The exponentially growing body of work on social bot detection, shown in Figure 2, suggests that there will be much effort to combat this problem in the coming years. However, it also poses some new challenges. First, it is becoming increasingly important to organize this huge body of work. Not only will this help to leverage knowledge more effectively, but it will also allow researchers to more effectively propose new solutions, avoiding exploring paths that have already failed.
Secondly, the expected growth in publications inevitably means that more bot detectors will be proposed. With the growing number of disparate detection methods, it becomes increasingly important to have standard tools such as benchmarks, baselines, and reference datasets to evaluate and compare them against. The current situation is that we have a suitcase full of all sorts of tools. Unfortunately, we don’t really know how to use them profitably, what the differences are between them, and ultimately, what they are really worth! Buying another tool won’t help much. Instead, a few targeted investments aimed at comprehensively evaluating and comparing current tools would greatly improve the usefulness of the entire suitcase.
One aspect that is often overlooked when evaluating bot detectors is their generalizability
I.e. the ability to achieve good detection results also for bot types that were not initially considered. In this regard, the analysis lays the foundations of a two-dimensional accurate mobile phone number list generalizability space, schematically depicted in Figure 7. A desirable future scenario would include the ability to evaluate any new bot detector against a variety of different social bot types, thus moving forward along the y-axis in Figure 7, following the promising approaches recently developed by Ekiverria et al. [11] and Yang et al. [36].
It would also be useful to compare detectors to different versions of existing bots, thus simulating the evolving characteristics of bots. This can be achieved by using the previously described adversarial approach to generate many adversarial examples, opening up the possibility of experimentation along the x-axis of the generalization space.
Combining these two evaluation parameters and thus comprehensively exploring the generalizability space would allow for a robust assessment of the detection capabilities of current and future methods, thus avoiding overestimation of detection performance. To achieve this ambitious goal, it would first be necessary to create benchmark datasets that would include several different types of malicious accounts, including social bots, cyborgs, and political trolls, thereby significantly increasing the scarce resources that exist today [j].
Challenges here include limited availability of the data itself
Missing or questionable information, and the aging of existing data sets that struggle to keep up with the rapid evolution of malicious accounts. Initiatives to continuously share data, such as Twitter’s, on accounts involved in information operations are therefore highly welcome, as they can facilitate the next wave of research into these issues.
Further ways to generate a wide range of diverse adversarial 6 proverbs about raising a leader that are worth taking note of examples must then be developed. This will require quantitative means to assess the contribution of different adversarial examples, for example in terms of their novelty and diversity relative to existing malicious accounts. These issues currently remain largely unsolved and require the greatest effort from the research community.
Figure 7. Two-dimensional generalizability space
The legend in Figure 7 : The axes represent the dimensions along which the generalization capabilities of the detectors can be test. Most existing detectors are evaluat under favorable conditions, i.e. only against a certain type of bot (b0) and with data collect at a certain point in time (t0), which may overestimate their capabilities. The actual detection performance for b ≠ b0 and t > t0 is unknown.
More realistic estimates can be obtaineby evaluating detectors in more general settings. Generalizations along the y-axis can be achie by adopting evaluation methodologies such as those propos by Echeverria et al. [11]. Generalizations along the y-axis can be obtain by applying adversarial approaches aim at generating variants of currently existing bots.
The caption to the left of the diagram is the types of social bots.
Summary of: bot scores (x-axis), bot types (y-axis), both (both axes).
Detection difficulty: easy (green circle), difficult (orange circle), most difficult (red circle).
A longitudinal analysis of the first decade of social bot detection research reveals some interesting trends. The early days were characteriz by simple supervis detectors that analyz accounts individually. Unsupervis detectors emerged in 2012–2013 and shift their target to groups of misbehaving european union phone number accounts. Finally, a new trend of increasing adversarial approaches was highlight.
The analysis shows that for more than a decade, we have been battling each of the threats pos by sophisticat social bots, cyborgs, trolls, and colluding humans separately. Now, thanks to the proliferation of AI-power deception techniques like deepfakes, the most sophisticat of these attackers will inevitably become indistinguishable from each other, and likely from legitimate accounts as well. It is therefore increasingly necessary to focus on identifying the techniques us to deceive and manipulate, rather than trying to classify individual accounts by their nature.
Inauthentic coordination is an important piece of the deception puzzle
As adversaries exploit it to gain attention and influence. Moreover, they [adversarial approaches] do not address different types of adversaries. In other words, the findings and recent reflections [in the literature][17][28] suggest that we should continue to move away from simple supervis approaches that focus on individual accounts and produce binary labels. Instead, we must take on the challenge of understanding the complexity of deception, manipulation, and automation, and develop unsupervis methods to detect suspicious coordination.
Furthermore, future methods should not provide overly simplistic binary labels, as is often done and just as often criticiz, but should instead produce multifacet measures of the degree of suspicious coordination.
In-depth analysis has reveal the emergence of group approaches several years before the general public and social platforms themselves recogniz “coordinat inauthentic behavior” as a major threat to our online social ecosystems. Among the most pressing issues in this line of research are the scalability of group detectors and the inherent fuzziness of “inauthentic coordination.”
In fact, scalable and generalizable coordination detection still remains a largely open problem, with only a few contributions propos so far.[12][25] Similarly, computational tools for distinguishing between genuine and inauthentic coordination have yet to be propos and evaluat.
Interestingly, the same analysis that anticipat the world’s interest in inauthentic coordination now suggests that adversarial approaches may give us an ge in the long-term fight against online deception.
To summarize the main propositions emerging from our extensive analysis, future deception detection methods should
- focus on identifying suspicious coordination regardless of the nature of individual accounts;
- avoid using binary labels in favor of more fuzzy and multiface indicators;
- favor unsupervis/semi-supervis approaches over supervis ones;
- take competition into account in design.
In addition, some of the enormous efforts devot to the detection task should also be reallocat to measuring the human impact of these phenomena and quantifying the potential impact. Only by making these changes can tools be develop that better reflect the current reality, thereby providing actionable results for the many scientific communities and stakeholders who see AI and big data tools as a compass for adventures in the dangerous landscape of online information.
These guiding lights appear before us as an exciting and rare opportunity that we have not had in the past. Reacting to this opportunity and using it is now entirely up to us.
Acknowlgments . This research is partly support by the EU H2020 Programme under the scheme INFRAIA-01-2018-2019: Research and Innovation Grant Agreement No 871042 SoBigData++: A European integrat infrastructure for social mining and big data analytics.