Beyond Content: Behavioral Policies Reveal Actors in Information Operations

The detection of online influence operations – coordinated campaigns by malicious actors to spread narratives – has traditionally depended on content analysis or network features. These approaches are increasingly brittle as generative models produce convincing text, platforms restrict access to behavioral data, and actors migrate to less-regulated spaces. We introduce a platform-agnostic framework that identifies malicious actors from their behavioral policies by modeling user activity as sequential decision processes. We apply this approach to 12,064 Reddit users, including 99 accounts linked to the Russian Internet Research Agency in Reddit’s 2017 transparency report, analyzing over 38 million activity steps from 2015-2018. Activity-based representations, which model how users act rather than what they post, consistently outperform content models in detecting malicious accounts. When distinguishing trolls – users engaged in coordinated manipulation – from ordinary users, policy-based classifiers achieve a median macro-$F_1$ of 94.9%, compared to 91.2% for text embeddings. Policy features also enable earlier detection from short traces and degrade more gracefully under evasion strategies or data corruption. These findings show that behavioral dynamics encode stable, discriminative signals of manipulation and point to resilient, cross-platform detection strategies in the era of synthetic content and limited data access.

💡 Research Summary

The paper tackles the growing challenge of detecting online influence operations (IOs) in an era where generative AI can produce highly convincing text and platforms increasingly restrict access to relational data. Traditional detection methods that rely on content analysis or network structure are becoming brittle, especially on platforms like Reddit that lack explicit follower graphs. To address this, the authors propose a platform‑agnostic framework that models each user’s activity as a sequential decision‑making process, formalized as a Markov Decision Process (MDP). In this formulation, a “state” captures recent engagement outcomes (e.g., recent up‑votes, subreddit context, time gaps) while an “action” corresponds to platform functions such as creating a new thread, posting a top‑level comment, or replying.

Using a dataset of 12,064 Reddit users spanning 2015‑2018, which includes 99 accounts identified by Reddit’s 2017 transparency report as linked to the Russian Internet Research Agency (IRA), the authors extract over 38 million activity steps. They infer individualized behavioral policies for each user through three increasingly expressive methods: (1) an empirical policy derived directly from observed state‑action frequencies, (2) a policy learned via Generative Adversarial Imitation Learning (GAIL), and (3) a maximum‑entropy deep Inverse Reinforcement Learning (IRL) approach that first recovers a reward function and then computes a stochastic policy via soft value iteration.

Each policy is vectorized and fed into supervised classifiers (XGBoost and Random Forest). As baselines, the study uses text embeddings generated from the same activity windows (e.g., BERT‑based sentence embeddings). Across stratified 5‑fold cross‑validation, policy‑based representations consistently outperform content‑based baselines. GAIL achieves the highest median macro‑F₁ of 94.9 % (95 % CI ≈ 92.0‑97.4 %), while the empirical and IRL policies also exceed 90 % macro‑F₁. By contrast, the best text‑embedding model reaches only 91.2 % macro‑F₁.

A key contribution is the evaluation of early detection. The authors train classifiers on only the first n state‑action pairs (n = 3 to full trajectory). Even with just three interactions, the empirical policy attains a macro‑F₁ of 91.4 %, far surpassing the embedding baseline’s 74.2 %. Performance quickly plateaus after 10‑20 actions, indicating that a short behavioral trace is sufficient for reliable identification.

Robustness is examined by randomly perturbing a fraction p of the state‑action pairs to simulate adversarial evasion. Policy‑based models degrade gracefully: at p = 0.3 (30 % corruption) they still maintain >80 % macro‑F₁, whereas text‑based models collapse dramatically. This resilience stems from the fact that policies encode structural decision patterns that are harder to disguise than surface text.

Clustering analysis of the learned policy vectors reveals sub‑communities within the troll cohort, suggesting that even coordinated actors exhibit distinct strategic “styles.” The authors also discuss failure modes such as “hijacked” accounts that mimic troll policies while displaying anomalous activity bursts.

Overall, the study demonstrates that behavioral dynamics—captured as decision policies—provide stable, platform‑agnostic signals for detecting malicious IO actors. The approach excels in accuracy, enables rapid early detection, and remains robust under data corruption or evasion attempts. Limitations include reliance on sufficient activity history (new accounts may be harder to classify) and the need to validate the framework on other platforms (e.g., X/Twitter, TikTok). Future work could explore automated state extraction, multimodal integration (text, images, video), and cross‑platform policy transfer.

In conclusion, modeling user behavior as sequential policies offers a powerful, resilient alternative to content‑centric detection, promising more effective defenses against sophisticated, AI‑generated influence campaigns in environments where data access is increasingly constrained.