Improving Implicit Hate Speech Detection via a Community-Driven Multi-Agent Framework

This work proposes a contextualised detection framework for implicitly hateful speech, implemented as a multi-agent system comprising a central Moderator Agent and dynamically constructed Community Agents representing specific demographic groups. Our approach explicitly integrates socio-cultural context from publicly available knowledge sources, enabling identity-aware moderation that surpasses state-of-the-art prompting methods (zero-shot prompting, few-shot prompting, chain-of-thought prompting) and alternative approaches on a challenging ToxiGen dataset. We enhance the technical rigour of performance evaluation by incorporating balanced accuracy as a central metric of classification fairness that accounts for the trade-off between true positive and true negative rates. We demonstrate that our community-driven consultative framework significantly improves both classification accuracy and fairness across all target groups.

💡 Research Summary

**
The paper addresses the persistent challenge of detecting implicit hate speech on social media, where subtle, coded language often evades conventional text‑centric classifiers and where existing models tend to under‑detect hateful content directed at marginalized groups. To remedy both the accuracy deficit and the fairness gap, the authors propose a novel multi‑agent framework that combines a central Moderator Agent with a set of dynamically instantiated Community Agents, each representing a specific demographic group (e.g., Black, Asian, Muslim, Jewish, Women, LGBTQ).

System Architecture
The Moderator Agent receives a post and an automatically extracted target‑group label. It performs an initial semantic assessment using a large language model (LLM) and outputs a provisional classification (Hate / Not Hate / Unsure), a justification, and a confidence score. If the confidence falls within a pre‑defined uncertainty interval (τ_low, τ_high), the Moderator flags the case for consultation.

Community Agents are created on‑the‑fly for the flagged group. The creation pipeline first generates multiple Wikipedia search queries tailored to the group, retrieves the corresponding articles, and encodes the retrieved text with a Transformer encoder. The group query embedding q_g is then combined with the token embeddings via cross‑attention, yielding a group‑specific embedding ψ_g that captures culturally relevant knowledge (historical context, legal frameworks, common euphemisms, etc.). This embedding is fed to the Community Agent LLM, which returns its own classification score and rationale.

The final decision is produced by fusing the Moderator’s initial score p_m with the Community Agent’s score p_c using a weighted combination or rule‑based logic, thereby preserving the Moderator’s authority while integrating expert‑in‑the‑loop insight. If no consultation is triggered, the Moderator’s initial decision stands.

Experimental Setup
The authors evaluate the framework on the ToxiGen dataset, a benchmark specifically designed for implicit toxicity. ToxiGen contains roughly 274 k short texts, of which 8 960 are manually annotated. The dataset covers multiple target groups; the authors sample 100 instances per group for six groups (Black, Asian, Muslim, Jewish, Women, LGBTQ). Evaluation metrics include the standard F1 score for overall detection performance and balanced accuracy as a fairness metric that equally weights true‑positive and true‑negative rates across groups.

Baseline systems comprise state‑of‑the‑art prompting strategies applied to the same LLM (Gemini‑2.5‑Flash): zero‑shot prompting, few‑shot prompting, Chain‑of‑Thought (CoT) prompting, and Decision‑Tree‑of‑Thought (DT‑oT) prompting.

Results
Across all groups, the multi‑agent consultative system outperforms the baselines. Relative improvements range from 4 to 7 percentage points in both F1 and balanced accuracy. Notably, the false‑negative rate for minority groups drops dramatically, indicating that the Community Agents successfully disambiguate coded language that would otherwise be missed. The authors also report qualitative examples where the Community Agent supplies historically grounded explanations (e.g., references to Jim Crow laws) that shift the final decision from “Not Hate” to “Hate.”

Technical Contributions

1. Automated Community Profiling – A pipeline that leverages publicly available knowledge bases (Wikipedia) to generate group‑specific contextual embeddings via cross‑attention.
2. Uncertainty‑Driven Consultation – A mechanism that mirrors human moderator practice: only when the primary model is uncertain does it solicit expert input, reducing unnecessary computational overhead.
3. AutoGen‑Based Multi‑LLM Coordination – The entire workflow is implemented using the AutoGen library, with distinct system prompts for each agent, ensuring reproducibility and clear interaction protocols.

Limitations and Future Work
The reliance on Wikipedia may introduce temporal gaps or systemic biases; integrating additional sources such as news feeds, community forums, or curated cultural lexicons could improve robustness. The current implementation supports six community agents; scaling to a broader spectrum of identities (regional, linguistic, generational) remains an open challenge. Real‑time latency and cost considerations for production deployment are not addressed, nor is a continuous learning loop that incorporates moderator or user feedback. Future research directions include (a) multi‑source knowledge fusion, (b) dynamic agent expansion based on emerging hate‑speech patterns, and (c) user‑in‑the‑loop evaluation to assess societal impact.

Conclusion
By embedding identity‑aware expertise directly into the moderation pipeline, the proposed community‑driven multi‑agent framework demonstrably improves both detection accuracy and fairness for implicit hate speech. The work offers a practical blueprint for “expert‑in‑the‑loop” AI systems that can adapt to culturally nuanced content, paving the way for more equitable and context‑sensitive moderation across online platforms.