Fairness-Aware Multi-Group Target Detection in Online Discussion

Target-group detection is the task of detecting which group(s) a piece of content is ``directed at or about’’. Applications include targeted marketing, content recommendation, and group-specific content assessment. Key challenges include: 1) that a single post may target multiple groups; and 2) ensuring consistent detection accuracy across groups for fairness. In this work, we investigate fairness implications of target-group detection in the context of toxicity detection, where the perceived harm of a social media post often depends on which group(s) it targets. Because toxicity is highly contextual, language that appears benign in general can be harmful when targeting specific demographic groups. We show our {\em fairness-aware multi-group target detection} approach both reduces bias across groups and shows strong predictive performance, surpassing existing fairness-aware baselines. To enable reproducibility and spur future work, we share our code online.

💡 Research Summary

The paper tackles the problem of target‑group detection in online discussions, where a single post may be directed at multiple demographic groups (e.g., race, gender, religion). While prior work has treated this as a single‑label task, the authors argue that real‑world content often implicates several groups simultaneously, and that detection accuracy must be equitable across all groups to avoid downstream harms in applications such as toxicity detection, personalized recommendation, and fact‑checking.

To measure fairness they adopt Accuracy Parity (AP), which requires that predictive accuracy be the same for every group. AP is appropriate because in target‑group detection both false positives (assigning a group that is not targeted) and false negatives (missing a targeted group) have symmetric costs. The authors demonstrate that Equalized Odds (EO), a common fairness metric, is ill‑suited for this setting: under realistic conditions with unequal base rates, AP and EO cannot be satisfied simultaneously (an impossibility theorem). Empirically, enforcing EO degrades performance for minority groups.

The core technical contribution is a new loss function called GAP_multi, an extension of the Group Accuracy Parity (GAP) loss originally designed for binary groups. GAP_multi computes pairwise cross‑entropy differences for every distinct pair of groups (j, k) and adds the squared L2 norm of these differences to the overall error term:

 GAP_multi = OE + λ ∑_{j≠k} ‖CE(g=j) – CE(g=k)‖²

where OE is the overall error (e.g., macro‑cross‑entropy) and λ controls the fairness‑utility trade‑off. By directly minimizing pairwise error gaps, GAP_multi avoids the serial “deviation‑from‑mean” computation that bottlenecks the original GAP formulation, enabling fully parallel GPU computation even as the number of groups grows.

The authors evaluate GAP_multi on two large, multi‑label hate‑speech corpora: the Measuring Hate Speech (MHS) corpus and HateXplain. Both datasets contain annotations for multiple demographic groups across several platforms (Twitter, Reddit, YouTube, Gab). Experiments compare GAP_multi against several baselines: the original GAP loss, re‑weighting schemes, adversarial debiasing, and EO‑based losses. Two primary metrics are reported: (1) overall macro‑accuracy (utility) and (2) inter‑group disparity (dev_overall), defined as the sum of absolute deviations of each group’s error from the mean error.

Results show that GAP_multi consistently reduces inter‑group disparity by 30‑45 % relative to the best baseline, with the most pronounced gains for small minority groups such as Native American and Pacific Islander. Overall macro‑accuracy is either maintained or improves by up to 2 % compared to baselines, demonstrating that fairness does not come at a substantial utility cost. A hyper‑parameter sweep over λ reveals a sweet spot (≈0.1–0.5) where both AP and accuracy are jointly optimized; larger λ values overly prioritize fairness and hurt utility, while λ = 0 reduces to standard cross‑entropy training.

Theoretical analysis confirms that GAP_multi is differentiable, preserves the convergence properties of GAP, and directly optimizes the AP metric. The impossibility theorem for AP vs. EO is formally proved, and empirical plots illustrate the trade‑offs.

In discussion, the authors note that while the current work focuses on text, the framework is agnostic to modality and could be extended to images or multimodal content. They also acknowledge the reliance on high‑quality multi‑label annotations, suggesting future work on weak supervision or leveraging user reports.

Overall, the paper makes three substantive contributions: (1) formalizing multi‑label target‑group detection as a fairness‑critical task, (2) introducing GAP_multi, a scalable loss that aligns training objectives with Accuracy Parity, and (3) providing extensive empirical and theoretical evidence that GAP_multi achieves better group‑fairness without sacrificing predictive performance. This work offers a practical solution for building fairer moderation and recommendation pipelines in real‑world online platforms.