Efficient LLM Moderation with Multi-Layer Latent Prototypes

Although modern LLMs are aligned with human values during post-training, robust moderation remains essential to prevent harmful outputs at deployment time. Existing approaches suffer from performance-efficiency trade-offs and are difficult to customize to user-specific requirements. Motivated by this gap, we introduce Multi-Layer Prototype Moderator (MLPM), a lightweight and highly customizable input moderation tool. We propose leveraging prototypes of intermediate representations across multiple layers to improve moderation quality while maintaining high efficiency. By design, our method adds negligible overhead to the generation pipeline and can be seamlessly applied to any model. MLPM achieves state-of-the-art performance on diverse moderation benchmarks and demonstrates strong scalability across model families of various sizes. Moreover, we show that it integrates smoothly into end-to-end moderation pipelines and further improves response safety when combined with output moderation techniques. Overall, our work provides a practical and adaptable solution for safe, robust, and efficient LLM deployment.

💡 Research Summary

The paper introduces the Multi‑Layer Prototype Moderator (MLPM), a lightweight yet high‑performing input‑moderation technique for large language models (LLMs). Existing moderation approaches fall into two camps: dedicated guard models, which achieve strong safety performance but require an additional model, incur extra training cost, and increase inference latency; and latent‑based methods, which reuse the LLM’s internal representations but typically lag behind guard models in accuracy. MLPM bridges this gap by exploiting the rich, hierarchical representations already computed inside any off‑the‑shelf LLM.

Methodology
For each transformer block (layer) of the target LLM, MLPM extracts the final‑token hidden state from the feed‑forward network (FFN). Using a labeled safety dataset, it computes class‑conditional prototypes: the mean vector µₗ,c and a shared covariance matrix Σ (estimated via a Bayesian ridge estimator). The Mahalanobis distance d_M(h, µₗ,c, Σ) captures the geometry of the latent space, and by exponentiating and normalising across classes it yields a per‑layer Gaussian Discriminant Analysis (GDA) probability Pₗ(x∈X_harm).

To combine information across layers, MLPM learns a sparse set of aggregation weights wₗ with an ℓ₁‑regularised objective. The final safety score is σ(∑ₗ wₗ·Pₗ), where σ is the sigmoid function. The ℓ₁ penalty forces many weights to zero, automatically selecting the most informative layers and reducing unnecessary computation at inference time.

Training and Efficiency
Training requires only a single forward pass over the prompt dataset; no back‑propagation or text generation is needed. Prototype computation and weight optimisation can be performed on a CPU in seconds, and the method remains data‑efficient—experiments show competitive performance with as few as 1 000 labeled examples. At inference, the required hidden states are already available from the prompt pre‑fill step, so MLPM adds less than 0.001 % of extra FLOPs and roughly 24 KB of memory per prototype for an 8‑billion‑parameter model.

Experimental Results
MLPM was evaluated on four base models (Mistral, Llama, OLMo, Qwen3) and eight harmful‑prompt benchmarks, including the sophisticated jailbreak datasets WildJailbreak and WildGuard‑Mix. Across all settings, MLPM outperformed prior latent‑based methods and matched or exceeded the best guard models (e.g., Aegis‑Defensive, LlamaGuard3, Granite Guardian). The gains were especially pronounced on complex jailbreak attacks, where multi‑layer aggregation resolved ambiguities that single‑layer approaches missed. Ablation studies confirmed robustness in low‑data regimes and under distribution shift.

Integration with End‑to‑End Pipelines
When combined with downstream output‑moderation or steering mechanisms, MLPM reduced unnecessary refusals and improved overall system safety, demonstrating that early input filtering can alleviate the burden on later safety components.

Contributions

1. A novel multi‑layer prototype‑based classifier that leverages Mahalanobis‑GDA for precise safety assessment.
2. Demonstration of state‑of‑the‑art performance with minimal training data and negligible inference overhead.
3. A flexible, model‑agnostic framework that can be seamlessly incorporated into existing moderation pipelines and easily adapted to custom safety policies.

In summary, MLPM offers a practical, scalable solution for safe LLM deployment, delivering guard‑level accuracy while preserving the efficiency and adaptability required for real‑world applications.