Agentic Adversarial QA for Improving Domain-Specific LLMs

Large Language Models (LLMs), despite extensive pretraining on broad internet corpora, often struggle to adapt effectively to specialized domains. There is growing interest in fine-tuning these models for such domains; however, progress is constrained by the scarcity and limited coverage of high-quality, task-relevant data. To address this, synthetic data generation methods such as paraphrasing or knowledge extraction are commonly applied. Although these approaches excel at factual recall and conceptual knowledge, they suffer from two critical shortcomings: (i) they provide minimal support for interpretive reasoning capabilities in these specialized domains, and (ii) they often produce synthetic corpora that are excessively large and redundant, resulting in poor sample efficiency. To overcome these gaps, we propose an adversarial question-generation framework that produces a compact set of semantically challenging questions. These questions are constructed by comparing the outputs of the model to be adapted and a robust expert model grounded in reference documents, using an iterative, feedback-driven process designed to reveal and address comprehension gaps. Evaluation on specialized subsets of the LegalBench corpus demonstrates that our method achieves greater accuracy with substantially fewer synthetic samples.

💡 Research Summary

The paper “Agentic Adversarial QA for Improving Domain‑Specific LLMs” tackles the persistent problem of adapting large language models (LLMs) to specialized domains where high‑quality labeled data are scarce. While recent synthetic‑data approaches such as EntiGraph and Knowledge‑Instruct have shown that expanding factual knowledge can improve performance, they fall short on two fronts: (i) they provide little support for interpretive, multi‑step reasoning that many domain tasks require, and (ii) they generate massive, often redundant corpora, leading to poor sample efficiency.

To address these gaps, the authors propose an adversarial question‑generation framework that directly probes the weaknesses of a target, smaller domain‑specific model (f_weak) by comparing its answers to those of a strong expert model (f_strong) that has full access to the domain documents C. The core of the method is an iterative optimization loop inspired by TextGrad’s differentiable prompting, but inverted: instead of minimizing loss, the system maximizes a disagreement score L(Q). At each iteration, both models answer a current question Q^(t)_i; a feedback model f_fb (implemented as the same strong LLM with a dedicated prompt) evaluates the discrepancy in terms of correctness, coverage, and reasoning alignment. A guide model f_guide then produces a natural‑language editing instruction that tells how to modify Q^(t)_i to accentuate the identified weakness. A revision model f_rev applies this instruction, yielding the next‑generation question Q^(t+1)_i. Repeating this process T times drives the questions toward the most challenging aspects for the weak model—often involving integration of multiple clauses, hypothetical scenarios, or nuanced terminology.

After the adversarial refinement, the final set of questions {Q^(T)_i} is paired with the expert model’s answers to form a synthetic dataset D_synthetic. Fine‑tuning f_weak on D_synthetic explicitly targets the uncovered gaps, improving both factual recall and, crucially, interpretive reasoning. The authors evaluate the approach on three frequently referenced contracts from the LegalBench benchmark (Cardlytics Maintenance Agreement, Buffalo Wild Wings Franchise Agreement, PF Hospitality Franchise Agreement), comprising 491 benchmark questions across 36 distinct tasks. Using a LLaMA‑3‑8B base model, the baseline few‑shot accuracy is 69.5% on average. The proposed adversarial QA fine‑tuning raises this to 82.7%, a 13.2‑percentage‑point gain, while using only 96 k training tokens. By contrast, paraphrase‑based fine‑tuning (≈149 k tokens) reaches 71.9%, a naive uninformed question set (≈147 k tokens) 75.8%, EntiGraph (≈6.7 M tokens) 79.6%, and Knowledge‑Instruct (≈159 k tokens) 75.0%. Thus the method achieves higher accuracy with roughly 70× fewer tokens, demonstrating superior sample efficiency.

A sensitivity analysis varying the number of adversarial refinement steps shows that 5–10 iterations yield the best trade‑off between computational cost and performance gain, confirming that iterative question sharpening is essential. The authors argue that the framework embodies an active‑learning style pedagogy: instead of exposing the model to a flood of generic facts, it is challenged with “exam‑style” questions that require synthesis, inference, and contextual judgment.

Key contributions include:

1. A novel adversarial QA generation loop that maximizes model disagreement to surface interpretive deficiencies.
2. Integration of differentiable prompting (guide and revision models) for automated, gradient‑guided question refinement.
3. Demonstration of dramatic token‑efficiency gains and reasoning improvements on a real‑world legal benchmark.
4. A domain‑agnostic pipeline that requires only raw domain documents, no external annotations or task‑specific supervision.

Limitations are acknowledged: the approach relies on a strong expert model (often a large, costly LLM) for feedback, and the design of the feedback function f_fb is somewhat heuristic, potentially introducing bias. Experiments are confined to legal contracts; extending to other domains such as medicine or finance will require further validation. Future work may explore ensembles of expert models, automated metric‑based feedback, and multi‑modal domain corpora.

Overall, the paper presents a compelling, technically sound method for generating high‑utility synthetic training data that directly addresses the interpretive reasoning gap in domain‑specific LLM adaptation, offering a practical path toward more capable, efficient specialized language models.