Parallelism Meets Adaptiveness: Scalable Documents Understanding in Multi-Agent LLM Systems

Large language model (LLM) agents have shown increasing promise for collaborative task completion. However, existing multi-agent frameworks often rely on static workflows, fixed roles, and limited inter-agent communication, reducing their effectiveness in open-ended, high-complexity domains. This paper proposes a coordination framework that enables adaptiveness through three core mechanisms: dynamic task routing, bidirectional feedback, and parallel agent evaluation. The framework allows agents to reallocate tasks based on confidence and workload, exchange structured critiques to iteratively improve outputs, and crucially compete on high-ambiguity subtasks with evaluator-driven selection of the most suitable result. We instantiate these principles in a modular architecture and demonstrate substantial improvements in factual coverage, coherence, and efficiency over static and partially adaptive baselines. Our findings highlight the benefits of incorporating both adaptiveness and structured competition in multi-agent LLM systems.

💡 Research Summary

The paper addresses a critical limitation of current large‑language‑model (LLM) based multi‑agent systems: they typically rely on static workflows, fixed role assignments, and limited inter‑agent communication. Such rigidity hampers performance in open‑ended, high‑complexity domains where tasks evolve, ambiguities arise, and agent competence varies. To overcome these challenges, the authors propose a comprehensive coordination framework that integrates three core mechanisms: dynamic task routing, bidirectional feedback loops, and parallel agent evaluation.

Dynamic Task Routing continuously monitors each sub‑task’s confidence score, estimated difficulty, current workload, and historical performance of agents. Based on these signals, the orchestrator can reassign a sub‑task to a more suitable peer at runtime, thereby avoiding bottlenecks and ensuring that specialized knowledge (e.g., regulatory expertise) is applied where needed.

Bidirectional Feedback Loops enable downstream agents to send structured critiques and revision requests back to upstream contributors. Feedback is transmitted via an asynchronous message bus, with explicit tags linking the request to the problematic output. The originating agent can then revise its result or forward the issue to a higher‑level orchestrator, reducing error propagation without rerunning the entire pipeline.

Parallel Agent Evaluation tackles high‑ambiguity or high‑stakes subtasks by assigning the same task to multiple agents in parallel. A centralized evaluator scores each candidate output using a hierarchical scoring function E(o) = w_f·S_fact + w_c·S_coh + w_r·S_rel, where S_fact measures the proportion of claims supported by retrieved context, S_coh assesses logical consistency via chain‑of‑thought critique, and S_rel captures semantic similarity to the query. In the financial domain the authors prioritize factual accuracy (w_f = 0.5) over coherence (0.3) and relevance (0.2). The evaluator selects the highest‑scoring output for downstream consumption while preserving alternatives for auditability.

The architecture comprises an orchestrator that parses documents into a dependency graph, role‑specific agents (e.g., risk‑factor extractor, MD&A summarizer, compliance QA), a shared long‑term memory for persisting intermediate results, an evaluator agent implementing the scoring function, and a feedback bus for asynchronous critique exchange. The system is modular: new agents, memory back‑ends, or domain‑specific scoring models can be plugged in without disrupting existing workflows.

Empirical validation focuses on SEC 10‑K filings, a high‑risk domain requiring precise extraction of risk factors, year‑over‑year financial summaries, and regulatory compliance answers. Three system variants are compared: (1) a static baseline with fixed roles, (2) an adaptive variant with dynamic routing and feedback only, and (3) the full system incorporating all three mechanisms. Using a mix of automatic metrics and human judgments, the full system achieves factual coverage of 0.92 and compliance accuracy of 0.94—improvements of 27 % and 74 % respectively over the static baseline. Compared to a state‑of‑the‑art LangGraph supervisor pattern, the full system yields a 14 % gain in compliance accuracy, especially on ambiguous subtasks where single‑path routing fails. Revision rates drop by over 70 % and redundancy penalties (repeated or contradictory information) fall by 73 %. Human evaluators also rate the full system higher on coherence, relevance, and logical structure.

The study demonstrates that structured competition (parallel evaluation) combined with real‑time adaptiveness (dynamic routing and feedback) can substantially reduce hallucinations and improve robustness in multi‑agent LLM pipelines. The modular design ensures that the approach can be transferred to other high‑stakes domains such as medical records, legal contracts, or technical manuals. Future work is outlined to explore reinforcement‑learning‑driven routing policies, multimodal inputs (tables, charts), and meta‑learning of scoring functions, further extending the scalability and reliability of adaptive multi‑agent LLM systems.