Unified-MAS: Universally Generating Domain-Specific Nodes for Empowering Automatic Multi-Agent Systems

Automatic Multi-Agent Systems (MAS) generation has emerged as a promising paradigm for solving complex reasoning tasks. However, existing frameworks are fundamentally bottlenecked when applied to knowledge-intensive domains (e.g., healthcare and law). They either rely on a static library of general nodes like Chain-of-Thought, which lack specialized expertise, or attempt to generate nodes on the fly. In the latter case, the orchestrator is not only bound by its internal knowledge limits but must also simultaneously generate domain-specific logic and optimize high-level topology, leading to a severe architectural coupling that degrades overall system efficacy. To bridge this gap, we propose Unified-MAS that decouples granular node implementation from topological orchestration via offline node synthesis. Unified-MAS operates in two stages: (1) Search-Based Node Generation retrieves external open-world knowledge to synthesize specialized node blueprints, overcoming the internal knowledge limits of LLMs; and (2) Reward-Based Node Optimization utilizes a perplexity-guided reward to iteratively enhance the internal logic of bottleneck nodes. Extensive experiments across four specialized domains demonstrate that integrating Unified-MAS into four Automatic-MAS baselines yields a better performance-cost trade-off, achieving up to a 14.2% gain while significantly reducing costs. Further analysis reveals its robustness across different designer LLMs and its effectiveness on conventional tasks such as mathematical reasoning.

💡 Research Summary

The paper addresses a critical limitation of current Automatic Multi‑Agent Systems (MAS): when applied to knowledge‑intensive domains such as healthcare, law, or finance, performance drops sharply because existing frameworks either rely on a static library of generic reasoning nodes (e.g., Chain‑of‑Thought, Debate) or attempt to generate new nodes on‑the‑fly while the orchestrator simultaneously handles macro‑level topology search. The latter creates a severe architectural coupling that forces the orchestrator to compensate for the LLM’s internal knowledge gaps, leading to hallucinations and inefficient search.

Unified‑MAS is proposed as a two‑stage solution that decouples granular node implementation from topological orchestration.

1. Search‑Based Node Generation – The system first extracts a multi‑dimensional keyword set from a validation sample buffer. Seven dimensions are considered: domain, task, entities, actions, constraints, desired outcomes, and implicit knowledge. These keywords are recombined into four targeted search strategies (background knowledge, system architecture, code implementation, evaluation). Using external sources (Google, GitHub, Google Scholar) the framework performs multi‑turn retrieval, aggregates the results into concise summaries, and prompts an LLM to synthesize a set of domain‑specific node blueprints (V_init). Each node includes its prompt, tool specifications, input/output schema, and any required sub‑agent calls. This stage expands the node pool from a fixed generic set V_fixed to a domain‑adapted set V_domain, thereby overcoming the LLM’s parametric knowledge limits.

2. Reward‑Based Node Optimization – The initially generated nodes are often coarse and logically brittle. Unified‑MAS treats MAS execution as a trajectory and measures the perplexity of each node’s generated text. Perplexity serves as a proxy for reasoning stability; higher values indicate uncertainty. A perplexity‑guided reward (negative perplexity scaled by a coefficient) is used to identify bottleneck nodes. The framework iteratively refines these nodes by adjusting prompts, adding missing constraints, or introducing additional sub‑agents, then re‑evaluates the reward. This reinforcement‑style loop gradually lowers perplexity, improves internal consistency, and strengthens the node’s contribution to the overall system.

The authors evaluate Unified‑MAS on four specialized benchmarks: TravelPlanner (constrained itinerary planning), HealthBench (clinical diagnosis and treatment recommendation), J1Bench (legal judgment and damages calculation), and DeepFund (financial decision‑making). They integrate the generated nodes into four state‑of‑the‑art Automatic‑MAS baselines—MAS‑Zero, AFlow, ScoreFlow, and MAS‑2—using four different orchestrator LLMs (GPT‑4, Claude‑2, LLaMA‑2, and a domain‑specific BioGPT). Across all combinations, Unified‑MAS yields an average accuracy gain of 8 %–14 % (up to 14.2 % on HealthBench) while reducing token usage and API cost by 10 %–20 %. Perplexity drops by roughly 35 % on bottleneck nodes, confirming the effectiveness of the reward mechanism.

Ablation studies compare the full pipeline against (a) a version without the reward‑based optimization (only fine‑tuning) and (b) a version that skips external search and relies solely on the LLM’s internal knowledge. Both baselines achieve marginal improvements (<2 % accuracy), underscoring that (i) external knowledge retrieval is essential for domain expertise, and (ii) perplexity‑guided refinement is crucial for turning that expertise into reliable reasoning.

The paper also demonstrates robustness: the same performance boost appears when swapping the orchestrator LLM, and the approach generalizes to a non‑specialized task (mathematical reasoning), where it still outperforms baseline MAS by about 5 % accuracy.

Key contributions are:

• A clear diagnosis of the two fundamental bottlenecks in Automatic‑MAS (static generic nodes and tightly coupled dynamic node generation).
• An offline node synthesis pipeline that expands the design space from V_fixed to V_domain via multi‑dimensional keyword extraction and targeted external retrieval.
• A novel perplexity‑based reward that quantitatively identifies and iteratively improves brittle nodes within a multi‑agent execution trajectory.
• Extensive empirical evidence across diverse domains, baselines, and LLMs showing simultaneous performance gains and cost reductions, as well as evidence of generalizability.

In conclusion, Unified‑MAS demonstrates that separating expert‑level node creation from macro‑level topology search enables Automatic‑MAS to achieve specialist‑level competence without sacrificing the flexibility of automated orchestration. Future work may explore automated strategy generation for the retrieval phase, multimodal external knowledge sources, and online continual updating of nodes during deployment.