Beyond Text-to-SQL: Autonomous Research-Driven Database Exploration with DAR

Large language models can already query databases, yet most existing systems remain reactive: they rely on explicit user prompts and do not actively explore data. We introduce DAR (Data Agnostic Researcher), a multi-agent system that performs end-to-end database research without human-initiated queries. DAR orchestrates specialized AI agents across three layers: initialization (intent inference and metadata extraction), execution (SQL and AI-based query synthesis with iterative validation), and synthesis (report generation with built-in quality control). All reasoning is executed directly inside BigQuery using native generative AI functions, eliminating data movement and preserving data governance. On a realistic asset-incident dataset, DAR completes the full analytical task in 16 minutes, compared to 8.5 hours for a professional analyst (approximately 32x times faster), while producing useful pattern-based insights and evidence-grounded recommendations. Although human experts continue to offer deeper contextual interpretation, DAR excels at rapid exploratory analysis. Overall, this work shifts database interaction from query-driven assistance toward autonomous, research-driven exploration within cloud data warehouses.

💡 Research Summary

The paper addresses a fundamental limitation of current text‑to‑SQL and database‑assistant systems: they are reactive, requiring an explicit natural‑language prompt from a user before any query is generated. In enterprise environments, massive volumes of “dark data” remain unused because human analysts must first notice an opportunity, formulate a research question, and manually craft the appropriate SQL. To move from reactive assistance to proactive, research‑driven exploration, the authors introduce DAR (Data Agnostic Researcher), a hierarchical multi‑agent system that autonomously discovers research questions, synthesizes and validates SQL‑plus‑AI queries, and produces a written report—all without any human‑initiated query.

DAR is built on Google Cloud’s BigQuery and its native generative‑AI functions (e.g., AI.GENERATE, AI.GENERATE_BOOL, AI.GENERATE_TABLE). By keeping all reasoning inside the warehouse, DAR eliminates data movement, reduces latency, and preserves strict data‑governance policies. The system is organized into three layers:

1. Initialization Layer – The Research Initiator (ARI) parses a high‑level research intent supplied by the user, extracts schema metadata via a Meta Extractor (list_table_ids, list_dataset_info, etc.), and uses a Plan Generator (TPG) to decompose the overall goal into concrete sub‑tasks and an execution plan.

2. Execution Layer – A sequential pipeline of four specialized agents:

   • Query Understanding (AQU) maps research objectives to specific tables, columns, and relationships.
   • Query Generation (AQG) creates concrete SQL statements, optionally embedding BigQuery‑native AI functions for in‑database text generation, classification, or summarization.
   • Query Execution (AQE) submits the queries to BigQuery using authenticated service‑account credentials and retrieves results.
   • Query Review (AQR) evaluates execution outcomes; if a result is empty or contains errors, it iteratively revises the query (bounded by a maximum number of retries). Validation succeeds only when |Result| > 0 ∧ Error(Result) = ∅.

3. Synthesis Layer – A second four‑agent workflow that builds a human‑readable report:

   • Structure Planner (ASP) defines the report’s hierarchy and narrative flow.
   • Scratch Research (ASR) drafts an initial synthesis limited to the validated query outputs.
   • Revision (ARV) refines the draft for coherence, readability, and completeness, guided by an internal quality score.
   • Report Composer (ARC) assembles the final Markdown document.

A dedicated Escalation Checker compares the report’s quality score against a threshold θ; reports meeting or exceeding θ are accepted, otherwise they are routed back to ARV for further refinement. This loop provides automated quality control without human review.

The reasoning engine relies on Google’s Agent Development Kit (ADK). ADK enables flexible model back‑ends (Gemini by default, with optional LiteLLM integration), supports session‑based short‑term and working memory, and provides composition patterns such as SequentialAgent for deterministic pipelines. Several agents employ chain‑of‑thought prompting and ReAct‑style interleaving of reasoning and tool use, while self‑reflection is built into the Query Review and report‑evaluation stages.

Experimental Evaluation
The authors implemented DAR as open‑source code on GitHub and conducted a case study on a realistic “asset‑incident” dataset comprising two relational tables (26 assets, 19 attributes, and incident records). The task required cross‑table joins, pattern detection, and recommendation generation. A professional data analyst performed the same analysis manually. Results:

• Time – DAR completed the full pipeline in 16 minutes, whereas the human analyst required 8.5 hours, a speed‑up of roughly 32×.
• Correctness – All DAR‑generated queries returned non‑empty results with no execution errors, satisfying the validation condition.
• Insight Quality – DAR identified a pattern: high‑risk assets in specific geographic regions exhibited elevated incident frequencies during certain time windows, and it produced evidence‑grounded mitigation recommendations.
• Report Quality – The internal quality score exceeded the predefined threshold (θ ≈ 0.8), leading to automatic acceptance of the final report.

The authors acknowledge that while DAR excels at rapid exploratory analysis, human experts still provide deeper contextual interpretation, strategic reasoning, and domain‑specific nuance that the system cannot yet replicate.

Contributions and Significance

1. Introduction of a fully autonomous, hierarchical multi‑agent architecture that formulates its own research questions and executes end‑to‑end database exploration without explicit prompts.
2. Demonstration that native in‑database LLM functions can replace external API calls, preserving data security and reducing latency/cost.
3. Empirical evidence of substantial time savings and comparable insight generation on a realistic business‑intelligence task.

Future Directions suggested include extending DAR to multimodal data (images, logs), integrating more sophisticated reinforcement‑learning based reward signals for agent policy optimization, and porting the approach to other cloud data warehouses (AWS Redshift, Azure Synapse). The paper positions DAR as a foundational step toward “research‑driven” data platforms where autonomous agents continuously surface actionable knowledge while respecting enterprise governance constraints.