From Scattered to Structured: A Vision for Automating Architectural Knowledge Management

Software architecture is inherently knowledge-centric. The architectural knowledge is distributed across heterogeneous software artifacts such as requirements documents, design diagrams, code, and documentation, making it difficult for developers to access and utilize this knowledge effectively. Moreover, as systems evolve, inconsistencies frequently emerge between these artifacts, leading to architectural erosion and impeding maintenance activities. We envision an automated pipeline that systematically extracts architectural knowledge from diverse artifacts, links them, identifies and resolves inconsistencies, and consolidates this knowledge into a structured knowledge base. This knowledge base enables critical activities such as architecture conformance checking and change impact analysis, while supporting natural language question-answering to improve access to architectural knowledge. To realize this vision, we plan to develop specialized extractors for different artifact types, design a unified knowledge representation schema, implement consistency checking mechanisms, and integrate retrieval-augmented generation techniques for conversational knowledge access.

💡 Research Summary

The paper addresses a fundamental pain point in software engineering: architectural knowledge is scattered across heterogeneous artifacts—requirements documents, design diagrams, source code, comments, documentation, and even meeting recordings—making it difficult for developers and architects to locate, understand, and keep this knowledge consistent as a system evolves. The authors propose a comprehensive, automated pipeline that extracts architectural knowledge from these diverse sources, links the extracted pieces, detects and resolves inconsistencies, and stores the result in a unified, structured knowledge base (KB). This KB then serves as the backbone for three high‑value activities: architecture conformance checking, change‑impact analysis, and natural‑language question‑answering (QA).

The pipeline consists of five tightly coupled components.

1. Knowledge Extraction Framework – Specialized extractors are built for each artifact type. Textual artifacts (requirements, documentation) are processed with state‑of‑the‑art natural‑language processing and large language models (LLMs) to identify architectural elements, decisions, and rationales. Source code is analyzed through static analysis and program‑understanding techniques to capture structural and behavioral information. Visual artifacts such as UML or SysML diagrams are parsed with dedicated diagram‑parsing engines to recover component relationships and architectural patterns. The framework is modular and supports a Human‑AI collaboration loop: users can supply missing or corrected information in a structured JSON format, allowing the system to refine its output for project‑specific terminology.

2. Unified Knowledge‑Base Schema – The authors design a meta‑model that blends ontology concepts with a knowledge‑graph representation. Traceability links (TLRs) are first‑class citizens, preserving the provenance of each extracted fact and enabling cross‑artifact navigation. The schema is deliberately expressive enough to support complex queries required for conformance checking, impact analysis, and QA while remaining tractable for storage and retrieval.

3. Consistency Checking and Resolution – Building on prior work, the system combines rule‑based, machine‑learning, and LLM‑driven techniques to spot contradictions (e.g., a component documented as “stateless” but implemented with stateful fields) and outdated information. Detected issues are classified by severity. Minor mismatches can be auto‑corrected; high‑impact conflicts trigger a semi‑automated workflow where the system formulates precise clarification questions for architects, presenting relevant context and suggested resolutions.

4. Agent‑Based Continuous Monitoring – An autonomous LLM‑powered agent watches the file system and version‑control events. Whenever an artifact changes, the agent re‑invokes the extraction pipeline, updates trace links, and runs the consistency checker. If a conflict is found, the agent decides whether to resolve it automatically or to seek human approval. Before performing any destructive operation (e.g., deleting obsolete entries or merging conflicting decisions), the agent explicitly asks the responsible stakeholder for confirmation. This creates a feedback loop that keeps the KB synchronized with the evolving codebase and documentation in near real‑time.

5. Retrieval‑Augmented Generation QA Interface – Leveraging the structured KB, the authors implement a QA front‑end that uses Retrieval‑Augmented Generation (RAG). A query first retrieves relevant sub‑graphs from the KB; the retrieved context is then fed to an LLM that generates a natural‑language answer, citing sources and confidence scores. This approach mirrors successful scientific‑knowledge QA systems such as HubLink and adapts them to the software‑engineering domain.

The paper also enumerates four major challenges. First, handling heterogeneous and multimodal data (code, diagrams, free‑form text, audio transcripts) requires flexible preprocessing pipelines. Second, scaling to large industrial systems stresses both computational resources and LLM context windows, demanding smart chunking and memory‑management strategies. Third, LLM hallucinations threaten the integrity of the KB; the authors propose hybrid verification that combines LLM output with formal reasoning and ontology constraints to improve explainability and trust. Fourth, the system must differentiate true architectural conflicts from extraction errors, avoiding false alarms that could erode user confidence.

In summary, the authors present a forward‑looking vision that unifies traceability, model consistency, knowledge‑graph engineering, and modern LLM capabilities into a single, continuously operating system for architectural knowledge management. If realized, this approach promises to reduce the manual effort required to keep architecture documentation in sync with implementation, enable rapid, evidence‑based decision making, and democratize access to architectural insight through conversational interfaces—ultimately improving maintainability and evolution of complex software systems.