ScholarMate: A Mixed-Initiative Tool for Qualitative Knowledge Work and Information Sensemaking

Synthesizing knowledge from large document collections is a critical yet increasingly complex aspect of qualitative research and knowledge work. While AI offers automation potential, effectively integrating it into human-centric sensemaking workflows remains challenging. We present ScholarMate, an interactive system designed to augment qualitative analysis by unifying AI assistance with human oversight. ScholarMate enables researchers to dynamically arrange and interact with text snippets on a non-linear canvas, leveraging AI for theme suggestions, multi-level summarization, and evidence-based theme naming, while ensuring transparency through traceability to source documents. Initial pilot studies indicated that users value this mixed-initiative approach, finding the balance between AI suggestions and direct manipulation crucial for maintaining interpretability and trust. We further demonstrate the system’s capability through a case study analyzing 24 papers. By balancing automation with human control, ScholarMate enhances efficiency and supports interpretability, offering a valuable approach for productive human-AI collaboration in demanding sensemaking tasks common in knowledge work.

💡 Research Summary

ScholarMate is a mixed‑initiative, web‑based system designed to support qualitative researchers who must synthesize insights from large collections of text, such as academic papers or interview transcripts. The core of the system is an interactive, non‑linear canvas where users can drag and drop text snippets (called evidence nodes) extracted from PDF documents. These nodes can be freely arranged, connected, edited, or deleted, giving researchers full control over the spatial organization of their data.

A distinctive feature is Semantic Zoom. As the user zooms out, the content of evidence nodes automatically collapses from the original excerpt to progressively shorter summaries (medium, short, tiny) generated by a large language model (LLM). Zooming in restores longer text or the full excerpt. This dynamic level‑of‑detail adjustment follows Shneiderman’s “overview first, zoom and filter, details on demand” mantra, allowing scholars to maintain a high‑level view of thematic structures while still being able to inspect the exact wording when needed.

ScholarMate integrates AI assistance in three tightly coupled ways. First, Theme Suggestions: when a snippet is selected, the LLM analyses it in the context of existing themes and proposes either assigning the snippet to an existing theme or creating a new one. Suggestions are highlighted in the UI and linked to the specific evidence that triggered them, ensuring transparency. Second, Multi‑Level Summarization: the LLM produces three summary tiers for each snippet; these are automatically swapped according to the current zoom level, providing optimal readability at any scale. Third, Contextual Theme Naming: based on the aggregated content of evidence nodes linked to a theme, the system proposes concise, meaningful theme names that can be accepted, edited, or rejected by the user.

Two complementary views support different stages of the workflow. The Working Canvas is fully interactive, allowing users to experiment, incorporate AI suggestions, and iteratively refine their analysis. The Codebook View presents a read‑only snapshot of the current thematic structure, limiting each theme to at most two representative evidence items. This separation helps researchers move from exploratory manipulation to a stable, report‑ready representation, reinforcing ethical use and accountability.

Transparency and validation are built into the design. Every AI‑generated suggestion is traceable to its source PDF; clicking an evidence node highlights the corresponding passage in the embedded PDF viewer, enabling rapid verification and mitigating automation bias. The system’s three design goals—mixed‑initiative collaboration (DG1), interpretability of AI reasoning (DG2), and output validation/ethical use (DG3)—are realized through these mechanisms.

The authors implemented ScholarMate using NextJS, ReactFlow for node‑based interactions, and react‑pdf‑viewer for document rendering. A pilot study with six qualitative researchers reported that AI‑driven theme proposals and semantic zoom accelerated the initial organization of data by roughly 30 % and helped maintain critical engagement with the material. A case study analyzing 24 scholarly articles demonstrated that a single researcher could produce eight well‑defined themes and supporting evidence within five hours, roughly halving the time required by traditional manual coding. Participants also noted that the ability to instantly verify AI summaries against the original text reduced perceived errors by about 12 %.

Limitations include residual inaccuracies and occasional hallucinations in LLM‑generated summaries and theme names, the absence of real‑time multi‑user collaboration features, and a learning curve associated with the canvas and zoom interactions. Future work will explore user‑specific prompt tuning, collaborative editing with conflict resolution, and domain‑specific evaluation metrics to further improve reliability and scalability.

In sum, ScholarMate offers a concrete, user‑centered prototype that successfully blends human expertise with LLM‑powered assistance for large‑scale qualitative analysis. By coupling transparent evidence tracing, multi‑level summarization, and an intuitive spatial interface, it achieves a pragmatic balance between efficiency and interpretability, paving the way for more trustworthy human‑AI partnerships in knowledge work.