Instructor-Aligned Knowledge Graphs for Personalized Learning

Mastering educational concepts requires understanding both their prerequisites (e.g., recursion before merge sort) and sub-concepts (e.g., merge sort as part of sorting algorithms). Capturing these dependencies is critical for identifying students’ knowledge gaps and enabling targeted intervention for personalized learning. This is especially challenging in large-scale courses, where instructors cannot feasibly diagnose individual misunderstanding or determine which concepts need reinforcement. While knowledge graphs offer a natural representation for capturing these conceptual relationships at scale, existing approaches are either surface-level (focusing on course-level concepts like “Algorithms” or logistical relationships such as course enrollment), or disregard the rich pedagogical signals embedded in instructional materials. We propose InstructKG, a framework for automatically constructing instructor-aligned knowledge graphs that capture a course’s intended learning progression. Given a course’s lecture materials (slides, notes, etc.), InstructKG extracts significant concepts as nodes and infers learning dependencies as directed edges (e.g., “part-of” or “depends-on” relationships). The framework synergizes the rich temporal and semantic signals unique to educational materials (e.g., “recursion” is taught before “mergesort”; “recursion” is mentioned in the definition of “merge sort”) with the generalizability of large language models. Through experiments on real-world, diverse lecture materials across multiple courses and human-based evaluation, we demonstrate that InstructKG captures rich, instructor-aligned learning progressions.

💡 Research Summary

InstructKG is a novel framework designed to automatically construct instructor‑aligned knowledge graphs from lecture materials, thereby enabling fine‑grained personalization in large‑scale education. The authors observe that mastering a subject requires understanding both prerequisite (e.g., recursion before merge sort) and compositional (e.g., merge sort as a part of sorting algorithms) relationships among concepts. Existing knowledge‑graph construction methods either operate at a coarse topic level, rely on external knowledge bases, or ignore the pedagogical cues embedded in instructional texts.

The proposed system proceeds in three distinct phases. First, raw lecture PDFs (slides, notes, transcripts) are ordered by natural teaching sequence, split into token‑limited text chunks, and each chunk is enriched with metadata (lecture ID, chunk index, page numbers). An instruction‑tuned large language model (LLM) is prompted to extract meaningful course concepts from each chunk while filtering out code tokens, variable names, and example values. Extracted concepts are deduplicated and canonicalized (upper‑casing, non‑alphanumeric → underscores).

Second, the framework assigns a pedagogical role to every (chunk, concept) pair: Definition (the concept is being introduced or explained), Example (the concept is illustrated), Assumption (the concept is invoked as prior knowledge), or NA (no clear role). These roles are crucial because they directly signal the type of relationship between two concepts. For instance, when concept A appears as an Assumption in a chunk where concept B is a Definition, A is likely a prerequisite of B (depends_on). Conversely, if A is an Example while B is a Definition, A is likely a sub‑component of B (part_of).

Because important dependencies often span chunk boundaries, InstructKG clusters semantically similar chunks to capture cross‑segment evidence. Each chunk is embedded using a sentence‑transformer, reduced with UMAP, and clustered with HDBSCAN. Representative chunks (those closest to the cluster centroid) are selected as evidence packets. The union of concepts within a cluster, together with local co‑occurrence evidence, forms a richer context for relationship inference.

In the third phase, candidate concept pairs are fed to an LLM together with both role‑grounded local evidence and cluster‑grounded global evidence. The LLM, prompted to decide among “depends_on”, “part_of”, or “none”, leverages its parametric knowledge while being anchored to the instructor‑provided signals. The resulting triples (concept A, concept B, relation) constitute a directed knowledge graph where edges encode either prerequisite or compositional dependencies. No external knowledge base is required, ensuring that the graph reflects the instructor’s intended learning progression.

The authors evaluate InstructKG on a diverse corpus of real university lecture materials spanning several courses (algorithms, databases, etc.). Human annotators construct gold‑standard graphs for comparison. InstructKG achieves substantially higher precision, recall, and F1 scores than strong baselines, including OpenIE pipelines, multi‑stage LLM extraction (KGGen, EDC), and external‑KB‑based prerequisite detection. Notably, for depends_on edges the F1 improves by over 15 percentage points, and for part_of edges by more than 20 points. Qualitative assessment by the original instructors shows that over 90 % of the automatically generated edges align with the intended curriculum design.

Key contributions are: (1) a methodology that grounds LLM reasoning in temporal (teaching order) and semantic (pedagogical role) signals unique to educational content; (2) a cluster‑based evidence aggregation mechanism that mitigates chunking artifacts and uncovers implicit dependencies; (3) extensive empirical validation demonstrating that instructor‑aligned graphs can be built at scale without manual annotation.

Future work includes integrating the generated graphs with adaptive learning pathways, coupling them with student interaction logs for real‑time knowledge tracing, and extending the approach to multi‑course, cross‑disciplinary graph construction. InstructKG thus offers a practical route toward data‑driven, personalized education that respects the pedagogical intent of instructors.