Representation Learning to Study Temporal Dynamics in Tutorial Scaffolding

Adaptive scaffolding enhances learning, yet the field lacks robust methods for measuring it within authentic tutoring dialogue. This gap has become more pressing with the rise of remote human tutoring and large language model-based systems. We introduce an embedding-based approach that analyzes scaffolding dynamics by aligning the semantics of dialogue turns, problem statements, and correct solutions. Specifically, we operationalize alignment by computing cosine similarity between tutor and student contributions and task-relevant content. We apply this framework to 1,576 real-world mathematics tutoring dialogues from the Eedi Question Anchored Tutoring Dialogues dataset. The analysis reveals systematic differences in task alignment and distinct temporal patterns in how participants ground their contributions in problem and solution content. Further, mixed-effects models show that role-specific semantic alignment predicts tutorial progression beyond baseline features such as message order and length. Tutor contributions exhibited stronger grounding in problem content early in interactions. In contrast, student solution alignment was modestly positively associated with progression. These findings support scaffolding as a continuous, role-sensitive process grounded in task semantics. By capturing role-specific alignment over time, this approach provides a principled method for analyzing instructional dialogue and evaluating conversational tutoring systems.

💡 Research Summary

The paper addresses a critical gap in the measurement of adaptive scaffolding within authentic tutoring dialogues, a problem that has become increasingly urgent with the rise of remote human tutoring and large‑language‑model (LLM) based tutoring systems. The authors propose a continuous, task‑grounded operationalization of scaffolding based on semantic alignment between dialogue turns and two task anchors: the problem statement and the correct solution. Using sentence embeddings generated by the all‑MiniLM‑L6‑v2 model (384‑dimensional vectors), they compute cosine similarity scores for each tutor and student utterance against the problem text and the solution text. These similarity scores serve as quantitative proxies for “problem‑focused” and “solution‑focused” scaffolding.

The empirical work is conducted on the Eedi‑2K “Question‑Anchored Tutoring Dialogues” corpus, which contains 1,576 real‑world, chat‑based mathematics tutoring sessions (55,322 conversational moves) involving 25 human tutors and multiple students. After preprocessing, three textual components are embedded: (1) each tutor and student message, (2) the diagnostic problem description, and (3) the reference solution. Cosine similarity between each message and the two anchors yields three primary alignment metrics: problem alignment, solution alignment, and a baseline question‑solution similarity. Two temporal baselines—absolute turn index and normalized position (n/N)—are also computed to allow cross‑dialogue comparison.

Three research questions guide the analysis: (RQ1) how do tutor and student roles differ in semantic alignment to the problem and solution? (RQ2) how does this alignment evolve over the course of a tutoring episode? (RQ3) does role‑specific alignment predict dialogue progression beyond simple turn‑order and length features?

For RQ1, density plots reveal a bimodal distribution for tutor problem alignment (peaks near 0.1 and 0.4) while student problem alignment clusters at low values (~0.1). Both roles show left‑skewed solution alignment with minimal differentiation. The bimodality suggests tutors alternate between a low‑alignment “prompting” mode and a higher‑alignment “re‑statement” mode, whereas students rarely reuse problem language.

For RQ2, the authors aggregate alignment scores by normalized dialogue position and apply Gaussian smoothing. Tutor problem alignment rises early, then gradually declines, indicating early grounding in the problem that tapers as the session proceeds. Student problem alignment remains flat and low throughout. Solution alignment for both roles stays relatively stable, with a slight upward trend toward the end, reflecting a gradual shift toward solution‑oriented language as the problem nears completion.

RQ3 is examined with a series of nested linear mixed‑effects models predicting logit‑transformed progression (the proportion of the dialogue completed at a given turn). Model 0 includes only absolute turn order; Model 1 adds message length; Model 2 adds both problem and solution cosine similarities; Model 3 allows these semantic effects to vary by role. Model comparisons using BIC and likelihood‑ratio tests show that each successive addition improves fit, with the full role‑varying model achieving the best balance of explanatory power and parsimony. Fixed‑effect estimates reveal: (i) turn order is the strongest positive predictor (β ≈ 1.72); (ii) message length contributes a modest positive effect (β ≈ 0.14); (iii) tutor problem similarity and tutor solution similarity are negatively associated with progression (β ≈ ‑0.04 and ‑0.07, respectively); (iv) student problem similarity is also negative (β ≈ ‑0.09); (v) student solution similarity is positively associated (β ≈ 0.07). These patterns suggest that early problem‑focused alignment (especially by tutors) tends to prolong the interaction, whereas later solution‑focused alignment by students signals imminent task completion.

The discussion situates these findings within the broader scaffolding literature. The bimodal tutor problem alignment aligns with theories that tutors interleave probing questions with occasional restatements of task constraints to reduce cognitive load. The relatively flat student problem alignment underscores learners’ tendency to generate explanatory language rather than echo problem phrasing. Stable solution alignment across time counters the “fluency heuristic” concern that excessive solution exposure would inflate perceived mastery. The modest but significant predictive effects of alignment demonstrate that semantic similarity is a meaningful, fine‑grained indicator of instructional dynamics, surpassing coarse metrics like turn count or hint usage.

Finally, the authors argue that embedding‑based alignment offers a scalable, domain‑agnostic tool for evaluating both human and AI tutoring systems. Future work could compare alignment patterns between human tutors and LLM tutors, integrate real‑time alignment feedback into adaptive scaffolding algorithms, or extend the approach to other subject areas and multimodal interactions.