Characterizing Human Semantic Navigation in Concept Production as Trajectories in Embedding Space

Semantic representations can be framed as a structured, dynamic knowledge space through which humans navigate to retrieve and manipulate meaning. To investigate how humans traverse this geometry, we introduce a framework that represents concept production as navigation through embedding space. Using different transformer text embedding models, we construct participant-specific semantic trajectories based on cumulative embeddings and extract geometric and dynamical metrics, including distance to next, distance to centroid, entropy, velocity, and acceleration. These measures capture both scalar and directional aspects of semantic navigation, providing a computationally grounded view of semantic representation search as movement in a geometric space. We evaluate the framework on four datasets across different languages, spanning different property generation tasks: Neurodegenerative, Swear verbal fluency, Property listing task in Italian, and in German. Across these contexts, our approach distinguishes between clinical groups and concept types, offering a mathematical framework that requires minimal human intervention compared to typical labor-intensive linguistic pre-processing methods. Comparison with a non-cumulative approach reveals that cumulative embeddings work best for longer trajectories, whereas shorter ones may provide too little context, favoring the non-cumulative alternative. Critically, different embedding models yielded similar results, highlighting similarities between different learned representations despite different training pipelines. By framing semantic navigation as a structured trajectory through embedding space, bridging cognitive modeling with learned representation, thereby establishing a pipeline for quantifying semantic representation dynamics with applications in clinical research, cross-linguistic analysis, and the assessment of artificial cognition.

💡 Research Summary

The paper proposes a novel framework for modeling human concept‑generation tasks as trajectories through high‑dimensional transformer‑based text embedding spaces. Rather than representing each produced word with an independent static vector, the authors construct cumulative embeddings: the embedding at step t encodes the entire prefix of words generated up to that point (e.g., “cat dog” for the second item). This design captures the context‑dependent nature of semantic retrieval, mirroring the role of working memory and inhibitory control in classic semantic‑fluency paradigms.

From each participant‑concept pair a time‑ordered sequence X = (x₁,…,x_N) is obtained, where N is the number of produced items. Five physics‑inspired metrics are then extracted:

1. Distance to Next – cosine distance between successive embeddings, summarised by the mean step size. Larger values indicate bigger semantic jumps (e.g., “dog” → “shark”).
2. Entropy – the binary‑coded distance series (high vs. low relative to the within‑trajectory median) is turned into a normalized Shannon entropy, quantifying the unpredictability of the search.
3. Velocity – the vector difference vₜ = xₜ₊₁ − xₜ, preserving both magnitude and direction of each semantic move.
4. Acceleration – the change in velocity aₜ = vₜ₊₁ − vₜ, reflecting how rapidly the search direction or speed changes. High acceleration corresponds to “unstable exploitation” or frequent switching.
5. Distance to Centroid – after collapsing repeated properties, a centroid vector (average of all unique property embeddings for a given concept and participant) is computed; each step’s cosine distance to this centroid measures how dispersed the search is relative to the participant’s overall semantic centre.

The framework is evaluated on four open datasets that differ in language, population, and task:

• Neurodegenerative – Spanish‑speaking Parkinson’s disease (PD), behavioral‑variant frontotemporal dementia (bvFTD), and healthy controls (HC) producing property lists for ten concrete concepts.
• Swear Fluency – English undergraduate participants generating items in several categories (animals, letters, swear words) within one minute.
• Italian Property Listing – 69 Italian students describing 50 concrete concepts across ten semantic categories.
• German Property Listing – 73 German students performing the same task as the Italian set.

For embeddings the authors primarily use OpenAI’s text‑embedding‑3‑large, and also test Google’s text‑embedding‑004, Qwen3‑Embedding‑0.6B, and a non‑cumulative fastText baseline. Results show that (i) the three transformer models yield highly similar patterns, suggesting a convergent geometry across distinct training pipelines; (ii) cumulative embeddings outperform the fastText baseline for longer sequences (e.g., the neurodegenerative dataset), whereas for short, one‑minute trials the non‑cumulative approach can be more stable; (iii) applying ZCA‑whitening to mitigate anisotropy does not substantially alter the discriminative power of the metrics.

Clinically, PD and bvFTD participants exhibit higher mean Distance to Next and higher Acceleration than HC, indicating more erratic, less focused semantic searches. Their Entropy values are also elevated, consistent with reduced executive control. bvFTD participants show the highest Distance to Centroid, reflecting a broader semantic spread. In contrast, healthy controls tend to cluster tightly around the centroid, suggesting more efficient exploitation of a semantic subspace.

Cross‑linguistic analysis reveals that the same set of metrics discriminates groups and concept types in Italian and German data, confirming that the approach is largely language‑agnostic when multilingual embeddings are used.

The authors discuss several limitations: (1) cumulative embeddings become computationally expensive for very long sequences; (2) the lack of real‑time timestamps forces the assumption of unit time steps, which may oversimplify the dynamics; (3) the mapping between embedding geometry and neural representations remains indirect. Future work is suggested to (a) develop memory‑efficient, possibly recurrent, encoders that can update cumulative representations incrementally; (b) integrate simultaneous neurophysiological recordings (EEG/fMRI) to align embedding‑derived trajectories with brain dynamics; and (c) combine the trajectory metrics with topic‑modeling or visualization tools to improve interpretability.

In sum, the paper bridges cognitive modeling of semantic search with modern NLP representation learning, offering a quantitative pipeline that captures fine‑grained, continuous aspects of human meaning navigation. The framework holds promise for clinical diagnostics, multilingual cognitive research, and for benchmarking artificial agents against human semantic behavior.