TIDE: Trajectory-based Diagnostic Evaluation of Test-Time Improvement in LLM Agents

Recent advances in autonomous LLM agents demonstrate their ability to improve performance through iterative interaction with the environment. We define this paradigm as Test-Time Improvement (TTI). However, the mechanisms under how and why TTI succeed or fail remain poorly understood, and existing evaluation metrics fail to capture their task optimization efficiency, behavior adaptation after erroneous actions, and the specific utility of working memory for task completion. To address these gaps, we propose Test-time Improvement Diagnostic Evaluation (TIDE), an agent-agnostic and environment-agnostic framework that decomposes TTI into three comprehensive and interconnected dimensions. The framework measures (1) the overall temporal dynamics of task completion and (2) identifies whether performance is primarily constrained by recursive looping behaviors or (3) by burdensome accumulated memory. Through extensive experiments across diverse agents and environments, TIDE highlights that improving agent performance requires more than scaling internal reasoning, calling for explicitly optimizing the interaction dynamics between the agent and the environment.

💡 Research Summary

The paper introduces a systematic framework for diagnosing and quantifying “Test‑Time Improvement” (TTI) in large language model (LLM) agents that interact with environments over multiple turns. TTI is defined as the process by which an autonomous LLM agent progressively refines its behavior during test‑time interaction, using feedback from the environment to correct mistakes and achieve goals that cannot be solved by static reasoning alone. Existing evaluation metrics such as success rate (SR) collapse the entire interaction trajectory into a binary outcome, ignoring three crucial aspects: (1) how efficiently the agent converts interaction budget into progress, (2) whether the agent adapts its behavior after errors or falls into repetitive loops, and (3) how the accumulated working memory contributes to performance.

To fill this gap, the authors propose the Test‑time Improvement Diagnostic Evaluation (TIDE), an agent‑agnostic and environment‑agnostic framework that decomposes TTI into three complementary, inter‑connected dimensions. Each dimension is captured by a dedicated metric:

1. Area Under Variation (AUV) – This metric measures temporal optimization efficiency. For each interaction step t, the cumulative proportion of tasks solved by step t (denoted Pₜ) is plotted, and the trapezoidal area under this curve from 0 to a predefined horizon t_max is computed and normalized. AUV ranges from 0 (no improvement) to 1 (instantaneous success) and captures both early‑stage speed and sustained convergence, revealing differences that SR masks.

2. Loop Ratio (LR) – LR quantifies behavior stagnation caused by recursive loops. The interaction trajectory is interpreted as a path over latent environment states; a cycle is defined when the agent returns to a previously visited state without progress. Repeated execution of the same cycle constitutes a “loop”. LR is the proportion of actions belonging to such redundant loops relative to total actions. Low LR indicates active adaptation, while high LR signals that the agent is stuck repeating ineffective actions.

3. Memory Index (MI) – MI isolates the utility of the working memory accumulated during interaction. The authors separate “useful memory” (information that directly improves decision making) from “harmful memory” (noise that misguides the agent). By varying the length of the retained context and measuring performance changes, MI quantifies whether additional memory is beneficial or detrimental.

The authors evaluate TIDE across five benchmark environments (BlocksWorld, FrozenLake, Sudoku, AlfWorld, WebShop) and a diverse set of LLMs, including non‑thinking models, “thinking” models, and proprietary large‑scale systems. Experiments are split into reasoning‑bound (MDP) tasks, where the solution can be inferred without external feedback, and information‑bound (POMDP) tasks, which require active probing of the environment.

Key findings include:

• AUV reveals temporal efficiency hidden from SR. Models with identical SR can have markedly different AUV scores, indicating that one solves most tasks early while another needs many more interaction steps. For example, Gemini 2.5 Pro and DeepSeek‑V3.2 both achieve 80.7 % SR on AlfWorld, yet AUV is 0.629 vs. 0.590, respectively.

• Loop behavior is pervasive and detrimental. Most evaluated agents exhibit LR values above 20 % in environments like FrozenLake, meaning they repeatedly execute the same ineffective action sequence. A strong inverse correlation is observed between LR and AUV across models, confirming that loops suppress test‑time improvement.

• Memory is a double‑edged sword. While longer context windows can provide useful information, beyond a certain length the added tokens introduce noise, causing AUV to drop and LR to rise. This demonstrates that effective memory management is essential for TTI.

• Agent‑environment match matters. Performance is not solely a function of model size; certain models excel in specific environments (e.g., Llama‑3.3‑70B‑Instruct in BlocksWorld) but falter in others (e.g., FrozenLake). Hence, TTI efficiency depends on how well the agent’s capabilities align with environmental dynamics.

Overall, TIDE offers a unified diagnostic lens that captures optimization efficiency, adaptive behavior, and memory utility simultaneously. The framework uncovers failure modes invisible to traditional metrics and provides actionable insights for designing more robust autonomous agents. The authors argue that improving test‑time performance requires more than scaling model parameters; it demands explicit optimization of interaction dynamics, loop avoidance strategies, and principled memory handling.

In summary, the paper makes three primary contributions: (1) formalizing Test‑Time Improvement as a multi‑dimensional, interaction‑driven process; (2) introducing the TIDE framework with AUV, LR, and MI metrics for comprehensive diagnosis; and (3) empirically demonstrating across diverse agents and tasks that temporal efficiency, loop avoidance, and memory management are critical levers for achieving genuine test‑time improvement.