Scene-Aware Memory Discrimination: Deciding Which Personal Knowledge Stays

Intelligent devices have become deeply integrated into everyday life, generating vast amounts of user interactions that form valuable personal knowledge. Efficient organization of this knowledge in user memory is essential for enabling personalized applications. However, current research on memory writing, management, and reading using large language models (LLMs) faces challenges in filtering irrelevant information and in dealing with rising computational costs. Inspired by the concept of selective attention in the human brain, we introduce a memory discrimination task. To address large-scale interactions and diverse memory standards in this task, we propose a Scene-Aware Memory Discrimination method (SAMD), which comprises two key components: the Gating Unit Module (GUM) and the Cluster Prompting Module (CPM). GUM enhances processing efficiency by filtering out non-memorable interactions and focusing on the salient content most relevant to application demands. CPM establishes adaptive memory standards, guiding LLMs to discern what information should be remembered or discarded. It also analyzes the relationship between user intents and memory contexts to build effective clustering prompts. Comprehensive direct and indirect evaluations demonstrate the effectiveness and generalization of our approach. We independently assess the performance of memory discrimination, showing that SAMD successfully recalls the majority of memorable data and remains robust in dynamic scenarios. Furthermore, when integrated into personalized applications, SAMD significantly enhances both the efficiency and quality of memory construction, leading to better organization of personal knowledge.

💡 Research Summary

The paper addresses the growing challenge of organizing the massive amounts of personal interaction data generated by intelligent devices. While prior work on memory‑assisted LLM agents has focused on writing, managing, and reading memory, it has largely ignored the fact that a substantial portion of collected interactions are irrelevant or redundant, leading to noisy memories and unnecessary computational overhead. To solve this, the authors introduce a new “memory discrimination” task: a binary decision for each interaction—whether it should be stored in the user’s personal memory given the current application’s needs.

Inspired by the human brain’s selective‑attention filter, they propose Scene‑Aware Memory Discrimination (SAMD), which consists of two complementary modules:

1. Gating Unit Module (GUM) – operates at the earliest stage of memory processing. It defines “memory scenes” (personal attributes, relations, events) and extracts salient keywords that typically appear in each scene. To ensure coverage, the authors simulate diverse user‑device contexts through multi‑view role‑playing, generating a rich set of scene‑specific vocabularies. Using a gating mechanism derived from selective‑attention theory, GUM scores each incoming token and blocks those below a learned threshold, thereby preventing non‑memorable data from ever reaching the LLM. This dramatically reduces the number of LLM calls and cuts computational cost.

2. Cluster Prompting Module (CPM) – creates adaptive memory standards and conveys them to the frozen LLM. User intent signals are harvested from voice assistants, dialogue systems, and QA modules. By analyzing large‑scale interaction logs, the authors build an intent‑scene affinity matrix, decompose it, and cluster similar intents. For each intent cluster, a set of discrimination rules is generated based on the associated memory scenes (e.g., “remember attribute information, discard routine commands”). These rules are embedded in prompts that explicitly tell the LLM what to retain or discard for any given interaction.

The paper evaluates SAMD through both direct and indirect experiments. Direct evaluation measures discrimination accuracy, recall of memorable data, false‑positive blocking rate, and resource usage (latency, GPU memory). Results show that GUM blocks roughly 70 % of non‑memorable inputs while preserving over 90 % of truly valuable information, and overall pipeline cost drops by about 30 %. Indirect evaluation integrates SAMD into downstream personalized applications such as knowledge‑graph construction, information retrieval, and recommendation. In these settings, agents equipped with SAMD achieve 8–12 % higher task accuracy and receive better user satisfaction scores, confirming that cleaner memories translate into better downstream performance.

A key advantage of SAMD is its dynamic adaptability. When application requirements evolve or new user intents emerge, only the scene identifiers in GUM and the discrimination rules in CPM need updating; the underlying frozen LLM remains unchanged. This enables rapid deployment of new services without costly re‑training.

In summary, the authors successfully translate the concept of selective attention into a practical memory‑discrimination framework, delivering both efficiency (by early filtering) and effectiveness (by context‑aware prompting). SAMD represents a significant step toward scalable, high‑quality personal memory management for LLM‑driven agents, and opens avenues for future work on multimodal data, real‑time streaming, and privacy‑preserving memory policies.