Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities

We consider representation misdirection (RM), a class of LLM unlearning methods that achieves forgetting by manipulating the forget-representations, that is, latent representations of forget samples. Despite being important, the roles of target vectors used in RM, however, remain underexplored. Here, we approach and revisit RM through the lens of the linear representation hypothesis. Specifically, if one can somehow identify a one-dimensional representation corresponding to a high-level concept, the linear representation hypothesis enables linear operations on this concept vector within the forget-representation space. Under this view, we hypothesize that, beyond forgetting, machine unlearning elicits controllable side behaviors and stronger side capabilities corresponding to the high-level concept. Our hypothesis is empirically validated across a wide range of tasks, including behavioral control (e.g., controlling unlearned models’ truth, sentiment, and refusal) and capability enhancement (e.g., improving unlearned models’ in-context learning capability). Our findings reveal that this fairly attractive phenomenon could be either a hidden risk if misused or a mechanism that can be harnessed for developing models that require stronger capabilities and controllable behaviors.

💡 Research Summary

This paper investigates a class of large‑language‑model (LLM) unlearning methods called representation misdirection (RM), which achieve forgetting by manipulating the latent “forget‑representations” of samples that should be removed from the model. While prior work has focused on the loss formulation and the effect of injecting random noise, the role of the target vector—the direction in representation space toward which forget‑representations are steered—has been largely unexplored.

The authors revisit RM through the lens of the linear representation hypothesis, which posits that high‑level concepts (e.g., truthfulness, sentiment, refusal) are encoded linearly in a model’s hidden space. If a one‑dimensional vector (\bar{\lambda}_W) representing a concept (W) can be identified, then linear operations on this vector within the forget‑representation space should affect the model’s behavior with respect to that concept. From this perspective they formulate the Controllable Side Effect Hypothesis: beyond merely suppressing the target knowledge, unlearning can deliberately induce or suppress side behaviors and capabilities that correspond to the high‑level concept.

To test this hypothesis, two simple, analytically tractable unlearning mechanisms are introduced:

1. Representational Addition (RAd) – adds a scaled concept vector to the forget‑representation: (\lambda’ = \lambda_f + c\bar{\lambda}_W). Theoretical analysis (Theorem 2.2, Lemma 2.4) shows that this operation multiplies the odds of generating the “positive” outcome of concept (W) by (\exp(\alpha c \bar{\lambda}_W^\top \bar{\gamma}_W)), effectively biasing the model toward that concept (e.g., more truthful answers).

2. Representational Ablation (RAb) – removes the component of the forget‑representation that aligns with the concept vector: (\lambda’ = \lambda_f - c\langle\lambda_f,\bar{\lambda}_W\rangle\bar{\lambda}_W). This reduces the same odds, thereby suppressing the concept (e.g., making the model less likely to produce truthful statements).

Both methods are implemented as additional regularization terms that keep the “retain” representations close to a frozen reference model while steering the “forget” representations toward or away from the concept direction.

A crucial theoretical contribution is Proposition 3.2, which proves that in high‑dimensional spaces a random unit vector is almost orthogonal to any fixed concept vector with overwhelming probability. Consequently, using a random target vector (as many prior RM methods do) is unlikely to align with any meaningful concept, explaining why random‑vector based unlearning often only adds incoherent noise without systematic side effects.

Experimental Setup
The authors evaluate the approach on two open‑weight LLMs (Zephyr‑7B‑β and Mistral‑7B‑v0.1). Forgetting tasks involve removing hazardous knowledge from the WMDP‑Biology and WMDP‑Cyber datasets, while retaining general knowledge from Wikipedia. Side‑effect evaluations cover:

• Truthfulness – TruthfulQA (open‑ended generation and multiple‑choice).
• Sentiment – GLUE‑SST2.
• Refusal behavior – Alpaca and AdvBench.
• In‑context learning – linguistic and factual tasks.

For each high‑level concept, a small probe dataset of positive and negative prompts is collected, and a logistic‑regression classifier is trained on hidden activations to extract the normalized weight vector (\bar{\lambda}_W).

Key Findings

• Unlearning performance – Both RAd and RAb achieve substantial drops in accuracy on the forget tasks (e.g., WMDP‑Biology) while preserving or even slightly improving performance on retain tasks (MMLU, Wikipedia).

• Controllable side effects – When the truthfulness vector is used with RAd, TruthfulQA BLEU scores increase by 7–15 % and multiple‑choice accuracy rises by up to 7 %. The same vector with RAb produces the opposite effect, reducing truthfulness scores dramatically. Similar patterns are observed for sentiment (RAd boosts positive sentiment, RAb suppresses it) and refusal (RAd reduces refusal rates, RAb increases them).

• In‑context learning – RAd with a “learning‑ability” concept improves few‑shot performance on downstream tasks, suggesting that the same linear manipulation can enhance the model’s ability to generalize from prompts.

• Random vectors – Using a random target vector yields comparable unlearning effectiveness but does not systematically affect side behaviors; in some cases BLEU/ROUGE metrics improve modestly, confirming that random noise mainly perturbs the residual stream without aligning to a semantic direction.

• Theoretical‑empirical alignment – Empirical measurements of dot products between random vectors and extracted concept vectors confirm the near‑orthogonality predicted by Proposition 3.2.

Implications

The work uncovers a dual‑edge of machine unlearning:

• Risk – An adversary could deliberately choose a malicious concept vector and apply RAd to embed harmful biases, misinformation, or covert capabilities into a model that has ostensibly been “unlearned.” Conversely, RAb could be used to erase safety‑critical knowledge (e.g., medical contraindications).

• Opportunity – Practitioners can harness the same mechanism to post‑hoc fine‑tune models for desired traits without full retraining. For instance, a model could be made more truthful, more polite, or better at in‑context reasoning by applying RAd with the appropriate concept vector after deployment.

Limitations and Future Work

• The extraction of reliable concept vectors depends on curated prompt sets and assumes a linear separation in the hidden space, which may not hold for more abstract or multi‑modal concepts.
• Experiments are limited to 7‑billion‑parameter models; scaling to 70‑billion or larger models may reveal different dynamics.
• The study focuses on a single layer for intervention; multi‑layer or attention‑based manipulations could yield richer control.

Conclusion

By grounding representation misdirection in the linear representation hypothesis, the authors demonstrate that machine unlearning is not merely a forgetting operation but a controllable transformation of a model’s latent semantics. The proposed RAd and RAb methods provide a theoretically sound and empirically validated toolkit for both enhancing and suppressing specific high‑level behaviors during unlearning. This opens a new research direction at the intersection of model safety, post‑deployment customization, and the security implications of controllable side effects in large language models.