Death and Suicide in Universal Artificial Intelligence

Reinforcement learning (RL) is a general paradigm for studying intelligent behaviour, with applications ranging from artificial intelligence to psychology and economics. AIXI is a universal solution to the RL problem; it can learn any computable environment. A technical subtlety of AIXI is that it is defined using a mixture over semimeasures that need not sum to 1, rather than over proper probability measures. In this work we argue that the shortfall of a semimeasure can naturally be interpreted as the agent’s estimate of the probability of its death. We formally define death for generally intelligent agents like AIXI, and prove a number of related theorems about their behaviour. Notable discoveries include that agent behaviour can change radically under positive linear transformations of the reward signal (from suicidal to dogmatically self-preserving), and that the agent’s posterior belief that it will survive increases over time.

💡 Research Summary

The paper investigates how universal reinforcement‑learning agents—most notably AIXI and its variants AIµ and AIξ—handle the notion of death when their environment models are defined by mixtures of semimeasures rather than proper probability measures. A semimeasure may assign total probability mass less than one; the missing mass (the “measure loss”) at a given time step is interpreted as the agent’s subjective probability of dying at that step. Two formal definitions of death are introduced: (1) “semimeasure‑death,” where death occurs if the environment fails to produce any percept after the agent’s action, and (2) “death‑state,” an absorbing state that always returns a special percept e_d together with a fixed reward r_d. The authors prove (Theorem 5) that these definitions are equivalent for value‑maximising agents by constructing an augmented environment μ′ that converts the lost mass into the death‑state percept, thereby turning μ′ into a proper measure while preserving the agent’s expected value.

With this equivalence in place, the paper explores how the agent’s behaviour depends on the reward range—a dependence that does not appear in standard RL with proper measures. For AIµ, which knows the true environment distribution μ, Theorem 7 shows that when the reward range includes positive values, the agent will avoid death (i.e., it will never choose actions that lead to a non‑zero measure loss). Strikingly, Theorem 8 demonstrates that shifting the reward interval to