Probabilistic Bisimulations for PCTL Model Checking of Interval MDPs

Verification of PCTL properties of MDPs with convex uncertainties has been investigated recently by Puggelli et al. However, model checking algorithms typically suffer from state space explosion. In this paper, we address probabilistic bisimulation to reduce the size of such an MDPs while preserving PCTL properties it satisfies. We discuss different interpretations of uncertainty in the models which are studied in the literature and that result in two different definitions of bisimulations. We give algorithms to compute the quotients of these bisimulations in time polynomial in the size of the model and exponential in the uncertain branching. Finally, we show by a case study that large models in practice can have small branching and that a substantial state space reduction can be achieved by our approach.

💡 Research Summary

The paper addresses the verification of probabilistic computation tree logic (PCTL) properties on Markov decision processes (MDPs) whose transition probabilities are given as intervals, a model commonly referred to as an Interval MDP (IMDP). Such models capture real‑world uncertainties arising from measurement errors, statistical estimates, or approximations, and they introduce two distinct sources of nondeterminism: (i) the choice of an action by a scheduler, and (ii) the selection of a concrete probability distribution within the prescribed interval by a “nature” component. The authors observe that the way these two nondeterministic choices are resolved has a profound impact on the behavioural equivalence notions that can be employed for state‑space reduction.

Two interpretations of nondeterminism are distinguished. In the cooperative (∀) setting, which is motivated by verification of parallel systems where both scheduler and nature may conspire to produce the worst‑case behaviour, the combined effect of the two choices is modelled as the convex hull of all feasible distributions for a given state. Under this interpretation the authors define a probabilistic (∀)-bisimulation that closely mirrors the classic bisimulation for ordinary MDPs: two states are equivalent if they share the same labeling and, for every distribution reachable from one state, there exists a matching distribution from the other state that assigns identical probability mass to each equivalence class of the bisimulation relation.

In the competitive (∃σ∀ and ∃π∀) setting, relevant for control synthesis and parameter synthesis, the scheduler seeks a strategy that works against the most adversarial choice of nature (or vice‑versa). Two asymmetric bisimulations are introduced: ∼(∃σ∀) assumes there exists a scheduler such that for all possible natures the equivalence holds, while ∼(∃π∀) assumes there exists a nature such that for all schedulers the equivalence holds. The paper proves that these two relations coincide, yielding a single competitive bisimulation notion.

The core technical contribution is an algorithmic framework for computing the quotients induced by both bisimulations. The authors represent each interval transition as a polytope of feasible probability distributions. Checking whether two states are bisimilar reduces to comparing these polytopes with respect to the current partition of the state space. The algorithms run in time polynomial in the number of states and actions, but exponential in two structural parameters: (a) the maximal dimension of the polytopes (i.e., the number of distinct successor states that an uncertain transition can reach) and (b), for the competitive case, the maximal number of outgoing uncertain transitions from a state. These parameters are often small in practice, making the approach tractable.

A detailed case study on a wireless sensor network (WSN) illustrates the practical impact. The network consists of N sensors communicating with a gateway over an unreliable channel whose loss probability lies in an interval