Case-Factor Diagrams for Structured Probabilistic Modeling

We introduce a probabilistic formalism subsuming Markov random fields of bounded tree width and probabilistic context free grammars. Our models are based on a representation of Boolean formulas that we call case-factor diagrams (CFDs). CFDs are similar to binary decision diagrams (BDDs) but are concise for circuits of bounded tree width (unlike BDDs) and can concisely represent the set of parse trees over a given string undera given context free grammar (also unlike BDDs). A probabilistic model consists of aCFD defining a feasible set of Boolean assignments and a weight (or cost) for each individual Boolean variable. We give an insideoutside algorithm for simultaneously computing the marginal of each Boolean variable, and a Viterbi algorithm for finding the mininum cost variable assignment. Both algorithms run in time proportional to the size of the CFD.

💡 Research Summary

The paper introduces a unified probabilistic formalism that simultaneously subsumes bounded‑tree‑width Markov random fields (MRFs) and probabilistic context‑free grammars (PCFGs). The core contribution is the case‑factor diagram (CFD), a data structure that extends binary decision diagrams (BDDs) with two crucial features: zero‑suppression, which efficiently represents assignments where most Boolean variables are false, and factor nodes, which allow the representation of independent sub‑problems as a product of their feasible sets.

A linear Boolean model (LBM) is defined as a triple (V, F, Ψ) where V is a set of Boolean variables, F ⊆ {0,1}^V is the feasible set of truth assignments, and Ψ assigns a weight to each variable. The probability of an assignment ρ∈F is proportional to exp(‑Ψ(ρ)), with Ψ(ρ) being the sum of the weights of variables set to 1. This formulation captures log‑linear models and provides a common ground for many structured models (MRFs, HMMs, CRFs, PCFGs, etc.). The main technical challenge is to encode F compactly.

For MRFs, each variable‑value pair “y_i = v” becomes a Boolean variable, and each energy term Ψ_k over a tuple of variables is represented by a Boolean variable that is true exactly when the corresponding tuple of assignments holds. Feasibility constraints enforce that each original variable takes exactly one value and that each energy term variable mirrors its underlying assignment. For PCFGs, the authors introduce phrase variables X_{i,j}, branch variables X_{i,k}→Y_{i,j}Z_{j,k}, and terminal variables X_{i,i+1}→a. A parse tree corresponds to a unique truth assignment over these variables, and the feasible set consists of all assignments that respect the grammar’s production rules.

A CFD is defined recursively by the grammar:  D ::= case(x, D₁, D₂) | factor(D₁, D₂) | unit | empty, with the semantics:  F(case(x, D₁, D₂)) = F(D₁)