Updating Probabilities: An Econometric Example

We demonstrate how information in the form of observable data and moment constraints are introduced into the method of Maximum relative Entropy (ME). A general example of updating with data and moments is shown. A specific econometric example is solved in detail which can then be used as a template for real world problems. A numerical example is compared to a large deviation solution which illustrates some of the advantages of the ME method.

💡 Research Summary

The paper presents a comprehensive treatment of updating probability distributions using the method of Maximum relative Entropy (ME) when both observed data and moment constraints are available. It begins by reviewing the classic Maximum Entropy (MaxEnt) principle and its extension to ME, which allows the incorporation of prior information together with new constraints to select a posterior distribution that maximizes relative entropy with respect to a joint prior.

In the first part, the authors show that when only a data point (x’) is observed, the ME framework imposes a delta‑function constraint (P(x)=\delta(x-x’)) on the joint distribution (P(x,\theta)). By introducing a Lagrange multiplier (\lambda(x)) and maximizing the relative entropy, they derive the posterior (P_{\text{new}}(\theta)=P_{\text{old}}(\theta\mid x’)). This result is identical to the standard Bayesian updating rule, establishing a formal equivalence between ME and Bayes’ theorem for pure data constraints.

The second, more novel, contribution is the simultaneous handling of a data constraint and a moment constraint of the form (\int f(\theta) P(x,\theta),dx,d\theta = F). Adding a second Lagrange multiplier (\beta) leads to a posterior that can be written as

\