Variational Message Passing (VMP)

Variational Message Passing (VMP) is a general purpose algorithm for applying variational inference to Bayesian Networks.  The variational distribution used is factorised with respect to each variable.  Like belief propagation, VMP proceeds by sending messages between nodes in the network and updating posterior beliefs using local operations at each node. Each such update increases a lower bound on the log evidence (unless already at a local maximum).

In contrast to belief propagation, VMP can be applied to a very general class of conjugate-exponential models. Furthermore, by introducing additional variational parameters, VMP can be applied to models containing non-conjugate distributions. The VMP framework also allows the lower bound to be evaluated, and this can be used both for model comparison and for detection of convergence.