Pseudolikelihood

In statistical theory, a pseudolikelihood is an approximation to the joint probability distribution of a collection of random variables. The practical use of this is that it can provide an approximation to the likelihood function of a set of observed data which may either provide a computationally simpler problem for estimation, or may provide a way of obtaining explicit estimates of model parameters.

The pseudolikelihood approach was introduced by Julian Besag^[1] in the context of analysing data having spatial dependence.

Definition

Given a set of random variables ${\displaystyle X=X_{1},X_{2},\ldots ,X_{n}}$ the pseudolikelihood of ${\displaystyle X=x=(x_{1},x_{2},\ldots ,x_{n})}$ is

${\displaystyle L(\theta ):=\prod _{i}\mathrm {Pr} _{\theta }(X_{i}=x_{i}\mid X_{j}=x_{j}{\text{ for }}j\neq i)=\prod _{i}\mathrm {Pr} _{\theta }(X_{i}=x_{i}\mid X_{-i}=x_{-i})}$

in discrete case and

${\displaystyle L(\theta ):=\prod _{i}p_{\theta }(x_{i}\mid x_{j}{\text{ for }}j\neq i)=\prod _{i}p_{\theta }(x_{i}\mid x_{-i})=\prod _{i}p_{\theta }(x_{i}\mid x_{1},\ldots ,{\hat {x}}_{i},\ldots ,x_{n})}$

in continuous one. Here ${\displaystyle X}$ is a vector of variables, ${\displaystyle x}$ is a vector of values, ${\displaystyle p_{\theta }(\cdot \mid \cdot )}$ is conditional density and ${\displaystyle \theta =(\theta _{1},\ldots ,\theta _{p})}$ is the vector of parameters we are to estimate. The expression ${\displaystyle X=x}$ above means that each variable ${\displaystyle X_{i}}$ in the vector ${\displaystyle X}$ has a corresponding value ${\displaystyle x_{i}}$ in the vector ${\displaystyle x}$ and ${\displaystyle x_{-i}=(x_{1},\ldots ,{\hat {x}}_{i},\ldots ,x_{n})}$ means that the coordinate ${\displaystyle x_{i}}$ has been omitted. The expression ${\displaystyle \mathrm {Pr} _{\theta }(X=x)}$ is the probability that the vector of variables ${\displaystyle X}$ has values equal to the vector ${\displaystyle x}$. This probability of course depends on the unknown parameter ${\displaystyle \theta }$. Because situations can often be described using state variables ranging over a set of possible values, the expression ${\displaystyle \mathrm {Pr} _{\theta }(X=x)}$ can therefore represent the probability of a certain state among all possible states allowed by the state variables.

The pseudo-log-likelihood is a similar measure derived from the above expression, namely (in discrete case)

${\displaystyle l(\theta ):=\log L(\theta )=\sum _{i}\log \mathrm {Pr} _{\theta }(X_{i}=x_{i}\mid X_{j}=x_{j}{\text{ for }}j\neq i).}$

One use of the pseudolikelihood measure is as an approximation for inference about a Markov or Bayesian network, as the pseudolikelihood of an assignment to ${\displaystyle X_{i}}$ may often be computed more efficiently than the likelihood, particularly when the latter may require marginalization over a large number of variables.

Properties

Use of the pseudolikelihood in place of the true likelihood function in a maximum likelihood analysis can lead to good estimates, but a straightforward application of the usual likelihood techniques to derive information about estimation uncertainty, or for significance testing, would in general be incorrect.^[2]

References