Exact Rank Hypothesis Tests

Introduction

The exact rank hypothesis testing strategy is based on Algorithm 2 by Gandy & Scott (2021)[GS2021].

ExactRankTest

The ranks are computed by simulating a single Markov chain backwards and forward. First, a random midpoint is simulated as

\[\begin{aligned} M \sim \mathrm{Uniform}(1,\;L), \end{aligned}\]

where $L = \texttt{n\_mcmc\_steps}$. Then, we simulate the Markov chain forward and backward as

\[\begin{alignat*}{3} \theta_{M-l} &\sim K\left(\theta_{L-l+1}, \cdot \right) \qquad &&\text{for}\; l = 1, \ldots, M-1 \\ \theta_{l} &\sim K\left(\theta_{l-1}, \cdot \right) \qquad &&\text{for}\; l = M+1, \ldots, L \end{alignat*}\]

forming the chain

\[\theta_1,\; \ldots, \; \theta_{M},\; \ldots, \; \theta_{L}.\]

The rank is the ranking of the statistics of $\theta_{M}$. If the sampler and the model are correct, the rank has an uniform distribution as long as the midpoint is independently sampled.

MCMCTesting.ExactRankTest — Type
ExactRankTest(n_samples, n_mcmc_steps; n_mcmc_thin)

Exact rank hypothesis testing strategy for reversible MCMC kernels. Algorithm 2 in Gandy & Scott (2021).

Arguments

  • n_samples::Int: Number of ranks to be simulated.
  • n_mcmc_steps::Int: Number of MCMC states to be simulated for simulating a single rank.
  • n_mcmc_thin::Int: Number of thinning applied to the MCMC chain.

Returns

  • pvalues: P-value computed for each dimension of the statistic returned from statistics.

Requirements

This test requires the following functions for model and kernel to be implemented:

  • markovchain_transition
  • sample_joint

Furthermore, this test explicitly assumes the following

  • kernel is reversible.

Applying this tests to an irreversible kernel will result in false negatives even if its stationary distribution is correct.

Keyword Arguments for Tests

When calling mcmctest or seqmcmctest, this tests has an additional keyword argument:

  • uniformity_test_pvalue: The p-value calculation strategy.

The default strategy is an $\chi^2$ test. Any function returning a single p-value from a uniformity hypothesis test will work. The format is as follows:

uniformity_test_pvalue(x::AbstractVector)::Real
source

References

  • GS2021Gandy, A., & Scott, J. (2020). Unit testing for MCMC and other Monte Carlo methods. arXiv preprint arXiv:2001.06465.