Efficient Interpolation of Computationally Expensive Posterior Densities With Variable Parameter Costs

Markov chain Monte Carlo (MCMC) is nowadays a standard approach to numerical computation of integrals of the posterior density pi of the parameter vector eta. Unfortunately, Bayesian inference using MCMC is computationally intractable when the posterior density pi is expensive to evaluate. In many such problems, it is possible to identify a minimal subvector beta of eta responsible for the expensive computation in the evaluation of pi. We propose two approaches, DOSKA and INDA, that approximate pi by interpolation in ways that exploit this computational structure to mitigate the curse of dimensionality. DOSKA interpolates pi directly while INDA interpolates pi indirectly by interpolating functions, for example, a regression function, upon which pi depends. Our primary contribution is derivation of a Gaussian processes interpolant that provably improves over some of the existing approaches by reducing the effective dimension of the interpolation problem from dim(eta) to dim(beta). This allows a dramatic reduction of the number of expensive evaluations necessary to construct an accurate approximation of pi when dim(eta) is high but dim(beta) is low. We illustrate the proposed approaches in a case study for a spatio-temporal linear model for air pollution data in the greater Boston area. Supplemental materials include proofs, details, and software implementation of the proposed procedures.