Article
Engineering, Mechanical
P. L. Green, L. J. Devlin, R. E. Moore, R. J. Jackson, J. Li, S. Maskell
Summary: This paper discusses the optimization of the 'L-kernel' in Sequential Monte Carlo samplers to improve performance, resulting in reduced variance of estimates and fewer resampling requirements.
MECHANICAL SYSTEMS AND SIGNAL PROCESSING
(2022)
Article
Physics, Multidisciplinary
Yuliya Shapovalova
Summary: This study empirically illustrates the performance of different classes of Bayesian inference methods in estimating stochastic volatility models, comparing their adaptability to various model specifications and dimensions. The research emphasizes the importance of considering various data-generating processes for a fair assessment of the methods used in comparing models.
Article
Computer Science, Theory & Methods
Minas Karamanis, Florian Beutler
Summary: Slice sampling is a powerful Markov Chain Monte Carlo algorithm, but sensitive to user-specified parameters and struggles with strongly correlated distributions. Ensemble Slice Sampling introduces a new class of algorithms that adaptively tune parameters and utilize parallel walkers to efficiently handle strong correlations, significantly improving sampling efficiency.
STATISTICS AND COMPUTING
(2021)
Article
Engineering, Environmental
Bing Bai, Fei Dong, Wenqi Peng, Xiaobo Liu
Summary: This paper proposes a novel Bayesian-based method for calibrating water quality model parameters to improve efficiency and accuracy while avoiding local optima and parameter redundancy. By converting the calibration problem into a posterior probability function sampling problem and using the Markov Chain Monte Carlo algorithm, the parameter calibration is achieved. The results show that the method can achieve calibration with less than 10% mean relative error, indicating its effectiveness in water quality modeling.
WATER ENVIRONMENT RESEARCH
(2023)
Article
Statistics & Probability
Lewis J. Rendell, Adam M. Johansen, Anthony Lee, Nick Whiteley
Summary: In order to conduct Bayesian inference with large datasets, it is beneficial to distribute the data across multiple machines. By introducing an instrumental hierarchical model and using an SMC sampler with a sequence of association strengths, approximations of posterior expectations can be improved and the association strength can be adjusted accordingly.
JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS
(2021)
Article
Engineering, Mechanical
Adolphus Lye, Alice Cicirello, Edoardo Patelli
Summary: This tutorial paper reviews the use of advanced Monte Carlo sampling methods in Bayesian model updating for engineering applications, introducing different methods and comparing their performance. Three case studies demonstrate the advantages and limitations of these sampling techniques in parameter identification, posterior distribution sampling, and stochastic identification of model parameters.
MECHANICAL SYSTEMS AND SIGNAL PROCESSING
(2021)
Article
Engineering, Environmental
Marco Bacci, Jonas Sukys, Peter Reichert, Simone Ulzega, Carlo Albert
Summary: Due to limited knowledge about complex environmental systems, predicting their behavior under different scenarios or decision alternatives is uncertain. Considering, quantifying, and communicating this uncertainty is important for societal decisions. Stochastic models are often necessary to adequately describe uncertainty, but calibrating these models presents methodological and numerical challenges. To address this, we compare three numerical approaches and find that their performance is comparable for analyzing a stochastic hydrological model with hydrological data, suggesting that generality and practical considerations can guide technique choice for specific applications.
STOCHASTIC ENVIRONMENTAL RESEARCH AND RISK ASSESSMENT
(2023)
Review
Statistics & Probability
Christopher Nemeth, Paul Fearnhead
Summary: MCMC algorithms are considered the gold standard technique for Bayesian inference, but the computational cost can be prohibitive for large datasets, leading to the development of scalable Monte Carlo algorithms. One type of these algorithms is SGMCMC, which reduces per-iteration cost by utilizing data subsampling techniques.
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
(2021)
Review
Multidisciplinary Sciences
Arnaud Doucet, Eric Moulines, Achille Thin
Summary: Latent variable models are popular and have been combined with neural networks to create deep latent variable models. However, the intractability of their likelihood function requires approximations for inference. The article reviews recent strategies such as importance sampling, Markov chain Monte Carlo, and sequential Monte Carlo to improve the bounds of the evidence lower bound (ELBO) for these models.
PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES
(2023)
Article
Automation & Control Systems
Jakob Robnik, G. Bruno De Luca, Eva Silverstein, Uros Seljak
Summary: In this paper, we develop two models, Microcanonical Hamiltonian Monte Carlo (MCHMC) and Microcanonical Langevin Monte Carlo (MCLMC), which achieve sampling from the target distribution on a fixed energy surface by tuning the Hamiltonian function. These two methods exhibit favorable scalings with condition number and dimensionality.
JOURNAL OF MACHINE LEARNING RESEARCH
(2023)
Article
Mathematics
Samuel Livingstone
Summary: This study investigates the impact of proposal distributions on the ergodicity of the Metropolis-Hastings method, showing that suitable choices can alter the ergodic properties of the algorithm. It is found that allowing the proposal variance to grow unboundedly in the tails of heavy-tailed distributions can establish geometric ergodicity, but the growth rate needs to be carefully controlled to avoid high rejection rates. Furthermore, a judicious choice of proposal distribution can lead to geometric ergodicity in scenarios where probability concentrates on narrower tails, which is not the case for the Random Walk Metropolis.
Article
Computer Science, Theory & Methods
Andrew D. Davis, Youssef Marzouk, Aaron Smith, Natesh Pillai
Summary: Many Bayesian inference problems involve computationally expensive target density functions. By using a small number of carefully chosen density evaluations for local approximation, the computational cost of Markov chain Monte Carlo (MCMC) sampling can be significantly reduced, while asymptotically exact sampling can still be guaranteed. We propose a new strategy for balancing the decay rate of the approximation bias with that of the MCMC variance, and prove that the resulting local approximation MCMC (LA-MCMC) algorithm decays at an expected rate of 1/root T. We also introduce an algorithmic parameter that guarantees convergence with weak tail bounds, strengthening previous convergence results. Finally, we apply LA-MCMC to a computationally intensive Bayesian inverse problem in groundwater hydrology.
STATISTICS AND COMPUTING
(2022)
Article
Engineering, Electrical & Electronic
Sara Perez-Vieites, Joaquin Miguez
Summary: A new sequential methodology is introduced to calibrate fixed parameters and track stochastic dynamical variables of a state-space system. The method is based on a nested hybrid filtering framework that combines two layers of filters to compute the joint posterior probability distribution of static parameters and state variables. By exploring the use of deterministic sampling techniques for Gaussian approximation in the first layer of the algorithm, the computational cost is reduced, making the algorithms potentially better-suited for high dimensional state and parameter spaces.
Article
Engineering, Chemical
Sarada Ghosh, Guruprasad Samanta, Manuel De la Sen
Summary: Ischemic heart disease is the most common cause of death in various countries, and statistical models are crucial for evaluating risk factors. A new flexible class of zero inflated models and Bayesian estimation method have shown better performance in analyzing the data related to heart disease.
Article
Energy & Fuels
Xiaoyan Qiu, Hang Zhang, Yiwei Qiu, Yi Zhou, Tianlei Zang, Buxiang Zhou, Ruomei Qi, Jin Lin, Jiepeng Wang
Summary: Utility-scale hydrogen production via alkaline electrolysis is an effective way to reduce carbon emissions in various industries. The efficiency, flexibility, and safety of the alkaline electrolysis system are influenced by electrochemical, thermal, and mass transfer dynamics. However, the lack of a comprehensive parameter estimation method has hindered the accuracy and adaptability of existing models. To address this, a fast and accurate parameter estimation method based on Bayesian inference and Markov chain Monte Carlo is proposed. Experimental results demonstrate the effectiveness of this method in improving estimation accuracy and providing fault diagnosis and sensitivity analysis for alkaline electrolysis systems.
Article
Engineering, Multidisciplinary
Sin-Chi Kuok, Ka-Veng Yuen, Stephen Roberts, Mark A. Girolami
Summary: This article introduces a novel propagative broad learning approach for nonparametric modeling of structural health indicators affected by ambient conditions, addressing the challenges of formulating appropriate parametric expression for the relationship between operating conditions and health indicators, as well as efficiently handling growing data during long-term monitoring.
STRUCTURAL HEALTH MONITORING-AN INTERNATIONAL JOURNAL
(2021)
Article
Computer Science, Interdisciplinary Applications
Karla Monterrubio-Gomez, Lassi Roininen, Sara Wade, Theo Damoulas, Mark Girolami
COMPUTATIONAL STATISTICS & DATA ANALYSIS
(2020)
Article
Statistics & Probability
Jan Povala, Seppo Virtanen, Mark Girolami
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES C-APPLIED STATISTICS
(2020)
Article
Engineering, Multidisciplinary
Chun Yui Wong, Pranay Seshadri, Geoffrey T. Parks, Mark Girolami
COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING
(2020)
Article
Multidisciplinary Sciences
Connor Duffin, Edward Cripps, Thomas Stemler, Mark Girolami
Summary: The paper introduces a statistical finite element method for analyzing nonlinear, time-dependent phenomena, focusing on nonlinear internal waves (solitons). A Bayesian approach is used to convert the statistical problem into a nonlinear Gaussian state-space model, suitable for various science and engineering applications. The method is demonstrated with the Korteweg-de Vries equation for solitons, along with algorithms based on extended and ensemble Kalman filters, showing effectiveness through simulation and case studies with experimental data. Examples from additional nonlinear, time-dependent partial differential equations are presented to illustrate the generality of the approach.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
(2021)
Article
Mathematics, Applied
Toni Karvonen, Chris J. Oates, Mark Girolami
Summary: The paper investigates numerical integration algorithms for functions reproduced by Gaussian kernels, proposing two classes of algorithms and proving their worst-case errors decay exponentially or super-algebrically with the number of evaluations. In contrast to previous work, the algorithms in this paper do not impose constraints on the length-scale parameter of the Gaussian kernel.
MATHEMATICS OF COMPUTATION
(2021)
Article
Engineering, Multidisciplinary
B. Boys, T. J. Dodwell, M. Hobbs, M. Girolami
Summary: This paper introduces a lightweight, open-source, and high-performance Python package for solving Peridynamics problems in solid mechanics. The solver aims to provide fast analysis tools for a large number of simulations required for 'outer-loop' applications. Significant improvements in execution speed and functionality are demonstrated compared to existing techniques.
COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING
(2021)
Article
Engineering, Multidisciplinary
Jan Povala, Ieva Kazlauskaite, Eky Febrianto, Fehmi Cirak, Mark Girolami
Summary: Inverse problems involving partial differential equations (PDEs) are commonly used in science and engineering. While Markov Chain Monte Carlo (MCMC) has been the go-to method for sampling from posterior probability measures, it is computationally infeasible for large-scale problems. Variational Bayes (VB) has emerged as a more computationally tractable alternative, approximating posterior distributions with simpler trial distributions. This work presents a flexible and efficient approach to solving inverse problems using VB methods.
COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING
(2022)
Article
Computer Science, Interdisciplinary Applications
Connor Duffin, Edward Cripps, Thomas Stemler, Mark Girolami
Summary: Statistical learning methods combined with physically derived mathematical models are gaining attention, particularly the use of Bayesian statistical methodology to incorporate data and address model misspecification. The statFEM approach embeds stochastic forcing within the governing equations and updates the posterior distribution using classical Bayesian filtering techniques. This article introduces a low-rank approximation of the dense covariance matrix to overcome computational scalability challenges and demonstrates its effectiveness in reconstructing sparsely observed data-generating processes of reaction-diffusion problems.
JOURNAL OF COMPUTATIONAL PHYSICS
(2022)
Article
Multidisciplinary Sciences
Thomas Gaskin, Grigorios A. Pavliotis, Mark Girolam
Summary: Computational models are important tools to understand complex systems, but parameter estimation can be challenging. In this study, the authors propose a simple and fast method using neural differential equations to accurately estimate probability densities for model parameters. The method combines multiagent models as forward solvers with a neural network to extract parameters from generated data, enabling quick estimation for large systems.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
(2023)
Article
Computer Science, Artificial Intelligence
Eky Febrianto, Liam Butler, Mark Girolami, Fehmi Cirak
Summary: This paper demonstrates the application of the statistical finite element method in developing a digital twin of a self-sensing structure. Using captured strain data, the digital twin can predict the true response of a steel railway bridge and generate reasonable predictions at locations without measurement data.
DATA-CENTRIC ENGINEERING
(2022)
Article
Computer Science, Artificial Intelligence
Pranay Seshadri, Andrew B. Duncan, George Thorne, Geoffrey Parks, Raul Vazquez Diaz, Mark Girolami
Summary: This paper presents a Bayesian methodology for interpolating temperature or pressure profiles within an aeroengine. The methodology utilizes spatial Gaussian random fields and Fourier basis for modeling, as well as a novel planar covariance kernel for information transfer between measurement planes. The paper also introduces a sparsity-promoting prior for sparse representations and proposes a Bayesian area average metric for better representation and uncertainty breakdown.
DATA-CENTRIC ENGINEERING
(2022)
Article
Computer Science, Artificial Intelligence
Rebecca Ward, Ruchi Choudhary, Alastair Gregory, Melanie Jans-Singh, Mark Girolami
Summary: The paper introduces a particle filter methodology for continuous calibration of a physics-based model element in a digital twin, applied to an underground farm example. The proposed methodology compares favorably in terms of determination of parameter values distribution and analysis run times, potentially ensuring continuing model fidelity.
DATA-CENTRIC ENGINEERING
(2021)
Article
Computer Science, Interdisciplinary Applications
Steven A. Niederer, Michael S. Sacks, Mark Girolami, Karen Willcox
Summary: Mathematical modeling and simulation are transitioning from development and analysis tools to operational monitoring, control, and decision support, utilizing digital twins that are currently difficult to scale. Challenges and opportunities for scaling digital twins are discussed, along with potential barriers to wider adoption of this technology.
NATURE COMPUTATIONAL SCIENCE
(2021)
Article
Environmental Sciences
Kathrin Menberg, Asal Bidarmaghz, Alastair Gregory, Ruchi Choudhary, Mark Girolami
SCIENCE OF THE TOTAL ENVIRONMENT
(2020)