Article
Computer Science, Artificial Intelligence
Stefan T. Radev, Marco D'Alessandro, Ulf K. Mertens, Andreas Voss, Ullrich Koethe, Paul-Christian Buerkner
Summary: Comparing mathematical models of complex processes is important in various scientific fields. The Bayesian probabilistic framework provides a principled way to compare models and extract useful metrics. However, many interesting models cannot be handled by standard Bayesian methods. Therefore, we propose a novel method using specialized deep learning architectures for Bayesian model comparison.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Multidisciplinary Sciences
Boris Hanin, Alexander Zlokapa
Summary: This article investigates how the depth, width, and dataset size of neural networks jointly affect model quality and presents a complete solution in the case of linear networks. The study reveals the joint role of depth, width, and dataset size through asymptotic expansions of Meijer-G functions. It shows that linear networks make provably optimal predictions at infinite depth.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
(2023)
Article
Automation & Control Systems
Shailesh Garg, Souvik Chakraborty
Summary: VB-DeepONet is a Bayesian operator learning framework that addresses the challenges faced by the deterministic DeepONet architecture. It provides better resistance against overfitting, improved generalization, and allows for the quantification of predictive uncertainty. The results from various mechanics problems demonstrate the effectiveness of VB-DeepONet in uncertainty quantification.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2023)
Article
Mathematics
Francisco Javier Diez, Manuel Arias, Jorge Perez-Martin, Manuel Luque
Summary: OpenMarkov is an open-source software tool designed for probabilistic graphical models, primarily in medicine but also used in other fields and education in over 30 countries. This paper explains how OpenMarkov can be used as a pedagogical tool to teach the main concepts of Bayesian networks and influence diagrams, as well as various inference algorithms.
Article
Computer Science, Artificial Intelligence
Martin Magris, Alexandros Iosifidis
Summary: The last decade has seen a growing interest in Bayesian learning, but its technicality and complexity in practical implementations have limited its widespread adoption. This survey introduces the principles and algorithms of Bayesian Learning for Neural Networks from a practical perspective, discussing standard and recent approaches for Bayesian inference. It also explores the use of manifold optimization as a state-of-the-art approach and provides pseudo-codes for implementation.
ARTIFICIAL INTELLIGENCE REVIEW
(2023)
Article
Automation & Control Systems
Jongchan Baek, Hayoung Jun, Jonghyuk Park, Hakjun Lee, Soohee Han
Summary: The proposed SVDPG algorithm integrates Bayesian pruning with policy update in reinforcement learning to achieve efficient policy network compression and competitive performance in continuous control benchmark tasks. Additionally, SVDPG demonstrates superiority in low-computing power devices and reliability in real-world scenarios.
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS
(2021)
Article
Biology
Jan Boelts, Jan-Matthis Lueckmann, Richard Gao, Jakob H. Macke
Summary: Inferring parameters of computational models is crucial in cognitive neuroscience. Simulation-based inference (SBI) using neural density estimators provides a more efficient way to capture decision-making data. Compared to traditional methods, this approach demonstrates higher accuracy and training efficiency.
Article
Physics, Multidisciplinary
Ali Unlu, Laurence Aitchison
Summary: The study introduces a Variational Laplace method for estimating ELBO without the need for stochastic sampling of neural network weights, outperforming maximum a posteriori inference and standard sampling-based variational inference in test performance and expected calibration errors. Care is necessary when benchmarking standard VI to avoid stopping before variance parameters have converged.
Article
Engineering, Multidisciplinary
Kevin Linka, Amelie Schafer, Xuhui Meng, Zongren Zou, George Em Karniadakis, Ellen Kuhl
Summary: Understanding real-world dynamical phenomena is challenging, and machine learning has become the go-to technology for analyzing and making decisions based on these phenomena. However, traditional neural networks often ignore the fundamental laws of physics and fail to make accurate predictions. In this study, the combination of neural networks, physics informed modeling, and Bayesian inference is used to integrate data, physics, and uncertainties, improving the predictive potential of neural network models.
COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING
(2022)
Article
Physics, Multidisciplinary
Ludwig Winkler, Cesar Ojeda, Manfred Opper
Summary: This paper proposes a method to leverage the Bayesian uncertainty information encoded in parameter distributions to inform the learning procedure for Bayesian models. By deriving a Bayesian stochastic differential equation and applying stochastic optimal control, individually controlled learning rates are obtained for variational parameters. The resulting optimizer shows robustness to large learning rates and can adaptively and individually control the learning rates.
Article
Computer Science, Artificial Intelligence
Haitao Xu, Lech Szymanski, Brendan McCane
Summary: Exploration in environments with continuous control and sparse rewards is a challenging task in reinforcement learning. VASE, a surprise-based exploration method, introduces intrinsic rewards to encourage more systematic and efficient exploration. Experimental results demonstrate that VASE outperforms other surprise-based exploration techniques in such environments.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Engineering, Multidisciplinary
Vahid Keshavarzzadeh, Robert M. Kirby, Akil Narayan
Summary: Inverse problems are common in engineering simulations, and Bayesian inference is a predominant approach to infer unknown parameters. This paper presents a variational inference method that incorporates observation data and the gradient information of the forward map to invert unknown latent parameters. The method utilizes a trained neural network to generate samples for statistical calculations. The effectiveness of the method is demonstrated through examples, and future research directions are discussed.
COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING
(2022)
Article
Computer Science, Artificial Intelligence
Tinghuan Chen, Bin Duan, Qi Sun, Meng Zhang, Guoqing Li, Hao Geng, Qianru Zhang, Bei Yu
Summary: This article proposes a sharing grouped convolution structure and Bayesian sharing framework to reduce parameter redundancy and improve prediction accuracy. Experimental results show that this method can significantly reduce parameters in different grouped convolutional neural networks.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Computer Science, Artificial Intelligence
Raquel Sanchez-Cauce, Iago Paris, Francisco Javier Diez
Summary: A sum-product network is a probabilistic model based on a directed acyclic graph, where terminal nodes represent probability distributions and non-terminal nodes represent convex sums and products of probability distributions. They can be used for building tractable models from data and are applicable to various problem domains.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2022)
Article
Multidisciplinary Sciences
Giulio Isacchini, Aleksandra M. Walczak, Thierry Mora, Armita Nourmohammad
Summary: Subclasses of lymphocytes work together with different functional roles to produce immune response and lasting immunity. The diversity of receptor chains is crucial for T and B cell lymphocytes in recognizing different pathogens.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
(2021)
Article
Psychology, Biological
Mischa von Krause, Stefan T. Radev, Andreas Voss
Summary: Response speeds in simple decision-making tasks decline in early and middle adulthood, but this is not solely due to mental speed differences. Using a Bayesian diffusion model and analysis of a large dataset with over one million participants, researchers find that the slowing is mainly caused by increases in decision caution and non-decisional processes. Slowing of mental speed is only observed after the age of approximately 60.
NATURE HUMAN BEHAVIOUR
(2022)
Article
Psychology, Experimental
Maria Wirth, Andreas Voss, Stefan Wirth, Klaus Rothermund
Summary: Research has shown that the current indicators used to capture affect dynamics have limited value in predicting well-being, indicating a need to identify more valid assessment methods.
COGNITION & EMOTION
(2022)
Article
Physics, Particles & Fields
Teresa Bister, Martin Erdmann, Ullrich Koethe, Josina Schulte
Summary: Inference of physical parameters from measured distributions is a crucial task in physics data analysis. Conditional invertible neural networks are an elegant deep learning method that can preserve probability and evaluate posterior distributions. This study compares the performance of conditional invertible neural networks with the traditional method and finds good agreement between the derived physics parameters.
EUROPEAN PHYSICAL JOURNAL C
(2022)
Article
Psychology
Lukas Schumacher, Andreas Voss
Summary: The human ability to discriminate the duration of stimuli is influenced by the presentation order and the history of previously encountered stimuli. Cognitive models that account for these effects suggest that participants' duration estimation is influenced by an internal reference that evolves throughout the experiment. In this study, a new diffusion model incorporating perceptual discrimination mechanisms accurately predicts performance in a duration discrimination task and is sensitive to different effects.
ATTENTION PERCEPTION & PSYCHOPHYSICS
(2023)
Article
Astronomy & Astrophysics
Jonas Haldemann, Victor Ksoll, Daniel Walter, Yann Alibert, Ralf S. Klessen, Willy Benz, Ullrich Koethe, Lynton Ardizzone, Carsten Rother
Summary: Researchers propose using conditional invertible neural networks to calculate the posterior probability of planetary structure parameters, which can speed up the inference process for characterizing exoplanets. By training the neural network on a large database of internal structure models, they show that cINNs can infer the composition of an exoplanet much faster than the commonly used MCMC method. However, computing a large database is still required for training the network.
ASTRONOMY & ASTROPHYSICS
(2023)
Article
Geriatrics & Gerontology
Maria Wirth, Andreas Voss, Klaus Rothermund
Summary: Emotional aging research focuses on age-related improvements in motivation. Socioemotional selectivity theory proposes that as individuals age, they favor emotion-related goals and savor positive but avoid negative emotions. Our computational approach provides partial support for these predictions.
JOURNALS OF GERONTOLOGY SERIES B-PSYCHOLOGICAL SCIENCES AND SOCIAL SCIENCES
(2023)
Article
Geriatrics & Gerontology
Eva Marie Wieschen, Aalim Makani, Stefan T. Radev, Andreas Voss, Julia Spaniol
Summary: Older adults exhibit longer response times in various cognitive domains. The diffusion model suggests that this is due to a cautious response style and slower non-decisional processes, rather than a difference in information accumulation rate. The Levy flight model extends the diffusion model by accommodating larger jumps in evidence accumulation and shows that older adults have a more gradual pattern of evidence accumulation compared to younger adults.
EXPERIMENTAL AGING RESEARCH
(2023)
Article
Mathematics, Interdisciplinary Applications
Matthias Kloft, Raphael Hartmann, Andreas Voss, Daniel W. W. Heck
Summary: Standard response formats like rating or visual analogue scales require respondents to condense distributions of latent states or behaviors into a single value, neglecting the variance of distributions. To address this, the dual-range slider is used to measure variability. An extension of the beta response model, the Dirichlet dual response model, is proposed and evaluated for parameter recovery.
Article
Multidisciplinary Sciences
Lukas Schumacher, Paul-Christian Buerkner, Andreas Voss, Ullrich Koethe, Stefan T. Radev
Summary: This study proposes a method to add a temporal dimension to cognitive models and estimate their dynamics using a superstatistics perspective. The results show that the deep learning approach is very efficient in capturing the temporal dynamics of the model, and that the erroneous assumption of static or homogeneous parameters will hide important temporal information.
SCIENTIFIC REPORTS
(2023)
Article
Computer Science, Artificial Intelligence
Stefan T. Radev, Marco D'Alessandro, Ulf K. Mertens, Andreas Voss, Ullrich Koethe, Paul-Christian Buerkner
Summary: Comparing mathematical models of complex processes is important in various scientific fields. The Bayesian probabilistic framework provides a principled way to compare models and extract useful metrics. However, many interesting models cannot be handled by standard Bayesian methods. Therefore, we propose a novel method using specialized deep learning architectures for Bayesian model comparison.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Psychology, Experimental
Annika Stump, Jan Rummel, Andreas Voss
Summary: The study found that negative emotions can reverse the truth effect, and individuals with higher need for cognitive closure are more susceptible to the truth effect. Additionally, the truth effect diminishes when the repetition interval is long.
PSYCHOLOGICAL RESEARCH-PSYCHOLOGISCHE FORSCHUNG
(2022)