Article
Computer Science, Artificial Intelligence
Haitao Xu, Lech Szymanski, Brendan McCane
Summary: Exploration in environments with continuous control and sparse rewards is a challenging task in reinforcement learning. VASE, a surprise-based exploration method, introduces intrinsic rewards to encourage more systematic and efficient exploration. Experimental results demonstrate that VASE outperforms other surprise-based exploration techniques in such environments.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Computer Science, Interdisciplinary Applications
Cristiano Villa, Stephen G. Walker
Summary: This paper presents a new perspective on the use of improper priors in Bayes factors for model comparison. It introduces an alternative approach that establishes the value of the constant in the Bayes factor by matching divergences between density functions. This method, unlike existing ones, does not require any input from the experimenter and is fully automated.
COMPUTATIONAL STATISTICS & DATA ANALYSIS
(2022)
Article
Biochemical Research Methods
Sam Gijsen, Miro Grundei, Robert T. Lange, Dirk Ostwald, Felix Blankenburg
Summary: Tracking statistical regularities of the environment using Bayesian principles is crucial for shaping human behavior and perception. This study investigates the cortical dynamics of somatosensory learning and reveals neural signatures of surprise in the brain during perception. Early surprise signals indicate the need for model updates, providing insights into how somatosensory processing contributes to the learning of environmental statistics.
PLOS COMPUTATIONAL BIOLOGY
(2021)
Article
Engineering, Multidisciplinary
G. F. Bomarito, P. E. Leser, N. C. M. Strauss, K. M. Garbrecht, J. D. Hochhalter
Summary: Interpretability and uncertainty quantification in machine learning are important for decision justification and improving understanding of model behavior. Symbolic regression is a form of machine learning that provides interpretability, but less research has focused on its use on noisy data and quantifying uncertainty. A new Bayesian framework for symbolic regression is introduced, which improves interpretability and robustness to noise while also quantifying parameter uncertainty and producing probabilistic predictions.
COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING
(2023)
Article
Neurosciences
Francoise Lecaignard, Olivier Bertrand, Anne Caclin, Jeremie Mattout
Summary: Predictive coding theory has a profound influence on brain functions and poses a fundamental paradox. Brain responses reflect precision-weighted prediction error, and it is necessary to differentiate the contributions of precision and prediction error in electrophysiology. By combining EEG and MEG, our study reveals adaptive learning of surprise in the brain and precision encoding through specific connections, which has important implications for applications in psychiatry.
JOURNAL OF NEUROSCIENCE
(2022)
Article
Computer Science, Artificial Intelligence
Di Wang, Hongbin Deng
Summary: In a multi-robot system, coordinating path planning effectively is crucial for interference avoidance, resource allocation, and information sharing. This paper proposes a multi-robot cooperative algorithm based on deep reinforcement learning to increase the autonomy of robots in target selection and movement. Through perception and understanding of the environment, the proposed algorithm allows robots to reach target positions without collision and move to any desired location.
EXPERT SYSTEMS WITH APPLICATIONS
(2021)
Article
Computer Science, Artificial Intelligence
Kleanthis Malialis, Christos G. Panayiotou, Marios M. Polycarpou
Summary: The study introduces a novel algorithm named AREBA, which performs significantly better than other algorithms in handling class imbalance and nonstationary data, with faster learning speed and higher learning quality. Experimental results demonstrate the superiority of AREBA in various scenarios.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2021)
Article
Automation & Control Systems
Min Wen, Ufuk Topcu
Summary: This article presents a method for using reinforcement learning (RL) algorithms to learn how to execute tasks in uncertain and adversarial environments. The interaction between an RL agent and its potentially adversarial environment is modeled as a turn-based zero-sum stochastic game. The task requirements are represented using linear temporal logic (LTL) specifications and a reward function. It is shown that a memoryless almost-sure winning strategy exists for each realizable LTL specification that can be transformed into a deterministic Buchi automaton, and a probably approximately correct (PAC) learning algorithm is proposed to efficiently learn such a strategy in an online manner with unknown reward functions and transition distributions.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2022)
Article
Multidisciplinary Sciences
Sebastian Hoehna, Michael J. Landis, John P. Huelsenbeck
Summary: The study presents a general parallelization strategy that can significantly reduce the runtime of marginal likelihood estimation, even with only two CPUs showing an average performance increase of nearly 2x. The performance increase is nearly linear with the number of available CPUs, with a substantial reduction in runtime for cluster nodes with 16 CPUs.
Article
Computer Science, Artificial Intelligence
Cuong Nguyen, Thanh-Toan Do, Gustavo Carneiro
Summary: We introduce a new PAC-Bayes meta-learning algorithm that solves few-shot learning by extending the framework to handle multiple tasks and samples. Our generative-based approach estimates task-specific model parameters more expressively, resulting in well-calibrated and accurate models. These models perform competitively on few-shot classification and regression benchmarks.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Chemistry, Applied
Gabriela de Freitas Laiber Pascoal, Marta Angela de Almeida Sousa Cruz, Joel Pimentel de Abreu, Millena Cristina Barros Santos, Gustavo Bernardes Fanaro, Mario Roberto Marostica Junior, Otniel Freitas Silva, Ricardo Felipe Alves Moreira, Luiz Claudio Cameron, Mariana Simoes Larraz Ferreira, Anderson Junger Teodoro
Summary: Bioactive compounds were extracted from hybrid Vitis vinifera L. varieties Sweet sapphire (SP) and Sweet surprise (SU) using two different extraction solvents. The antioxidant potential and anthocyanin content of SP acetone extract from grape skin were found to be the highest among the samples tested. Various analytical techniques were used for characterization and quantification of the compounds.
Article
Computer Science, Artificial Intelligence
Tomer Panker, Nir Nissim
Summary: This paper presents the first trusted framework for detecting unknown malware in Linux VM cloud-environments using machine-learning algorithms and informative traces from volatile memory. The framework was rigorously evaluated in experiments and showed high accuracy in detecting unknown malware and categorizing them by attack category.
KNOWLEDGE-BASED SYSTEMS
(2021)
Article
Computer Science, Artificial Intelligence
J. Christmas
Summary: The online variational Bayesian model introduced can perform truly online processing on an infinitely long set of observations, ensuring posterior probability distributions do not become overly tight and keeping the algorithm's time complexity constant. Demonstrations on synthetic and real datasets have shown the effectiveness of the model.
PATTERN RECOGNITION
(2022)
Article
Computer Science, Software Engineering
Jinhan Kim, Robert Feldt, Shin Yoo
Summary: The rapid adoption of Deep Learning (DL) systems in safety critical domains necessitates the testing of their correctness and robustness. In this article, we propose Surprise Adequacy (SA) as a test adequacy criterion, which measures the difference between the behavior of a DL system for a given input and its behavior for training data. We demonstrate that SA can predict model behavior correctness and detect adversarial examples.
ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY
(2023)
Article
Physics, Multidisciplinary
Joseph Colantonio, Igor Bascandziev, Maria Theobald, Garvin Brod, Elizabeth Bonawitz
Summary: Bayesian models can provide insights into how surprise is related to belief revision in children, particularly in terms of physiological responses like pupil dilation. The likelihood of an observed event given prior beliefs (Shannon Information) and the dissimilarity between prior and updated beliefs following observations (Kullback-Leibler divergence) are two computational measures used to understand surprise. Our study shows that pupillometric responses in children are correlated with Kullback-Leibler divergence only when they actively make predictions, suggesting that pupillary responses may indicate the degree of divergence between a child's current beliefs and updated beliefs.
Article
Computer Science, Artificial Intelligence
Samuel P. Muscinelli, Wulfram Gerstner, Johanni Brea
NEURAL COMPUTATION
(2017)
Review
Neurosciences
Wulfram Gerstner, Marco Lehmann, Vasiliki Liakoni, Dane Corneil, Johanni Brea
FRONTIERS IN NEURAL CIRCUITS
(2018)
Article
Neurosciences
Alireza Modirshanechi, Mohammad Mahdi Kiani, Hamid Aghajan
Article
Computer Science, Artificial Intelligence
Bernd Illing, Wulfram Gerstner, Johanni Brea
Article
Multidisciplinary Sciences
Piero Amodio, Johanni Brea, Benjamin G. Farrar, Ljerka Ostojic, Nicola S. Clayton
Summary: The study explored prospective caching behaviors in corvids and found that neither the Compensatory Caching Hypothesis nor the Future Planning Hypothesis were supported by the experimental results. Instead, the best explanation was a uniform distribution of food caches across different caching locations.
SCIENTIFIC REPORTS
(2021)
Article
Neurosciences
Vahid Esmaeili, Keita Tamura, Samuel P. Muscinelli, Alireza Modirshanechi, Marta Boscaglia, Ashley B. Lee, Anastasiia Oryshchuk, Georgios Foustoukos, Yanqi Liu, Sylvain Crochet, Wulfram Gerstner, Carl C. H. Petersen
Summary: This study used a combination of calcium imaging, electrophysiology, and optogenetics to track the cortical activity sequence in mice from whisker stimulation to delayed licking. It found that enhanced activity in the secondary whisker motor cortex and transient reduction in orofacial sensorimotor cortex played important roles in completing the task, while sustained activity in the frontal cortex was crucial for licking during the response period.
Article
Biochemical Research Methods
He A. Xu, Alireza Modirshanechi, Marco P. Lehmann, Wulfram Gerstner, Michael H. Herzog
Summary: This research demonstrates the importance of incorporating surprise and novelty into reinforcement learning theories to explain human behavior. It is found that human decisions are mainly influenced by model-free action choices, with the world-model playing a role in detecting surprising events.
PLOS COMPUTATIONAL BIOLOGY
(2021)
Article
Neurosciences
Vasiliki Liakoni, Marco P. Lehmann, Alireza Modirshanechi, Johanni Brea, Antoine Lutti, Wulfram Gerstner, Kerstin Preuschoff
Summary: Humans have a remarkable capability of learning how to reach a reward over long series of actions, potentially guided by multiple parallel learning modules. This study introduces a complex decision-making task and tests various learning algorithms to identify model-free and surprise-based learning signals in the brain.
Article
Mathematics, Interdisciplinary Applications
Alireza Modirshanechi, Johanni Brea, Wulfram Gerstner
Summary: Surprising events have a measurable impact on brain activity and human behavior, affecting learning, memory, and decision-making. The definition of surprise lacks consensus. In this study, the authors identify and classify 18 mathematical definitions of surprise, proposing a taxonomy based on the measured quantity. This research lays the foundation for studying the functional roles and physiological signatures of surprise in the brain.
JOURNAL OF MATHEMATICAL PSYCHOLOGY
(2022)
Article
Multidisciplinary Sciences
Johanni Brea, Nicola S. Clayton, Wulfram Gerstner
Summary: In this study, the authors present computational models of neural networks to explain how the 'what', 'where', and 'when' of past experiences are stored and retrieved in episodic memories for suitable decisions, using food-caching birds as examples. They propose a model and neural implementation of food-caching behavior, suggesting that associative reinforcement learning without mental time-travel is sufficient to explain the results of behavioral experiments with food-caching birds.
NATURE COMMUNICATIONS
(2023)
Article
Neurosciences
Alireza Modirshanechi, Sophia Becker, Johanni Brea, Wulfram Gerstner
Summary: This article systematically investigates the relationship between different aspects of surprise and novelty with brain functions and physiological signals, addressing concerns about the existence of same functionalities and mechanisms in the brain. It emphasizes the significance of computational modeling and quantifiable definitions in providing novel interpretations of previous findings.
CURRENT OPINION IN NEUROBIOLOGY
(2023)
Article
Computer Science, Interdisciplinary Applications
Jamie Fairbrother, Christopher Nemeth, Maxime Rischard, Johanni Brea, Thomas Pinder
Summary: This paper introduces the GaussianProcesses.jl package developed for the Julia programming language, utilizing the computational benefits of Julia for fast, flexible and user-friendly Gaussian process modeling. The package provides various mean and kernel functions, inference tools, alternative likelihood functions for non-Gaussian data, and sparse approximations.
JOURNAL OF STATISTICAL SOFTWARE
(2022)