Article
Neurosciences
Vasiliki Liakoni, Marco P. Lehmann, Alireza Modirshanechi, Johanni Brea, Antoine Lutti, Wulfram Gerstner, Kerstin Preuschoff
Summary: Humans have a remarkable capability of learning how to reach a reward over long series of actions, potentially guided by multiple parallel learning modules. This study introduces a complex decision-making task and tests various learning algorithms to identify model-free and surprise-based learning signals in the brain.
Article
Clinical Neurology
Shinsuke Suzuki, Yuichi Yamashita, Kentaro Katahira
Summary: This study found that a trans-diagnostic dimension of psychiatric symptoms related to compulsive behavior and intrusive thought was negatively correlated with overall decision-making performance in both reward-seeking and loss-avoidance tasks. Further analysis revealed that this psychiatric dimension was associated with lower preference for options that recently led to better outcomes in both tasks. These findings suggest that psychiatric symptoms influence decision-making processes for seeking rewards and avoiding losses through both common and distinct computational processes.
PSYCHIATRY AND CLINICAL NEUROSCIENCES
(2021)
Article
Chemistry, Multidisciplinary
Xiaofeng Lu, Ao Xue, Pietro Lio, Pan Hui
Summary: In this paper, a method combining dynamic influence map and deep reinforcement learning is proposed to address the issue of sparse reward in intelligent decision making. The experimental results demonstrate the effectiveness of the proposed method in improving scores and training speed, as well as reducing video memory and overall memory consumption.
APPLIED SCIENCES-BASEL
(2022)
Article
Engineering, Electrical & Electronic
Longquan Chen, Ying He, Qiang Wang, Weike Pan, Zhong Ming
Summary: This paper introduces a novel autonomous driving scheme that integrates the sensing, decision-making, and motion-controlling modules using reinforcement learning for joint learning and optimization. By leveraging an attention layer and a CNN layer, the proposed scheme achieves better representation of states and implicitly considers the relevance between decision-making and motion-controlling.
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
(2022)
Article
Neurosciences
A. Wiehler, K. Chakroun, J. Peters
Summary: Research suggests that individuals with gambling disorder show specific deficits in exploration strategy, possibly linked to altered processing in a fronto-parietal network and/or changes in dopamine neurotransmission in the midbrain.
JOURNAL OF NEUROSCIENCE
(2021)
Article
Geriatrics & Gerontology
Ping Ren, Guozhi Luo, Jiayin Huang, Meiling Tan, Donghui Wu, Han Rong
Summary: Aging is often accompanied by cognitive decline and altered decision making. This study found that older adults have difficulty in processing reward/risk information, leading to suboptimal decision strategy. It also investigated the neural substrates of risky decision-making under ambiguity in aging and found that older adults are more sensitive to high punishment frequency.
FRONTIERS IN AGING NEUROSCIENCE
(2023)
Article
Biology
Goni Naamani, Nitzan Shahar, Yoav Ger, Yossi Yovel
Summary: This study examined the bats' ability to adjust their decision strategy in different environments and found that their decisions were influenced by natural priors.
Article
Psychology, Clinical
Paola Fuentes-Claramonte, Maria Angeles Garcia-Leon, Pilar Salgado-Pineda, Nuria Ramiro, Joan Soler-Vidal, Maria Llanos Torres, Ramon Cano, Isabel Argila-Plaza, Francesco Panicali, Carmen Sarri, Nuria Jaurrieta, Manel Sanchez, Ester Boix-Quintana, Auria Albacete, Teresa Maristany, Salvador Sarro, Joaquim Radua, Peter. J. McKenna, Raymond Salvador, Edith Pomarol-Clotet
Summary: The negative symptoms of schizophrenia may be due to reduced responsiveness to rewarding stimuli, which is associated with abnormal dopamine function in the disorder. However, few imaging studies have examined whether patients with negative symptoms show reduced activation related to reward prediction error (RPE). The findings suggest that negative symptoms are not caused by a generalized reduction in RPE signaling, but rather by specific dysfunction in the lateral frontal and possibly the orbitofrontal cortex.
PSYCHOLOGICAL MEDICINE
(2023)
Article
Physics, Multidisciplinary
Ruihai Chen, Hao Li, Guanwei Yan, Haojie Peng, Qian Zhang
Summary: This paper proposes a hierarchical reinforcement learning framework for air combat training to address the non-convergence problem caused by the curse of dimensionality in the state space during air combat tactical pursuit. By using hierarchical reinforcement learning, three-dimensional problems are transformed into two-dimensional problems, resulting in improved training performance compared to other baselines. To further enhance overall learning performance, a meta-learning-based algorithm is established with a corresponding reward function designed to improve the agent's performance in the air combat tactical chase scenario. The results demonstrate that the proposed framework achieves better performance than the baseline approach.
Article
Chemistry, Multidisciplinary
Hongpeng Zhang, Yujie Wei, Huan Zhou, Changqiang Huang
Summary: This paper proposes an air combat maneuver decision-making method based on final reward estimation and proximal policy optimization. By constructing an air combat environment, designing reward mechanisms, and improving the training performance and efficiency of reinforcement learning, it provides a solution to maneuver decision-making problems in autonomous air combat.
APPLIED SCIENCES-BASEL
(2022)
Article
Computer Science, Artificial Intelligence
Jianghui Sang, Yongli Wang, Weiping Ding, Zaki Ahmadkhan, Lin Xu
Summary: This paper presents a reward shaping method called HGT, which propagates reward information through hierarchical graph topology to shape potential functions for complex tasks. Compared to cutting-edge RL techniques, HGT achieves faster learning rates in experiments on Atari and Mujoco tasks.
PATTERN RECOGNITION
(2023)
Article
Computer Science, Theory & Methods
Shangdong Yang, Huihui Wang, Shaokang Dong, Xingguo Chen
Summary: In reinforcement learning, agents learn policies from spatiotemporal data generated through interaction with the environment. However, the reward signals in the data are often sparse, making policy learning challenging. Prior knowledge of task structure has been used to address this issue, and in this paper, we consider the Hard-Transiting task structure. We propose two novel algorithms for efficient exploration and test them on various tasks.
FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE
(2023)
Article
Computer Science, Artificial Intelligence
Guofeng Zhu, Fei Zhu
Summary: The article proposes a policy framework called PGMP, which utilizes multi-step prediction to guide exploration in reinforcement learning. The framework includes a curiosity mechanism and a safety bonus model to encourage exploration in safe and task-relevant areas. Additionally, a looking-ahead model is introduced to predict future states, actions, and rewards, allowing the agent to optimize its policy for predicted future states.
EXPERT SYSTEMS WITH APPLICATIONS
(2024)
Article
Psychology, Clinical
Wei Lei, Kezhi Liu, Guangxiang Chen, Serenella Tolomeo, Cuizhen Liu, Zhenlei Peng, Boya Liu, Xuemei Liang, Chaohua Huang, Bo Xiang, Jia Zhou, Fulin Zhao, Rongjun Yu, Jing Chen
Summary: This study found that patients with Internet gaming disorder (IGD) have impaired reinforcement learning and blunted reward prediction error (RPE) signals in the brain reward system, as well as hyperconnectivity between regions of the reward system. These results suggest that reinforcement learning deficits may be crucial characteristics of IGD pathophysiology.
PSYCHOLOGICAL MEDICINE
(2022)
Article
Computer Science, Artificial Intelligence
Jianghui Sang, Yongli Wang
Summary: This study proposes a method called Graph Convolution with Topology Refinement (GTR) for automatic reinforcement learning, which constructs a new latent graph to enhance reward shaping. The most suitable state node is identified using graph entropy, and the original graph is adaptively mapped to a subset of nodes to form a more compact latent graph. GTR utilizes trainable projection vectors for node feature projection, ensuring consistent inter-connections between nodes in the new latent graph.
Article
History & Philosophy Of Science
Russell A. Poldrack
Summary: The concept of representation is widely and uncontroversially used in neuroscience, contrasting its highly controversial status in philosophy and cognitive science. The paper discusses the use of the term in neuroscience, particularly in characterizing representations empirically, and relates it to the field of machine learning, arguing that the success of artificial neural networks in certain tasks reflects inherent inductive biases similar to biological brains.
Editorial Material
Neurosciences
A. Zeynep Enkavi, Russell A. Poldrack
BIOLOGICAL PSYCHIATRY-COGNITIVE NEUROSCIENCE AND NEUROIMAGING
(2021)
Article
Biology
Leor Zmigrod, Ian W. Eisenberg, Patrick G. Bissett, Trevor W. Robbins, Russell A. Poldrack
Summary: This study explores the relationships between ideological attitudes and psychological traits, finding that cognitive dispositions and personality characteristics play significant roles in shaping individuals' ideological worldviews, extremist beliefs, and resistance to evidence.
PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY B-BIOLOGICAL SCIENCES
(2021)
Article
Multidisciplinary Sciences
Patrick G. Bissett, Henry M. Jones, Russell A. Poldrack, Gordon D. Logan
Summary: The stop-signal paradigm, based on race models, shows severe violations of the independence assumption at short stop-signal delays (SSDs) across various conditions. Existing data may need to be reanalyzed, and adjustments to models are necessary to accommodate this finding.
Article
Neurosciences
Elizabeth Beam, Christopher Potts, Russell A. Poldrack, Amit Etkin
Summary: A data-driven framework for understanding human brain circuits was developed using computational methods applied to a large number of neuroimaging articles. This framework outperformed traditional knowledge frameworks in explaining the relationships between brain structure and function. The approach divided the literature into modular subfields, providing insights into key structure-function patterns.
NATURE NEUROSCIENCE
(2021)
Article
Neurosciences
Christian Paret, Nike Unverhau, Franklin Feingold, Russell A. Poldrack, Madita Stirner, Christian Schmahl, Maurizio Sicorello
Summary: The replicability and reproducibility of scientific findings are crucial for progress and quality assurance in neuroscience. Open science practices such as preregistration, data sharing, and adherence to data standards are widely adopted in neuroimaging research. However, there are challenges hindering the full adoption of these practices. A survey showed that while half of the participants had experience with preregistration, their willingness to preregister studies in the future was modest. Most participants had experience with sharing primary neuroimaging data and were interested in implementing standardized data structures. Demographic factors such as career level and geographical location were found to influence researchers' attitudes towards open science practices.
Article
Psychiatry
Matilde M. Vaghi, McKenzie P. Hagen, Henry M. Jones, Jeanette A. Mumford, Patrick G. Bissett, Russell A. Poldrack
Summary: This study examined the relationship between self-regulation changes and transdiagnostic dimensions of psychopathology in the early phase of the COVID-19 pandemic. The results revealed independent relationships between psychopathology dimensions and longitudinal alterations in specific domains of self-regulation. Symptoms related to anxiety and depression were associated with more cautious decision-making, while social withdrawal was associated with faster non-decision processes. Self-reported measures of self-regulation predicted changes in psychiatric symptoms, highlighting the relevance of psychological dimensions in separate transdiagnostic dimensions of psychiatry.
TRANSLATIONAL PSYCHIATRY
(2022)
Article
Neurosciences
Christin Scholz, Hang-Yee Chan, Russell A. Poldrack, Denise T. D. de Ridder, Ale Smidts, Laura Nynke van der Laan
Summary: Self-control is crucial for human wellbeing, and Hare et al.'s study on the neural correlates of self-control has made a significant impact. To support the self-control theory, a replication study was conducted, which found that decision-making is associated with value signals in the vmPFC. Contrary to the hypothesis, the study did not find a significant role for the dlPFC in self-control. Additionally, the results show that the analytical strategy can affect the findings.
HUMAN BRAIN MAPPING
(2022)
Article
Biochemical Research Methods
Rastko Ciric, William H. H. Thompson, Romy Lorenz, Mathias Goncalves, Eilidh E. E. MacNicol, Christopher J. J. Markiewicz, Yaroslav O. O. Halchenko, Satrajit S. S. Ghosh, Krzysztof J. J. Gorgolewski, Russell A. A. Poldrack, Oscar Esteban
Summary: This article introduces TemplateFlow as a publicly available framework for human and non-human brain models. It combines an open database with software to enable scientists to share their resources under FAIR principles, allowing multifaceted brain analyses.
Editorial Material
Computer Science, Interdisciplinary Applications
Cyril Pernet, Claus Svarer, Ross Blair, John D. Van Horn, Russell A. Poldrack
Summary: FAIR data sharing aims to achieve access to research data at any time, but the sustainability of maintaining all datasets for rapid access is questioned. This article addresses the issue of cold data storage: when to archive data for offline storage, how to maintain FAIR principles while archiving, and who should be responsible for cold archiving and long-term preservation.
Article
Psychology, Multidisciplinary
Patrick G. Bissett, Russell A. Poldrack
Summary: This article discusses the cognitive function of response inhibition and the assessment method using the stop-signal paradigm. It also highlights the violations of the independence assumption in existing stop models and introduces promising new models of response inhibition.
CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE
(2022)
Article
Psychology, Clinical
Jeffrey L. Birk, Michael W. Otto, Talea Cornelius, Russell A. Poldrack, Donald Edmondson
Summary: Various fields rely on effective interventions to change human behaviors, but the lack of a systematic approach to identifying and targeting mechanisms of action hinders systematic progress in behavior change. The CheckList for Investigating Mechanisms in Behavior-change Research (CLIMBR) was developed to guide researchers in understanding the active ingredients of successful behavior change. The rationale, development process, and final version of CLIMBR are presented.
Article
Neurosciences
Armin W. Thomas, Christopher Re, Russell A. Poldrack
Summary: Deep learning models are widely used in mental state decoding to accurately identify the mapping between mental states and brain activity. Researchers often use explainable AI methods to understand the learned mappings of the model. A study benchmarks different explanation methods in mental state decoding and provides guidance for neuroimaging researchers on choosing the appropriate explanation method.
Article
Psychology, Clinical
Gina L. Mazza, Heather L. Smyth, Patrick G. Bissett, Jessica R. Canning, Ian W. Eisenberg, A. Zeynep Enkavi, Oscar Gonzalez, Sunny Jung Kim, Stephen A. Metcalf, Felix Muniz, William E. Pelham, Emily A. Scherer, Matthew J. Valente, Haiyi Xie, Russell A. Poldrack, Lisa A. Marsch, David P. MacKinnon
Summary: Self-regulation is studied across various disciplines, and the integration of cross-disciplinary measures of self-regulation reveals substantial variability and challenges the notion that different measures assess the same construct.
JOURNAL OF PERSONALITY ASSESSMENT
(2021)
Article
Psychology, Clinical
Patrick D. Manapat, Michael C. Edwards, David P. MacKinnon, Russell A. Poldrack, Lisa A. Marsch
Summary: The study evaluated the Brief Self-Control Scale using modern psychometric literature and provided a comprehensive item analysis using the item response theory (IRT) framework. Results supported both unidimensional and multidimensional factor structures for the 13-item version of the BSCS, with an additional perspective on item- and test-level functioning from the IRT analysis. The goal of establishing a more defensible psychometric grounding for the BSCS is to promote greater consistency, stability, and trust in future results.