Review
Neurosciences
Yujun Deng, Da Song, Junjun Ni, Hong Qing, Zhenzhen Quan
Summary: Learning is a complex process where our opinions and decisions can be easily influenced by unexpected information. The neural mechanism underlying revision and correction during learning is still unclear.
FRONTIERS IN NEUROSCIENCE
(2023)
Article
Computer Science, Artificial Intelligence
Guofeng Zhu, Fei Zhu
Summary: The article proposes a policy framework called PGMP, which utilizes multi-step prediction to guide exploration in reinforcement learning. The framework includes a curiosity mechanism and a safety bonus model to encourage exploration in safe and task-relevant areas. Additionally, a looking-ahead model is introduced to predict future states, actions, and rewards, allowing the agent to optimize its policy for predicted future states.
EXPERT SYSTEMS WITH APPLICATIONS
(2024)
Article
Psychology, Clinical
Paola Fuentes-Claramonte, Maria Angeles Garcia-Leon, Pilar Salgado-Pineda, Nuria Ramiro, Joan Soler-Vidal, Maria Llanos Torres, Ramon Cano, Isabel Argila-Plaza, Francesco Panicali, Carmen Sarri, Nuria Jaurrieta, Manel Sanchez, Ester Boix-Quintana, Auria Albacete, Teresa Maristany, Salvador Sarro, Joaquim Radua, Peter. J. McKenna, Raymond Salvador, Edith Pomarol-Clotet
Summary: The negative symptoms of schizophrenia may be due to reduced responsiveness to rewarding stimuli, which is associated with abnormal dopamine function in the disorder. However, few imaging studies have examined whether patients with negative symptoms show reduced activation related to reward prediction error (RPE). The findings suggest that negative symptoms are not caused by a generalized reduction in RPE signaling, but rather by specific dysfunction in the lateral frontal and possibly the orbitofrontal cortex.
PSYCHOLOGICAL MEDICINE
(2023)
Article
Neurosciences
Huw Jarvis, Isabelle Stevenson, Amy Q. Huynh, Emily Babbage, James Coxon, Trevor T. -J. Chong
Summary: Recent research suggests that the act of investing effort may influence learning. This study tested whether effort modulates teaching signals in a reinforcement learning paradigm. The results showed that effort resulted in more efficient learning from positive outcomes and less efficient learning from negative outcomes. Interestingly, this effect varied across individuals and was more pronounced in those who were more averse to investing effort in the first place. These findings highlight the importance of motivational factors in a common framework of reward-based learning, integrating the computational principles of reinforcement learning with those of value-based decision-making.
JOURNAL OF NEUROSCIENCE
(2022)
Article
Psychology, Clinical
Wei Lei, Kezhi Liu, Guangxiang Chen, Serenella Tolomeo, Cuizhen Liu, Zhenlei Peng, Boya Liu, Xuemei Liang, Chaohua Huang, Bo Xiang, Jia Zhou, Fulin Zhao, Rongjun Yu, Jing Chen
Summary: This study found that patients with Internet gaming disorder (IGD) have impaired reinforcement learning and blunted reward prediction error (RPE) signals in the brain reward system, as well as hyperconnectivity between regions of the reward system. These results suggest that reinforcement learning deficits may be crucial characteristics of IGD pathophysiology.
PSYCHOLOGICAL MEDICINE
(2022)
Article
Neurosciences
Naveen Sendhilnathan, Michael E. Goldberg, Anna E. Ipata
Summary: Recent studies have shown that the cerebellum is not only involved in motor control but also plays a role in reward processing. In an experiment with monkeys, researchers found that the discharge patterns of cerebellar cortex changed during the learning of associations between movements and visual symbols. Despite being related to both reward and movement, these signals were independent of each other.
JOURNAL OF NEUROSCIENCE
(2022)
Article
Clinical Neurology
Shinsuke Suzuki, Yuichi Yamashita, Kentaro Katahira
Summary: This study found that a trans-diagnostic dimension of psychiatric symptoms related to compulsive behavior and intrusive thought was negatively correlated with overall decision-making performance in both reward-seeking and loss-avoidance tasks. Further analysis revealed that this psychiatric dimension was associated with lower preference for options that recently led to better outcomes in both tasks. These findings suggest that psychiatric symptoms influence decision-making processes for seeking rewards and avoiding losses through both common and distinct computational processes.
PSYCHIATRY AND CLINICAL NEUROSCIENCES
(2021)
Article
Neurosciences
Jonathan Nicholas, Christian Amlang, Chi-Ying R. Lin, Leila Montaser-Kouhsari, Natasha Desai, Ming-Kai Pan, Sheng-Han Kuo, Daphna Shohamy
Summary: Recent studies have shown that individuals with cerebellar ataxia are impaired in learning reward associations through trial-and-error feedback, while retaining the ability to predict reward based on episodic memory. This suggests a specific and necessary role for the cerebellum in incremental learning of reward associations based on reinforcement, in addition to its role in motor learning.
Article
Chemistry, Multidisciplinary
Xiaofeng Lu, Ao Xue, Pietro Lio, Pan Hui
Summary: In this paper, a method combining dynamic influence map and deep reinforcement learning is proposed to address the issue of sparse reward in intelligent decision making. The experimental results demonstrate the effectiveness of the proposed method in improving scores and training speed, as well as reducing video memory and overall memory consumption.
APPLIED SCIENCES-BASEL
(2022)
Article
Neurosciences
Shuyuan Xu, Wei Ren
Summary: This study used electroencephalogram to investigate the neural correlates of state prediction errors (SPEs) in goal-directed reinforcement learning. The results suggest that the parietal correlate is responsible for explicit learning of state transition structure, while the frontal and central correlates may be involved in cognitive control.
Article
Biology
Goni Naamani, Nitzan Shahar, Yoav Ger, Yossi Yovel
Summary: This study examined the bats' ability to adjust their decision strategy in different environments and found that their decisions were influenced by natural priors.
Article
Computer Science, Artificial Intelligence
Xiaoshu Zhou, Fei Zhu, Peiyao Zhao
Summary: The method of prediction based on uncertainty exploration (SPE) improves the quality of exploration and reduces noise interference in deep reinforcement learning, leading to significant improvements in simulated environments.
EXPERT SYSTEMS WITH APPLICATIONS
(2022)
Article
Physics, Multidisciplinary
Ruihai Chen, Hao Li, Guanwei Yan, Haojie Peng, Qian Zhang
Summary: This paper proposes a hierarchical reinforcement learning framework for air combat training to address the non-convergence problem caused by the curse of dimensionality in the state space during air combat tactical pursuit. By using hierarchical reinforcement learning, three-dimensional problems are transformed into two-dimensional problems, resulting in improved training performance compared to other baselines. To further enhance overall learning performance, a meta-learning-based algorithm is established with a corresponding reward function designed to improve the agent's performance in the air combat tactical chase scenario. The results demonstrate that the proposed framework achieves better performance than the baseline approach.
Article
Psychology, Clinical
Qiang Shen, Shiguang Fu, Xiaoying Jiang, Xiaoyu Huang, Doudou Lin, Qingyan Xiao, Sitti Khadijah, Yaping Yan, Xiaoxing Xiong, Jia Jin, Richard P. Ebstein, Ting Xu, Yiquan Wang, Jun Feng
Summary: This study investigates the differences in learning behavior between adolescent depressive patients and healthy controls using an instrumental learning task. The results show that depressive patients perform worse, have slower learning rates, and exhibit pessimistic biases and counterfactual outcome biases. These biases are also linked with the severity of depressive symptoms.
PSYCHOLOGICAL MEDICINE
(2023)
Article
Psychology, Multidisciplinary
Heinrich Peters, Sandra C. Matz, Moran Cerf
Summary: This paper explores a novel approach to improving human decision-making through sensory substitution. The results from a within-subject design study show that translating numerical information into sensory experiences leads to higher decision accuracy. The benefits of sensory substitution are attributed to a shift from explicit rule abstraction to intuitive configural learning.
COMPUTERS IN HUMAN BEHAVIOR
(2023)
Article
Neurosciences
James W. Antony, Thomas H. Hartshorne, Ken Pomeroy, Todd M. Gureckis, Uri Hasson, Samuel D. McDougle, Kenneth A. Norman
Summary: Surprise signals a discrepancy between past and current beliefs, affecting how ongoing experiences are segmented into discrete perceived events. These effects are influenced by whether surprising moments contradict or bolster current predominant beliefs.
Article
Surgery
Matthew J. Crossley, Christopher L. Hewitson, John Cartmill, David M. Kaplan
ANZ JOURNAL OF SURGERY
(2021)
Article
Behavioral Sciences
Milena Rmus, Samuel D. McDougle, Anne G. E. Collins
Summary: Reinforcement learning models have advanced understanding of animal learning, decision making, and brain support, but fall short in explaining many sophisticated aspects of human learning. Executive functions play an important role in instrumental learning, providing inputs that enhance the flexibility and applicability of canonical RL computations in the brain.
CURRENT OPINION IN BEHAVIORAL SCIENCES
(2021)
Article
Neurosciences
Samuel D. McDougle, Ian C. Ballard, Beth Baribault, Sonia J. Bishop, Anne G. E. Collins
Summary: This study found that novel rewards can act as substitutes for rewards during instrumental learning, generating reward-like signals in dopaminergic circuits. Furthermore, prefrontal correlates of executive control may play a role in shaping flexible responses in reward circuits. The interaction between high-level representations in prefrontal cortex and low-level responses in subcortical reward circuits support learning from novel outcomes.
Article
Multidisciplinary Sciences
Christopher L. Hewitson, Sinan T. Shukur, John Cartmill, Matthew J. Crossley, David M. Kaplan
Summary: The study shows that camera realignment imposes a reliable cost on laparoscopic task performance for both naive controls and experienced surgeons. This finding could influence the strategies used by surgeons in the operating room.
SCIENTIFIC REPORTS
(2021)
Article
Neurosciences
Samuel D. McDougle, Sarah A. Wilterson, Nicholas B. Turk-Browne, Jordan A. Taylor
Summary: Classic taxonomies of memory distinguish between explicit and implicit memory systems, but this study suggests that the declarative memory processes in the medial temporal lobe play a significant role in motor learning, which requires a revision of the classic model.
JOURNAL OF COGNITIVE NEUROSCIENCE
(2022)
Article
Neurosciences
Faisal Mushtaq, Samuel D. McDougle, Matt P. Craddock, Darius E. Parvin, Jack Brookes, Alexandre Schaefer, Mark Mon-Williams, Jordan A. Taylor, Richard B. Ivry
Summary: This study examined the brain's response to different types of errors by observing feedback-related EEG activity. The results revealed that selection errors elicited a larger feedback-related negativity (FRN) and were correlated with behavioral adjustment, while execution errors produced a different pattern of FRN and error positivity, which were not correlated with subsequent changes in behavior.
JOURNAL OF COGNITIVE NEUROSCIENCE
(2022)
Article
Neurosciences
Samuel D. McDougle
Summary: Post-error slowing (PES) is a phenomenon in human decision-making studies, characterized by an increase in response time for a decision following an error. PES reflects the deployment of executive resources to restore task performance. This study found that PES is influenced by multiple distinct processes, with a significant impact from working memory recruitment.
Review
Neurosciences
Juan A. Gallego, Tamar R. Makin, Samuel D. McDougle
Summary: This study argues that brain-computer interfaces (BCIs) for movement restoration need to better decode the user's intent. The researchers point out that the neural activity in the primary motor cortex provides both too little and too much information, which may affect the accuracy of decoding. To address this issue, the researchers suggest integrating additional information from multiple brain regions to enhance the interpretability of BCIs.
TRENDS IN NEUROSCIENCES
(2022)
Article
Multidisciplinary Sciences
Olivia A. Kim, Alexander D. Forrence, Samuel D. McDougle
Summary: Prediction errors play a crucial role in guiding learning. Recent research suggests that the brain can generate sensory predictions based on motor planning alone, challenging the traditional assumption that movement execution is required for error computation. This study shows that the brain can compute errors that drive implicit adaptation without generating overt movements, leading to the adaptation of motor commands that are not overtly produced.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
(2022)
Article
Biology
Guy Avraham, Jordan A. Taylor, Assaf Breska, Richard B. Ivry, Samuel D. McDougle
Summary: This study examines the relevance of associative learning concepts for sensorimotor adaptation and provides evidence for the existence of associative learning effects in sensorimotor adaptation tasks. The findings suggest a potential integration of different cerebellar-dependent learning paradigms within a common theoretical framework.
Review
Psychology, Clinical
Ashleigh Rutherford, Samuel D. McDougle, Jutta Joormann
Summary: Anhedonia and rumination are key features of depression, yet they have often been studied separately. In this paper, we argue that by examining the relationship between cognitive constructs and deficits in positive affect, we can better understand anhedonia in depression and improve prevention and intervention efforts.
CLINICAL PSYCHOLOGY REVIEW
(2023)
Article
Neurosciences
Christopher L. Hewitson, Naser Al-Fawakhiri, Alexander D. Forrence, Samuel D. McDougle
Summary: People form metacognitive representations of their own abilities across tasks, but the influence of errors during learning on these representations is poorly understood. Our study used computational modeling to show that people's confidence judgments during motor learning are best explained by a recency-weighted averaging of visually observed errors. These confidence judgments are adaptive and sensitive to the volatility of the learning environment, integrating recent motor errors differently based on the environment's stability. Additionally, confidence tracked motor errors in both implicit and explicit motor learning, but only influenced behavior in explicit learning.
JOURNAL OF NEUROPHYSIOLOGY
(2023)
Article
Psychology, Mathematical
Lisa Langsdorf, Jana Maresch, Mathias Hegele, Samuel D. McDougle, Raphael Schween
Summary: Research has shown that incomplete learning in visuomotor adaptation tasks may primarily be a result of a speed-accuracy tradeoff due to time-consuming motor planning, leading to under-compensation.
PSYCHONOMIC BULLETIN & REVIEW
(2021)