Article
Physics, Multidisciplinary
Danyang Jia, Hao Guo, Zhao Song, Lei Shi, Xinyang Deng, Matjaz Perc, Zhen Wang
Summary: Reinforcement learning is an alternative to imitation and exploration in resolving social dilemmas, where individuals adjust strategies based on their own past performance and preset aspirations. Stimuli play a crucial role in determining whether a strategy should be retained.
NEW JOURNAL OF PHYSICS
(2021)
Article
Engineering, Marine
Xiangwei Wei, Hao Wang, Yixuan Tang
Summary: This paper proposes a novel multi-USV formation path planning algorithm based on deep reinforcement learning. The algorithm utilizes goal-based hierarchical reinforcement learning and an improved artificial potential field algorithm to address training speed and planning conflicts, achieving optimal path planning and obstacle avoidance through a formation geometry model and a composite reward function.
Article
Nutrition & Dietetics
Alexia Duriez, Clemence Bergerot, Jackson J. Cone, Mitchell F. Roitman, Boris Gutkin
Summary: Seeking and consuming nutrients is crucial for survival and animals learn complex behavioral strategies to obtain them. Recent research suggests that these strategies result from reinforcement learning processes, with phasic dopamine signals playing a key role. This study explores the links between homeostatic and reinforcement learning processes in the context of sodium appetite.
Article
Mathematics, Applied
Danyang Jia, Tong Li, Yang Zhao, Xiaoqin Zhang, Zhen Wang
Summary: This paper investigates individual behavior patterns in public goods games using expectation-based reinforcement learning methods. The study finds that conditional cooperation shows opposite trends in networks with empty nodes, while an appropriate population density can facilitate the maintenance and development of cooperation.
APPLIED MATHEMATICS AND COMPUTATION
(2022)
Article
Mathematics, Applied
Yini Geng, Yifan Liu, Yikang Lu, Chen Shen, Lei Shi
Summary: Recent studies show that the Q-learning algorithm has an impact on the evolutionary outcomes in square lattice. The mixed strategy update rule, which combines Q-learning and the Fermi function, can facilitate cooperation.
APPLIED MATHEMATICS AND COMPUTATION
(2022)
Article
Computer Science, Artificial Intelligence
Liwei Huang, Mingsheng Fu, Hong Qu, Siying Wang, Shangqian Hu
Summary: The paper discusses the important issue of multiagent defense and attack in the field of multi-agent cooperation, based on deep reinforcement learning algorithms, especially Multi-agent DDPG (MADDPG). By reconstructing and redefining the environment, experimental results show that learning with MADDPG can improve the decision-making abilities of agents better than other DRL models.
EXPERT SYSTEMS WITH APPLICATIONS
(2021)
Article
Engineering, Civil
Zhiyu Huang, Haochen Liu, Jingda Wu, Chen Lv
Summary: Making safe and human-like decisions in autonomous driving systems is crucial, and this study proposes a predictive behavior planning framework that learns from human driving data. The framework includes a behavior generation module, a conditional motion prediction network, and a scoring module using inverse reinforcement learning. Comprehensive experiments on real-world urban driving dataset validate the effectiveness of this framework in predicting future trajectories and selecting human-like driving plans.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Rafael Pina, Varuna De Silva, Joosep Hook, Ahmet Kondoz
Summary: The study introduces the concept of residual Q-networks (RQNs) for multiagent reinforcement learning (MARL), which leads to improved efficiency and stability, demonstrating more robust performance in various environments.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Energy & Fuels
Rendong Shen, Shengyuan Zhong, Xin Wen, Qingsong An, Ruifan Zheng, Yang Li, Jun Zhao
Summary: Under the backdrop of high global building energy consumption, utilizing renewable energy to meet the increasing demand of building energy systems can promote clean energy transformation and carbon neutrality. However, the complexity of BES control is increased with the introduction of renewable energy, and addressing the mismatch between supply and demand sides is challenging. The proposed multi-agent deep reinforcement learning framework optimized energy management in buildings, improving device control efficiency and renewable energy utilization in BES.
Article
Multidisciplinary Sciences
Eric S. Dickson, Sanford C. Gordon, Gregory A. Huber
Summary: The extent to which individuals perceive legitimacy affects their intrinsic motivations to comply with authority. An experimental approach is proposed to separate the effects of an authority's costly actions on citizen behavior through both intrinsic and extrinsic channels. The findings provide credible evidence that an authority's actions can directly shape citizens' behavior by enhancing their legitimacy.
Article
Anesthesiology
Angelos-Miltiadis Krypotos, Geert Crombez, Maryna Alves, Nathalie Claes, Johan W. S. Vlaeyen
Summary: This study investigates how individuals solve the exploration-exploitation dilemma when facing pain and finds that participants tend to choose the safest option, prioritize rewards over pain, and are more inclined to explore after experiencing pain.
Article
Multidisciplinary Sciences
Gautam Reddy
Summary: Recent experiments with mice navigating a labyrinth have shown a sharp discontinuity in learning, contradicting the gradual nature of reinforcement learning. By combining biologically plausible reinforcement learning rules with persistent exploration, discontinuous learning is shown to be a common occurrence.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
(2022)
Article
Automation & Control Systems
Pengjie Xu, Yuanzhe Cui, Yichao Shen, Wei Zhu, Yiheng Zhang, Bingzheng Wang, Qirong Tang
Summary: This study proposes a coordinated control method based on reinforcement learning to address the challenges of strong constraints and close coupling in tightly cooperative tasks involving multiple mobile manipulators. The reinforcement learning strategy is tailored to handle unknown vibrations between the manipulators and the common object. By converting the problem into a Markov decision process, optimizing the grasping forces of the end-effectors, and employing an advantage actor-critic algorithm, the system states and learning framework are described. Simulations and experiments employing two mobile manipulators demonstrate the superior control effects of the proposed method compared to well-known controllers. Overall, this study combines the strengths of reinforcement learning and model-based methods through a coordinated controller designed for tight cooperation.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2023)
Article
Mathematics, Interdisciplinary Applications
Zhen-Wei Ding, Guo-Zhong Zheng, Chao-Ran Cai, Wei-Ran Cai, Li Chen, Ji-Qiang Zhang, Xu-Ming Wang
Summary: Cooperation is essential in ecosystems and human society, and reinforcement learning plays a crucial role in understanding its emergence. This study focuses on the individual level dynamics of cooperation in a two-agent system. It is found that strong memory and long-sighted expectation lead to the emergence of Coordinated Optimal Policies (COPs) which maintain high cooperation levels. However, when memory weakens and expectation decreases, cooperation becomes unstable, and the policy of defection prevails. The study also suggests that tolerance can be a precursor to a crisis in cooperation. The findings provide insights into the stability of cooperation and have implications for more complex scenarios.
CHAOS SOLITONS & FRACTALS
(2023)
Article
Ecology
Xin Wang, Shuo Liu, Yifan Yu, Shengzhi Yue, Ying Liu, Fumin Zhang, Yuanshan Lin
Summary: This study proposes a new method of modeling collective motion for fish schooling via multi-agent reinforcement learning. Each fish individual is modeled as an artificial learning agent using mean field Q-learning. The experimental results show that the learned policy can produce collective motion in groups of various sizes and three different collective motion patterns observed in nature emerged during the training process.
ECOLOGICAL MODELLING
(2023)
Article
Neurosciences
Marinho A. Lopes, Jiaxiang Zhang, Dominik Krzeminski, Khalid Hamandi, Qi Chen, Lorenzo Livi, Naoki Masuda
Summary: Brain network dynamics are crucial for brain function and dysfunction, and a new framework using recurrence analysis has been proposed to assess the dynamics of brain networks. The recurrence of dFNs may serve as a biomarker for epilepsy, and recurrence analysis can also help detect seizures and inform neurostimulation strategies. This framework can be applied not only to epilepsy but also to understand dFNs in healthy brain function and other neurological disorders.
EUROPEAN JOURNAL OF NEUROSCIENCE
(2021)
Article
Neurosciences
Takahiro Ezaki, Yu Himeno, Takamitsu Watanabe, Naoki Masuda
Summary: Recent studies have suggested that summarizing brain activity into hidden states dynamics is a promising tool for understanding brain function. Hidden Markov models (HMMs) are commonly used to infer neural dynamics, but the impact of assuming Markovian structure in neural time series data needs further examination. Comparisons between GMM and HMM show that HMM generally provides better accuracy in estimating hidden states, but GMM can be a viable alternative under certain conditions like low sampling frequency or short data length.
EUROPEAN JOURNAL OF NEUROSCIENCE
(2021)
Article
Multidisciplinary Sciences
Sadamori Kojaku, Giacomo Livan, Naoki Masuda
Summary: The intense competition in the academic publishing market drives journal editors to pursue higher impact factors, but the fixation on impact factors can lead to some journals artificially boosting their impact factors. The rise of citation cartel behavior is becoming more common, and the CIDRE algorithm is able to detect these anomalies effectively.
SCIENTIFIC REPORTS
(2021)
Article
Psychology, Multidisciplinary
Yutaka Horita
Summary: The research shows that individuals with high levels of paranoia are more likely to perceive harmful intent in others, potentially leading to increased aggressive behavior. Paranoia is also related to aggressiveness, as well as assumptions about others' self-interests and competitiveness.
FRONTIERS IN PSYCHOLOGY
(2021)
Article
Biology
Lingqi Meng, Naoki Masuda
Summary: Metapopulation models are a useful tool for studying epidemic dynamics. In this study, the effects of different mobility rules on epidemic dynamics in a metapopulation model were compared. It was found that using a second-order random-walk mobility rule called node2vec tends to suppress epidemic spreading, while simple random walks have less impact. The change in epidemic threshold induced by node2vec mobility is generally smaller than the change induced by the diffusion rate.
JOURNAL OF THEORETICAL BIOLOGY
(2022)
Article
Multidisciplinary Sciences
Penghang Liu, Naoki Masuda, Tomomi Kito, Ahmet Erdem Sariyuce
Summary: This study examines the relationship between patent oppositions and collaborations by constructing a two-layer temporal network. The findings suggest that large companies are more likely to engage in patent oppositions with multiple companies, and adversarial companies are more likely to collaborate in the future.
SCIENTIFIC REPORTS
(2022)
Article
Multidisciplinary Sciences
Prosenjit Kundu, Neil G. MacLaren, Hiroshi Kori, Naoki Masuda
Summary: In this paper, a degree-based mean-field theory is developed for understanding the coupled tipping dynamics in a network of double-well systems. The study provides evidence for multistage tipping point transitions in such networks.
PROCEEDINGS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES
(2022)
Article
Multidisciplinary Sciences
Yutaka Horita
Summary: This study found that paranoid thinking exists even in the general population. It is characterized by the expectation that others are competitors who aim to maximize differences in payoffs rather than maximizing their own payoffs. The study used a modified Dictator Game and found that individuals with high-level paranoid thinking were more likely to anticipate competitive allocations by their opponents, even when it was costly for the Dictators. However, paranoid thinking was not associated with selecting sure rewards or competitive allocations.
Article
Psychology, Multidisciplinary
Yutaka Horita, Miku Yamazaki
Summary: The social environment has an impact on individuals' psychology and social networks. Generalized trust is associated with an individual's social ties, while perception of relational mobility has no significant association with trust or social network measures.
JAPANESE PSYCHOLOGICAL RESEARCH
(2023)
Article
Neuroimaging
Arthur P. C. Spencer, Jonathan C. W. Brooks, Naoki Masuda, Hollie Byrne, Richard Lee-Kelland, Sally Jary, Marianne Thoresen, Marc Goodfellow, Frances M. Cowan, Ela Chakkarapani
Summary: Therapeutic hypothermia in children treated for neonatal hypoxic-ischaemic encephalopathy reduces severe motor disability but leads to motor deficits and altered white matter connectivity in those without cerebral palsy. Diffusion-weighted imaging showed significant correlations between white matter tracts and motor performance in cases but not controls. Network analysis revealed associations between impaired motor function and brain organization in cases, highlighting the impact of therapeutic hypothermia on brain development.
NEUROIMAGE-CLINICAL
(2021)
Article
Physics, Fluids & Plasmas
Hang-Hyun Jo, Naoki Masuda
Summary: The study shows that in lattice networks, the convergence time increases as the system size increases, while in regular random graphs, uncorrelated scale-free networks, and complete graphs, the convergence time is almost independent of the system size unless individuals perfectly copy their neighbors' opinions in each update. A mean-field analysis of the complete graph case is also provided.
Article
Physics, Multidisciplinary
Kashin Sugishita, Mason A. Porter, Mariano Beguerisse-Diaz, Naoki Masuda
Summary: Opinion dynamics on tie-decay networks involve interaction patterns that change over time, with tie strength increasing instantaneously and decaying exponentially between interactions. Numerical computations for continuous-time Laplacian dynamics reveal the spectral gaps of combinatorial Laplacian matrices of tie-decay networks. The study compares spectral gaps of empirical tie-decay networks with randomized and aggregate networks, showing smaller gaps in empirical networks, and explores the relationship between spectral gap, tie-decay rate, and time. The results emphasize the importance of the interplay between edge strengthening and decaying in temporal networks.
PHYSICAL REVIEW RESEARCH
(2021)
Article
Neuroimaging
Arthur P. C. Spencer, Jonathan C. W. Brooks, Naoki Masuda, Hollie Byrne, Richard Lee-Kelland, Sally Jary, Marianne Thoresen, James Tonks, Marc Goodfellow, Frances M. Cowan, Ela Chakkarapani
Summary: Therapeutic hypothermia for neonatal encephalopathy following birth asphyxia reduces death and cerebral palsy risks, but school-age children treated with this method still show cognitive and motor deficits compared to controls. Diffusion-weighted imaging revealed widespread changes in white matter structure in these children. Network-based statistic analysis identified brain regions associated with visuo-spatial processing and attention as being affected.
NEUROIMAGE-CLINICAL
(2021)
Review
Multidisciplinary Sciences
Naoki Masuda, Joel C. Miller, Petter Holme
Summary: Diseases spread through temporal networks of interaction events, with the structure of these networks being crucial for understanding epidemic propagation. Concurrency, the concept of individuals forming time-overlapping 'partnerships', plays a key role in this understanding, particularly in sexual transmitted infections. Despite conflicting evaluations and various definitions, concurrency remains significant in epidemiology research.
JOURNAL OF THE ROYAL SOCIETY INTERFACE
(2021)
Editorial Material
Economics
Teruyoshi Kobayashi, Naoki Masuda
JAPANESE ECONOMIC REVIEW
(2021)