Article
Clinical Neurology
Sinem Balta Beylergil, Angela M. Noecker, Mikkel Petersen, Palak Gupta, Sarah Ozinga, Mark F. Walker, Camilla Kilbane, Cameron C. McIntyre, Aasef G. Shaikh
Summary: The study revealed that patients with Parkinson's disease showed lower accuracy in vestibular perception compared to healthy controls, but significant improvement was observed after deep brain stimulation treatment. However, there were no significant differences in visual heading perception between patients and controls.
JOURNAL OF NEUROLOGY
(2022)
Article
Behavioral Sciences
Shir Shalom-Sperber, Aihua Chen, Adam Zaidel
Summary: This study investigated cross-sensory (visual-vestibular) adaptation of self-motion perception. The results showed that even several short-duration stimuli can lead to functional adaptation of perception, suggesting that the brain monitors and adapts to supra-modal statistics of events in the environment.
Review
Biology
Zhao Zeng, Ce Zhang, Yong Gu
Summary: Multi-sensory decision making (MSDM) is crucial for making accurate decisions in complex environments. Recent research in computational theory, psychophysical behavior, and neurophysiology has made significant progress in understanding MSDM. By studying a visuo-vestibular heading model system, researchers have uncovered the complex temporal dynamics of vestibular signals in various brain regions, challenging the brain's ability to integrate cues across time and sensory modalities. Moreover, new evidence from higher-level decision-related areas has revised our understanding of how signals from different sensory modalities are processed and accumulated to form a unified perceptual decision.
PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY B-BIOLOGICAL SCIENCES
(2023)
Article
Neurosciences
Sinan Yumurtaci, Oliver W. Layton
Summary: Human heading perception is accurate for directions close to the straight-ahead, but systematic biases emerge in the periphery. A weak-to-moderate overrepresentation of peripheral headings in the MSTd model demonstrated high accuracy and precision. Physiological tuning characteristics play a crucial role in shaping the accuracy and precision of neural heading signals.
Article
Computer Science, Artificial Intelligence
Xitao Zou, Song Wu, Erwin M. Bakker, Xinzhi Wang
Summary: In this paper, a novel multi-label enhancement based self-supervised deep cross-modal hashing approach is proposed to capture semantic affinity more accurately and avoid noise in modalities, achieving state-of-the-art performance in cross-modal hashing retrieval applications.
Review
Biology
Steven J. Jerjian, Devin R. Harsch, Christopher R. Fetsch
Summary: In order to navigate and guide adaptive behaviour in a dynamic environment, animals need to accurately estimate their own motion relative to the external world. This process involves the integration of visual, vestibular and kinesthetic inputs. Recent research has shown that time and certainty are crucial for self-motion perception and decision-making in navigation. By extending current models, researchers have been able to study confidence in heading discrimination and explore the connection between self-motion perception and navigation. Overall, this study offers promise for a deeper understanding of spatial perception and decision-making in behaving animals.
PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY B-BIOLOGICAL SCIENCES
(2023)
Article
Neurosciences
Wei Gao, Yipeng Lin, Jiangrong Shen, Jianing Han, Xiaoxiao Song, Yukun Lu, Huijia Zhan, Qianbing Li, Haoting Ge, Zheng Lin, Wenlei Shi, Jan Drugowitsch, Huajin Tang, Xiaodong Chen
Summary: Gaze changes can disrupt spatial reference frames encoding visual and vestibular signals in the brain, impacting heading discrimination. Through altering eye and head positions, the study examined the effects of gaze changes on heading discrimination using visual, vestibular, and combined stimuli. The results revealed that gaze changes led to biases in perceived heading, increased discrimination thresholds, and reaction time across all stimulus conditions. The effects differed for visual and vestibular stimuli, with visual gaze effects biasing heading perception in the opposite direction of gaze and vestibular gaze effects biasing heading perception in the same direction of gaze. Integration of visual and vestibular signals deviated from predictions of an optimal diffusion model. These findings highlight the diverse effects of gaze changes on heading discrimination and emphasize the importance of spatial reference frame transformation.
Article
Psychology, Experimental
Qi Sun, Ruifang Yan, Jingyi Wang, Xinyu Li
Summary: Recent studies have found a central tendency in the perception of physical features, but this does not apply to heading perception. However, perceived headings are generally biased towards the left side.
Article
Multidisciplinary Sciences
Klaus Gramann, Friederike U. Hohlefeld, Lukas Gehrke, Marius Klug
Summary: The study employed a Mobile Brain/Body Imaging approach to investigate the role of the human RSC in heading computation during physical movement, revealing synchronization in wide frequency bands in RSC, parietal, and occipital cortices. Results suggest the human RSC is involved in heading computation based on visual, vestibular, and proprioceptive input, challenging traditional findings of alpha band desynchronization in navigation network areas during spatial orientation in movement-restricted participants.
SCIENTIFIC REPORTS
(2021)
Article
Biology
Wei Gao, Jiangrong Shen, Yipeng Lin, Kejun Wang, Zheng Lin, Huajin Tang, Xiaodong Chen
Summary: To navigate in space, it is crucial to predict headings based on neural responses in the brain to vestibular and visual signals. The ventral intraparietal area (VIP) is a key brain area for this purpose, but its representation of heading perception on a population level is still unknown. This study successfully decoded headings from VIP population responses using a sequential sparse autoencoder (SSAE) model, achieving high accuracy and demonstrating robustness and efficiency for real-time prediction.
COMPUTERS IN BIOLOGY AND MEDICINE
(2023)
Article
Computer Science, Information Systems
Qinglin He, Xiaobing Zhang, Jianxin Hu, Zehua Sheng, Qi Li, Si-Yuan Cao, Hui-Liang Shen
Summary: Tobacco leaf grading is crucial for ensuring the quality of tobacco production. Traditional manual grading is limited by the high visual similarity among reflection images and the inconsistent visual appearances caused by different planting locations. Therefore, we propose CMENet, an end-to-end Cross-modal Enhancement Network, which integrates multimodal information to achieve automatic grading and achieves high accuracy.
Article
Computer Science, Information Systems
Chengyuan Zhang, Jiayu Song, Xiaofeng Zhu, Lei Zhu, Shichao Zhang
Summary: The Hybrid Cross-Modal Similarity Learning model (HCMSL) proposed in this article effectively addresses the similarity measurement issue in cross-modal retrieval by capturing semantic information and establishing a common subspace between different modalities. Comprehensive experiments demonstrate significant improvements over existing techniques on real datasets.
ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
(2021)
Article
Computer Science, Information Systems
Wenxin Tao, Xiaohong Su, Jiayuan Wan, Hongwei Wei, Weining Zheng
Summary: This paper proposes a new multimodal deep learning based vulnerability detection method that achieves improved performance through cross-modal feature enhancement and fusion. It uses a compilation and debugging method to establish alignment relationships between source code statements and assembly instructions, as well as between source code variables and assembly code registers. By generating bimodal program slices using a cross-slicing method based on alignment relationships and program slicing technology, the method captures fine-grained semantic correlation between source code and assembly code with a cross-modal feature enhanced code representation learning model utilizing co-attention mechanisms. Vulnerability detection is then achieved through feature level fusion of semantic features captured in fine-grained aligned source code and assembly code.
COMPUTERS & SECURITY
(2023)
Article
Cell Biology
Qihao Zheng, Luxin Zhou, Yong Gu
Summary: The study investigates the integration of optic flow and vestibular cues in precise heading perception, finding that adjusting visual stimuli to lead vestibular cues can improve heading performance. This alignment is associated with nonlinear gain modulation effects, facilitating cue integration in certain brain areas.
Article
Computer Science, Information Systems
Qiubin Lin, Wenming Cao, Zhiquan He, Zhihai He
Summary: The rapid development of deep learning has led to significant progress in cross-modal retrieval and the recent attention towards cross-modal hashing. The existing semantic heterogeneity gap between different modalities presents a challenging problem. To address this, we propose the MDCH approach, which introduces semantic mask information and alternately trains intra-modal and inter-modal networks to improve hash code effectiveness.
IEEE TRANSACTIONS ON MULTIMEDIA
(2021)
