Article
Computer Science, Artificial Intelligence
Rana Ali Amjad, Kairen Liu, Bernhard C. Geiger
Summary: In this study, three information-theoretic quantities were used to analyze the behavior of trained neural networks, revealing that class selectivity is not a reliable indicator for classification performance. However, when examining individual layers, mutual information and class selectivity show a positive correlation with classification performance.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Computer Science, Artificial Intelligence
Taojiannan Yang, Sijie Zhu, Matias Mendieta, Pu Wang, Ravikumar Balakrishnan, Minwoo Lee, Tao Han, Mubarak Shah, Chen Chen
Summary: Existing deep neural networks are limited in their ability to perform inference at different resource constraints. In this work, the authors propose the MutualNet method, which trains a single network to run at various resource constraints. By training a cohort of model configurations with different widths and resolutions, MutualNet achieves consistent improvements on various tasks and datasets.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Tianyu Ma, Alan Q. Wang, Adrian V. Dalca, Mert R. Sabuncu
Summary: The convolutional neural network (CNN) is a commonly used architecture for computer vision tasks. A new building block called hyper-convolution is presented in this paper, which encodes the convolutional kernel using spatial coordinates and enables a more flexible architecture design. Experimental results showed that replacing regular convolutions with hyper-convolutions improved performance with fewer parameters and increased robustness against noise.
MEDICAL IMAGE ANALYSIS
(2023)
Article
Chemistry, Physical
Rafael B. Araujo, Ilknur Bayrak Pehlivan, Tomas Edvinsson
Summary: In this study, a computational approach using density functional theory and deep neural networks is introduced to predict the performance of high-entropy alloys (HEAs) in the electrochemical reduction of nitrogen. The approach quantifies the catalytic activity of HEA catalysts by optimizing the elements and concentration to increase the probability of specific atomic arrangements on the surfaces. The approach provides a quick and effective way to filter HEA candidates without the need for time-consuming calculations.
Article
Computer Science, Artificial Intelligence
Fan Feng, Qi Liu, Zhanglin Peng, Ruimao Zhang, Rosa H. M. Chan
Summary: The layer-wise structure of DNNs hinders efficient learning by isolating channel interactions. Existing methods focus on learning channel interdependence but suppress channel diversity. Our work proposes CC-Net with a community-based graph topology for optimal channel interaction. It outperforms baselines on image classification tasks with lower computational costs.
PATTERN RECOGNITION
(2023)
Article
Automation & Control Systems
Tania B. Lopez-Garcia, Jose A. Dominguez-Navarro
Summary: This study proposes a typed graph neural network-based approach to solve the power flow problem. The proposed solver achieves unsupervised training by training on benchmark power grid cases and generating voltage magnitude and phase values that minimize the violation of physical laws governing the system. The solver has linear time complexity and can generalize to grids with different conditions.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2023)
Article
Computer Science, Information Systems
Quan Li, Xinhua Xu, Jinjun Liu, Guangmin Li
Summary: A recent study shows that recommendation systems not only rely on static user preferences, but also dynamic preferences. This has led to the emergence of session-based recommendation systems. A hybrid neural model called SGPD is proposed in this study to learn sequential general patterns and dependencies for session-based recommendation. The experimental results demonstrate that SGPD significantly improves precision rate, recall rate, and mean reciprocal ranking compared to state-of-the-art methods.
Article
Quantum Science & Technology
Junyu Liu, Francesco Tacchino, Jennifer R. Glick, Liang Jiang, Antonio Mezzacapo
Summary: In this paper, the design and performance prediction of variational quantum circuits for learning and optimization tasks are discussed using the theory of neural tangent kernels. Quantum neural tangent kernels are defined, and dynamical equations for their loss function in optimization tasks are derived. Analytical solutions in the frozen limit and dynamical settings are explored, showing that a hybrid quantum classical neural network has approximate Gaussian behavior.
Article
Physics, Multidisciplinary
Lorenzo Squadrani, Nico Curti, Enrico Giampieri, Daniel Remondini, Brian Blais, Gastone Castellani
Summary: In this study, an improved BCM model is proposed by combining the classical framework with modern deep learning methods. Through numerical simulations, the model's efficiency in learning, memorization capacity, and feature extraction is demonstrated. The model also shows selectivity and interpretability.
Article
Computer Science, Information Systems
Saeed Reza Kheradpisheh, Maryam Mirsadeghi, Timothee Masquelier
Summary: We propose a new learning algorithm that trains spiking neural networks using conventional artificial neural networks as proxies. By aligning the outputs of the two networks and backpropagating the error, we can effectively train deep neural networks and achieve high classification accuracy on benchmark datasets.
Article
Computer Science, Artificial Intelligence
Enzo Tartaglione, Stephane Lathuiliere, Attilio Fiandrotti, Marco Cagnazzo, Marco Grangetto
Summary: The study formulates the entropy of a quantized artificial neural network as a differentiable function that can be plugged into the cost function minimized by gradient descent. By training the network to minimize the entropy of quantized parameters, optimal compression can be achieved. This approach shows significant benefits in terms of storage size compressibility without compromising the model's performance.
Article
Computer Science, Artificial Intelligence
Pablo Morala, Jenny Alexandra Cifuentes, Rosa E. Lillo, Inaki Ucar
Summary: This article discusses a mathematical framework relating neural networks and polynomial regression, and proposes a new method for their relationship. Through experimental validation, it is shown that this method can produce polynomials that correctly approximate data when learning from polynomial generated data.
Article
Computer Science, Artificial Intelligence
Xiaoyu Xu, Xiaoyu Shi, Mingsheng Shang
Summary: This research introduces a novel approach for effectively learning node representations in disassortative graph structures. The proposed method synthesizes the feature semantic space and the structure semantic space to find friendly neighbor spaces and learns the interrelationship between aggregated information and separated information using contrastive learning. Experimental results show that the proposed approach outperforms eight state-of-the-art GNN models in various graph mining tasks, demonstrating its superior graph representation ability.
EXPERT SYSTEMS WITH APPLICATIONS
(2023)
Article
Computer Science, Information Systems
Danyang Xiao, Yuan Mei, Di Kuang, Mengqiang Chen, Binbin Guo, Weigang Wu
Summary: Distributed deep learning faces challenges in communication overhead, and a proposed entropy-based gradient compression mechanism aims to reduce this overhead. Experimental results and analysis show that algorithms based on EGC can achieve high compression ratios while maintaining accuracy.
INFORMATION SCIENCES
(2021)
Article
Computer Science, Artificial Intelligence
Wei Zeng, Ge Fan, Shan Sun, Biao Geng, Weiyi Wang, Jiacheng Li, Weibo Liu
Summary: The deep neural network has been successfully applied to the collaborative filtering problem, capturing side information of users and items and modeling interactions between them. Research trends towards utilizing neural networks with mixed structures to learn better representations, achieving high accuracy with minimal additional computation.
APPLIED SOFT COMPUTING
(2021)
Article
Chemistry, Multidisciplinary
Sung Hyun Kim, Hyunwoo Kim, Hawoong Jeong, Tae-Young Yoon
Summary: The study demonstrates the use of single-molecule FRET technology to encode virtual signals in DNA barcodes, allowing for precise measurement of FRET efficiency for each binding event and differentiation of six DNA barcodes.
Article
Multidisciplinary Sciences
Hyewon Kim, Hang-Hyun Jo, Hawoong Jeong
Summary: The impact of environmental changes on the dynamics of temporal networks is important, with interaction patterns varying in different environments. By studying a temporal network model and considering environmental changes, it is possible to successfully reproduce empirical results regarding multiscale temporal correlations.
Article
Multidisciplinary Sciences
Seungwoong Ha, Hawoong Jeong
Summary: The study introduces AgentNet, a model-free data-driven framework utilizing deep neural networks to reveal and analyze the hidden interactions in complex systems. AgentNet successfully captured a wide variety of simulated complex systems and demonstrated potential applications with real bird flock data.
SCIENTIFIC REPORTS
(2021)
Article
Physics, Multidisciplinary
Jae Sung Lee, Sangyun Lee, Hyukjoon Kwon, Hyunggyu Park
Summary: Landauer's bound is the minimum thermodynamic cost for erasing one bit of information. Finite-time operation incurs additional energetic costs, with different scaling behavior depending on the degree of irreversibility of the process. Optimal dynamics can lead to the equality of the bound.
PHYSICAL REVIEW LETTERS
(2022)
Article
Physics, Multidisciplinary
Sangyun Lee, Dong-Kyum Kim, Jong-Min Park, Won Kyu Kim, Hyunggyu Park, Jae Sung Lee
Summary: Entropy production is a key quantity in thermodynamics, but measuring it has remained challenging. However, a newly introduced estimator called multidimensional entropic bound (MEB) utilizing an ensemble of trajectories can accurately estimate the entropy production of overdamped Langevin systems with any time-dependent protocol. The MEB also provides a unified platform to estimate entropy production of underdamped Langevin systems under certain conditions and has the advantage of computational efficiency. Numerical simulations confirm the validity and efficiency of this method by applying it to three physical systems driven by time-dependent protocols in optical tweezers experiments: a dragged Brownian particle, the pulling process of a harmonic chain, and the unfolding process of an RNA hairpin.
PHYSICAL REVIEW RESEARCH
(2023)
Article
Physics, Multidisciplinary
Youngkyoung Bae, Dong-Kyum Kim, Hawoong Jeong
Summary: This paper presents a method to estimate and visualize the dissipation pattern in living organisms at mesoscopic scales through analyzing recorded videos. The estimator accurately measures the stochastic entropy production (EP) and provides a locally heterogeneous dissipation map. This method can contribute to understanding complex nonequilibrium phenomena and their dissipation mechanisms.
PHYSICAL REVIEW RESEARCH
(2022)
Article
Physics, Multidisciplinary
Dong-Kyum Kim, Sangyun Lee, Hawoong Jeong
Summary: In this study, a machine-learning method is developed to estimate the entropy production in a stochastic system with odd-parity variables using multiple neural networks. This method enables the measurement of entropy production solely based on trajectory data and parity information.
PHYSICAL REVIEW RESEARCH
(2022)
Article
Physics, Multidisciplinary
Mi Jin Lee, Eun Lee, Byunghwee Lee, Hawoong Jeong, Deok-Sun Lee, Sang Hoon Lee
Summary: This study focuses on the framework for discovering hidden dependent relationships in weighted networks by selecting essential interactions for individual nodes based on information entropy. The analysis reveals that nations in the world trade network exhibit more asymmetric dependent relations compared to their random counterparts, while relationships among individuals in the historical record of Korea are more mutual.
PHYSICAL REVIEW RESEARCH
(2021)
Article
Physics, Multidisciplinary
Seungwoong Ha, Hawoong Jeong
Summary: ConservNet is a neural network that learns hidden invariants from grouped data with an intuitive loss function, making it robust to various noise and data conditions, and directly applicable to experimental data for discovering hidden conservation laws and general relationships between variables.
PHYSICAL REVIEW RESEARCH
(2021)
Article
Physics, Fluids & Plasmas
Youngkyoung Bae, Sangyun Lee, Juin Kim, Hawoong Jeong
Summary: The research found that the dynamics and energetics of the Brownian gyrator are influenced by mass, with inertia helping to reduce nonequilibrium effects. In the Langevin model, rotation is maximized at a particular anisotropy while stability decreases at a specific anisotropy or mass.
Article
Physics, Fluids & Plasmas
Sangyun Lee, Meesoon Ha, Hawoong Jeong
Summary: By studying the quantum and classical Otto cycles, it was found that quantumness can reduce productivity and precision in the quasistatic limit but increase them in the finite-time mode. Moreover, the precision of the quantum Otto cycle surpasses that of the classical one as the strength between the system and the bath increases. Additionally, both quantum and classical Otto cycles violate the conventional TUR in the region where entropy production is small in the finite-time mode, suggesting the need for a modified TUR to cover such scenarios.
