Article
Computer Science, Artificial Intelligence
Liangjian Wen, Haoli Bai, Lirong He, Yiji Zhou, Mingyuan Zhou, Zenglin Xu
Summary: This study proposes a gradient estimation method for information measures based on score estimation, aiming to address the issues of high variance and high bias in deep learning. By directly estimating the gradients of information measures with respect to model parameters, stable and efficient stochastic backpropagation is achieved, leading to higher accuracy and lower variance in gradient estimation of information measures.
KNOWLEDGE-BASED SYSTEMS
(2021)
Article
Physics, Multidisciplinary
Peter Grassberger
Summary: In this paper, a new class of estimators for severely undersampled discrete distributions is presented. These estimators are based on a generalization of a previous estimator proposed by T. Schurmann and myself. Interestingly, for a specific set of parameters, they are unbiased and have finite variance, challenging the widely believed notion of impossibility. Detailed numerical tests are conducted to compare these estimators with recent ones and exact results, revealing a conflict with Bayesian estimators for mutual information.
Article
Physics, Multidisciplinary
Andrew Rolph
Summary: In this article, we study the entanglement structure and dynamics in CFTs and black holes, employing a local entanglement measure known as the entanglement contour. We calculate the entanglement contour for different systems, including 1+1-dimensional condensed matter systems and simplified models of black hole evaporation. Our findings reveal universal results for the entanglement contours in low energy non-equilibrium states of 2D CFTs and illustrate the presence of an island phase transition in the entanglement contour of a non-gravitational bath coupled to an extremal AdS(2) black hole.
Article
Physics, Multidisciplinary
Barbara Bedowska-Sojka, Agata Kliber
Summary: This study compares the mutual information shared by different liquidity and volatility estimators within each group, finding that volatility measures are more coherent while liquidity measures are more dispersed.
PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS
(2021)
Article
Physics, Multidisciplinary
Yuan Zhai, Bo Yang, Zhengjun Xi
Summary: This paper investigates the characteristics of the Belavkin-Staszewski relative entropy in relation to the noncommutativity of quantum states. By introducing new conditional entropy and mutual information terms and replacing the quantum relative entropy, the basic properties of these terms are analyzed, focusing on classical-quantum settings. The paper establishes the weak concavity of the Belavkin-Staszewski conditional entropy and the chain rule for the Belavkin-Staszewski mutual information. Additionally, the subadditivity of the Belavkin-Staszewski relative entropy is proven, along with a certain subadditivity of the geometric Renyi relative entropy.
Article
Computer Science, Information Systems
Doron Cohen, Aryeh Kontorovich, Aaron Koolyk, Geoffrey Wolfer
Summary: We seek an entropy estimator for discrete distributions with fully empirical accuracy bounds. Under the assumption of a certain information moment, we provide the first finite-sample entropy estimates for infinite alphabets, nearly recovering the known minimax rates. Our empirical bounds are significantly sharper than the state-of-the-art bounds, for various natural distributions and non-trivial sample regimes. We also give a dimension-free analogue of the Cover-Thomas result on entropy continuity for finite alphabets and resolve all of the open problems posed by Jurgensen and Matthews, 2010.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2023)
Article
Computer Science, Interdisciplinary Applications
Stephane Robin, Luca Scrucca
Summary: Entropy is a crucial measure in information theory, and this paper proposes a semi-parametric estimation method based on a mixture model for calculating entropy from data samples. The accuracy and versatility of the method are demonstrated using simulation studies and real-life examples.
COMPUTATIONAL STATISTICS & DATA ANALYSIS
(2023)
Article
Multidisciplinary Sciences
Qian Zeng, Jin Wang
Summary: This study focuses on refining the concept of Maxwell's demon to explore the limit of energy dissipation in open systems, uncovering a previously unexplored set of fluctuation theorems. These theorems reveal the existence of an intrinsic nonequilibrium state in the system, guided by nonnegative demon-induced dissipative information. The analysis suggests that the bounds of both work and heat in the system are tighter than previously predicted, and proposes a potential experimental test to verify these boundaries.
Article
Geography
Hexiang Bai, Hui Wang, Deyu Li, Yong Ge
Summary: Spatial stratified heterogeneity, a common form of spatial heterogeneity, is observed in geographical phenomena. This study proposes two measures for nominal and continuous target variables based on mutual information and relative entropy. Permutation tests are used to determine the statistical significance.
ANNALS OF THE AMERICAN ASSOCIATION OF GEOGRAPHERS
(2023)
Article
Physics, Multidisciplinary
Jacob A. Barandes, David Kagan
Summary: We introduce a new form of quantum conditional probability to establish new measures of quantum information in a dynamic setting. We investigate the interconnections between our novel measures and conventional measures like von Neumann entropy. These measures serve as a basis for new proofs of established results in quantum information theory.
Article
Computer Science, Artificial Intelligence
Yuval Shalev, Amichai Painsky, Irad Ben-Gal
Summary: Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. In this study, a practical solution is proposed to improve the accuracy of entropy estimation using deep neural networks' cross-entropy estimation abilities. A family of estimators for other information-theoretic measures is also introduced.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Physics, Multidisciplinary
Hiroki Murakami, Norimasa Yamada
Summary: Fitts studied information capacity and transfer in the speed-accuracy motor paradigm using a theoretical approach based on information theory. By estimating information entropy and mutual information of multiple human movement trajectories, Fitts found that information was processed from the first half of the trajectory in difficult tasks.
Article
Biochemical Research Methods
Antoine Passemiers, Yves Moreau, Daniele Raimondi
Summary: This article presents a novel method called PORTIA for inferring gene regulatory networks (GRNs). The method is based on robust precision matrix estimation and is shown to outperform state-of-the-art methods in terms of speed while still maintaining good accuracy. The authors extensively validated PORTIA using benchmark datasets and propose a new scoring metric based on graph-theoretical concepts.
Article
Chemistry, Analytical
Huipeng Li, Bo Xu, Fengxing Zhou, Baokang Yan, Fengqi Zhou
Summary: In this paper, an adaptive empirical variational mode decomposition (EVMD) method based on a binary tree model is proposed to decompose non-stationary signals with complex components. This method effectively solves the problem of VMD parameter selection and reduces computational complexity using an intelligent optimization algorithm. The experimental results demonstrate that the proposed EVMD algorithm can decompose non-stationary signals adaptively with lower complexity, good decomposition effect, and strong robustness.
Article
Computer Science, Interdisciplinary Applications
Ruediger Mutz
Summary: Diversity is a central concept in ecology, social sciences, and bibliometrics. This study proposes a probability-based diversity indicator that reconceptualizes the components of diversity as entropy masses, addressing an inconsistency issue in existing indicators. The overall diversity of research projects in terms of entropy is estimated, with journal articles being the most balanced output type across research areas.
