Article
Physics, Fluids & Plasmas
O. V. Usatenko, S. S. Melnyk, G. M. Pritula, V. A. Yampol'skii
Summary: This study proposes two different approaches for introducing the information temperature of binary Nth-order Markov chains. The first approach compares Markov sequences with equilibrium Ising chains at given temperatures, while the second approach utilizes the probabilities of finite-length subsequences to determine their entropies. The results indicate that both approaches yield similar results for the case of nearest-neighbor spin-symbol interaction.
Article
Engineering, Electrical & Electronic
Francois Desbouvries, Yohan Petetin, Achille Salaun
Summary: In this paper, HMC and RNN are considered as generative models and embedded in a common GUM. The expressivity of these models is compared by assuming linearity and Gaussianity, and using structured covariance series to characterize the probability distributions.
IEEE TRANSACTIONS ON SIGNAL PROCESSING
(2023)
Article
Mathematics, Applied
Abdessatar Souissi, El Gheteb Soueidy, Mohamed Rhaima
Summary: This paper investigates quantum Markov chains (QMCs) on graphs and trees in relation to important models in quantum statistical mechanics and quantum information. These QMCs generate significant properties, such as quantum phase transition and clustering properties. The paper proposes a construction of QMCs associated with an XX-Ising model over the comb graph N >0 Z and proves that the QMC associated with the disordered phase enjoys a clustering property.
Article
Computer Science, Information Systems
Yixuan Wang, Liping Yuan, Harish Garg, Ali Bagherinia, Hamid Parvin, Kim-Hung Pho, Zulkefli Mansor
Summary: This paper proposes a new fuzzy clustering ensemble method that introduces Reliability Based weighted co-association matrix Fuzzy C-Means (RBFCM), Reliability Based Graph Partitioning (RBGP) and Reliability Based Hyper Clustering (RBHC) as three new fuzzy clustering consensus functions to improve performance and robustness.
CMC-COMPUTERS MATERIALS & CONTINUA
(2021)
Review
Physics, Multidisciplinary
Jan Mielniczuk
Summary: This paper reviews the information theoretic tools and their application in feature selection, focusing on classification problems with discrete features. The authors discuss various ways of constructing counterparts to conditional mutual information and their properties and limitations. They propose a unified method based on truncation for the Mobius expansion of conditional mutual information. The paper also discusses the main approaches to feature selection using the introduced measures of conditional dependence, along with methods for assessing the quality of the obtained predictors, including recent results on asymptotic distributions of empirical criteria and advances in resampling.
Article
Chemistry, Multidisciplinary
Yinan Chen, Chuanpeng Wang, Dong Li
Summary: Community structure is a prevalent characteristic in social, biological, and technological networks, where nodes can be naturally divided into densely connected groups. Understanding community structure helps in exploring the interactions and associations between elements in the network and uncovering their potential information. However, defining the quality of a community and finding the best partition are challenging due to the complexity of the network.
APPLIED SCIENCES-BASEL
(2022)
Article
Computer Science, Artificial Intelligence
Tom Lefebvre
Summary: In this study, we extend and handle the problem of imitation from observations, specifically in the case of feature-only demonstrations. Our approach combines elements from probability and information theory to develop a behavioral cloning method that extracts an executable policy directly from the given features.
PATTERN RECOGNITION LETTERS
(2022)
Article
Mathematics, Applied
Luigi Accardi, Amenallah Andolsi, Farrukh Mukhamedov, Mohamed Rhaima, Abdessatar Souissi
Summary: The MCL algorithm is a highly efficient approach in detecting clustered structures in networks. It utilizes a stochastic matrix derived from the adjacency matrix of the graph network. Quantum clustering algorithms are proven to be more efficient than classical ones, and in this study, a clustering property for quantum Markov chains on Cayley trees associated with open quantum random walks is demonstrated.
Article
Computer Science, Information Systems
Eyuri Wakakuwa
Summary: This study focuses on quantifying the non-Markovianity of tripartite quantum states using operational viewpoints, introducing an Omega* class of operations performed by three distant parties. It establishes that the non-Markovianity of formation is a faithful measure of non-Markovianity, and provides an operational meaning to a measure of bipartite entanglement called the c-squashed entanglement.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Biology
Alaguraj Veluchamy, Preeti Mehta, K. V. Srividhya, Hirendra Vikram, M. K. Govind, Ramneek Gupta, Abdul Aziz Bin Dukhyil, Raed Abdullah Alharbi, Saleh Abdullah Aloyuni, Mohamed M. Hassan, S. Krishnaswamy
Summary: Shannon's information theoretic perspective sheds light on the storage and processing of information in one-dimensional sequences, with an analysis of prokaryotic and eukaryotic genomes revealing patterns of information content and compositional rules among chromosomes. The study confirms that information content can offer insights into genome clustering and the evolution of messaging strategies.
SAUDI JOURNAL OF BIOLOGICAL SCIENCES
(2021)
Article
Computer Science, Artificial Intelligence
Syed Fawad Hussain, Khadija Khan, Rashad Jillani
Summary: The combination of multi-view clustering with co-clustering has resulted in a new Weighted Multi-View Co-Clustering (WMVCC) algorithm, which optimizes the clustering by leveraging the diversity of features provided by multiple sources of information while exploiting the power of co-clustering.
APPLIED INTELLIGENCE
(2022)
Article
Computer Science, Artificial Intelligence
Shizhe Hu, Ruobin Wang, Yangdong Ye
Summary: This paper proposes an interactive information bottleneck method to address high-dimensional data clustering by simultaneously performing data clustering and low-dimensional feature subspace learning. The method preserves the correlations between the data clusters and the learned dimension-reduced features, iteratively improving the results. Experimental results demonstrate the effectiveness and superiority of the proposed method.
APPLIED SOFT COMPUTING
(2021)
Article
Engineering, Electrical & Electronic
Ya-Jun Leng, Xin Yue, Yi-Qin Lu, Shu-Ping Zhao
Summary: The article proposes a black-start decision-making method based on information-theoretic co-clustering, which can solve the problem of black-start scheme selection with incomplete information. Experimental results demonstrate that the proposed method performs even better than some methods considering complete information.
ELECTRIC POWER COMPONENTS AND SYSTEMS
(2022)
Article
Computer Science, Artificial Intelligence
Tomasz Klonecki, Pawel Teisseyre, Jaesung Lee
Summary: Feature selection is crucial in multi-label classification for building predictive models. Existing methods often disregard cost information associated with considered features. We address the problem of cost-constrained multilabel feature selection, aiming to select a feature subset relevant to multiple labels while adhering to a user-defined budget. Our approach ensures high predictive power without exceeding the specified budget per prediction. We propose a novel criterion combining relevance and cost for feature selection, along with an effective method for determining the trade-off between relevancy and cost. Experimental results demonstrate the superiority of our method over conventional methods on multilabel datasets.
PATTERN RECOGNITION
(2023)
Article
Engineering, Electrical & Electronic
Christopher Robbiano, Mahmood R. Azimi-Sadjadi, Edwin K. P. Chong
Summary: This paper addresses an autonomous exploration problem in which a mobile sensor uses an information-theoretic navigation cost function to efficiently detect and classify objects of interest in an unseen search area. The proposed cost function consists of two information gain terms and a novel closed-form representation derived from the definition of mutual information. Experimental results show that using the proposed cost function can improve search efficiency compared to other methods.
IEEE TRANSACTIONS ON SIGNAL PROCESSING
(2021)
Article
Energy & Fuels
Achilles Kefalas, Andreas B. Ofner, Gerhard Pirker, Stefan Posch, Bernhard C. Geiger, Andreas Wimmer
Summary: The study proposes an efficient method for detecting knocking combustion in internal combustion engines using continuous wavelet transformation and convolutional neural network. The approach outperformed existing methods, improving accuracy by 6.15 percentage points. The CWT + CNN method does not require calibrating threshold values for different engines or operating conditions, as long as diverse data is used for training.
Article
Computer Science, Artificial Intelligence
Christian Toth, Denis Helic, Bernhard C. Geiger
Summary: This paper presents Synwalk, a random walk-based community detection method, which detects communities in networks by synthesizing random walks. The results indicate that Synwalk performs robustly in various network scenarios.
DATA MINING AND KNOWLEDGE DISCOVERY
(2022)
Article
Automation & Control Systems
Andreas Benjamin Ofner, Achilles Kefalas, Stefan Posch, Bernhard Claus Geiger
Summary: This study utilizes a 1-D convolutional neural network to detect knocking occurrences in an internal combustion engine. The network achieves an accuracy of over 92% in distinguishing between knocking and non-knocking cycles in tenfold cross-validation. The network outperforms existing methods and demonstrates remarkable generalization ability.
IEEE-ASME TRANSACTIONS ON MECHATRONICS
(2022)
Article
Chemistry, Multidisciplinary
Johannes G. Hoffer, Bernhard C. Geiger, Roman Kern
Summary: The avoidance of scrap and adherence to tolerances are important goals in manufacturing. Researchers propose a simulation method using Gaussian Process surrogate model that considers real manufacturing process uncertainties, acting as a substitute for expensive and computationally intensive finite element method (FEM) simulation, resulting in a fast and robust method to adequately depict reality.
APPLIED SCIENCES-BASEL
(2022)
Article
Chemistry, Analytical
Achilles Kefalas, Andreas B. Ofner, Gerhard Pirker, Stefan Posch, Bernhard C. Geiger, Andreas Wimmer
Summary: The study investigates the potential of using a virtual sensor based on vibration signals acquired by a knock sensor for controlling the combustion process. A data-driven approach utilizing discrete wavelet transform as a preprocessing step and extreme gradient boosting regression models for regression tasks of combustion parameters is introduced. The methodology will be applied to data from two different spark-ignited, single cylinder gas engines, with analysis to identify important features based on the model's decisions.
Review
Metallurgy & Metallurgical Engineering
Johannes G. Hoffer, Andreas B. Ofner, Franz M. Rohrhofer, Mario Lovric, Roman Kern, Stefanie Lindstaedt, Bernhard C. Geiger
Summary: This article introduces the application of theory-inspired machine learning in engineering fields, which combines the advantages of traditional models and modern data-driven methods. This approach can often result in models that are more accurate, simpler, better at extrapolating, and allow for faster model training or inference.
WELDING IN THE WORLD
(2022)
Article
Environmental Sciences
Mario Lovric, Mario Antunovic, Iva Sunic, Matej Vukovic, Simonas Kecorius, Mark Kroell, Ivan Beslic, Ranka Godec, Gordana Pehnec, Bernhard C. Geiger, Stuart K. Grange, Iva Simic
Summary: The authors investigated the changes in particulate matter (PM) concentration during the COVID-19 lockdown in Zagreb, Croatia. The results showed that there were no significant differences in PM concentration during the lockdown compared to pre-lockdown and new normal periods.
INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH
(2022)
Article
Engineering, Mechanical
Andreas B. Ofner, Achilles Kefalas, Stefan Posch, Gerhard Pirker, Bernhard C. Geiger
Summary: This study introduces a novel approach to reconstructing in-cylinder pressure trace using vibration signals recorded by knock sensors. The proposed data-driven methodology employs a convolutional neural network with two distinct branches. The model architecture incentivizes each branch to learn low-frequency and high-frequency contents of the pressure trace. The reconstruction achieves high correlation and small errors, and can also extract peak firing pressure and peak pressure position.
MECHANICAL SYSTEMS AND SIGNAL PROCESSING
(2023)
Review
Computer Science, Artificial Intelligence
Bernhard C. Geiger
Summary: This article reviews the literature on information plane analysis of neural network classifiers, discussing the causal relationship between information bottleneck theory and generalization. The research found conflicting empirical evidence in IP analysis and emphasized the importance of detailed estimation of information quantities. It suggests that compression visualized in IPs may not necessarily be information-theoretic, but often compatible with geometric compression of latent representations.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Computer Science, Artificial Intelligence
Rana Ali Amjad, Kairen Liu, Bernhard C. Geiger
Summary: In this study, three information-theoretic quantities were used to analyze the behavior of trained neural networks, revealing that class selectivity is not a reliable indicator for classification performance. However, when examining individual layers, mutual information and class selectivity show a positive correlation with classification performance.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Computer Science, Interdisciplinary Applications
J. G. Hoffer, S. Ranftl, B. C. Geiger
Summary: This article discusses how to find an input such that the output of a stochastic black box function is as close as possible to a target value. It fills the gap in current approaches by deriving acquisition functions for common criteria and demonstrating their compatibility with certain extensions of Gaussian processes. The experiments show that these derived acquisition functions can outperform classical Bayesian optimization.
COMPUTERS & INDUSTRIAL ENGINEERING
(2023)
Review
Computer Science, Information Systems
Philipp Gabler, Bernhard C. Geiger, Barbara Schuppler, Roman Kern
Summary: Superficially, read and spontaneous speech are two main types of training data in automatic speech recognition, but they are fundamentally different due to the way the audio signal is generated. This review introduces causal reasoning into automatic speech recognition, highlighting the impact of data generation processes on inference and performance. By applying a causal perspective, this work discusses the relationship between data generation mechanisms, learning, and prediction in speech data. Furthermore, the authors argue that a causal perspective can enhance the understanding of models in speech processing.
Article
Computer Science, Information Systems
Franz M. M. Rohrhofer, Stefan Posch, Clemens Gossnitzer, Bernhard C. Geiger
Summary: This study investigates the impact of system parameters on multi-objective optimization in PINNs and the selection of loss weights. The authors find that system parameters effectively scale loss residuals and cause imbalances in MO optimization. However, they demonstrate that loss weights can compensate for the scaling of system parameters and enable the selection of an optimal solution on the Pareto front.
Article
Automation & Control Systems
Andreas Benjamin Ofner, Achilles Kefalas, Stefan Posch, Bernhard Claus Geiger
Summary: This study demonstrates the effective detection of knocking events in an internal combustion engine using a 1-D convolutional neural network. The model is capable of accurately distinguishing between knocking and non-knocking cycles, showing remarkable generalization ability to unseen operating points. By training on a small number of non-knocking cycles, the model also achieved increased accuracy in classifying knocking cycles in unseen engines.
IEEE-ASME TRANSACTIONS ON MECHATRONICS
(2022)