Article
Computer Science, Artificial Intelligence
Semih Kaya, Elif Vural
Summary: Numerous approaches exist in the literature for learning low-dimensional representations of multi-modal data collections, yet the generalizability of multi-modal nonlinear embeddings to unseen data has been overlooked. The study highlights the importance of the regularity of interpolation functions for successful generalization in multi-modal classification and retrieval problems, alongside criteria such as between-class separation and cross-modal alignment. The proposed multi-modal nonlinear representation learning algorithm, inspired by theoretical findings, shows promising performance in applications such as multi-modal image classification and cross-modal image-text retrieval.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2021)
Article
Engineering, Electrical & Electronic
Fei Zhou, Wei Wei, Lei Zhang, Yanning Zhang
Summary: In few-shot learning, the meta-learning approach focuses on learning transferable knowledge for fast generalization to new tasks. However, due to the distribution discrepancy between classes, each class requires a specific mapping function to map samples into an ideal semantic space. To address this issue, a new meta-learning approach has been proposed to adaptively manipulate the features of samples for accurate classification.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
(2023)
Article
Engineering, Mechanical
Soheil Sadeghi Eshkevari, Liam Cronin, Soheila Sadeghi Eshkevari, Shamim N. Pakzad
Summary: The study introduces a machine learning approach for input estimation in nonlinear dynamic systems, showing promise in various applications. Data-driven methods have the potential to capture hidden and subtle nonlinearities in different domains. Experimental results confirm the efficacy of input estimations in real-world applications.
MECHANICAL SYSTEMS AND SIGNAL PROCESSING
(2022)
Article
Mathematical & Computational Biology
Duowei Tang, Maja Taseska, Toon van Waterschoot
Summary: This paper proposes a parametric embedding method that maps binaural cues to a low-dimensional space for source localization. The proposed embedding performs well in various acoustic conditions and outperforms previous unsupervised embeddings and direct estimation models, especially with limited training data.
FRONTIERS IN NEUROINFORMATICS
(2022)
Article
Computer Science, Artificial Intelligence
Yanghao Zhang, Wenjie Ruan, Fu Wang, Xiaowei Huang
Summary: Previous studies have shown that deep neural networks can be fooled by universal adversarial attacks using a single human-invisible perturbation. This paper introduces a novel unified framework called GUAP, which enables both additive and non-additive perturbations for universal adversarial attacks. Extensive experiments demonstrate that GUAP outperforms state-of-the-art methods in terms of attack success rates on various datasets and computer vision tasks.
Article
Computer Science, Artificial Intelligence
Javier Marin, Aritro Biswas, Ferda Ofli, Nicholas Hynes, Amaia Salvador, Yusuf Aytar, Ingmar Weber, Antonio Torralba
Summary: This paper introduces Recipe 1M+, a large-scale corpus of cooking recipes and food images, and demonstrates how training neural networks on this data can improve image-recipe retrieval tasks. Regularization through the addition of a high-level classification objective not only enhances retrieval performance but also enables semantic vector arithmetic.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Multidisciplinary Sciences
Maria Littmann, Michael Heinzinger, Christian Dallago, Tobias Olenyi, Burkhard Rost
Summary: This study proposes a GO term prediction method based on SeqVec embedding and protein proximity, with promising results especially for proteins from smaller families or with intrinsically disordered regions.
SCIENTIFIC REPORTS
(2021)
Article
Computer Science, Artificial Intelligence
Marwah A. Helaly, Sherine Rady, Mostafa M. Aref
Summary: This paper focuses on biological sequence classification and inference, exploring efficient representations of biological sequences. A pretrained BERT model and CNN are used to provide a complete prediction model. Data augmentation is applied to enhance classification accuracy, showing promising performance of using contextual embeddings to represent biological sequences.
EXPERT SYSTEMS WITH APPLICATIONS
(2022)
Article
Computer Science, Artificial Intelligence
Manuel Lopez-Martin, Antonio Sanchez-Esguevillas, Juan Ignacio Arribas, Belen Carro
Summary: Contrastive learning enables the establishment of similarities by comparing distances between sample features and labels in a shared embedding space, allowing for supervised classification and improved model performance through reduced pairwise comparisons.
INFORMATION FUSION
(2022)
Article
Computer Science, Artificial Intelligence
Jiae Kim, Yoonkyung Lee, Zhiyu Liang
Summary: Fisher's linear discriminant analysis is limited to linear features, while kernel discriminant analysis overcomes this limitation with nonlinear feature mapping. This study examines the geometry of nonlinear embeddings in discriminant analysis using polynomial and Gaussian kernels. The discriminant function is obtained by solving a generalized eigenvalue problem with covariance operators. The results provide insight into the interaction between data distribution and kernel in determining the nonlinear embedding for discrimination, guiding the choice of kernel and its parameters.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Information Systems
Lingyun Liu, Yifan Zhang, Haoyuan Gao, Xingtao Yu, Jian Cheng
Summary: In this paper, we propose a method for face verification in a federated learning setting, where equivalent class embeddings are transferred to clients to separate their embeddings far away from each other.
MULTIMEDIA SYSTEMS
(2022)
Article
Robotics
Ming Sun, Yue Gao
Summary: This article introduces the challenges that intelligent service robots face in performing tasks and proposes a knowledge representation framework suitable for unstructured environments. By modeling the relationship among grasping tools, actions, and target objects, the algorithm GATER is presented, and its effectiveness is demonstrated through experiments.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2022)
Article
Chemistry, Analytical
Marius Dediu, Costin-Emanuel Vasile, Calin Bira
Summary: This paper presents a deep learning method that extends the PhotoNet network architecture for photorealistic universal style transfer. The method improves the fusion of content and style information by adding extra feature-aggregation modules and deeper aggregation across decoding layers. The proposed deep layer aggregation architectures enhance the stylization and quality of the output image.
Article
Biochemical Research Methods
Wayland Yeung, Zhongliang Zhou, Sheng Li, Natarajan Kannan
Summary: Protein language modeling is a new deep learning method in bioinformatics with various applications. This study presents a method for estimating sequence conservation using sequence embeddings generated from protein language models. The ESM2 models show the best performance to computational cost ratio for conservation estimation. The method can identify conserved functional sites in any full-length protein sequence and estimate conservation without the need for sequence alignment.
BRIEFINGS IN BIOINFORMATICS
(2023)
Review
Computer Science, Artificial Intelligence
Jun Liu, Shuang Zheng, Guangxia Xu, Mingwei Lin
Summary: The study extended the CBoW word vector model and proposed a cross-domain sentiment-aware word embedding learning model, which can capture both the sentiment information and domain relevance of a word. The experimental results demonstrate that the model has higher accuracy and Macro-F1 value when dealing with sentiment information.
INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS
(2021)
Article
Automation & Control Systems
Krithika Manohar, J. Nathan Kutz, Steven L. Brunton
Summary: This article utilizes balanced model reduction and greedy optimization to determine optimal sensor and actuator selections that optimize observability and controllability. The results are demonstrated on a high-dimensional system, approximating known optimal placements.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2022)
Article
Mechanics
Jared L. Callaham, Steven L. Brunton, Jean-Christophe Loiseau
Summary: This work explores the use of nonlinear dimensionality reduction to enhance the accuracy and stability of reduced-order models for advection-dominated flows. By leveraging nonlinear correlations between temporal Proper Orthogonal Decomposition (POD) coefficients, latent low-dimensional structures can be identified and approximated using a minimal set of driving modes and a manifold equation for the remaining modes. This approach can stabilize POD-Galerkin models and serve as a state space for data-driven model identification.
JOURNAL OF FLUID MECHANICS
(2022)
Article
Multidisciplinary Sciences
Peter J. Baddoo, Benjamin Herrmann, Beverley J. McKeon, Steven L. Brunton
Summary: Research in modern data-driven dynamical systems tackles the challenges of high dimensionality, unknown dynamics, and nonlinearity. This work presents a kernel method for learning interpretable data-driven models for high-dimensional, nonlinear systems. The method efficiently handles high-dimensional data and incorporates partial knowledge of system physics.
PROCEEDINGS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES
(2022)
Article
Multidisciplinary Sciences
U. Fasel, J. N. Kutz, B. W. Brunton, S. L. Brunton
Summary: In this work, the authors propose an ensemble approach to improve the robustness of the sparse identification of nonlinear dynamics (SINDy) algorithm. The ensemble-SINDy (E-SINDy) algorithm is demonstrated to be able to discover accurate models from extremely noisy and limited data. The authors also show that the ensemble statistics from E-SINDy can be utilized for active learning and improved model predictive control.
PROCEEDINGS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES
(2022)
Article
Mathematics, Applied
Steven L. Brunton, Marko Budisic, Eurika Kaiser, J. Nathan Kutz
Summary: The field of dynamical systems is undergoing a transformation due to the emergence of mathematical tools and algorithms from modern computing and data science. Data-driven approaches that use operator-theoretic or probabilistic frameworks are replacing first-principles derivations and asymptotic reductions. The Koopman spectral theory, which represents nonlinear dynamics using an infinite-dimensional linear operator, has the potential to enable the prediction, estimation, and control of nonlinear systems with standard textbook methods developed for linear systems. However, a challenge remains in obtaining finite-dimensional coordinate systems and embeddings that approximately linearize the dynamics. The success of Koopman analysis is attributed to its rigorous theoretical connections, measurement-based approach suitable for leveraging big data and machine learning techniques, and the development of simple yet powerful numerical algorithms.
Article
Computer Science, Interdisciplinary Applications
Romit Maulik, Dimitrios K. Fytanidis, Bethany Lusch, Venkatram Vishwanath, Saumil Patel
Summary: The tool is a general-purpose Python-based data analysis tool for OpenFOAM, allowing for arbitrary data analysis and manipulation on flow-field information and supporting online singular value decomposition accessible through the OpenFOAM solver.
JOURNAL OF COMPUTATIONAL SCIENCE
(2022)
Article
Engineering, Aerospace
Michelle K. Hickner, Urban Fasel, Aditya G. Nair, Bingni W. Brunton, Steven L. Brunton
Summary: This study extends the classical unsteady aerodynamic models to include the deformation of a flexible wing, and develops low-order linear models based on data from direct numerical simulations. Model predictive control is used to track maneuvers and limit wing deformation, providing an interpretable and accurate representation of the aeroelastic system for applications where transients are important.
Article
Physics, Fluids & Plasmas
J. D. Lore, S. De Pascuale, P. Laiu, B. Russo, J. -S. Park, J. M. Park, S. L. Brunton, J. N. Kutz, A. A. Kaptanoglu
Summary: Time-dependent SOLPS-ITER simulations were used to identify reduced models using the SINDy method and develop model-predictive control of the boundary plasma state. The identified reduced models show good predictive accuracy and can be applied to other time-dependent data from boundary simulations or experimental data.
Article
Physics, Fluids & Plasmas
Alan A. Kaptanoglu, Christopher Hansen, Jeremy D. Lore, Matt Landreman, Steven L. Brunton
Summary: Many scientific problems can be solved by sparse regression, which finds parsimonious and interpretable solutions by exploring high-dimensional spaces and assuming that some parameters are zero or negligible. This technique has been widely used in signal and image processing, system identification, optimization, and parameter estimation methods like Gaussian process regression. In this paper, the authors illustrate the importance of sparse regression in plasma physics and discuss recent contributions and challenges in solving related problems, especially in constrained and high-dimensional scenarios.
PHYSICS OF PLASMAS
(2023)
Article
Multidisciplinary Sciences
Peter J. J. Baddoo, Benjamin Herrmann, BeverleyJ. J. McKeon, J. Nathan Kutz, Steven L. L. Brunton
Summary: In this work, the integration of physical principles into the dynamic mode decomposition (DMD) is demonstrated. A physics-informed DMD (piDMD) optimization is proposed to restrict the models to a matrix manifold that respects the physical structure of the system. Several closed-form solutions and efficient algorithms for the corresponding piDMD optimizations are derived based on fundamental physical principles. The piDMD models outperform standard DMD algorithms in various applications, showing advantages in spectral identification, state prediction, and estimation.
PROCEEDINGS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES
(2023)
Article
Mechanics
Jared L. Callaham, Jean-Christophe Loiseau, Steven L. Brunton
Summary: In this study, we introduce a projection-based model reduction method that takes into account the nonlinear interactions between the resolved and unresolved scales of the flow in a low-dimensional dynamical systems model. The method uses a separation of time scales and a perturbation series approximation to derive a reduced-order model with closure terms, which improves the stability and accuracy of the flow models.
JOURNAL OF FLUID MECHANICS
(2023)
Article
Multidisciplinary Sciences
Joseph Bakarji, Kathleen Champion, J. Nathan Kutz, Steven L. Brunton
Summary: A central challenge in data-driven model discovery is the presence of hidden, or latent, variables that are not directly measured but are dynamically important. We designed a deep autoencoder network to learn a coordinate transformation from the delay embedded space into a new space, where it is possible to represent the dynamics in a sparse, closed form. This framework combines deep learning and the sparse identification of nonlinear dynamics methods to uncover interpretable models within effective coordinates.
PROCEEDINGS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES
(2023)
Article
Computer Science, Information Systems
Kartik Krishna, Steven L. Brunton, Zhuoyuan Song
Summary: Finite-time Lyapunov exponents (FTLEs) can be used to compute time-varying analogs of invariant manifolds in unsteady fluid flow fields, providing insight into optimal transport routes and effective deployment locations for controlled agents.
Proceedings Paper
Computer Science, Hardware & Architecture
Shilpika Shilpika, Bethany Lusch, Murali Emani, Filippo Simini, Venkatram Vishwanath, Michael E. Papka, Kwan-Liu Ma
Summary: To maintain a robust and reliable supercomputing facility, monitoring and understanding the hardware system events and behaviors are essential. In this work, we built an end-to-end error log analysis system that examines the job logs and extracts insights from their correlation with hardware error logs and environment logs. Our machine learning pipeline achieved an accuracy of 92% in predicting the job exit status and provides sufficient lead time for preventive measures before the actual failure occurs.
2022 22ND IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND INTERNET COMPUTING (CCGRID 2022)
(2022)
Article
Geosciences, Multidisciplinary
Romit Maulik, Vishwas Rao, Jiali Wang, Gianmarco Mengaldo, Emil Constantinescu, Bethany Lusch, Prasanna Balaprakash, Ian Foster, Rao Kotamarthi
Summary: Data assimilation is crucial in geophysical sciences for robust forecasts. Traditional methods are computationally expensive, but this paper proposes a new approach using a low-dimensional, data-driven, and differentiable emulator to accelerate data assimilation. The results show improved computational efficiency by four orders of magnitude and increased accuracy compared to traditional methods.
GEOSCIENTIFIC MODEL DEVELOPMENT
(2022)