4.7 Article

Minimax Sparse Logistic Regression for Very High-Dimensional Feature Selection

Journal

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2013.2263427

Keywords

Feature selection; minimax problem; single-nucleotide polymorphism (SNP) detection; smoothing method; sparse logistic regression

Funding

  1. Singapore NTU A*SERC [112 172 0013]

Ask authors/readers for more resources

Because of the strong convexity and probabilistic underpinnings, logistic regression (LR) is widely used in many real-world applications. However, in many problems, such as bioinformatics, choosing a small subset of features with the most discriminative power are desirable for interpreting the prediction model, robust predictions or deeper analysis. To achieve a sparse solution with respect to input features, many sparse LR models are proposed. However, it is still challenging for them to efficiently obtain unbiased sparse solutions to very high-dimensional problems (e. g., identifying the most discriminative subset from millions of features). In this paper, we propose a new minimax sparse LR model for very high-dimensional feature selections, which can be efficiently solved by a cutting plane algorithm. To solve the resultant nonsmooth minimax subproblems, a smoothing coordinate descent method is presented. Numerical issues and convergence rate of this method are carefully studied. Experimental results on several synthetic and real-world datasets show that the proposed method can obtain better prediction accuracy with the same number of selected features and has better or competitive scalability on very high-dimensional problems compared with the baseline methods, including the l(1)-regularized LR.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Computer Science, Information Systems

Toward Equivalent Transformation of User Preferences in Cross Domain Recommendation

Xu Chen, Ya Zhang, Ivor W. Tsang, Yuangang Pan, Jingchao Su

Summary: This article discusses cross-domain recommendation in scenarios where different domains have the same set of users but no overlapping items. Most existing methods focus on shared-user representation, but fail to capture domain-specific features. In this article, an equivalent transformation learner (ETL) is proposed to preserve both domain-specific and overlapped features by modeling the joint distribution of user behaviors across domains.

ACM TRANSACTIONS ON INFORMATION SYSTEMS (2023)

Article Engineering, Industrial

Operation twins: production-intralogistics synchronisation in Industry 4.0

Mingxing Li, Daqiang Guo, Ming Li, Ting Qu, George Q. Huang

Summary: The widespread adoption of Industry 4.0 technologies is revolutionising manufacturing operations. This paper introduces a novel concept of operations twins (OT) for achieving synchronisation between production and intralogistics (PiL) through the use of Industry 4.0 technologies and innovative operations management strategies.

INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH (2023)

Article Computer Science, Artificial Intelligence

LADDER: Latent boundary-guided adversarial training

Xiaowei Zhou, Ivor W. Tsang, Jie Yin

Summary: Deep Neural Networks have achieved great success in classification tasks, but they are vulnerable to adversarial attacks. Adversarial training is an effective strategy to improve the robustness of DNN models, but existing methods fail to generalize well to standard test data. To achieve a better trade-off between standard accuracy and adversarial robustness, a novel adversarial training framework called LADDER is proposed, which generates high-quality adversarial examples through perturbations on latent features.

MACHINE LEARNING (2023)

Article Computer Science, Artificial Intelligence

Coarse-to-Fine Contrastive Learning on Graphs

Peiyao Zhao, Yuangang Pan, Xin Li, Xu Chen, Ivor W. Tsang, Lejian Liao

Summary: Inspired by the success of contrastive learning, graph augmentation strategies have been used to learn node representations. Existing methods add perturbations to construct contrastive samples. However, they ignore the prior information assumption, leading to decreased similarity and increased discrimination among nodes. In this article, a general ranking framework is proposed to incorporate these prior information into contrastive learning. Experimental results on benchmark datasets show the effectiveness of the proposed algorithm compared to supervised and unsupervised models.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2023)

Article Computer Science, Artificial Intelligence

Open Set Domain Adaptation With Soft Unknown-Class Rejection

Yiming Xu, Lin Chen, Lixin Duan, Ivor W. Tsang, Jiebo Luo

Summary: This article studies the problem of open set domain adaptation and proposes a method that performs soft rejection of unknown target classes and simultaneously matches the source and target domains. Extensive experiments on three standard datasets validate the effectiveness of the proposed method.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2023)

Article Engineering, Multidisciplinary

Privacy Protection Method Based On Multidimensional Feature Fusion Under 6G Networks

Mianjie Li, Yuan Liu, Zhihong Tian, Chun Shan

Summary: In this article, an information hiding method based on multidimensional feature fusion for privacy protection in 6G networks is proposed. The method is based on the strong attack resistance of carriers carrying private information, and it involves studying multidimensional feature fusion arbitration methods and constructing an efficient and accurate feature-based information hiding technology.

IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING (2023)

Article Computer Science, Artificial Intelligence

Latent Class-Conditional Noise Model

Jiangchao Yao, Bo Han, Zhihan Zhou, Ya Zhang, Ivor W. Tsang

Summary: Learning with noisy labels is important in the Big Data era to save costs. Previous noise-transition-based methods achieved good performance but relied on impractical anchor sets. Our approach introduces a Bayesian framework for parameterizing the noise transition and solves the problem of ill-posed stochastic learning in back-propagation.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2023)

Article Computer Science, Artificial Intelligence

DifFormer: Multi-Resolutional Differencing Transformer With Dynamic Ranging for Time Series Analysis

Bing Li, Wei Cui, Le Zhang, Ce Zhu, Wei Wang, Ivor W. Tsang, Joey Tianyi Zhou

Summary: Time series analysis is crucial in various fields such as economics, finance, and surveillance. However, traditional Transformer models have limitations in representing nuanced patterns in time series data. To overcome these challenges, we propose a novel Transformer architecture called DifFormer, which incorporates a multi-resolutional differencing mechanism. DifFormer outperforms existing models in classification, regression, and forecasting tasks, while also exhibiting efficiency and lower time consumption.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2023)

Article Computer Science, Artificial Intelligence

Noisy Label Learning With Provable Consistency for a Wider Family of Losses

Defu Liu, Wen Li, Lixin Duan, Ivor W. Tsang, Guowu Yang

Summary: Deep models have achieved impressive performance in various visual recognition tasks, but their generalization ability is compromised by noisy labels. This paper presents a dynamic label learning algorithm that allows the use of different loss functions for classification in the presence of label noise, ensuring that the search for the optimal classifier of noise-free samples is not hindered by label noise.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2023)

Article Computer Science, Artificial Intelligence

Missingness-Pattern-Adaptive Learning With Incomplete Data

Yongshun Gong, Zhibin Li, Wei Liu, Xiankai Lu, Xinwang Liu, Ivor W. W. Tsang, Yilong Yin

Summary: Many real-world problems involve data with missing values, which can hinder learning achievements. Existing methods use a universal model for all incomplete data, resulting in suboptimal models for each missingness pattern. This paper proposes a general model that can adjust to different missingness patterns, minimizing competition between data. The model is based on observable features and does not rely on data imputation, and a low-rank constraint is introduced to improve generalization ability.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2023)

Article Computer Science, Artificial Intelligence

MetaCAR: Cross-Domain Meta-Augmentation for Content-Aware Recommendation

Hui Xu, Changyu Li, Yan Zhang, Lixin Duan, Ivor W. Tsang, Jie Shao

Summary: Cold-start is crucial for recommendations, especially when there is limited user-item interaction data. Meta-learning-based approaches have shown success in addressing this issue by leveraging their strong generalization capabilities to quickly adapt to new tasks in cold-start settings. However, these methods are prone to meta-overfitting when trained with single and sparse ratings because a single rating cannot capture a user's diverse interests under different circumstances. To overcome this, a meta-augmentation technique is proposed to convert non-mutually-exclusive (Non-ME) tasks into mutually-exclusive (ME) tasks without changing inputs, thus relieving the issue of meta-overfitting. Inspired by this technique, this paper proposes a cross-domain meta-augmentation technique for content-aware recommendation systems (MetaCAR) to construct ME tasks in the recommendation scenario. The proposed method consists of two stages: meta-augmentation and meta-learning. In the meta-augmentation stage, domain adaptation is performed using a dual conditional variational autoencoder (CVAE) with a multi-view information bottleneck constraint, and the learned CVAE is applied to generate ratings for users in the target domain. In the meta-learning stage, both the true and generated ratings are used to construct ME tasks, enabling meta-learning recommendations to avoid meta-overfitting. Experiments conducted on real-world datasets demonstrate the significant superiority of MetaCAR over competing baselines, including cross-domain, content-aware, and meta-learning-based recommendations, in dealing with the cold-start user issue.

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING (2023)

Article Computer Science, Artificial Intelligence

A Multi-View Multi-Task Learning Framework for Multi-Variate Time Series Forecasting

Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, Ivor W. Tsang

Summary: Multi-variate time series (MTS) data is a common type of data abstraction in the real world, generated from a hybrid dynamical system. MTS data can be categorized into spatial and temporal attributes, and can be analyzed from the spatial view or temporal view. A novel multi-view multi-task (MVMT) learning framework is proposed to extract hidden MVMT information from MTS data while predicting. The framework improves effectiveness and efficiency of canonical architectures according to extensive experiments on three datasets.

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING (2023)

Article Computer Science, Artificial Intelligence

Distribution Matching for Machine Teaching

Xiaofeng Cao, Ivor W. Tsang

Summary: Machine teaching is a reverse problem of machine learning, aiming to guide the student towards its target hypothesis using known learning parameters. Previous studies focused on balancing teaching risk and cost to find the best teaching examples. However, when the student doesn't disclose any cue of the learning parameters, the optimization solver becomes ineffective. This article presents a distribution matching-based machine teaching strategy that iteratively shrinks teaching cost to eliminate boundary perturbations, providing an effective solution.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2023)

Article Computer Science, Artificial Intelligence

Structure-Informed Shadow Removal Networks

Yuhao Liu, Qing Guo, Lan Fu, Zhanghan Ke, Ke Xu, Wei Feng, Ivor W. Tsang, Rynson W. H. Lau

Summary: In this paper, a novel structure-informed shadow removal network (StructNet) is proposed to address the problem of shadow remnants in existing deep learning-based methods. StructNet reconstructs the structure information of the input image without shadows and uses it to guide the image-level shadow removal. Two main modules, MSFE and MFRA, are developed to extract image structural features and regularize feature consistency. Additionally, an extension called MStructNet is proposed to exploit multi-level structure information and improve shadow removal performance.

IEEE TRANSACTIONS ON IMAGE PROCESSING (2023)

Article Computer Science, Artificial Intelligence

Data-Efficient Learning via Minimizing Hyperspherical Energy

Xiaofeng Cao, Weiyang Liu, Ivor W. Tsang

Summary: This paper addresses the problem of data-efficient learning from scratch in scenarios where data or labels are expensive to collect. It proposes the MHEAL algorithm based on active learning on homeomorphic tubes of spherical manifolds, and provides comprehensive theoretical guarantees. Empirical results demonstrate the effectiveness of MHEAL in various applications for data-efficient learning.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2023)

No Data Available