Article
Statistics & Probability
Zeng Li, Qinwen Wang, Runze Li
Summary: This paper examines the limiting spectral behaviors of large dimensional Kendall's rank correlation matrices generated by samples with independent and continuous components. The study establishes the central limit theorem for the linear spectral statistics of the Kendall's rank correlation matrices under the Marchenko-Pastur asymptotic regime. Three nonparametric procedures for high dimensional independent test are proposed, with their limiting null distributions derived by implementing the central limit theorem.
ANNALS OF STATISTICS
(2021)
Article
Health Care Sciences & Services
Abdullah Qayed, Dong Han
Summary: Gene sets analysis characterizes intra-subject variation using gene expression profiling by collecting multiple sets per subject in microarray data. The study presents tests of multi-set sphericity and multi-set identity of covariance structures across subjects (tumor types), demonstrating good properties through theoretical and empirical studies. The proposed tests were applied on The Cancer Genome Atlas (TCGA) to test covariance structures for gene expressions across several tumor types.
STATISTICAL METHODS IN MEDICAL RESEARCH
(2021)
Article
Biology
Long Yu, Jiahui Xie, Wang Zhou
Summary: In this paper, we propose test statistics for the Kronecker product covariance matrix based on linear spectral statistics of renormalized sample covariance matrices. A central limit theorem is proved for the linear spectral statistics, with explicit formulas for the mean and covariance functions, thereby filling a gap in the literature. We then show theoretically that the proposed test statistics have well-controlled size and high power. We further propose a bootstrap resampling algorithm to approximate the limiting distributions of the associated linear spectral statistics. Consistency of the bootstrap procedure is guaranteed under mild conditions. The proposed test procedure is also applicable to the Kronecker product covariance model with additional random noise. In our simulations, the empirical sizes of the proposed test procedure and its bootstrapped version are close to the corresponding theoretical values, while the power converges to 1 quickly as the dimension and sample size increase.
Article
Mathematics, Applied
Xiaozhuo Zhang, Zhidong Bai, Jiang Hu
Summary: This paper investigates the limiting spectral distribution and analytic behavior of high-dimensional noncentral Fisher matrices and presents the determination criterion for the support of the limiting spectral distribution.
SCIENCE CHINA-MATHEMATICS
(2023)
Article
Physics, Fluids & Plasmas
Zdzislaw Burda, Andrzej Jarosz
Summary: The article discusses the problem of estimating large-dimensional covariance matrices when correlations exist between samples. It introduces a generalized approach using random matrix theory and free probability, and develops an efficient algorithm and an open-source Python library called SHRINKAGE for practical estimation tasks. An example of estimating synthetic data with exponentially decaying autocorrelations is provided.
Article
Statistics & Probability
Zeyu Wu, Cheng Wang
Summary: This paper studies the empirical spectral distribution of Spearman's rank correlation matrices, assuming that the observations are independent and identically distributed random vectors and the features are correlated. The paper shows that the limiting spectral distribution follows the generalized Marcenko-Pastur law with the covariance matrix of the observation after standardized transformation. Several classical covariance/correlation matrices are compared, including the sample covariance matrix, Pearson's correlation matrix, Kendall's correlation matrix, and Spearman's correlation matrix.
JOURNAL OF MULTIVARIATE ANALYSIS
(2022)
Article
Chemistry, Multidisciplinary
Hyeongmin Cho, Sangkyun Lee
Summary: This paper proposes two data quality measures that can compute class separability and in-class variability for a given dataset, focusing on large-scale high-dimensional data such as images and videos. The measures are efficient and compatible with classical measures on small-scale data, offering statistical benefits on large-scale datasets.
APPLIED SCIENCES-BASEL
(2021)
Article
Statistics & Probability
Yangchang Xu, Ningning Xia
Summary: This paper investigates the limiting behavior of eigenvectors of the sample spatial sign covariance matrix (SSCM) by introducing the eigenvector empirical spectral distribution (VESD) with weights depending on the eigenvectors. The results show that the VESD of a large-dimensional sample SSCM converges to a generalized Marcenko-Pastur distribution when both the dimension p of observations and the sample size n tend to infinity proportionally. In addition, the central limit theorem of linear spectral statistics of VESD is established, implying that the eigenmatrix of sample SSCM and the classical sample covariance matrix are asymptotically the same.
JOURNAL OF MULTIVARIATE ANALYSIS
(2023)
Article
Statistics & Probability
Long Feng, Tiefeng Jiang, Binghui Liu, Wei Xiong
Summary: This article investigates the testing problem for cross-sectional independence in high-dimensional panel data using linear regression models. Three tests, including the sum, the max, and the max-sum tests, are studied and evaluated through theoretical analysis and simulation. The results show that the max-sum test outperforms the other two tests and demonstrates good performance and robustness in practical applications.
ANNALS OF STATISTICS
(2022)
Article
Physics, Fluids & Plasmas
Wojciech Tarnowski
Summary: This paper investigates the properties of eigenvalues when randomness is introduced at the level of real matrix elements. It is found that in the limit of large matrix size, the density of real eigenvalues is proportional to the square root of the asymptotic density of complex eigenvalues continuated to the real line.
Article
Statistics & Probability
Yanqing Yin, Yanyuan Ma
Summary: This paper studies the properties of eigenvalues and eigenvectors of high-dimensional sample correlation matrices, improves existing results, and establishes central limit theorems. The study reveals the difference between the functional central limit theorem of sample covariance matrices and sample correlation matrices, influenced by nonrandom projection vector direction. Additionally, the normalization method affects the asymptotic properties of the eigenmatrix. The theoretical results are applied in the field of communications.
ANNALS OF APPLIED PROBABILITY
(2022)
Article
Computer Science, Artificial Intelligence
Naoko Koide-Majima, Kei Majima
Summary: The study introduces a quantum-inspired computational approach for approximating CCA, which demonstrates higher computational efficiency and performance compared to traditional methods when dealing with high-dimensional data. Experimental evaluations show that the proposed qiCCA algorithm can extract more correlations and achieve comparable performance with state-of-the-art nonlinear variants of CCA on multiple datasets.
Article
Statistics & Probability
Zhanting Long, Zeng Li, Ruitao Lin, Jiaxin Qiu
Summary: In this paper, we investigate the limiting behavior of the singular values of a lag-τ sample auto-correlation matrix Rετ in a high-dimensional factor model with vector white noise process. We establish the limiting spectral distribution (LSD) that characterizes the global spectrum of Rετ and derive the limit of its largest singular value. Under certain assumptions, we show that the LSD of Rετ is the same as the lag-τ sample auto-covariance matrix. Based on this equivalence, we propose two estimators of the total number of factors in a factor model using lag-τ sample auto-correlation matrices. The theoretical results are supported by numerical experiments.
JOURNAL OF MULTIVARIATE ANALYSIS
(2023)
Article
Statistics & Probability
Tao Qiu, Qintong Zhang, Yuanyuan Fang, Wangli Xu
Summary: This article introduces a method for testing the homogeneity of two random vectors. The method involves selecting two subspaces and projecting them onto one-dimensional spaces, using the Cramer-von Mises distance to construct the test statistic. The performance is enhanced by repeating this procedure and the effectiveness is demonstrated through numerical simulations.
JOURNAL OF MULTIVARIATE ANALYSIS
(2024)
Article
Physics, Multidisciplinary
S. Ejaz Ahmed, Saeid Amiri, Kjell Doksum
Summary: Regression models provide prediction frameworks based on information concepts for multivariate mutual information analysis, especially when dealing with high dimensional data. Recent research has shown that ensemble approaches can improve on penalty methods like Lasso in certain scenarios, particularly when covariates are strongly associated with the response variable and the model complexity is high. In such cases, the trimmed average version of ensemble Lasso often performs best as a predictor.
Article
Statistics & Probability
Ningning Zhao, Zhidong Bai
COMMUNICATIONS IN STATISTICS-SIMULATION AND COMPUTATION
(2020)
Article
Statistics & Probability
Zhidong Bai, Jiang Hu, Chen Wang, Chao Zhang
Summary: This paper proposes a new test on the linear combinations of covariance matrices in high-dimensional data, which can be applied to many hypothesis tests on covariance matrices. The empirical example in financial portfolio allocation illustrates the results of the proposed test.
STATISTICAL PAPERS
(2021)
Article
Physics, Mathematical
Zhiqiang Hou, Yan Liu, Zhidong Bai, Jiang Hu
Summary: Roy's largest root test is a common test statistic with high power under rank-one alternatives. Previous research has focused on studying its asymptotic distribution under rank-one alternatives, but our study extends this to wider alternatives using approximate power derived from spiked eigenvalues in a high-dimensional setting, applicable to cases involving rank-finite alternatives.
RANDOM MATRICES-THEORY AND APPLICATIONS
(2021)
Article
Statistics & Probability
Qiuyan Zhang, Jiang Hu, Zhidong Bai
JOURNAL OF STATISTICAL PLANNING AND INFERENCE
(2020)
Article
Statistics & Probability
Wenya Luo, Zhidong Bai, Shurong Zheng, Yongchang Hui
STATISTICS & PROBABILITY LETTERS
(2020)
Article
Statistics & Probability
Dandan Jiang, Zhidong Bai
Summary: The study focuses on a more generalized spiked covariance matrix, categorizing it based on whether the maximum absolute value of the eigenvector of the corresponding spikes tends to zero or not. A Generalized Four Moment Theorem (G4MT) is proposed by relaxing the matching of the 3rd and 4th moment, showing the universality of the asymptotic law. Removing the restrictive condition of block wise diagonal assumption on the population covariance matrix, the new Central Limit Theorem (CLT) has a much broader application domain.
Article
Statistics & Probability
Gregory Tai Xiang Ang, Zhidong Bai, Kwok Pui Choi, Yasunori Fujikoshi, Jiang Hu
Summary: This paper examines the applicability of different computational methods in Roy's Largest Root Test (RLRT) and investigates the robustness of RLRT in high-dimensional datasets through simulation studies.
COMMUNICATIONS IN STATISTICS-SIMULATION AND COMPUTATION
(2023)
Article
Statistics & Probability
Zhidong Bai, Jack W. Silverstein
Summary: The authors reminisce on their association with P.R. Krishnaiah, highlighting their individual collaborations and applied work in the field of random matrices and eigenvalue behavior.
JOURNAL OF MULTIVARIATE ANALYSIS
(2022)
Article
Statistics & Probability
Tingting Zou, Shurong Zheng, Zhidong Bai, Jianfeng Yao, Hongtu Zhu
Summary: This paper investigates the central limit theorem for linear spectral statistics of high dimensional sample covariance matrices, proposing a new model that covers most known models. By generalizing previous results with the central limit theorem of this model, it can be applied to structural testing of causal processes and analysis of large datasets.
STATISTICAL PAPERS
(2022)
Article
Physics, Mathematical
Gao-Fan Ha, Qiuyan Zhang, Zhidong Bai, You-Gan Wang
Summary: This paper develops a ridgelized Hotelling's T-2 test for a hypothesis on a large-dimensional mean vector, relaxing the Gaussian assumption made in a previous study. By establishing an exact four-moment theorem, which is a simplified version of another work, the proposed test is shown to outperform traditional methods in high-dimensional scenarios, as demonstrated by simulation results.
RANDOM MATRICES-THEORY AND APPLICATIONS
(2022)
Review
Physics, Mathematical
Jungang Ge, Ying-Chang Liang, Zhidong Bai, Guangming Pan
Summary: Large-dimensional random matrix theory, originating from quantum physics research, offers profound insights into large-dimensional systems. It finds important applications in wireless communications and deep learning, aiding in algorithm design and performance analysis.
RANDOM MATRICES-THEORY AND APPLICATIONS
(2021)
Article
Statistics & Probability
Dandan Jiang, Zhidong Bai
Summary: This paper provides a complementary proof of the Generalized Four Moment Theorem (G4MT) and introduces a new theorem called PG4MT, which is found to be more general than G4MT. The detailed proof of PG4MT is necessary, as it is discovered to derive a Central Limit Theorem for spiked eigenvalues of sample covariance matrices, covering previous works by Bai and Yao.
Article
Statistics & Probability
Huanchao Zhou, Zhidong Bai, Jiang Hu
Summary: This paper investigates the limiting spectral distribution of large-dimensional general information-plus-noise-type matrices. The empirical distribution of eigenvalues is shown to converge weakly to a non-random probability distribution characterized by a system of equations involving its Stieltjes transform.
JOURNAL OF THEORETICAL PROBABILITY
(2023)
Article
Economics
Hua Li, Zhidong Bai, Wing-Keung Wong, Michael McAleer
Summary: This study addresses the portfolio problem for high dimensional data and analyzes the limitations of the traditional Markowitz mean-variance portfolio. A new spectrally corrected method is proposed to improve the expected return and risk of the portfolio. Simulation experiments and empirical analysis demonstrate the superiority of the spectrally corrected estimates in both portfolio return and risk.
ECONOMETRICS AND STATISTICS
(2022)
Article
Statistics & Probability
Qiuyan Zhang, Jiang Hu, Zhidong Bai
ELECTRONIC JOURNAL OF STATISTICS
(2019)
