☆ 4.5 Article

On Various Confidence Intervals Post-Model-Selection

STATISTICAL SCIENCE (2015)

Journal

STATISTICAL SCIENCE

Volume 30, Issue 2, Pages 216-227

Publisher

INST MATHEMATICAL STATISTICS

DOI: 10.1214/14-STS507

Keywords

Confidence intervals; model selection; nonstandard coverage target; AIC; BIC; Lasso

Categories

Statistics & Probability

Funding

Deutsche Forschungsgemeinschaft (DFG) [FOR916]
FWF [P26354]
Austrian Science Fund (FWF) [P26354] Funding Source: Austrian Science Fund (FWF)

Ask authors/readers for more resources

Protocol

Community support

Reagent

Community support

Abstract

We compare several confidence intervals after model selection in the setting recently studied by Berk et al. [Ann. Statist. 41 (2013) 802-837], where the goal is to cover not the true parameter but a certain nonstandard quantity of interest that depends on the selected model. In particular, we compare the PoSI-intervals that are proposed in that reference with the naive confidence interval, which is constructed as if the selected model were correct and fixed a priori (thus ignoring the presence of model selection). Overall, we find that the actual coverage probabilities of all these intervals deviate only moderately from the desired nominal coverage probability. This finding is in stark contrast to several papers in the existing literature, where the goal is to cover the true parameter.

Authors

I am an author on this paper

Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.5

Not enough ratings

Secondary Ratings

Novelty

-

Significance

-

Scientific rigor

-

Rate this paper

Recommended

Article Statistics & Probability

On the Length of Post-Model-Selection Confidence Intervals Conditional on Polyhedral Constraints

Danijel Kivaranovic, Hannes Leeb

Summary: Research on valid inference after model selection is actively exploring the polyhedral method for constructing confidence intervals, with varying lengths depending on the model and computation complexity. Simulation results suggest that the sufficient condition for infinite length is met unless the selected model includes almost all or almost none of the available regressors. Additionally, kappa-quantiles exhibit a similar behavior for kappa close to 1 in the distribution of confidence interval lengths.

JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION (2021)

Add to Collection

Article Computer Science, Artificial Intelligence

Spectral information criterion for automatic elbow detection

Luca Martino, Roberto San Millan-Castillo, Eduardo Morgado

Summary: We propose a generalized information criterion that encompasses other well-known criteria such as BIC and AIC. Our proposed SIC is even more general as it does not strictly require knowledge of a likelihood function. It extracts geometric features of the error curve and can be considered an automatic elbow detector.

EXPERT SYSTEMS WITH APPLICATIONS (2023)

Add to Collection

Article Biology

General ways to improve false coverage rate-adjusted selective confidence intervals

Haibing Zhao

Summary: Post-selection inference on thousands of parameters has been studied in recent years. This article proposes two methods for narrowing the confidence intervals, by incorporating the selection event into the calculation and focusing on parameters with very small selection probabilities.

BIOMETRIKA (2022)

Add to Collection

Article Environmental Sciences

Association between short-term exposure to air pollution and COVID-19 mortality in all German districts: the importance of confounders

Gregor Miller, Annette Menzel, Donna P. Ankerst

Summary: This study assessed the associations between air pollution variables and COVID-19 mortality in Germany, taking into account potential confounders. The results showed that the associations became non-significant when other risk factors were considered in the model, highlighting the importance of adequately accounting for confounders in the analysis.

ENVIRONMENTAL SCIENCES EUROPE (2022)

Add to Collection

Article Biology

Hybrid confidence intervals for informative uniform asymptotic inference after model selection

A. Mccloskey

Summary: I propose a new type of confidence interval that allows correct asymptotic inference after model selection without assuming any model is correctly specified. This hybrid confidence interval combines techniques from selective inference and post-selection inference to provide a short interval across various data realizations. The results show that hybrid confidence intervals have correct asymptotic coverage over a broad class of probability distributions with no bound on scaled model parameters. Monte Carlo experiments and an empirical application on diabetes disease progression predictors demonstrate the desirable length and coverage properties of these confidence intervals in small samples.

BIOMETRIKA (2023)

Add to Collection

Article Computer Science, Artificial Intelligence

Model identification for ARMA time series through convolutional neural networks

Wai Hoh Tang, Adrian Rollin

Summary: By training convolutional neural networks on synthetic data with known ground truths, we found that in ARMA time series models, this approach significantly outperforms traditional likelihood-based methods in terms of accuracy and speed, particularly in statistical inference and time series forecasting. This study demonstrates the feasibility of using artificial neural networks for statistical inference in situations where classical likelihood-based methods are difficult or costly to implement.

DECISION SUPPORT SYSTEMS (2021)

Add to Collection

Article Environmental Sciences

Correlation Coefficient-Based Information Criterion for Quantification of Dependence Characteristics in Hydrological Time Series

Ping Xie, Jingqun Huo, Yan-Fang Sang, Yaqing Li, Jie Chen, Ziyi Wu, Vijay P. Singh

Summary: Quantifying the dependence characteristics in hydrological time series is crucial for understanding hydrological variability and managing water resources. This article proposes a correlation coefficient-based information criterion (CCIC) to determine the optimal model order for quantifying dependence characteristics. Experimental results verify the accuracy of CCIC and its superiority over existing criteria, and its application to annual precipitation in China further demonstrates its advantages.

WATER RESOURCES RESEARCH (2022)

Add to Collection

Article Physics, Multidisciplinary

A resampling approach for confidence intervals in linear time-series models after model selection

Wei Dai, Ka Wai Tsang

Summary: Linear models are widely used in econometrics to analyze relationships and estimate parameters. However, empirical studies have shown that these models often do not fit real-world data well, and their selection is a critical issue. In the era of big data, one approach is to consider a large number of covariates and use model selection. However, many model selection methods perform poorly in practice due to insufficient sample size, especially for financial data that are often correlated and have small samples. This study addresses the challenge of constructing accurate confidence intervals after model selection, and proposes a resampling approach with consistent estimators. Theoretical, simulation, and empirical results demonstrate the advantages of this method.

PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS (2023)

Add to Collection

Article Economics

Modified harmonic mean method for spatial autoregressive models

Osman Dogan

Summary: In this paper, the modified harmonic mean method is proposed for estimating marginal likelihood functions of cross-sectional spatial autoregressive models. The method is shown to be applicable for popular cross-sectional spatial autoregressive models in a Bayesian estimation setting. A simulation study is conducted to investigate the performance of this estimator along with other popular information criteria for model selection problems. The simulation results demonstrate that the modified harmonic mean estimator performs well and can be useful for specification search exercises in spatial econometrics.

ECONOMICS LETTERS (2023)

Add to Collection

Article Mathematics

Normalized Information Criteria and Model Selection in the Presence of Missing Data

Nitzan Cohen, Yakir Berchenko

Summary: This article proposes a new approach that enables the use of classic information criteria for model selection with missing data by normalizing the information criteria theory, which is found to be exponentially better in computational complexity than traditional imputation methods, leading to increased statistical efficiency.

MATHEMATICS (2021)

Add to Collection

Article Statistics & Probability

Model selection for time series with nonlinear trend

R. Alraddadi, Q. Shao

Summary: The study proposes a two-step model selection procedure for ARMA models, suitable for time series contaminated with nonlinear trends. The method is able to effectively identify the true model and maintains asymptotic properties as sample size increases.

COMMUNICATIONS IN STATISTICS-THEORY AND METHODS (2022)

Add to Collection

Article Computer Science, Hardware & Architecture

Defect Prediction Using Akaike and Bayesian Information Criterion

Saleh Albahli, Ghulam Nabi Ahmad Hassan Yar

Summary: This study focuses on data cleaning and selection of best parameter values for software defect prediction. The Akaike information criterion (AIC) and the Bayesian information criterion (BIC) were used to select the best variables, and a simple ANN model was trained. The results show that the combination of two variables, ns and entropy, is the best for software defect prediction.

COMPUTER SYSTEMS SCIENCE AND ENGINEERING (2022)

Add to Collection

Article Biochemical Research Methods

Comparison of the effectiveness of variable selection method for creating a diagnostic panel of biomarkers for mass spectrometric lipidome analysis

Alisa O. Tokareva, Vitaliy V. Chagovets, Alexey S. Kononikhin, Natalia L. Starodubtseva, Evgeny N. Nikolaev, Vladimir E. Frankevich

Summary: A reliable diagnostic model can be built by selecting lipid species with the most discriminative potential and developing the model based on these lipids.

JOURNAL OF MASS SPECTROMETRY (2021)

Add to Collection

Article Statistics & Probability

Melded Confidence Intervals Do Not Provide Guaranteed Coverage

Jesse Frey, Yimin Zhang

Summary: Melded confidence intervals are proposed to combine two one-sample confidence intervals, but they do not guarantee the nominal coverage when calculating the difference in population quantiles.

AMERICAN STATISTICIAN (2023)

Add to Collection

Article Health Care Sciences & Services

New adaptive lasso approaches for variable selection in automated pharmacovigilance signal detection

Emeline Courtois, Pascale Tubert-Bitter, Ismail Ahmed

Summary: This study proposes a new signal detection methodology based on the adaptive lasso, which shows equivalent or better performances compared to other competitors in simulations and real data applications. The CISL-based adaptive weights have an advantage over others, making the adaptive lasso a solid alternative for signal detection in pharmacovigilance.

BMC MEDICAL RESEARCH METHODOLOGY (2021)

Add to Collection

Article Statistics & Probability

CONDITIONAL PREDICTIVE INFERENCE POST MODEL SELECTION

Hannes Leeb

ANNALS OF STATISTICS (2009)

Add to Collection

Article Statistics & Probability

ON THE CONDITIONAL DISTRIBUTIONS OF LOW-DIMENSIONAL PROJECTIONS FROM HIGH-DIMENSIONAL DATA

Hannes Leeb

ANNALS OF STATISTICS (2013)

Add to Collection

Article Statistics & Probability

Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process

Hannes Leeb

BERNOULLI (2008)

Add to Collection

Article Statistics & Probability

Shrinkage Estimators for Prediction Out-of-Sample: Conditional Performance

Nina Huber, Hannes Leeb

COMMUNICATIONS IN STATISTICS-THEORY AND METHODS (2013)

Add to Collection

Article Statistics & Probability

CONCENTRATION OF THE SPECTRAL MEASURE OF LARGE WISHART MATRICES WITH DEPENDENT ENTRIES

Adityanand Guntuboyina, Hannes Leeb

ELECTRONIC COMMUNICATIONS IN PROBABILITY (2009)

Add to Collection

Article Economics

Sparse estimators and the oracle property, or the return of Hodges' estimator

Hannes Leeb, Benedikt M. Poetscher

JOURNAL OF ECONOMETRICS (2008)

Add to Collection

Article Statistics & Probability

On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

Benedikt M. Poetscher, Hannes Leeb

JOURNAL OF MULTIVARIATE ANALYSIS (2009)

Add to Collection

Correction Economics

Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results (vol 22, pg 69, 2006)

Hannes Leeb, Benedikt M. Poetscher

ECONOMETRIC THEORY (2008)

Add to Collection

Article Statistics & Probability

Admissibility of the usual confidence interval in linear regression

Paul Kabaila, Khageswor Giri, Hannes Leeb

ELECTRONIC JOURNAL OF STATISTICS (2010)

Add to Collection

Article Economics

Can one estimate the unconditional distribution of post-model-selection estimators?

Hannes Leeb, Benedikt M. Poetscher

ECONOMETRIC THEORY (2008)

Add to Collection

Article Statistics & Probability

Can one estimate the conditional distribution of post-model-selection estimators?

Hannes Leeb, Benedikt M. Poetscher

ANNALS OF STATISTICS (2006)

Add to Collection

Article Statistics & Probability

On the large-sample minimal coverage probability of confidence intervals after model selection

Paul Kabaila, Hannes Leeb

JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION (2006)

Add to Collection

Article Economics

Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results

H Leeb, BM Pötscher

ECONOMETRIC THEORY (2006)

Add to Collection

Article Economics

Model selection and inference:: Facts and fiction

H Leeb, BM Pötscher

ECONOMETRIC THEORY (2005)

Add to Collection

No Data Available

© Peeref 2019-2024. All rights reserved.