Nonreplication of functional connectivity differences in autism spectrum disorder across multiple sites and denoising strategies

A rapidly growing number of studies on autism spectrum disorder (ASD) have used resting-state fMRI to identify alterations of functional connectivity, with the hope of identifying clinical biomarkers or underlying neural mechanisms. However, results have been largely inconsistent across studies, and there remains a pressing need to determine the primary factors influencing replicability. Here, we used resting-state fMRI data from the Autism Brain Imaging Data Exchange to investigate two potential factors: denoising strategy and data site (which differ in terms of sample, data acquisition, etc.). We examined the similarity of both group-averaged functional connectomes and group-level differences (ASD vs. control) across 33 denoising pipelines and four independently-acquired datasets. The group-averaged connectomes were highly consistent across pipelines (r = 0.92 +/- 0.06) and sites (r = 0.88 +/- 0.02). However, the group differences, while still consistent within site across pipelines (r = 0.76 +/- 0.12), were highly inconsistent across sites regardless of choice of denoising strategies (r = 0.07 +/- 0.04), suggesting lack of replication may be strongly influenced by site and/or cohort differences. Across-site similarity remained low even when considering the data at a large-scale network level or when considering only the most significant edges. We further show through an extensive literature survey that the parameters chosen in the current study (i.e., sample size, age range, preprocessing methods) are quite representative of the published literature. These results highlight the importance of examining replicability in future studies of ASD, and, more generally, call for extra caution when interpreting alterations in functional connectivity across groups of individuals.