Past the power law: Complex systems and the limiting law of restricted diversity

Probability distributions have proven effective at modeling diversity in complex systems. The two most common are the Gaussian normal and skewed-right. While the mechanics of the former are well-known; the latter less so, given the significant limitations of the power-law. Moving past the power-law, we demonstrate that there exists, hidden-in-full-view, a limiting law governing the diversity of complexity in skewed-right systems; which can be measured using a case-based version Cc of Shannon entropy, resulting in a 60/40 rule. For our study, given the wide range of approaches to measuring complexity (i.e., descriptive, constructive, etc), we examined eight different systems, which varied significantly in scale and composition (from galaxies to genes). We found that skewed-right complex systems obey the law of restricted diversity; that is, when plotted for a variety of natural and human-made systems, as the diversity of complexity (primarily in terms of the number of types; but also, secondarily, in terms of the frequency of cases) a limiting law of restricted diversity emerges, constraining the majority of cases to simpler types. Even more compelling, this limiting law obeys a scale-free 60/40 rule: when measured using Cc, 60%(or more) of the cases in these systems reside within the first 40% (or less) of the lower bound of equiprobable diversity typeswith or without long-tail and whether or not the distribution fits a power-law. Furthermore, as an extension of the Pareto Principle, this lower bound accounts for only a small percentage of the total diversity; that is, while the top 20% of cases constitute a sizable percentage of the total diversity in a system, the bottom 60% are highly constrained. In short, as the central limit theorem governs the diversity of complexity in normal distributions, restricted diversity seems to govern the diversity of complexity in skewed-right distributions. (c) 2016 Wiley Periodicals, Inc. Complexity 21: 99-112, 2016