Nassim Nicholas Taleb describes limits to what kinds of systems we can apply statistical tools to, especially with respect to distribution properties. Distributions that are “thin-tailed”–having known variances, known sampling properties, ergodic properties–then statistical tools work well. Those systems in which unknown properties may exist are subject to the emergence of a “Black Swan”–an event that retrospectively reveals that you did not actually have a correct model of the system’s statistical properties to start with.
Taleb has summarized these ideas into a four-quadrant classification of domains: those with thin-tailed and those with heavy or unknown tails, and those with simple and complex payoffs. The fourth quadrant is, unfortunately, where most real-world systems exist, and our ability to reduce them to tractable models of the other three domains is limited outside of idealized Gaussian models.
