> Standard deviation, STD, should be left to mathematicians, physicists and mathematical statisticians deriving limit theorems.
E.g., for positive integer n and a sequence of n random variables with the same expectation and with finite variance and, as n grows to infinity, with the variance converging to zero, the random variables, actually points in the Hilbert space commonly called L^2, converge in the norm of that space, and then a subsequence must converge almost surely, that is, the strongest case of convergence. Of course, this is a very old result and standard when consider convergence of random variables.
But standard deviation still has an important role in common applications of statistics without "deriving limit theorems". And, with some irony, we don't derive a limit theorem but use one, indeed, likely the most important one, the central limit theorem (CLT).
With the CLT, under mild assumptions, for positive integer n, as n grows to infinity, the probability distribution of the mean of n independent and identically distributed (the i.i.d. case) converges to a Gaussian. Likely the mildest assumptions are from the Lindeberg-Feller case (don't ask but look it up if you wish, and to read the proof set aside much of an afternoon).
Now, when have convergence to a Gaussian and have the standard deviation of that Gaussian, we can calculate any and all confidence intervals we want on our estimate of the mean of that Gaussian. So, THAT'S one case of where and why even in just common work we still want standard deviation.
Yes, how fast the convergence is to a Gaussian can be relevant in protecting against Talib's "black swans" and avoiding, say, the disaster of Long Term Capital Management (LTCM) in their estimates of volatility.
That is, suppose we want to estimate the standard deviation of an average (as above). Suppose the random variables we are averaging have a distribution that has in its probability density function a bump way, way, way out in a tail. The way out in the tail means that if get such a value, then it's really large (in absolute value, and in practice really far from the expectation of that random variable). So, if get a value in that bump, then can have a "black swan". But the probability of the bump is quite small. So, we can take samples from the distribution of that random variable and average them for weeks before we ever get a sample from the bump, before ever see a black swan.
So, doing this, in our sampling never seeing a black swan, we can have an estimate of standard deviation that is significantly too small. So, with that small standard deviation, can believe that some highly leveraged financial positions are relatively safe, that is, also have low volatility.
Then, bad day, the Russians default on something, we get a "black swan", and suddenly lose some billions of dollars where before we were really sure that wouldn't happen for millennia. Sorry 'bout that.
Roughly, that is what happened in the famous, expensive crash of LTCM.
> Standard deviation, STD, should be left to mathematicians, physicists and mathematical statisticians deriving limit theorems.
E.g., for positive integer n and a sequence of n random variables with the same expectation and with finite variance and, as n grows to infinity, with the variance converging to zero, the random variables, actually points in the Hilbert space commonly called L^2, converge in the norm of that space, and then a subsequence must converge almost surely, that is, the strongest case of convergence. Of course, this is a very old result and standard when consider convergence of random variables.
But standard deviation still has an important role in common applications of statistics without "deriving limit theorems". And, with some irony, we don't derive a limit theorem but use one, indeed, likely the most important one, the central limit theorem (CLT).
With the CLT, under mild assumptions, for positive integer n, as n grows to infinity, the probability distribution of the mean of n independent and identically distributed (the i.i.d. case) converges to a Gaussian. Likely the mildest assumptions are from the Lindeberg-Feller case (don't ask but look it up if you wish, and to read the proof set aside much of an afternoon).
Now, when have convergence to a Gaussian and have the standard deviation of that Gaussian, we can calculate any and all confidence intervals we want on our estimate of the mean of that Gaussian. So, THAT'S one case of where and why even in just common work we still want standard deviation.
Yes, how fast the convergence is to a Gaussian can be relevant in protecting against Talib's "black swans" and avoiding, say, the disaster of Long Term Capital Management (LTCM) in their estimates of volatility.
That is, suppose we want to estimate the standard deviation of an average (as above). Suppose the random variables we are averaging have a distribution that has in its probability density function a bump way, way, way out in a tail. The way out in the tail means that if get such a value, then it's really large (in absolute value, and in practice really far from the expectation of that random variable). So, if get a value in that bump, then can have a "black swan". But the probability of the bump is quite small. So, we can take samples from the distribution of that random variable and average them for weeks before we ever get a sample from the bump, before ever see a black swan.
So, doing this, in our sampling never seeing a black swan, we can have an estimate of standard deviation that is significantly too small. So, with that small standard deviation, can believe that some highly leveraged financial positions are relatively safe, that is, also have low volatility.
Then, bad day, the Russians default on something, we get a "black swan", and suddenly lose some billions of dollars where before we were really sure that wouldn't happen for millennia. Sorry 'bout that.
Roughly, that is what happened in the famous, expensive crash of LTCM.