Two sites, two charts of daily sessions. The first swings by almost 400 sessions a day, the second by just 52: one would be tempted to call the first far less stable.
The opposite is true, and to notice it the mean is not enough: we need to know how far the data stray from it.
That is the job of measures of dispersion (or of variability): describing how much the observations spread around a central value, the mean or the median. Two datasets with the same centre can tell opposite stories, and these measures are what reveal it.
The techniques for measuring the variability of datasets are numerous. Among them, the most widely known (and most commonly used) are:
- the range
- the mean deviation and the variance
- the standard deviation
- the interquartile range
- the coefficient of variation
We will also visualise the concepts of central tendency and dispersion by revisiting skewness and introducing the concept of kurtosis.
In short: measures of dispersion tell us how far the data stray from the centre. The range (maximum minus minimum) is the quickest but depends on two values only; the variance is the mean of the squared deviations from the mean, the standard deviation its square root, in the same unit as the data; the interquartile range (Q3 − Q1) is robust to outliers; the coefficient of variation (standard deviation divided by the mean) is a pure number that lets us compare different scales.
| Measure | What it is | Strength | Limit | In R |
|---|---|---|---|---|
| Range | maximum minus minimum | immediate | depends on two values only | diff(range(x)) |
| Variance | mean of the squared deviations from the mean | the basis of almost all inference | squared units, hard to read | var(x) |
| Standard deviation | square root of the variance | same unit as the data | inflated by outliers | sd(x) |
| Interquartile range | Q3 − Q1, the width of the middle half | robust to outliers | ignores the tails | IQR(x) |
| Coefficient of variation | standard deviation divided by the mean | compares different scales | meaningless with means near zero | sd(x) / mean(x) |
What We’ll Cover
The Range
The range is the difference between the maximum and the minimum value of ungrouped data in a frequency distribution.
It is a very quick calculation, which in R can be computed as follows:
max(var) - min(var)The maximum and minimum can also be displayed with:
range(var)and they appear as the first and last terms in:
fivenum(var)For grouped data, the range is defined as the difference between the upper boundary of the highest class and the lower boundary of the lowest class.
A trimmed range is a range from which a certain percentage of extreme values has been removed from both ends of the distribution (for instance, the middle 80 per cent).
The Mean Deviation
The mean deviation is a measure of variability based on the difference between each data point and the mean. If we calculated the average by summing the positive and negative differences between each value and the arithmetic mean, the result would always be zero. For this reason, we sum the absolute values of the differences:
\( SM = \frac{\Sigma|X – \mu|}{N} \\ \)Those “absolute values” lend themselves poorly to algebra (they cannot easily be expanded or differentiated), which is why the mean deviation is rarely used. There is another way to eliminate negative values, and so we introduce the important concept of…
The Variance
Variance is analogous to the mean deviation, since it is based on the differences between each data point and the mean, but these differences are squared before being summed. Variance is denoted by the lowercase sigma squared symbol, and the formula is:
\( \sigma^{2}=\frac{\Sigma(X – \mu)^{2}}{N} \\ \\ \)R has the var() function for computing variance, but it uses (n-1) in the denominator. To obtain the variance with N in the denominator, we can define a custom function:
pop_var <- function(x) { var(x) * (1 - 1/length(x)) }In general, interpreting the value of a variance is difficult because the units it is expressed in are not the same as those of the original observations.
For this reason, the standard deviation was introduced.
The Standard Deviation: The Most Widely Used Measure of Variability
The standard deviation is simply the square root of the variance:
\( \sigma = \sqrt{\frac{\Sigma(X - \mu)^{2}}{N}} \\ \\ \)The standard deviation is of fundamental utility in statistics, particularly (as we shall see) in conjunction with the normal distribution.
For grouped data, we assume that the midpoint of each class represents all the measurements within that class. The variance formula then becomes:
\( \sigma^{2}=\frac{\Sigma f(X - \mu)^{2}}{N} \\ \\ \)and the standard deviation:
\( \sigma = \sqrt{\frac{\Sigma f(X - \mu)^{2}}{N}} \\ \\ \)In R, the function for computing the standard deviation is sd(). However, R uses (n-1) in the denominator. So, if we want the population standard deviation (with n in the denominator), we can define a dedicated function:
pop_sd <- function(x) { sqrt(sum((x - mean(x))^2) / length(x)) }The Interquartile Range (IQR)
All the measures seen so far revolve around the mean, and inherit its weakness: a single outlier is enough to inflate them. The interquartile range (IQR) takes another road: it is the distance between the third and the first quartile, that is the width of the middle half of the data.
\( IQR = Q_{3} - Q_{1} \)Precisely because it leaves out the lowest and the highest quarter of the data, the interquartile range is not dragged by extreme values: it is the natural measure of dispersion to pair with the median, and it is the height of the box in the boxplot. Let's compute it in R on the same data we will use for the coefficient of variation:
data <- c(24,17,21,23,15,30,24,21,24,19,25,28,22,20,14,19,26,29,23,25,24,18,27,21)
quantile(data, c(0.25, 0.75))
# 25% 75%
# 19.75 25.00
IQR(data)
# [1] 5.25The middle half of the data therefore lies in an interval 5.25 wide.
(Note: R computes quartiles by interpolating between values; other software, or the rules for hand calculation, can give slightly different results. Nothing to worry about.) The interquartile range is also the yardstick of the most widely used rule for flagging outliers: points lying more than 1.5 times the IQR beyond the first or third quartile are suspect.
The Coefficient of Variation (CV)
The coefficient of variation indicates the relative magnitude of the standard deviation with respect to the distribution mean. It is extremely useful for comparing phenomena expressed in different units of measurement, since the CV is a "pure" number, independent of the unit of measurement:
\( CV = \frac{\sigma}{\mu} \\ \)As is often the case in R, there is a ready-made function: we can use cv(), defined in an external library, labstatR. Its usage is straightforward:
library(labstatR)
data <- c(24, 17, 21, 23, 15, 30, 24, 21, 24, 19,
25, 28, 22, 20, 14, 19, 26, 29, 23, 25,
24, 18, 27, 21)
cv(data)
# [1] 0.1817708We can also calculate the value quite simply without resorting to external libraries:
data <- c(24, 17, 21, 23, 15, 30, 24, 21, 24, 19,
25, 28, 22, 20, 14, 19, 26, 29, 23, 25,
24, 18, 27, 21)
pop_sd <- function(x) { sqrt(sum((x - mean(x))^2) / length(x)) }
cv_data <- pop_sd(data) / mean(data)
cv_data
# [1] 0.1817708The Shape of a Distribution
Frequency distributions can take on the most varied shapes. Among all of them, the one by far most important in statistics is the normal distribution, also known as the bell curve or Gaussian distribution.
In a normal distribution, data are arranged symmetrically around the mean. A first clue to the shape of the distribution comes from comparing the mean with the median: if they are close, the distribution is probably symmetric (a clue, not a proof). If the mean is greater than the median, we have positive skewness (with a longer "tail" on the right); if the mean is less than the median, the skewness is negative (with the longer "tail" on the left).
The best-known formula for calculating the skewness of a distribution is Pearson's coefficient of skewness:
\( Skewness = \frac{3(\mu - med)}{\sigma} \\ \\ \)A perfectly symmetric distribution has a skewness value of 0. A right-skewed (positive) distribution has a positive value, while a left-skewed distribution has a negative value.
Skewness values typically fall between -3 and 3, and the fact that the standard deviation appears in the denominator makes the value independent of the unit of measurement.
How do we calculate the skewness index in R? The simplest way is to use a library that provides the functions we need "ready to go":
library(moments)
data <- c(24, 17, 21, 23, 15, 30, 24, 21, 24, 19,
25, 28, 22, 20, 14, 19, 26, 29, 23, 25,
24, 18, 27, 21)
skewness(data)
# [1] -0.1918578The moments library serves us well. Let us see, however, how to calculate the index without relying on a library. It is very simple. The first step is to remember that R uses n-1 in the denominator of the variance. We are reasoning about a population, however, with n in the denominator. So, let us define a function that gives us the value we need:
pop_var <- function(x) { var(x) * (1 - 1/length(x)) }At this point we can calculate the skewness index:
data <- c(24, 17, 21, 23, 15, 30, 24, 21, 24, 19,
25, 28, 22, 20, 14, 19, 26, 29, 23, 25,
24, 18, 27, 21)
pop_var <- function(x) { var(x) * (1 - 1/length(x)) }
z <- (data - mean(data)) / sqrt(pop_var(data))
skew <- mean(z^3)
skew
# [1] -0.1918578Kurtosis
Kurtosis compares the shape of a distribution with that of the normal: how peaked it is at the centre and, above all, how «heavy» its tails are, that is how often values far from the mean turn up.
We have three cases:
- a tall curve, called leptokurtic, which is highly concentrated around its mean
- a normal curve, called mesokurtic
- a low and flat curve, called platykurtic, with little concentration around its mean
Kurtosis can be measured by dividing the fourth moment by the standard deviation raised to the fourth power. Sounds difficult? It is easier done than said. Here is the formula:
\( Kurtosis = \frac{\Sigma f(X -\mu)^{4}}{N \sigma^{4}} \\ \\ \)The kurtosis of a mesokurtic curve has a value of 3. Naturally, a kurtosis coefficient less than 3 indicates a platykurtic curve, while a value greater than 3 indicates a leptokurtic curve.
Note: many tools, Excel's KURT function included, return the excess kurtosis, that is the value minus 3 (with a small correction for samples): there the normal is 0. The 2.48 we are about to obtain corresponds to an excess of about −0.52: a slightly platykurtic curve.
As with the skewness index, the moments library provides a convenient ready-made function:
library(moments)
data <- c(24, 17, 21, 23, 15, 30, 24, 21, 24, 19,
25, 28, 22, 20, 14, 19, 26, 29, 23, 25,
24, 18, 27, 21)
kurtosis(data)
# [1] 2.480035But we are not averse to computing it ourselves:
data <- c(24, 17, 21, 23, 15, 30, 24, 21, 24, 19,
25, 28, 22, 20, 14, 19, 26, 29, 23, 25,
24, 18, 27, 21)
pop_var <- function(x) { var(x) * (1 - 1/length(x)) }
z <- (data - mean(data)) / sqrt(pop_var(data))
kurt <- mean(z^4)
kurt
# [1] 2.480035We have seen that measures of central tendency alone are not enough: two datasets can have the same mean yet behave in profoundly different ways. Variability tells us how much the data fluctuate, and the shape of the distribution tells us how they are arranged. In the next article, we will take the first steps into the world of probability—the essential bridge between descriptive statistics and the inferential tools that let us draw conclusions from data.
A Practical SEO Case: the Coefficient of Variation to Compare Two Sites
The standard deviation, on its own, can mislead when we compare quantities of different scale. Take two sites: a big one, around 5,000 sessions a day, and a small one, around 300. Let's measure the daily variability of each in R.
big <- c(4800, 5300, 4600, 5500, 5100, 4400, 5600, 4900, 5200, 4700)
small <- c(260, 340, 220, 380, 300, 240, 360, 280, 320, 300)
sd(big) # 395.7
sd(small) # 51.6
sd(big) / mean(big) # CV = 0.079 (7.9%)
sd(small) / mean(small) # CV = 0.172 (17.2%)In absolute terms the big site swings by almost 400 sessions a day, the small one by just 52: one would be tempted to call the big site far more “jumpy”. But that is an illusion of scale. The coefficient of variation — the standard deviation divided by the mean, a pure number read as a percentage — tells the opposite story: 7.9% for the big site, 17.2% for the small one. Relative to its own size, the small site is more than twice as unstable. This is exactly what the CV is for: it makes comparable variabilities that, in raw units, would not be.
The operational rule is clear: when we compare the variability of metrics on different scales — two sites, two channels, two periods with very different volumes — the coefficient of variation is the tool to use, not the bare standard deviation. The latter remains irreplaceable when the scale is the same.
Try It Yourself
Let's try a similar case. We have a week of daily clicks for two channels: organic, robust, and a niche referral. Which of the two is steadier, net of the difference in volume?
organic: 1200 1350 1100 1280 1420 1180 1320 referral: 80 140 60 170 95 130 110
The task: for each channel, compute the CV with sd(x) / mean(x).
To check the numbers: organic has a CV of 8.7%, the referral one of 33.5%. Although it brings ten times fewer clicks, the referral is nearly four times more volatile in relative terms: a channel on which to base forecasts is far riskier. The raw standard deviation (110 against 38) would have suggested the exact opposite.
Frequently asked questions
What are the main measures of dispersion?
The range, the variance, the standard deviation, the interquartile range and the coefficient of variation. Variance, standard deviation and coefficient of variation are based on the distance of the data from the mean; the interquartile range on the quartiles, which is why it is robust to outliers.
What is the difference between variance and standard deviation?
The standard deviation is the square root of the variance. They measure the same thing, but the variance is expressed in the square of the data's unit (sessions squared), the standard deviation in the same unit (sessions): that is why it is much easier to read and communicate.
Why does R divide by n − 1 and not by n?
Because var() and sd() estimate the variability of a population from a sample: dividing by n would systematically underestimate the true variance, and n − 1 corrects that bias. If the data are the whole population, you divide by n.
When should you use the coefficient of variation?
When comparing the variability of quantities on different scales: two sites with different volumes, two channels, metrics with different units. The CV is the standard deviation divided by the mean, a pure number read as a percentage. It loses meaning with means close to zero or with data that can be negative.
What does platykurtic mean?
A distribution flatter than the normal, with lighter tails: the kurtosis is below 3 (the excess kurtosis below 0). The opposite is a leptokurtic curve, more peaked and with heavier tails; the normal is mesokurtic.
Which measure of dispersion should you use when there are outliers?
The interquartile range, which measures the width of the middle half of the data and therefore ignores the tails. Variance, standard deviation and range are instead inflated even by a single extreme value, such as a traffic spike or a session left open by a bot.
Range, variance, standard deviation, interquartile range, coefficient of variation, kurtosis: we can now say not only where the centre of a distribution lies, but how tightly the data cluster around it and what shape they take. It is the minimum vocabulary to describe any set of data — and the springboard for the next step: modelling it with a probability distribution, starting with the most famous of all, the normal distribution, where the mean and the standard deviation become the only two numbers that say almost everything.
You might also like
- Descriptive Statistics: Measures of Position and Central Tendency
- The Normal Distribution
- Anomaly Detection: How to Identify Outliers in Your Data
Further Reading
If you want a gentle yet rigorous introduction to the concepts behind descriptive statistics—including variability, distributions, and the art of making sense of data—The Art of Statistics by David Spiegelhalter is an excellent starting point. Spiegelhalter has the rare ability to explain statistical concepts without dumbing them down, and his treatment of uncertainty and dispersion is particularly illuminating.