statistics

Non-Parametric Tests: The Wilcoxon Test for Non-Normal Data (with R Examples)

Imagine comparing session durations between two groups of pages: those with a featured snippet and those without. You run a Student’s t-test and get a p-value of 0.08. Not significant, apparently. But looking at the data, you notice the distributions are heavily skewed: a few pages with very long sessions, many with very short ones. The t-test assumes normality, and here we are light-years away.

The Wilcoxon test is a non-parametric test that makes no assumptions about the shape of the distribution. It works on ranks, not on original values: it orders all data from smallest to largest and assigns scores based on position. This makes it especially useful in SEO, where many metrics (sessions, CTR, rankings) are anything but normal.

When to Use Wilcoxon

Parametric tests like the t-test assume the data follows a normal distribution. In SEO, this assumption is almost always violated:

Sessions per page follow a skewed distribution (a few pages drive most of the traffic).
CTR is concentrated near zero with a long tail upward.
Rankings are ordinal variables, not continuous.
Page load times have a heavy right tail.

When data is non-normal, the t-test can give misleading results. The Wilcoxon test, being rank-based, is robust to these violations.

Two Versions of the Wilcoxon Test

The name “Wilcoxon test” actually covers two distinct variants:

Wilcoxon-Mann-Whitney (or U test): for comparing two independent samples. It is the non-parametric alternative to the two-sample t-test.
Wilcoxon signed-rank: for comparing two paired samples (before/after measurements on the same group). Alternative to the paired t-test.

Let’s look at an SEO example with the independent samples version.

Comparing Two Groups: An SEO Case

Suppose we want to compare session durations between pages with featured snippets and pages without. Real session data is typically skewed: most sessions last a few seconds, but some last minutes. This is exactly the scenario where Wilcoxon shines.

The figure below shows two simulated distributions similar to what we observe in real cases. The t-test gives a p-value that could be misleading, while Wilcoxon correctly captures the difference between groups.

Two skewed session duration distributions: the t-test may fail to detect the difference, Wilcoxon does not.

The reason Wilcoxon works better in this scenario lies in the ranking mechanism. Instead of comparing means (which are sensitive to extreme values), it transforms each value into its rank within the union of the two groups and compares the sums of ranks.

Raw values are sorted and converted to ranks: sums of ranks are compared, not original values.

After converting to ranks, we lose information about the distance between values, but we gain robustness: extreme values no longer distort the result.

Wilcoxon Test in R

Let’s compute Wilcoxon on the same data from the figure:

set.seed(20260718)
# Session duration in seconds, two groups
with_snippet <- rlnorm(40, meanlog = 4.8, sdlog = 0.7)
without_snippet <- rlnorm(40, meanlog = 5.2, sdlog = 0.7)

# Wilcoxon-Mann-Whitney test
wilcox.test(with_snippet, without_snippet)

The output returns the W statistic (the sum of ranks of the first group) and the p-value. If the p-value is below 0.05, we can conclude that the difference between the two groups is statistically significant.

For paired samples (same pages before and after an intervention), use the signed-rank variant:

# Same pages, before and after optimization
before <- rlnorm(30, meanlog = 4.5, sdlog = 0.6)
after <- before + rnorm(30, mean = 15, sd = 25)
# some values may be negative (decreased sessions)

# Wilcoxon for paired samples
wilcox.test(before, after, paired = TRUE)

A common mistake: thinking that Wilcoxon compares medians. It doesn’t. Wilcoxon tests whether the distributions of the two groups are stochastically shifted relative to each other, not whether their medians differ. In practice, if the two distributions have similar shapes, it behaves like a median test — but it’s not the same thing. A second mistake: using Wilcoxon even when the data is normal. In that case the t-test has more statistical power (it can detect smaller differences). The rule is: normal → t-test; non-normal → Wilcoxon.

Try It Yourself

Here are session data for two groups of pages (Group A: pages with optimized titles; Group B: pages without).

group_A <- c(45, 132, 28, 67, 312, 54, 89, 43, 156, 78,
             92, 34, 201, 67, 88, 43, 156, 67, 234, 55)
group_B <- c(67, 156, 89, 123, 445, 98, 134, 67, 189, 102,
             145, 56, 267, 89, 123, 78, 198, 89, 312, 78)

# Question 1: run the Wilcoxon test. What does the p-value suggest?
# Question 2: also run a t-test. Do the two tests agree?
# Question 3: what happens if you add an extreme outlier to group_A (value 5000)?

Solution:

wilcox.test(group_A, group_B)
t.test(group_A, group_B)

# With outlier
group_A_out <- c(group_A, 5000)
wilcox.test(group_A_out, group_B)
t.test(group_A_out, group_B)

You’ll notice that the t-test changes drastically with the outlier, while Wilcoxon remains much more stable: that’s the power of ranks.

Wilcoxon is the first line of defense when data is non-normal: it works with small samples, resists outliers, and in R it’s just one line of code. But it has a limitation: it only compares two groups. When we have more than two groups and the data is still non-normal, we need an extension — the non-parametric analysis of variance. We’ll cover that in the next article.