The term “analysis of variance” sounds counterintuitive: we are comparing means, not variances. The name comes from the way the test uses variances to determine whether the means differ. ANOVA compares the variance of the means between<\/em> groups with the variance within<\/em> each group. If the former is sufficiently larger than the latter, we have evidence that the groups are not equivalent.<\/p>\n\n\n\n Imagine three school classes. In each class the students have fairly similar grades (low within-group variance), but the class averages are very different (high between-group variance). In this case it is easy to conclude that the classes are genuinely different.<\/p>\n\n\n\n If instead the grades within each class vary wildly \u2014 brilliant students next to struggling ones \u2014 distinguishing one class from another becomes much harder, even if the averages look different. The within-group variability “drowns out” the differences between groups.<\/p>\n\n\n\n This is what ANOVA is all about: a ratio between the variability between<\/em> groups and the variability within<\/em> groups. When the former clearly dominates the latter, the F value grows and we have evidence against the null hypothesis.<\/p>\n\n\n\n ANOVA is a parametric test. It requires a few conditions:<\/p>\n\n\n\n A natural question arises: why use ANOVA when we could simply compare every pair of groups with a t-test?<\/p>\n\n\n\n The problem is not the inconvenience of running many tests. The problem is that each test has a probability \u03b1 of producing a false positive<\/strong> (Type I error). When we run many independent tests, these probabilities accumulate.<\/p>\n\n\n\n For k groups, the number of pairwise comparisons is:<\/p>\n\n\n\n\\( n = \\frac{k(k-1)}{2} \\\\ \\)\n\n\n\n With 4 groups that would be 6 t-tests. The probability of committing at least one<\/em> Type I error becomes:<\/p>\n\n\n\n\\( 1 – (1 – \\alpha)^n \\\\ \\)\n\n\n\n With \u03b1 = 0.05 and n = 6:<\/p>\n\n\n\n\\( 1 – (1 – 0.05)^6 = 1 – 0.735 = 0.265 \\\\ \\)\n\n\n\n That is 26.5%: more than one in four. Unacceptable.<\/p>\n\n\n\n When comparing three or more groups, ANOVA is preferable<\/strong> because it runs a single test with a single \u03b1, controlling the overall error rate.<\/p>\n\n\n\n One-way ANOVA is the simplest form. The data are organised into groups based on a single categorical variable (the factor). The question is:<\/p>\n\n\n\n > Are the observed differences between the means too large to be attributed to chance?<\/p>\n\n\n\n Let us formalise the hypotheses:<\/p>\n\n\n\n\\( H_0: \\mu_1 = \\mu_2 = \\dots = \\mu_k \\\\ \\)\n\n\n\n\\( H_a: \\text{at least one mean differs from the others} \\\\ \\)\n\n\n\n Important<\/strong>: ANOVA tells us THAT a difference exists, not WHERE it is. To identify which groups differ, we need a post-hoc<\/em> test, which we will see shortly.<\/p>\n<\/div><\/div>\n\n\n\n The core of ANOVA is the F ratio:<\/p>\n\n\n\n\\( F = \\frac{\\text{variance between groups}}{\\text{variance within groups}} = \\frac{MS_b}{MS_e} \\\\ \\)\n\n\n\n If the null hypothesis is true, both \\( MS_b \\) and \\( MS_e \\) estimate the same variance, and F is close to 1. If F is significantly greater than 1, we have evidence that the means are not all equal.<\/p>\n\n\n\n ANOVA was developed by Ronald Fisher<\/strong> in the 1920s. The F statistic it relies on is named in his honour as the Fisher-Snedecor F distribution<\/strong>.<\/p>\n<\/div><\/div>\n\n\n\n Let us take a concrete case. We want to know whether the type of device (desktop, mobile, tablet) affects goal completions on the site. We track monthly data for 6 months:<\/p>\n\n\n\n Group means:<\/p>\n\n\n\n\\( \\bar{x}_{\\text{desktop}} = 56 \\quad \\bar{x}_{\\text{mobile}} = 49.3 \\quad \\bar{x}_{\\text{tablet}} = 35.8 \\\\ \\)\n\n\n\n And the grand mean<\/strong> (mean of all observations): \\( \\bar{x} = 47 \\).<\/p>\n\n\n\n To understand where the F statistic comes from, we need to decompose the total variability into two parts: the variability between<\/em> groups and the variability within<\/em> groups. Without going through every manual calculation \u2014 nobody does them by hand anymore, as we will see shortly \u2014 from these we derive the respective variances (Mean Squares) and finally the F ratio.<\/p>\n\n\n\n F = 4.86 exceeds the critical value (3.68 for df = 2,15 and \u03b1 = 0.05). With a p-value of 0.0236<\/strong>, we reject the null hypothesis: at least one device has a different mean.<\/p>\n\n\n\n The p-value in ANOVA is read like any hypothesis test: it is the probability of observing an F value equal to or more extreme than the one obtained, assuming H\u2080 is true. A small p-value (typically < 0.05) tells us that the observed differences are unlikely to be due to chance.<\/p>\n\n\n\n But note: statistical significance does not mean practical importance<\/strong>. A very small p-value can hide a negligible effect if the sample is large. That is why we need effect size.<\/p>\n\n\n\n The p-value tells us whether<\/em> a difference exists. Eta-squared tells us how large<\/em> it is.<\/p>\n\n\n\n\\( \\eta^2 = \\frac{SS_b}{SS_{\\text{total}}} = \\frac{1266.4}{3220.6} \\approx 0.39 \\\\ \\)\n\n\n\n A very small p-value tells us that a difference probably exists. But it does not tell us whether that difference is large or negligible. That is why it is good practice to always accompany ANOVA with a measure of effect size.<\/p>\n<\/div><\/div>\n\n\n\n Rules of thumb for \u03b7\u00b2:<\/p>\n\n\n\n Our \u03b7\u00b2 = 0.39 is a very large effect: the type of device explains 39% of the variability in goal completions.<\/p>\n\n\n\n ANOVA tells us that a difference exists, but not which groups differ. For that we need post-hoc<\/strong> tests.<\/p>\n\n\n\n The most common is Tukey’s HSD<\/strong> (Honestly Significant Difference), which compares all pairs of means while correcting for multiple comparisons.<\/p>\n\n\n\n In our example, Tukey HSD would tell us that the significant difference is between tablet and desktop, while the other pairs do not reach significance.<\/p>\n\n\n\n Let us put ANOVA to work on a question anyone looking at site data eventually asks: do users coming from different channels behave differently?<\/p>\n\n\n\n We have time on page (in seconds) for 8 sessions from each of four channels:<\/p>\n\n\n\n Compute in R:<\/p>\n\n\n\n F = 46.82 and the p-value is on the order of 10\u207b\u00b9\u00b9: well below any reasonable threshold. We reject the null hypothesis: at least one channel<\/strong> behaves differently.<\/p>\n\n\n\n Let us look at the means:<\/p>\n\n\n\n Organic is the most engaged channel (100 seconds), followed by direct (93). Social stops at 52 seconds \u2014 traffic from feeds tends to bounce quickly.<\/p>\n\n\n\n Let us calculate \u03b7\u00b2 to understand the effect size:<\/p>\n\n\n\n 0.83<\/strong>: the channel explains 83% of the variability in time on page. A huge effect. In real-world settings such high values are quite rare: this example is deliberately constructed to make the method’s workings evident.<\/p>\n\n\n\n The post-hoc test confirms that all differences involving paid and social versus organic and direct are significant. The difference between organic and direct, however, is not statistically significant.<\/p>\n\n\n\n Before trusting the results, it is wise to verify the assumptions:<\/p>\n\n\n\n Normality of residuals:<\/strong><\/p>\n\n\n\n Homogeneity of variances (homoscedasticity):<\/strong><\/p>\n\n\n\n If these assumptions are violated, non-parametric alternatives such as the Kruskal-Wallis test<\/strong> can be used.<\/p>\n\n\n\n Nobody computes an ANOVA table by hand anymore. Programs like R, Python (scipy.stats), SPSS, or Jamovi perform the calculation in seconds. Understanding what the different components represent \u2014 the decomposition of variance, the F ratio, the degrees of freedom \u2014 remains essential for correctly interpreting the result.<\/p>\n\n\n\n An e-commerce store ran three campaigns (A, B, C) and wants to know whether the average order value differs significantly.<\/p>\n\n\n\n Run the test, read F and p-value, calculate \u03b7\u00b2.<\/p>\n\n\n\n F = 10.77<\/strong>, p = 0.0006<\/strong>, \u03b7\u00b2 = 0.44<\/strong>. Campaign B (mean \u20ac50.5) performs better than A (\u20ac43.2) and C (\u20ac44.4), and the difference is statistically significant. Tukey HSD would confirm this by pointing at B.<\/p>\n\n\n\n Does ANOVA work with only two groups?<\/strong> What does “one-way” and “two-way” mean?<\/strong> Can I use ANOVA with small samples?<\/strong> ANOVA is one of those ideas that, once understood, you see everywhere: whenever we compare a continuous metric across multiple groups \u2014 channels, devices, variants, time periods \u2014 it is the right tool for not confusing noise with signal. And it is not an island: under the hood, analysis of variance and linear regression are the same model, seen from two different angles. It is from that model \u2014 the linear model \u2014 that the most powerful part of applied statistics flows, and that is where our journey continues.<\/p>\n\n\n\n Analysis of variance in all its forms \u2014 one-way, two-way, repeated measures \u2014 is covered in Statistics<\/em> by Newbold, Carlson and Thorne, for those who want to go beyond the introduction in this article. Amazon<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" You have three Google Ads campaigns. Or four landing pages. Or five versions of a newsletter. The averages seem different, but are these real differences or just statistical noise? This is exactly the problem that Analysis of Variance (ANOVA) solves: determining whether at least one group differs genuinely from the others, without falling into the … Leggi tutto “Analysis of Variance: ANOVA Explained Simply”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","footnotes":""},"categories":[161],"tags":[1228,287,1226],"class_list":["post-3303","post","type-post","status-publish","format-standard","hentry","category-statistics","tag-analysis-of-variance","tag-anova","tag-parametric-test"],"lang":"en","translations":{"en":3303,"it":2342},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false,"post-thumbnail":false},"uagb_author_info":{"display_name":"paolo","author_link":"https:\/\/www.gironi.it\/blog\/author\/paolo\/"},"uagb_comment_info":199,"uagb_excerpt":"You have three Google Ads campaigns. Or four landing pages. Or five versions of a newsletter. The averages seem different, but are these real differences or just statistical noise? This is exactly the problem that Analysis of Variance (ANOVA) solves: determining whether at least one group differs genuinely from the others, without falling into the…","_links":{"self":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3303","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/comments?post=3303"}],"version-history":[{"count":5,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3303\/revisions"}],"predecessor-version":[{"id":4200,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3303\/revisions\/4200"}],"wp:attachment":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/media?parent=3303"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/categories?post=3303"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/tags?post=3303"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}The Intuition in a Metaphor<\/h2>\n\n\n\n
When to Use ANOVA<\/h2>\n\n\n\n
Why ANOVA Instead of Multiple t-Tests?<\/h2>\n\n\n\n
The Simplest Case: One-Way ANOVA<\/h2>\n\n\n\n
How to Interpret the F Value<\/h2>\n\n\n\n
![\"F](\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/f-distribution.png\")
A Step-by-Step Example<\/h2>\n\n\n\n
Desktop<\/th> Mobile<\/th> Tablet<\/th><\/tr><\/thead> 39<\/td> 45<\/td> 30<\/td><\/tr> 67<\/td> 54<\/td> 45<\/td><\/tr> 78<\/td> 64<\/td> 22<\/td><\/tr> 59<\/td> 52<\/td> 39<\/td><\/tr> 42<\/td> 46<\/td> 38<\/td><\/tr> 51<\/td> 35<\/td> 41<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n Source<\/th> SS<\/th> df<\/th> MS<\/th> F<\/th> p<\/th><\/tr><\/thead> Device (between)<\/td> 1266.4<\/td> 2<\/td> 633.2<\/td> 4.86<\/strong><\/td> 0.0236<\/strong><\/td><\/tr> Residual (within)<\/td> 1954.2<\/td> 15<\/td> 130.3<\/td> <\/td> <\/td><\/tr> Total<\/td> 3220.6<\/td> 17<\/td> <\/td> <\/td> <\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n How to Interpret the p-Value<\/h2>\n\n\n\n
Effect Size: \u03b7\u00b2 (Eta-Squared)<\/h2>\n\n\n\n
What to Do After ANOVA: Post-Hoc Tests<\/h2>\n\n\n\n
A Practical SEO Case: Does Time on Page Depend on Channel?<\/h2>\n\n\n\n
organic <- c(98, 112, 85, 105, 92, 120, 88, 101)\ndirect <- c(90, 102, 78, 95, 110, 84, 99, 88)\npaid <- c(72, 65, 80, 58, 70, 75, 62, 68)\nsocial <- c(55, 48, 62, 50, 44, 58, 52, 46)\n\ntime <- c(organic, direct, paid, social)\nchannel <- factor(rep(c(\"organic\",\"direct\",\"paid\",\"social\"), each = 8))<\/code><\/pre>\n\n\n\nsummary(aov(time ~ channel))<\/code><\/pre>\n\n\n\n Df Sum Sq Mean Sq F value Pr(>F)\nchannel 3 11913 3971 46.82 4.89e-11 ***\nResiduals 28 2375 85<\/code><\/pre>\n\n\n\ntapply(time, channel, mean)<\/code><\/pre>\n\n\n\n direct organic paid social\n 93.2 100.1 68.8 51.9<\/code><\/pre>\n\n\n\n11913 \/ (11913 + 2375) # \u2248 0.83<\/code><\/pre>\n\n\n\nTukeyHSD(aov(time ~ channel))<\/code><\/pre>\n\n\n\n diff lwr upr p adj\ndirect-organic -6.875 -24.423 10.6730 0.7165876\npaid-organic -31.250 -48.798 -13.7020 0.0003217\nsocial-organic -48.250 -65.798 -30.7020 0.0000001\npaid-direct -24.375 -41.923 -6.8270 0.0041674\nsocial-direct -41.375 -58.923 -23.8270 0.0000073\nsocial-paid -17.000 -34.548 0.5480 0.0598936<\/code><\/pre>\n\n\n\nChecking ANOVA Assumptions<\/h2>\n\n\n\n
model <- aov(time ~ channel)\nshapiro.test(residuals(model))<\/code><\/pre>\n\n\n\nbartlett.test(time ~ channel)<\/code><\/pre>\n\n\n\nIn Everyday Practice<\/h2>\n\n\n\n
Try It Yourself<\/h2>\n\n\n\n
camp_A <- c(42, 38, 45, 50, 41, 47, 39, 44)\ncamp_B <- c(48, 52, 46, 55, 50, 53, 49, 51)\ncamp_C <- c(44, 40, 49, 43, 46, 41, 45, 47)\n\nvalue <- c(camp_A, camp_B, camp_C)\ncampaign <- factor(rep(c(\"A\",\"B\",\"C\"), each = 8))\nsummary(aov(value ~ campaign))<\/code><\/pre>\n\n\n\nFAQ<\/h2>\n\n\n\n
Yes, but in that case it is equivalent to a two-tailed t-test (and F = t\u00b2).<\/p>\n\n\n\n
One-way: a single factor (e.g., channel). Two-way: two factors simultaneously (e.g., channel \u00d7 device), which also allows studying their interaction.<\/p>\n\n\n\n
Yes, but check normality and homoscedasticity. With unequal group sizes, ANOVA is still robust if the deviations are not extreme.<\/p>\n\n\n\n
\n\n\n\n
\n\n\n\nFurther Reading<\/h3>\n\n\n\n