In the posts on multiple regression, analysis of variance and time series we came across three terms that seem made on purpose to scare the reader. Multicollinearity, heteroscedasticity, autocorrelation: if we want to convince someone that statistics is a dark and inaccessible subject, these are the magic words to say. 🙂
Surprise: behind the names there are three simple ideas, and above all three very concrete problems. To see them at work, let’s take the model that sooner or later anyone working in digital marketing builds: how many conversions does each euro spent on Google Ads and on Meta bring us? We’ll watch it fall into all three traps, one after the other, and learn how to recognise them and get out.
In short: these are three different problems of regression. Multicollinearity concerns the predictors: two or more explanatory variables are strongly correlated with each other, and the model can’t separate their effects (unstable coefficients, sometimes with the wrong sign). Heteroscedasticity concerns the errors: their spread is not constant, but grows or shrinks with the predicted values. Autocorrelation also concerns the errors, ordered in time: today’s error is linked to yesterday’s. In the last two cases the coefficients remain correct on average, but standard errors, p-values and confidence intervals become unreliable. In R they are diagnosed with the VIF, the Breusch-Pagan test and the Durbin-Watson test.
What we’ll cover:
Let’s imagine an e-commerce site that for two years has been investing every week in Google Ads and Meta. The business is growing, spend follows the seasons, and in the spreadsheet we have 104 rows: for each week the two budgets and the conversions obtained. The data are simulated, so we know the truth: both channels work, Google a little more than Meta.
Let’s generate the data in R:
library(lmtest) # Breusch-Pagan and Durbin-Watson tests
library(sandwich) # robust standard errors
simulate <- function() {
week <- 1:104
growth <- seq(5000, 15000, length.out = 104) # the business grows
season <- 1 + 0.35 * sin(2 * pi * week / 52) # high and low season
google <- round(growth * season * exp(rnorm(104, 0, 0.10)))
meta <- round(0.45 * google * exp(rnorm(104, 0, 0.08))) # follows Google
error <- as.numeric(arima.sim(list(ar = 0.6), 104, sd = 0.12))
conversions <- round(exp(0.6 + 0.40 * log(google) + 0.20 * log(meta) + error))
data.frame(week, google, meta, conversions)
}
set.seed(114)
data <- simulate() The three traps are already written into these lines, even if we can’t see them yet: Meta’s budget is set as a share of Google’s, the error is multiplicative (it grows with the size of the week) and it follows a pattern that drags on from one week to the next. Now let’s do what anyone would do, a regression of conversions on the two kinds of spend:
model <- lm(conversions ~ google + meta, data = data)
summary(model)
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 179.480922 19.529960 9.190 5.45e-15 ***
# google 0.030642 0.008473 3.616 0.000468 ***
# meta -0.024130 0.018618 -1.296 0.197920
#
# Multiple R-squared: 0.5323, Adjusted R-squared: 0.5231
# F-statistic: 57.48 on 2 and 101 DF, p-value: < 2.2e-16 Let’s read it the way a marketing manager would: every €1,000 on Google brings about 31 conversions. And every €1,000 on Meta takes away 24. Obvious conclusion: switch Meta off.
It would be a costly mistake, because in the data Meta works. The model as a whole explains half of the variability in conversions (R² 0.53, F test with a tiny p-value), but the split between the two channels is an illusion. Welcome to multicollinearity.
We speak of multicollinearity when two or more explanatory variables in a regression model are strongly correlated with each other. The model knows that together they matter, but it can’t establish how much each one does.
The reason is intuitive. To estimate the effect of Meta “holding Google fixed”, the regression needs weeks in which Meta varies and Google doesn’t. But in our data, when Google goes up Meta goes up too: the correlation between the two kinds of spend is 0.975. The weeks that help tell the two effects apart are very few, and the estimate rests on those few random wobbles.
The standard tool to measure the problem is the VIF, the variance inflation factor. It’s computed with an auxiliary regression: we explain one predictor with all the others and see how well we manage. If the R² of this regression is high, the predictor is almost a copy of the others:
\(VIF = \frac{1}{1 – R^2_{\text{auxiliary}}}\)
The name says exactly what it measures: how many times the variance of the coefficient is inflated compared with the case of independent predictors. Let’s compute it:
r2_aux <- summary(lm(google ~ meta, data = data))$r.squared
r2_aux
# [1] 0.9511464
1 / (1 - r2_aux)
# [1] 20.46933 The car package computes it directly with vif(model). The most widespread rule of thumb considers a VIF above 5 worrying and above 10 serious. Ours is 20.5: the variance of the estimates is twenty times what we’d have with two independent channels, and the standard error (its square root) more than four times.
What does that mean in practice? We see it by repeating the same experiment 500 times: same rules, same true effects, just a different random draw.
set.seed(2026)
estimates <- t(replicate(500, {
d <- simulate()
1000 * c(coef(lm(conversions ~ google + meta, data = d))[2:3],
total = coef(lm(conversions ~ I(google + meta), data = d))[[2]])
}))
mean(estimates[, "meta"] < 0)
# [1] 0.146
cor(estimates[, "google"], estimates[, "meta"])
# [1] -0.9068232 The two estimates compete for the same effect: when one goes up, the other goes down (correlation −0.91 between the estimates). In one case in seven Meta even comes out negative, and our dataset is one of those. And yet the sum of the two effects is stable: total spend has a standard deviation of 2.5 conversions per €1,000, against 8.7 for Google and 17.7 for Meta.
Here is the hallmark of multicollinearity: the model as a whole works, the predictions are good, but the individual coefficients are not credible. It often shows up exactly like this: a fair or high R², a significant F test, and coefficients with huge standard errors or signs that defy common sense.
There’s no test that “cures” multicollinearity, because it isn’t a flaw in the calculation: it’s a lack of information in the data. There are four routes.
lm(conversions ~ I(google + meta)) we get 13.6 conversions per €1,000, with a standard error of 1.3: a solid estimate.From here on we follow the first route and work with total spend:
data$spend <- data$google + data$meta
model_tot <- lm(conversions ~ spend, data = data) Well, this term seems made on purpose to scare. But the concept, it turns out, isn’t that complicated.
Heteroscedasticity basically means unequal spread: it happens when the variance of the errors of a regression model is not constant, but changes with the predicted values or with an explanatory variable. The opposite case, the one ordinary least squares assumes, is called homoscedasticity: errors with the same spread at every level.
In marketing, heteroscedasticity is almost the rule. A 300-conversion week can be off by 30 either way; a 600-conversion week, other things equal, is off by 60. Errors grow with size, because effects are proportional rather than additive. The simplest way to notice it is to look at the residuals against the predicted values: if they fan out, the variance isn’t constant.
For a formal check we use the Breusch-Pagan test, which checks whether the squared residuals can be explained by the predictors. The null hypothesis is homoscedasticity:
bptest(model_tot)
# BP = 9.3544, df = 1, p-value = 0.002224 With a p-value of 0.002, the constant-variance hypothesis doesn’t hold.
What happens if we ignore it? Here there’s good news that textbooks often leave in the shade: the coefficients remain correct on average. What breaks is the estimate of their precision: standard errors computed with the classical formula are wrong, and so are p-values and confidence intervals. In our case the robust standard errors, which don’t assume constant variance, are 20% larger (from 0.00131 to 0.00157):
coeftest(model_tot, vcov = vcovHC(model_tot, type = "HC3"))
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 1.7769e+02 2.0789e+01 8.5472 1.310e-13 ***
# spend 1.3622e-02 1.5723e-03 8.6638 7.273e-14 *** How do we fix it? There are three routes, often combined:
In our case the second route is also the most natural, because the error is multiplicative by construction. Let’s move to logarithms:
model_log <- lm(log(conversions) ~ log(spend), data = data)
coef(summary(model_log))
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 0.2257489 0.45602741 0.4950336 6.216411e-01
# log(spend) 0.5953304 0.04792515 12.4220879 3.942384e-22
bptest(model_log)
# BP = 1.1777, df = 1, p-value = 0.2778 The fan closes, and the test finds nothing anymore. On top of that, the coefficient has gained a very handy interpretation: in a model with logarithms on both sides it’s an elasticity. The value of 0.595 says that 10% more spend brings about 5.8% more conversions (1.1 raised to 0.595 is 1.058). In the simulation the true value is 0.6: the model got very close.
The last trap remains, the sneakiest one when data are ordered in time.
We have autocorrelation (or serial correlation) when the errors of a regression model are correlated with each other over time: the error of one period says something about the error of the next one. In the most common case, first-order positive autocorrelation, a week that went better than expected tends to be followed by another week that went better than expected.
In marketing it happens all the time: a competitor’s campaign, a period of favourable prices, a Google algorithm update, a good review going round. They’re all things the model doesn’t see and that last more than a week. The errors, then, don’t jump randomly above and below zero: they stay on the same side for weeks on end. The plot of residuals in time order shows it at a glance:
The classical test is Durbin-Watson. Its statistic d is about 2 with no autocorrelation, drops towards 0 with positive autocorrelation and rises towards 4 with negative autocorrelation. In R we no longer need the tables: the test returns the p-value directly.
dwtest(model_log)
# DW = 0.81187, p-value = 2.573e-10
acf(resid(model_log), plot = FALSE)$acf[2]
# [1] 0.5881333 A statistic of 0.81 and a residual autocorrelation of 0.59 between one week and the next: the problem is there, and it’s strong. For longer series or higher-order autocorrelation (between an error and the one two or more periods earlier) we also use the acf() function and the Ljung-Box test, which we met when talking about time series.
What happens if we ignore it? As with heteroscedasticity, the estimates remain correct on average, but the classical standard-error formula assumes that each week brings independent information. With positive autocorrelation that isn’t so: 104 weeks that resemble each other in groups are worth less than 104 independent observations. The model believes it’s more precise than it is. Newey-West standard errors, which account for both autocorrelation and heteroscedasticity, show it right away:
coeftest(model_log, vcov = NeweyWest(model_log))
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 0.22575 1.00072 0.2256 0.822
# log(spend) 0.59533 0.10624 5.6039 1.791e-07 *** The standard error of the elasticity goes from 0.048 to 0.106, more than double. The 95% confidence interval, which with the classical formula ran from 0.50 to 0.69, widens to 0.38 to 0.81. The effect of advertising remains clearly positive, but our uncertainty about how much it’s worth is more than double what we thought. With a smaller effect, the difference between the two formulas would be the one between a “significant” result and one that isn’t.
How do we fix it?
gls() from the nlme package, with correlation = corAR1()). They are the modern descendants of Durbin’s two-step method and of the Cochrane-Orcutt procedure.At this point we can put the three concepts side by side. The most important distinction is where the problem lies: multicollinearity concerns the explanatory variables, the other two concern the errors.
| Multicollinearity | Heteroscedasticity | Autocorrelation | |
|---|---|---|---|
| Where the problem lies | in the predictors, correlated with each other | in the errors, with non-constant variance | in the errors, correlated over time |
| Coefficients | correct on average, but very unstable | correct on average | correct on average |
| Standard errors | large (inflated) | wrong | usually too small |
| How to spot it | VIF above 5-10 | fan-shaped residuals, Breusch-Pagan test | residuals in runs, Durbin-Watson test |
| Typical remedies | aggregate spend, experiments, ridge | logarithms, robust standard errors | Newey-West, omitted variables, gls() |
There’s a subtle but important difference between the first and the other two. With multicollinearity the standard errors are large and they know it: the model warns us, in its own way, that it can’t separate the effects. With heteroscedasticity and autocorrelation, instead, classical standard errors are wrong silently, and often in the worst direction: they make us look more confident than we are. That’s why diagnostic tests aren’t an appendix for nitpickers, but a check to run every time a regression has to guide a decision, from an advertising ROI model to a forecast of organic traffic.
Multicollinearity concerns the explanatory variables, correlated with each other; autocorrelation concerns the model’s errors, correlated over time. The first makes the coefficients unstable and is measured with the VIF; the second makes the standard errors too optimistic and is checked with the Durbin-Watson test.
Heteroscedasticity concerns the errors, whose variance is not constant; multicollinearity concerns the predictors, which are strongly correlated. With heteroscedasticity the coefficients are fine but the classical standard errors are wrong; with multicollinearity the standard errors are honestly large because the data can’t separate the effects.
Both concern the model’s errors and leave the coefficients correct on average. Heteroscedasticity is about their spread, which changes from one observation to another; autocorrelation is about their link over time. They can occur together, and Newey-West standard errors account for both.
For multicollinearity, avoid moving all channels together, or use experiments that vary one channel at a time; otherwise model total spend. For heteroscedasticity, use logarithms when effects are proportional, or robust standard errors. For autocorrelation in weekly data, add seasonality and trend to the model and use Newey-West standard errors.
There is no universal threshold. The most widespread rule of thumb considers a VIF above 5 worth watching and above 10 serious. A VIF of 10 means the variance of the coefficient is ten times what it would be with independent predictors.
About 2 with no autocorrelation. Values close to 0 indicate positive autocorrelation, values close to 4 negative autocorrelation. In R, dwtest() from the lmtest package also returns the p-value, with no need for tables.
The article’s code lends itself to a few experiments.
simulate() function, raise the standard deviation of Meta’s noise from 0.08 to 0.40, so the two budgets move more independently. What happens to the VIF? And to the share of negative Meta estimates in the 500 repetitions?ar = 0 in arima.sim(): does the Durbin-Watson statistic go back close to 2? And do the Newey-West standard errors get close to the classical ones?lm(log(conversions) ~ log(google) + log(meta)): does multicollinearity disappear with logarithms?The last exercise has an instructive answer: transformations cure heteroscedasticity, but not multicollinearity. If the two budgets move together, so do their logarithms.
With this we have the tools to trust (or not trust) a regression. It’s time to put them to work on a very concrete SEO problem: building a model of the CTR we expect at each position, and using it to find the pages that perform worse than they should. That’s the topic of expected vs actual CTR, the next stop on our journey.
The three problems we discussed are covered in depth in Introduzione all’econometria by Stock and Watson (Italian edition), the book I recommend: it devotes very clear pages to heteroscedasticity-robust standard errors and to Newey-West errors for time series, always starting from concrete economic examples.
The September report is almost ready, and there is one number that does not fit:…
The redesign is approved, the launch date is set, and sooner or later the question…
The annual report lands on the desk with good news on the front page: the…
A site's technical guides climb in the search results, show up more often, and yet…
Clicks and sessions vary more than Poisson allows: how to detect it in R and…
An annual report often ends with a single line: clicks from Google fell 11.8% compared…