The month the form stopped counting: finding and reconstructing tracking gaps

The September report is almost ready, and there is one number that does not fit: in August the contact form collected 11 requests. In July there had been 82, in June 98.
The first explanation is close at hand: it is August, companies are closed, people are at the beach. And indeed the year before, too, August had been the worst month. The report goes out, with a sentence about the summer dip. Then, months later, somebody notices that the form submission tag had been broken by a theme update, and that almost all of those requests had arrived: they simply had not been counted.

Every analyst, sooner or later, meets a tracking gap: a period in which an event stopped being recorded, because a selector changed, a tag was paused, a plugin was updated, a consent banner was reconfigured. The real damage is not the gap itself but the time that passes before we notice it. In this article we see how to find one with statistical reasoning rather than by chance, how to reconstruct the missing data honestly, and when instead it is better not to reconstruct it at all.

In short: a tracking gap is recognised by comparing the observed with the expected: if the recorded number is too improbable against a reasonable reference, something has to be checked. The strongest clue comes from a twin event, a linked event that meanwhile kept being recorded (the opening of the form next to its submission, for example): if the twin is normal and the ratio between the two collapses, the problem lies in the submission or in its measurement, and to tell the two apart we check the form, the notification emails, the CRM and the technical changes of the period. If the failure lies only in the tracking, the missing data can be reconstructed by applying to the twin the ratio estimated on clean periods, with a prediction interval. Reconstructed data, though, is not measured data: it has to be marked as such and documented. The best remedy remains prevention: an automatic weekly check that compares the observed with the expected, and that in our example sounds the alarm already in the first week.

What we will cover:


The case: summer closing or a broken tag?

Let us build an example with entirely simulated data. A B2B site with a contact form, two years of daily data, from January 2024 to December 2025, and two recorded events:

  • the opens of the form, that is, the times someone starts filling it in: about twenty a day on weekdays, fewer at the weekend, with a little seasonality, a slow August (almost half of the other months) and slight growth over the two years;
  • submissions, that is, forms actually sent: about one every six opens.

From 28 July to 24 August 2025, four weeks exactly, the submission tag stops working. Opens continue to be recorded, submissions do not.

I build the data in R:

set.seed(2026)
day <- seq.Date(as.Date("2024-01-01"), as.Date("2025-12-31"), by = "day")
n <- length(day)

weekday  <- ifelse(format(day, "%u") %in% c("6", "7"), 0.4, 1)
doy      <- as.numeric(format(day, "%j"))
seasonal <- 1 + 0.15 * cos(2 * pi * (doy - 60) / 365)
august   <- ifelse(format(day, "%m") == "08", 0.55, 1)
growth   <- 1 + 0.10 * (seq_len(n) - 1) / n

rate             <- 22 * weekday * seasonal * august * growth
opens            <- rnbinom(n, mu = rate, size = 15)
true_submissions <- rbinom(n, opens, 0.17)

# the gap: the tag records only 4% of submissions
gap  <- day >= as.Date("2025-07-28") & day <= as.Date("2025-08-24")
submissions <- true_submissions
submissions[gap] <- rbinom(sum(gap), true_submissions[gap], 0.04)

data <- data.frame(day, opens, submissions, true_submissions, gap)
data$month <- format(day, "%Y-%m")
monthly <- aggregate(cbind(opens, submissions, true_submissions) ~ month, data, sum)

The true_submissions column is the luxury of a simulation: in reality we do not have it, and we will use it only at the end, to see how good the reconstruction is. Let us look at what whoever prepares the report would see, that is, recorded submissions month by month.

Recorded form submissions month by month, from January 2024 to December 2025. Almost every month sits between 70 and 140 submissions; the two Augusts are the lowest, 37 in 2024 and 11 in 2025.
Recorded form submissions month by month, from January 2024 to December 2025. Almost every month sits between 70 and 140 submissions; the two Augusts are the lowest, 37 in 2024 and 11 in 2025.

The chart is exactly the reason these gaps go unnoticed. August is low every year, and an August lower than usual does not set off any alarm: it looks like a slightly stronger version of something we already know. The right question is not “is August low?” but “is it as low as we would expect?”.


Finding it: expected against observed

To know how much we expect, we need a forecast, even a rough one. The simplest takes the August of the previous year and corrects it for the site’s growth, measured over a clean period: March to June.

I compute the expected value in R:

ref25 <- sum(monthly$submissions[monthly$month %in% c("2025-03", "2025-04", "2025-05", "2025-06")])
ref24 <- sum(monthly$submissions[monthly$month %in% c("2024-03", "2024-04", "2024-05", "2024-06")])
growth_year <- ref25 / ref24
expected <- monthly$submissions[monthly$month == "2024-08"] * growth_year
observed <- monthly$submissions[monthly$month == "2025-08"]

From March to June the site went from 392 to 444 submissions, growth of 13.3%. August 2024 had 37, so for August 2025 we expect about 41.9. We observed 11.

How improbable is that? If submissions followed a Poisson distribution with a mean of 41.9, the probability of seeing 11 or fewer would be the cumulative distribution function evaluated at 11.

I compute the probability in R:

ppois(observed, expected)

The result is about 1.5 in a hundred million. But caution is needed here, for two reasons. The first is that the expected value, 41.9, is itself an estimate, built on counts that have their own variability, and the calculation treats it as if it were known. The second is that real counts almost always vary more than the Poisson predicts, which is what we have called overdispersion. Let us redo the calculation with a negative binomial whose variance is five times its mean, a generous assumption about variability (but an assumption we chose all the same).

I repeat the calculation in R:

pnbinom(observed, mu = expected, size = expected / 4)

The probability rises to 0.0026, roughly one in four hundred. Much higher, but still small. Neither calculation is proof: they are exploratory checks, valid under the assumptions we made. But they say the same thing: against this reference, the data are anomalous and have to be checked. The calculation tells us there is something to understand, but not what it is. It could be a real decline (an aggressive competitor, a problem on the site, a page that vanished from Google) or a gap in the measurement. To tell the two apart we need another witness.


The litmus test: the twin event

A tracking gap usually hits one event only, or a few. Neighbouring events, the ones sitting on the same path of the user but recorded in another way, keep counting. In our case the twin event is the opening of the form: if people had really stopped trying to contact the company, they would have stopped opening the form too.

August 2025 opens are 280, against 263 in August 2024. No decline, in fact a small increase, in line with the rest of the year. People kept opening the form as always: it is the submissions that vanished.

The clearest way to see it is to look at the ratio between the two events, week by week. A ratio has a big advantage: if seasonality, weekends and the site’s growth act in the same proportion on opens and submissions, they cancel out in the ratio, and what is left is the behaviour of the form. In the simulation this holds by construction; in real data it holds only partly (in August, for instance, the type of visitor may change, and with it the willingness to submit), but usually enough to make the ratio far more stable than the two counts taken on their own.

I compute the weekly ratio in R:

data$week <- as.Date(cut(data$day, "week"))
weekly <- aggregate(cbind(opens, submissions) ~ week, data, sum)
weekly$ratio <- weekly$submissions / weekly$opens
Top: form opens and recorded submissions by week, from 3 March to 30 November 2025; bottom: the ratio between submissions and opens. The ratio mostly oscillates between 0.12 and 0.21, with a few lower weeks, except four weeks, from 28 July to 24 August, when it collapses to zero while opens stay normal.
Top: form opens and recorded submissions by week, from 3 March to 30 November 2025; bottom: the ratio between submissions and opens. The ratio mostly oscillates between 0.12 and 0.21, with a few lower weeks, except four weeks, from 28 July to 24 August, when it collapses to zero while opens stay normal.

In normal weeks the ratio mostly oscillates between 0.12 and 0.21: about one submission every six opens. A few weeks go lower (in early November it drops to 0.09, for pure chance, since the simulation changes nothing in that period), and that is a good reminder of how much a weekly ratio can swing. But then there are four weeks in a row when it is zero. Not 0.10, not 0.05: zero submissions out of 60–70 opens a week. It is hard to imagine a season or a change in people’s behaviour that takes a ratio to zero from one Monday to the next and brings it back up at once four weeks later, while for a technical failure this is the typical signature. A drop this sharp does not prove the failure on its own, but it makes checking it the first thing to do. It remains to understand which failure: a tag that does not record submissions, or a form that cannot send? The two things are very different, because in the first case the requests arrived and in the second they did not. The notification emails, the CRM, a submission tried by hand will tell us: if the requests were arriving, the failure was only in the measurement. In our example we assume we have verified that this is the case.

The twin also gives us the dates. Looking at the days, the last recorded submission is on Sunday 27 July, and from Monday 28 July there are days with 11, 14, 18 opens and no submission. The first submission comes back on Monday 25 August. With the dates in hand we can go looking for the cause: a site release, a theme update, a change to the Tag Manager container. Finding the cause matters as much as finding the gap, because it is the only way to be sure it does not happen again.

A perfect twin does not always exist. Possible alternatives are the notification emails the form sends to the company (if they arrive, the requests left), the CRM data, visits to the thank-you page, or the same conversions recorded by another platform, such as Google Ads. Each has its flaws, but it is enough that it is recorded independently of the broken event.


Reconstructing with a ratio

We know the gap is there and where it is. Can we estimate how many submissions were lost? The idea is to use that very ratio: if normally there is one submission every six opens or so, and during the gap there were so many opens, the submissions will have been about one sixth of those opens.

The ratio must be estimated on a clean period close to the gap: the eight weeks before and the eight weeks after, so that it reflects the behaviour of the form at that moment and not the one of a year earlier.

I compute the reconstruction in R:

calibration <- (data$day >= as.Date("2025-06-02") & data$day <= as.Date("2025-07-27")) |
               (data$day >= as.Date("2025-08-25") & data$day <= as.Date("2025-10-19"))
opens_cal      <- sum(data$opens[calibration])
subs_cal       <- sum(data$submissions[calibration])
ratio          <- subs_cal / opens_cal
binom.test(subs_cal, opens_cal)$conf.int

opens_gap <- sum(data$opens[data$gap])
ratio * opens_gap

In the sixteen calibration weeks there are 339 submissions out of 2,004 opens: a ratio of 0.169, with a 95% confidence interval from 0.153 to 0.186. In the gap there are 259 opens, so the estimate of the lost submissions is 0.169 × 259 = 43.8.

A single number, however, promises a precision we do not have. The sources of uncertainty we can measure are two: the ratio itself, which we only know through the interval we have just seen, and chance, because even with the exact ratio, 259 opens do not always produce the same number of submissions. We combine them with a simulation: we draw a plausible ratio, then a number of submissions compatible with that ratio, and repeat ten thousand times.

I simulate the prediction interval in R:

set.seed(1)
sim <- replicate(10000, {
  rr <- rbeta(1, subs_cal + 0.5, opens_cal - subs_cal + 0.5)   # a plausible ratio
  rbinom(1, opens_gap, rr)                                     # the submissions that follow from it
})
quantile(sim, c(0.025, 0.975))

To draw the ratio we use a beta distribution built on the submissions and the opens of the calibration period (it is the posterior distribution we get starting from a beta(0.5, 0.5), the so-called Jeffreys prior, weakly informative): a simple way of representing how much we know. The 95% prediction interval runs from 32 to 57 submissions. It is wide, and it should be: we are estimating a number we have never seen.
And it holds under its assumptions: that every open becomes a submission with the same probability, at most once and independently of the other opens (the same person opening the form three times creates dependence between opens, and if the dependence is positive, ignoring it makes the interval too narrow), and that during the gap the ratio stayed the same as in the calibration period. The uncertainties these assumptions exclude, such as a ratio that changes or a twin that in turn miscounts, are not in the interval.

Now we can use the luxury of the simulation and look at the true value: during the gap the true submissions were 47, and the tag recorded none of them. The point estimate, 43.8, is off by little, and the true value falls comfortably inside the interval. Over whole months the picture is this:

monthrecordedreconstructedtrue
July 20258290.389
August 20251146.551

August was not a month of 11 requests but of some fifty: more than the August of the year before, in line with the site’s growth. The conclusion of the report turns upside down: not a summer collapse, but an ordinary August.

The reconstruction rests on an assumption that must be stated clearly: that during the gap the ratio between submissions and opens stayed the one it had always been. If in the same period the form itself had also changed (an extra required field, a validation error that prevented sending), the ratio would really have changed, and the reconstruction would overestimate the requests. That is why finding the cause of the gap is not a detail: if the cause is only the tag, the assumption is reasonable; if the cause also touched the form, it is not.

A warning: reconstructed data is not measured data. It must be marked as such in the series, with a note explaining how it was obtained, and it must never be used to support a conclusion that the measured data alone cannot carry. Reconstruction serves to avoid drawing wrong conclusions from a gap, not to produce new certainties.


Reconstruct or document the gap?

Having an estimate does not oblige us to use it. There are two paths, and the choice depends on how the data will be used.

Reconstructing is worthwhile when the series must stay continuous: a forecasting model, a year-over-year comparison that next year will put August 2026 next to August 2025, a calculation of the cost per contact of the campaigns. Leaving 11 in place of some fifty, in these cases, poisons all the calculations that come after. In the series we keep the reconstructed figure, with a column that marks it as estimated and a note on the method.

Documenting the gap without reconstructing the data is worthwhile when the data have already entered delivered documents, or when the uncertainty of the reconstruction is too large relative to the decision to be taken. In this case we keep the measured data, add a note that explains the gap, its dates and the estimate of the missing data with its interval, and avoid using that period for conclusions.

The two paths have one thing in common: the choice must always be stated. The worst case is not reconstructing badly or not reconstructing: it is a report in which part of the numbers is estimated and nobody knows it. And if the gap is discovered after a report has been delivered, the right thing is to go back to whoever received it and correct it, however uncomfortable that is.


Preventing it: the Monday check

In our example the gap lasted four weeks, and was discovered, at best, at the end of the month. With an automatic check we can discover it in seven days. The idea again relies on the ratio: every Monday, for the week that has just closed, we compute how many submissions we expected given the opens, using the ratio of the previous eight weeks, and we check how probable the observed number is.

I write the check in R:

weekly_check <- function(s) {
  before <- weekly[weekly$week < s & weekly$week >= s - 56, ]
  rr <- sum(before$submissions) / sum(before$opens)
  q  <- weekly[weekly$week == s, ]
  c(expected = rr * q$opens, observed = q$submissions, p = pbinom(q$submissions, q$opens, rr))
}
weekly_check(as.Date("2025-07-28"))

The number of submissions, given the opens, can be approximated with a binomial: every open becomes a submission with a certain probability, at most once and independently of the others. pbinom() gives the probability of seeing a number of submissions that low or lower. For the week of 28 July we expected 11.8 submissions and we saw zero: the probability is 2.5 in a million. The alarm would have sounded on Monday 4 August, at the first opportunity, instead of in September.

A reasonable alarm threshold is a probability below 0.001. Applying the check to all the weeks from March 2024 to December 2025, 95 in all, only the four weeks of the gap trigger an alarm; in the other 91 the lowest probability is 0.0075, far from the threshold.
The check is an approximation, though, and the threshold does not guarantee one false alarm every thousand weeks: the reference ratio is estimated, not known, and with many weeks and many event pairs under control the chances of a false alarm multiply. In real data, which vary more than simulated data, there will be some false alarms, and the threshold has to be calibrated by looking at how many alarms it produces over past periods in which we know the tracking was working; but a false alarm costs a five-minute check, a gap discovered after a year costs a wrong report.

Two practical precautions. The first: the weeks of a confirmed failure must be excluded from the reference ratio computed for the following weeks, otherwise the gap lowers the expected value and the check becomes less sensitive exactly when it is needed (the function in the example, for simplicity, does not do this). It is better to exclude confirmed failures than all flagged weeks: automatically removing every unusual week would build too optimistic a reference. The second: the check must be run on all the event pairs that matter, but the binomial applies directly only when the events are matched one to one, like the opening and the submission of the same form. For pairs like sessions and conversions, where one session can bring several conversions, the ratio remains a good indicator, but we need a different model (for instance a Poisson or a negative binomial on the number of conversions, with sessions as exposure). All of this is a direct application of anomaly detection to a very concrete problem: not a strange day, but a measuring instrument that stopped counting.


Frequently asked questions

How can we tell whether a drop in conversions is real or a tracking problem?
We compare the event with a twin event recorded independently, for example the form submission with its open, or with the notification emails or the CRM data. If the twin is normal and the ratio between the two collapses, the problem lies in the conversion or in its measurement, and the notifications or the CRM tell us which of the two; if both fall, we have to tell a real decline apart from a failure that hits both (a consent banner, the Tag Manager container), using a source independent of the site’s tracking.

Can the missing conversions be reconstructed in GA4?
GA4 does not allow replacing the counts of a gap like this one with estimates, when it is discovered weeks or months later, but we can reconstruct them in an external analysis: we estimate the ratio between the conversion and a linked event over a clean period and apply it to the period of the gap, with an interval that accounts for uncertainty. Reconstructed data must always be marked as estimated.

How reliable is a reconstruction with a ratio?
It depends on the stability of the ratio and on how much data there is in the calibration period. The key assumption is that during the gap the ratio did not change: if the same cause that broke the tag also changed the form or the page, the reconstruction is not reliable.

How can we notice a tracking gap in time?
With an automatic weekly check that compares, for each pair of linked events, the observed number with the expected one given the ratio of the previous weeks, and flags the too improbable cases. In our example the alarm would have sounded in the first week of the gap.


Try it yourself

The simulation can be modified in many ways. Three suggestions:

  1. A partial gap. Instead of recording 4% of submissions, make the tag record 60% (because it breaks only on mobile, for example). Does the Monday check still see it? After how many weeks?
  2. A real decline in the same period. Along with the broken tag, make the ratio between submissions and opens really fall from 0.17 to 0.12 during the gap (an extra required field). By how much is the reconstruction wrong? Does the true value still fall inside the interval?
  3. The broken twin. Break the opens as well, in the same period. What is left, to notice the gap, apart from the comparison with the previous year?

Hint: the second test is the most instructive, because it shows the main limit of the reconstruction. The ratio reconstructs well only what the tag lost, not what the site lost.


The gap in our example was easy to find because it was total: zero submissions for four weeks. The subtler problems are the ones where the numbers do not vanish but swell or double: the same user sending the form three times, the same conversion counted twice by two tags. When we count events instead of people, comparisons between groups can mislead us even with perfectly working tracking. It is the subject of the next article: counting events or counting people.


Further reading

The book that takes data quality most seriously is Trustworthy Online Controlled Experiments by Kohavi, Tang and Xu: built for online experiments, it devotes whole chapters to the automatic checks that unmask measurement problems before they become wrong conclusions, and it is written by people who built those checks at companies like Microsoft, Google and LinkedIn. For the underlying idea, that a number is worth as much as the way it was measured, How to Lie with Statistics by Darrell Huff remains the quickest and most entertaining read.

Leave a Reply

Your email address will not be published. Required fields are marked *