A site receives 612 requests this year and received 640 the year before. The arithmetic is quick: −4.4%. The search for a cause begins, and with it the meeting where someone suggests rebuilding the service pages, someone else blames seasonality, someone else «the general market downturn».
Before explaining a drop, though, it is worth asking whether there is anything to explain. Because 612 against 640 is not a number: it is a count, and counts fluctuate.
The numbers in this article are made up, but the shape of the problem is the one we meet every month: requests, form submissions, phone calls, sign-ups, conversions without a reliable denominator. They are all event counts, and to compare them across two periods we need a test that reasons about their rate, not their total. That test is the Poisson test, and the good news is that it fits in one line of R.
In short: when what we measure is a count, two periods are compared by their rates, not their totals. The Poisson test tells us whether the rate changed: poisson.test(c(x2, x1), T = c(t2, t1)) returns the rate ratio, its confidence interval and a p-value. In our example, 612 requests against 640 are a 4.4% drop that chance alone produces more than one time in five: the ratio is 0.96, the interval runs from 0.85 to 1.07, and there is nothing to explain. The same test works when the periods have different lengths, where comparing totals is the mistake we see most often in reports.
What we will cover:
- Why requests are a Poisson process
- Comparing the rates
- Reading the output without being fooled
- Different lengths: the case almost everyone gets wrong
- When the difference is real
- How much data do we need to see a 10% drop?
- An odd month: the bridge to anomalies
- Frequently asked questions
- Try it yourself
- Further reading
Why requests are a Poisson process
We met the Poisson distribution while talking about rare events happening at a constant average rate: calls to a switchboard, typing errors, arrivals at a counter. Requests on a website are close relatives of those examples. Every visit can produce a submitted form, every submitted form is an event, and over a period of time the number of events oscillates around an average.
The property we need is just one, and it has a name that sounds scarier than it is: in a Poisson distribution the variance equals the mean. In plainer terms: if we expect 640 requests in a year, the natural oscillation of the count is on the order of the square root of 640, that is about 25 requests.
So a «normal» year can end at 615 or at 665 with nothing having happened. Twenty-five requests out of 640 is 4%: exactly the order of magnitude of the drop we started from.
A quick example, to fix the idea. If the true rate were identical in the two years, and each year’s count were drawn from a Poisson distribution with mean 640, how often would we observe a drop of at least 4.4%?
set.seed(2026)
true_rate <- 640
last_year <- rpois(10000, true_rate) # last year
this_year <- rpois(10000, true_rate) # this year, same rate
change <- (this_year - last_year) / true_rate
mean(change <= -0.044)The result is 0.214: more than one time in five. And if we look in both directions, at a drop or a rise of at least 4.4%, we reach 43%. Half the time, with the same true rate, the year-over-year comparison tells a story that is not there.

There is a limit, and it is worth stating right away: Poisson is fine as long as events are independent and the rate is stable. Campaigns, bots, a form that breaks for two weeks all break the assumption, and we will come back to that at the end. For now let us keep the clean case, which is already most monthly reports.
Comparing the rates
The starting point is that the two totals, 612 and 640, are not independent: together they make 1,252 events. If the rate were the same in both years, each of the 1,252 events would have the same probability of falling in the first or the second period, and the number of events in the second year would be a binomial with 1,252 trials and probability one half.
This is the bridge between Poisson and the hypothesis test: once the total is known, the question «are the two rates equal?» becomes «is the split of events between the two periods compatible with a fair coin?».
The Poisson test does exactly that, and the ratio it returns is the ratio of events per unit of time:
\(\hat{R} = \frac{x_2 / T_2}{x_1 / T_1}\)
where x₂ and T₂ are the count and the length of the most recent period, x₁ and T₁ those of the previous one. In R we write it like this. The first argument is the pair of counts, the second the length of the two periods, which here is the same:
poisson.test(c(612, 640), T = c(365, 365))The result contains three numbers that must be read together:
- the rate ratio: 0.956;
- the 95% confidence interval: from 0.855 to 1.070;
- the p-value: 0.445.
The ratio is the number we really care about: it says that this year’s rate is 95.6% of last year’s, that is a 4.4% drop. The interval tells us how precise that estimate is with the data we have: from a 14.5% drop to a 7% rise. The p-value, finally, tells us that a ratio this far from 1 is compatible with pure chance in almost half of all possible worlds.

Reading the output without being fooled
The first temptation is to translate «p = 0.445» into «nothing changed». That is not what the test says. A high p-value does not prove that the two rates are equal: it says the data are not enough to prove that they differ. The difference between absence of evidence and evidence of absence is the same as between «I did not find the wallet» and «the wallet does not exist», and in daily practice it makes quite a difference.
The confidence interval is more useful than the p-value, because it delimits what we can rule out. In our case the interval runs from −14.5% to +7%: we can reasonably rule out a collapse, but we cannot rule out an 8% drop or a 5% rise. If the client asks «are we stable?», the honest answer is «we have ruled out drops beyond 15%, not those beyond 5%».
There is also a detail about the direction of the ratio. poisson.test() returns the ratio of the first group to the second. Reversing the order of the counts turns the ratio into 1.046, and a reader might believe in a rise. It is not a bug in the test: it is a reminder to always state which period sits in the numerator. In practice it helps to put the most recent period first, so that the ratio reads as «this year against last year» and numbers below 1 are drops.
Different lengths: the case almost everyone gets wrong
The test is at its best when the two periods do not last the same time, which is the norm in real reports: a site launched mid-year, a 28-day month against a 31-day one, a 180-day period against a full year.
Imagine a form that received 7 submissions in 180 days and 15 submissions in the previous 365 days. At first glance the second figure is more than double the first, and a first draft of a report might talk about «more than doubled». But the periods do not last the same: the first rate is 7 submissions every 180 days, the second 15 every 365. We normalise and compare:
poisson.test(c(7, 15), T = c(180, 365))The rate ratio is 0.946, with an interval from 0.33 to 2.47 and a p-value of 1. In words: the two rates are indistinguishable, and with so few events the interval is so wide that it lets us say almost nothing.
This is where the test is most valuable and most uncomfortable. Valuable, because it avoids telling a rise that does not exist; uncomfortable, because it forces us to write «we do not know» instead of a percentage.
It is worth noting that the ratio computed on the totals would have been 15/7 = 2.14, that is +114%. The difference between +114% and −5% is not a methodological detail: it is the difference between two reports that say opposite things.
When the difference is real
Of course the test is not only there to dampen enthusiasm. Imagine a period with 420 requests and one, of the same length, with 500. The ratio is 0.84:
poisson.test(c(420, 500))Here the interval runs from 0.74 to 0.96 and the p-value is 0.009. The drop is 16%, and the data let us rule out with some confidence any value above −4%. Significant, however, does not mean large: a 16% drop on 500 requests is an operational problem, not a catastrophe, and the size of the effect must be read separately from its significance. We discussed this at length when talking about effect size and power.
There is another use worth knowing: instead of the p-value, we can ask the interval whether the drop exceeds a threshold we care about. If the decision threshold is «below −10% we change the form», we just look at the interval: if its upper end is below −10%, the decision is made; if it crosses it, the data are not enough. It is a way of using the test that is closer to how decisions are actually taken, and further from the hunt for p-values.
How much data do we need to see a 10% drop?
The case of 7 submissions in 180 days leaves an open question: how many events do we need to notice a 10% drop? The answer comes from simulating the test across many possible worlds. Fix the true rate of the second period at 1,600 events a year, imagine the first is 10% lower, and count how often the test finds the difference:
power <- function(n, drop = 0.9, B = 4000) {
mean(replicate(B, poisson.test(c(rpois(1, n * drop), rpois(1, n)))$p.value < 0.05))
}
power(1600)The result is about 0.82: with 1,600 events a year, a 10% drop is detected eight times out of ten. With 800 events power falls to about 53%, with 400 to a little under a third. In months: if the site receives 80 requests a month, 1,600 events are twenty months of data. A 10% drop cannot be seen in a quarter, and this is probably the most useful thing to put on a slide.
An odd month: the bridge to anomalies
The same tool helps find holes, not only drops. If in a month we expect 70 events and observe 3, the question is no longer «did the rate change?» but «is it plausible that a rate of 70 produces 3 events?». The cumulative probability of observing 3 or fewer events from a Poisson with mean 70 takes a moment:
ppois(3, 70)The result is 2.4×10⁻²⁶. It is not seasonality, not chance, not an odd month: it is almost certainly a measurement problem. It is the same calculation behind anomaly detection, and it will come back when we talk about missing data and how to rebuild it without inventing it.
A warning: the Poisson test assumes that the variance equals the mean. Web data often do not comply — bots, spikes, campaigns, days of the week — and in that case intervals come out too narrow and p-values too small. Before trusting a test on counts, it is worth checking the variance-to-mean ratio; if it is well above 1, the test should be redone with a model that accounts for overdispersion.
Frequently asked questions
Why can’t I simply compare the percentages of two periods?
Because a percentage only makes sense if there is a reliable denominator. If requests are the numerator and sessions the denominator, and sessions are measured badly, the percentage inherits the error. The Poisson test works on counts and lengths, and needs no denominator: that is why it is more robust when the denominator is uncertain.
What is the difference between the Poisson test and a test on proportions?
A test on proportions (prop.test()) is used when each trial has a yes/no outcome and the number of trials is known: 1,000 sessions, 30 conversions. The Poisson test is used when the number of trials is unknown and we count events directly over a period of time. If the denominator exists and is reliable, the proportions test is the natural choice; if it does not, we stay with rates.
Does a high p-value mean nothing changed?
No. It means the data are not enough to prove a change. The confidence interval is more informative, because it says which changes we can rule out and which remain compatible with the data. A wide interval is valuable information: it tells us we need more data, not that everything is fine.
Can I use it to compare more than two periods?
Yes, but not with a single call: we compare them in pairs, or move to a Poisson regression with the period as a variable. In that case, watch out for multiple comparisons: with many periods and many pages, some «significant» difference always turns up by chance.
Try it yourself
A form received 55 submissions in a 31-day month and 48 in a 30-day month. Did the rate change?
poisson.test(c(55, 48), T = c(31, 30))The ratio is about 1.11, the interval crosses 1 and the p-value is well above 0.05: with these numbers there is nothing to report. Now try 12 submissions in 7 days against 3 in the previous 7 days: the ratio is 4, but the interval is so wide that even an identical rate remains plausible. It is the most useful exercise: find the point where the test stops being informative, and write it in the report.
Comparing two periods solves half the problem. The other half begins when the periods are twelve consecutive months and seasonality turns the month-on-month comparison into a puzzle: December cannot be compared with November, and classic decomposition with only two years of data refuses to start. That is the subject of the next article: removing seasonality without having enough seasons.
Further reading
To put the simulations of this article into practice, R for Data Science by Hadley Wickham, Mine Çetinkaya-Rundel and Garrett Grolemund is the practical companion, and the second edition is also free online. For a gentler start on distributions and probability, The Art of Statistics by David Spiegelhalter is the book I recommend to anyone who wants the intuition before the formulas.