{"id":3942,"date":"2026-06-29T09:22:50","date_gmt":"2026-06-29T08:22:50","guid":{"rendered":"https:\/\/www.gironi.it\/blog\/?p=3942"},"modified":"2026-07-11T21:50:07","modified_gmt":"2026-07-11T20:50:07","slug":"expected-vs-actual-ctr","status":"publish","type":"post","link":"https:\/\/www.gironi.it\/blog\/en\/expected-vs-actual-ctr\/","title":{"rendered":"Expected vs Actual CTR: finding the pages that earn fewer clicks than their position deserves"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Anyone who spends their days inside Search Console knows that little nagging feeling: a page sits steadily in third position, yet the clicks are few, a CTR that looks like it belongs at the bottom of the page.<br>The question we usually ask is the wrong one: not &#8220;how many clicks does it get?&#8221;, but the more uncomfortable one \u2014 &#8220;how many clicks <em>should<\/em> it get, sitting where it sits?&#8221;. Without a benchmark, a 3% CTR tells us nothing: for position 8 it would be excellent, for position 2 a small disaster.<br>What we are missing, in order to judge, is an expected CTR: the value to compare the actual one against.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We have already seen, talking about <a href=\"https:\/\/www.gironi.it\/blog\/en\/correlation\/\">correlation<\/a>, that position and CTR move together along a steep curve; and that the next step \u2014 using one variable to predict another \u2014 is the job of <a href=\"https:\/\/www.gironi.it\/blog\/en\/correlation-and-regression-analysis-linear-regression\/\">linear regression<\/a>.<br>Here the two threads tie together: we turn that curve into an <strong>expected CTR<\/strong> and measure, page by page, how far each one deviates from it. It is the way to stop reading CTRs as absolute numbers and start reading them for what they really are: deviations from a norm.<\/p>\n\n\n\n<!--more-->\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What we will cover<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"#why-baseline\">Why a CTR, on its own, means nothing<\/a><\/li><li><a href=\"#modelling-the-curve\">Modelling the CTR curve: three roads<\/a><\/li><li><a href=\"#the-example\">An example with Search Console data<\/a><\/li><li><a href=\"#loess-nls\">The other two roads: loess and nls at work<\/a><\/li><li><a href=\"#residuals\">Residuals: who earns less than they should<\/a><\/li><li><a href=\"#reading-deviations\">Reading the deviations without fooling ourselves<\/a><\/li><li><a href=\"#try-it-yourself\">Try it yourself<\/a><\/li><li><a href=\"#further-reading\">Further reading<\/a><\/li><\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"why-baseline\">Why a CTR, on its own, means nothing<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">There are industry tables telling us what the CTR of each position &#8220;should&#8221; be: the first around 25-30%, the second below half, and so on going down. They are useful as a general horizon, but to judge <em>our<\/em> pages they lead us astray: CTR depends on the type of query (a heavily clicked brand term or a cold informational search), on the sector, on how crowded the SERP is with ads and <em>rich snippets<\/em>.<br>The average CTR of &#8220;position 3&#8221; on an American e-commerce benchmark has almost nothing to say to our technical blog in Italian.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The way out is to stop comparing ourselves with an external table and build the reference curve <em>on our own data<\/em>.<br>We take all the pages, their average positions and their CTRs, and we trace the curve that describes the typical behaviour of CTR as position changes <em>for the way our own site works<\/em>. That curve becomes the yardstick: the expected CTR of a page is the value the curve assigns it, given its position.<br>The gap between the actual CTR and that expected value is the information we were after.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>A CTR only makes sense next to the position that produced it: it is the deviation from the curve, not the absolute number, that tells us whether a page is working well or leaving clicks on the table.<\/strong><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"modelling-the-curve\">Modelling the CTR curve: three roads<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Building the curve means estimating a function that, given the position, returns the expected CTR. We already know the shape of that curve by eye: it starts high, plummets across the first positions and then flattens towards zero. A straight line does not describe it; we need something that curves.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There is, however, a detail that changes the whole way of reasoning, and it is the kind of deviation we care about.<br>We do not care that a page gets &#8220;two CTR points less&#8221; than expected: at the top of the SERP two points are crumbs, at the bottom they are a doubling. We care about the <em>multiplicative<\/em> deviation \u2014 &#8220;it earns half of what it should&#8221;, &#8220;it earns double&#8221;.<br>And a multiplicative deviation is best handled on a logarithmic scale, where a ratio becomes a difference.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The most natural shape for a curve of this kind is the <strong>power law<\/strong>, that is the idea that CTR is proportional to position raised to a negative exponent:<\/p>\n\n\n\n\\( \\text{CTR} = a \\cdot position^{b} \\\\ \\)\n\n\n\n<p class=\"wp-block-paragraph\">where \\( a \\) sets the general level and \\( b \\) (negative) governs how fast the descent is. The beauty arrives when we take the logarithm of both sides, which turns that curve into a straight line:<\/p>\n\n\n\n\\( \\log(\\text{CTR}) = \\log(a) + b \\cdot \\log(position) \\\\ \\)\n\n\n\n<p class=\"wp-block-paragraph\">In other words: the logarithm of CTR is a <em>linear<\/em> function of the logarithm of position. And estimating a straight line is exactly what we know how to do with regression. From here, three roads to build the curve.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The first, and the one I recommend as the workhorse, is a <strong>linear regression on the logarithms<\/strong> \u2014 <code>lm(log(ctr) ~ log(position))<\/code>. It is in base R, it is interpretable (the slope \\( b \\) is the elasticity of CTR to position: by what percentage CTR drops for each percentage point of extra position), and its residuals are already on a logarithmic scale, hence <em>multiplicative<\/em>, exactly as we need.<br>It also extrapolates to rarely observed positions, and it can be <strong>weighted by impressions<\/strong> (<code>weights = impression<\/code>), so that pages with a handful of clicks do not skew the curve as much as those with tens of thousands of views. It is a pragmatic choice rather than the theoretically optimal weight (for a proportion the variance also depends on the CTR itself), but in practice it works very well.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The second is <strong>non-linear regression<\/strong> with <code>nls<\/code>, which estimates \\( a \\) and \\( b \\) directly on the natural scale of CTR without going through logarithms. It is an elegant refinement, but it must be primed with sensible starting values (which we fish out precisely from the log-log regression) and on messy data it may fail to converge. I keep it for when I need a clean parameter to put in a report, not as a starting point.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The third is <strong>local smoothing<\/strong> with <code>loess<\/code>, which imposes no shape on the curve and lets the data &#8220;draw it&#8221;. It is perfect for <em>seeing<\/em> the trend at a glance, but it wobbles on the tails (few pages in first position) and above all it does not extrapolate: outside the observed range it has nothing to say. It is an exploratory tool, not the model on which to base judgements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So: we start from the log-log regression weighted by impressions as the working model, we compare it by eye with a <code>loess<\/code> to check we are not forcing the wrong shape, and we move to <code>nls<\/code> only if we need the explicit exponent. Let us see it at work.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"the-example\">An example with Search Console data<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We start from an extract like the one anyone can download from Search Console: one row per page, with impressions, clicks and average position. In reality CTR is the ratio of clicks to impressions; here, since these are example data, we go the other way round \u2014 we set a plausible CTR and reconstruct the clicks.<br>I build the table in R with twelve example pages (with, on purpose, a couple of anomalous cases):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gsc &lt;- data.frame(\n  page       = c(\"\/technical-seo-guide\",\"\/seo-audit-checklist\",\n                 \"\/keyword-research-guide\",\"\/campaign-roi-calculator\",\n                 \"\/google-analytics-tutorial\",\"\/statistics-glossary\",\n                 \"\/seo-tool-review\",\"\/link-building-guide\",\n                 \"\/competitor-analysis\",\"\/attribution-model\",\n                 \"\/ranking-report\",\"\/meta-tag-optimization\"),\n  impression = c(  9800, 5400, 12500, 2100, 7600, 1500,\n                   8300, 4200,  6100,  900, 3300, 1800),\n  position   = c(  1.3,  2.1,   3.4,  4.0,  4.6,  5.2,\n                   2.8,  6.1,   7.0,  8.3,  9.1, 10.2)\n)\n# observed CTR (usually computed as click \/ impression)\ngsc$ctr   &lt;- c(0.232, 0.150, 0.034, 0.071, 0.066, 0.060,\n               0.171, 0.048, 0.041, 0.012, 0.031, 0.028)\ngsc$click &lt;- round(gsc$impression * gsc$ctr)<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">I now estimate the expected-CTR curve with the regression on logarithms, weighting each page by its impressions:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>fit &lt;- lm(log(ctr) ~ log(position), data = gsc, weights = impression)\nround(coef(fit), 3)\n# (Intercept)  log(position)\n#      -1.240          -1.088<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The slope is <strong>\u22121.088<\/strong>: a value close to \u22121 describes an almost inversely proportional curve, where doubling the position (going, say, from 3 to 6) cuts the CTR roughly in half.<br>It is the same steep drop we had glimpsed when measuring correlation, but now written in a formula we can <em>query<\/em>: given a position number, it returns the typical CTR that position implies on our site.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"loess-nls\">The other two roads: loess and nls at work<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We picked the log-log regression as our working model, but we had promised all three roads. It is worth seeing the other two genuinely at work on the same twelve data points, to understand what they add \u2014 and above all where they stumble.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Non-linear regression<\/strong> estimates \\( a \\) and \\( b \\) directly on the CTR scale, without going through logarithms. It must be primed with sensible starting values, which we fish out precisely from the log-log we just estimated: the intercept brought back to the natural scale with <code>exp<\/code> is our \\( a \\), the slope is our \\( b \\). I set it running in R:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>start &lt;- list(a = exp(coef(fit)[1]), b = coef(fit)[2])  # primed from the log-log\nfit_nls &lt;- nls(ctr ~ a * position^b, data = gsc,\n               weights = impression, start = start)\nround(coef(fit_nls), 3)\n#      a       b\n#  0.315  -1.073<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">It converges, and returns an exponent of <strong>\u22121.073<\/strong>, practically the same as the log-log, but with a bonus: now \\( a \\) and \\( b \\) are numbers you can read on the true CTR scale. An \\( a \\) of 0.315 says, in plain terms, that at the top of the SERP the typical CTR is around 31% \u2014 exactly the kind of clean parameter to put in a report.<br>The price we pay is fragility: without those starting values, or on noisier data, <code>nls<\/code> may fail to converge at all and hand us back only an error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Local smoothing<\/strong> with <code>loess<\/code> does the opposite: it imposes no shape, it lets the data draw the curve. I estimate it and ask it for the expected CTR at a few positions, including the first:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>fit_lo &lt;- loess(ctr ~ position, data = gsc, span = 0.9)\nround(predict(fit_lo, data.frame(position = c(1, 2, 3, 5))), 3)\n# [1]    NA 0.173 0.111 0.029<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And here is the limit in a single output: at position 1, <code>loess<\/code> returns <strong><code>NA<\/code><\/strong>. The minimum our data observe is 1.3, and outside that range <code>loess<\/code> refuses to commit \u2014 it <em>does not extrapolate<\/em>.<br>For a CTR curve this is a serious flaw: the first position, the one we care about most, is often the least populated, and that is exactly where the local method leaves us empty-handed.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Put on the same chart, the three roads tell the whole story at a glance:<\/p>\n\n\n\n<div class=\"wp-block-group has-background\" style=\"background-color:#f5f7f9;margin-top:2.5rem;margin-bottom:2.5rem;padding-top:1.5rem;padding-right:1.5rem;padding-bottom:1rem;padding-left:1.5rem\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-container-core-group-is-layout-eed7543b wp-block-group-is-layout-constrained\">\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1020\" height=\"690\" src=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/ctr-tre-metodi-en.png\" alt=\"The three estimates of the expected-CTR curve on the twelve example points. In the middle, where pages abound, they almost coincide; on the tails they diverge. loess (purple) stops at position 1.3 \u2014 it does not extrapolate \u2014 while lm (blue) and nls (orange) continue below the first position. The nls curve sits a touch above the log-log because it estimates the level directly on the CTR scale.\" class=\"wp-image-4077\" srcset=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/ctr-tre-metodi-en.png 1020w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/ctr-tre-metodi-en-300x203.png 300w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption class=\"wp-element-caption\">The three estimates of the expected-CTR curve on the twelve example points. In the middle, where pages abound, they almost coincide; on the tails they diverge. loess (purple) stops at position 1.3 \u2014 it does not extrapolate \u2014 while lm (blue) and nls (orange) continue below the first position. The nls curve sits a touch above the log-log because it estimates the level directly on the CTR scale.<\/figcaption><\/figure>\n\n<\/div><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">In the middle, where pages abound, the three curves almost coincide: any method works when the data speak clearly. It is on the tails that they part ways \u2014 <code>loess<\/code> wobbles behind the few pages it finds and stops dead at the edge of the data, while <code>lm<\/code> and <code>nls<\/code> continue smoothly even where observations are scarce.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>So<\/strong>: the weighted log-log stays the workhorse \u2014 interpretable, extrapolable, with residuals already on a multiplicative scale. <code>nls<\/code> refines it when we need a clean exponent to write down; <code>loess<\/code> is the critical eye that, before we trust the model, tells us whether we are forcing the wrong shape. Three different tools for a single craft: turning a position into an expected CTR.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"residuals\">Residuals: who earns less than they should<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Having the curve means being able to compute, for each page, its expected CTR and compare it with the actual one. The comparison, as we said, must be made in terms of a ratio and not a difference: <code>ratio = actual_ctr \/ expected_ctr<\/code>. A value around 1 says the page earns as predicted; well below 1 that it is leaving clicks on the table; well above 1 that it captures more than its share.<br>I compute the expected CTR, the ratio, and flag the cases that truly deviate \u2014 but only if they have enough impressions to make their CTR reliable:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gsc$ctr_exp &lt;- exp(predict(fit))           # back from the log scale to the natural one\ngsc$ratio   &lt;- gsc$ctr \/ gsc$ctr_exp\n\ngsc$flag &lt;- ifelse(gsc$ratio &lt; 0.6 &amp; gsc$impression &gt;= 1000, \"UNDER\",\n             ifelse(gsc$ratio &gt; 1.4 &amp; gsc$impression &gt;= 1000, \"OVER\", \"ok\"))\n\ngsc[order(gsc$ratio),\n    c(\"page\",\"position\",\"impression\",\"ctr\",\"ctr_exp\",\"ratio\",\"flag\")]<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">n.b. <code>predict<\/code> gives us the logarithm of the expected CTR, because that is the scale on which we estimated the model: <code>exp<\/code> brings it back to an actual CTR. Strictly speaking <code>exp<\/code> returns the <em>median<\/em> of the expected CTR, not the arithmetic mean (under log-normal errors the true mean is a touch higher), but for the relative ratios we care about the distinction is immaterial.<br>The output, sorted from the lowest ratio to the highest:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>page<\/th><th>position<\/th><th>impression<\/th><th>ctr<\/th><th>ctr_exp<\/th><th>ratio<\/th><th>flag<\/th><\/tr><\/thead><tbody><tr><td>\/attribution-model<\/td><td>8.3<\/td><td>900<\/td><td>0.012<\/td><td>0.029<\/td><td>0.41<\/td><td>ok<\/td><\/tr><tr><td>\/keyword-research-guide<\/td><td>3.4<\/td><td>12500<\/td><td>0.034<\/td><td>0.076<\/td><td>0.44<\/td><td><strong>UNDER<\/strong><\/td><\/tr><tr><td>\/technical-seo-guide<\/td><td>1.3<\/td><td>9800<\/td><td>0.232<\/td><td>0.218<\/td><td>1.07<\/td><td>ok<\/td><\/tr><tr><td>\/campaign-roi-calculator<\/td><td>4.0<\/td><td>2100<\/td><td>0.071<\/td><td>0.064<\/td><td>1.11<\/td><td>ok<\/td><\/tr><tr><td>\/seo-audit-checklist<\/td><td>2.1<\/td><td>5400<\/td><td>0.150<\/td><td>0.129<\/td><td>1.16<\/td><td>ok<\/td><\/tr><tr><td>\/competitor-analysis<\/td><td>7.0<\/td><td>6100<\/td><td>0.041<\/td><td>0.035<\/td><td>1.18<\/td><td>ok<\/td><\/tr><tr><td>\/ranking-report<\/td><td>9.1<\/td><td>3300<\/td><td>0.031<\/td><td>0.026<\/td><td>1.18<\/td><td>ok<\/td><\/tr><tr><td>\/link-building-guide<\/td><td>6.1<\/td><td>4200<\/td><td>0.048<\/td><td>0.040<\/td><td>1.19<\/td><td>ok<\/td><\/tr><tr><td>\/google-analytics-tutorial<\/td><td>4.6<\/td><td>7600<\/td><td>0.066<\/td><td>0.055<\/td><td>1.20<\/td><td>ok<\/td><\/tr><tr><td>\/meta-tag-optimization<\/td><td>10.2<\/td><td>1800<\/td><td>0.028<\/td><td>0.023<\/td><td>1.21<\/td><td>ok<\/td><\/tr><tr><td>\/statistics-glossary<\/td><td>5.2<\/td><td>1500<\/td><td>0.060<\/td><td>0.048<\/td><td>1.25<\/td><td>ok<\/td><\/tr><tr><td>\/seo-tool-review<\/td><td>2.8<\/td><td>8300<\/td><td>0.171<\/td><td>0.094<\/td><td>1.81<\/td><td><strong>OVER<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The case that jumps out is <strong>\/keyword-research-guide<\/strong>: it sits in third position, where the curve would expect a CTR of 7.6%, and instead it gathers a meagre 3.4% \u2014 less than half of what it should, on twelve thousand five hundred impressions that make the figure rock solid.<br>It is a strong, immediately actionable hypothesis: in all likelihood the <em>title<\/em> and the <em>meta description<\/em> are not doing their job, and a rewrite could unlock clicks the position had already earned.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">At the opposite end there is <strong>\/seo-tool-review<\/strong>, which in second-to-third position earns almost double the expected. It is not a problem, it is a lesson: something in that <em>snippet<\/em> works beautifully \u2014 a magnetic title, a <em>rich card<\/em>, a perfect match with intent \u2014 and it is worth understanding what, to try to replicate it elsewhere. <strong>Residuals are not only there to find the sick ones: over-performances are the case studies from which to learn what, on our site, makes people click.<\/strong><\/p>\n\n\n\n<div class=\"wp-block-group has-background\" style=\"background-color:#f5f7f9;margin-top:2.5rem;margin-bottom:2.5rem;padding-top:1.5rem;padding-right:1.5rem;padding-bottom:1rem;padding-left:1.5rem\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-container-core-group-is-layout-eed7543b wp-block-group-is-layout-constrained\">\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1110\" height=\"690\" src=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/ctr-curva-attesa-en.png\" alt=\"The twelve example pages: average SERP position against CTR, with each point's area proportional to impressions. The blue curve is the expected CTR from the weighted log-log; the vertical segment below each point is its residual, the distance from the curve. Two pages stand out: \/keyword-research-guide at 0.44\u00d7 expected (red, under-performs) and \/seo-tool-review at 1.81\u00d7 (green, over-performs).\" class=\"wp-image-4078\" srcset=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/ctr-curva-attesa-en.png 1110w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/ctr-curva-attesa-en-300x186.png 300w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/ctr-curva-attesa-en-1024x637.png 1024w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption class=\"wp-element-caption\">The twelve example pages: average SERP position against CTR, with each point&#8217;s area proportional to impressions. The blue curve is the expected CTR from the weighted log-log; the vertical segment below each point is its residual, the distance from the curve. Two pages stand out: \/keyword-research-guide at 0.44\u00d7 expected (red, under-performs) and \/seo-tool-review at 1.81\u00d7 (green, over-performs).<\/figcaption><\/figure>\n\n<\/div><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"reading-deviations\">Reading the deviations without fooling ourselves<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">There is a detail in the table that is the heart of the whole matter, and that is easy to miss. <strong>\/attribution-model<\/strong> has a ratio of 0.41 \u2014 even lower than \/keyword-research-guide \u2014 yet we did not flag it. The reason is in the impressions column: nine hundred.<br>A CTR computed on so little data is almost pure noise, and next month it will drift back towards its mean regardless of anything we do. Flagging it as a &#8220;page to optimise&#8221; would send us chasing a ghost. Two pages with the same deviation, two opposite verdicts, and the only thing making the difference is the amount of data behind them.<\/p>\n\n\n\n<p class=\"has-light-gray-background-color has-background wp-block-paragraph\">A word of caution: the expected CTR is a <strong>conditional typical value<\/strong> \u2014 the median the curve associates with a position \u2014 not a law of nature. A page can &#8220;under-perform&#8221; for reasons that have nothing to do with the <em>title<\/em>: a brand query inflating competitors&#8217; CTR, a <em>featured snippet<\/em> or a block of ads eating the clicks before the first organic result, a purely informational intent already satisfied by reading the <em>snippet<\/em>. And a CTR built on few impressions measures almost nothing: it will regress towards its mean on its own, as we saw talking about <a href=\"https:\/\/www.gironi.it\/blog\/en\/regression-to-the-mean\/\">regression to the mean<\/a>. <strong>A negative residual is a hypothesis to verify \u2014 &#8220;maybe the title earns little here&#8221; \u2014 not a verdict to execute.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It is also worth remembering that we estimated the curve on our own data, and that data influences it: a handful of very anomalous pages can tilt it just enough to shift the judgements on the others.<br>We can already see it in our table: ten pages out of twelve have a ratio above 1, but it is not that the site &#8220;over-performs&#8221; almost everywhere. It is that the single large negative deviation, <code>\/keyword-research-guide<\/code>, weighs a great deal (twelve thousand five hundred impressions) and pulls the curve downwards, raising the ratio of all the others as a side effect. The &#8220;centre&#8221; of the cloud, in short, is not exactly 1, and it is better to read the ratios in relative terms \u2014 who sits well below and who well above the bulk of the group \u2014 rather than against the hard threshold of one.<br>This is why the impression weighting is precious, and why it pays to recompute the curve whenever the picture changes, instead of treating it as a constant carved in stone.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"try-it-yourself\">Try it yourself<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The best way to internalise the mechanism is to get your hands on it. Building on the code above, there are three interesting directions to explore:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Aggregate by <strong>query<\/strong> instead of by page: the same page can appear on dozens of searches with different positions and CTRs, and often it is there \u2014 on the single query \u2014 that the missed opportunity hides. The curve and the residuals are built in the exact same way.<\/li><li>Drop the impression weighting \u2014 <code>lm(log(ctr) ~ log(position))<\/code> without <code>weights<\/code> \u2014 and compare the exponent and the verdicts with the weighted version: how much do the conclusions shift once the pages with more data no longer have a louder voice? It is the way to <em>feel<\/em> how much the weighting matters, instead of taking it on faith.<\/li><li>Change the impression threshold below which you do not trust the CTR: going from 1000 to 3000, which pages leave the radar? The right number does not exist in the abstract, it depends on how much traffic your site moves.<\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">A hint: the structure never changes \u2014 you estimate the curve, you predict the expected value, you look at the ratio. It is by playing with the threshold and the level of aggregation that you really understand how much of what we call an &#8220;under-performing page&#8221; is signal and how much is, simply, noise.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\">Spotting a page that earns less than expected <em>for its position<\/em> is a close cousin of another problem every analyst knows: spotting a day that earns less than expected <em>over time<\/em>, a drop or a spike in traffic that does not square with the usual trend.<br>It is the same reasoning on residuals \u2014 observed value against expected value \u2014 moved from the space of positions to the axis of time, where the expected value is provided by the historical <a href=\"https:\/\/www.gironi.it\/blog\/en\/time-series-analysis-and-forecasting-in-r\/\">trend of the series<\/a>. From there springs <a href=\"https:\/\/www.gironi.it\/blog\/en\/anomaly-detection-how-to-identify-outliers-in-your-data\/\">anomaly detection<\/a>: telling signal from noise when the numbers move over time, and the next step of our path.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"further-reading\">Further reading<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If you want to go deeper into regression, logarithmic transformations and the reading of residuals \u2014 the very backbone of the model we built here \u2014 and then push beyond the power law towards local methods like <code>loess<\/code>, <em><a href=\"https:\/\/www.amazon.it\/dp\/1461471370?tag=consulenzeinf-21\" rel=\"nofollow sponsored noopener\" target=\"_blank\">An Introduction to Statistical Learning<\/a><\/em> by James, Witten, Hastie and Tibshirani is the book I recommend: it covers both the &#8220;why&#8221; of the logarithms and the &#8220;how&#8221; of interpreting coefficients, with hands-on labs in R, always starting from applied problems.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Anyone who spends their days inside Search Console knows that little nagging feeling: a page sits steadily in third position, yet the clicks are few, a CTR that looks like it belongs at the bottom of the page.The question we usually ask is the wrong one: not &#8220;how many clicks does it get?&#8221;, but the &hellip; <a href=\"https:\/\/www.gironi.it\/blog\/en\/expected-vs-actual-ctr\/\" class=\"more-link\">Leggi tutto<span class=\"screen-reader-text\"> &#8220;Expected vs Actual CTR: finding the pages that earn fewer clicks than their position deserves&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","footnotes":""},"categories":[161],"tags":[],"class_list":["post-3942","post","type-post","status-publish","format-standard","hentry","category-statistics"],"lang":"en","translations":{"en":3942,"it":3941},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false,"post-thumbnail":false},"uagb_author_info":{"display_name":"Paolo Gironi","author_link":"https:\/\/www.gironi.it\/blog\/author\/autore-articoli\/"},"uagb_comment_info":10,"uagb_excerpt":"Anyone who spends their days inside Search Console knows that little nagging feeling: a page sits steadily in third position, yet the clicks are few, a CTR that looks like it belongs at the bottom of the page.The question we usually ask is the wrong one: not &#8220;how many clicks does it get?&#8221;, but the&hellip;","_links":{"self":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3942","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/comments?post=3942"}],"version-history":[{"count":2,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3942\/revisions"}],"predecessor-version":[{"id":4079,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3942\/revisions\/4079"}],"wp:attachment":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/media?parent=3942"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/categories?post=3942"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/tags?post=3942"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}