{"id":3902,"date":"2026-06-28T09:27:35","date_gmt":"2026-06-28T08:27:35","guid":{"rendered":"https:\/\/www.gironi.it\/blog\/?p=3902"},"modified":"2026-07-08T10:04:17","modified_gmt":"2026-07-08T09:04:17","slug":"naive-bayes-search-intent","status":"publish","type":"post","link":"https:\/\/www.gironi.it\/blog\/en\/naive-bayes-search-intent\/","title":{"rendered":"Naive Bayes: classifying search intent with Bayes&#8217; theorem"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">In the article on the <a href=\"https:\/\/www.gironi.it\/blog\/en\/thompson-sampling-multi-armed-bandit\/\">multi-armed bandit<\/a> we used Bayes to <em>decide between variants<\/em>: shifting traffic toward the one converting best while the test is still running. Now we take a step sideways, while staying within the same line of reasoning: instead of choosing between options, we want to <em>classify<\/em>, that is to attach to each new observation the most probable label given its features.<br> The concrete case is one that anyone doing SEO knows well: <strong>the intent behind a search query<\/strong>. Someone searching &#8220;how to bake a cake&#8221; wants to learn something; someone searching &#8220;buy shoes online&#8221; is ready to pull out a credit card. They are two different worlds, and they call for different content: a guide, a tutorial, a glossary for the first; a product page, a price list, a clearly visible <em>call to action<\/em> for the second. Getting the intent wrong means answering the right question in the wrong way.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The trouble is that queries are many and always new, and classifying them by hand does not scale. We need a method that learns from a handful of labelled examples and then copes on its own with queries it has never seen. The algorithm that does this with almost disarming elegance is <em>Naive Bayes<\/em>, and \u2014 as the name hints \u2014 it starts once again from the <a href=\"https:\/\/www.gironi.it\/blog\/en\/bayesian-statistics-how-to-learn-from-data-one-step-at-a-time\/\">Bayes&#8217; theorem<\/a> that has accompanied us throughout this path.<\/p>\n\n\n\n<!--more-->\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What we will cover<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"#from-theorem\">From the theorem to the classifier<\/a><\/li><li><a href=\"#training\">Training on labelled queries<\/a><\/li><li><a href=\"#classifying\">Classifying new queries<\/a><\/li><li><a href=\"#limits\">The limits of &#8220;naive&#8221;<\/a><\/li><li><a href=\"#try-it-yourself\">Try it yourself<\/a><\/li><li><a href=\"#further-reading\">Further reading<\/a><\/li><\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"from-theorem\">From the theorem to the classifier<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Let us pick up the thread. Bayes&#8217; theorem tells us how to update the probability of a hypothesis in the light of the observed data. Here the hypothesis is &#8220;this query belongs to the <em>informational<\/em> class&#8221; (or <em>transactional<\/em>), and the data are the words that make up the query. We want, in other words, the probability of a class <em>given<\/em> the text: in symbols, <em>P(class | words)<\/em>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Computing it directly would be a nightmare, because the possible combinations of words are boundless. Bayes lets us flip the problem around: instead of asking how probable the class is given the words, we ask how probable those words are given the class \u2014 a question the training data can answer. The rule, first in words and then in a formula, is that the probability of a class given a query is proportional to the prior probability of the class times the probability of observing those words if the class were that one:<\/p>\n\n\n\n\\( P(\\text{class} \\mid \\text{words}) \\propto P(\\text{class}) \\cdot \\prod_i P(\\text{word}_i \\mid \\text{class}) \\\\ \\)\n\n\n\n<p class=\"wp-block-paragraph\">Let us unpack the symbols. <em>P(class)<\/em> is the <strong>prior<\/strong>: how frequent that class is to begin with, before looking at the text (if half of the example queries are informational, the prior is 0.5). <em>P(word | class)<\/em> is how often that word appears in the queries of that class. The symbol \u220f is simply the product: we multiply the contributions of all the words in the query. The sign \u221d (&#8220;proportional to&#8221;) reminds us that we are dropping a constant denominator, identical across classes: since in the end we only care <em>which<\/em> class wins, we can ignore it with no harm.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">And here it is, the spot where &#8220;naive&#8221; hides. <strong>Multiplying the probabilities of the individual words as if they were independent of one another amounts to assuming that, given the class, the presence of one word says nothing about the presence of the others.<\/strong> It is a plainly false assumption about real language \u2014 &#8220;credit&#8221; and &#8220;card&#8221; certainly do not appear at random independently of each other \u2014 and it is precisely this naivety that gives the algorithm its name. The surprising thing, as we shall see, is that despite starting from so crude an assumption the method works beautifully in practice.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"training\">Training on labelled queries<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Training a Naive Bayes means one thing only: counting. For each class we count how many times each word appears in the example queries, and from those counts we derive the <em>P(word | class)<\/em>. No optimisation, no iterations: we scan the data once and fill a table of frequencies.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let us start from a small set of hand-labelled queries, five informational and five transactional. I train the classifier in R like this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>train &lt;- list(\n  info  = c(\"how to bake a cake\", \"what is statistics\", \"seo guide for beginners\",\n            \"free r tutorial\", \"how bayesian works\"),\n  trans = c(\"buy shoes online\", \"iphone price deal\", \"best cheap hosting\",\n            \"purchase seo course\", \"gym membership discount\")\n)\ntok &lt;- function(s) unlist(strsplit(tolower(s), \"\\\\s+\"))\nvocab &lt;- unique(unlist(lapply(unlist(train), tok)))\ncounts &lt;- lapply(train, function(docs) {\n  w &lt;- table(factor(unlist(lapply(docs, tok)), levels = vocab)); w + 1\n})\ntots &lt;- sapply(counts, sum); V &lt;- length(vocab)\nprior &lt;- sapply(train, length) \/ length(unlist(train))   # 0.5 \/ 0.5<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Let us see what happens line by line. The <code>tok<\/code> function splits each query into lowercase words (a spartan tokenisation, but enough). <code>vocab<\/code> is the vocabulary, the list of all distinct words seen in training. For each class, <code>table(factor(...))<\/code> counts the occurrences of each vocabulary word; <code>tots<\/code> is the total of the counts per class, and <code>prior<\/code> here is 0.5 and 0.5 because the two classes have the same number of examples.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There is a detail in that <code>w + 1<\/code> worth pausing on, because it is the trick that holds the whole edifice up. <strong>If a word never appears in the queries of a class, its count is zero, and with it the entire product of probabilities would collapse to zero: a single unknown word would be enough to drive the class probability to zero, wiping out the contribution of all the others.<\/strong> It is the classic case where &#8220;multiplying by zero&#8221; ruins the party. The fix is called <strong>Laplace smoothing<\/strong>: we add 1 to the count of every word, in every class, before computing the proportions. No word any longer has exactly zero probability, only a very small one.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The price of smoothing is that the per-class totals gain the number of vocabulary words (the units added one per word): that is why <code>tots<\/code> is not the raw token counts of the two classes but the smoothed totals. With these numbers, for example, the word &#8220;buy&#8221; (present once among the transactional queries, never among the informational ones) ends up clearly more probable under <em>trans<\/em> than under <em>info<\/em> \u2014 and it is exactly this imbalance that pushes a query toward the right intent. A neutral word like &#8220;seo&#8221;, appearing once on each side, stays roughly balanced between the two classes and does not move the needle.<\/p>\n\n\n\n<div class=\"wp-block-group has-background\" style=\"background-color:#f5f7f9;margin-top:2.5rem;margin-bottom:2.5rem;padding-top:1.5rem;padding-right:1.5rem;padding-bottom:1rem;padding-left:1.5rem\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-container-core-group-is-layout-eed7543b wp-block-group-is-layout-constrained\">\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"990\" height=\"630\" src=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/naive-bayes-impronta-en.png\" alt=\"The lexical fingerprint of the two classes: P(word | class) for a few vocabulary terms, after Laplace smoothing. \"buy\" and \"discount\" weigh more than twice toward transactional, \"how\" toward informational, \"seo\" stays almost balanced: it is this word-by-word imbalance that pushes a query toward the right intent.\" class=\"wp-image-4035\" srcset=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/naive-bayes-impronta-en.png 990w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/naive-bayes-impronta-en-300x191.png 300w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption class=\"wp-element-caption\">The lexical fingerprint of the two classes: P(word | class) for a few vocabulary terms, after Laplace smoothing. &#8220;buy&#8221; and &#8220;discount&#8221; weigh more than twice toward transactional, &#8220;how&#8221; toward informational, &#8220;seo&#8221; stays almost balanced: it is this word-by-word imbalance that pushes a query toward the right intent.<\/figcaption><\/figure>\n\n<\/div><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"classifying\">Classifying new queries<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">With the frequency table ready, classifying a new query is genuinely child&#8217;s play: we tokenise it, sum the logarithms of the probabilities of each word (summing logarithms instead of multiplying probabilities avoids the product of many tiny numbers going into numerical <em>underflow<\/em>, but the result is the same), add the logarithm of the prior, and pick the class with the highest score. I compute it in R:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>classify &lt;- function(query) {\n  w &lt;- tok(query)\n  logp &lt;- log(prior)\n  for (cl in names(train)) {\n    p &lt;- counts[[cl]][w]; p[is.na(p)] &lt;- 1            # out-of-vocab word: neutral weight\n    logp[cl] &lt;- logp[cl] + sum(log(as.numeric(p) \/ tots[cl]))\n  }\n  names(which.max(logp))\n}\ncat(\"'buy seo course discount' -&gt;\", classify(\"buy seo course discount\"), \"\\n\")\ncat(\"'how to learn statistics'   -&gt;\", classify(\"how to learn statistics\"), \"\\n\")<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The output is clear-cut:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>'buy seo course discount' -&gt; trans\n'how to learn statistics'   -&gt; info<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">As we can see, the classifier assigns the first query to the transactional intent and the second to the informational one, exactly as a human would have. It is worth noting how it gets there. In the first query &#8220;buy&#8221; and &#8220;discount&#8221; pull decisively toward <em>trans<\/em>, &#8220;seo&#8221; stays neutral, and not even &#8220;course&#8221; \u2014 which in training appeared in the transactional &#8220;purchase seo course&#8221; \u2014 rows against it: the verdict is solid. In the second, &#8220;statistics&#8221; is a markedly informational word, and &#8220;learn&#8221;, though out of vocabulary, does no harm thanks to that <code>p[is.na(p)] &lt;- 1<\/code> which assigns never-seen words a neutral weight, identical for both classes: having nothing to say, it simply does not vote.<\/p>\n\n\n\n<div class=\"wp-block-group has-background\" style=\"background-color:#f5f7f9;margin-top:2.5rem;margin-bottom:2.5rem;padding-top:1.5rem;padding-right:1.5rem;padding-bottom:1rem;padding-left:1.5rem\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-container-core-group-is-layout-eed7543b wp-block-group-is-layout-constrained\">\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1110\" height=\"585\" src=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/naive-bayes-tiro-alla-fune-en.png\" alt=\"The words' tug of war: for each query, the log-likelihood ratio of each word. To the right (orange) the word pushes toward transactional, to the left (blue) toward informational, near zero the neutral or out-of-vocabulary words (like \"learn\", which does not vote). The sum of the contributions decides the verdict.\" class=\"wp-image-4036\" srcset=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/naive-bayes-tiro-alla-fune-en.png 1110w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/naive-bayes-tiro-alla-fune-en-300x158.png 300w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/naive-bayes-tiro-alla-fune-en-1024x540.png 1024w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption class=\"wp-element-caption\">The words&#8217; tug of war: for each query, the log-likelihood ratio of each word. To the right (orange) the word pushes toward transactional, to the left (blue) toward informational, near zero the neutral or out-of-vocabulary words (like &#8220;learn&#8221;, which does not vote). The sum of the contributions decides the verdict.<\/figcaption><\/figure>\n\n<\/div><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">A handful of example queries per side is very little, and yet the mechanism is already all here. In a real case it is enough to replace the handfuls of labelled queries with a few hundred or thousand queries pulled from Search Console and annotated for intent, and the same code \u2014 unchanged in structure \u2014 becomes an intent classifier you can actually use.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"limits\">The limits of &#8220;naive&#8221;<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Before diving in, though, it is worth knowing where the method shows its seams, because its simplicity has a flip side.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The most obvious limit is precisely the independence assumption it takes its name from. By treating each word as detached from the others, Naive Bayes ignores order and context entirely: to it &#8220;cheap running shoes&#8221; and &#8220;the economics of running shoes&#8221; are the same <em>bag of words<\/em>. In intent classification this matters little, but in subtler tasks it can mislead. Then there is the question of <strong>out-of-vocabulary words<\/strong>: a query made only of terms never seen in training would be decided by the prior alone, that is by a pure coin toss \u2014 the sign that the example dataset is too thin and needs widening.<\/p>\n\n\n\n<p class=\"has-light-gray-background-color has-background wp-block-paragraph\">A note of caution that holds for any classifier, and all the more for one trained on little data: the model learns <em>exactly<\/em> what we show it, biases included. If the transactional example queries all contain the word &#8220;buy&#8221;, the classifier will associate purchase intent with that term and will struggle on an equally transactional but lexically different &#8220;add to cart&#8221;. The quality and representativeness of the labelled data matter more than the sophistication of the algorithm: a Naive Bayes fed with varied, balanced examples beats a sophisticated model trained badly.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That said, Naive Bayes remains a genuinely valuable tool to have in the box: it is extremely fast to train, needs little data to get going, is interpretable (we can always go and look at which words drove the decision) and in text classification it holds its own against far more complex models. It is often the <em>baseline<\/em> to beat before bringing in heavier artillery \u2014 and this is where the door opens onto machine learning proper, where classification is done with trees, regressions and networks that drop the independence assumption in exchange for more power (and less transparency).<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"try-it-yourself\">Try it yourself<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The code above is a perfect playground for building intuition. Search queries are not only informational or transactional: at least a third family is missing, the <strong>navigational<\/strong> one (someone searching &#8220;facebook login&#8221; or &#8220;gironi blog&#8221; just wants to reach a specific site). Here are a few changes to try:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Add a <code>nav<\/code> class to <code>train<\/code> with four or five navigational queries (&#8220;facebook login&#8221;, &#8220;youtube&#8221;, &#8220;amazon sign in&#8221;, &#8220;gmail inbox&#8221;), then retrain: the <code>prior<\/code> will no longer be 0.5 but roughly a third per class. How do the classifications of the earlier queries change?<\/li><li>Feed the classifier an ambiguous query like &#8220;iphone review&#8221; (informational? transactional?) and see which way it leans. Did the verdict make sense, given the words in training?<\/li><li>Remove the smoothing (replace <code>w + 1<\/code> with <code>w<\/code>) and try classifying a query with a rare word: what happens to the score when a count is zero? It is the quickest way to see with your own eyes why Laplace is needed.<\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">The nice part is that the structure of the code never changes: adding a class just means lengthening the <code>train<\/code> list, and everything else \u2014 vocabulary, counts, prior, classification rule \u2014 adapts on its own.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\">With this we close the Bayesian thread we have unspooled article after article: from <a href=\"https:\/\/www.gironi.it\/blog\/en\/bayesian-conversion-rate-estimation\/\">estimating a conversion rate<\/a> to comparing variants, from the adaptive allocation of traffic to this last leap, from deciding to classifying. The same theorem, reworn each time in a different guise, has proved a surprisingly sturdy common thread. From here the road forks toward machine learning in the full sense \u2014 decision trees, logistic regression, neural networks \u2014 where the methods give up the most comfortable assumptions in exchange for power, and where Bayes stays in the background as the grammar by which, in the end, we always learn from data.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"further-reading\">Further reading<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If you want to move from Naive Bayes to machine learning proper while keeping your feet on the ground (and R at hand), <a href=\"https:\/\/www.amazon.it\/dp\/1461471370?tag=consulenzeinf-21\" rel=\"nofollow sponsored noopener\" target=\"_blank\"><em>An Introduction to Statistical Learning<\/em><\/a> by James, Witten, Hastie and Tibshirani is the book I recommend. It is the most accessible doorway into applied machine learning: it explains classification, trees and regression with the right rigour but without intimidating mathematics, and every chapter has R labs you can redo step by step. The second edition devotes space to Naive Bayes itself, so the jump from this article to the rest of the path is a natural one.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This article is part of the <a href=\"https:\/\/www.gironi.it\/blog\/en\/bayesian-approach\/\">&#8220;The Bayesian Approach&#8221;<\/a> path, a guided route through the articles on Bayesian statistics and inference for SEO.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the article on the multi-armed bandit we used Bayes to decide between variants: shifting traffic toward the one converting best while the test is still running. Now we take a step sideways, while staying within the same line of reasoning: instead of choosing between options, we want to classify, that is to attach to &hellip; <a href=\"https:\/\/www.gironi.it\/blog\/en\/naive-bayes-search-intent\/\" class=\"more-link\">Leggi tutto<span class=\"screen-reader-text\"> &#8220;Naive Bayes: classifying search intent with Bayes&#8217; theorem&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","footnotes":""},"categories":[161],"tags":[],"class_list":["post-3902","post","type-post","status-publish","format-standard","hentry","category-statistics"],"lang":"en","translations":{"en":3902,"it":3901},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false,"post-thumbnail":false},"uagb_author_info":{"display_name":"Paolo Gironi","author_link":"https:\/\/www.gironi.it\/blog\/author\/autore-articoli\/"},"uagb_comment_info":2,"uagb_excerpt":"In the article on the multi-armed bandit we used Bayes to decide between variants: shifting traffic toward the one converting best while the test is still running. Now we take a step sideways, while staying within the same line of reasoning: instead of choosing between options, we want to classify, that is to attach to&hellip;","_links":{"self":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3902","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/comments?post=3902"}],"version-history":[{"count":4,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3902\/revisions"}],"predecessor-version":[{"id":4037,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3902\/revisions\/4037"}],"wp:attachment":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/media?parent=3902"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/categories?post=3902"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/tags?post=3902"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}