{"id":3260,"date":"2024-01-11T15:50:29","date_gmt":"2024-01-11T14:50:29","guid":{"rendered":"https:\/\/www.gironi.it\/blog\/?p=3260"},"modified":"2026-07-14T14:46:47","modified_gmt":"2026-07-14T13:46:47","slug":"how-to-use-decision-trees-to-classify-data","status":"publish","type":"post","link":"https:\/\/www.gironi.it\/blog\/en\/how-to-use-decision-trees-to-classify-data\/","title":{"rendered":"How to Use Decision Trees to Classify Data"},"content":{"rendered":"\n<div class=\"wp-block-group has-background\" style=\"background-color:#f5f7f9;margin-top:2.5rem;margin-bottom:2.5rem;padding-top:1.5rem;padding-right:1.5rem;padding-bottom:1.5rem;padding-left:1.5rem\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p><strong>In this article<\/strong>:<\/p>\n<ul>\n<li><a href=\"#general-idea\">The general idea<\/a><\/li>\n<li><a href=\"#gini\">Impurity and split criteria<\/a><\/li>\n<li><a href=\"#r-example\">A decision tree in R with the iris data<\/a><\/li>\n<li><a href=\"#seo-example\">An SEO example: predicting top 10 pages<\/a><\/li>\n<li><a href=\"#accuracy\">Evaluating accuracy<\/a><\/li>\n<li><a href=\"#overfitting\">Overfitting and pruning<\/a><\/li>\n<li><a href=\"#feature-importance\">Variable importance<\/a><\/li>\n<li><a href=\"#random-forest\">From single trees to forests: Random Forest<\/a><\/li>\n<li><a href=\"#faq\">FAQ<\/a><\/li>\n<\/ul>\n<\/div><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">When we analyse a dataset with many variables and want to predict a category \u2014 whether a user will convert or not, whether a page will reach the top 10, whether a keyword is high or low competition \u2014 the first problem is figuring out which variables actually matter and how to combine them. <br> <strong>Decision trees<\/strong> tackle this problem in the most natural way possible: a sequence of yes\/no questions that, step by step, separate the data into increasingly homogeneous groups.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"general-idea\">The general idea<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A decision tree is a structure that makes decisions sequentially. Each <strong>internal node<\/strong> asks a question (for instance: &#8220;is petal length greater than 2.45 cm?&#8221;), and depending on the answer it follows a branch to the next node. Eventually we reach a <strong>leaf<\/strong>, which assigns the predicted class.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The power of the idea is that the result is <strong>intrinsically interpretable<\/strong>: you can understand why the tree made a particular decision by simply following the path from root to leaf. No black box.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"gini\">Impurity and split criteria<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">How does the tree decide which question to ask first? The goal is to find the split that separates the data as &#8220;cleanly&#8221; as possible. The core concept is <strong>impurity<\/strong>: a perfect split is one where all elements in a group belong to the same class.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The most common measure is the <strong>Gini index<\/strong>. For a node with \\( K \\) classes, where \\( p_k \\) is the proportion of elements in class \\( k \\):<\/p>\n\n\n\n\\( G = \\sum_{k=1}^{K} p_k (1 &#8211; p_k) \\\\ \\)\n\n\n\n<p class=\"wp-block-paragraph\">The index is 0 when the node is pure (all elements belong to the same class), and reaches its maximum when classes are uniformly distributed. <br> The tree tries all possible splits on all variables and chooses the one that most reduces the weighted average impurity of the two child nodes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Other measures exist, such as <strong>entropy<\/strong> (used in C4.5 trees), but the idea is the same: minimise uncertainty.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"r-example\">A decision tree in R with the iris data<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We build a decision tree using the <code>iris<\/code> dataset, which contains measurements of sepals and petals for 150 flowers of three species. Our goal is to classify the species based on the measurements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s prepare the data:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>data(iris)\nlibrary(rpart)\n\nset.seed(123)\ntrain_idx &lt;- sample(1:nrow(iris), 0.8 * nrow(iris))\ntrain &lt;- iris[train_idx, ]\ntest  &lt;- iris[-train_idx, ]\n\ntree &lt;- rpart(Species ~ ., data = train, method = \"class\")<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Visualise the tree:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>library(rpart.plot)\nrpart.plot(tree, type = 2, extra = 104)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1050\" height=\"675\" src=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-decisionali-iris-en.png\" class=\"wp-image-4269\" alt=\"Decision tree: classification of iris species\" srcset=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-decisionali-iris-en.png 1050w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-decisionali-iris-en-300x193.png 300w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-decisionali-iris-en-1024x658.png 1024w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption class=\"wp-element-caption\">Decision tree: classification of iris species<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The tree is very simple. It starts at the root with a question about petal length. If it is less than 2.45 cm, we can be certain the species is <em>setosa<\/em> (all 41 training cases go there). If it is greater, we move to the next branch, which separates <em>versicolor<\/em> from <em>virginica<\/em> again based on petal length, this time with a threshold of 4.75 cm.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Notice that the tree did not use sepal measurements at all: it found them irrelevant for classification and ignored them automatically. This is an implicit form of <strong>variable selection<\/strong>: the tree chooses which metrics matter, and in which order.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"seo-example\">An SEO example: predicting top 10 pages<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Iris is fine for understanding the mechanism, but let&#8217;s try something closer to our daily work. We generate synthetic data for 200 web pages with real SEO metrics \u2014 word count, readability, speed, backlinks, images, keyword in title, meta description length \u2014 and build a tree that predicts whether a page will reach the top 10.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>set.seed(2026)\n\nn &lt;- 200\nword_count      &lt;- round(runif(n, 300, 3000))\nreadability     &lt;- round(runif(n, 20, 80), 1)\npage_speed_ms   &lt;- round(runif(n, 500, 8000))\nbacklinks       &lt;- round(runif(n, 0, 150))\nimages          &lt;- round(runif(n, 0, 20))\nkeyword_in_title &lt;- rbinom(n, 1, 0.35)\nmeta_desc_len   &lt;- round(runif(n, 0, 165))\n\n# simulated target: top_10 based on a combination of factors\nscore &lt;- (\n  (backlinks &gt; 30) * 0.30 +\n  (keyword_in_title == 1) * 0.20 +\n  (readability &gt; 45 &amp; readability &lt; 70) * 0.15 +\n  (word_count &gt; 800 &amp; word_count &lt; 2200) * 0.15 +\n  (page_speed_ms &lt; 3000) * 0.10 +\n  (meta_desc_len &gt; 110 &amp; meta_desc_len &lt; 155) * 0.10\n)\nscore &lt;- score + rnorm(n, 0, 0.12)\ntop_10 &lt;- factor(ifelse(score &gt; 0.50, \"yes\", \"no\"))\n\nseo_data &lt;- data.frame(word_count, readability, page_speed_ms,\n                       backlinks, images, keyword_in_title,\n                       meta_desc_len, top_10)<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The target variable <code>top_10<\/code> is constructed so that it depends on all these factors, each with a different weight \u2014 backlinks are the most influential, followed by the presence of the keyword in the title, readability in the right range, and so on. We also add a random component to simulate what we cannot measure.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Train the tree:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>library(rpart)\ntrain_idx &lt;- sample(1:n, 0.7 * n)\ntree_seo &lt;- rpart(top_10 ~ ., data = seo_data[train_idx, ],\n                  method = \"class\", control = rpart.control(cp = 0.01))<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Visualise the resulting tree:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>library(rpart.plot)\nrpart.plot(tree_seo, type = 2, extra = 104)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1125\" height=\"750\" src=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-seo-albero-en.png\" class=\"wp-image-4275\" alt=\"Decision tree: predicting top 10 pages\" srcset=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-seo-albero-en.png 1125w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-seo-albero-en-300x200.png 300w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-seo-albero-en-1024x683.png 1024w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption class=\"wp-element-caption\">Decision tree: predicting top 10 pages<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The tree tells us something very interesting. The first question \u2014 the most discriminant variable \u2014 is about <strong>backlinks<\/strong>: if they are few, the page can hardly reach the top 10, regardless of other factors. Only pages with enough backlinks move to the next branch, where the tree evaluates <strong>readability<\/strong> and <strong>keyword presence in the title<\/strong>. This is exactly the kind of hierarchy we would expect from a serious SEO analysis: first authority (backlinks), then on-page quality.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Feature importance<\/strong> confirms this hierarchy:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>tree_seo$variable.importance<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"975\" height=\"600\" src=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-seo-importance-en.png\" class=\"wp-image-4277\" alt=\"SEO variable importance\" srcset=\"https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-seo-importance-en.png 975w, https:\/\/www.gironi.it\/blog\/wp-content\/uploads\/2026\/07\/alberi-seo-importance-en-300x185.png 300w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption class=\"wp-element-caption\">SEO variable importance<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Backlinks dominate, as expected. Readability and keyword in title come next, while speed and meta description length contribute less. The tree did not select the number of images as relevant \u2014 in our synthetic data it was not, and the tree ignored it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">n.b. These data are synthetic: in a real project with real data, the tree would reveal the actual hierarchy of variables for your specific context. The example is meant to show the kind of answer you get, not to establish universal truths.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>An important caveat<\/strong>: the tree identifies recurring patterns in observed data, not cause-effect relationships. A variable can be very useful for predicting a page&#8217;s behaviour without being its direct cause. For instance, if the tree selects CTR as a discriminant variable, it does not mean that increasing CTR will push the page into the top 10 \u2014 it could be that pages already in the top 10 tend to have higher CTR precisely because they are in the top 10. The tree tells us what is correlated, not what is causal.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"accuracy\">Evaluating accuracy<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s check how it performs on the test data:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pred &lt;- predict(tree, newdata = test, type = \"class\")\ntable(pred, test$Species)<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The contingency table (confusion matrix) shows how many predictions are correct and how many are not. Accuracy is calculated as:<\/p>\n\n\n\n\\( \\text{accuracy} = \\frac{\\text{correct predictions}}{\\text{total predictions}} \\\\ \\)\n\n\n\n<p class=\"wp-block-paragraph\">Our tree makes very few mistakes: it is a simple model and the iris data are well separated. In the real world, things are almost always more complex.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"overfitting\">Overfitting and pruning<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Here is the delicate part. A decision tree can keep splitting until every leaf contains a single element. In that case, accuracy on the training set will be 100%. But the tree will have simply <strong>memorised the training dataset<\/strong> instead of learning its regularities.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This phenomenon is called <strong>overfitting<\/strong>: the model adapts too closely to the training data, capturing noise and specific details that will not appear in new data.<\/p>\n\n\n\n<div class=\"wp-block-group has-background\" style=\"background-color:#f5f7f9;margin-top:2.5rem;margin-bottom:2.5rem;padding-top:1.5rem;padding-right:1.5rem;padding-bottom:1.5rem;padding-left:1.5rem\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>Overfitting is the main problem with decision trees. A tree that is too deep memorises the training set but fails on new data; one that is too shallow cannot capture the real structure of the data. The optimal balance is found through cross-validation and <strong>pruning<\/strong>: building an intentionally large tree and then cutting the branches that do not improve prediction error.<\/p>\n<\/div><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">In R, you can prune with <code>rpart<\/code> using the <code>cp<\/code> parameter (complexity parameter), which prevents the tree from making splits that do not reduce error by at least a certain threshold. The <code>printcp(tree)<\/code> function shows cross-validated error for different tree sizes, helping you choose the cut-off point.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"feature-importance\">Variable importance<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">One of the most useful byproducts of a decision tree is <strong>feature importance<\/strong>: a measure of how much each variable contributes to the overall reduction in impurity. In <code>rpart<\/code>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>tree$variable.importance<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">This tells us, for example, that petal length is by far the most discriminant variable for iris \u2014 and sepal measurements count for little or nothing. <br> In web marketing, a similar analysis can reveal which metrics truly separate high-performing pages from low-performing ones.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"random-forest\">From single trees to forests: Random Forest<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A single decision tree is unstable: a small change in the input data can produce a completely different tree. To reduce this problem, <strong>Random Forests<\/strong> build hundreds of trees on slightly different versions of the data (bootstrap sampling) and average their predictions. The result is much more stable and accurate than a single tree, while retaining much of its interpretability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If a single tree is like asking one expert for their opinion, a Random Forest is like consulting hundreds of independent experts and going with the majority.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We will cover this in a future article.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"faq\">FAQ<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>When should I use a decision tree instead of logistic regression?<\/strong><br> When the data have complex interactions between variables (for example, the effect of age on conversion changes depending on the acquisition channel), and when interpretability is the priority.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Can a tree handle numerical and categorical variables together?<\/strong><br> Yes, <code>rpart<\/code> handles both types natively. Categorical variables are internally converted to binary splits.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How deep should a tree be?<\/strong><br> It depends on the data. The rule of thumb is to use cross-validation (the <code>plotcp()<\/code> function in <code>rpart<\/code>) to find the depth that minimises prediction error on unseen data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Does the decision tree work for regression too?<\/strong><br> Yes. Using <code>method = \"anova\"<\/code> instead of <code>\"class\"<\/code>, the tree predicts numerical values (for example, expected time on page).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why did the tree only use petal length?<\/strong><br> Because in the iris data it is the most discriminant variable: it alone suffices to separate one of the three species and almost separates the other two. The tree, being a greedy algorithm, chooses the variable that gives the best immediate split.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\">From decision trees to <a href=\"https:\/\/www.gironi.it\/blog\/lalgoritmo-di-discesa-del-gradiente-gradient-descent-spiegato-semplice\/\">gradient descent<\/a> and dimensionality reduction, there is a common thread: all solve the problem of giving structure to complex data, each with its own approach. Our next step in this journey is <strong>gradient descent<\/strong>, the engine that powers much of modern machine learning.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Further reading<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If you want to dive deeper into decision trees, <a href=\"https:\/\/www.amazon.it\/dp\/1461471370?tag=consulenzeinf-21\" rel=\"nofollow sponsored noopener\" target=\"_blank\"><em>An Introduction to Statistical Learning<\/em><\/a> by James, Witten, Hastie and Tibshirani is the reference: it covers trees, Random Forests and split criteria with the right balance of intuition and formalism. The R labs are freely available online.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this article: The general idea Impurity and split criteria A decision tree in R with the iris data An SEO example: predicting top 10 pages Evaluating accuracy Overfitting and pruning Variable importance From single trees to forests: Random Forest FAQ When we analyse a dataset with many variables and want to predict a category &hellip; <a href=\"https:\/\/www.gironi.it\/blog\/en\/how-to-use-decision-trees-to-classify-data\/\" class=\"more-link\">Leggi tutto<span class=\"screen-reader-text\"> &#8220;How to Use Decision Trees to Classify Data&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","footnotes":""},"categories":[161],"tags":[1189],"class_list":["post-3260","post","type-post","status-publish","format-standard","hentry","category-statistics","tag-decision-trees"],"lang":"en","translations":{"en":3260,"it":3065},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false,"post-thumbnail":false},"uagb_author_info":{"display_name":"paolo","author_link":"https:\/\/www.gironi.it\/blog\/author\/paolo\/"},"uagb_comment_info":5,"uagb_excerpt":"In this article: The general idea Impurity and split criteria A decision tree in R with the iris data An SEO example: predicting top 10 pages Evaluating accuracy Overfitting and pruning Variable importance From single trees to forests: Random Forest FAQ When we analyse a dataset with many variables and want to predict a category&hellip;","_links":{"self":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3260","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/comments?post=3260"}],"version-history":[{"count":5,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3260\/revisions"}],"predecessor-version":[{"id":4283,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/posts\/3260\/revisions\/4283"}],"wp:attachment":[{"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/media?parent=3260"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/categories?post=3260"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gironi.it\/blog\/wp-json\/wp\/v2\/tags?post=3260"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}