Paolo Gironi - notes on data analysis, SEO, statistics and retro computing

The Gradient Descent Algorithm Explained Clearly: From Intuition to Practice

A blindfolded person on a mountain

Imagine standing on a mountainous terrain, completely blindfolded. Your goal: reach the lowest point in the valley. You can’t see anything, but you can feel the slope of the ground beneath your feet. What do you do? You move in the direction where the ground goes down, one step at a time. If it slopes more steeply to the left, you go left. If it drops more to the right, you go right. With each step, you feel the slope again and redirect yourself.

This strategy, so simple and natural, is exactly what neural networks use to learn. Every time an AI model improves — learning to recognize a face, translate a sentence, or generate text — it does so by descending through a mathematical landscape, one step at a time, following the slope.

It’s called gradient descent, and it’s arguably the most important algorithm in modern machine learning.

Infographic: the blindfolded explorer metaphor for gradient descent, with three steps: Sensor, Action, Cycle

The Hypergeometric Distribution

We have seen that the binomial distribution is based on the hypothesis of an infinite population N, a condition that can be practically realized by sampling from a finite population with replacement.

If this does not occur, meaning if we are sampling from a population without replacement, we must use the hypergeometric distribution. (In reality, if N is large, the hypergeometric probability density function tends towards the binomial).

The hypergeometric distribution is used to calculate the probability of obtaining a certain number of successes in a series of binary trials (yes or no), which are dependent and have a variable probability of success.

The hypergeometric distribution allows us to answer questions like:

If I take a sample of size N, in which M elements meet certain requirements, what is the probability of drawing x elements that meet those requirements?

The Negative Binomial Distribution (or Pascal Distribution)

The negative binomial distribution describes the number of trials needed to achieve a certain number of successes in a series of independent trials. For example, it answers the question: flipping a balanced coin (50% chance of heads on each flip), what is the probability that the third head comes up exactly on the fifth flip?

The negative binomial distribution is useful in many fields, including statistics, economics, biology, and physics. And also in “our” SEO.

(If instead we want the probability of a certain number of successes in a fixed number of trials, say three heads in five flips, the right tool is the binomial distribution: the negative binomial turns the question around.)

Probability: Sample Space, Events, Permutations and Combinations

How likely is it that a page reaches the first results page? Or that an A/B test gives a reliable answer? Questions like these — seemingly very different — share the same root: to answer them we need probability and combinatorics. In this post we lay the foundations, starting from the sample space and exploring together the basic tools that will accompany us throughout the whole path.

Logistic Regression: What It Is, How to Read Odds Ratios, and an Example in R

Sooner or later, every traffic analysis arrives at the question that really matters: did this visit turn into a customer, yes or no? We don’t care how much a session converted, we care whether it did. And right behind it comes the next question: which sessions are more likely to convert? The ones arriving from a search for the brand name? The ones browsing more pages? The ones on desktop?

With linear regression we learned to predict a number. Here, though, the outcome is a switch, on or off, and a straight line is not the right tool: we’ll see it even ends up predicting negative probabilities. The right tool is logistic regression, one of the most widely used models of all, from marketing to medicine. Let’s build it together, starting from an example with conversions.