First Steps into the World of Probability: Sample Space, Events, Permutations, and Combinations

How likely is it that a page reaches the first results page? Or that an A/B test gives a reliable answer? Questions like these — seemingly very different — share the same root: to answer them we need probability and combinatorics. In this post we lay the foundations, exploring together the basic tools that will accompany us throughout the whole path.

Probability

Probability is a mathematical measure that indicates the likelihood of an event occurring. In other words, probability is the ratio between the number of favorable cases and the number of possible cases.

Probability is based on two fundamental concepts: the sample space and the event.

The sample space is the set of all possible outcomes of a random experiment.
For example, if we flip a coin, the sample space is {heads, tails}. If we roll two dice, the sample space is {(1,1), (1,2), …, (6,6)}.

An event is a subset of the sample space that we are interested in.
For instance, if we flip a coin and we care whether it lands heads or tails, the event is {heads} or {tails}. If we roll two dice and want to know whether the sum of the numbers is even or odd, the event is {(2,2), (2,4), …, (6,6)} or {(1,2), (1,4), …, (5,6)}.

The probability of an event is calculated by dividing the number of favorable cases for the event by the number of possible cases in the sample space.

For example:

if we have a six-sided die and want to know the probability of getting a 4 by rolling it, we have 1 favorable case (the face with the number 4) out of 6 possible cases (the six faces of the die).
So the probability of getting a 4 is 1/6.

Other simple examples:

• The probability of getting heads when flipping a coin is 1/2
• The probability that the sum of the numbers is even when rolling two dice is 18/36 = 1/2

Probability is expressed as a number between 0 and 1, where 0 indicates the impossibility of the event and 1 indicates its certainty.

A probability value close to 0 indicates a low chance that the event occurs, while a value close to 1 indicates a high chance that it occurs.

The additivity principle for incompatible events

The additivity principle for incompatible events states that the probability of the union of two or more incompatible events is equal to the sum of their probabilities.

Incompatible events are events that cannot occur at the same time: if one occurs, the other cannot.
For example, in rolling a die, the events “rolling a 3” and “rolling a 5” are incompatible. In this case the probability of the union of the events (that is, rolling a 3 or a 5) equals the sum of their probabilities (1/6 + 1/6 = 1/3).

The multiplication principle for probabilities

The multiplication principle states that the probability of the intersection of two events equals the product of their individual probabilities, if the events are independent.

In other words, if A and B are two independent events in a probability experiment, then the probability that both occur at the same time is given by the product of their individual probabilities:

P(A ∩ B) = P(A) x P(B).

---

So far we have computed very simple probabilities: one die, two dice, a coin. But when the number of possible cases becomes large, counting them one by one is impossible. This is where combinatorics comes in: it lets us determine quickly how many favorable and how many possible cases exist, without having to list them all.

There are three fundamental tools in combinatorics: permutations, arrangements and combinations. The difference between them is simple: it depends on how many elements we choose and whether the order matters or not. Let’s keep this map in mind, then we’ll look at them one by one.

Permutation

Permutations are the ways in which n distinct objects can be ordered in n different positions. For example:

• The permutations of the letters A, B, C are ABC ACB BAC BCA CAB CBA
• The number of permutations of n distinct objects is calculated with the factorial n!, that is the product of the natural numbers from 1 to n
• The number of permutations of A, B, C is 3! = 3 x 2 x 1 = 6

Let’s see a few more examples:

How many different ways are there to arrange 4 books on a shelf?

The answer is: n!
Recall that “!” denotes the factorial, that is the product of all positive integers from 1 to n.

Solution: 4! (4 factorial) = 24 different ways

How many permutations are possible for a set of 5 letters?

5! = 5 x 4 x 3 x 2 x 1 = 120

There are 120 possible permutations of 5 letters.

Arrangements

So far we have ordered all the elements. If instead we choose only some elements from a set and the order matters, we speak of simple arrangements. The formula is n! / (n – r)!, where n is the total number of elements and r is how many we choose. Arrangements are an intermediate case between permutations and combinations: we do not use all the available elements, only a part of them, while keeping the order meaningful.

How many arrangements are possible from 5 letters taken in groups of 3?

n! / (n – r)!

where “n” is the total number of objects (in this case, the 5 letters), and “r” is the number of objects we want to choose and arrange in a specific order (in this case, 3 letters).

So, substituting the values, we get:

5! / (5 – 3)! = 5! / 2! = (5 x 4 x 3 x 2 x 1) / (2 x 1) = 60

So there are 60 possible arrangements of 5 letters taken in groups of 3. The order matters: ABC is different from ACB, which is different from BAC, and so on.

It is important to note that, when we choose a group of objects from a larger set, the order in which the objects are chosen matters. If the order matters we are in the case of arrangements; if instead it did not matter, we would use the combinations formula.

The concept of Combination

Combinations are the ways in which k objects can be chosen from n distinct objects without considering the order. For example,

• The combinations of two letters from A, B, C are AB AC BC
• The number of combinations of k objects from n distinct objects is calculated with the binomial coefficient C(n,k) = n! / (k! x (n-k)!)
• The number of combinations of two letters from A, B, C is C(3, 2) = 3! / (2! x (3-2)!) = 3

Let’s see a few more examples:

How many combinations are possible for a set of 10 people taken in groups of 3?

To calculate the number of combinations possible for a set of 10 people taken in groups of 3, we can use the combinations formula:

n! / (k! * (n – k)!)

where “n” is the total number of objects (in this case, the 10 people) and “k” is the number of objects we want to choose without caring about the order (in this case, 3 people).

So, substituting the values, we get:

10! / (3! * (10 – 3)!) = (10 x 9 x 8 x 7 x 6 x 5 x 4 x 3 x 2 x 1) / ((3 x 2 x 1) * (7 x 6 x 5 x 4 x 3 x 2 x 1)) = 120

A class is made up of 12 boys and 4 girls. Three of the sixteen students are chosen at random: what is the probability that they are all boys?

This example is taken from the Italian State Exam, mathematics topic no. 1 (PNI, school year 2000-2001, standard course, Scientific High School).

The probability of choosing three all-boy students can be computed as the ratio between the number of ways we can choose three boys and the total number of ways we can choose three students out of all sixteen.
If we want to choose three all-boy students, we must consider all the possible groups of 3 boys that can be formed by choosing them from the 12 boys.

The number of ways we can choose three boys from the class of 12 boys is given by the combination of 3 elements chosen from the 12 boys. Let’s compute it with the combinations formula:

C(12, 3) = 12! / (3! * (12-3)!) = 220

The total number of ways we can choose three students from the class of 16 is given by the combination of 3 elements chosen from the 16 students.

C(16, 3) = 16! / (3! * (16-3)!) = 560

So the probability of choosing three all-boy students is:

P(three boys) = C(12, 3) / C(16, 3) = 220 / 560 = 11 / 28

A concrete example: choosing the keywords for a landing page

Let’s now look at an example that perhaps touches more closely the work of those who deal with data analysis and SEO. Suppose we have a list of 15 keywords and we want to choose 3 of them as the focus of a landing page.

Scenario 1: I only care which keywords to choose
The order does not matter: the group {technical SEO, content optimization, link building} is the same as {link building, technical SEO, content optimization}.
This is a combination: C(15, 3) = 455 different ways.

Scenario 2: I also want to set the order in the title
If the first keyword must go in the title, the second in the H1 and the third in the body, then the order matters. The same group of keywords in a different order produces a different page.
This is an arrangement: 15! / (15-3)! = 2730 different ways.

The difference is substantial: 455 possibilities if the order does not matter, 2730 if it does. This is why it is essential to understand first of all whether the order is relevant in the problem we are facing.

A common mistake
Many students — and not only students — confuse permutations and combinations. The simplest criterion to tell them apart is to ask: if I swap the order of the elements, do I get a different result?

If yes, the order matters: we are working with permutations or arrangements.
If no, the order does not matter: we are in the case of combinations.

This single question is all we need in order not to go wrong.

How to choose the right formula

Do I need to order all the elements? → Permutations: n!
Do I need to choose some and the order matters? → Arrangements: n!/(n-r)!
Do I need to choose some and the order does not matter? → Combinations: C(n,k)

With this scheme, tackling any combinatorial problem becomes much simpler: just identify the type of question and apply the corresponding formula.

The binomial distribution as an application of probability and combinatorics

In a post specifically dedicated to probability distributions I examined in detail the properties of the binomial distribution. I refer you to that post for all the details.
Here, though, I would like to introduce it briefly in a direct and practical way, only to answer questions like:

• What is the probability that in 10 coin tosses heads comes up 5 times or fewer?
• If in 20 multiple-choice questions (each with 4 options) we answer completely at random, what is the probability of answering 15 or more correctly?
• In 100 draws with replacement from an urn with 10 white balls and 90 black ones, what is the probability of drawing fewer than 20 white?

Let’s take the first question. We want to know the probability that in 10 coin tosses heads comes up 5 times or fewer.
Proceeding by logic, we should compute the sum of the binomial probabilities for k = 0, 1, 2, 3, 4 and 5. That is:

P(X <= 5) = P(X = 0) + P(X = 1) + P(X = 2) + P(X = 3) + P(X = 4) + P(X = 5)

Using the binomial probability formula and substituting n = 10 and p = 1/2, we get:

P(X <= 5) = C(10 ,0) x (1/2)^0 x (1/2)^10 + C(10 ,1) x (1/2)^1 x (1/2)^9 + … + C(10 ,5) x (1/2)^5 x (1/2)^5

Simplifying the calculations and using a calculator, we get:

P(X <= 5) = <0.001 + <0.01 + <0.04 + <0.12 + <0.21 + <0.25

P(X <= 5) = 0.63

So the probability that in 10 coin tosses heads comes up at most five times is about 63%.

Is there a simpler way to reach the correct result?

We can introduce the cumulative distribution function of the binomial distribution.

The cumulative distribution function computes the probability that the random variable X is less than or equal to a certain value k. It is written F(k) and defined as:

F(k) = P(X <= k) = sum of the binomial probabilities for i = 0, 1, …, k

This function can be computed with an approximate formula or with a precompiled table. For example, using an online table like this one:

https://www.statisticshowto.com/tables/binomial-distribution-table/

where we can find the value of F(5) for n = 10 and p = 1/2.

We just look in the row corresponding to n = 10 and the column corresponding to p = 0.5 and read the value at k = 5. The value is 0.623.

Of course, it is much more convenient to use R or Python, especially for larger numbers.

Let’s compute it in R with the pbinom function, which returns the cumulative probability of a given number of successes in a given number of trials:

# Probability of getting 5 or fewer heads in 10 tosses
pbinom(5, size = 10, prob = 0.5)
# Result: 0.6230469

The same calculation in Python uses the binom class from the scipy.stats library:

# Import the library
from scipy.stats import binom

# Probability of getting 5 or fewer heads in 10 tosses
binom.cdf(5, n = 10, p = 0.5)
# Result: 0.623046875

Let’s move on to the second question.

We want to compute the probability of answering 15 or more of 20 multiple-choice questions correctly. If we assume each question has 4 options and only one is correct, then the probability of success is p = 0.25. So we must compute:

P(X >= 15) = P(X = 15) + P(X = 16) + P(X = 17) + P(X = 18) + P(X = 19) + P(X = 20)

The binomial probability formula is:

\( P(X = x) = {n choose x} p^x (1-p)^{n-x} \ \)

Using the binomial probability formula, we get:

\( P(X geq 15) approx 0.0002 \)

So the probability is very, very low… better to study!

Let’s move on to the third question.

We want to find the probability of drawing fewer than 20 white balls in 100 draws from an urn with 10 white balls and 90 black ones. If we assume the draws are with replacement, then the probability of success (drawing a white ball) is p = 0.1. So we must compute:

$$ P(X < 20) = P(X leq 19) = sum_{x=0}^{19} {100 choose x} (0.1)^x (0.9)^{100-x} $$ and we get: $$ P(X < 20) approx 0.9988 $$

So in this case the probability is very high.

These are the basics — sample space, events, permutations, arrangements, combinations, and a first taste of the binomial. Tools that look simple, yet much of the statistics we use in everyday work rests on them. In the next article on the binomial distribution we look more closely at this distribution, seeing how to compute it and when to apply it — and we will discover it is not nearly as complex as it might seem at first sight.

You might also like

• Bayesian statistics: how to learn from data, one step at a time

---

Further Reading

Two complementary reads to consolidate probabilistic reasoning: The Art of Statistics by David Spiegelhalter for the conceptual framework, and Finalmente ho capito la statistica by Maurizio De Pra (Italian edition) for a gradual, example-rich path.