Desktop 5.5% CTR, mobile 2.5%. A difference that looks huge: but is it real, or could it vanish with another month of data?
To answer with numbers rather than gut feeling there is one of the most widely used tests of all, the chi-square test (χ²).
The chi-square test was proposed by Karl Pearson in 1900, and it is widely used to estimate how effectively the distribution of a categorical variable represents an expected distribution (in this case, we talk about the “Goodness of Fit Test”) or to estimate when two categorical variables are independent of each other (and then we talk about the “Test of Independence”).
Such is the importance and widespread use of this test that it was listed by the magazine Scientific American among the 20 most important scientific discoveries of the 20th century.
Continue reading “The Chi-Square Test: Goodness of Fit and Test of Independence”