Descriptive Statistics for Machine Learning

Introduction

Descriptive statistics is the set of tools we use to summarise, organise and describe a dataset. Instead of staring at thousands of raw numbers, we compute a few meaningful values (an average, a spread, a shape) or draw a chart that tells the story of the data.

Descriptive statistics only describes the data you have. It does not try to make claims about a larger population; that is the job of inferential statistics (the next lesson).

In machine learning, descriptive statistics is always the first step. Before training any model you should know the typical values of your features, how spread out they are, whether there are outliers or missing values, and how different features relate. This is called exploratory data analysis (EDA).

Types of data

Knowing the type of a variable decides which statistics make sense.

Type	Sub-type	Description	Examples
Categorical	Nominal	Categories with no order	Colour, city, blood group
	Ordinal	Categories with a natural order	Rating (poor, fair, good), education level
Numerical	Discrete	Countable numbers	Number of children, clicks
	Continuous	Any value in a range	Height, temperature, price

Numerical data is further described by its scale: interval (differences are meaningful, no true zero, e.g. temperature in °C) and ratio (true zero exists, ratios are meaningful, e.g. weight or income).

You can compute a mean for numerical data but not for nominal data; for nominal data the natural summary is the mode or a frequency table.

Measures of central tendency

These describe the "typical" or "central" value.

Mean (arithmetic average)

x̄ = (x₁ + x₂ + ... + xₙ) / n = Σxᵢ / n

Uses every value, so it is efficient, but it is sensitive to outliers.
Weighted mean: when values matter unequally, x̄w = Σ(wᵢxᵢ) / Σwᵢ. A grade point average is a weighted mean.

Median

The middle value after sorting.

Odd n: the value in position (n + 1)/2.
Even n: the average of the two middle values.
Robust to outliers, so it is the better "typical value" for skewed data such as income or house prices.

Mode

The most frequent value. A dataset can have no mode, one mode (unimodal), or several (bimodal, multimodal). The mode is the only average that works for nominal data.

Which one should I use?

Situation	Best measure
Symmetric numerical data without outliers	Mean
Skewed data or data with outliers	Median
Categorical data	Mode
Values with different importance	Weighted mean

Measures of position

Percentile: the value below which a given percentage of the data falls. The 90th percentile means 90% of values are lower.
Quartiles: Q1 (25th percentile), Q2 (50th, the median) and Q3 (75th).
Five-number summary: minimum, Q1, median, Q3, maximum. It is what a box plot draws.

Measures of dispersion (spread)

Two datasets can have the same mean but very different spread, so you must always report both.

Measure	Formula	Notes
Range	max − min	Very sensitive to outliers
Interquartile range (IQR)	Q3 − Q1	Spread of the middle 50%, robust
Variance	σ² = Σ(x − μ)² / n	Average squared distance from the mean
Standard deviation	σ = √variance	Same units as the data
Coefficient of variation	CV = (σ / mean) × 100%	Compares spread across different units or scales

Population versus sample

If you have the entire population: divide by n.
If you have only a sample: divide by n − 1. This is Bessel's correction. Because a sample's values are closer to the sample mean than to the true mean, dividing by n would underestimate the variance.

Population variance:  σ² = Σ(xᵢ − μ)² / n
Sample variance:      s²  = Σ(xᵢ − x̄)² / (n − 1)

Detecting outliers

An outlier is a value far from the rest of the data. Two common rules:

IQR rule: a value is an outlier if it is below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR. (This is what box plots use.)
Z-score rule: compute z = (x − mean) / standard deviation. Values with |z| > 3 (sometimes > 2.5) are flagged.

Outliers are not automatically errors. They may be data-entry mistakes, or they may be the most interesting observations (fraud, faults). Investigate before removing.

Shape of the data

Symmetric: mean ≈ median.
Right-skewed: long tail to the right, mean > median.
Left-skewed: long tail to the left, mean < median.

Skewness and kurtosis are studied in detail in their own lesson.

Summarising and visualising

Chart	Best for
Histogram	Distribution of one numerical variable
Box plot	Median, quartiles and outliers; comparing groups
Bar chart	Frequencies of categories
Scatter plot	Relationship between two numerical variables
Frequency table	Counts and percentages of each value

Real-life applications

Business: average order value, monthly revenue variability.
Education: class average and the spread of exam scores.
Healthcare: median recovery time, blood-pressure ranges.
Sports: batting averages, consistency of a player's scores.
Government: census summaries, average household income (usually reported as a median).

Where it appears in machine learning

EDA: describe(), histograms and box plots at the start of every project.
Standardisation: z = (x − mean) / std puts every feature on the same scale, which helps k-NN, SVM, PCA and gradient descent.
Imputation: missing values are often replaced with the mean or median.
Outlier handling: clipping or removing extreme values using the IQR rule.
Baselines: predicting the mean (regression) or the mode (classification) is the simplest baseline model.

Python implementation

import numpy as np
import pandas as pd
from scipy import stats

data = np.array([12, 15, 11, 18, 20, 15, 14])

print("Mean:", np.mean(data))
print("Median:", np.median(data))
print("Mode:", stats.mode(data, keepdims=False).mode)

print("Population variance:", np.var(data))            # divides by n
print("Sample variance:", np.var(data, ddof=1))        # divides by n - 1
print("Sample std:", np.std(data, ddof=1))

q1, q3 = np.percentile(data, [25, 75])
iqr = q3 - q1
print("IQR:", iqr)
print("Outlier fences:", q1 - 1.5 * iqr, q3 + 1.5 * iqr)

# pandas gives everything at once
df = pd.DataFrame({"score": data})
print(df.describe())

Watch out: NumPy's var and std use ddof=0 by default (population), while pandas uses ddof=1 (sample). Always check which one you need.

Standardising a feature

from sklearn.preprocessing import StandardScaler

X = np.array([[50_000], [60_000], [80_000], [120_000]])
X_scaled = StandardScaler().fit_transform(X)
print(X_scaled.mean(), X_scaled.std())   # about 0 and 1

Solved examples

Example 1: Mean, median, mode and variance

Find the mean, median, mode, variance and standard deviation of: 12, 15, 11, 18, 20, 15, 14 (treat as a sample).

Solution. Sorted: 11, 12, 14, 15, 15, 18, 20 (n = 7).

Mean = 105 / 7 = 15
Median = the 4th value = 15
Mode = 15 (appears twice)
Deviations from the mean: −4, −3, −1, 0, 0, 3, 5. Squares: 16, 9, 1, 0, 0, 9, 25. Sum = 60.
Population variance = 60 / 7 ≈ 8.57
Sample variance = 60 / 6 = 10
Sample standard deviation = √10 ≈ 3.16

Example 2: Median with an even number of values

Find the mean and median of 3, 7, 8, 10, 12, 20.

Solution. n = 6. Mean = 60 / 6 = 10. The two middle values are 8 and 10, so median = (8 + 10)/2 = 9.

Example 3: Weighted mean

A course grade is 70% exam and 30% project. A student scores 80 on the exam and 90 on the project. What is the final grade?

Solution. Weighted mean = 0.7 × 80 + 0.3 × 90 = 56 + 27 = 83.

Example 4: Quartiles, IQR and outliers

Data: 4, 7, 9, 10, 12, 15, 18, 21, 50. Does it contain an outlier?

Solution. Median = 12. Using the "median of each half" method: lower half (4, 7, 9, 10) gives Q1 = (7 + 9)/2 = 8, upper half (15, 18, 21, 50) gives Q3 = (18 + 21)/2 = 19.5.

IQR = 19.5 − 8 = 11.5. Upper fence = 19.5 + 1.5 × 11.5 = 36.75. Since 50 > 36.75, it is an outlier.

(NumPy's default linear interpolation gives Q1 = 9 and Q3 = 18, so IQR = 9 and the upper fence is 31.5. Different tools use slightly different quartile methods, but 50 is flagged either way.)

Example 5: Z-scores

A test has mean 60 and standard deviation 8. Find the z-scores of a student who scored 76 and one who scored 52.

Solution. z = (76 − 60)/8 = 2.0 and z = (52 − 60)/8 = −1.0. The first student is two standard deviations above the mean; the second is one below.

Example 6: Coefficient of variation

Product A costs on average 50 with a standard deviation of 5. Product B costs on average 200 with a standard deviation of 16. Which price is relatively more variable?

Solution. CV(A) = 5/50 = 10%. CV(B) = 16/200 = 8%. Product A is relatively more variable, even though B has a larger standard deviation in absolute terms.

Common mistakes

Reporting a mean without any measure of spread.
Using the mean for heavily skewed data (use the median).
Mixing up population (÷ n) and sample (÷ n − 1) formulas.
Removing outliers automatically without checking whether they are real.
Computing a mean of ordinal categories (such as the "average" of ratings) without thinking about what it means.

Practice problems

Find the mean and median of 5, 8, 9, 12, 16.
Find the mean, median and mode of 2, 3, 3, 5, 7, 7, 7, 10.
Compute the sample variance and standard deviation of 4, 8, 6, 5, 7.
Scores have mean 500 and standard deviation 100. What is the z-score of 650?
A student takes three courses with 4, 3 and 2 credits and gets grade points 3.5, 3.0 and 4.0. Compute the weighted GPA.
Q1 = 20 and Q3 = 30 for a dataset. Is a value of 48 an outlier by the IQR rule?

Answers

Mean = 50/5 = 10; median = 9
Mean = 44/8 = 5.5; median = (5 + 7)/2 = 6; mode = 7
Mean = 6; squared deviations sum to 10; sample variance = 2.5; standard deviation ≈ 1.58
(650 − 500)/100 = 1.5
(4×3.5 + 3×3.0 + 2×4.0)/9 = 31/9 ≈ 3.44
IQR = 10; upper fence = 30 + 15 = 45. Since 48 > 45, yes, it is an outlier.

Key takeaways

Descriptive statistics summarises data with measures of centre, spread, position and shape.
Use the median and IQR for skewed data, the mean and standard deviation for symmetric data.
Sample variance divides by n − 1; watch the defaults of your software.
The IQR rule and z-scores help detect outliers.
Always explore your data with statistics and charts before you train a model.

Descriptive Statistics