Multivariate Analysis
Multivariate data has three or more variables for each observation. Real-world outcomes rarely depend on just one or two factors, so multivariate analysis is the backbone of predictive analytics, econometrics and machine learning.
Goals
- Understand how several variables jointly affect an outcome.
- Separate the effect of one variable from the others ("holding others constant").
- Reduce complexity when there are too many variables.
- Find groups (clusters) of similar observations.
Common techniques
| Technique | Purpose |
|---|---|
| Multiple linear / logistic regression | Predict an outcome from many inputs |
| MANOVA | Compare group means on several dependent variables |
| PCA | Reduce many correlated variables to a few components |
| Clustering (k-means, hierarchical) | Discover natural groups |
| Factor analysis | Find hidden factors behind observed variables |
| Heatmaps, pair plots, 3D scatter | Visual exploration |
Worked example: ad click rates
An advertiser records the advertisement type, the gender of the viewer and the click rate.
| Advertisement | Gender | Click rate |
|---|---|---|
| Ad1 | Male | 80 |
| Ad3 | Female | 55 |
| Ad2 | Female | 123 |
| Ad1 | Male | 66 |
| Ad3 | Male | 35 |
Three variables interact: the ad, who sees it, and how they respond. We fit a multiple regression:
import statsmodels.api as sm
data = {
"Ad_Type": [1, 2, 3, 1, 3],
"Gender": [0, 1, 1, 0, 0], # 0 = Male, 1 = Female
"Click_Rate": [80, 123, 55, 66, 35],
}
df3 = pd.DataFrame(data)
X = sm.add_constant(df3[["Ad_Type", "Gender"]])
y = df3["Click_Rate"]
model = sm.OLS(y, X).fit()
print(model.summary())
The output table gives, for each predictor, a coefficient (how much click rate changes when that predictor increases by one unit, others fixed), a p-value (is the effect statistically meaningful?) and R-squared (how much variation the model explains).
With only 5 rows this example is for learning. A real study needs far more data; otherwise the coefficients are unreliable.
Visual multivariate tools
# Heatmap of all numeric correlations
sns.heatmap(df.corr(numeric_only=True), annot=True, cmap="coolwarm", center=0)
# Scatter with a third variable as colour and a fourth as size
sns.scatterplot(data=df, x="alcohol", y="density", hue="quality", size="sulphates")
# Pair plot coloured by a category
sns.pairplot(df[["alcohol", "sulphates", "density", "quality"]], hue="quality")
Applications
- Customer segmentation: group customers using age, income and buying habits.
- Economic forecasting: GDP, inflation, employment and exports modelled together.
- Medicine: predicting disease risk from age, weight, blood tests and lifestyle.
Advantages and limitations
Advantages: captures real-world complexity; enables prediction and deeper insight.
Limitations:
- Needs more data and more computation.
- Harder to interpret.
- Multicollinearity: when predictors are strongly correlated with each other, regression coefficients become unstable. Check correlations or the Variance Inflation Factor (VIF) first.