Applications, Advantages and Limitations of Correlation
Applications
| Field | How correlation is used |
|---|---|
| Economics and finance | Relating supply and demand, inflation and interest rates; portfolio diversification (choose assets that are not strongly correlated) |
| Business analytics | Finding which factors relate to revenue, churn or productivity |
| Market research and promotions | Linking market trends and campaigns to customer behaviour |
| Medical research | Understanding relations between symptoms, biomarkers and diseases, including genetic ones |
| Weather forecasting | Relating temperature, pressure, humidity and wind to predict conditions |
| Customer service | Linking response time, resolution and satisfaction scores |
| Environmental analysis | Relating pollution, traffic, temperature and health outcomes to design better policy |
| Machine learning | Feature selection, detecting redundant features, spotting data leakage |
Example: feature selection in ML
target = "quality"
corr_target = df.corr(numeric_only=True)[target].drop(target)
# Keep features with at least a weak link to the target
selected = corr_target[corr_target.abs() > 0.1].index.tolist()
print(selected)
# Remove one of each highly-correlated feature pair
corr = df[selected].corr().abs()
upper = corr.where(np.triu(np.ones(corr.shape), k=1).astype(bool))
to_drop = [c for c in upper.columns if (upper[c] > 0.85).any()]
print("Drop:", to_drop)
Example: portfolio diversification
Two stocks with correlation +0.95 move almost together - owning both gives little protection. A stock and a bond with correlation near 0 or negative reduce overall risk.
Advantages
- Shows how two variables relate to each other.
- Simple and easy to interpret - one number and one plot.
- Helps decision making in business, healthcare, marketing and more.
- Helps in feature selection for machine learning.
- Gives a quantitative measure that can be compared across pairs.
Disadvantages and limitations
- Correlation does not imply causation. A variable may not cause another even when they are strongly correlated.
- Outliers can create, hide or reverse a correlation if not handled.
- Pearson detects only linear relationships; a perfect curve can give r near 0.
- Works well for pairs (bivariate); it does not describe multi-variable interactions, so complex relations cannot be analysed accurately by simple correlation.
- Sensitive to restricted range - if you only study top students, the study-hours vs marks correlation shrinks.
- Simpson's paradox: a trend that appears in the combined data may reverse inside each subgroup.
- With many variables, some "significant" correlations will appear by pure chance (multiple-comparison problem).
Good-practice checklist
- Plot before you compute.
- Check outliers and remove or explain them.
- Try both Pearson and Spearman.
- Report r and sample size and p-value or confidence interval.
- Think about confounders and reverse causation before writing conclusions.
- Do not confuse "strong correlation" with "important" or "causal".