What is Exploratory Data Analysis (EDA)?
Exploratory Data Analysis (EDA) is the process of examining a dataset before building any model, so that you understand its structure, quality, patterns, relationships and surprises. The term was popularised by the statistician John Tukey, who argued that we should "look at the data" before we test hypotheses on it.
Think of EDA as a health check-up for your data. A doctor does not prescribe medicine before examining the patient; a data scientist should not train a model before examining the data.
Why is EDA important?
- Data quality: reveals missing values, duplicates, wrong data types and impossible values (for example, an age of 250).
- Understanding: shows the distribution, spread and typical values of each variable.
- Relationships: shows which variables move together and which ones matter for the target.
- Model choice: helps you decide between linear models, tree models, scaling, encoding and so on.
- Avoiding costly mistakes: a model trained on dirty or misunderstood data gives confident but wrong answers.
Core EDA techniques
| Technique | Question it answers | Example |
|---|---|---|
| Data inspection | How big is the data? What types are the columns? | df.shape, df.info() |
| Missing / duplicate handling | Is the data complete and unique? | df.isnull().sum() |
| Univariate analysis | How is one variable distributed? | histogram, box plot |
| Bivariate analysis | How are two variables related? | scatter plot, correlation |
| Multivariate analysis | How do three or more variables interact? | heatmap, pair plot, PCA |
A typical EDA workflow
- Define the goal - what question are we trying to answer or predict?
- Load and inspect - shape, types, first rows, summary statistics.
- Clean - handle missing values, duplicates and wrong types.
- Explore one variable at a time - distributions and outliers.
- Explore pairs and groups - correlations and group comparisons.
- Explore many variables together - interactions and dimensionality reduction.
- Write down findings - decisions for the next stage (cleaning, feature engineering, modelling).
Python libraries used in EDA
- pandas - tables (DataFrame), grouping, summary statistics.
- NumPy - fast numerical arrays and maths.
- Matplotlib - the base plotting library.
- Seaborn - beautiful statistical plots on top of Matplotlib.
- Plotly - interactive charts.
- SciPy / scikit-learn - statistical tests, outlier detection, PCA.
Real-world example
Suppose a wine company wants to predict wine quality from chemical measurements. Before modelling, EDA can tell them: how many wines got each quality score, whether alcohol is higher in better wines, whether some columns are almost identical (redundant), and whether a few laboratory readings are clearly wrong. Each of these discoveries changes how the model should be built.
Key idea: EDA is not a one-time step. You return to it whenever a model behaves unexpectedly.