Image and Audio Features, and Best Practices

Feature extraction for images

Images are grids of pixels. Feeding raw pixels to a model works for small images, but extracting features usually helps.

Approach	Description
Raw pixels	Flatten the image into a vector (simple, high-dimensional)
Color histograms	Distribution of colors
Edges (Sobel, Canny)	Outlines and shapes
HOG	Histogram of Oriented Gradients, good for object shapes
SIFT / ORB	Keypoint descriptors robust to scale and rotation
CNN features (transfer learning)	Take the output of a late layer of a pretrained network (ResNet, VGG, EfficientNet)

import numpy as np
from skimage import io, color, feature

img = io.imread("photo.jpg")
gray = color.rgb2gray(img)

hog_vec, hog_img = feature.hog(gray, pixels_per_cell=(16, 16), cells_per_block=(2, 2), visualize=True)
print("HOG feature vector length:", hog_vec.shape[0])

hist = np.histogram(gray, bins=32, range=(0, 1))[0]      # simple intensity histogram

Feature extraction for audio and signals

Feature	Meaning
MFCC	Mel-frequency cepstral coefficients, standard for speech
Spectrogram	Frequency content over time
Chroma	Pitch classes, useful for music
Zero-crossing rate	How often the signal changes sign
Statistical features	Mean, variance, skewness, FFT components

import librosa

y, sr = librosa.load("speech.wav", sr=16000)
mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=13)
features = np.concatenate([mfcc.mean(axis=1), mfcc.std(axis=1)])   # fixed-size vector
print(features.shape)

Pros and cons of feature extraction

Advantages	Disadvantages
Reduces dimensionality and training time	New features are harder to interpret
Removes redundancy and noise	Some information can be lost
Helps visualization	Linear methods miss non-linear structure
Handles unstructured data (text, images, audio)	Needs scaling and careful tuning

Best practices

1. Scale features before PCA or LDA.
2. Fit the extractor on the training data only; use transform on validation and test data.
3. Choose the number of components using explained variance or cross-validation.
4. Use t-SNE or UMAP for visualization, not as the main model input.
5. Compare model performance with and without extraction.
6. Keep the fitted extractor (joblib) so production data goes through the same transformation.
7. Combine with feature selection: extract first, then drop weak components if needed.

Summary of the Data Preparation journey

Chapter	What you learned
Introduction	Why and how to prepare data, pipelines and leakage
Data Cleaning	Duplicates, formats, structural errors
Missing Values	Types of missingness and imputation
Outliers	Detection and treatment
Splitting Data	Train, validation and test sets, cross-validation
Feature Selection	Filter, wrapper and embedded methods
Feature Scaling	Normalization and standardization
Feature Engineering	Creating, transforming and encoding features
Feature Extraction	PCA, LDA, text, image and audio features
Imbalanced Data	Metrics, resampling, class weights

Key takeaways

• Feature extraction creates new features; feature selection picks existing ones.
• PCA (unsupervised) maximizes variance; LDA (supervised) maximizes class separation.
• Text and images need extraction (TF-IDF, embeddings, CNN features) before modeling.