PCA & Dimensionality Reduction: Fewer Columns, Same Signal

When columns are highly correlated, they repeat each other's information. Principal Component Analysis finds new axes — the principal components — that point along the directions of greatest variance, as the arrows above show. A few of them often capture most of the signal.

Note

PCA doesn't delete columns; it rotates the data into new directions ordered by how much variance each explains, so you can keep just the top few.

How It Works

Find the direction of maximum variance — that's PC1
Find the next direction, at right angles to it — PC2, and so on
Each component explains a share of the total variance
Keep enough components to cover, say, 90% of the variance

The components are ordered, so the first few are the ones worth keeping.

Explained Variance

The key output is how much variance each component explains. Plot the cumulative share and you'll usually see it climb steeply then flatten — the "elbow" tells you how many components are enough.

Tip

Choose the number of components by the cumulative explained variance you need — often 90–95% — not by a fixed count picked in advance.

Scale First, Always

PCA chases variance, so a column measured in large units will dominate purely because its numbers are bigger. Standardize every feature first, or the components will just track your units instead of your structure.

Watch out

Running PCA on unscaled data is the classic mistake — the component directions end up driven by unit choice, not real variation. Z-score your features first.

When to Use PCA

Too many correlated features slowing a model or causing instability
Visualizing high-dimensional data in 2D or 3D
Removing noise by dropping low-variance components

The trade-off: the new components are combinations of originals, so they're harder to interpret than the raw columns.

PCA in the SKARI Workflow

SKARI's Data Editor standardizes your features, and the Model Lab applies dimensionality reduction as a preprocessing step — so you feed models a compact, scaled representation and can view the explained variance behind it.

Standardization in the Data Editor before reduction
Dimensionality reduction as a modeling step
Explained-variance view to pick how many components to keep

Takeaway

Scaling and reduction sit in the same flow — so PCA reflects your structure, not your units.

Frequently Asked Questions

How many components should I keep?

Enough to reach your target cumulative variance — commonly 90–95%. The elbow in the curve is a good guide.

Does PCA improve accuracy?

Sometimes — it can cut noise and instability — but it can also drop useful signal. Validate with and without it.

Is PCA the only option?

No — t-SNE and UMAP are better for visualization, though PCA remains the fast, linear default for compression.

Conclusion

PCA compresses correlated columns into a handful of variance-rich directions. Scale first, keep enough components to hold the signal, and you get a smaller, faster dataset that still says what the original did.

Takeaway

Fewer dimensions, same story — as long as you scale first and let explained variance decide how many to keep.

Data Normalization

The scaling PCA depends on

Complete Clustering Guide

PCA as a preprocessing step

Overfitting & Regularization

Another way to fight too many features