Outlier Detection

Outliers come in more than one shape

An outlier is not always a very large number. Some values are perfectly ordinary alone and impossible in combination; others are odd not as points but as a stretch.

• •
  Point outliers — far from everything on a single variable. The most common kind, and the easiest to find.
• •
  Contextual outliers — the value is fine, the context is not. 28°C in January.
• •
  Multivariate outliers — each variable is unremarkable, the combination is not. 150 cm tall and 110 kg.
• •
  Collective outliers — every value is normal but the run is not. A server returning the identical figure for thirty minutes.

The three rules you will meet

Nearly every detection rule in practical use is one of these.

What separates them is which statistics they use for the centre and the width. The first takes the mean and the standard deviation; the other two take the median and the quartiles. Everything else follows from that choice.

The z rule is blunted by what it is hunting

The mean and the standard deviation are the two statistics most sensitive to outliers — and the z-score is built out of both. Add an extreme value and σ grows; once σ grows, that value's own z shrinks.

With one extreme point z is 4.71 and catches it cleanly. At three points it drops to 2.84 and slips under the threshold. Over the same range the modified z stays above 20 and the fence never stops flagging. This is masking.

z has a ceiling it cannot pass

There is a deeper problem. In a sample of size n, no value can have a |z| larger than$(n-1)/\sqrt{n}$. Not unlikely — arithmetically impossible.

On ten observations, $|z| > 3$ cannot return a result, no matter how absurd a value you plant in there. Run it on a small sample and "no outliers found" is not a finding about the data — it is a fact about the rule.

The IQR fence has its own failure mode

The fence is immune to outliers but assumes symmetry. On anything naturally right-skewed — incomes, wait times, claim sizes — it flags the entire legitimate tail.

1. Match the rule to the shape of the data

2. Detection is followed by diagnosis

All a rule ever tells you is "this value is far away". Why it is far away can only be answered by knowing how the data came to exist.

Note that deletion is the right answer in none of the six rows. Typos get fixed or nulled, unit mix-ups get converted, a different population gets analysed separately, and a genuine extreme gets kept.

3. Alternatives to deleting

• •
  Transform — a log or square root shortens the tail and reduces the pull of extremes without discarding anything.
• •
  Winsorise — replace the top and bottom 1% with the boundary value. Order is preserved, influence is not.
• •
  Robust models — quantile regression, Huber loss, tree ensembles. Extremes stop steering the fit.
• •
  Weighting — down-weight extreme observations. A continuous dial between keeping and dropping.
• •
  Segment them — if the outliers form a coherent group, that group is a segment, not an error.

4. Log it, and report both ways

Without a record of what you removed, nobody can check the work. And when the call is genuinely ambiguous, reporting the result with and without is the honest option.

• •
  Record how many were removed and by which rule
• •
  Summarise the removed observations separately
• •
  Compute the headline result both ways
• •
  If the conclusion flips, say so — that is the finding
• •
  Fit outlier thresholds on the training rows only

In practice: what automatic removal was deleting

A team running automatic outlier removal in their pipeline finally looked at what it had been throwing away. None of it was an error.

All five were findings that deletion erased. The third is the worst of them: it was a real outage, and outlier removal was quietly disabling the monitoring.

In practice: odd only in combination

Health screening data passed univariate checks on height and weight with almost nothing flagged. The scatter plot showed twelve points sitting well off the main body.

Univariate rules alone would have passed all twelve. Once there is more than one variable, you need a rule that looks at the combination.

Common misunderstandings

Drag one point and watch three rules disagree

Twenty-four fixed observations plus one you control. Push the slider right and watch when each rule reacts — and where z stops climbing.

What to look for

• •
  Which rule reacts first?
• •
  Push the point to the far end — what does z converge to, and why?
• •
  At that same position, how high does the modified z go?
• •
  Turn on masking. What happens to z?
• •
  With masking on, why does the fence barely move?