Population vs Sample

What is a population?

A population means the entire group you want to know about. Every adult in a country, every employee at a company, every unit produced this year, every university student nationally — the whole set you are interested in is the population.

What is a sample?

A sample is data taken from part of that population. If the population is every adult in the country and 1,000 of them are surveyed at random, those 1,000 are the sample.

In other words, a sample is the part chosen to stand in for the population.

Why survey only a sample?

The main reason for using a sample is that surveying an entire population is rarely feasible.

Asking every voter in the country directly before an election is essentially impossible. Instead a portion of them are surveyed and the mood of the whole is estimated from that.

Choose the sample well and the characteristics of the population can be estimated remarkably accurately without surveying all of it.

Why does representativeness matter?

A sample has to stand in for its population properly if the conclusion is to be any good.

Survey 500 basketball players for the average adult height and the answer comes back far above the truth. Survey only primary school children and it lands somewhere else entirely.

Which is to say, a lopsided sample gives a distorted result.

So statistics uses random sampling, to give as many people as possible an equal chance of being picked.

A lopsided sample does not improve with size

Here is where the commonest misunderstanding turns up. "Could you not just survey more people?" A real case settles it.

An apartment complex of 1,200 households had money left over and had to choose between enlarging the playground and adding parking spaces. The management office set up a booth in the courtyard on a Tuesday at 2pm and collected 1,000 responses. Playground won, 71% to 29%.

The residents' committee separately drew 100 of the 1,200 households at random and phoned them in the evening — a tenth as many. There, parking won with 68%.

The one that asked ten times fewer people was far closer. The reason is not subtle: ask who is able to be standing in a courtyard at 2pm on a weekday. Anyone who drove to work cannot possibly be there. The households who most wanted the parking were precisely the ones the booth could never reach.

Take the booth to 3,000 responses and nothing changes. However many people you stop in the courtyard, not one of the households that drove to work is among them. All that grows is your confidence in the wrong answer.

How does a part tell you about the whole?

The average moves a little from sample to sample.

One draw of 1,000 gives an average height of 173 cm; another gives 172.7. A different thousand people produces a slightly different number every time.

But draw the sample large enough, and fairly, and the sample average closes in on the population average.

Estimating a population's average, proportion or variance on that principle is what inferential statistics is.

Two things have to be told apart here

There are two quite different reasons a sample's answer moves around, and what to do about them is opposite.

The top row is solved by a bigger sample. The bottom row is not. The courtyard booth was the bottom row.

So when reading a survey, the question that comes before "how many were asked" is "who was asked".

Where does it come up?

Population and sample are the starting point of very nearly every statistical analysis. Cases like these, for instance.

• •
  opinion polling
• •
  customer satisfaction surveys
• •
  medical clinical trials
• •
  quality inspection
• •
  market research

Every one of them is a process of surveying a sample to estimate a population's characteristics.

What is the population in each case?

Writing down "who does my conclusion apply to?" in one sentence before starting makes it immediately obvious whether the sample is fit for it.

The right-hand column is the important one. Who is missing is not in the data, so no amount of looking at the data will reveal them. Only knowing how it was collected will.

Common misunderstandings

❌ A bigger sample is a more representative one. They are separate problems. As the courtyard booth showed, a lopsided sample of 1,000 is worse than a random sample of 100. Size reduces the wobble from chance; representativeness removes the tilt.

❌ What matters is what percentage of the population you sampled. The size of the population has almost no bearing on the sample size you need. A thousand drawn from fifty million is enough, and the same thousand is just as enough in a country of a hundred million.

❌ People who volunteered are a sample too. They are — but not a random one. Those who opt in tend to be the ones who care most about the topic or are most annoyed by it. Internet surveys are especially prone to this.

❌ A full census has no error. The error from drawing a sample disappears, but omissions, duplicates and mistaken answers remain. If anything, quality control gets harder as the operation grows.

Change it and see for yourself

Raise the sample size and watch how differently a random sample and a lopsided one behave. What size fixes, and what it does not, separates cleanly.

What to look for

• •
  As the sample grows, do both settle down?
• •
  Does the random sample settle on the truth?
• •
  Does the lopsided one move towards the truth as the sample grows?
• •
  At a very large sample, could you tell from the results alone which one was lopsided?