Margin of error and sample size: a practical guide
How many interviews does a survey need? The answer depends on the precision you want, the confidence level and the groups you want to analyze. This guide explains each of these points with numbers.
- The margin of error shows how far the sample result may be from the true value in the population, at a given confidence level (usually 95%).
- At 95% confidence, 400 interviews give a margin of about ± 4.9 percentage points; 1,000 interviews, about ± 3.1.
- For ± 3 points, about 1,068 interviews are needed, almost the same number for a mid-sized city as for an entire country.
- Subgroups have a larger margin than the total: plan the sample around the smallest group you need to analyze.
- The margin of error doesn’t capture questionnaire, coverage or non-response errors; those come down to quality control.
What is the margin of error?
A sample survey interviews part of the population to estimate what the whole population thinks or does. Since every sample is slightly different, a survey’s result is rarely identical to the true value. The margin of error shows the likely size of that gap.
When a survey reports that 42% of customers are satisfied, with a margin of error of 3 percentage points at 95% confidence, the true value is most likely between 39% and 45%. The confidence level tells you how much you can rely on that range: if the survey were repeated many times, about 95% of the ranges would contain the true value.
How to calculate it
For a simple random sample, the margin of error of a proportion is calculated as follows:
- z is the value associated with the confidence level: 1.96 for 95%.
- p is the expected proportion. If you don’t know it, use 50%, the most conservative case, which gives the largest margin.
- n is the number of interviews.
Working the formula backward gives you the sample size you need for a target margin (e): n = z² × p × (1 − p) ÷ e². For ± 3 points at 95% confidence, that comes to 1,068 interviews.
Margin of error by sample size
The table shows the maximum margin of error (p = 50%) at 95% confidence. Notice that precision improves quickly at first and slowly after that: to cut the margin in half, you need four times the sample.
| Interviews | Margin of error | Chart |
|---|---|---|
| 100 | ± 9.8 pp | |
| 200 | ± 6.9 pp | |
| 300 | ± 5.7 pp | |
| 400 | ± 4.9 pp | |
| 600 | ± 4.0 pp | |
| 800 | ± 3.5 pp | |
| 1.000 | ± 3.1 pp | |
| 1.500 | ± 2.5 pp | |
| 2.000 | ± 2.2 pp | |
| 2.500 | ± 2.0 pp |
Does population size matter?
Much less than you might think. For large populations, the number of interviews you need barely changes, whether it’s a city of 100,000 people or an entire country. Population size only matters when it’s small, which is where the finite population correction comes in.
| Population size | Interviews needed |
|---|---|
| 1.000 | 517 |
| 10.000 | 965 |
| 100.000 | 1.056 |
| 1.000.000 | 1.066 |
Watch out for subgroups
The margin of error you see reported usually applies to the total sample. When results are broken down by region, age group or customer type, each group has fewer interviews and therefore a wider margin. In a survey of 1,000 interviews, a subgroup of 200 people has a margin of about ± 6.9 points.
That’s why the rule of thumb is to plan the sample around the smallest group you need to analyze, not just the total. Small differences between groups also need a significance test before you treat them as findings.
What the margin of error doesn’t measure
The margin of error only measures the error that comes from drawing a sample. It doesn’t capture other problems that can skew a survey:
- Coverage: part of the audience was left out of the list or the data collection channel.
- Non-response: people who agree to answer differ from those who refuse.
- Questionnaire: ambiguous or leading questions.
- Fieldwork: interviews poorly conducted or recorded.
These errors are kept in check through sample design, questionnaire testing and fieldwork supervision. At Checon, these processes follow the ISO 20252 standard, with external audits and internal quality control on every project. With quota, cluster or weighted samples, the real margin also tends to be wider than the simple formula suggests, a phenomenon known as the design effect.