Main navigation

What is sampling?


Sampling is used to make inferences about a population from a relatively small number of observations, that are assumed to be representative of the population.

More detailed guidance is contained in the Background Document (pdf - 192kb)

Why use sampling methods?

In a social researcher's ideal world all data would be collected by census. However, in practice this approach is not always practical because of time and cost constraints. As an alternative sampling methods are used to provide estimates based on data from a very small percentage of the population.

How does sampling work?

Sampling works because:

Sample bias

Population estimates based on survey data will be inaccurate if the sample is biased. There are a number of reasons why a sample may be biased:

Sampling frames

The ideal sampling frame is a straightforward list of the elements you are trying to sample. So, for a population sample, a comprehensive list of members of the population would be ideal. And for a household survey a comprehensive list of households would be the ideal.

In practice the ideal sampling frame hardly ever exists. In the UK, for instance, there is no population register that can be used for sampling purposes. The closest is the electoral register, but because inclusion on the electoral register is voluntary large sections of the population are excluded. This means that the electoral register tends to give biased samples.

Popular sampling frames

The Postcode Address File (PAF)

The PAF is the Post Office's list of all addresses in the UK, which receive less than 25 items of post per day. The list is primarily residential addresses (about 94%), and the list covers approximately 99% of all residential addresses.

The PAF closely approximates to a sampling frame of households, which makes it ideal for surveys of households (once non-residential and unoccupied addresses have been screened out).

Random Digit Dialling (RDD)

A sample frame can be generated for a telephone survey using Random Digit Dialling (RDD). RDD involves:

  • Selecting 11 digit numbers at random, the first 7 digits being randomly selected from the published list of prefix numbers (such as 020 8693 XXXX) and the last four digits being generated entirely at random.

  • The selected numbers are then dialled and numbers not in use, and business numbers, are screened out to leave just residential numbers.

Sample size

The main decision needed in deciding on a sample design is sample size. To decide on this a number of questions need to be decided on:

Clustering

A 'clustered' sample is defined as a sample that is selected in two or more hierarchical stages, different 'units' being selected at each stage, and with multiple sub-units being selected within higher order units.

A few examples will help to clarify this:

  • A sample of children is selected by (a) sampling schools and then (b) selecting children within schools. This is a two-stage clustered sample, the clustering being of children within schools.

  • On a general population survey a PAF sample is used to generate a sample of households. Within each household up to two adults are selected at random. This is a two-stage clustered sample, the clustering being of adults within households.
    Note that, had the instruction been to select just one adult per household, this would not be described as a clustered sample, because there would no clustering of the adult sample within a smaller number of households.

The most common design for PAF samples is:

Under this design adults are clustered within households (assuming more than one adult is selected per household) and households are clustered within postcode sectors.

Clustering, or multi-stage sampling, is adopted on surveys for a number of reasons. The two main reasons are:

The impact of clustering on standard errors

The main objection to clustered samples is that they tend to give estimates with larger standard errors than unclustered samples.

The reasoning here is that the more the sample is clustered the greater the chance of drawing a sample that is extreme. For instance, imagine a scenario where you are selecting a sample of 1000 people. Then if you choose to select the 1000 people by selecting just 10 postcode sectors and 100 people per sector, then if you are unlucky enough to select one or two sectors that are outliers in some sense then you will get a sample mean that is quite different to the population mean.

From this example it is hopefully clear that the more the total sample is spread across clusters the lower the chance of taking an extreme sample and the lower the standard error. This translates into: for a fixed sample size, the smaller the sample size per cluster the smaller the standard error.

Stratification

Stratification essentially means dividing the sampling frame into groups (strata) before sampling. Stratification reduces the risk of drawing an extreme sample, unrepresentative of the population.

A simple example would be to take a sampling frame of, say, business establishments and then to sort them into size strata before sampling. The sample would then be described as a sample stratified by size.

There are two methods of stratified sampling: proportionate and disproportionate.

  1. In a proportionate stratified sample the sampling frame is divided into strata but the same sampling fraction is applied per stratum. This means that each stratum is sampled from in its correct proportion.

  2. In a disproportionate stratified sample the sampling fraction differs between strata. This means that individuals from the strata with the highest sampling fractions will be over-represented in the sample. Disproportionate sampling is generally used when there is a need to boost the sample size within a particular stratum or strata (e.g. for boosting the number of young people or minority ethnic groups).

Quota sampling

Quota sampling allows substitution of non-respondents with others willing to respond, but it makes use of stratification to ensure that the final sample reflects the population on at least some key variables.

For instance, the quota sample method uses population estimates of the numbers within each combination of age group, sex, and social classes to determine the number needed within each of these combinations for a survey. Interviewers are then given quotas based on these numbers that they are asked to achieve.

Quota sampling assumes that:

Quota samples can be biased and for this reason the UK government has tended to use random sampling even though quota-sampling methods are cheaper and faster.

Sampling special populations using screening

Some surveys have a particular focus on sub-groups of the general population. Some examples include:

The usual approach to selecting the sample in these instances is:

The success of screening largely depends on our ability to ask screening questions quickly, and at the start of interviews.

Survey Weighting

In most surveys it will be the case that some groups are over-represented in the raw data and others under-represented. These mis-representations are usually dealt with by weighting the data.

The idea behind weighting is that:

Weights for disproportionate sampling are relatively non-controversial, but weights to adjust for non-response biases are largely dependent upon judgement.

The Government Statistical Service has established some principles for weighting survey data. These are:

Further reading

> Magenta Book contents