Sampling distributions, simulation and the central limit theorem

What the exam asks

• Distinguish three distributions: Identify whether a prompt describes the population distribution, the distribution of data in a single sample, or the sampling distribution of a statistic.
• Interpret simulated sampling distributions: Read a dotplot or histogram formed by repeatedly taking random samples of size $n$ from a population with known parameters, and use the proportion of simulated statistics at or beyond an observed value to evaluate a claim.
• Interpret randomization distributions: Explain how repeatedly reallocating observed response values to treatment groups models what happens when there is no treatment effect, and use the simulated differences to decide if an observed difference is statistically significant.
• Apply the central limit theorem (CLT): Predict how the shape and variability of the sampling distribution of a sample mean $\bar{x}$ change as the sample size $n$ increases.

Core ideas

Three distinct distributions

A statistic (such as a sample proportion $\hat{p}$ or sample mean $\bar{x}$) is a numerical summary calculated from sample data; it estimates a population parameter ($p$ or $\mu$). Because different random samples yield different values of a statistic, statistics exhibit .

The sampling distribution of a statistic is the distribution of values taken by the statistic in all possible samples of the same size $n$ from the same population.

Simulating sampling and randomization distributions

Because taking all possible samples is rarely practical, we approximate distributions using computer simulation:

1. Simulated sampling distribution (random sampling): Begin with a population model with known (or assumed) parameter values. Repeatedly generate random samples of size $n$, calculate the statistic for each sample, and plot the results. This models random sampling variability.
2. Randomization distribution (randomized experiments): When comparing treatments in an experiment, assume the treatments have identical effects (the null hypothesis of no difference). Keep the observed response values fixed, randomly reallocate them into groups matching the original sample sizes, and calculate the difference in group statistics (such as $\bar{x}_1 - \bar{x}_2$ or ). Repeating this many times shows the differences that could occur by random assignment alone.