Type I and Type II errors and power

What the exam asks

• Identify and describe a Type I or Type II error in the context of a specific hypothesis test.
• Describe a consequence (practical outcome or action taken) of committing a Type I or Type II error in context.
• Calculate or determine the probability of a Type I error ($α$), the probability of a Type II error, or the power of a test given one of the other values.
• Explain or determine how changing the sample size $n$, the significance level $α$, the standard error, or the true parameter value affects the power of the test and the probability of a Type II error.
• Justify the choice of a smaller or larger significance level $α$ (such as $α = 0.01$ versus $α = 0.10$) or a larger sample size based on the severity of the two errors.

Core ideas

Truth versus decision: the four outcomes

A significance test weighs sample evidence against a null hypothesis $H_0$ in favor of an alternative hypothesis $H_a$. Because we only observe a sample, our conclusion might not match the unknown truth about the population parameter.

• Type I error: Rejecting $H_0$ when $H_0$ is actually true. In context: finding convincing statistical evidence that [$H_a$ in context] when in reality [ is true in context].