Z-Test for Two Proportions
What Is the Z-Test for Two Proportions?
The z-test for two proportions is a statistical method that helps you determine whether the success rates (or any proportions) observed in two independent groups are significantly different from each other. If you have two groups and you want to compare the percentage of each group that meets some criterion, this is the test to use.
Imagine a researcher running a clinical trial. She gives 200 patients a new medication and 200 patients a placebo. After treatment, 140 of the medication group recover, compared to 110 of the placebo group. That is a recovery rate of 70% versus 55%. The z-test for two proportions tells her whether this 15-percentage-point difference is statistically significant or whether it could plausibly be due to random variation.
Why Do We Need It?
Comparing proportions between groups is one of the most common tasks in research. A company might want to know whether its website redesign improved the conversion rate compared to the old design. A public health official might compare vaccination rates between two regions. A political analyst might compare the approval ratings of a policy across two demographic groups. In all these cases, you are looking at two independent samples and asking whether the proportions differ by more than chance would predict.
How Does It Work?
The test begins with a null hypothesis stating that the two population proportions are equal. To evaluate this, the test first computes a pooled proportion, which is the overall success rate across both groups combined. In the clinical trial example, the pooled proportion would be (140 + 110) / (200 + 200) = 0.625, or 62.5%.
Using this pooled proportion, the test calculates a z-statistic. The z-statistic measures how many standard errors the observed difference between the two proportions lies from zero (the value expected under the null hypothesis). A z-statistic near zero suggests the two proportions are similar. A z-statistic far from zero—typically beyond roughly ±1.96—suggests the difference is unlikely to be due to chance.
The z-statistic is then converted into a p-value using the standard normal distribution. This p-value tells you the probability of observing a difference as large as (or larger than) yours, assuming there is actually no difference in the population.
Interpreting the Result
If the p-value is below your significance level (typically 0.05), you reject the null hypothesis and conclude that the two proportions are significantly different. In the clinical trial example, a very small p-value would indicate that the medication genuinely improves recovery rates compared to the placebo.
If the p-value is above 0.05, you do not have enough evidence to conclude that the proportions differ. This does not prove they are equal; it simply means the observed difference could easily have occurred by chance given the sample sizes involved.
When to Use This Test
Use the z-test for two proportions when you want to compare the proportion of successes (or any binary outcome) between two independent groups. Typical scenarios include:
- Comparing treatment and control groups in an experiment.
- Comparing click-through or conversion rates between two versions of a web page (A/B testing).
- Comparing survey response rates across two demographic groups.
- Comparing pass rates between two schools or training programs.
Key Assumptions
Like all statistical tests, the z-test for two proportions requires certain conditions to produce reliable results:
- The two samples must be independent of each other. Participants in one group should not influence participants in the other.
- Each observation within a group must be independent. One person’s outcome should not affect another’s.
- The sample sizes should be large enough that the normal approximation is reasonable. A common guideline is that both n×p and n×(1−p) should be at least 5 in each group, where p is the sample proportion.
- The data should come from random or representative sampling.
When sample sizes are very small and the conditions above are not met, the normal approximation underlying the z-test becomes unreliable. In those situations, an exact test such as the Fisher exact test is a better choice. For moderate to large samples, however, the z-test for two proportions is efficient, well-understood, and widely used across the social and health sciences.