Calculator
Online calculator: enter the means, standard deviations and sample sizes of both groups.
Explanation
This test is used to assess whether two population means are equal when comparing two independent groups without assuming a common variance. That's why the Welch version is normally used, since it is more robust when the dispersion of the two groups is not the same.
Besides the statistical decision, it is worth interpreting the magnitude of the difference \(\bar{x}_1-\bar{x}_2\), its confidence interval and its relevance in the applied context.
Hypotheses and test statistic
\(H_0: \mu_1 - \mu_2 = 0\)
\( t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{s_1^2/n_1 + s_2^2/n_2}} \)
- \(\bar{x}_1, \bar{x}_2\): sample means of groups 1 and 2.
- \(s_1, s_2\): sample standard deviations of each group.
- \(n_1, n_2\): sizes of groups 1 and 2.
- df: Welch-Satterthwaite degrees of freedom (approximate, not necessarily an integer):
\( \nu = \dfrac{(s_1^2/n_1 + s_2^2/n_2)^2}{\dfrac{(s_1^2/n_1)^2}{n_1-1} + \dfrac{(s_2^2/n_2)^2}{n_2-1}} \)
Quick check
The Welch formula is used because it does not require assuming equal variances between groups. In applied practice, it is a robust option when \(s_1\) and \(s_2\), or \(n_1\) and \(n_2\), differ.
That is also why approximate degrees of freedom (Welch-Satterthwaite) are used, which may not be integers.
Is there also a version with known population standard deviation?
Yes. Just as with the one-mean test, for two means there are two approaches: a two-sample z-test when the population standard deviations \(\sigma_1\) and \(\sigma_2\) are known, and a t-test when only the sample standard deviations \(s_1\) and \(s_2\) are known.
In real practice \(\sigma_1\) and \(\sigma_2\) are almost never known, which is why this calculator applies the most common and robust option: Welch's t-test. If in your case you do know the true population standard deviations, the correct approach would be a two-sample z-test.
Worked example
Two treatments for lowering blood pressure are compared. Group 1 (treatment A): \(\bar{x}_1 = 82\) mmHg, \(s_1 = 12\) mmHg, \(n_1 = 30\). Group 2 (treatment B): \(\bar{x}_2 = 76\) mmHg, \(s_2 = 15\) mmHg, \(n_2 = 28\). The test is \(H_0\colon \mu_1 = \mu_2\) versus \(H_1\colon \mu_1 \neq \mu_2\) with \(\alpha = 0.05\) two-tailed, using Welch's method.
The standard error of the difference is:
\( SE = \sqrt{\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}} = \sqrt{\dfrac{144}{30} + \dfrac{225}{28}} = \sqrt{4.8 + 8.036} \approx 3.583 \)
Welch's t statistic is:
\( t = \dfrac{\bar{x}_1 - \bar{x}_2}{SE} = \dfrac{82 - 76}{3.583} \approx 1.675 \)
The Welch degrees of freedom are approximately \(\nu \approx 52\). The two-tailed critical value is \(t_{0.025,\,52} \approx 2.007\). Since \(|t| = 1.675 < 2.007\), \(H_0\) is not rejected. The two-tailed p-value is \(p \approx 0.100 > 0.05\).
Conclusion: \(H_0\) is not rejected. With this data there is not enough statistical evidence to claim that the two treatments produce different mean reductions at the 5% level. The observed difference of 6 mmHg is compatible with sample variability.
How to interpret the result
Rejecting \(H_0\) (p-value < α) means that the observed difference between \(\bar{x}_1\) and \(\bar{x}_2\) is too large to attribute to chance if both populations had the same mean. However, statistical significance does not imply practical relevance: with large samples, differences of small magnitude can be highly significant. Always report the estimated difference \(\hat{\Delta} = \bar{x}_1 - \bar{x}_2\) and its confidence interval to quantify the effect size.
Failing to reject \(H_0\) (p-value ≥ α) does not amount to proving that the means are equal: it only indicates that, with the available sample, there is not enough evidence to reject equality. Low power — caused by small samples, high variability or a small true difference — can prevent detection. Compute the confidence interval for \(\Delta\) and assess whether the plausible range of differences excludes values of practical relevance.
The t statistic (or z when \(\sigma\) is known) quantifies how many standard errors the sample difference is away from zero. In the chart, the green zone represents the non-rejection region, the red zones are the critical regions and the amber line marks the observed statistic: if it falls in red, p-value < α. If the test uses equal variances (pooled) or unequal variances (Welch), the standard error calculation changes, but the interpretive logic is the same.
Frequently asked questions
- Does this work for paired samples? No; for paired data you should use the paired-means test.
- What hypothesis does it test? Usually \(H_0: \mu_1=\mu_2\), against a two-tailed or one-tailed alternative.
- What happens if the sample sizes are very different? It can still be computed, but it is important to check assumptions and variances.
- How can I improve the interpretation? Also report the estimated difference, its confidence interval and the context of the problem.
Reference: Student's t-test (two-sample) — Wikipedia