Calculator
Select the test type and variable, enter the data and the margin δ to get the z statistics, p-value, decision and confidence interval.
Explanation
Non-inferiority and equivalence tests invert the usual hypothesis-testing logic. Instead of trying to show that two treatments differ, they try to show that they are similar enough within a predefined clinical or practical margin \(\delta\).
Non-inferiority: the goal is to show that the new treatment (cheaper, less invasive, etc.) is not unacceptably worse than the control. The null hypothesis is that the treatment actually is worse by a relevant amount: \(H_0\colon \Delta \leq -\delta\).
Equivalence (TOST): the goal is to show that the new treatment is neither better nor worse in a relevant way. It requires two simultaneous one-sided tests (Two One-Sided Tests), one for non-inferiority and one for non-superiority. Equivalence is declared when both are rejected.
Hypotheses and test statistics
Let \(\hat{\Delta} = \hat{\theta}_1 - \hat{\theta}_2\) be the estimated difference (of means or proportions) and \(\text{SE}\) its standard error:
\( \hat{\Delta} = \bar{x}_1 - \bar{x}_2 \quad \text{or} \quad \hat{\Delta} = \hat{p}_1 - \hat{p}_2 \)
\( \text{SE} = \sqrt{\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}} \quad \text{(means)} \)
\( \text{SE} = \sqrt{\dfrac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \dfrac{\hat{p}_2(1-\hat{p}_2)}{n_2}} \quad \text{(proportions)} \)
Non-inferiority (\(H_0\colon \Delta \leq -\delta\), \(H_1\colon \Delta > -\delta\)):
\( z_{\text{NI}} = \dfrac{\hat{\Delta} + \delta}{\text{SE}}, \quad p_{\text{NI}} = 1 - \Phi(z_{\text{NI}}) \)
Non-superiority (\(H_0\colon \Delta \geq +\delta\), \(H_1\colon \Delta < +\delta\)):
\( z_{\text{NS}} = \dfrac{\delta - \hat{\Delta}}{\text{SE}}, \quad p_{\text{NS}} = 1 - \Phi(z_{\text{NS}}) \)
TOST equivalence: \(p_{\text{TOST}} = \max(p_{\text{NI}}, p_{\text{NS}})\). Equivalence is declared if \(p_{\text{TOST}} < \alpha\).
Quick interpretation guide
- Non-inferiority: if \(p_{\text{NI}} < \alpha\), \(H_0\) is rejected and non-inferiority is declared. The \(100(1-2\alpha)\%\) CI of \(\hat{\Delta}\) must lie entirely above \(-\delta\).
- TOST: if \(p_{\text{TOST}} < \alpha\), equivalence is declared. This is equivalent to checking that the \(100(1-2\alpha)\%\) CI of \(\hat{\Delta}\) is contained in \([-\delta, +\delta\)].
- With \(\alpha=0.05\), the CI criterion for TOST uses the 90% CI (not the 95% CI).
- The margin \(\delta\) must be defined a priori using clinical judgment, not statistical criteria.
Worked example: non-inferiority of proportions
A new treatment has a success rate \(\hat{p}_1 = 0.72\) in \(n_1 = 200\) patients. The reference treatment has \(\hat{p}_2 = 0.75\) in \(n_2 = 200\) patients. The non-inferiority margin is \(\delta = 0.10\) with \(\alpha = 0.05\).
\( \hat{\Delta} = 0.72 - 0.75 = -0.03 \)
\( \text{SE} = \sqrt{\dfrac{0.72\cdot0.28}{200} + \dfrac{0.75\cdot0.25}{200}} = \sqrt{0.001008 + 0.0009375} = \sqrt{0.0019455} \approx 0.04411 \)
\( z_{\text{NI}} = \dfrac{-0.03 + 0.10}{0.04411} \approx 1.587, \quad p \approx 0.056 \)
Since \(p = 0.057 > \alpha = 0.05\), non-inferiority is not declared at the 5% level (a borderline result). The 90% CI of \(\hat{\Delta}\) does not lie entirely above \(-0.10\).
How to interpret the result
Non-inferiority (one-sided test): rejecting \(H_0\) (p-value < α) means that the lower bound of the \((1-2\alpha)\cdot100\%\) confidence interval for the difference \(\Delta = \mu_T - \mu_C\) exceeds the non-inferiority margin \(-\delta\). In practical terms, this concludes that the experimental treatment is not worse than the control by more than the clinically tolerable amount. Not rejecting \(H_0\) (p-value ≥ α) indicates that the data do not rule out the treatment being worse than the control beyond the defined margin.
Equivalence (TOST): equivalence is declared if both one-sided tests simultaneously reject \(H_0\) (p_TOST < α) and the 90% confidence interval for \(\Delta\) lies entirely within \([-\delta, +\delta]\). An equivalence result does not claim the treatments are identical, only that any difference is small enough to be clinically acceptable. If only one of the two one-sided tests rejects, the conclusion is one-directional: non-inferiority, but not equivalence.
Choosing the margin \(\delta\) is the most critical design decision: it must be set a priori using clinical or regulatory judgment, not statistical criteria. In the chart, the green zone corresponds to the equivalence or non-inferiority region, the red zone is the unacceptable-inferiority region, and the amber line shows the estimated difference with its confidence interval. A narrow interval centered near zero is evidence of both non-inferiority and practical equivalence; a wide interval signals uncertainty and possibly an insufficient sample.
Frequently asked questions
- What is the margin δ? The maximum tolerable difference in favor of the control. It must be defined a priori using clinical or regulatory judgment.
- How does non-inferiority differ from equivalence? Non-inferiority is one-sided (the treatment is not worse). Equivalence (TOST) is two-sided (it is neither better nor worse in a relevant way).
- Why the 90% CI in TOST? With α=0.05, TOST is equivalent to checking that the 90% CI of Δ̂ is within [−δ, +δ].
- How is δ chosen? Using clinical or regulatory judgment, not statistical criteria. It is usually a fraction of the active comparator's effect.
- Can non-inferiority coexist with non-superiority? Yes, that is equivalent to declaring equivalence (TOST).
Reference: Non-inferiority trial — Wikipedia