Comparing categorical outcomes
Outcomes counted in categories are compared through a contingency table, with a test chosen by whether the observations are paired or matched or come from independent groups, and associations measured within several strata can be pooled into one comparison or tested for whether they differ.
The contingency table and the choice of test
Pain scores on a 10-point scale can instead be compared as categories by defining what counts as adequate pain relief, for example a decrease of 3, and sorting the patients into responders and nonresponders.
Tests of potential associations among categorical data start from a contingency table, often 2 × 2. The tests divide into those for paired or matched data, such as McNemar's test, and those for unpaired data, such as Fisher's exact test and the chi-squared test.
For unpaired data, Fisher's exact test is typically used with smaller samples, including cell values of ≤5 to 10, and the chi-squared test is appropriate for large samples. When the numbers in the rows and columns of a contingency table exceed 10, calculating Fisher's exact test becomes inefficient even for a computer, and the chi-squared test is used instead.
McNemar's test for paired or matched data
McNemar's test evaluates the dependence of categorical data that are paired or matched, and it is considered a special application of the chi-squared test. One major application of it in biostatistics is to determine genetic linkage disequilibrium.
The McNemar statistic, with 1 degree of freedom, is (|b − c| − 1)² / (b + c). In the contingency table of pairs, b counts the pairs in which the case has the factor and the control does not, and c the pairs in which the control has it and the case does not.
In a hypothetical set of 14 pairs from a respiratory clinic, each a case with chronic obstructive pulmonary disease and a control without it, classified by smoking history, 9 pairs had a smoking case and a nonsmoking control and none the reverse, giving a statistic of 7.1, significant at α < 0.01.
In 20 hypothetical adults classified for Enterobacteriaceae bacteremia before and after treatment with an investigational carbapenem, 11 changed from infection present to absent and 3 from absent to present, giving a statistic of 3.5, not significant at α = 0.05.
Fisher's exact test
Fisher's exact test evaluates the independence of categorical variables in a contingency table through the hypergeometric formula, and it yields an exact P value. One manual says that in modern biostatistics it is often used to compare categorical descriptive data, such as baseline sociodemographic or clinical characteristics, across treatment or other groups. Elsewhere the same manual says that intergroup differences in these baseline characteristics are sometimes analysed with it. It is also often called the Fisher–Irwin test, because J.O. Irwin developed a similar formulation in parallel; Fisher reported his in 1935 and Irwin his in 1936.
Fisher tested a claim that a taster could tell whether tea or milk had been poured first into her cup by presenting 8 cups in a random sequence unknown to her, 4 prepared each way. She categorised all 8 correctly, and because the likelihood of doing so by chance alone was 1/70 (1.43%), the null hypothesis that she could not reliably discern the sequence was rejected at P < 5%.
The chi-squared test
The chi-squared test determines whether observed differences between unpaired categorical data from large samples can be attributed to chance. It is used when there is at least one category of frequency data from a simple random sample of 10 or more independent observations.
The chi-squared statistic sums, across the cells of the table, the squared difference between each observed count and the count expected from chance alone if the columns were independent of the rows, divided by that expected count. The expected counts are calculated from the observed values. For a table with r rows and c columns the degrees of freedom are (r − 1)(c − 1). The calculation for a table with more than two columns is more complex than for a 2 × 2 table, but the basic concepts are the same. The test gives an approximate rather than an exact P value, because the computed sampling distribution approximates the theoretical chi-squared distribution when the null hypothesis is true.
In a hypothetical 2 × 2 table of 318 patients with HIV not treated with highly active antiretroviral therapy, conversion to AIDS occurred in 110 of the 177 who had either a 1 log₁₀ increase in viral load or a 100-cell/mm³ decrease in CD4⁺ T lymphocytes over 1 year and in 43 of the 141 who had neither. The statistic of 31.4 with 1 degree of freedom exceeds the critical value of 3.84, and the association is significant at a two-tailed α < 0.0005.
In an example study of an active analgesic against placebo in two groups of 50 patients, 35 and 13 respectively were responders, and the 2 × 2 chi-squared statistic of 19.4 with 1 degree of freedom gives P < 0.001.
Pooled and stratum-specific comparisons across strata
The Cochran–Mantel–Haenszel test, also known simply as the Mantel–Haenszel test, evaluates the conditional independence of binary variables by testing associations in a set of k 2 × 2 tables, such as tables of two groups measured at more than one time or setting, in different clinical trials or in distinct clinical settings. It can also generate a common odds ratio, pooled across the tables, to assess effect size, and its null hypothesis of conditional independence assumes that the odds ratio in every table is 1.0. It can evaluate the potential effects of confounding variables on associations, including the effects of at least two confounders.
In a hypothetical example of asthma symptoms in children stratified by race, the odds ratio for symptoms in the lower socioeconomic stratum, below 50% of the federal poverty level, against the higher, at least 50%, was 4.94 among non-Hispanic Black and 4.90 among non-Hispanic White children, and the common Mantel–Haenszel odds ratio of 4.92 lies midway between them. The null hypothesis of no association between socioeconomic status and asthma symptoms was rejected.
The chi-squared test of heterogeneity can test for effect modification: its null hypothesis is that the odds ratio in each stratum does not differ significantly from the overall Mantel–Haenszel odds ratio, and its degrees of freedom are the number of strata minus 1. In the asthma example the statistic of 0.00067 with 1 degree of freedom is much less than the critical value, and the null hypothesis that race does not modify the association between socioeconomic status and asthma symptoms is not rejected.