Correlation and agreement
A correlation coefficient measures how closely two variables are associated, and which coefficient fits depends on the data. A coefficient can be statistically significant without being strong, a report of a correlation or an association analysis has items of its own, and whether two methods of measurement agree is a separate question that no correlation answers.
What a correlation coefficient measures
Correlation assesses the extent of the association between two variables in a bivariate analysis, and its coefficients, which include Pearson's r, Spearman's ρ and Kendall's τ, run from −1, a perfect inverse association, to +1, a perfect direct one. Squaring Pearson's r gives the proportion of the variance in one variable that the other explains. An r of 0.975 is consistent with a strong correlation, and its square, 0.950, means that 95% of the variance in one data set is explained by the variance in the other.
Two variables can be correlated with statistical significance but not necessarily strongly. An r of 0.189 with P = 0.04, for example, lies close to 0, so the correlation is not strong. Its square, 0.036, means that only about 4% of the variance in the dependent variable is explained by the independent one.
Choosing a coefficient
Pearson's r needs continuous data that have a linear relationship and are homoscedastic, their points lying symmetrically around the regression line in the shape of a tube rather than a cone. Daily sodium content in grams and waking blood pressure in mmHg among patients with salt-sensitive hypertension are such a pair of variables. Pearson's r is the ratio of the covariance of the two variables to the product of their standard deviations, and in a regression model it equals the product of the regression slope and the ratio of the standard deviations.
Spearman's ρ and Kendall's τ measure correlation in paired ranked data with asymmetric distributions, and they can be thought of as nonparametric counterparts of Pearson's r. Spearman's ρ measures the association between two ranked, ordinal variables and makes no assumption about the distribution of the data, but it requires a monotonic relationship between them. Kendall's τ works from concordant and discordant pairs, two pairs being concordant where both of their ranks rise, or both fall, from one to the next. It requires ranked values but can analyse data that are not normally distributed or whose relationship is monotonic but not linear.
In hypothetical rheumatoid factor levels from 11 patients before and after three months of treatment with a tumour necrosis factor α inhibitor, both distributions are skewed to the right, each mean exceeding its median, and the ranks give 45 concordant and 9 discordant pairs.
Reporting a correlation or an association analysis
A methods section should say which coefficient a correlational analysis used, whether Pearson's r, Spearman's ρ, Kendall's τ or another.
For a correlation analysis, the Statistical Analyses and Methods in the Published Literature (SAMPL) guidelines recommend that a report:
- Describe the purpose of the analysis.
- Summarise each variable with the appropriate descriptive statistics.
- Identify the correlation coefficient used, such as Pearson's or Spearman's.
- Confirm that the assumptions of the analysis were met.
- Report the alpha level that indicates whether the coefficient is statistically significant.
- Report the value of the coefficient, and do not describe a correlation as low, moderate or high unless the ranges for those categories have been defined; even then, consider the wisdom of using such categories, given their biological implications or realities.
- For primary comparisons, report the 95% confidence interval for the coefficient, whether or not it is statistically significant.
- For primary comparisons, consider reporting the results as a scatter plot, whose data field can include the sample size, the coefficient with its confidence interval, and the P value.
- Name the statistical package used.
For an analysis of association, they recommend that a report:
- Describe the association of interest.
- Identify the variables used and summarise each with descriptive statistics.
- Identify the test of association used.
- Indicate whether the test was one- or two-tailed, and justify any one-tailed test.
- For a test of association, such as a chi-square test, report the P value, because association is defined there as a statistically significant result.
- For a measure of association, such as the phi coefficient, report the value of the coefficient and a confidence interval, and do not describe the association as low, moderate or high unless the ranges for those categories have been defined; even then, consider the wisdom of using such categories, given their biological implications or realities.
- For primary comparisons, consider including the full contingency table.
- Name the statistical package used.
Agreement between methods
Quantitative measures from two methods can be correlated without agreeing. A correlation coefficient characterises the relationship between two sets of data but not their agreement. In a worked example of two laboratory methods, 30 paired results were closely correlated, with an r² of 0.992, while Method A read 27.2 units, or 17.4%, below Method B on average.
A Bland–Altman plot is a straightforward, graphical way to assess the bias between two sets of paired measurements by plotting the difference within each pair against the pair's mean, with a horizontal line at the mean difference. The difference can be plotted in units or as a percentage of the pair's mean. A horizontal line at zero means that the two measurements show no bias, and any differences between them can be ascribed to analytical variability alone. Four kinds of anomaly can be seen in the plot: systematic error, a mean offset; proportional error, a trend; inconsistent variability; and excessive or erratic variability. In the worked example the differences between low values cluster closely and those between higher values spread more widely.
If the data are normally distributed, which a plot or a Shapiro–Wilk or Kolmogorov–Smirnov test can show, a 95% confidence interval can be constructed around the mean difference. In the worked example, with 30 pairs and a t value of 2.05 for 29 degrees of freedom, that interval runs from −40.2 to −14.2 around the mean difference of −27.2. Bias is present when the line of equality, a difference of zero, lies outside the confidence interval of the mean difference, as it does in the worked example.
The plot shows the degree of bias and not the clinical or biological meaning of a difference between the methods. The 95% confidence interval is the statistical convention for normally distributed data, but it may be advisable to construct more biologically informed precision limits, as is done in non-inferiority analyses, to enrich the meaning of the plot.