AkCalculators

Correlation — Measuring Association Between Variables

Correlation quantifies the degree to which two variables move together. While causation requires controlled experiments and domain knowledge, correlation is a critical first step to understanding relationships in data. The most commonly used correlation measures are Pearson's correlation coefficient (r), which assesses linear association, and Spearman's rank correlation (ρ), which measures monotonic association using ranks. Covariance, a related concept, captures the direction and magnitude (in units of the variables) of joint variability.

Pearson's correlation (r)

Pearson's r measures linear association between two variables x and y. It is defined as the covariance of x and y divided by the product of their standard deviations:

r = Σ (x_i − x̄)(y_i − ȳ) / sqrt( Σ (x_i − x̄)² × Σ (y_i − ȳ)² )

Values range from −1 (perfect negative linear correlation) through 0 (no linear correlation) to +1 (perfect positive linear correlation). Pearson's r is sensitive to outliers and assumes both variables are at least interval-scale and approximately jointly normally distributed for inference; however, it can be a useful descriptive statistic more generally.

Spearman's rank correlation (ρ)

Spearman's ρ is a nonparametric measure of monotonic association based on ranks. Convert x and y values to ranks (handling ties by assigning average ranks), then compute Pearson's r on the ranks. For data without ties, there is a simplified formula:

ρ = 1 − (6 Σ d_i²) / (n (n² − 1))

where d_i is the difference between the ranks of x_i and y_i. Spearman's ρ is robust to non-linear monotonic relationships and to some outliers because it uses ranks.

Covariance

Covariance measures joint variability and has units (product of units of x and y):

Cov(x,y) = Σ (x_i − x̄)(y_i − ȳ) / (n − 1)

Unlike correlation, covariance does not normalize by variability and so is not scale-free. Positive covariance indicates variables tend to increase together; negative covariance indicates one increases while the other decreases.

Practical interpretation and caveats

• Correlation ≠ Causation: two variables may correlate due to common causes or coincidence.
• Outliers: Pearson's r can be dominated by outliers; always inspect data and consider robust measures or Spearman's ρ.
• Ties in ranks: Spearman's method handles ties by assigning average ranks; many datasets contain ties (e.g., Likert scales).
• Nonlinear relationships: Pearson's r may be near zero for strong nonlinear associations (e.g., quadratic), while Spearman's ρ may capture monotonic patterns better.

Step-by-step calculation outline

For Pearson's r:

1. Compute means x̄ and ȳ.
2. Compute deviations (x_i − x̄) and (y_i − ȳ).
3. Compute Σ (x_i − x̄)(y_i − ȳ), Σ (x_i − x̄)², and Σ (y_i − ȳ)².
4. Apply the formula for r above.

For Spearman's ρ:

1. Replace x and y with their ranks (average ranks for ties).
2. Compute Pearson's r on the rank variables, or use the simplified d_i formula when there are no ties.

Use cases

Correlation is used across sciences: checking linear association before modeling, variable selection, exploratory data analysis, psychometrics, finance (asset returns), biology, and many more fields. Given its simplicity, correlation is often the first diagnostic step.

Best practices

• Plot data (scatter plot) to inspect patterns and outliers before relying solely on numeric coefficients.
• Consider Spearman's ρ when data are ordinal or when relationships are monotonic but not linear.
• Report sample size and confidence intervals when making inferential claims; small samples produce noisy correlation estimates.
• For inference and hypothesis testing, use appropriate tests (e.g., t-test for correlation) and check assumptions.

Correlation measures are powerful but must be interpreted carefully. Use them together with visual inspection, domain knowledge and, when needed, more advanced modeling techniques.