categorical
Cohen's kappa
Measure agreement between two categorical ratings of the same observations.
Requirements
- two paired rating columns
- a shared category scale
- independent observations
Avoid when
- ratings are unpaired
- more than two raters
- ordinal weighting is essential
Formula
κ = (pₒ − pₑ) / (1 − pₑ)
Interpretation
Kappa is observed agreement corrected for agreement expected by chance; inspect the agreement table alongside it.
Worked example
Measure agreement between two clinicians rating the same cases categorically.
Open example datasets