Search Ctrl K
Home / Tools
THE FEUTEX TOOLKIT

Binary Rater Agreement

Compare observed agreement with chance agreement and calculate Cohen’s kappa for two binary raters.

Statistics workspace● Runs in your browser
01 Your input
02 Your result
View or copy raw result data
FormulaObserved Agreement = (a+d)/(a+b+c+d); Chance Agreement = ((a+b)*(a+c)+(c+d)*(b+d))/(a+b+c+d)**2; Kappa = ((a+d)/(a+b+c+d)-(((a+b)*(a+c)+(c+d)*(b+d))/(a+b+c+d)**2))/(1-(((a+b)*(a+c)+(c+d)*(b+d))/(a+b+c+d)**2))
Sources and model references

Worked example

Inputs

Group 1 events
20 count
Group 1 non-events
80 count
Group 2 events
10 count
Group 2 non-events
90 count

Expected results

Observed Agreement
0.55 number
Chance Agreement
0.5 number
Kappa
0.1 number

Load the example in the workspace to reproduce these results. Display rounding does not change the underlying calculation.

How to use this tool

  1. Enter the labeled inputs using consistent units.
  2. Check the stated assumptions and calculate.
  3. Review each output and compare another input scenario.

Try this example

{
  "a": 20,
  "b": 80,
  "c": 10,
  "d": 90
}

Accuracy and scope

Table is two raters on the same items; a and d are agreement cells, b and c disagreements. Kappa is sensitive to prevalence. Finite floating-point arithmetic; results can lose precision near singular or extreme inputs.

Is my data uploaded?

This tool runs locally in your browser. Input and output are not sent to a processing server. Copy or download your result before leaving the page.