Statistics - Cohen's kappa coefficient

Statistics – Cohen’s kappa coefficient

Cohen’s kappa coefficient is a statistic which measures inter-rater settlement for qualitative (categorical) gadgets. It’s usually considered a extra strong measure than easy p.c settlement calculation, since okay takes into consideration the settlement occurring by probability. Cohen’s kappa measures the settlement between two raters who every classify N gadgets into C mutually unique classes.

Cohen’s kappa coefficient is outlined and given by the next operate:

Formulation

okay=p0pe1pe=11po1peokay=p0−pe1−pe=1−1−po1−pe

The place −

  • p0p0 = relative noticed settlement amongst raters.
  • pepe = the hypothetical likelihood of probability settlement.

p0p0 and pepe are computed utilizing the noticed knowledge to calculate the possibilities of every observer randomly saying every class. If the raters are in full settlement then okayokay = 1. If there isn’t any settlement among the many raters aside from what can be anticipated by probability (as given by pepe), okayokay ≤ 0.

Instance

Downside Assertion:

Suppose that you just had been analyzing knowledge associated to a bunch of 50 folks making use of for a grant. Every grant proposal was learn by two readers and every reader both stated “Sure” or “No” to the proposal. Suppose the disagreement depend knowledge had been as follows, the place A and B are readers, knowledge on the diagonal slanting left exhibits the depend of agreements and the info on the diagonal slanting proper, disagreements:

B
Sure No
A Sure 20 5
No 10 15

Calculate Cohen’s kappa coefficient.

Resolution:

Observe that there have been 20 proposals that had been granted by each reader A and reader B and 15 proposals that had been rejected by each readers. Thus, the noticed proportionate settlement is

p0=20+1550=0.70p0=20+1550=0.70

To calculate pepe (the likelihood of random settlement) we observe that:

  • Reader A stated “Sure” to 25 candidates and “No” to 25 candidates. Thus reader A stated “Sure” 50% of the time.
  • Reader B stated “Sure” to 30 candidates and “No” to 20 candidates. Thus reader B stated “Sure” 60% of the time.

Utilizing system P(A and B) = P(A) x P(B) the place P is likelihood of occasion occuring.

The likelihood that each of them would say “Sure” randomly is 0.50 x 0.60 = 0.30 and the likelihood that each of them would say “No” is 0.50 x 0.40 = 0.20. Thus the general likelihood of random settlement is pepe = 0.3 + 0.2 = 0.5.

So now making use of our system for Cohen’s Kappa we get:

okay=p0pe1pe=0.700.5010.50=0.40