Reader and case sample combinations needed for 0.80 power based on the data of:

Franken EA, Berbaum KS, Marley SM, Smith WL, Sato Y, Kao SCS, Milam SG. Evaluation of a digital workstation for interpreting neonatal examinations - A receiver operating characteristic study. Investigative Radiology 1992;27:732-737.

Readers: 4 pediatric radiologists

Cases: 100 neonatal intensive care patients, 67 with abnormalities

Cases: 67 neonatal intensive care patients with subtle abnormalities, 33 neonatal intensive care patients without abnormalities (100)

Rating: probability of abnormality on a 5-point scale (1=definite abnormality)

Conditions: (1) Digital display of digitized plain film images. (Also referred to as PACS Display (Picture Archiving and Communications System.) [PACS], (2) Plain film display. [PF]

 

ROC Model = propROC, Parameter = AUC,

AUCs = 0.870, 0.853, respectively.

Readers = 4, Cases = 100,

MS( Treatment*Reader ) = 0.00327658,

MS( Treatment*Case ) = 0.07242917,

MS( Treatment*Reader *Case ) = 0.07854222.

AUC Difference = 0.03

Readers	Readers and Cases Random	Cases Random	Readers Random
3	99	459	99
4	93	345	93
5	87	276	87
6	81	231	81
7	77	198	77
8	73	174	73
9	69	155	69
10	66	139	66
11	63	127	63
12	60	117	60
13	57	108	57
14	55	100	55
15	53	94	53

 

 

 

 

AUC Difference = 0.05

Readers	Readers and Cases Random	Cases Random	Readers Random
3	91	167	91
4	77	126	77
5	66	101	66
6	58	85	58
7	52	73	52
8	47	64	47
9	43	57	43
10	40	52	40
11	37	47	37
12	34	44	34
13	32	40	32
14	30	38	30
15	28	35	28

 

 

AUC Difference = 0.10

Readers	Readers and Cases Random	Cases Random	Readers Random
3	65	44	65
4	43	33	43
5	32	27	32
6	25	23	25
7	21	20	21
8	20	20	20
9	20	20	20
10	20	20	20
11	20	20	20
12	20	20	20
13	20	20	20
14	20	20	20
15	20	20	20

 

Cautionary note:  For this data set the estimated required number of cases when only cases are treated as random sometimes considerably exceeds the required number when both cases and readers are treated as random. If the variance components were all known, this would not be possible. It happens here because we are using unbiased estimates of the variance components rather than the true unknown parameter values; in particular, it happens because the treatment-by-reader variance component estimate is negative.  In this situation a conservative approach is to rerun the program with the treatment-by-reader variance component estimate set to zero.  Alternatively, in this situation you may want to consider pooling information from several similar studies, resulting in more precise variance component estimates and hence more precise sample size estimates.  The reader is referred to the Details section of the MRMC Sample Size User's Guide for further information.