BURKHEIMER -"- ZIMMERMAN "- WILLIAMS 93Substituting this result in [ 29] and simplifying, wehave(a- 1)? sf[ 31] ae iSoThis expression gives maximum reliability intermsof number of choices, variance of true scores, andvariance of observed scores. Substituting the valuefor s6 given by [14] leads to the following alterna-tive result:(a - 1) sfr = nenOe ere + eT[ 32]This equation, then, gives the maximum relia-bility of a multiple-choice test as afunction of num-ber of items, number of choices, variance of truescores, and mean of true scores. It indicates thatmaximum reliability depends on group heterogeneityas well as test length and number of choices. _ioe Pee, ee Si ag &Since O = T+ E and, from [4], E ="",;" , wecan writePay ew aea* 2Solving [ 14] for ar; substituting the results, to-gether with [33], in [31], and simplifying givesanother expression for maximum reliability:N-O[ 34] Jes i= zeeALTERNATIVE EQUATIONS FOR CORRELATIONBETWEEN ERROR SCORES ON PARALLELFORMSSubstituting [13] in [27] and simplifying, wehaveSf[35] r_ = - " ?nae ee Nome dewhich is similar in form to [32]. Equation[ 34] canbe written in this form:N-O[36] s* Q- ro,) =Equation [ 28] can be written as follows:[87] s5 - 1r5,) = 82 (1 - Tee):Substituting the right hand side of [37] in [36] andsimplifying, we haveN-Oog asas?�,�[38] rCOMPUTER CHECKSThe equations presented above give the values ofYoo and reg which would be expected if chance suc-cess due to guessing were the only source of errorin multiple-choice tests. The reliabilities of actualtests would be expected to be less thanthese valuesbecause of the presence of other sources of error.In addition, if reliability were determinedfrom afinite number of ordered pairs of observed scoreson parallel forms of atest, with only error due toguessing present, there would be sampling variabil-ity of the reliability coefficient. The binomial dis-tribution of error scores assumed in derivation ofthe equations, in other words, would be only approx-imated for any finite number of true scores.As the number of ordered pairs of scores on par-allel forms increases without limit, however, the re-liability coefficient would be expected to come clos-er and closer to the values given by the equations.In a previous paper (8) a method of determining thereliability coefficient by a computer simulationmethod was described. It was shown that for fairlylarge numbers of scores (samples of 100, 400, 700,and 1000) the estimates given by the method werestable. For example, for ten samples of 400 scores,the reliability of a 100-item, two-choice test wasindicated as . 89, .88, .89, .87, .90, .88, . 89,.89,. 89, and . 87.In Table 1 the reliabilities given by the computersimulation method are compared to the values givenanalytically by equations [ 31], [ 32], and[ 34] above.Also, the correlations between error scores on par-allel forms given by the computer program are com-pared to the values given by equations [27], [35],and [38] above. In making these checks we beginwith a distribution of true scores having a certainmean and a certain variance. The computer pro-gram then generates error scores which depend up-on the magnitude of the true scores, as a model ofguessing error, and these are added tothe truescores to give observed scores. Repeating the pro-cedure gives results comparable to observed scoreson parallel forms of a test, when guessing is theonly source of error. Finally, product-momentcorrelations between the two sets of observed scores 'give an indication of test reliability. Also, corre-lation between the two sets of error scoresis found,as well as the means and variances of all distribu-tions.It can be seen from the table that the values givenby the computer program correspond closely to thevalues predicted from the equations presented inthis paper.REFERENCES1. Carroll, J. B., ~~The Effect of Difficulty andChance Success on Correlations Between Items