IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v36y2011i5p586-615.html
   My bibliography  Save this article

Sampling Variability and Axioms of Classical Test Theory

Author

Listed:
  • Donald W. Zimmerman

Abstract

Many well-known equations in classical test theory are mathematical identities in populations of individuals but not in random samples from those populations. First, test scores are subject to the same sampling error that is familiar in statistical estimation and hypothesis testing. Second, the assumptions made in derivation of formulas in test theory are not necessarily satisfied in small samples. The present study derived modified equations relating test scores and components of scores that are identities in samples of any size and that reduce to the more familiar equations when various correlations are zero. Simulations determined the accuracy of both the familiar and the modified equations when applied to samples of various sizes from populations with known reliability coefficients. The programs also determined the variability of the sample values for different parameters in the equations and for different sample sizes, as well as the means and variances of discrepancies between population and sample values.

Suggested Citation

  • Donald W. Zimmerman, 2011. "Sampling Variability and Axioms of Classical Test Theory," Journal of Educational and Behavioral Statistics, , vol. 36(5), pages 586-615, October.
    • Handle: RePEc:sae:jedbes:v:36:y:2011:i:5:p:586-615
    DOI: 10.3102/1076998610397052
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998610397052
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998610397052?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Louis Guttman, 1953. "Reliability formulas that do not assume experimental independence," Psychometrika, Springer;The Psychometric Society, vol. 18(3), pages 225-239, September.
    2. Lee Cronbach, 1951. "Coefficient alpha and the internal structure of tests," Psychometrika, Springer;The Psychometric Society, vol. 16(3), pages 297-334, September.
    3. Melvin Novick & Charles Lewis, 1967. "Coefficient alpha and the reliability of composite measurements," Psychometrika, Springer;The Psychometric Society, vol. 32(1), pages 1-13, March.
    4. Leonard Feldt, 1965. "The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty," Psychometrika, Springer;The Psychometric Society, vol. 30(3), pages 357-370, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tarkkonen, L. & Vehkalahti, K., 2005. "Measurement errors in multivariate measurement scales," Journal of Multivariate Analysis, Elsevier, vol. 96(1), pages 172-189, September.
    2. Walter Kristof, 1971. "On the theory of a set of tests which differ only in length," Psychometrika, Springer;The Psychometric Society, vol. 36(3), pages 207-225, September.
    3. Klaas Sijtsma & Julius M. Pfadt, 2021. "Part II: On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha: Discussing Lower Bounds and Correlated Errors," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 843-860, December.
    4. Christmann, A. & Van Aelst, S., 2006. "Robust estimation of Cronbach's alpha," Journal of Multivariate Analysis, Elsevier, vol. 97(7), pages 1660-1674, August.
    5. Klaas Sijtsma, 2012. "Future of Psychometrics: Ask What Psychometrics Can Do for Psychology," Psychometrika, Springer;The Psychometric Society, vol. 77(1), pages 4-20, January.
    6. Christmann, Andreas & Aelst, Stefan van, 2002. "Robust estimation of Cronbach's alpha," Technical Reports 2002,42, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    7. Dheeraj Sharma, 2016. "Understanding Coefficient Alpha: Assumptions and Interpretations," Working Papers id:11088, eSocialSciences.
    8. Sharma, Dheeraj, 2016. "Understanding Coefficient Alpha: Assumptions and Interpretations," IIMA Working Papers WP2016-03-38, Indian Institute of Management Ahmedabad, Research and Publication Department.
    9. Walter Kristof, 1974. "Estimation of reliability and true score variance from a split of a test into three arbitrary parts," Psychometrika, Springer;The Psychometric Society, vol. 39(4), pages 491-499, December.
    10. Kentaro Hayashi & Akihito Kamata, 2005. "A note on the estimator of the alpha coefficient for standardized variables under normality," Psychometrika, Springer;The Psychometric Society, vol. 70(3), pages 579-586, September.
    11. Salim Moussa, 2016. "A Comment on the Estimation of the Reliability of Multidimensional Marketing Constructs: A Store Personality Scale Application," Global Business Review, International Management Institute, vol. 17(5), pages 1125-1144, October.
    12. Kent W. Smith, 1974. "On Estimating the Reliability of Composite Indexes Through Factor Analysis," Sociological Methods & Research, , vol. 2(4), pages 485-510, May.
    13. Klaas Sijtsma & Ivo Molenaar, 1987. "Reliability of test scores in nonparametric item response theory," Psychometrika, Springer;The Psychometric Society, vol. 52(1), pages 79-97, March.
    14. Emily Kistner & Keith Muller, 2004. "Exact distributions of intraclass correlation and Cronbach's alpha with Gaussian data and general covariance," Psychometrika, Springer;The Psychometric Society, vol. 69(3), pages 459-474, September.
    15. Massimiliano Pastore & Luigi Lombardi, 2014. "The impact of faking on Cronbach’s alpha for dichotomous and ordered rating scores," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(3), pages 1191-1211, May.
    16. Ke-Hai Yuan & Peter Bentler, 2002. "On robusiness of the normal-theory based asymptotic distributions of three reliability coefficient estimates," Psychometrika, Springer;The Psychometric Society, vol. 67(2), pages 251-259, June.
    17. Jos Berge & Willem Hofstee, 1999. "Coefficients alpha and reliabilities of unrotated and rotated components," Psychometrika, Springer;The Psychometric Society, vol. 64(1), pages 83-90, March.
    18. Ping, Robert Jr., 2004. "On assuring valid measures for theoretical models using survey data," Journal of Business Research, Elsevier, vol. 57(2), pages 125-141, February.
    19. Tyler Hunt & Peter Bentler, 2015. "Quantile Lower Bounds to Reliability Based on Locally Optimal Splits," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 182-195, March.
    20. David Woodruff & Leonard Feldt, 1986. "Tests for equality of several alpha coefficients when their sample estimates are dependent," Psychometrika, Springer;The Psychometric Society, vol. 51(3), pages 393-413, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:36:y:2011:i:5:p:586-615. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.