IDEAS home Printed from https://ideas.repec.org/a/spr/reihed/v57y2016i4d10.1007_s11162-015-9394-7.html
   My bibliography  Save this article

The Reliability and Validity of Using Regression Residuals to Measure Institutional Effectiveness in Promoting Degree Completion

Author

Listed:
  • Aaron S. Horn

    (Midwestern Higher Education Compact)

  • Giljae Lee

    (Chungbuk National University)

Abstract

A relatively simple way of measuring institutional effectiveness in relation to degree completion is to estimate the difference between an actual and predicted graduation rate, but the reliability and validity of this method have not been thoroughly examined. Longitudinal data were obtained from IPEDS for both public and private not-for-profit 4-year institutions (n = 1496). Hierarchical panel regression was used to predict 4- and 6-year graduation rates based on structural, demographic, financial, and contextual attributes. A direct effects model yielded effectiveness scores that were highly correlated between consecutive data years (r = 0.65–0.80), which indicated acceptable to good test–retest reliability. A test of convergent validity indicated that effectiveness scores were positively associated with students’ perceptions of a supportive campus environment (r = 0.32–0.45). A test of discriminant validity revealed relatively small correlations between effectiveness scores and institutional attributes, such as educational expenditures (r = 0.07–0.16). The modeling of interaction effects in relation to institutional type marginally improved the validity of effectiveness scores among public but not private institutions. The results suggest that correct model specification can yield residual scores that reliably and validly measure institutional effectiveness in promoting timely degree completion.

Suggested Citation

  • Aaron S. Horn & Giljae Lee, 2016. "The Reliability and Validity of Using Regression Residuals to Measure Institutional Effectiveness in Promoting Degree Completion," Research in Higher Education, Springer;Association for Institutional Research, vol. 57(4), pages 469-496, June.
    • Handle: RePEc:spr:reihed:v:57:y:2016:i:4:d:10.1007_s11162-015-9394-7
    DOI: 10.1007/s11162-015-9394-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11162-015-9394-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11162-015-9394-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Webber, Douglas A. & Ehrenberg, Ronald G., 2010. "Do expenditures other than instructional expenditures affect graduation and persistence rates in American higher education?," Economics of Education Review, Elsevier, vol. 29(6), pages 947-958, December.
    2. Corman, Hope & Davidson, Patricia K., 1984. "Economic aspects of post-secondary schooling decisions," Economics of Education Review, Elsevier, vol. 3(2), pages 131-139, April.
    3. Monks, James, 2000. "The returns to individual and college characteristics: Evidence from the National Longitudinal Survey of Youth," Economics of Education Review, Elsevier, vol. 19(3), pages 279-289, June.
    4. George D. Kuh & Ty M. Cruce & Rick Shoup & Jillian Kinzie & Robert M. Gonyea, 2008. "Unmasking the Effects of Student Engagement on First-Year College Grades and Persistence," The Journal of Higher Education, Taylor & Francis Journals, vol. 79(5), pages 540-563, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zavale, Nelson Casimiro & Santos, Luísa A. & Manuel, Lourenço & da Conceição L. Dias, Maria & Khan, Maida A. & Tostão, Emílio & Mondjana, Ana M., 2017. "Decision-making in African universities demands rigorous data: Evidence from graduation rates at Eduardo Mondlane University in Mozambique," International Journal of Educational Development, Elsevier, vol. 52(C), pages 122-134.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Amanda L. Griffith & Kevin N. Rask, 2016. "The Effect Of Institutional Expenditures On Employment Outcomes And Earnings," Economic Inquiry, Western Economic Association International, vol. 54(4), pages 1931-1945, October.
    2. Bratti, Massimiliano, 2002. "Does the choice of university matter?: a study of the differences across UK universities in life sciences students' degree performance," Economics of Education Review, Elsevier, vol. 21(5), pages 431-443, October.
    3. Daniel C. Hickman & Andrew G. Meyer, 2017. "Does Athletic Success Influence Persistence At Higher Education Institutions? New Evidence Using Panel Data," Contemporary Economic Policy, Western Economic Association International, vol. 35(4), pages 658-676, October.
    4. Eaton, C. & Kulkarni, K. & Birgeneau, Robert & Brady, Henry & Hout, Michael, 2017. "AFFORDING THE DREAM: Student Debt and State Need-Based Grant Aid for Public University Students," University of California at Berkeley, Center for Studies in Higher Education qt24j8945b, Center for Studies in Higher Education, UC Berkeley.
    5. Sharron Xuanren Wang & Arthur Sakamoto, 2021. "Can Higher Education Ameliorate Racial/Ethnic Disadvantage? An Analysis of the Wage Assimilation of College-Educated Hispanic Americans," SAGE Open, , vol. 11(2), pages 21582440211, April.
    6. Cécile Bonneau, 2020. "The Concentration of investment in education in the US (1970-2018)," Working Papers halshs-02875965, HAL.
    7. Amanda Pallais, 2015. "Small Differences That Matter: Mistakes in Applying to College," Journal of Labor Economics, University of Chicago Press, vol. 33(2), pages 493-520.
    8. Angela Boatman & Bridget Terry Long, 2016. "Does Financial Aid Impact College Student Engagement?," Research in Higher Education, Springer;Association for Institutional Research, vol. 57(6), pages 653-681, September.
    9. Basmah Al-Bogami & Tariq Elyas, 2020. "Promoting Middle School Students’ Engagement Through Incorporating iPad Apps in EFL/ESL Classes," SAGE Open, , vol. 10(2), pages 21582440209, May.
    10. Stacy Dale & Alan B. Krueger, "undated". "Estimating the Return to College Selectivity Over the Career Using Administrative Earning Data," Mathematica Policy Research Reports d76ec29a0bbb4b1bb9d285b5a, Mathematica Policy Research.
    11. Long, Mark C., 2010. "Changes in the returns to education and college quality," Economics of Education Review, Elsevier, vol. 29(3), pages 338-347, June.
    12. Gartell, Marie, 2009. "Stability of college rankings - A study of relative earnings estimates applying different methods and models on Swedish data," Working Paper Series 2009:19, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    13. Eliasson, Kent, 2006. "The Role of Ability in Estimating the Returns to College Choice: New Swedish Evidence," Umeå Economic Studies 691, Umeå University, Department of Economics.
    14. Ann M. Gansemer-Topf & Jillian Downey & Katherine Thompson & Ulrike Genschel, 2018. "Did the Recession Impact Student Success? Relationships of Finances, Staffing and Institutional Type on Retention," Research in Higher Education, Springer;Association for Institutional Research, vol. 59(2), pages 174-197, March.
    15. Eaton, Charlie & Kulkarni, Sheisha & Birgeneau, Robert & Brady, Henry & Hout, Michael, 2017. "AFFORDING THE DREAM: Student Debt and State Need-Based Grant Aid for Public University Students," University of California at Berkeley, Center for Studies in Higher Education qt093215zt, Center for Studies in Higher Education, UC Berkeley.
    16. Zara Daghbashyan & Björn Hårsman, 2014. "University choice and entrepreneurship," Small Business Economics, Springer, vol. 42(4), pages 729-746, April.
    17. Havranek, Tomas & Irsova, Zuzana & Zeynalova, Olesia, 2017. "Tuition Reduces Enrollment Less Than Commonly Thought," MPRA Paper 78813, University Library of Munich, Germany.
    18. Dr. Brunilda Zenelaga & Blerina Hamzallari, 2018. "The Role of the Family in the Education of Children in Conflict with the Law: Empirical Evidence from Albanian Context," European Journal of Education Articles, Revistia Research and Publishing, vol. 1, September.
    19. Javier García-Estévez & Néstor Duch-Brown, 2012. "Student graduation: to what extent does university expenditure matter?," Working Papers 2012/4, Institut d'Economia de Barcelona (IEB).
    20. Yajun Wu & Xia Kang, 2021. "A Moderated Mediation Model of Expectancy-Value Interactions, Engagement, and Foreign Language Performance," SAGE Open, , vol. 11(4), pages 21582440211, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:reihed:v:57:y:2016:i:4:d:10.1007_s11162-015-9394-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.