IDEAS home Printed from https://ideas.repec.org/p/zbw/rwirep/603.html

The effect of supplemental instruction on academic performance: An encouragement design experiment

Author

Listed:
  • Paloyo, Alfredo R.
  • Rogan, Sally
  • Siminski, Peter

Abstract

While randomized controlled trials (RCTs) are the 'gold standard' for impact evaluation, they face numerous practical barriers to implementation. In some circumstances, a randomized-encouragement design (RED) is a viable alternative, but applications are surprisingly rare. We discuss the strengths and challenges of RED and apply it to evaluate a mature Supplemental Instruction (SI) or PASS (Peer Assisted Study Session) program at an Australian university. A randomly selected subgroup of students from first-year courses (N = 6954) was offered large incentives (worth AUD 55,000) to attend PASS, which increased attendance by an estimated 0.47 hours each. This first-stage (inducement) effect did not vary with the size of the incentive and was larger (0.89) for students from disadvantaged backgrounds. Instrumental variable estimates suggest that one hour of PASS improved grades by 0.065 standard deviations, which is consistent with the non-experimental literature. However, this estimate is not statistically significant, reflecting limited statistical power. The estimated effect is largest for students in their first semester at university.

Suggested Citation

  • Paloyo, Alfredo R. & Rogan, Sally & Siminski, Peter, 2016. "The effect of supplemental instruction on academic performance: An encouragement design experiment," Ruhr Economic Papers 603, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    • Handle: RePEc:zbw:rwirep:603
    DOI: 10.4419/86788700
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/128629/1/848646029.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.4419/86788700?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pugatch, Todd & Wilson, Nicholas, 2018. "Nudging study habits: A field experiment on peer tutoring in higher education," Economics of Education Review, Elsevier, vol. 62(C), pages 151-161.
    2. Todd Pugatch & Nicholas Wilson, 2024. "Nudging Demand for Academic Support Services: Experimental and Structural Evidence from Higher Education," Journal of Human Resources, University of Wisconsin Press, vol. 59(5), pages 1637-1682.
    3. Tarek Azzam & Michael Bates & David Fairris, 2019. "Do Learning Communities Increase First Year College Retention? Testing Sample Selection and External Validity of Randomized Control Trials," Working Papers 202002, University of California at Riverside, Department of Economics.
    4. Gordanier, John & Hauk, William & Sankaran, Chandini, 2019. "Early intervention in college classes and improved student outcomes," Economics of Education Review, Elsevier, vol. 72(C), pages 23-29.
    5. Azzam, Tarek & Bates, Michael D. & Fairris, David, 2022. "Do learning communities increase first year college retention? Evidence from a randomized control trial," Economics of Education Review, Elsevier, vol. 89(C).
    6. Matilde Reséndiz-Castro & Rosalba Zepeda-Bautista & Ignacio Enrique Peón-Escalante, 2022. "Transdisciplinary Cyber-systemic Design of Instruments to Measure Academic Performance in Middle and Higher Education Systems," Systemic Practice and Action Research, Springer, vol. 35(3), pages 395-440, June.
    7. Hardt, David & Nagler, Markus & Rincke, Johannes, 2023. "Tutoring in (online) higher education: Experimental evidence," Economics of Education Review, Elsevier, vol. 92(C).
    8. Alicia Brandt & Hamid R. Oskorouchi & Alfonso Sousa-Poza, 2024. "The effect of SMS nudges on higher education performance," Empirical Economics, Springer, vol. 66(5), pages 2311-2334, May.
    9. Paloyo, Alfredo R. & Rogan, Sally & Siminski, Peter, 2016. "The effect of supplemental instruction on academic performance: An encouragement design experiment," Economics of Education Review, Elsevier, vol. 55(C), pages 57-69.
    10. Andrew Leigh, 2021. "Australian Mobility Report Cards: Which Universities Admit the Most Disadvantaged Students?," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 54(3), pages 331-342, September.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:rwirep:603. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/rwiesde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.