IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v68y2018icp81-89.html

A counterfactual impact evaluation of a bilingual program on students’ grade point average at a spanish university

Author

Listed:
  • Arco-Tirado, J.L.
  • Fernández-Martín, F.
  • Ramos-García, A.M.
  • Littvay, L.
  • Villoria, J.
  • Naranjo, J.A.

Abstract

This observational study intends to estimate the causal effects of an English as a Medium of Instruction (EMI) program (as predictor) on students Grade Point Average (GPA) (as outcome) at a particular University in Spain by using a Counterfactual Impact Evaluation (CIE). The need to address the crucial question of causal inferences in EMI programs to produce credible evidences of successful interventions contrasts, however, with the absence of experimental or quasi-experimental research and evaluation designs in the field. CIE approach is emerging as a methodologically viable solution to bridge that gap. The program evaluated here consisted in delivering an EMI program in a Primary Education Teacher Training Degree group. After achieving balance on the observed covariates and recreating a situation that would have been expected in a randomized experiment, three matching approaches such as genetic matching, nearest neighbor matching and Coarsened Exact Matching were used to analyze observational data from a total of 1288 undergraduate students, including both treatment and control group. Results show unfavorable effects of the bilingual group treatment condition. Potential interpretations and recommendations are provided in order to strengthen future causal evidences of bilingual education programs’ effectiveness in Higher Education.

Suggested Citation

  • Arco-Tirado, J.L. & Fernández-Martín, F. & Ramos-García, A.M. & Littvay, L. & Villoria, J. & Naranjo, J.A., 2018. "A counterfactual impact evaluation of a bilingual program on students’ grade point average at a spanish university," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 81-89.
    • Handle: RePEc:eee:epplan:v:68:y:2018:i:c:p:81-89
    DOI: 10.1016/j.evalprogplan.2018.02.013
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S014971891730335X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2018.02.013?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Card, David & Ibarrarán, Pablo & Villa, Juan Miguel, 2011. "Building in an Evaluation Component for Active Labor Market Programs: A Practitioner's Guide," IZA Discussion Papers 6085, IZA Network @ LISER.
    2. Vincent Tinto, 1997. "Classrooms as Communities," The Journal of Higher Education, Taylor & Francis Journals, vol. 68(6), pages 599-623, November.
    3. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2012. "Causal Inference without Balance Checking: Coarsened Exact Matching," Political Analysis, Cambridge University Press, vol. 20(1), pages 1-24, January.
    4. repec:idb:brikps:58518 is not listed on IDEAS
    5. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    6. Brindusa Anghel & Antonio Cabrales & Jesus M. Carro, 2016. "Evaluating A Bilingual Education Program In Spain: The Impact Beyond Foreign Language Learning," Economic Inquiry, Western Economic Association International, vol. 54(2), pages 1202-1223, April.
    7. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pelucha, Martin & Kveton, Viktor & Potluka, Oto, 2019. "Using mixed method approach in measuring effects of training in firms: Case study of the European Social Fund support," Evaluation and Program Planning, Elsevier, vol. 73(C), pages 146-155.
    2. del Campo, Cristina & Urquía-Grande, Elena & Pascual-Ezama, David, 2023. "Internationalizing the business school: A comparative analysis of English-medium and Spanish-medium instruction impact on student performance," Evaluation and Program Planning, Elsevier, vol. 98(C).
    3. Teng Lin & Jun Lei, 2021. "English-Medium Instruction and Content Learning in Higher Education: Effects of Medium of Instruction, English Proficiency, and Academic Ability," SAGE Open, , vol. 11(4), pages 21582440211, November.
    4. Lucia Švábová & Katarína Kramárová & Marek Ďurica, 2021. "Evaluation of the Effects of the Graduate Practice in Slovakia: Comparison of Results of Counterfactual Methods," Central European Business Review, Prague University of Economics and Business, vol. 2021(4), pages 1-31.
    5. Lucia Svabova & Eva Nahalkova Tesarova & Marek Durica & Lenka Strakova, 2021. "Evaluation of the impacts of the COVID-19 pandemic on the development of the unemployment rate in Slovakia: counterfactual before-after comparison," Equilibrium. Quarterly Journal of Economics and Economic Policy, Institute of Economic Research, vol. 16(2), pages 261-284, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adeola Oyenubi & Martin Wittenberg, 2021. "Does the choice of balance-measure matter under genetic matching?," Empirical Economics, Springer, vol. 61(1), pages 489-502, July.
    2. Adeola Oyenubi, 2020. "On sensitivity of Genetic Matching to the choice of balance measure," ERSA Working Paper Series, Economic Research Southern Africa, vol. 0.
    3. Gerhard Krug, 2017. "Augmenting propensity score equations to avoid misspecification bias – Evidence from a Monte Carlo simulation [Erweiterung der Propensity Score Gleichung zur Vermeidung von Fehlspezifikationen? Eine Monte Carlo Simulation]," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 11(3), pages 205-231, December.
    4. Dudel Christian & Garbuszus Jan Marvin & Ott Notburga & Werding Martin, 2017. "Matching as Non-Parametric Preprocessing for the Estimation of Equivalence Scales," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 237(2), pages 115-141, April.
    5. Ndiaye, Abdou Salam, 2025. "Does access to credit promote the use of modern energy services? Evidence from rural Nigeria," Energy Economics, Elsevier, vol. 148(C).
    6. Gary King & Christopher Lucas & Richard A. Nielsen, 2017. "The Balance‐Sample Size Frontier in Matching Methods for Causal Inference," American Journal of Political Science, John Wiley & Sons, vol. 61(2), pages 473-489, April.
    7. Jones, A.M.; & Pastore, C.; & Rice, N.;, 2018. "Tracking pupils into adulthood: selective schools and long-term well-being in the 1958 British cohort," Health, Econometrics and Data Group (HEDG) Working Papers 18/32, HEDG, c/o Department of Economics, University of York.
    8. Jason J. Sauppe & Sheldon H. Jacobson, 2017. "The role of covariate balance in observational studies," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(4), pages 323-344, June.
    9. Federico Biagi & Daniele Bondonio & Alberto Martini, 2015. "Counterfactual Impact Evaluation of Enterprise Support Programmes. Evidence from a Decade of Subsidies to Italian Firm," ERSA conference papers ersa15p1619, European Regional Science Association.
    10. Verena Lauber & Johanna Storck, 2016. "Helping with the Kids? How Family-Friendly Workplaces Affect Parental Well-Being and Behavior," Discussion Papers of DIW Berlin 1630, DIW Berlin, German Institute for Economic Research.
    11. Jian Jiu Chen & Sai Yin Ho & Wing Man Au & Man Ping Wang & Tai Hing Lam, 2015. "Family Smoking, Exposure to Secondhand Smoke at Home and Family Unhappiness in Children," IJERPH, MDPI, vol. 12(11), pages 1-14, November.
    12. Kube, Roland & von Graevenitz, Kathrine & Löschel, Andreas & Massier, Philipp, 2019. "Do voluntary environmental programs reduce emissions? EMAS in the German manufacturing sector," Energy Economics, Elsevier, vol. 84(S1).
    13. Helena Ting & Martina Bozzola & Chiara Ravetti, 2024. "Tuberculosis and labour market participation: Evidence from South Africa," South African Journal of Economics, Economic Society of South Africa, vol. 92(4), pages 444-474, December.
    14. Blackman, Allen, 2013. "Evaluating forest conservation policies in developing countries using remote sensing data: An introduction and practical guide," Forest Policy and Economics, Elsevier, vol. 34(C), pages 1-16.
    15. K. Poehlmann & R. Helm & O. Mauroner & J. Auburger, 2021. "Corporate spin-offs’ success factors: management lessons from a comparative empirical analysis with research-based spin-offs," Review of Managerial Science, Springer, vol. 15(6), pages 1767-1796, August.
    16. Cao, Hung & Phan, Hieu V. & Silveri, Sabatino, 2024. "Data breach disclosures and stock price crash risk: Evidence from data breach notification laws," International Review of Financial Analysis, Elsevier, vol. 93(C).
    17. Cappelletti, Matilde & Giuffrida, Leonardo M., 2022. "Targeted bidders in government tenders," ZEW Discussion Papers 22-030, ZEW - Leibniz Centre for European Economic Research.
    18. García, Sandra & Harker, Arturo & Cuartas, Jorge, 2019. "Building dreams: The short-term impacts of a conditional cash transfer program on aspirations for higher education," International Journal of Educational Development, Elsevier, vol. 64(C), pages 48-57.
    19. Owusu, Eric S. & Bravo-Ureta, Boris E., 2022. "Reap when you sow? The productivity impacts of early sowing in Malawi," Agricultural Systems, Elsevier, vol. 199(C).
    20. Idil Uz Akdogan, 2020. "Understanding the dynamics of foreign reserve management: The central bank intervention policy and the exchange rate fundamentals," International Economics, CEPII research center, issue 161, pages 41-55.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:68:y:2018:i:c:p:81-89. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.