IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v79y2014i1p130-153.html
   My bibliography  Save this article

School System Evaluation by Value Added Analysis Under Endogeneity

Author

Listed:
  • Jorge Manzi
  • Ernesto San Martín
  • Sébastien Van Bellegem

Abstract

Value added is a common tool in educational research on effectiveness. It is often modeled as a (prediction of a) random effect in a specific hierarchical linear model. This paper shows that this modeling strategy is not valid when endogeneity is present. Endogeneity stems, for instance, from a correlation between the random effect in the hierarchical model and some of its covariates. This paper shows that this phenomenon is far from exceptional and can even be a generic problem when the covariates contain the prior score attainments, a typical situation in value added modeling. Starting from a general, model-free definition of value added, the paper derives an explicit expression of the value added in an endogeneous hierarchical linear Gaussian model. Inference on value added is proposed using an instrumental variable approach. The impact of endogeneity on the value added and the estimated value added is calculated accurately. This is also illustrated on a large data set of individual scores of about 200,000 students in Chile. Copyright The Psychometric Society 2014

Suggested Citation

  • Jorge Manzi & Ernesto San Martín & Sébastien Van Bellegem, 2014. "School System Evaluation by Value Added Analysis Under Endogeneity," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 130-153, January.
    • Handle: RePEc:spr:psycho:v:79:y:2014:i:1:p:130-153
    DOI: 10.1007/s11336-013-9338-0
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11336-013-9338-0
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11336-013-9338-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Alan B. Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 69-85, Fall.
    2. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    3. San Martin, Ernesto & Rolin, Jean-Marie, 2013. "Identification of parametric Rasch-type models," LIDAM Reprints ISBA 2013002, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    4. MANZI, Jorge & SAN MARTIN, Ernesto & VAN BELLEGEM, Sébastien, 2010. "School system evaluation by value-added analysis under endogeneity," LIDAM Discussion Papers CORE 2010046, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    5. Picci, Giorgio, 1989. "Parametrization of factor analysis models," Journal of Econometrics, Elsevier, vol. 41(1), pages 17-38, May.
    6. Jee-Seon Kim & Edward Frees, 2007. "Multilevel Modeling with Correlated Effects," Psychometrika, Springer;The Psychometric Society, vol. 72(4), pages 505-533, December.
    7. Blundell, Richard & Bond, Stephen, 1998. "Initial conditions and moment restrictions in dynamic panel data models," Journal of Econometrics, Elsevier, vol. 87(1), pages 115-143, August.
    8. Peter Ebbes & Ulf Böckenholt & Michel Wedel, 2004. "Regressor and random‐effects dependencies in multilevel models," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 58(2), pages 161-178, May.
    9. Florens, Jean-Pierre & Johannes, Jan & Van Bellegem, Sebastien, 2012. "Instrumental regression in partially linear models," LIDAM Reprints ISBA 2012017, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    10. Andrew Ray & Tanya McCormack & Helen Evans, 2009. "Value Added in English Schools," Education Finance and Policy, MIT Press, vol. 4(4), pages 415-438, October.
    11. Claudio Sapelli & Bernardita Vial, 2002. "The Performance of Private and Public Schools in the Chilean Voucher System," Latin American Journal of Economics-formerly Cuadernos de Economía, Instituto de Economía. Pontificia Universidad Católica de Chile., vol. 39(118), pages 423-454.
    12. Jean‐Pierre Florens & Jan Johannes & Sébastien Van Bellegem, 2012. "Instrumental regression in partially linear models," Econometrics Journal, Royal Economic Society, vol. 15(2), pages 304-324, June.
    13. Florens, Jean-Pierre & Mouchart, Michel, 1985. "A Linear Theory for Noncausality," Econometrica, Econometric Society, vol. 53(1), pages 157-175, January.
    14. Neil H. Spencer & Antony Fielding, 2002. "A Comparison of Modelling Strategies for Value-Added Analyses of Educational Data," Computational Statistics, Springer, vol. 17(1), pages 103-116, March.
    15. Saïd Hanchane & Tarek Mostafa, 2012. "Solving endogeneity problems in multilevel estimation: an example using education production functions," Journal of Applied Statistics, Taylor & Francis Journals, vol. 39(5), pages 1101-1114, November.
    16. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    17. Jee-Seon Kim & Edward Frees, 2006. "Omitted Variables in Multilevel Models," Psychometrika, Springer;The Psychometric Society, vol. 71(4), pages 659-690, December.
    18. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    19. Derek C. Briggs & Jonathan P. Weeks, 2011. "The Persistence of School-Level Value-Added," Journal of Educational and Behavioral Statistics, , vol. 36(5), pages 616-637, October.
    20. Hendry, David F., 1995. "Dynamic Econometrics," OUP Catalogue, Oxford University Press, number 9780198283164, Decembrie.
    21. Joshua Angrist & Alan Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Working Papers 834, Princeton University, Department of Economics, Industrial Relations Section..
    22. Paredes, Ricardo D. & Paredes, Valentina, 2009. "Chile: academic performance and educational management under a rigid employment regime," Revista CEPAL, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL), December.
    23. Sean F. Reardon & Stephen W. Raudenbush, 2009. "Assumptions of Value-Added Models for Estimating School Effects," Education Finance and Policy, MIT Press, vol. 4(4), pages 492-519, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Melanie Birke & Sebastien Van Bellegem & Ingrid Van Keilegom, 2017. "Semi-parametric Estimation in a Single-index Model with Endogenous Variables," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 44(1), pages 168-191, March.
    2. Vanhems, Anne & Van Keilegom, Ingrid, 2019. "Estimation Of A Semiparametric Transformation Model In The Presence Of Endogeneity," Econometric Theory, Cambridge University Press, vol. 35(1), pages 73-110, February.
    3. Garritt L. Page & Ernesto San Martín & Javiera Orellana & Jorge González, 2017. "Exploring complete school effectiveness via quantile value added," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 315-340, January.
    4. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.
    5. Luis Alejandro Lopez-Agudo & Oscar David Marcenaro Gutierrez, 2016. "Identifying effective teachers: The case study of Spain," Investigaciones de Economía de la Educación volume 11, in: José Manuel Cordero Ferrera & Rosa Simancas Rodríguez (ed.), Investigaciones de Economía de la Educación 11, edition 1, volume 11, chapter 18, pages 349-366, Asociación de Economía de la Educación.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Manzi, Jorge & San Martin, Ernesto & Van Bellegem, Sébastien, 2010. "School System Evaluation By Value-Added Analysis under Endogeneity," IDEI Working Papers 631, Institut d'Économie Industrielle (IDEI), Toulouse.
    2. Vanhems, Anne & Van Keilegom, Ingrid, 2019. "Estimation Of A Semiparametric Transformation Model In The Presence Of Endogeneity," Econometric Theory, Cambridge University Press, vol. 35(1), pages 73-110, February.
    3. Tesfaye, Wondimagegn & Tirivayi, Nyasha, 2020. "Crop diversity, household welfare and consumption smoothing under risk: Evidence from rural Uganda," World Development, Elsevier, vol. 125(C).
    4. Guilhem Bascle, 2008. "Controlling for endogeneity with instrumental variables in strategic management research," Post-Print hal-00576795, HAL.
    5. Katherine E. Castellano & Sophia Rabe-Hesketh & Anders Skrondal, 2014. "Composition, Context, and Endogeneity in School and Teacher Comparisons," Journal of Educational and Behavioral Statistics, , vol. 39(5), pages 333-367, October.
    6. Aleksey Oshchepkov & Anna Shirokanova, 2020. "Multilevel Modeling For Economists: Why, When And How," HSE Working papers WP BRP 233/EC/2020, National Research University Higher School of Economics.
    7. Xiaohong Chen & Andres Santos, 2018. "Overidentification in Regular Models," Econometrica, Econometric Society, vol. 86(5), pages 1771-1817, September.
    8. Asadul Islam & Dietrich K. Fausten, 2008. "Skilled Immigration and Wages in Australia," The Economic Record, The Economic Society of Australia, vol. 84(s1), pages 66-82, September.
    9. Xiaohong Chen & Victor Chernozhukov & Sokbae Lee & Whitney K. Newey, 2014. "Local Identification of Nonparametric and Semiparametric Models," Econometrica, Econometric Society, vol. 82(2), pages 785-809, March.
    10. Markus Brückner & Antonio Ciccone, 2011. "Rain and the Democratic Window of Opportunity," Econometrica, Econometric Society, vol. 79(3), pages 923-947, May.
    11. Ricardo Bebczuk & Klaus Schmidt-Hebbel, 2010. "Revisiting the Feldstein-Horioka Puzzle: An institutional sector view," Económica, Instituto de Investigaciones Económicas, Facultad de Ciencias Económicas, Universidad Nacional de La Plata, vol. 0, pages 69-104, January-D.
    12. Ali Uyar & Simone Pizzi & Fabio Caputo & Cemil Kuzey & Abdullah S. Karaman, 2022. "Do shareholders reward or punish risky firms due to CSR reporting and assurance?," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 43(5), pages 1596-1620, July.
    13. Mathenge, Mary K. & Smale, Melinda & Olwande, John, 2012. "The Impact of Maize Hybrids on Income, Poverty, and Inequality among Smallholder Farmers in Kenya," Food Security International Development Working Papers 146931, Michigan State University, Department of Agricultural, Food, and Resource Economics.
    14. Joan Costa-Font & Sergi Jiménez-Martín & Cristina Vilaplana-Prieto, 2016. "Thinking of Incentivizing Care? The Effect of Demand Subsidies on Informal Caregiving and Intergenerational Transfers," Working Papers 2016-08, FEDEA.
    15. Deodat E. Adenutsi & Meshach J. Aziakpono & Matthew K. Ocran, 2011. "The Changing Impact Of Macroeconomic Environment On Remittance Inflows In Sub-Saharan Africa," Journal of Academic Research in Economics, Spiru Haret University, Faculty of Accounting and Financial Management Constanta, vol. 3(2 (July)), pages 136-167.
    16. Monica Schuster & Miet Maertens, 2013. "8 Private Food Standards and Firm-Level Trade Effects: A Dynamic Analysis of the Peruvian Asparagus Export Sector," Frontiers of Economics and Globalization, in: Nontariff Measures with Market Imperfections: Trade and Welfare Implications, pages 187-213, Emerald Group Publishing Limited.
    17. Melanie Birke & Sebastien Van Bellegem & Ingrid Van Keilegom, 2017. "Semi-parametric Estimation in a Single-index Model with Endogenous Variables," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 44(1), pages 168-191, March.
    18. Irene Botosaru & Chris Muris & Senay Sokullu, 2022. "Time-Varying Linear Transformation Models with Fixed Effects and Endogeneity for Short Panels," Department of Economics Working Papers 2022-01, McMaster University.
    19. Chen, Sheng-Syan & Hsu, Ching-Yu & Huang, Chia-Wei, 2016. "The white squire defense: Evidence from private investments in public equity," Journal of Banking & Finance, Elsevier, vol. 64(C), pages 16-35.
    20. Antonio Ruiz Porras, 2016. "La investigación econométrica mediante paneles de datos:historia, modelos y usos en México," Archivos Revista Economía y Política., Facultad de Ciencias Económicas y Administrativas, Universidad de Cuenca., vol. 24, pages 11-32, Julio.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:79:y:2014:i:1:p:130-153. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.