IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v38y2014i2p99-132.html
   My bibliography  Save this article

Trying to Raise (Low) Math Achievement and to Promote (Rigorous) Policy Evaluation in Italy: Evidence From a Large-Scale Randomized Trial

Author

Listed:
  • Gianluca Argentin
  • Aline Pennisi
  • Daniele Vidoni
  • Giovanni Abbiati
  • Andrea Caputo

Abstract

Background: Italy is a country showing low math achievement, especially in the Southern regions. Moreover, national student assessments are recent and rigorous policy evaluation is lacking. This study presents the results of one of the first randomized controlled trials implemented in Italian schools in order to measure the effects of a professional development (PD) program for teachers on student math achievement. The program was already at scale when it was being evaluated. Objective: Assessing the effects of a PD program for math teachers on their students’ achievement and making suggestions for future policy evaluations. Design: A large-scale clustered randomized control trial has been conducted. It involves 175 lower secondary schools (sixth - eighth grade) in four among the Italian lowest performing regions. Alongside national standard math assessments, the project collected a wide amount of information. Subjects: Math in lower secondary schools. Measures: Math achievement as measured by standardized tests provided by the National Education Assessment Institute (Istituto Nazionale per la Valutazione del Sistema di Istruzione e Formazione); teacher and student practices and attitudes collected through questionnaires. Results: Findings suggest that the program had no significant impact on math scores during the first year (when the program was held). Nonetheless some heterogeneity was detected, as the treatment does seem “to work†with middle-aged teachers. Moreover, effects on teaching practice and student attitudes appear. Conclusion: Some effects attributable to the intervention have been detected. Moreover, this project shows that a rigorous approach to evaluation is feasible also in a context lacking attention towards evidence-based policies, such the Italian school system.

Suggested Citation

  • Gianluca Argentin & Aline Pennisi & Daniele Vidoni & Giovanni Abbiati & Andrea Caputo, 2014. "Trying to Raise (Low) Math Achievement and to Promote (Rigorous) Policy Evaluation in Italy: Evidence From a Large-Scale Randomized Trial," Evaluation Review, , vol. 38(2), pages 99-132, April.
    • Handle: RePEc:sae:evarev:v:38:y:2014:i:2:p:99-132
    DOI: 10.1177/0193841X14529125
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X14529125
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X14529125?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bertoni, Marco & Brunello, Giorgio & Rocco, Lorenzo, 2013. "When the cat is near, the mice won't play: The effect of external examiners in Italian schools," Journal of Public Economics, Elsevier, vol. 104(C), pages 65-77.
    2. Maria De Paola & Vincenzo Scoppa & Rosanna Nisticò, 2012. "Monetary Incentives and Student Achievement in a Depressed Labor Market: Results from a Randomized Experiment," Journal of Human Capital, University of Chicago Press, vol. 6(1), pages 56-85.
    3. Massimiliano Bratti & Daniele Checchi & Antonio Filippin, 2007. "Geographical Differences in Italian Students' Mathematical Competencies: Evidence from Pisa 2003," Giornale degli Economisti, GDE (Giornale degli Economisti e Annali di Economia), Bocconi University, vol. 66(3), pages 299-333, November.
    4. Barbieri, Gianna & Rossetti, Claudio & Sestito, Paolo, 2011. "The determinants of teacher mobility: Evidence using Italian teachers’ transfer applications," Economics of Education Review, Elsevier, vol. 30(6), pages 1430-1444.
    5. De Paola, Maria & Scoppa, Vincenzo, 2011. "Frequency of examinations and student achievement in a randomized experiment," Economics of Education Review, Elsevier, vol. 30(6), pages 1416-1429.
    6. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2011. "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood," NBER Working Papers 17699, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adriana Di Liberto & Fabiano Schivardi & Giovanni Sulis, 2015. "Managerial practices and student performance," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 30(84), pages 683-728.
    2. Tommaso Agasisti & Patrizia Falzetti, 2017. "Between-classes sorting within schools and test scores: an empirical analysis of Italian junior secondary schools," International Review of Economics, Springer;Happiness Economics and Interpersonal Relations (HEIRS), vol. 64(1), pages 1-45, March.
    3. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    4. Cavalieri, Marina & Finocchiaro Castro, Massimo & Guccio, Calogero, 2023. "Organised crime and educational outcomes in Southern Italy: An empirical investigation," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    5. Lee, Se Woong & Lee, Eunjung Alice, 2020. "Teacher qualification matters: The association between cumulative teacher qualification and students’ educational attainment," International Journal of Educational Development, Elsevier, vol. 77(C).
    6. De Paola, Maria & Gioia, Francesca & Scoppa, Vincenzo, 2018. "The adverse consequences of tournaments: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 151(C), pages 1-18.
    7. Marina Cavalieri & Massimo Finocchiaro Castro & Calogero Guccio, 2020. "Does the Fish Rot from the Head? Organised Crime and Educational Outcomes in Southern Italy," Working papers 97, Società Italiana di Economia Pubblica.
    8. Tommaso Agasisti & Veronica Minaya, 2018. "Evaluating the Stability of School Performance Estimates for School Choice: Evidence for Italian Primary Schools," Working papers 67, Società Italiana di Economia Pubblica.
    9. Marco Paccagnella & Paolo Sestito, 2014. "School cheating and social capital," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 367-388, August.
    10. Tommaso Agasisti & Francesca Ieva & Anna Maria Paganoni, 2017. "Heterogeneity, school-effects and the North/South achievement gap in Italian secondary education: evidence from a three-level mixed model," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 26(1), pages 157-180, March.
    11. Adam M. Lavecchia & Heidi Liu & Philip Oreopoulos, 2014. "Behavioral Economics of Education: Progress and Possibilities," NBER Working Papers 20609, National Bureau of Economic Research, Inc.
    12. Sergio Longobardi & Patrizia Falzetti & Margherita Maria Pagliuca, 2018. "Quis custiodet ipsos custodes? How to detect and correct teacher cheating in Italian student data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(3), pages 515-543, August.
    13. Luis Aguiar & Jörg Claussen & Christian Peukert, 2018. "Catch Me If You Can: Effectiveness and Consequences of Online Copyright Enforcement," Information Systems Research, INFORMS, vol. 29(3), pages 656-678, September.
    14. Dan Goldhaber & Roddy Theobald, 2013. "Managing the Teacher Workforce in Austere Times: The Determinants and Implications of Teacher Layoffs," Education Finance and Policy, MIT Press, vol. 8(4), pages 494-527, October.
    15. Emmanuel Dechenaux & Dan Kovenock & Roman Sheremeta, 2015. "A survey of experimental research on contests, all-pay auctions and tournaments," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 609-669, December.
    16. Sergio Longobardi & Margherita Maria Pagliuca & Andrea Regoli, 2018. "Can problem-solving attitudes explain the gender gap in financial literacy? Evidence from Italian students’ data," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1677-1705, July.
    17. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    18. Shawn Ni & Michael Podgursky, 2016. "How Teachers Respond to Pension System Incentives: New Estimates and Policy Applications," Journal of Labor Economics, University of Chicago Press, vol. 34(4), pages 1075-1104.
    19. Kevin C. Bastian & Gary T. Henry & Charles L. Thompson, 2013. "Incorporating Access to More Effective Teachers into Assessments of Educational Resource Equity," Education Finance and Policy, MIT Press, vol. 8(4), pages 560-580, October.
    20. Stacy, Brian, 2014. "Ranking Teachers when Teacher Value-Added is Heterogeneous Across Students," EconStor Preprints 104743, ZBW - Leibniz Information Centre for Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:38:y:2014:i:2:p:99-132. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.