IDEAS home Printed from https://ideas.repec.org/p/tin/wpaper/20170019.html
   My bibliography  Save this paper

Comparing Econometric Methods to Empirically Evaluate Job-Search Assistance

Author

Listed:
  • Paul Muller

    (University of Gothenburg, Sweden)

  • Bas van der Klaauw

    (VU Amsterdam, The Netherlands)

  • Arjan Heyma

    (SEO Economic Research, The Netherlands)

Abstract

We test whether different empirical methods give different results when evaluating job-search assistance programs. Budgetary problems at the Dutch unemployment insurance (UI) administration in March 2010, caused a sharp drop in the availability of these programs. Using administrative data provided by the UI administration, we evaluate the effect of the program using (1) the policy discontinuity as a quasi-experiment, (2) conventional matching methods, and (3) the timing-of-events model. All three methods use the same data to consider the same program in the same setting, and also yield similar results. The program reduces job finding during the first six months after enrollment. At longer durations, the quasi-experimental estimates are not significantly different from zero, while the non-experimental methods show a small negative effect.

Suggested Citation

  • Paul Muller & Bas van der Klaauw & Arjan Heyma, 2017. "Comparing Econometric Methods to Empirically Evaluate Job-Search Assistance," Tinbergen Institute Discussion Papers 17-019/V, Tinbergen Institute.
    • Handle: RePEc:tin:wpaper:20170019
    as

    Download full text from publisher

    File URL: https://papers.tinbergen.nl/17019.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. de Groot, Nynke & van der Klaauw, Bas, 2019. "The effects of reducing the entitlement period to unemployment insurance benefits," Labour Economics, Elsevier, vol. 57(C), pages 195-208.
    2. Gerard J. van den Berg & Bas van der Klaauw, 2006. "Counseling And Monitoring Of Unemployed Workers: Theory And Evidence From A Controlled Social Experiment," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 47(3), pages 895-936, August.
    3. Luc Behaghel & Bruno Cr?pon & Marc Gurgand, 2014. "Private and Public Provision of Counseling to Job Seekers: Evidence from a Large Controlled Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 6(4), pages 142-174, October.
    4. Gerard J. van den Berg & Antoine Bozio & Mónica Costa Dias, 2020. "Policy discontinuity and duration outcomes," Quantitative Economics, Econometric Society, vol. 11(3), pages 871-916, July.
    5. Jaap H. Abbring & Gerard J. Berg & Jan C. Ours, 2005. "The Effect of Unemployment Insurance Sanctions on the Transition Rate from Unemployment to Employment," Economic Journal, Royal Economic Society, vol. 115(505), pages 602-630, July.
    6. Gerard J. van den Berg & Bas van der Klaauw & Jan C. van Ours, 2004. "Punitive Sanctions and the Transition Rate from Welfare to Work," Journal of Labor Economics, University of Chicago Press, vol. 22(1), pages 211-241, January.
    7. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    8. Michael Gerfin & Michael Lechner, 2002. "A Microeconometric Evaluation of the Active Labour Market Policy in Switzerland," Economic Journal, Royal Economic Society, vol. 112(482), pages 854-893, October.
    9. Michael Lechner & Conny Wunsch, 2009. "Are Training Programs More Effective When Unemployment Is High?," Journal of Labor Economics, University of Chicago Press, vol. 27(4), pages 653-692, October.
    10. Kluve, Jochen, 2010. "The effectiveness of European active labor market programs," Labour Economics, Elsevier, vol. 17(6), pages 904-918, December.
    11. repec:adr:anecst:y:2012:i:107-108:p:7 is not listed on IDEAS
    12. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labour Market Policy Evaluations: A Meta-Analysis," Economic Journal, Royal Economic Society, vol. 120(548), pages 452-477, November.
    13. Graversen, Brian Krogh & van Ours, Jan C., 2008. "How to help unemployed find jobs quickly: Experimental evidence from a mandatory activation program," Journal of Public Economics, Elsevier, vol. 92(10-11), pages 2020-2035, October.
    14. Barbara Sianesi, 2004. "An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 133-155, February.
    15. Abbring, Jaap H & van den Berg, Gerard J, 2005. "Social experiments and intrumental variables with duration outcomes," Working Paper Series 2005:11, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    16. Abadie, Alberto & Imbens, Guido W., 2011. "Bias-Corrected Matching Estimators for Average Treatment Effects," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 1-11.
    17. Cockx, Bart & Dejemeppe, Muriel, 2012. "Monitoring job search effort: An evaluation based on a regression discontinuity design," Labour Economics, Elsevier, vol. 19(5), pages 729-737.
    18. David Card & Pablo Ibarrarán & Ferdinando Regalia & David Rosas-Shady & Yuri Soares, 2011. "The Labor Market Impacts of Youth Training in the Dominican Republic," Journal of Labor Economics, University of Chicago Press, vol. 29(2), pages 267-300.
    19. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    20. Bruno Crépon & Marc Ferracci & Denis Fougere, 2012. "Training the Unemployed in France: How Does it Affect Unemployment Duration and Recurrence?," Annals of Economics and Statistics, GENES, issue 107-108, pages 175-199.
    21. Michael Lechner & Ruth Miquel & Conny Wunsch, 2011. "Long‐Run Effects Of Public Sector Sponsored Training In West Germany," Journal of the European Economic Association, European Economic Association, vol. 9(4), pages 742-784, August.
    22. Guido W. Imbens, 2015. "Matching Methods in Practice: Three Examples," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 373-419.
    23. Abbring, Jaap H. & Heckman, James J., 2007. "Econometric Evaluation of Social Programs, Part III: Distributional Treatment Effects, Dynamic Treatment Effects, Dynamic Discrete Choice, and General Equilibrium Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 72, Elsevier.
    24. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    25. Card, David & Sullivan, Daniel G, 1988. "Measuring the Effect of Subsidized Training Programs on Movements in and out of Employment," Econometrica, Econometric Society, vol. 56(3), pages 497-530, May.
    26. Stephen Kastoryano & Bas van der Klaauw, 2022. "Dynamic evaluation of job search assistance," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 227-241, March.
    27. Rafael Lalive & Jan C. van Ours & Josef Zweimüller, 2005. "The Effect Of Benefit Sanctions On The Duration Of Unemployment," Journal of the European Economic Association, MIT Press, vol. 3(6), pages 1386-1417, December.
    28. Bas Van der Klaauw & Jan C. Van Ours, 2013. "Carrot And Stick: How Re‐Employment Bonuses And Benefit Sanctions Affect Exit Rates From Welfare," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 275-296, March.
    29. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    30. Jaap H. Abbring & Gerard J. van den Berg, 2003. "The Nonparametric Identification of Treatment Effects in Duration Models," Econometrica, Econometric Society, vol. 71(5), pages 1491-1517, September.
    31. repec:hal:pseose:halshs-01067926 is not listed on IDEAS
    32. Rafael Lalive & Jan C. Van Ours & Josef Zweimüller, 2008. "The Impact of Active Labour Market Programmes on The Duration of Unemployment in Switzerland," Economic Journal, Royal Economic Society, vol. 118(525), pages 235-257, January.
    33. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    34. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    35. Peter Dolton & Donal O'Neill, 2002. "The Long-Run Effects of Unemployment Monitoring and Work-Search Programs: Experimental Evidence from the United Kingdom," Journal of Labor Economics, University of Chicago Press, vol. 20(2), pages 381-403, Part.
    36. J.J. Heckman & E.E. Leamer (ed.), 2007. "Handbook of Econometrics," Handbook of Econometrics, Elsevier, edition 1, volume 6, number 6b.
    37. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    38. Busso, Matias & DiNardo, John & McCrary, Justin, 2009. "New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators," IZA Discussion Papers 3998, Institute of Labor Economics (IZA).
    39. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    40. J.J. Heckman & E.E. Leamer (ed.), 2007. "Handbook of Econometrics," Handbook of Econometrics, Elsevier, edition 1, volume 6, number 6a.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. van den Berg, Gerard J. & Vikström, Johan, 2019. "Long-Run Effects of Dynamically Assigned Treatments: a New Methodology and an Evaluation of Training Effects on Earnings," Working Paper Series 2019:18, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    2. Paul Muller & Bas van der Klaauw & Arjan Heyma, 2020. "Comparing econometric methods to empirically evaluate activation programs for job seekers," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 526-547, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul Muller & Bas van der Klaauw & Arjan Heyma, 2020. "Comparing econometric methods to empirically evaluate activation programs for job seekers," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 526-547, August.
    2. Stephen Kastoryano & Bas van der Klaauw, 2022. "Dynamic evaluation of job search assistance," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 227-241, March.
    3. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    4. Lukas Fervers, 2016. "Fast track to the labour market or highway to hell? The effect of activation policies on quantity and quality of labour market integration," IAW Discussion Papers 125, Institut für Angewandte Wirtschaftsforschung (IAW).
    5. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    6. Michael C. Knaus & Michael Lechner & Anthony Strittmatter, 2022. "Heterogeneous Employment Effects of Job Search Programs: A Machine Learning Approach," Journal of Human Resources, University of Wisconsin Press, vol. 57(2), pages 597-636.
    7. McGuinness, Seamus & O'Connell, Philip J. & Kelly, Elish & Walsh, John R., 2011. "Activation in Ireland: An Evaluation of the National Employment Action Plan," Research Series, Economic and Social Research Institute (ESRI), number RS20.
    8. Aderonke Osikominu, 2013. "Quick Job Entry or Long-Term Human Capital Development? The Dynamic Effects of Alternative Training Schemes," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(1), pages 313-342.
    9. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    10. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    11. Mergele, Lukas & Weber, Michael, 2020. "Public employment services under decentralization: Evidence from a natural experiment," Journal of Public Economics, Elsevier, vol. 182(C).
    12. Bruno Crépon & Muriel Dejemeppe & Marc Gurgand, 2005. "Counseling the unemployed: does it lower unemployment duration and recurrence?," Working Papers halshs-00590769, HAL.
    13. Rainer Eppel & Helmut Mahringer, 2019. "Getting a lot out of a little bit of work? The effects of marginal employment during unemployment," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 46(2), pages 381-408, May.
    14. Tito Boeri & Jan van Ours, 2013. "The Economics of Imperfect Labor Markets: Second Edition," Economics Books, Princeton University Press, edition 1, number 10142.
    15. Alessio Brown & Johannes Koettl, 2015. "Active labor market programs - employment gain or fiscal drain?," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-36, December.
    16. Hagen, Tobias, 2016. "Econometric evaluation of a placement coaching program for recipients of disability insurance benefits in Switzerland," Working Paper Series 10, Frankfurt University of Applied Sciences, Faculty of Business and Law.
    17. Bernhard Boockmann & Tobias Brändle, 2019. "Coaching, Counseling, Case‐Working: Do They Help the Older Unemployed Out of Benefit Receipt and Back Into the Labor Market?," German Economic Review, Verein für Socialpolitik, vol. 20(4), pages 436-468, November.
    18. van den Berg, Gerard J. & Bonev, Petyo & Mammen, Enno, 2016. "Nonparametric Instrumental Variable Methods for Dynamic Treatment Evaluation," IZA Discussion Papers 9782, Institute of Labor Economics (IZA).
    19. van der Klaauw, Bas & Ziegler, Lennart, 2019. "A Field Experiment on Labor Market Speeddates for Unemployed Workers," IZA Discussion Papers 12140, Institute of Labor Economics (IZA).
    20. Gerard J. van den Berg & Petyo Bonev & Enno Mammen, 2020. "Nonparametric Instrumental Variable Methods for Dynamic Treatment Evaluation," The Review of Economics and Statistics, MIT Press, vol. 102(2), pages 355-367, May.

    More about this item

    Keywords

    empirical policy evaluation; job-search assistance; unemployment duration;
    All these keywords.

    JEL classification:

    • J64 - Labor and Demographic Economics - - Mobility, Unemployment, Vacancies, and Immigrant Workers - - - Unemployment: Models, Duration, Incidence, and Job Search
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tin:wpaper:20170019. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tinbergen Office +31 (0)10-4088900 (email available below). General contact details of provider: https://edirc.repec.org/data/tinbenl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.