IDEAS home Printed from https://ideas.repec.org/p/diw/diwwpp/dp1368.html
   My bibliography  Save this paper

Data and Model Cross-Validation to Improve Accuracy of Microsimulation Results: Estimates for the Polish Household Budget Survey

Author

Listed:
  • Michal Myck
  • Mateusz Najsztub

Abstract

We conduct detailed analysis of the Polish Household Budget Survey data for the years 2006-2011 with the focus on its representativeness from the point of view of microsimulation analysis. We find important discrepancies between the data weighted with baseline grossing-up weights and official statistics from other sources. A number of re-weighting exercises is examined from the point of view of the accuracy of microsimulation results and we show that using a combination of demographic calibration targets with several economic status variables or tax identifiers from the microsimulation model substantially improves the correspondence of model results and administrative data. While demographic re-weighting is neutral from the point of view of income distribution, calibrating the grossing-up weights to adjust for economic status and tax identifiers significantly increases income inequality. We argue that although data reweighting can substantially improve the accuracy of microsimulation it should be used with caution.

Suggested Citation

  • Michal Myck & Mateusz Najsztub, 2014. "Data and Model Cross-Validation to Improve Accuracy of Microsimulation Results: Estimates for the Polish Household Budget Survey," Discussion Papers of DIW Berlin 1368, DIW Berlin, German Institute for Economic Research.
    • Handle: RePEc:diw:diwwpp:dp1368
    as

    Download full text from publisher

    File URL: https://www.diw.de/documents/publikationen/73/diw_01.c.441305.de/dp1368.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lixin Cai & John Creedy & Guyonne Kalb, 2006. "Accounting For Population Ageing In Tax Microsimulation Modelling By Survey Reweighting," Australian Economic Papers, Wiley Blackwell, vol. 45(1), pages 18-37, March.
    2. Peter Haan & Michał Myck, 2012. "Multi-family households in a labour supply model: a calibration method with application to Poland," Applied Economics, Taylor & Francis Journals, vol. 44(22), pages 2907-2919, August.
    3. Christopher Giles & Julian McCrae, 1995. "TAXBEN: the IFS microsimulation tax and benefit model," IFS Working Papers W95/19, Institute for Fiscal Studies.
    4. Morawski, Leszek & Myck, Michal, 2010. "'Klin'-ing up: Effects of Polish tax reforms on those in and on those out," Labour Economics, Elsevier, vol. 17(3), pages 556-566, June.
    5. Regina Riphahn & Oliver Serfling, 2005. "Item non-response on income and wealth questions," Empirical Economics, Springer, vol. 30(2), pages 521-538, September.
    6. Jörg-Peter Schräpler, 2002. "Respondent Behavior in Panel Studies: A Case Study for Income-Nonresponse by Means of the German Socio-Economic Panel (GSOEP)," Discussion Papers of DIW Berlin 299, DIW Berlin, German Institute for Economic Research.
    7. Michal Myck & Anna Kurowska & Michal Kundera, 2013. "Financial support for families with children and its trade-offs: balancing redistribution and parental work incentives," Baltic Journal of Economics, Baltic International Centre for Economic Policy Studies, vol. 13(2), pages 59-83, December.
    8. Jekaterina Navicke & Olga Rastrigina & Holly Sutherland, 2014. "Nowcasting Indicators of Poverty Risk in the European Union: A Microsimulation Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 119(1), pages 101-119, October.
    9. repec:diw:diwwpp:dp1315 is not listed on IDEAS
    10. Anders Klevmarken, 2022. "Statistical Inference in Micro-simulation Models: Incorporating External Information," International Journal of Microsimulation, International Microsimulation Association, vol. 15(1), pages 111-120.
    11. Bargain, Olivier B. & Morawski, Leszek & Myck, Michal & Socha, Mieczyslaw, 2007. "As SIMPL As That: Introducing a Tax-Benefit Microsimulation Model for Poland," IZA Discussion Papers 2988, Institute of Labor Economics (IZA).
    12. Cathal O'Donoghue & Jason Loughrey, 2014. "Nowcasting in Microsimulation Models: A Methodological Survey," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 17(4), pages 1-12.
    13. Anton Korinek & Johan Mistiaen & Martin Ravallion, 2006. "Survey nonresponse and the distribution of income," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 4(1), pages 33-55, April.
    14. Adrian Mander, 2007. "RADAR: Stata module to draw radar (spider) plots," Statistical Software Components S456829, Boston College Department of Economics, revised 06 Dec 2021.
    15. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
    16. Cameron,A. Colin & Trivedi,Pravin K., 2005. "Microeconometrics," Cambridge Books, Cambridge University Press, number 9780521848053.
    17. John Creedy & Ivan Tuckwell, 2004. "Reweighting Household Surveys for Tax Microsimulation Modelling: An Application to the New Zealand Household Economic Survey," Australian Journal of Labour Economics (AJLE), Bankwest Curtin Economics Centre (BCEC), Curtin Business School, vol. 7(1), pages 71-88, March.
    18. Daniele Pacifico, 2010. "REWEIGHT: The Stata command for survey reweighting," Center for the Analysis of Public Policies (CAPP) 0079, Universita di Modena e Reggio Emilia, Dipartimento di Economia "Marco Biagi".
    19. Michal Brzezinski, 2010. "Income Affluence in Poland," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 99(2), pages 285-299, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Piotr Arak & Piotr Lewandowski & Piotr Zakowiecki, 2014. "Dual labour market in Poland – proposals for overcoming the deadlock," IBS Policy Papers 1/2014, Instytut Badan Strukturalnych.
    2. Michał Brzeziński & Michał Myck & Mateusz Najsztub, 2019. "Reevaluating distributional consequences of the transition to market economy in Poland: new results from combined household survey and tax return data," Working Papers 2019-18, Faculty of Economic Sciences, University of Warsaw.
    3. Brzezinski, Michal & Najsztub, Mateusz, 2021. "The impact of "Family 500+" programme on household incomes, poverty and inequality," SocArXiv vkr6h, Center for Open Science.
    4. Barbara Liberda & Katarzyna Sałach & Marek Pęczkowski, 2023. "The Effects of Child Benefit on Household Saving," Journal of Family and Economic Issues, Springer, vol. 44(2), pages 447-460, June.
    5. Tobias Schoch & André Müller, 2020. "Treatment of sample under-representation and skewed heavy-tailed distributions in survey-based microsimulation: An analysis of redistribution effects in compulsory health care insurance in Switzerland," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 14(3), pages 267-304, December.
    6. Michał Myck & Kajetan Trzciński, 2019. "From Partial to Full Universality: The Family 500+ Programme in Poland and its Labor Supply Implications," ifo DICE Report, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 17(03), pages 36-44, October.
    7. Mehmet Burak Turgut & Tomasz Tratkiewicz, 2023. "Estimate of the Underground Economy in Poland Based on Household Expenditures and Incomes," Central European Journal of Economic Modelling and Econometrics, Central European Journal of Economic Modelling and Econometrics, vol. 15(1), pages 1-29, March.
    8. Jarosław Oczki, 2016. "Gender Pay Gap in Poland," International Economics, University of Lodz, Faculty of Economics and Sociology, issue 14, pages 106-113, June.
    9. Piotr Lewandowski & Katarzyna Salach, 2018. "Pomiar ubostwa energetycznego na podstawie danych BBGD - metodologia i zastosowanie," IBS Research Reports 01/2018, Instytut Badan Strukturalnych.
    10. Brzezinski, Michal & Sałach, Katarzyna, 2021. "Nierówności dochodowe i majątkowe w Polsce: nowe wyniki wykorzystujące dane pozaankietowe," SocArXiv s43yr, Center for Open Science.
    11. Stefan Bouzarovski & Aneta Kie³czewska & Piotr Lewandowski & Jakub Soko³owski, 2019. "Measuring energy poverty in Poland with the Multidimensional Energy Poverty Index," IBS Working Papers 07/2019, Instytut Badan Strukturalnych.
    12. Brzezinski, Michal & Myck, Michał & Najsztub, Mateusz, 2022. "Sharing the gains of transition: Evaluating changes in income inequality and redistribution in Poland using combined survey and tax return data," European Journal of Political Economy, Elsevier, vol. 73(C).
    13. Arkadiusz Florczak & Janusz Jabłonowski, 2016. "Consumption over the life cycle in Poland," NBP Working Papers 252, Narodowy Bank Polski.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tobias Schoch & André Müller, 2020. "Treatment of sample under-representation and skewed heavy-tailed distributions in survey-based microsimulation: An analysis of redistribution effects in compulsory health care insurance in Switzerland," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 14(3), pages 267-304, December.
    2. Brzezinski, Michal & Najsztub, Mateusz, 2021. "The impact of "Family 500+" programme on household incomes, poverty and inequality," SocArXiv vkr6h, Center for Open Science.
    3. Konopczak, Karolina & Skibicki, Jakub, 2012. "Mikrosymulacyjny model podatkowo-zasiłkowy Ministerstwa Finansów – dokumentacja," MF Working Papers 33, Ministry of Finance in Poland.
    4. Michal Myck & Anna Kurowska & Michal Kundera, 2013. "Financial support for families with children and its trade-offs: balancing redistribution and parental work incentives," Baltic Journal of Economics, Baltic International Centre for Economic Policy Studies, vol. 13(2), pages 59-83, December.
    5. repec:diw:diwwpp:dp1315 is not listed on IDEAS
    6. Cristina Barceló, 2008. "The impact of alternative imputation methods on the measurement of income and wealth: Evidence from the Spanish survey of household finances," Working Papers 0829, Banco de España.
    7. Tim Goedemé & Karel Van den Bosch & Lina Salanauskaite & Gerlinde Verbist, 2013. "Testing the Statistical Significance of Microsimulation Results: Often Easier than You Think. A Technical Note," ImPRovE Working Papers 13/10, Herman Deleeck Centre for Social Policy, University of Antwerp.
    8. O'Donoghue, Cathal & Sologon, Denisa Maria, 2023. "The Transformation of Public Policy Analysis in Times of Crisis – A Microsimulation-Nowcasting Method Using Big Data," IZA Discussion Papers 15937, Institute of Labor Economics (IZA).
    9. Leszek Morawski & Aneta Semeniuk, 2013. "Zakres ubóstwa a reformy podatkowo-świadczeniowe w latach 2006-2010," Gospodarka Narodowa. The Polish Journal of Economics, Warsaw School of Economics, issue 4, pages 21-40.
    10. Giancarlo MANZI & Pier Alda FERRARI, "undated". "Statistical methods for evaluating satisfaction with public services Abstract: Contrary to private enterprises, public enterprises can be unaware of the impact of their performance when providing serv," CIRIEC Working Papers 1404, CIRIEC - Université de Liège.
    11. Brunori, Paolo & Salas-Rojo, Pedro & Verme, Paolo, 2022. "Estimating Inequality with Missing Incomes," GLO Discussion Paper Series 1138, Global Labor Organization (GLO).
    12. Gary Cornwall & Marina Gindelsky, 2024. "Nowcasting Distributional National Accounts for the United States: A Machine Learning Approach," BEA Papers 0130, Bureau of Economic Analysis.
    13. Leszek Morawski, 2009. "Efekty wprowadzenia dwóch stóp w podatku dochodowym od osób fizycznych w 2009 roku," Gospodarka Narodowa. The Polish Journal of Economics, Warsaw School of Economics, issue 7-8, pages 37-58.
    14. Anna Kurowska & Michał Myck & Katharina Wrohlich, 2017. "Making work pay: increasing labour supply of secondary earners in low income families with children," Contemporary Economics, University of Economics and Human Sciences in Warsaw., vol. 11(2), June.
    15. Brzeziński, Michał & Myck, Michal & Najsztub, Mateusz, 2019. "Reevaluating Distributional Consequences of the Transition to Market Economy in Poland: New Results from Combined Household Survey and Tax Return Data," IZA Discussion Papers 12734, Institute of Labor Economics (IZA).
    16. Denisa M. Sologon & Cathal O’Donoghue & Iryna Kyzyma & Jinjing Li & Jules Linden & Raymond Wagener, 2022. "The COVID-19 resilience of a continental welfare regime - nowcasting the distributional impact of the crisis," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 20(4), pages 777-809, December.
    17. Joachim R. Frick & Markus M. Grabka, 2003. "Missing Income Data in the German SOEP: Incidence, Imputation and its Impact on the Income Distribution," Discussion Papers of DIW Berlin 376, DIW Berlin, German Institute for Economic Research.
    18. O'Donoghue, Cathal & M. Sologon, Denisa & Kyzyma, Iryna & McHale, John, 2020. "Modelling the distributional impact of the Covid-19 crisis in Ireland," Centre for Microsimulation and Policy Analysis Working Paper Series CEMPA4/20, Centre for Microsimulation and Policy Analysis at the Institute for Social and Economic Research.
    19. Cathal O'Donoghue & Denisa M. Sologon & Iryna Kyzyma & John McHale, 2020. "Modelling the Distributional Impact of the COVID‐19 Crisis," Fiscal Studies, John Wiley & Sons, vol. 41(2), pages 321-336, June.
    20. Navicke, Jekaterina & Kump, Nataša, 2014. "Re-weighting EUROMOD for demographic change: an application on Slovenian and Lithuanian data," EUROMOD Working Papers EM13/14, EUROMOD at the Institute for Social and Economic Research.
    21. Elisa Baroni & Matteo Richiardi, 2007. "Orcutt’s Vision, 50 years on," LABORatorio R. Revelli Working Papers Series 65, LABORatorio R. Revelli, Centre for Employment Studies.

    More about this item

    Keywords

    microsimulation; re-weighting; household data analysis;
    All these keywords.

    JEL classification:

    • D31 - Microeconomics - - Distribution - - - Personal Income and Wealth Distribution
    • I38 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Government Programs; Provision and Effects of Welfare Programs

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:diw:diwwpp:dp1368. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bibliothek (email available below). General contact details of provider: https://edirc.repec.org/data/diwbede.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.