IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2004.03865.html
   My bibliography  Save this paper

Manipulation-Proof Machine Learning

Author

Listed:
  • Daniel Bjorkegren
  • Joshua E. Blumenstock
  • Samsun Knight

Abstract

An increasing number of decisions are guided by machine learning algorithms. In many settings, from consumer credit to criminal justice, those decisions are made by applying an estimator to data on an individual's observed behavior. But when consequential decisions are encoded in rules, individuals may strategically alter their behavior to achieve desired outcomes. This paper develops a new class of estimator that is stable under manipulation, even when the decision rule is fully transparent. We explicitly model the costs of manipulating different behaviors, and identify decision rules that are stable in equilibrium. Through a large field experiment in Kenya, we show that decision rules estimated with our strategy-robust method outperform those based on standard supervised learning approaches.

Suggested Citation

  • Daniel Bjorkegren & Joshua E. Blumenstock & Samsun Knight, 2020. "Manipulation-Proof Machine Learning," Papers 2004.03865, arXiv.org.
  • Handle: RePEc:arx:papers:2004.03865
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2004.03865
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Stefano DellaVigna & Devin Pope, 2018. "Predicting Experimental Results: Who Knows What?," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2410-2456.
    2. Thomas S. Dee & Will Dobbie & Brian A. Jacob & Jonah Rockoff, 2019. "The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 382-423, July.
    3. Rema Hanna & Benjamin A. Olken, 2018. "Universal Basic Incomes versus Targeted Transfers: Anti-Poverty Programs in Developing Countries," Journal of Economic Perspectives, American Economic Association, vol. 32(4), pages 201-226, Fall.
    4. Michael Greenstone & Guojun He & Ruixue Jia & Tong Liu, 2022. "Can Technology Solve the Principal-Agent Problem? Evidence from China's War on Air Pollution," American Economic Review: Insights, American Economic Association, vol. 4(1), pages 54-70, March.
    5. J. A. Mirrlees, 1971. "An Exploration in the Theory of Optimum Income Taxation," Review of Economic Studies, Oxford University Press, vol. 38(2), pages 175-208.
    6. Yanhao Wei & Pinar Yildirim & Christophe Van den Bulte & Chrysanthos Dellarocas, 2016. "Credit Scoring with Social Network Data," Marketing Science, INFORMS, vol. 35(2), pages 234-258, March.
    7. Joshua E. Blumenstock, 2018. "Estimating Economic Characteristics with Phone Data," AEA Papers and Proceedings, American Economic Association, vol. 108, pages 72-76, May.
    8. Holmstrom, Bengt & Milgrom, Paul, 1987. "Aggregation and Linearity in the Provision of Intertemporal Incentives," Econometrica, Econometric Society, vol. 55(2), pages 303-328, March.
    9. Banerjee, Abhijit & Hanna, Rema & Olken, Benjamin A. & Sumarto, Sudarno, 2018. "The (Lack of) Distortionary Effects of Proxy-Means Tests: Results from a Nationwide Experiment in Indonesia," Working Paper Series rwp18-041, Harvard University, John F. Kennedy School of Government.
    10. Nichols, Albert L & Zeckhauser, Richard J, 1982. "Targeting Transfers through Restrictions on Recipients," American Economic Review, American Economic Association, vol. 72(2), pages 372-377, May.
    11. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, Oxford University Press, vol. 133(1), pages 237-293.
    12. Gabriel Carroll, 2015. "Robustness and Linear Contracts," American Economic Review, American Economic Association, vol. 105(2), pages 536-563, February.
    13. Eliaz, Kfir & Spiegler, Ran, 2022. "On incentive-compatible estimators," Games and Economic Behavior, Elsevier, vol. 132(C), pages 204-220.
    14. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
    15. Gonzalez-Lira, Andres & Mobarak, Ahmed Mushfiq, 2019. "Slippery Fish: Enforcing Regulation under Subversive Adaptation," IZA Discussion Papers 12179, Institute of Labor Economics (IZA).
    16. Vivi Alatas & Abhijit Banerjee & Rema Hanna & Benjamin A. Olken & Ririn Purnamasari & Matthew Wai-Poi, 2016. "Self-Targeting: Evidence from a Field Experiment in Indonesia," Journal of Political Economy, University of Chicago Press, vol. 124(2), pages 371-427.
    17. Paul Niehaus & Antonia Atanassova & Marianne Bertrand & Sendhil Mullainathan, 2013. "Targeting with Agents," American Economic Journal: Economic Policy, American Economic Association, vol. 5(1), pages 206-238, February.
    18. Michael Spence, 1973. "Job Market Signaling," The Quarterly Journal of Economics, Oxford University Press, vol. 87(3), pages 355-374.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Delprato, Marcos & Frola, Alessia & Antequera, Germán, 2022. "Indigenous and non-Indigenous proficiency gaps for out-of-school and in-school populations: A machine learning approach," International Journal of Educational Development, Elsevier, vol. 93(C).
    2. Roshni Sahoo & Stefan Wager, 2022. "Policy Learning with Competing Agents," Papers 2204.01884, arXiv.org, revised Dec 2022.
    3. Aiken, Emily & Bellue, Suzanne & Blumenstock, Joshua & Karlan, Dean S. & Udry, Christopher, 2021. "Machine Learning and Mobile Phone Data Can Improve the Targeting of Humanitarian Assistance," CEPR Discussion Papers 16385, C.E.P.R. Discussion Papers.
    4. Elliott Ash & Sergio Galletta & Tommaso Giommoni, 2021. "A Machine Learning Approach to Analyze and Support Anti-Corruption Policy," CESifo Working Paper Series 9015, CESifo.
    5. Meena Jagadeesan & Celestine Mendler-Dunner & Moritz Hardt, 2021. "Alternative Microfoundations for Strategic Classification," Papers 2106.12705, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Emily Aiken & Suzanne Bellue & Dean Karlan & Christopher R. Udry & Joshua Blumenstock, 2021. "Machine Learning and Mobile Phone Data Can Improve the Targeting of Humanitarian Assistance," NBER Working Papers 29070, National Bureau of Economic Research, Inc.
    2. Rema Hanna & Benjamin A. Olken, 2018. "Universal Basic Incomes vs. Targeted Transfers: Anti-Poverty Programs in Developing Countries," NBER Working Papers 24939, National Bureau of Economic Research, Inc.
    3. Jia Barwick, Panle & Liu, Yanyan & Patacchini, Eleonora & Wu, Qi, 2019. "Information, Mobile Communication, and Referral Effects," CEPR Discussion Papers 13786, C.E.P.R. Discussion Papers.
    4. Robin Boadway, 2011. "Viewpoint: Innovations in the theory and practice of redistribution policy," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 44(4), pages 1138-1183, November.
    5. Hilary W. Hoynes & Jesse Rothstein, 2019. "Universal Basic Income in the US and Advanced Countries," NBER Working Papers 25538, National Bureau of Economic Research, Inc.
    6. Fredrik Andersson, 1994. "On the screening power of incentive schemes," Review of Economic Design, Springer;Society for Economic Design, vol. 1(1), pages 251-274, December.
    7. Bottan, Nicolas & Hoffmann, Bridget & Vera-Cossio, Diego A., 2021. "Stepping up during a crisis: The unintended effects of a noncontributory pension program during the Covid-19 pandemic," Journal of Development Economics, Elsevier, vol. 150(C).
    8. Meena Jagadeesan & Celestine Mendler-Dunner & Moritz Hardt, 2021. "Alternative Microfoundations for Strategic Classification," Papers 2106.12705, arXiv.org.
    9. Haseeb, Muhammad & Vyborny, Kate, 2022. "Data, discretion and institutional capacity: Evidence from cash transfers in Pakistan," Journal of Public Economics, Elsevier, vol. 206(C).
    10. Aubrey Clark & Giovanni Reggiani, 2021. "Contracts for acquiring information," Papers 2103.03911, arXiv.org.
    11. Oyer, Paul & Schaefer, Scott, 2011. "Personnel Economics: Hiring and Incentives," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 20, pages 1769-1823, Elsevier.
    12. Dosis, Anastasios, 2016. "Bertand Competition and the Existence of Pure Strategy Nash Equilibrium in Markets with Adverse Selection," ESSEC Working Papers WP1606, ESSEC Research Center, ESSEC Business School.
    13. Cairo, Sofie & Mahlstedt, Robert, 2021. "Transparency of the Welfare System and Labor Market Outcomes of Unemployed Workers," IZA Discussion Papers 14940, Institute of Labor Economics (IZA).
    14. Ni, Jian & Zhao, Jun & Chu, Lap Keung, 2021. "Supply contracting and process innovation in a dynamic supply chain with information asymmetry," European Journal of Operational Research, Elsevier, vol. 288(2), pages 552-562.
    15. Zeckhauser, Richard, 2021. "Strategic sorting: the role of ordeals in health care," Economics and Philosophy, Cambridge University Press, vol. 37(1), pages 64-81, March.
    16. Ghazala Azmat & Marc Möller, 2018. "The Distribution of Talent Across Contests," Economic Journal, Royal Economic Society, vol. 128(609), pages 471-509, March.
    17. Daníelsson, Jón & Macrae, Robert & Uthemann, Andreas, 2022. "Artificial intelligence and systemic risk," Journal of Banking & Finance, Elsevier, vol. 140(C).
    18. Tomer Blumkin & Efraim Sadka & Yoram Margalioth, 2008. "The Role of Stigma in the Design of Welfare Programs," Working Papers 0806, Ben-Gurion University of the Negev, Department of Economics.
    19. Iossa, Elisabetta & Martimort, David, 2015. "Pessimistic information gathering," Games and Economic Behavior, Elsevier, vol. 91(C), pages 75-96.
    20. Robert J. Gary-Bobo & Alain Trannoy, 2015. "Optimal student loans and graduate tax under moral hazard and adverse selection," RAND Journal of Economics, RAND Corporation, vol. 46(3), pages 546-576, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2004.03865. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.