IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2111.07889.html
   My bibliography  Save this paper

An Outcome Test of Discrimination for Ranked Lists

Author

Listed:
  • Jonathan Roth
  • Guillaume Saint-Jacques
  • YinYin Yu

Abstract

This paper extends Becker (1957)'s outcome test of discrimination to settings where a (human or algorithmic) decision-maker produces a ranked list of candidates. Ranked lists are particularly relevant in the context of online platforms that produce search results or feeds, and also arise when human decisionmakers express ordinal preferences over a list of candidates. We show that non-discrimination implies a system of moment inequalities, which intuitively impose that one cannot permute the position of a lower-ranked candidate from one group with a higher-ranked candidate from a second group and systematically improve the objective. Moreover, we show that that these moment inequalities are the only testable implications of non-discrimination when the auditor observes only outcomes and group membership by rank. We show how to statistically test the implied inequalities, and validate our approach in an application using data from LinkedIn.

Suggested Citation

  • Jonathan Roth & Guillaume Saint-Jacques & YinYin Yu, 2021. "An Outcome Test of Discrimination for Ranked Lists," Papers 2111.07889, arXiv.org.
    • Handle: RePEc:arx:papers:2111.07889
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2111.07889
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kevin Lang & Ariella Kahn-Lang Spitzer, 2020. "Race Discrimination: An Economic Perspective," Journal of Economic Perspectives, American Economic Association, vol. 34(2), pages 68-89, Spring.
    2. Roth, Alvin E, 1984. "The Evolution of the Labor Market for Medical Interns and Residents: A Case Study in Game Theory," Journal of Political Economy, University of Chicago Press, vol. 92(6), pages 991-1016, December.
    3. Francesca Molinari, 2020. "Microeconometrics with Partial Identification," Papers 2004.11751, arXiv.org.
    4. Castillo, Marco & Petrie, Ragan, 2010. "Discrimination in the lab: Does information trump appearance?," Games and Economic Behavior, Elsevier, vol. 68(1), pages 50-59, January.
    5. Donald W. K. Andrews & Gustavo Soares, 2010. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Econometrica, Econometric Society, vol. 78(1), pages 119-157, January.
    6. Ashesh Rambachan & Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan, 2020. "An Economic Perspective on Algorithmic Fairness," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 91-95, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. YinYin Yu & Guillaume Saint-Jacques, 2022. "Choosing an algorithmic fairness metric for an online marketplace: Detecting and quantifying algorithmic bias on LinkedIn," Papers 2202.07300, arXiv.org, revised Aug 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shuowen Chen & Hiroaki Kaido, 2022. "Robust Tests of Model Incompleteness in the Presence of Nuisance Parameters," Papers 2208.11281, arXiv.org, revised Sep 2023.
    2. Phillip Heiler, 2022. "Heterogeneous Treatment Effect Bounds under Sample Selection with an Application to the Effects of Social Media on Political Polarization," Papers 2209.04329, arXiv.org, revised Jan 2024.
    3. Levon Barseghyan & Maura Coughlin & Francesca Molinari & Joshua C. Teitelbaum, 2021. "Heterogeneous Choice Sets and Preferences," Econometrica, Econometric Society, vol. 89(5), pages 2015-2048, September.
    4. Jorg Stoye, 2020. "A Simple, Short, but Never-Empty Confidence Interval for Partially Identified Parameters," Papers 2010.10484, arXiv.org, revised Dec 2020.
    5. Kim, Dongwoo, 2023. "Partially identifying competing risks models: An application to the war on cancer," Journal of Econometrics, Elsevier, vol. 234(2), pages 536-564.
    6. Pietro Tebaldi & Alexander Torgovitsky & Hanbin Yang, 2023. "Nonparametric Estimates of Demand in the California Health Insurance Exchange," Econometrica, Econometric Society, vol. 91(1), pages 107-146, January.
    7. Peter Coles & John Cawley & Phillip B. Levine & Muriel Niederle & Alvin E. Roth & John J. Siegfried, 2010. "The Job Market for New Economists: A Market Design Perspective," Journal of Economic Perspectives, American Economic Association, vol. 24(4), pages 187-206, Fall.
    8. Victor Chernozhukov & Denis Chetverikov & Kengo Kato, 2013. "Testing Many Moment Inequalities," CeMMAP working papers 65/13, Institute for Fiscal Studies.
    9. Demet Yilmazkuday & Hakan Yilmazkuday, 2017. "The role of direct flights in trade costs," Review of World Economics (Weltwirtschaftliches Archiv), Springer;Institut für Weltwirtschaft (Kiel Institute for the World Economy), vol. 153(2), pages 249-270, May.
    10. Yuichi Kitamura & Jörg Stoye, 2013. "Nonparametric analysis of random utility models: testing," CeMMAP working papers 36/13, Institute for Fiscal Studies.
    11. Menzel, Konrad, 2014. "Consistent estimation with many moment inequalities," Journal of Econometrics, Elsevier, vol. 182(2), pages 329-350.
    12. Paula Jaramillo & Çaǧatay Kayı & Flip Klijn, 2014. "On the exhaustiveness of truncation and dropping strategies in many-to-many matching markets," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 42(4), pages 793-811, April.
    13. Muriel Niederle & Alvin E. Roth, 2009. "The Effects of a Centralized Clearinghouse on Job Placement, Wages, and Hiring Practices," NBER Chapters, in: Studies of Labor Market Intermediation, pages 235-271, National Bureau of Economic Research, Inc.
    14. Kevin Lang & Ariella Kahn-Lang Spitzer, 2020. "Race Discrimination: An Economic Perspective," Journal of Economic Perspectives, American Economic Association, vol. 34(2), pages 68-89, Spring.
    15. Steven T Berry & Giovanni Compiani, 2023. "An Instrumental Variable Approach to Dynamic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(4), pages 1724-1758.
    16. Scott Duke Kominers & Alexander Teytelboym & Vincent P Crawford, 2017. "An invitation to market design," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 33(4), pages 541-571.
    17. Parag A. Pathak & Alex Rees-Jones & Tayfun Sönmez, 2020. "Immigration Lottery Design: Engineered and Coincidental Consequences of H-1B Reforms," NBER Working Papers 26767, National Bureau of Economic Research, Inc.
    18. Donald R. Davis & Jonathan I. Dingel & Joan Monras & Eduardo Morales, 2019. "How Segregated Is Urban Consumption?," Journal of Political Economy, University of Chicago Press, vol. 127(4), pages 1684-1738.
    19. Du, Qingyuan & Wei, Shang-Jin, 2013. "A theory of the competitive saving motive," Journal of International Economics, Elsevier, vol. 91(2), pages 275-289.
    20. Chen, Peter & Egesdal, Michael & Pycia, Marek & Yenmez, M. Bumin, 2016. "Median stable matchings in two-sided markets," Games and Economic Behavior, Elsevier, vol. 97(C), pages 64-69.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2111.07889. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.