IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_12573.html

A Brave New World of Hiring: A Natural Field Experiment on How Asynchronous Interviews and AI Assessment Reshape Recruitment

Author

Listed:
  • Mallory Avery
  • Edwin Ip
  • Andreas Leibbrandt
  • Joseph Vecci

Abstract

Recent technological advancements are reshaping pathways to employment by automating the interview process. Asynchronous interviews, in which job applicants submit answers to interview questions via an online platform without interacting with an interviewer, are replacing more traditional face-to-face job interviews. At the same time, AI algorithms are now widely used to assess these interview answers. In this paper, we use a field experiment to comprehensively study how these new technologies affect applicants and employers in the recruitment process. Over 3,000 job applicants are randomized into asynchronous audio or video interviews, live online interviews, and a control group. Their job interviews are then assessed by both professional recruiters and a commercial AI recruitment tool used by most Fortune 100 companies. We find that asynchronous interviews cause an over 50% decrease in application continuation, including among the most qualified applicants, and that this decline is largest for women. A complementary vignette experiment provides evidence that this deterrence is driven by perceptions about the competitiveness and fairness of the recruitment process. In terms of assessments, we find that the AI evaluation tool scores women and underrepresented racial minorities higher than human evaluators, while the opposite is true for men, Whites and Asians. We track our applicants' subsequent labor market outcomes and find that the AI assessment tool predicts subsequent employment success substantially better than human recruiters, suggesting that AI captures soft skills and potential that humans overlook. In addition, we provide evidence that, unlike AI, human recruiters' assessments suffer from multiple cognitive biases. Our findings provide some of the first key evidence on how recent technological advances are transforming the hiring process.

Suggested Citation

  • Mallory Avery & Edwin Ip & Andreas Leibbrandt & Joseph Vecci, 2026. "A Brave New World of Hiring: A Natural Field Experiment on How Asynchronous Interviews and AI Assessment Reshape Recruitment," CESifo Working Paper Series 12573, CESifo.
    • Handle: RePEc:ces:ceswps:_12573
    as

    Download full text from publisher

    File URL: https://www.ifo.de/DocDL/cesifo1_wp12573.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Marie-Pierre Dargnies & Rustamdjan Hakimov & Dorothea Kübler, 2026. "Aversion to Hiring Algorithms: Transparency, Gender Profiling, and Self-Confidence," Management Science, INFORMS, vol. 72(1), pages 285-301, January.
    2. Jan Feld & Edwin Ip & Andreas Leibbrandt & Joseph Vecci, 2022. "Identifying and Overcoming Gender Barriers in Tech: A Field Experiment on Inaccurate Statistical Discrimination," Discussion Papers 2205, University of Exeter, Department of Economics.
    3. David H. Autor, 2001. "Wiring the Labor Market," Journal of Economic Perspectives, American Economic Association, vol. 15(1), pages 25-40, Winter.
    4. Amer, Abdelrahman & Craig, Ashley C & Van Effenterre, Clémentine, 2024. "Decoding Gender Bias: The Role of Personal Interaction," IZA Discussion Papers 17077, IZA Network @ LISER.
    5. Denzer, Manuel & Schank, Thorsten & Upward, Richard, 2021. "Does the internet increase the job finding rate? Evidence from a period of expansion in internet use," Information Economics and Policy, Elsevier, vol. 55(C).
    6. Peter Kuhn & Hani Mansour, 2014. "Is Internet Job Search Still Ineffective?," Economic Journal, Royal Economic Society, vol. 124(581), pages 1213-1233, December.
    7. Kausel, Edgar E. & Culbertson, Satoris S. & Madrid, Hector P., 2016. "Overconfidence in personnel selection: When and why unstructured interview information can hurt hiring decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 137(C), pages 27-44.
    8. Bo Cowgill & Fabrizio Dell'Acqua & Samuel Deng & Daniel Hsu & Nakul Verma & Augustin Chaintreau, 2020. "Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics," Papers 2012.02394, arXiv.org.
    9. Furnham, Adrian & Boo, Hua Chu, 2011. "A literature review of the anchoring effect," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 40(1), pages 35-42, February.
    10. Markus M. Möbius & Muriel Niederle & Paul Niehaus & Tanya S. Rosenblat, 2022. "Managing Self-Confidence: Theory and Experimental Evidence," Management Science, INFORMS, vol. 68(11), pages 7793-7817, November.
    11. repec:ces:ceswps:_11268 is not listed on IDEAS
    12. Joseph P. Simmons & Leif D. Nelson & Jeff Galak & Shane Frederick, 2011. "Intuitive Biases in Choice versus Estimation: Implications for the Wisdom of Crowds," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 38(1), pages 1-15.
    13. Jan Feld & Edwin Ip & Andreas Leibbrandt & Joseph Vecci, 2023. "Do Financial Incentives Encourage Women to Apply for a Tech Job? Evidence from a Natural Field Experiment," AEA Papers and Proceedings, American Economic Association, vol. 113, pages 432-435, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mallory Avery & Andreas Leibbrandt & Joseph Vecci, 2023. "Does Artificial Intelligence Help or Hurt Gender Diversity? Evidence from Two Field Experiments on Recruitment in Tech," Monash Economics Working Papers 2023-09, Monash University, Department of Economics.
    2. Choi, Eleanor Jawon, 2023. "Does the internet help the unemployed find jobs?," Information Economics and Policy, Elsevier, vol. 62(C).
    3. Hang Thu Nguyen-Phung & Miki Kohara & Secil Er, 2024. "The impact of ICT development on female employment and household’s well-being in Vietnam," The Japanese Economic Review, Springer, vol. 75(4), pages 951-978, December.
    4. Gürtzgen, Nicole & (né Nolte), André Diegmann & Pohlan, Laura & van den Berg, Gerard J., 2021. "Do digital information technologies help unemployed job seekers find a job? Evidence from the broadband internet expansion in Germany," European Economic Review, Elsevier, vol. 132(C).
    5. Czernich, Nina, 2014. "Does broadband internet reduce the unemployment rate? Evidence for Germany," Information Economics and Policy, Elsevier, vol. 29(C), pages 32-45.
    6. Gomez-Herrera, Estrella & Mueller-Langer, Frank, 2024. "Does information disclosure affect the gender gap in bidding behavior? Empirical evidence from a natural experiment on a large online labor platform," Labour Economics, Elsevier, vol. 90(C).
    7. Qing Wang & Wenjing Xu & Yanghua Huang & Jidong Yang, 2022. "The Effect of Fast Internet on Employment: Evidence from a Large Broadband Expansion Program in China," China & World Economy, Institute of World Economics and Politics, Chinese Academy of Social Sciences, vol. 30(3), pages 100-134, May.
    8. Bastian Stockinger, 2019. "Broadband internet availability and establishments’ employment growth in Germany: evidence from instrumental variables estimations," Journal for Labour Market Research, Springer;Institute for Employment Research/ Institut für Arbeitsmarkt- und Berufsforschung (IAB), vol. 53(1), pages 1-23, December.
    9. Jain, Ritika & Chatterjee, Tirtha, 2024. "Information and communication technology and female employment in India," Information Economics and Policy, Elsevier, vol. 69(C).
    10. Ana Dammert & Jose Galdo & Virgilio Galdo, 2015. "Integrating mobile phone technologies into labor-market intermediation: a multi-treatment experimental design," IZA Journal of Labor & Development, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-27, December.
    11. Năstasă Anamaria & Mocanu Cristina, 2025. "Digital divides in online job search: Insights from Europe," Management & Marketing, Sciendo, vol. 20(4), pages 32-40.
    12. Denzer, Manuel & Schank, Thorsten & Upward, Richard, 2021. "Does the internet increase the job finding rate? Evidence from a period of expansion in internet use," Information Economics and Policy, Elsevier, vol. 55(C).
    13. Bhuller, Manudeep & Kostøl, Andreas & Vigtel, Trond Christian, 2019. "How Broadband Internet Affects Labor Market Matching," Memorandum 10/2019, Oslo University, Department of Economics.
    14. Wang, Jiancheng & Chen, Jialing & Li, Xiaoying & Li, Weiping, 2024. "Broadband acceleration and employment: Evidence from China," Technological Forecasting and Social Change, Elsevier, vol. 202(C).
    15. Lutgen, Vanessa & Van der Linden, Bruno, 2015. "Regional equilibrium unemployment theory at the age of the Internet," Regional Science and Urban Economics, Elsevier, vol. 53(C), pages 50-67.
    16. Matthieu Manant & Serge Pajak & Nicolas Soulié, 2019. "Can social media lead to labor market discrimination? Evidence from a field experiment," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 28(2), pages 225-246, April.
    17. Miguel A. Fonseca & Ashley McCrea, 2023. "The role of shortlisting in shifting gender beliefs on performance: experimental evidence," Discussion Papers 2315, University of Exeter, Department of Economics.
    18. Nuarpear Lekfuangfu & Voraprapa Nakavachara & Paphatsorn Sawaengsuksant, 2017. "Glancing at Labour Market Mismatch with User-generated Internet Data," PIER Discussion Papers 53, Puey Ungphakorn Institute for Economic Research.
    19. Sergey Roshchin & Sergey Solntsev & Dmitry Vasilyev, 2017. "Recruiting and Job Search Technologies in the Age of Internet," Foresight and STI Governance, National Research University Higher School of Economics, vol. 11(4), pages 33-43.
    20. Fonseca, Miguel A. & McCrea, Ashley, 2025. "The role of experience in shifting gender beliefs on performance: Experimental evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 237(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • J23 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Labor Demand
    • J71 - Labor and Demographic Economics - - Labor Discrimination - - - Hiring and Firing
    • J78 - Labor and Demographic Economics - - Labor Discrimination - - - Public Policy (including comparable worth)

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_12573. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.