IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2504.06215.html
   My bibliography  Save this paper

Randomization Inference in Two-Sided Market Experiments

Author

Listed:
  • Jizhou Liu
  • Azeem M. Shaikh
  • Panos Toulis

Abstract

Randomized experiments are increasingly employed in two-sided markets, such as buyer-seller platforms, to evaluate treatment effects from marketplace interventions. These experiments must reflect the underlying two-sided market structure in their design (e.g., sellers and buyers), making them particularly challenging to analyze. In this paper, we propose a randomization inference framework to analyze outcomes from such two-sided experiments. Our approach is finite-sample valid under sharp null hypotheses for any test statistic and maintains asymptotic validity under weak null hypotheses through studentization. Moreover, we provide heuristic guidance for choosing among multiple valid randomization tests to enhance statistical power, which we demonstrate empirically. Finally, we demonstrate the performance of our methodology through a series of simulation studies.

Suggested Citation

  • Jizhou Liu & Azeem M. Shaikh & Panos Toulis, 2025. "Randomization Inference in Two-Sided Market Experiments," Papers 2504.06215, arXiv.org.
    • Handle: RePEc:arx:papers:2504.06215
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2504.06215
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Susan Athey & Michael Luca, 2019. "Economists (and Economics) in Tech Companies," Journal of Economic Perspectives, American Economic Association, vol. 33(1), pages 209-230, Winter.
    2. Ramesh Johari & Hannah Li & Inessa Liskovich & Gabriel Y. Weintraub, 2022. "Experimental Design in Two-Sided Platforms: An Analysis of Bias," Management Science, INFORMS, vol. 68(10), pages 7069-7089, October.
    3. Guillaume Basse & Peng Ding & Avi Feller & Panos Toulis, 2024. "Randomization Tests for Peer Effects in Group Formation Experiments," Econometrica, Econometric Society, vol. 92(2), pages 567-590, March.
    4. Susan Athey & Dean Eckles & Guido W. Imbens, 2018. "Exact p-Values for Network Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 230-240, January.
    5. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    6. Stefan Wager & Kuang Xu, 2021. "Experimenting in Equilibrium," Management Science, INFORMS, vol. 67(11), pages 6694-6715, November.
    7. Cyrus J. DiCiccio & Joseph P. Romano, 2017. "Robust Permutation Tests For Correlation And Regression Coefficients," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1211-1220, July.
    8. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    9. Jason Wu & Peng Ding, 2021. "Randomization Tests for Weak Null Hypotheses in Randomized Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1898-1913, October.
    10. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    11. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    12. G W Basse & A Feller & P Toulis, 2019. "Randomization tests of causal effects under interference," Biometrika, Biometrika Trust, vol. 106(2), pages 487-494.
    13. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, Enero.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David M. Ritzwoller & Joseph P. Romano & Azeem M. Shaikh, 2024. "Randomization Inference: Theory and Applications," Papers 2406.09521, arXiv.org, revised Feb 2025.
    2. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org, revised Apr 2025.
    3. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    4. Han, Kevin & Basse, Guillaume & Bojinov, Iavor, 2024. "Population interference in panel experiments," Journal of Econometrics, Elsevier, vol. 238(1).
    5. Iavor Bojinov & David Simchi-Levi & Jinglong Zhao, 2023. "Design and Analysis of Switchback Experiments," Management Science, INFORMS, vol. 69(7), pages 3759-3777, July.
    6. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org.
    7. Stefan Wager & Kuang Xu, 2021. "Experimenting in Equilibrium," Management Science, INFORMS, vol. 67(11), pages 6694-6715, November.
    8. Tadao Hoshino & Takahide Yanagi, 2024. "Causal Inference with Noncompliance and Unknown Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 119(548), pages 2869-2880, October.
    9. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    10. Stefan Wager & Kuang Xu, 2019. "Experimenting in Equilibrium," Papers 1903.02124, arXiv.org, revised Jun 2020.
    11. Luofeng Liao & Christian Kroer, 2024. "Statistical Inference and A/B Testing in Fisher Markets and Paced Auctions," Papers 2406.15522, arXiv.org, revised Mar 2025.
    12. Julius Owusu, 2023. "Randomization Inference of Heterogeneous Treatment Effects under Network Interference," Papers 2308.00202, arXiv.org, revised Jan 2025.
    13. Shuze Chen & David Simchi-Levi & Chonghuan Wang, 2024. "Experimenting on Markov Decision Processes with Local Treatments," Papers 2407.19618, arXiv.org, revised Oct 2024.
    14. Xiaokang Luo & Tirthankar Dasgupta & Minge Xie & Regina Y. Liu, 2021. "Leveraging the Fisher randomization test using confidence distributions: Inference, combination and fusion learning," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(4), pages 777-797, September.
    15. Zhao, Anqi & Ding, Peng, 2024. "No star is good news: A unified look at rerandomization based on p-values from covariate balance tests," Journal of Econometrics, Elsevier, vol. 241(1).
    16. Nathan Kallus, 2023. "Treatment Effect Risk: Bounds and Inference," Management Science, INFORMS, vol. 69(8), pages 4579-4590, August.
    17. Rauf Ahmad & Per Johansson & Mårten Schultzberg, 2024. "Is Fisher inference inferior to Neyman inference for policy analysis?," Statistical Papers, Springer, vol. 65(6), pages 3425-3445, August.
    18. Ariel Boyarsky & Hongseok Namkoong & Jean Pouget-Abadie, 2023. "Modeling Interference Using Experiment Roll-out," Papers 2305.10728, arXiv.org, revised Aug 2023.
    19. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    20. Young, Alwyn, 2024. "Asymptotically robust permutation-based randomization confidence intervals for parametric OLS regression," LSE Research Online Documents on Economics 120933, London School of Economics and Political Science, LSE Library.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2504.06215. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.