IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2504.06215.html

Randomization Inference in Two-Sided Market Experiments

Author

Listed:
  • Jizhou Liu
  • Azeem M. Shaikh
  • Panos Toulis

Abstract

Randomized experiments are increasingly employed in two-sided markets, such as buyer--seller platforms, to evaluate the effects of marketplace interventions. These experiments must reflect the underlying two-sided market structure in their design and can therefore be challenging to analyze. In this paper, we develop a randomization inference framework for outcomes from two-sided experiments, with a focus on testing and inference for two-sided spillover effects. Our approach is finite-sample valid under sharp null hypotheses. Regarding weak null hypotheses, we find that the commonly used Neyman-style studentization does not universally ensure asymptotic validity, and we document how it depends on the specific formulation of the null. We then propose a two-way variance estimator for studentization that restores asymptotic validity. We further propose methods to improve testing power by exploiting the two-sided structure of the problem, which we validate empirically. We demonstrate our methods through a series of simulation studies and an applied example from a network experiment in micro-lending.

Suggested Citation

  • Jizhou Liu & Azeem M. Shaikh & Panos Toulis, 2025. "Randomization Inference in Two-Sided Market Experiments," Papers 2504.06215, arXiv.org, revised Mar 2026.
  • Handle: RePEc:arx:papers:2504.06215
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2504.06215
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Susan Athey & Michael Luca, 2019. "Economists (and Economics) in Tech Companies," Journal of Economic Perspectives, American Economic Association, vol. 33(1), pages 209-230, Winter.
    2. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    3. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    4. Ramesh Johari & Hannah Li & Inessa Liskovich & Gabriel Y. Weintraub, 2022. "Experimental Design in Two-Sided Platforms: An Analysis of Bias," Management Science, INFORMS, vol. 68(10), pages 7069-7089, October.
    5. Guillaume Basse & Peng Ding & Avi Feller & Panos Toulis, 2024. "Randomization Tests for Peer Effects in Group Formation Experiments," Econometrica, Econometric Society, vol. 92(2), pages 567-590, March.
    6. G W Basse & A Feller & P Toulis, 2019. "Randomization tests of causal effects under interference," Biometrika, Biometrika Trust, vol. 106(2), pages 487-494.
    7. Susan Athey & Dean Eckles & Guido W. Imbens, 2018. "Exact p-Values for Network Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 230-240, January.
    8. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    9. Stefan Wager & Kuang Xu, 2021. "Experimenting in Equilibrium," Management Science, INFORMS, vol. 67(11), pages 6694-6715, November.
    10. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, November.
    11. Cyrus J. DiCiccio & Joseph P. Romano, 2017. "Robust Permutation Tests For Correlation And Regression Coefficients," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1211-1220, July.
    12. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    13. Jason Wu & Peng Ding, 2021. "Randomization Tests for Weak Null Hypotheses in Randomized Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1898-1913, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jizhou Liu & Liang Zhong, 2026. "Randomization Tests in Switchback Experiments," Papers 2602.23257, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jizhou Liu & Liang Zhong, 2026. "Randomization Tests in Switchback Experiments," Papers 2602.23257, arXiv.org.
    2. David M. Ritzwoller & Joseph P. Romano & Azeem M. Shaikh, 2024. "Randomization Inference: Theory and Applications," Papers 2406.09521, arXiv.org, revised Feb 2025.
    3. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org, revised Apr 2025.
    4. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    5. Han, Kevin & Basse, Guillaume & Bojinov, Iavor, 2024. "Population interference in panel experiments," Journal of Econometrics, Elsevier, vol. 238(1).
    6. Iavor Bojinov & David Simchi-Levi & Jinglong Zhao, 2023. "Design and Analysis of Switchback Experiments," Management Science, INFORMS, vol. 69(7), pages 3759-3777, July.
    7. Sadegh Shirani & Mohsen Bayati, 2025. "On Evolution-Based Models for Experimentation Under Interference," Papers 2511.21675, arXiv.org.
    8. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org, revised May 2025.
    9. Chonghuan Wang, 2026. "Experimental Design for Matching," Papers 2601.21036, arXiv.org.
    10. Stefan Wager & Kuang Xu, 2021. "Experimenting in Equilibrium," Management Science, INFORMS, vol. 67(11), pages 6694-6715, November.
    11. Tadao Hoshino & Takahide Yanagi, 2024. "Causal Inference with Noncompliance and Unknown Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 119(548), pages 2869-2880, October.
    12. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    13. Stefan Wager & Kuang Xu, 2019. "Experimenting in Equilibrium," Papers 1903.02124, arXiv.org, revised Jun 2020.
    14. Luofeng Liao & Christian Kroer, 2024. "Statistical Inference and A/B Testing in Fisher Markets and Paced Auctions," Papers 2406.15522, arXiv.org, revised Mar 2025.
    15. Wang, Yuhao & Li, Xinran, 2025. "Asymptotic theory of the best-choice rerandomization using the Mahalanobis distance," Journal of Econometrics, Elsevier, vol. 251(C).
    16. Julius Owusu, 2023. "Randomization Inference of Heterogeneous Treatment Effects under Network Interference," Papers 2308.00202, arXiv.org, revised Jun 2025.
    17. Zhexiao Lin & Peng Ding, 2025. "Unifying regression-based and design-based causal inference in time-series experiments," Papers 2510.22864, arXiv.org.
    18. Shuze Chen & David Simchi-Levi & Chonghuan Wang, 2024. "Improving the Estimation of Lifetime Effects in A/B Testing via Treatment Locality," Papers 2407.19618, arXiv.org, revised Sep 2025.
    19. Xiaokang Luo & Tirthankar Dasgupta & Minge Xie & Regina Y. Liu, 2021. "Leveraging the Fisher randomization test using confidence distributions: Inference, combination and fusion learning," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(4), pages 777-797, September.
    20. Zhao, Anqi & Ding, Peng, 2024. "No star is good news: A unified look at rerandomization based on p-values from covariate balance tests," Journal of Econometrics, Elsevier, vol. 241(1).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2504.06215. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.