IDEAS home Printed from https://ideas.repec.org/a/eee/irlaec/v68y2021ics0144818821000442.html
   My bibliography  Save this article

The efficacy of measuring judicial ideal points: The mis-analogy of IRTs

Author

Listed:
  • Lerner, Joshua Y.
  • McCubbins, Mathew D.
  • Renberg, Kristen M.

Abstract

IRT models are among the most commonly used latent trait models in all of political science, particularly in the estimation of ideal points of political actors in institutions. While widely used, IRT models are often misapplied, and a key element of their estimation, the item parameters, are almost always ignored and discarded. In this paper, we look into the application of IRT models to the estimation of judicial ideology scores by Martin and Quinn (2002). Building off of a replication and extension of Martin and Quinn (2002), we demonstrate that the often-ignored item parameters are, in fact, inconsistent with the assumptions of IRTs. Then, using a post-estimation fix that is designed to ameliorate the problem, we run the model again, generating new scores. We then compare our new ideal points to the existing ideal points and discuss the implications for both ideal point modeling generally and in judicial politics specifically. We conclude by replicating a prominent study in judicial politics that demonstrates how inconsistencies in the estimation of IRT models can be consequential and bring up concerns with the implications for what this could mean for the usefulness of scores estimated via IRT models.

Suggested Citation

  • Lerner, Joshua Y. & McCubbins, Mathew D. & Renberg, Kristen M., 2021. "The efficacy of measuring judicial ideal points: The mis-analogy of IRTs," International Review of Law and Economics, Elsevier, vol. 68(C).
    • Handle: RePEc:eee:irlaec:v:68:y:2021:i:c:s0144818821000442
    DOI: 10.1016/j.irle.2021.106020
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0144818821000442
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.irle.2021.106020?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Imai, Kosuke & Lo, James & Olmsted, Jonathan, 2016. "Fast Estimation of Ideal Points with Massive Data," American Political Science Review, Cambridge University Press, vol. 110(4), pages 631-656, November.
    2. Tausanovitch, Chris & Warshaw, Christopher, 2014. "Representation in Municipal Government," American Political Science Review, Cambridge University Press, vol. 108(3), pages 605-641, August.
    3. Bartels, Brandon L., 2009. "The Constraining Capacity of Legal Doctrine on the U.S. Supreme Court," American Political Science Review, Cambridge University Press, vol. 103(3), pages 474-495, August.
    4. Londregan, John, 1999. "Estimating Legislators' Preferred Points," Political Analysis, Cambridge University Press, vol. 8(1), pages 35-56, January.
    5. Matthew E.K. Hall, 2014. "The Semiconstrained Court: Public Opinion, the Separation of Powers, and the U.S. Supreme Court's Fear of Nonimplementation," American Journal of Political Science, John Wiley & Sons, vol. 58(2), pages 352-366, April.
    6. Jackman, Simon, 2001. "Multidimensional Analysis of Roll Call Data via Bayesian Simulation: Identification, Estimation, Inference, and Model Checking," Political Analysis, Cambridge University Press, vol. 9(3), pages 227-241, January.
    7. Chad Westerland & Jeffrey A. Segal & Lee Epstein & Charles M. Cameron & Scott Comparato, 2010. "Strategic Defiance and Compliance in the U.S. Courts of Appeals," American Journal of Political Science, John Wiley & Sons, vol. 54(4), pages 891-905, October.
    8. Martin, Andrew D. & Quinn, Kevin M., 2002. "Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999," Political Analysis, Cambridge University Press, vol. 10(2), pages 134-153, April.
    9. Adam Bonica, 2014. "Mapping the Ideological Marketplace," American Journal of Political Science, John Wiley & Sons, vol. 58(2), pages 367-386, April.
    10. Clinton, Joshua & Jackman, Simon & Rivers, Douglas, 2004. "The Statistical Analysis of Roll Call Data," American Political Science Review, Cambridge University Press, vol. 98(2), pages 355-370, May.
    11. Chad Westerland, 2007. "The Judicial Common Space 1," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 23(2), pages 303-325, June.
    12. Chris W. Bonneau & Thomas H. Hammond & Forrest Maltzman & Paul J. Wahlbeck, 2007. "Agenda Control, the Median Justice, and the Majority Opinion on the U.S. Supreme Court," American Journal of Political Science, John Wiley & Sons, vol. 51(4), pages 890-905, October.
    13. Tom S. Clark & Benjamin Lauderdale, 2010. "Locating Supreme Court Opinions in Doctrine Space," American Journal of Political Science, John Wiley & Sons, vol. 54(4), pages 871-890, October.
    14. Shawn Treier & Simon Jackman, 2008. "Democracy as a Latent Variable," American Journal of Political Science, John Wiley & Sons, vol. 52(1), pages 201-217, January.
    15. Aldrich, John H. & Montgomery, Jacob M. & Sparks, David B., 2014. "Polarization and Ideology: Partisan Sources of Low Dimensionality in Scaled Roll Call Analyses," Political Analysis, Cambridge University Press, vol. 22(4), pages 435-456.
    16. Tahk, Alexander, 2018. "Nonparametric Ideal-Point Estimation and Inference," Political Analysis, Cambridge University Press, vol. 26(2), pages 131-146, April.
    17. Cliff Carrubba & Barry Friedman & Andrew D. Martin & Georg Vanberg, 2012. "Who Controls the Content of Supreme Court Opinions?," American Journal of Political Science, John Wiley & Sons, vol. 56(2), pages 400-412, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Spruk, Rok & Kovac, Mitja, 2019. "Replicating and extending Martin-Quinn scores," International Review of Law and Economics, Elsevier, vol. 60(C).
    2. Yonatan Lupu & James H. Fowler, 2013. "Strategic Citations to Precedent on the U.S. Supreme Court," The Journal of Legal Studies, University of Chicago Press, vol. 42(1), pages 151-186.
    3. Richard F. Potthoff, 2018. "Estimating Ideal Points from Roll-Call Data: Explore Principal Components Analysis, Especially for More Than One Dimension?," Social Sciences, MDPI, vol. 7(1), pages 1-27, January.
    4. Bonica, Adam & Chilton, Adam S. & Goldin, Jacob & Rozema, Kyle & Sen, Maya, 2016. "Measuring Judicial Ideology Using Law Clerk Hiring," Working Paper Series 16-031, Harvard University, John F. Kennedy School of Government.
    5. Cindy Cheng & Joan Barceló & Allison Spencer Hartnett & Robert Kubinec & Luca Messerschmidt, 2020. "COVID-19 Government Response Event Dataset (CoronaNet v.1.0)," Nature Human Behaviour, Nature, vol. 4(7), pages 756-768, July.
    6. Crane, Harry, 2017. "A hidden Markov model for latent temporal clustering with application to ideological alignment in the U.S. Supreme Court," Computational Statistics & Data Analysis, Elsevier, vol. 110(C), pages 19-36.
    7. Paola Annoni & Nicholas Charron, 2019. "Measurement Assessment in Cross-Country Comparative Analysis: Rasch Modelling on a Measure of Institutional Quality," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 31-60, January.
    8. F. Swen Kuh & Grace S. Chiu & Anton H. Westveld, 2019. "Modeling National Latent Socioeconomic Health and Examination of Policy Effects via Causal Inference," Papers 1911.00512, arXiv.org.
    9. Iaryczower, Matias & Lewis, Garrett & Shum, Matthew, 2013. "To elect or to appoint? Bias, information, and responsiveness of bureaucrats and politicians," Journal of Public Economics, Elsevier, vol. 97(C), pages 230-244.
    10. Christopher J Fariss & James Lo, 2020. "Innovations in concepts and measurement for the study of peace and conflict," Journal of Peace Research, Peace Research Institute Oslo, vol. 57(6), pages 669-678, November.
    11. Cindy Cheng & Joan Barcelo & Allison Spencer Hartnett & Robert Kubinec & Luca Messerschmidt, 2020. "CoronaNet: A Dyadic Dataset of Government Responses to the COVID-19 Pandemic," Working Papers 20200042, New York University Abu Dhabi, Department of Social Science, revised Apr 2020.
    12. James Lo, 2018. "Dynamic ideal point estimation for the European Parliament, 1980–2009," Public Choice, Springer, vol. 176(1), pages 229-246, July.
    13. Bonica, Adam & Sen, Maya, 2017. "The Politics of Selecting the Bench from the Bar: The Legal Profession and Partisan Incentives to Introduce Ideology into Judicial Selection," Working Paper Series rwp17-048, Harvard University, John F. Kennedy School of Government.
    14. Mario Quaranta, 2018. "The Meaning of Democracy to Citizens Across European Countries and the Factors Involved," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(3), pages 859-880, April.
    15. Swen Kuh & Grace S. Chiu & Anton H. Westveld, 2020. "Latent Causal Socioeconomic Health Index," Papers 2009.12217, arXiv.org, revised Oct 2023.
    16. Bjørn Høyland, 2010. "Procedural and party effects in European Parliament roll-call votes," European Union Politics, , vol. 11(4), pages 597-613, December.
    17. Christopher Hare & Keith T. Poole, 2015. "Measuring ideology in Congress," Chapters, in: Jac C. Heckelman & Nicholas R. Miller (ed.), Handbook of Social Choice and Voting, chapter 18, pages 327-346, Edward Elgar Publishing.
    18. René Lindstädt & Ryan Wielen, 2011. "Timely shirking: time-dependent monitoring and its effects on legislative behavior in the U.S. Senate," Public Choice, Springer, vol. 148(1), pages 119-148, July.
    19. Sara Hagemann, 2007. "Applying Ideal Point Estimation Methods to the Council of Ministers," European Union Politics, , vol. 8(2), pages 279-296, June.
    20. Christopher Hare & Tzu-Ping Liu & Robert N. Lupton, 2018. "What Ordered Optimal Classification reveals about ideological structure, cleavages, and polarization in the American mass public," Public Choice, Springer, vol. 176(1), pages 57-78, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:irlaec:v:68:y:2021:i:c:s0144818821000442. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/irle .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.