Can we learn anything about pleading changes from existing data?

In light of the gateway role that the pleading standard can play in our civil litigation system, measuring the empirical effects of pleading policy changes embodied in the Supreme Court's controversial Twombly and Iqbal cases is important. In my earlier paper, Locking the Doors to Discovery, I argued that in doing so, special care is required in formulating the object of empirical study. Taking party behavior seriously, as Locking the Doors does, leads to empirical results suggesting that Twombly and Iqbal have had substantial effects among cases that face Rule 12(b)(6) motions post-Iqbal. This paper responds to potentially important critiques of my empirical implementation made by the FJC's Joe Cecil and Professor David Engstrom. An additional contribution of the present paper is to elucidate some important challenges for empirical work in civil procedure. First, researchers should carefully consider which covariates belong in statistical models, while also taking care in assessing the empirical importance of controlling for covariates. Second, data collection protocols should be designed with behavioral assumptions in mind. But third, researchers should not let the perfect be the enemy of the good: even data protocols that are less than perfectly designed may be broadly useful.