Causal inference in conjoint analysis: Logit models vs. potential outcomes

Estimating average marginal effects (AME) from logit models is a common approach for assessing attribute importance in conjoint analysis, but it rests on strong parametric and behavioral assumptions. Recent work has formalized conjoint analysis within Rubin’s potential outcomes framework and introduced the Average Marginal Component Effect (AMCE) as a nonparametric causal estimand, but AMCE relies on additional identification assumptions. The Conditional Randomization Test (CRT) does not require these assumptions for testing whether an attribute has any causal effect and can also be used to diagnose violations of key AMCE assumptions. This research note presents the first comprehensive empirical comparison of AME, AMCE, and CRT using four public datasets that vary in task complexity and attribute count, with the aim of providing practical guidance for applied researchers across conjoint designs of differing complexity. We find that designs with many attributes and many tasks are more prone to violating AMCE assumptions, whereas simpler designs tend to be more robust, making AMCE a preferred causal estimand in such settings. When AMCE assumptions fail, AME estimates, when consistent with CRT results, provide a more credible basis for inference than AMCE. When AME is not statistically significant but CRT suggests an effect, interaction terms can be explored within the logit model to capture preference heterogeneity and reconcile the discrepancy. CRT can therefore inform both assumption checking and parametric specification, although it should be used with caution because Lasso-based CRT implementations can occasionally yield counterintuitive outcomes in applied settings.