Research

Social scientists use survey experiments to study the effect of information on individual attitudes and behaviors. However, such experiments may fail to provide respondents with the information as intended. If the theorized mechanism is correct, noncompliance attenuates results, but noncompliance can also arise if the experiment exposes respondents to unintended information, affecting the substantive interpretation of results. In this letter, we propose a diagnostic test and recommendations for treatment design that will help researchers evaluate theoretical mechanisms in survey experiments. This placebo test repurposes treatment-relevant manipulation checks to evaluate responses under control conditions. This approach offers a path toward more robust and more informative survey experiments.

Difference-in-differences (DiD) designs for estimating causal effects have grown in popularity throughout political science. It is common for DiD studies report their main results using a "dynamic" or "event study" two-way fixed effects (TWFE) regression. This regression combines estimates of average treatment effects for multiple post-treatment time periods alongside placebo tests of the main identifying assumption: parallel trends. Despite their ubiquity, there is little clear and consistent guidance in the discipline for how researchers should estimate dynamic treatment effects. This paper develops a novel decomposition of the dynamic TWFE regression coefficients in terms of their component 2x2 difference-in-differences comparisons in the style of Goodman-Bacon (2021). We use this decomposition to illustrate how bias can result from the incorrect specification of baseline time periods, the inclusion of units and time periods where all observations are treated, and heterogeneity in the dynamic treatment effects across different treatment timing groups. Our results provide additional intuition for the source of bias due to effect heterogeneity—what Sun and Abraham (2021) term "contamination bias"—by directly characterizing the contaminated 2x2 comparisons. We then provide a common framework for connecting the many proposed “heterogeneity-robust” estimators in the literature, noting that they vary primarily in which 2x2 comparisons they choose to include. Through a replication of three studies published in prominent political science journals, we conclude by showing how attentiveness to baseline selection and specification can alter findings.

This research note investigates the relative explanatory power of pre- and post-Communist institutions for the variation in contemporary development of towns in Guangxi Province in South China. I use a regression discontinuity (RD) design to test whether historical institutional membership of Guangdong, a currently rich coastal province that was historically a center of maritime trade in China, has persisted since the administrative boundary changed between the provinces shortly after the Chinese Communist Party came to power. The study finds little evidence that pre-Communist institutional membership affects contemporary development and suggests that post-Communist institutional evolution at the local level may have contributed more to differences in contemporary development within Guangxi.