research

Researchers use survey experiments to study the effect of informational beliefs on individual attitudes and behaviors. When the manipulation of interest is a belief about some fact, such studies must assume respondents receive the information treatment and, further, that their beliefs change in the intended manner. These assumptions, however, are often not addressed in survey experimental designs. We suggest that researchers collect and analyze post-treatment measures of both treatment compliance and belief change. With these measures, researchers can model the reception and causal effect of information and perform placebo tests of their proposed theoretical mechanisms. We demonstrate the utility of our framework by re-analyzing three prominent survey experiments in political science and with an original study on how changes in a factual belief can affect downstream judgments. We find that accounting for treatment compliance and belief change can better connect experimental studies and the substantive theories of politics they seek to test.

Difference-in-differences (DiD) designs for estimating causal effects have grown in popularity throughout political science. Many DiD papers present their central results through an “event study plot” – a visualization that combines estimated average treatment effects for multiple post-treatment time periods alongside placebo tests of the main identifying assumption: parallel trends. Despite their ubiquity, the methods used in practice for the creation of these plots are inconsistent and in many cases can result in misleading inferences about both the treatment effects and the placebo checks. Building on and synthesizing recent contributions in the econometric literature on differences-in-differences designs, this paper illustrates some common pitfalls through a replication of three recently published papers in major political science journals. We identify three notable problems related to the incorrect specification of the baseline comparison time, incorrect inclusion of “always-treated” units, and sensitivity to effect homogeneity assumptions. We help provide researchers with additional intuition for the problems that arise due to effect heterogeneity and for the “contamination bias” result of Sun and Abraham (2021) through a novel decomposition of the dynamic event study regression.

This research note investigates the relative explanatory power of pre- and post-Communist institutions for the variation in contemporary development of towns in Guangxi Province in South China. I use a regression discontinuity (RD) design to test whether historical institutional membership of Guangdong, a currently rich coastal province that was historically a center of maritime trade in China, has persisted since the administrative boundary changed between the provinces shortly after the Chinese Communist Party came to power. The study finds little evidence that pre-Communist institutional membership affects contemporary development and suggests that post-Communist institutional evolution at the local level may have contributed more to differences in contemporary development within Guangxi.