Journal Article

Abstraction and Detail in Experimental Design

Brutger, Ryan, Tingley, Dustin, Renshon, Jonathan, Weiss, Chagai M., Kertzer, Joshua D.

2023

Overview

Political scientists designing experiments often face the question of how abstract or detailed their experimental stimuli should be. Typically, this question is framed in terms of trade-offs relating to experimental control and generalizability: the more context introduced into studies, the less control, and the more difficulty generalizing the results. Yet, we have reason to question this trade-off, and there is relatively little systematic evidence to rely on when calibrating the degree of abstraction in studies. We make two contributions. First, we provide a theoretical framework that identifies and considers the consequences of three dimensions of abstraction in experimental design: situational hypotheticality, actor identity, and contextual detail. Second, we replicate and extend three survey experiments, varying these levels of abstraction. We find no evidence that situational hypotheticality substantively changes results in any of our studies, but do find that increased contextual detail dampens treatment effects, and that the salience of actor identities moderates results in our endorsement experiment.

• Publisher: Wiley,Blackwell Publishing Ltd
• Subject: Context / Contextual information / Experiments / Generalizability / Identity / Political scientists / Questions / Research design / Salience / Terms of trade