Replication Data for: How dropping subjects who failed manipulation checks can bias your experimental results. An illustrative case
<|begin_of_text|><|start_header_id|>system<|end_header_id|> You are a research archivist expert in dataset metadata. Answer the question truthfully.  <|eot_id|><|start_header_id|>user<|end_header_id|> Here is the Description of a dataset. Description:'Manipulations checks are post-experimental measures widely used to verify that subjects understood the treatment. Some researchers drop subjects who failed manipulation checks in order to limit the analyses to attentive subjects. This short report offers a novel illustration on how this practice may bias experimental results: in the present case, through confirming a hypothesis that is likely false. In a survey experiment, subjects were primed with fictional news stories depicting an economic decline versus prosperity. Subjects were then asked whether the news story depicted an economic decline or prosperity. Results indicate that responses to this manipulation check captured subjects’ pre-existing beliefs about the economic situation. As a consequence, dropping subjects who failed the manipulation check mixes the effects of pre-existin' . Please return only a json list of at most 3 Keywords corresponding to the Description:.<|eot_id|><|start_header_id|>assistant<|end_header_id|> {"Answer" : ["Manipulation checks","Survey experiments","Type I error"]} <|eot_id|>
 Here is the Description of a dataset. Description:'Manipulations checks are post-experimental measures widely used to verify that subjects understood the treatment. Some researchers drop subjects who failed manipulation checks in order to limit the analyses to attentive subjects. This short report offers a novel illustration on how this practice may bias experimental results: in the present case, through confirming a hypothesis that is likely false. In a survey experiment, subjects were primed with fictional news stories depicting an economic decline versus prosperity. Subjects were then asked whether the news story depicted an economic decline or prosperity. Results indicate that responses to this manipulation check captured subjects’ pre-existing beliefs about the economic situation. As a consequence, dropping subjects who failed the manipulation check mixes the effects of pre-existin' . Please return only a json list of at most 3 Keywords corresponding to the Description:.
Manipulations checks are post-experimental measures widely used to verify that subjects understood the treatment. Some researchers drop subjects who failed manipulation checks in order to limit the analyses to attentive subjects. This short report offers a novel illustration on how this practice may bias experimental results: in the present case, through confirming a hypothesis that is likely false. In a survey experiment, subjects were primed with fictional news stories depicting an economic decline versus prosperity. Subjects were then asked whether the news story depicted an economic decline or prosperity. Results indicate that responses to this manipulation check captured subjects’ pre-existing beliefs about the economic situation. As a consequence, dropping subjects who failed the manipulation check mixes the effects of pre-existin
Manipulation checks; Survey experiments; Type I error