Replication Data for: Topics, Concepts, and Measurement: A Crowdsourced Procedure for Validating Topics as Measures
<|begin_of_text|><|start_header_id|>system<|end_header_id|> You are a research archivist expert in dataset metadata. Answer the question truthfully.  <|eot_id|><|start_header_id|>user<|end_header_id|> Here is the Description of a dataset. Description:'Topic models, as developed in computer science, are effective tools for exploring and summarizing large document collections.  When applied in social science research, however, they are commonly used for measurement, a task that requires careful validation to ensure that the model outputs actually capture the desired concept of interest. In this paper, we review current practices for topic validation in the field and show that extensive model validation is increasingly rare, or at least not systematically reported. To supplement current practices, we refine an existing crowd-sourcing method for validating topic quality (Chang et al., 2009) and go on to create new procedures for validating conceptual labels provided by the researcher. We illustrate our method with an analysis of Facebook posts by U.S. Senators and provide software and gu' . Please return only a json list of at most 3 Keywords corresponding to the Description:.<|eot_id|><|start_header_id|>assistant<|end_header_id|> {"Answer" : ["text-as-data, topic model, validation, crowd-sourcing, measurement"]} <|eot_id|>
 Here is the Description of a dataset. Description:'Topic models, as developed in computer science, are effective tools for exploring and summarizing large document collections.  When applied in social science research, however, they are commonly used for measurement, a task that requires careful validation to ensure that the model outputs actually capture the desired concept of interest. In this paper, we review current practices for topic validation in the field and show that extensive model validation is increasingly rare, or at least not systematically reported. To supplement current practices, we refine an existing crowd-sourcing method for validating topic quality (Chang et al., 2009) and go on to create new procedures for validating conceptual labels provided by the researcher. We illustrate our method with an analysis of Facebook posts by U.S. Senators and provide software and gu' . Please return only a json list of at most 3 Keywords corresponding to the Description:.
Topic models, as developed in computer science, are effective tools for exploring and summarizing large document collections.  When applied in social science research, however, they are commonly used for measurement, a task that requires careful validation to ensure that the model outputs actually capture the desired concept of interest. In this paper, we review current practices for topic validation in the field and show that extensive model validation is increasingly rare, or at least not systematically reported. To supplement current practices, we refine an existing crowd-sourcing method for validating topic quality (Chang et al., 2009) and go on to create new procedures for validating conceptual labels provided by the researcher. We illustrate our method with an analysis of Facebook posts by U.S. Senators and provide software and gu
text-as-data, topic model, validation, crowd-sourcing, measurement