Making data accessible: lessons learned from computational reproducibility of impact evaluations
<|begin_of_text|><|start_header_id|>system<|end_header_id|> You are a research archivist expert in dataset metadata. Answer the question truthfully.  <|eot_id|><|start_header_id|>user<|end_header_id|> Here is the Description of a dataset. Description:'The study aims to check the reusability of impact evaluation data by verifying the results presented in published 3ie reports. In order to verify results, we conduct push button replications on the original data and code submitted by the authors.We use the push button replication protocol developed at 3ie to determine the level of comparability of the replication results to the original findings. Our sample includes closed 3ie-funded impact evaluations commissioned between 2008 and 2018.Of the 74 studies in our sample, we successfully reproduced results from 38 studies (51%). 24 (32%) studies were categorized as incomplete and 12 (16%) studies were categorized as having major differences. The cumulative replication rate in 2018 increased to 51%, as compared to the below-40% replication rate in previous years. Overall, on average, it' . Please return only a json list of at most 3 Keywords corresponding to the Description:.<|eot_id|><|start_header_id|>assistant<|end_header_id|> {"Answer" : ["agriculture","economics","impact evaluations","open data","open research","replication","reproducibility","transparency"]} <|eot_id|>
 Here is the Description of a dataset. Description:'The study aims to check the reusability of impact evaluation data by verifying the results presented in published 3ie reports. In order to verify results, we conduct push button replications on the original data and code submitted by the authors.We use the push button replication protocol developed at 3ie to determine the level of comparability of the replication results to the original findings. Our sample includes closed 3ie-funded impact evaluations commissioned between 2008 and 2018.Of the 74 studies in our sample, we successfully reproduced results from 38 studies (51%). 24 (32%) studies were categorized as incomplete and 12 (16%) studies were categorized as having major differences. The cumulative replication rate in 2018 increased to 51%, as compared to the below-40% replication rate in previous years. Overall, on average, it' . Please return only a json list of at most 3 Keywords corresponding to the Description:.
The study aims to check the reusability of impact evaluation data by verifying the results presented in published 3ie reports. In order to verify results, we conduct push button replications on the original data and code submitted by the authors.We use the push button replication protocol developed at 3ie to determine the level of comparability of the replication results to the original findings. Our sample includes closed 3ie-funded impact evaluations commissioned between 2008 and 2018.Of the 74 studies in our sample, we successfully reproduced results from 38 studies (51%). 24 (32%) studies were categorized as incomplete and 12 (16%) studies were categorized as having major differences. The cumulative replication rate in 2018 increased to 51%, as compared to the below-40% replication rate in previous years. Overall, on average, it
agriculture; economics; impact evaluations; open data; open research; replication; reproducibility; transparency