document: Framework and algorithms for network bucket testing Bucket testing , also known as split testing , A\/B testing , or 0\/1 testing , is a widely used method for evaluating users ' satisfaction with new features , products , or services . In order not to expose the whole user base to the new service , the mean user satisfaction rate is estimated by exposing the service only to a few uniformly chosen random users . In a recent work , Backstrom and Kleinberg , defined the notion of network bucket testing for social services . In this context , users ' interactions are only valid for measurement if some minimal number of their friends are also given the service . The goal is to estimate the mean user satisfaction rate while providing the service to the least number of users . This constraint makes uniform sampling , which is optimal for the traditional case , grossly inefficient . In this paper we introduce a simple general framework for designing and evaluating sampling techniques for network bucket testing . The framework is constructed in a way that sampling algorithms are only required to generate sets of users to which the service should be provided . Given an algorithm , the framework produces an unbiased user satisfaction rate estimator and a corresponding variance bound for any network and any user satisfaction function . Furthermore , we present several simple sampling algorithms that are evaluated using both synthetic and real social networks . Our experiments corroborate the theoretical results and demonstrate the effectiveness of the proposed framework and algorithms .
doc_bio_tags: O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O