How can an existing bot detection system by customized for health-related research?
An existing bot detection score for each user  can be used as a feature in training
With the increasing use of social media data for health-related research, the credibility of the information from this source has been questioned as the posts may originate from automated accounts or "bots". While automatic bot detection approaches have been proposed, there are none that have been evaluated on users posting health-related information. In this paper, we extend an existing bot detection system and customize it for health-related research. Using a dataset of Twitter users, we first show that the system, which was designed for political bot detection, underperforms when applied to health-related Twitter users. We then incorporate additional features and a statistical machine learning classifier to significantly improve bot detection performance. Our approach obtains F_1 scores of 0.7 for the "bot" class, representing improvements of 0.339. Our approach is customizable and generalizable for bot detection in other health-related social media cohorts.
In recent years, social media has evolved into an important source of information for various types of health-related research. Social networks encapsulate large volumes of data associated with diverse health topics, generated by active user bases in continuous growth. Twitter, for example, has 330 million monthly active users worldwide that generate almost 500 million micro-blogs (tweets) per day. For some years, the use of the platform to share personal health information has been growing, particularly amongst people living with one or more chronic conditions and those living with disability. Twenty percent of social network site users living with chronic conditions gather and share health information on the sites, compared with 12% of social network site users who report no chronic conditions. Social media data is thus being widely used for health-related research, for tasks such as adverse drug reaction detection BIBREF0, syndromic surveillance BIBREF1, subject recruitment for cancer trials BIBREF2, and characterizing drug abuse BIBREF3, to name a few. Twitter is particularly popular in research due to the availability of the public streaming API, which releases a sample of publicly posted data in real time. While early health-related research from social media focused almost exclusively on population-level studies, some very recent research tasks have focused on performing longitudinal data analysis at the user level, such as mining health-related information from cohorts of pregnant women BIBREF4. When conducting user-level studies from social media, one challenge is to ascertain the credibility of the information posted. Particularly, it is important to verify, when deriving statistical estimates from user cohorts, that the user accounts represent humans and not bots (accounts that can be controlled to automatically produce content and interact with other profiles)BIBREF5, BIBREF6. Bots may spread false information by automatically retweeting posts without a human verifying the facts or to influence public opinions on particular topics on purpose BIBREF5, BIBREF7, BIBREF8. For example, a recent study BIBREF9 showed that the highest proportion of anti-vaccine content is generated by accounts with unknown or intermediate bot scores, meaning that the existing methods were not able to fully determine if they were indeed bots. Automatic bot detection techniques mostly rely on extracting features from users' profiles and their social networks BIBREF10, BIBREF11. Some studies have used Honeypot profiles on Twitter to identify and analyze bots BIBREF12, while other studies have analyzed social proximity BIBREF13 or both social and content proximities BIBREF10, tweet timing intervals BIBREF14, or user-level content-based and graph-based features BIBREF15. However, in response to efforts towards keeping Twitter bot-free, bots have evolved and changed to overcome the detection techniques BIBREF16. The objectives of this study are to (i) evaluate an existing bot detection system on user-level datasets selected for their health-related content, and (ii) extend the bot detection system for effective application within the health realm. Bot detection approaches have been published in the past few years, but most of the code and data necessary for reproducing the published results were not made available BIBREF17, BIBREF18, BIBREF19. The only system for which we found both operational code and data available, Botometer BIBREF20 (formerly BotOrNot), was chosen as the benchmark system for this study. To the best of our knowledge, this paper presents the first study on health-related bot detection. We have made the classification code and training set of annotated users available at (we will provide a URL with the camera-ready version of the paper).