any of their derivatives obtained by suffixation). Moreover, in all experiments in which the difference between tweets related and unrelated to ASD was examined, the ground truth was defined by the presence of these words. This was necessitated by the scale of the data which made it practically impossible to perform labelling manually. Consequently, these keywords had to be excluded from the consideration in classification experiments. Clearly this would not be done in any practical application which means that the results we presented are worse, likely significantly so, than they would have been had we been able to use the entirety of tweet content. The high classification accuracy we obtained of approximately 80% provides strong evidence that even with these most salient words removed, the remaining content of ASD-related tweets is highly informative and characteristic. In the final set of experiments we report here we examined the sensitivity of learning to the exact keywords used to select the learning corpus. 

We repeated the classification experiments from the previous section but using for train- ing only a part of the previous training corpus – the part that is matched by three of the four keywords. Thus four experiments were performed, with one of the keywords being left out in each. Then we first evaluated the performance of thus learnt classifier on the test corpus, just as before. We additionally evaluated the performance of the classifier specifically on those tweets which contained the left out keyword but none of the other three. Our results are summarized in the bar plot in Fig. 10. Firstly, observe that in no experiment was the classification performance on the test set greatly negatively affected – in all cases 83-85% of control tweets were correctly classified and 71-76% of ASD-related tweets. This is in line with our previous observations and is highly reassuring since it suggests that the same type of characteristic context is learnt regardless of which specific keywords are used to collect training data. However, the classification results on the tweets only matched by the left out keyword is interesting and provides novel insight. In particular, observe the particularly bad performance in the experiment in which the keyword “adhd” was left out – only about 44% of the tweets matched by this keyword were correctly classified. A manual examination of the misclassified tweets readily revealed the answer – in nearly all cases these seemingly 

