s2fieldsofstudy:
Computer Science
Biology

text:
Mega-COV: A Billion-Scale Dataset of 100+ Languages for COVID-19

We describe Mega-COV, a billion-scale dataset from Twitter for studying COVID-19. The dataset is diverse (covers 268 countries), longitudinal (goes as back as 2007), multilingual (comes in 100+ languages), and has a significant number of location-tagged tweets (~169M tweets). We release tweet IDs from the dataset. We also develop two powerful models, one for identifying whether or not a tweet is related to the pandemic (best F1=97%) and another for detecting misinformation about COVID-19 (best F1=92%). A human annotation study reveals the utility of our models on a subset of Mega-COV. Our data and models can be useful for studying a wide host of phenomena related to the pandemic. Mega-COV and our models are publicly available.

Introduction
The seeds of the coronavirus disease 2019  pandemic are reported to have started as a local outbreak in Wuhan (Hubei, China) in December, 2019, but soon spread around the world (WHO, 2020). As of January 24, 2021, the number of confirmed cases around the world exceeded 99.14M and the number of confirmed deaths exceeded 2.13M. 1 In response to this ongoing public health emergency, researchers are mobilizing to track the pandemic and study its impact on all types of life in the planet. Clearly, the different ways the pandemic has its footprint on human life is a question that will be studied for years to come. Enabling scholarship on the topic by providing relevant data is an important endeavor. Toward this goal, we collect and release Mega-Cov, a billion-scale multilingual Twitter dataset with geo-location information. As a result of the pandemic, most countries around the world went into lockdown and the public health emergency has restricted physical aspects of human communication considerably. As hundreds of millions of people spend more time sheltering in place, communication over social media became more important than ever. In particular, the content of social media communication promises to capture significant details about the lives of tens of millions of people. Mega-Cov is intended as a repository of such a content.
There are several ongoing efforts to collect Twitter data, and our goal is to complement these. More specifically, we designed our methods to harvest a dataset that is unique in multiple ways, as follows: Massive Scale: Very large datasets lend themselves to analyses that are not possible with smaller data. Given the global nature of COVID-19, we realize that a large-scale dataset will be most useful as the scale allows for slicing and dicing the data across different times, communities, languages, and regions that are not possible otherwise. For this reason, we dedicated significant resources to harvesting and preparing the dataset. Mega-COV has solid international coverage and brings data from 1M users from 268 countries (see Section 3.1). Overall, our dataset has ∼ 1.5B tweets (Section 2). This is one order of magnitude larger than #COVID-19 (Chen et al., 2020), the largest dataset we know of (∼ 144M tweets as of June 1, 2020). 2 Topic Diversity: We do not restrict our collection to tweets carrying certain hashtags. This makes the data general enough to involve content and topics directly related to COVID-19, regardless of existence of accompanying hashtags. This also allows for investigating themes that may not be directly linked to the pandemic but where the pandemic may have some bearings which should be taken into account when investigating such themes. This is important because users can, and indeed do, post about activities impacted by the health crisis without using any hashtags. In fact, users may not mention COVID-19 at all, even though what they are posting about could be affected by the pandemic one way or another (e.g., "eating habits", "shopping behavior"). Section B and Section C in the Appendix provide a general overview of issues discussed in the dataset. Longitudinal Coverage: We collect multiple data points (up to 3,200) from each user, with a goal to allow for comparisons between the present and the past across the same users, communities, and geographical regions (Section 3.2). Again, this is desirable since without data from pre-COVID-19 time it will be challenging to hold any such comparisons. For example, some users may have stopped posting about "exercising" during the pandemic but we cannot definitely identify this without access to these users' previous data where they may have been posting about their physical activities. Language Diversity: Since our collection method targets users, rather than hashtag-based content, Mega-COV is linguistically diverse. In theory, any language posted to Twitter by a user whose data we have collected should be represented. Based on Twitter-assigned language codes, we identify a total of 65 languages. However, applying two different language detection tools to the whole dataset, we identify more than 100 languages. (Section 3.3). No Distribution Shift: Related to the two previous points, but from a machine learning perspective, by collecting the data without conditioning on existence of specific (or any) hashtags we avoid introducing distribution bias. In other words, the data can be used to study various phenomena in-the-2 Both our own dataset and that of Chen et al. (2020) are growing over time. All our statistics in the current paper are based on our collection as of May 15, 2020. As of October 6, 2020, authors of #COVID-19 report 649.9M tweets on their GitHub (https://github.com/echen102/ COVID-19-TweetIDs), and our own dataset has exceeded 5B tweets.
wild. This warrants more generalizable findings and models. A dataset as large as Mega-COV can be hard to navigate. In particular, an informative description of the dataset is necessary for navigating it. In this paper, we provide an explanation of a number of global aspects of the dataset, including its geographic, temporal, and linguistic coverage. We also provide a high-level content analysis of the data, and explore user sharing of content from particular web domains with a focus on news media. In the context of our investigation of Mega-COV, we make an array of important discoveries. For example, we strikingly discover that, perhaps for the first time in Twitter history, users address one another and retweet more than they post tweets. We also find a noticeable rise in ranks for news sites (based on how frequent their URLs are shared) during 2020 as compared to 2019, with a shift toward global (rather than local) news media. A third finding is how use of the Twitter platform surged in March, perhaps making it the busiest time in the history of the network. Furthermore, we develop two groups of effective neural models: (1) COVID-relevance models (for detecting whether a tweet is related to COVID-19 or not).
(2) COVID-misinformation models (for detecting whether a text carries fake information or not). In addition to releasing our best models, we also apply them to a total of 30M tweets from Mega-COV and release our tags to accelerate further research on the topic.
The rest of the paper is organized as follows: In Section 2, we describe our data collection methods. Section 3 is where we investigate geographic, linguistic, and temporal dimensions of our data. We describe our models for detecting COVID-19 tweets and COVID-misinformation in Section 4. Section 5 is where we apply our relevance and misinformation models to a large sample of Mega-COV. Section 6 is about data release and ethics. We provide a literature review in Section 7, and conclude in Section 8.

Data Collection
To collect a sufficiently large dataset, we put crawlers using the Twitter streaming API 3 on Africa, Asia, Australia, Europe, North America, and South America starting in early January, 2020. This allows us to acquire a diverse set of tweets from which we can extract a random set of user IDs whose timelines (up to 3,200 tweets) we then iteratively crawl every two weeks. This gave us data from July 30 th , 2020 backwards, depending on how prolific of a poster a user is (see Table 4a for a breakdown.). In this paper, we describe and analyze the version of Mega-COV collected up to May 15, 2020 and use the term Mega-COV to refer it. Mega-COV comprises a total of 1, 023, 972 users who contribute 1, 487, 328, 805 tweets. For each tweet, we collect the whole json object. This gives us access to various types of information, such as user location and the language tag (including "undefined") Twitter assigns to each tweet. We then use the data streaming and processing engine, Spark, to merge all user files and run our analyses. To capture a wide range of behaviors, we keep tweets, retweets, and responses (i.e., direct user-touser interactions) as independent categories. Table 1 offers a breakdown of the distribution of the different types of posts in Mega-COV. Tweet IDs of the dataset are publicly available at our GitHub 4 and can be downloaded for research. To the extent it is possible, we intend to provide semi-regular updates to the dataset repository.

Geographic Diversity
A region from which a tweet is posted can be associated with a specific 'point' location or a Twitter place with a 'bounding box' that describes a larger area such as city, town, or country. We refer to tweets in this category as geo-located. A smaller fraction of tweets are also geo-tagged with longitude and latitude. As Table 2 shows, Mega-COV has ∼ 187M geo-located tweets from ∼ 740K users and ∼ 31M geo-tagged tweets from ∼ 267K users. Table 2 also shows the distribution of tweets and users over the top two countries represented in the dataset, the U.S. and Canada (North America), and other locations (summed up as one category, but see also Table 3 for countries in the data by con- Figure 2: World map coverage of Mega-COV. Each dot is a point co-ordinate (longitude and latitude) from which at least one tweet was posted. Clearly, users tweet while traveling, whether by air or sea. tinent). As explained, to allow comparisons over time (including behavioral changes during COVID-19), we include pre-2020 data in Mega-COV. For the year 2020, Mega-COV has ∼ 66M geo-located tweets from ∼ 670K users and ∼ 3M geo-tagged tweets from ∼ 109K users. 5 We note that significant parts from the data could still belong to the different countries but just not geo-located in the original json files retrieved from Twitter. Figure 2 shows actual point co-ordinates of locations from which the data were posted. Figure 3 shows the geographical diversity in Mega-COV based on geo-located data. We show the distribution in terms of the number of cities over the 20 countries from which we retrieved the highest number of locations in the dataset, broken by all-time and the year 2020. Overall, Mega-COV has data posted from a total of 167, 202 cities that represent 268 countries. Figure A.2 in Appendix A shows the distribution of data over countries. The top 5 countries in the data are the U.S., Canada, Brazil, the U.K., and Japan. As we mention earlier, other top countries in the data across the various continents are shown in Table 3.

Temporal Coverage
Our goal is to make it possible to exploit Mega-COV for comparing user social content over time. Since we crawl user timelines, the dataset comprises content going back as early as 2007.    of 2020 compared to the same period in 2019. This is expected, both due to physical distancing and a wide range of human activity (e.g., "work", "shopping") moving online. More precisely, moving activities online causes users to be on their machines for longer times and hence have easier access to social media. The clear spike in the month of March 2020 is striking. It is particularly so given a shifted pattern of use: retweeting and replying (to others) are both observably more frequent than tweeting itself. This especially takes place during the month of March, and somewhat continues in April, as shown in Figure 4b. Figure 4a and Figure 4b also show a breakdown of tweets, retweets, and replies.
A striking discovery is that, for 2020, users are engaged in conversations with one another more than tweeting directly to the platform. This may be the first time this pattern exists, perhaps in the history of the network. At least based on our massive dataset, this conclusion can be made. In addition, for 2020, we also see users retweeting more than tweeting. Based on Mega-COV, this is also happening for the first time.

Linguistic Diversity
We perform the language analysis based on tweets (n=∼ 1.5B), including retweets and replies. Twitter assigns 65 language ids to ∼ 1.4B tweets, while the rest are tagged as "und" (for "undefined").
Mega-COV has ∼ 104M (∼ 7%) tweets tagged as "und". We run two language identification tools, langid (Lui and Baldwin, 2012) and Compact Language Detector (Ooms and Sites, 2018) 6 langid (Lui and Baldwin, 2012), 7 on the whole dataset (including tweets tagged "und" by Twitter). 8 After merging language tags from Twitter and the 2 tools, we acquire a total of 104 labels. This makes Mega-COV very linguistically rich. Table 4 shows the top 20 languages identified by Twitter (left) and the top 20 languages tagged by one of the two tools, langid (Lui and Baldwin, 2012), after removing the 65 Twitter languages (right).

Models
We develop two groups of models suited for answering important questions related to COVID-19, including making use of Mega-COV. These are (1) COVID-relevance, where a classifier will   label a tweet as relevant to COVID-19 or not and (2) COVID-misinformation, where a model predicts text veracity pertaining COVID-19 (i.e., whether a text carries true or fake information related to the pandemic). We now describe our methods.

Methods
For all our models, we fine-tune 3 popular pre-

Hyper-Parameters and Optimization
For each model, we use the same pre-processing in the respective code released by the authors. For all models, we typically use a sequence length of 50 tokens. We use a learning rate of 5e−6 and a batch size of 32. We train each model for 20 epochs and identify the best epoch on a development set. We report performance on both development and test sets. We describe our baseline for each of the relevance and misinformation models in the respective sections below. We now introduce each of these two model groups.

COVID-Relevance Models
Data. Our COVID-relevance models predict whether a tweet is related to COVID-19 or not (i.e., not related). To train the models, we sample ∼ 2.3M multilingual tweets (65 languages TRAIN (n=3,146,334), 10% DEV (n=393,567), and 10% TEST (n=392,918). We then remove all hashtags which were used by  for collecting the data and fine-tune each of the 3 language models on TRAIN.
Results. As shown in Table 5, XLM-R Large acquires best results with 97.95 acc and 97.93 macro F 1 on TEST. These results are significantly better than a majority class baseline (based on TRAIN) and another arbitrarily-chosen (yet quite competitive) baseline model that chooses the related class (majority class in TRAIN) 75% of the time.
Model Generalization. Our COVID-relevance models are trained with distant supervision (hashtags as surrogate labels). It is conceivable that content related to COVID-19 would still occur in real world without accompanying hashtags. To test the extent to which our best model would perform on external data, we evaluate it on two external Twitter datasets, CoAID (Cui and Lee, 2020) and ReCOVery (Zhou et al., 2020), both of which are claimed by the authors to be completely (100%) related to  As Tabel 6 shows, We do observe a drop in model performance as compared to our best model on our own TEST set in Table 5 (acc drops on average by 15.5% and 7.6% F 1 ). However, the best model is still highly effective. It acquires an average acc of 82.46% and F 1 of 90.38% on the CoAID and ReCOVery datasets. We now introduce our misinformation models. 9 Each of the two datasets are also labeled for fake news (true vs. fake) focused on COVID-19, but our focus here is exclusively on using the two datasets as gold-labeled TEST sets for evaluating our COVID-relevance model. Note that we will use these two datasets again as explained in Section 4.4 as well.

COVID-Misinformation Models
To train models for detecting the veracity of news related to COVID-19, we exploit two recent and publicly available fake news datasets (in English): CoAID (Cui and Lee, 2020), and ReCOVery (Zhou et al., 2020). We now describe each of these datasets:   CoAID. Cui and Lee (2020) present a Covid-19 heAthcare mIsinformation Dataset (CoAID), with diverse COVID-19 healthcare misinformation, including fake news on websites and social platforms, along with related user engagements (i.e., tweets and replies) about such news. CoAID includes 3, 235 news articles and claims, 294, 692 user engagement, and 851 social platform posts about COVID-19. The dataset is collected from December 1, 2019 to July 1, 2020. Table 7 shows class distribution of news articles and tweets in CoAID.
More information about CoAID is in Appendix F. ReCOVery. Zhou et al. (2020) choose 60 news publishers with 'extreme' levels of credibility (i.e., true vs. fake classes) from an original list of ∼ 2, 000 to collect a total of 2, 029 news articles on COVID-19, published between January and May 2020. They also collect 140, 820 tweets related to the news articles, considering those tweets related to true articles to be true and vice versa. Table 7 shows class distribution of news articles and tweets in ReCOVery. Splits and Cleaning. Table 8 shows the distribution of tweets in CoAID and Recovery before and after the de-duplication process. As Table 8 shows, de-duplication results in significantly reducing the sizes of DEV and TEST sets in the two resources. The distribution of news article is shown in Table F.1 (Appendix F).
Training. We use both CoAID and ReCOVery after de-duplication for training neural models to detect fake news related to Covid-19. Using the same hyper-parameters and training setup as the COVIDrelevance models, we fine-tune the pre-trained language models on the Twitter dataset and the news dataset, independently. 10 Since Mega-COV is a social media dataset, we only focus on training Twitter models here and provide the news models in Appendix F. For the Twitter models, we develop one model on CoAID, another on ReCOVery, independently, and a third model for CoAID+ReCOVery (concatenated). Again, for each of these 3 datasets, we fine-tune on TRAIN and identify the best model on DEV. We then report the best model on both DEV and TEST.
Results. Since our focus is on detecting fake texts, we show results on the positive class only in Table 9. We report results in terms of precision, recall and F 1 . Our baseline is a small LSTM with 2 hidden layers, each of which has 50 nodes. We add a dropout of 0.2 after the first layer and arbitrarily train the LSMT for 3 epochs. As Table 9 shows, our best results for fake tweet detection on TEST for CoAID is at 90% F 1 (mBERT/XLM-R Large ), for ReCOV 68% (mBERT), and for these two combined is 92%. All results are above the LSTM baseline. We show results of the COVIDmisinformation news models in Table F.2 (Appendix F).

Applications on Mega-COV
Now that we have developed two highly effective models, one for COVID-relevance and another for COVID-misinformation, we can employ these models to make discoveries using Mega-COV. Since 10 Even though we could have used the monolingual versions of the transformer-based language models (i.e., BERT and RoBERTa), we stick to the multilingual versions for consistency.  our misinformation models are focused only on English (due to the external gold data we used for training being English only), we will restrict this analysis to the English language. 11 We were curious whether model predictions will have different distributions on the different types of Twitter posts (i.e., tweets, retweets, and replies). Hence, to enable such comparisons, we extract a random sample of 10M samples from each of these post types (for a total of 30M) from the year 2020 in Mega-COV.
We then apply the XLM-R Large relevance and misinformation models on the extracted samples. Table 10 shows the distribution of predicted labels from each of the two models across the 3 posting types (tweets, retweets, and replies). Strikingly, as the top half of the table shows, while only 7.77% of tweets are predicted as COVID-related, almost all retweets (99.84%) are predicted as related. This shows that users' retweets were focused almost exclusively on COVID-19. The table (bottom half) also shows that retweets are highest carriers of content predicted as fake (3.67%), followed by tweets (2.3%). From the table, we can also deduce that only 2.45% of all English language Twitter content (average across the 3 posting types) are predicted as fake. Given the global use of English, and the large volume of English posts Twitter receives daily, this percentage of fake content is still problematically high.

Annotation Study
We perform a human annotation study on a small sample of 150 random posts from those the model predicted as both COVID-related and fake. Two annotators labeled the 150 samples for two types of 11 But we emphasize the multilingual capacity of our COVID-relevance model. tags, relevance and veracity. For relevance, all the 150 posts were found relevant by the two annotators (perfect agreement). For veracity, since some posts can be very challenging to identify, we asked annotators to assign one of the 3 tags in the set {true, fake, unknown}. We did not ask annotators to consult any outside sources (e.g., Wikipedia or independent fact-checking sites) to identify veracity of the samples. Inter-annotator agreement is at Kappa (K)=77.81%, thus indicating almost perfect agreement. On average, annotators assigned the fake class 39.39% of the time, the true class 3.02%, and the unknown class 57.05%. While these findings show that it is hard for humans to identify data veracity without resorting to external sources, it also demonstrates the utility of the model in detecting actual fake stories in the wild. We provide a number of samples from the posts that were automatically tagged as COVID-related and either true or false by our misinformation/veracity model in Table 11.  Table 10: Distribution of predicted labels from our COVID-relevance and COVID-misinformation models on randomly selected 30M English samples from Mega-COV data.

Data Release and Ethics
Data Distribution. The size of the data makes it an attractive object of study. Collection and exploration of the data required significant computing infrastructure and use of powerful data streaming and processing tools. To facilitate use of the dataset, we organize the tweet IDs we release by time (month and year) and language. This should enable interested researchers to work with the exact parts of the data related to their research questions even if they do not have large computing infrastructure.
Ethical Considerations. We collect Mega-COV from the public domain (Twitter). In compliance with Twitter policy, we do not publish hydrated tweet content. Rather, we only publish publicly available tweet IDs. All Twitter policies, including respect and protection of user privacy, apply. We decided not to assign geographic region tags to the tweet IDs we distribute, but these already exist on the json object retrievable from Twitter. Still, location information should be used with caution. Twitter does not allow deriving or inferring, or storing derived or inferred, potentially sensitive characteristics about users. Sensitive user attributes identified by Twitter include health (e.g., pregnancy), negative financial status or condition, political affiliation or beliefs, religious or philosophical affiliation or beliefs, sex life or sexual orientation, trade union membership, and alleged or actual commission of a crime. If they decide to use Mega-COV, we expect researchers to review Twitter policy 12 and applicable laws, including the European Union's General Data Protection Regulation (GDPR) 13 , beforehand. We encourage use of Mega-COV for social good, including applications that can improve health and well-being and enhance online safety. Is COVID -19 airborne contagious ? New study shows that coronavirus may be caught from the air * 3 -hours * after it has been exposed . True A close relative of SARS-CoV -2 found in bats offers more evidence it evolved naturally URL True Antiviral remdesivir prevents disease progression in monkeys with COVID -19 -National Institutes of Health ( NIH ) URL True COVID Surges Among Young Adults URL True  Osho et al., 2020;Pierri et al., 2020;Koubaa, 2020), estimating the rate of misinformation in COVID-19 associated tweets (Kouzy et al., 2020), the use of bots (Ferrara, 2020), predicting whether a user is COVID19 positive or negative (Karisani and Karisani, 2020), and the quality of shared links Singh et al. (2020).

Conclusion
We presented Mega-COV, a billion-scale dataset of 104 languages for studying COVID-19 pandemic. In addition to being large and highly multilingual, our dataset comprises data pre-dating the pandemic. This allows for comparative and longitudinal investigations. We provided a global description of Mega-COV in terms of its geographic and temporal coverage, over-viewed its linguistic diversity, and provided analysis of its content based on hashtags and top domains. We also provided a case study of how the data can be used to track global human mobility. The scale of the Mega-COV has also allowed us to make a number of striking discoveries, including (1) the shift toward retweeting and replying to other users rather than tweeting in 2020 and (2) the role of international news sites as key sources of information during the pandemic. In addition, we developed effective models for detecting COVID relevance and COVID misinformation and applied them to a large sample of our dataset. Our dataset and models are publicly available.

B Hashtag Content Analysis
Hashtags usually correlate with the topics users post about. We provide the top 30 hashtags in the data in  shows (e.g., Big Boss), doctors, and even fake news along with the pandemic-related hashtags.
An interesting observation from the Chinese language word cloud is the use of hashtags such as ChinaPneumonia and WuhanPneumonia to refer to the pandemic. We did not observe these same hash- Figure B.1: Word clouds for hashtags in tweets from the top 10 languages in the data. We note that tweets in non-English can still carry English hashtags or employ Latin script. tags in any of the other languages. Additionally, for some reason, Apple seems to be trending during the first 4 months of 2020 in China owing to hashtags such as appledaily and appledailytw. Some languages, such as Romanian and Vietnamese, involve discussions of bitcoin and crypto-currency. This was also seen in the Chinese language word cloud, but not as prominently.

C Domain Sharing Analysis
Domains in URLs shared by users also provide a window on what is share-worthy. We perform an analysis of the top 200 domains shared in each of 2019 and 2020. The major observation we reach is the surge in tweets involving news websites, and the rise in ranks for the majority of these websites compared to 2019. Table C.1 shows the top 40 news domains in the 2020 data and their change in rank compared to 2019. Such a heavy sharing of news domains reflects users' needs: Intuitively, at times of global disruption, people need more frequent updates on ongoing events. Of particular importance, especially relative to other ongoing political polarization in the U.S., is the striking rise of the conservative news network Fox News, which has moved from a rank of 118 in 2019 to 67 in 2020 with a swooping 51 positions jump. We also note the rank of some news sites (e.g., The Globe and Mail and The Star going down. This is perhaps due to people resorting to international (and more diverse) sources of information to remain informed about countries other than their own. Other domains: Other noteworthy domain activities include those related to gaming, video and music, and social media tools. Ranks of these domains have not necessarily shifted higher than 2019 but remain prominent. This shows these themes still being relevant in 2020. In spite of the eco-  nomic impact of the pandemic, shopping domains such as etsy.me and poshmark.com have markedly risen in rank as people moved to shopping online in more significant ways. We now introduce a case study as to how our data can be used for mobility tracking.

D Case Study: Mapping Human Mobility with Mega-COV
Geolocation information in Mega-COV can be used to characterize and track human mobility in various ways. We investigate some of these next.
Inter-Region Mobility. Mega-COV can be exploited to generate responsive maps where end users can check mobility patterns between different regions over time. In particular, geolocation information can show mobility patterns between regions. As an illustration of this use case, albeit Intra-Region Mobility. We also use information in Mega-COV to map each user to a single home region (i.e., city, state/province, and country). We follow Geolocation literature (Roller et al., 2012;Graham et al., 2014;Han et al., 2016;Do et al., 2018) in setting a condition that a user must have posted at least 10 tweets from a given region. However, we also condition that at least 60% of all user tweets must have been posted from the same region. We use the resulting set of users whose home location we can verify to map user weekly mobility within their own city, state, and country exclusively for both Canada and the U.S. as illustrating examples. We provide the related visualization in supplementary material under "User Weekly Intra-Region Mobility". 14 Here, due to increased posting in 2020, we normalize the number of visits between states by the total number of all tweets posted during a given month.

D.1 User Weekly Intra-Region Mobility
We can also visualize user mobility as a distance from an average mobility score on a weekly basis. Namely, we calculate an average weekly mobility score for the year 2019 using geo-tag information (longitude and latitude) and use it as a baseline against which we plot user mobility for each week of 2019 and 2020 up until April. In general, we observe a drop in user mobility in Canada starting from mid-March. For U.S. users, we notice a very high mobility surge starting around end of February and early March, only waning down the last week of March and continuing in April as shown in Figure D.8. For both the U.S. and Canada, we hypothesize the surge in early March (much more noticeable in the U.S.) is a result of people moving back to their hometowns, returning from travels, moving for basic needs stocking, etc.

E.1 Dataset
We randomly sample 200K tweets from the English data in Chen et al. (2020) and a maximum of 100K from each of the rest of languages. For languages where there is < 100K tweets, we take all data. For the negative class, we extract data from Jan-Nov, 2019 from Mega-COV. For each language, we take roughly the same number of tweets we sampled for the positive class. Table E    • User Engagement. Queries based on the true and fake articles and claims were used to build a dataset of user engagement from Twitter where the goal was to acquire the tweets discussing the news in question and related Twitter replies.


extfieldsofstudy:
Computer Science
Medicine

text:
Impact of Visual Design Elements and Principles in Human Electroencephalogram Brain Activity Assessed with Spectral Methods and Convolutional Neural Networks

The visual design elements and principles (VDEPs) can trigger behavioural changes and emotions in the viewer, but their effects on brain activity are not clearly understood. In this paper, we explore the relationships between brain activity and colour (cold/warm), light (dark/bright), movement (fast/slow), and balance (symmetrical/asymmetrical) VDEPs. We used the public DEAP dataset with the electroencephalogram signals of 32 participants recorded while watching music videos. The characteristic VDEPs for each second of the videos were manually tagged for by a team of two visual communication experts. Results show that variations in the light/value, rhythm/movement, and balance in the music video sequences produce a statistically significant effect over the mean absolute power of the Delta, Theta, Alpha, Beta, and Gamma EEG bands (p < 0.05). Furthermore, we trained a Convolutional Neural Network that successfully predicts the VDEP of a video fragment solely by the EEG signal of the viewer with an accuracy ranging from 0.7447 for Colour VDEP to 0.9685 for Movement VDEP. Our work shows evidence that VDEPs affect brain activity in a variety of distinguishable ways and that a deep learning classifier can infer visual VDEP properties of the videos from EEG activity.

Overview
Visual communication design is the process of creating visually engaging content for the creative industries, to communicate specific messages and persuade in a way that eases the comprehension of the message and ensures a lasting impression on its recipient. Therefore, the main objective of visual communication design as a discipline, is the creation of visually attractive content that impacts the target audience through persuasion, the stimulation of emotions and the generation of sentiments. This objective is achieved through conscious manipulation of the inherent elements and principles, such as shape, texture, direction harmony, and colour that make up the designed product [1].
The elements of visual design describe the building blocks of a product's aesthetics, and the principles of design tell how these elements should go together for the best results. Although there is no consensus on how many visual design elements and principles (VDEPs) there are, some of these design reference values have been thoroughly studied for centuries and are considered the basis of modern graphic design [2]. One of the most important breakthroughs in graphic design history is the theory of the Gestalt in the 1930s, which studied these VDEPs from a psychological point of view, and discovered that the notion of visual experience is inherently structured by the nature of the stimulus triggering it as it interacts with the visual nervous system [3]. Since then, several studies have explored the psychological principles of different VDEPs and their influence on their observers [4][5][6]. As an example, in colour theory, some colours have been experimentally proven to convey some semantic meanings, and they are often used to elicit particular emotions in the viewers of visual content [7][8][9].
Within this context, in recent years diverse research has been carried out to deepen our understanding of brain activity and visual perception [10]. Research findings on visual perception have been applied to real-world applications in a wide range of fields, including visual arts, branding, product management, and packaging design to increase consumer spending, creating engaging pedagogic content, and survey design, among others [11][12][13][14]. These works mostly focus on the VDEPs capability of eliciting specific emotions in the viewer and how to trigger the desired response from the viewer, such as favouring the learning of content or increasing the probability of purchasing some goods. Multiple approaches have recently addressed the study of human visual perception from within the fields of attention and perception psychology and computational intelligence with studies focused on the neuropsychology of attention and brain imaging techniques and their applications (fMRI or electroencephalogram, EEG) [14,15], pupillometry [16] or eye-tracking [17,18] as a scanning technique of information processing as well as neuroaesthetics [19], computational neuroscience [20], computational aesthetics [21], or neuromarketing, and consumer neuroscience studies [22,23], among others [24].
However, it is not yet known how the VDEPs impact human brain activity. In this regard, we turn our attention to the DEAP dataset [25], a multimodal dataset for the analysis of human affective states with EEG recordings and synchronous excerpts of music videos. Previous research carried out with DEAP dataset pursued emotion recognition from the EEG records by using 3D convolutional neural networks [26], a fusing of learned multimodal representations and dense trajectories for emotional analysis in videos [27], studying arousal and valence classification model based on long short-term memory [28], DEAP data for mental healthcare management [28], or accurate EEG-based emotion recognition on combined features using deep convolutional neural networks, among others [29].
Beyond the information about levels of valence, arousal, like/dislike, familiarity, and dominance in the DEAP dataset, we sought to analyse the EEGs and videos to extract some of the most important VDEPs-Balance, Colour, Light, and Movement-and search for measurable effects of these on the observable EEG brain activity. The Balance, Colour, Light and Movement principles are fundamental for professional design pipelines and are universally considered to influence the emotions of the viewer, as designers rely on the contrasts, schemes, harmony, and interaction of colours to evoke reactions and moods in those who look at their creations [30][31][32][33].
The Balance refers to the visual positioning or distribution of objects in a composition, gives the visual representation of a sense of equality, and can be achieved through a symmetrical or asymmetrical composition to create a relationship of force between the various elements represented, either to compensate or, to the contrary, to decompensate the composition [6,34]. We speak of symmetrical balance when the elements are placed evenly on both sides of the axes. Secondly, the Colour helps communicate the message by attracting attention, setting the tone of the message, guiding the eye to where it needs to go and determines 90% of the choice of a product [35,36]. The warmth or coldness of a colour attends to subjective thermal sensations; it can be cold or warm depending on how it is perceived by the human eye and the interpretation of the thermal sensation it causes [36]. Thirdly, the Light or "value" refers to relative lightness and darkness and is perceived in terms of varying levels of contrast; it determines how light or dark an area looks [34,37]. Finally, Movement refers to the suggestion of motion using various elements and it is the strongest visual inducement to attention. Different studies have dissected the exact nature of our eye movement habits and the patterns our eyes trace over when viewing specific things [17,18]. Strongly connected with direction VDEPs, rhythm/movement provide designers with the chance to create final pieces with good flow from top to bottom, left to right, corner A to corner B, etc., [37]. By layering simple shapes of varying opacities, an abrupt change in the camera, a counterstroke or a tracking shot, or a sudden change in the subject's action, it is possible to create a strong sense of speed and motion and determine an effect of mobility or immobility.

EEG Symmetry Perception
Symmetry is known to be an important determinant of aesthetic preference. Sensitivity to symmetry has been studied extensively in adults, children, and infants with diverse research ranging from behavioural psychology to neuroscience. Adults detect symmetrical visual presentations more quickly and accurately than asymmetrical ones and remember them better [38,39]. The perception and appreciation of visual symmetry have been studied in several EEG/fMRI experiments, some of the most recent studies are focused on symmetric patterns with different luminance polarity (anti-symmetry) [40], the role of colour and attention-to-colour in mirror-symmetry perception [41], or contour polarity [42], among others.

EEG Colour Perception
The perception of colour is an important cognitive feature of the human brain and is a powerful descriptor that considerably expands our ability to characterize and distinguish objects by facilitating interactions with a dynamic environment [43]. Some of the most recent studies that have addressed the interactions of colour and human perception through EEG are the response of a human visual system to continuous colour variation [44], human brain perception and reasoning of image complexity for synthetic colour fractal and natural texture images via EEG [45], or neuromarketing studies based on the study of colour perception and the application of EEG power for the prediction and interpretation of consumer decision-making [46], and so forth.

EEG Brightness Perception
Brightness is one of the most important sources of perceptual processing and may have a strong effect on brain activity during visual processing [47]. Brain responses of the brain to changes in brightness were explored in different studies centred on the luminance and spatial attention effects on early visual processing [48], the grounding valence in brightness through shared relational structures [49] or the interaction of brightness and semantic content in the extrastriate visual cortex [50].

EEG Visual Motion Perception
Detecting the displacement of retinal image features has been studied for many years in both psychophysical and neurobiological experiments [51]. Visual motion perception has been explored through EEG technique in several lines of research such as the speed of neural visual motion perception and processing in the visuomotor reaction time of young elite table tennis athletes [52], visual motion perception for prospective control [53], or visual perception of motion to cortical activity, by evaluation of the association of quantified EEG parameters with a video film projection [54], the analysis of neural responses to motion-in-depth using EEG [55], or the examination of the time course of motion-in-depth perception [56].
In addition, recent research has sought to delve deeper into Brain-Computer Interfaces (BCI), an emerging area of research that aims to improve the quality of human-computer ap- plications [57], and the relationship with Steady-State Visually Evoked Potentials (SSVEPS), a stimulus-locked oscillatory response to periodic visual stimulation commonly recorded in an electroencephalogram (EEG) studies in humans, through the use and execution of Convolutional Neural Networks (CNN) [58,59]. Some of these research works have focused on the review of the steady-state evoked activity, its properties, and the mechanisms of SSVEP generation, as well as the SSVEP-BCI paradigm and the recently developed SSVEP-based BCI systems [60]; studies focused on analysing Deep Learning-based classification for BCI through comparisons among various traditional classification algorithms to the newer methods of deep learning, exploring two different types of deep learning methods: CNN and Recurrent Neural Networks (RNN) with long short-term memory (LSTM) architecture [61]. In addition, recent research works proposed a novel CNN for the robust classification of an SSVEPS paradigm, where the authors measured electroencephalogram (EEG)-based SSVEPs for a brain-controlled exoskeleton under ambulatory conditions in which numerous artefacts may deteriorate decoding [62]. Another work that analyses SSVEPS by using CNNs is the work of S.Stober et al. [63], which analysed and classified EEG data recorded within a rhythm perception study. In this last case, the authors investigated the impact of the data representation and the pre-processing steps for this classification task and compared the different network structures.

Objective
It is not known whether predictors can be constructed to classify these VDEPs based on brain activity alone. The large body of previous research explored the effects of the VDEPs as isolated features, even though during human visual processing and perception many of them act simultaneously and are not appreciated individually by the viewer in different situations.
In this work, we present a novel methodology for exploring this relationship between VDEPs and brain activity in the form of EEGs available on the DEAP dataset. Our methodology consisted of combining the expert recognition and classification of VDEPs, statistical analysis, and deep learning techniques, which we used to successfully predict VDEPs solely from the EEG of the viewer. We tested whether: There is a statistical relationship between the VDEPs of video fragments and the mean EEG frequency bands (δ, θ, α, β, γ) of the viewers. A simple Convolutional Neural Network model can accurately predict the VDEPs in video content from the EEG activity of the viewer.

DEAP Dataset
The DEAP dataset is composed of the EEG records and peripheral physiological signals of 32 participants, which were recorded as each watched 40 1-min-long excerpts of music videos, relating to the levels of valence, arousal, like/dislike, familiarity, and dominance reported by each participant. Firstly, the ratings came from an online selfassessment where 120 1-min extracts of music videos were rated by volunteers based on emotion classification variables (namely arousal, valence, and dominance) [25]. Secondly, the ratings came from the participants' ratings on these emotion variables, face video, and physiological signals (including EEG) of an experiment where 32 volunteers watched a subset of 40 of the abovementioned videos. The official dataset also includes the YouTube links of the videos used and the pre-processed physiological data (down sampling, EOG removal, filtering, segmenting, etc.) in MATLAB and NumPy (Python) format.
The data were down sampled to 128 Hz.

3.
A bandpass frequency filter from 4 to 45 Hz was applied.

4.
The data was averaged to the common reference.

5.
The EEG channels were reordered so that all the samples followed the same order. 6. The data was segmented into 60-s trials and a 3-s pre-trial baseline was removed. 7.
The trials were reordered from presentation order to video (Experiment_id) order.
In our experiments, we use the provided pre-processed EEG data in NumPy format, as recommended by the dataset summary, since it is especially useful for testing classification and regression techniques without the need of processing the electrical signal first. This pre-processed signal contains 32 arrays corresponding to each of the 32 participants, with a shape of 40 × 40 × 8064. Each array contains data for each of the 40 videos/trials of the participant, signal data from 40 different channels (the first 32 of them being EEG signals and the remainder 8 being peripheral signals such as temperature and respiration) and 8064 EEG samples (63 s × 128 Hz). As we are not working with emotion in this work, the labels provided for the videos on the DEAP dataset were not used in this experiment. Instead, we retrieved the original videos from the URLs provided and performed an exhaustive classification of them on the studied VDEPs. Of the original 40 videos, 14 URLs pointed to videos that were taken down from YouTube as of the 4 June 2019, therefore, only 26 of the videos used in the original DEAP experiment were retrieved and classified. The generated dataset was updated on the 5 May 2021.
The timestamps tagged as "unclear" or where experts disagreed were discarded. On the other hand, the number of timestamps per class was unbalanced, because the music videos exhibited different visual aesthetics and characteristics among different sections within the same videos; for example, most of the timestamps were tagged as asymmetrical. Therefore, we computed a sub-sample for each VDEP, selecting n v random timestamps, where n v represents the number of cases in the smaller class. The resultant number of timestamps belonging to each class is displayed in Table 1. It is important to notice that all the participants provided the same amount of data points, therefore each second from Table 1 corresponds to 32 epochs, one for each participant. The four VDEPs are being considered as independent of each other, therefore, each second of video will appear classified only once in each of the four VDEPs, either within one of the two classes or as an unclear timestamp. The VDEPs tagging of the selected video samples from the DEAP dataset is available in Appendix A (Table A11). Since the DEAP dataset provides the EEG signal and the experts provided the VDEP timestamps for the videos used in the original experiment, an extraction-transformation process (ETP) was performed. In this process, we obtained a large set of 1-s samples from the DEAP dataset classified according to their VDEPs. The steps of this ETP are shown in Figure 1.

Light 795 654 111
Since the DEAP dataset provides the EEG signal and the experts provided the VDEP timestamps for the videos used in the original experiment, an extraction-transformation process (ETP) was performed. In this process, we obtained a large set of 1-s samples from the DEAP dataset classified according to their VDEPs. The steps of this ETP are shown in Figure 1. This work uses statistical analysis as well as convolutional neural networks to analyse the relationship between VDEPs and the EEG, so we designed the ETP to return the outputs in two different formats, readily available to perform each of these techniques. The first output is a set of spreadsheets containing the EEG bands for the studied samples and their VDEP classes, which are used to find statistical relationships. The second output is a set of NumPy matrices with the shape of 128 × 32, each containing the 32 channels of a second of an EEG signal at 128 Hz and classified by their VDEP class that we used to train and test the artificial neural networks. This ETP was repeated four times, one for each VDEP studied in this work.
The first step of the ETP is the extraction of the intervals from the DEAP dataset according to the VDEP timestamps provided by the experts. In this step, we must take into consideration that the first 3 s of the EEG signal for each video trial in the DEAP dataset corresponds to a pre-trial for calibration purposes and therefore must be ignored when processing the signals. We also discard the channels numbered 33-40 in the DEAP dataset since they do not provide EEG data but peripheral signals, which are out of the scope of this work.
In the second step, we trimmed the first and last 0.5 s of each interval, and in the third step, we split the remainder of the signal in 1-s intervals. Given that the visual design experts extracted the VDEPs from the videos by 1-s segments, the removal of a full second of each interval (half a second on each end) strengthens the quality of the remaining samples by reducing the truncation error. An epoch is one second of a subject's EEG with information about the VDEPs that is currently being displayed. When extracting epochs, subjects are not considered separately. For the authors, the subject has attributed a series of samples, as well as the rest of the subjects, to finally put them all together and obtain a set of epochs independently of the subject they come from. Therefore, from each second of the video, we extract 32 epochs (one for each subject). The samples are extracted from the remainder of the interval by taking 1-s samples, this approach mitigates the effects of the perception time, which is the delay between the occurrence of the stimuli and its perception by the brain [64]. In Figure 2 we show the resulting samples obtained from a 5-s interval classified in the same class. This work uses statistical analysis as well as convolutional neural networks to analyse the relationship between VDEPs and the EEG, so we designed the ETP to return the outputs in two different formats, readily available to perform each of these techniques. The first output is a set of spreadsheets containing the EEG bands for the studied samples and their VDEP classes, which are used to find statistical relationships. The second output is a set of NumPy matrices with the shape of 128 × 32, each containing the 32 channels of a second of an EEG signal at 128 Hz and classified by their VDEP class that we used to train and test the artificial neural networks. This ETP was repeated four times, one for each VDEP studied in this work.
The first step of the ETP is the extraction of the intervals from the DEAP dataset according to the VDEP timestamps provided by the experts. In this step, we must take into consideration that the first 3 s of the EEG signal for each video trial in the DEAP dataset corresponds to a pre-trial for calibration purposes and therefore must be ignored when processing the signals. We also discard the channels numbered 33-40 in the DEAP dataset since they do not provide EEG data but peripheral signals, which are out of the scope of this work.
In the second step, we trimmed the first and last 0.5 s of each interval, and in the third step, we split the remainder of the signal in 1-s intervals. Given that the visual design experts extracted the VDEPs from the videos by 1-s segments, the removal of a full second of each interval (half a second on each end) strengthens the quality of the remaining samples by reducing the truncation error. An epoch is one second of a subject's EEG with information about the VDEPs that is currently being displayed. When extracting epochs, subjects are not considered separately. For the authors, the subject has attributed a series of samples, as well as the rest of the subjects, to finally put them all together and obtain a set of epochs independently of the subject they come from. Therefore, from each second of the video, we extract 32 epochs (one for each subject). The samples are extracted from the remainder of the interval by taking 1-s samples, this approach mitigates the effects of the perception time, which is the delay between the occurrence of the stimuli and its perception by the brain [64]. In Figure 2 we show the resulting samples obtained from a 5-s interval classified in the same class. These samples can be represented as (128 × 32) matrices and can be used to obtain EEG characteristics such as the delta, theta, alpha, beta, and gamma channels that we used in our statistical analysis. In our case, we used a basic FFT (Fast-Fourier Transform) filtering function to extract these time-domain waveforms [65].
The second output was used to train artificial neural networks (ANN), so in the ETP we normalized the values of the samples from their original values to the [0, 1] interval by using Equation (1), as normalized inputs have proven an increase on the learning rate of convolutional neural networks [66]. min max min . (1)

Convolutional Neural Network (CNN)
In this work, we used a generic CNN architecture typically used in image analysis consisting of a sequence of two-dimensional convolutional layers followed by a fully connected layer with sigmoid activation. The same network architecture was used in each of the VDEPs, and the neural network was trained using the previously extracted epochs, given that the epochs do not consider the subject from which they were extracted, it could be considered that all the subjects were equally fed into the model. The training process of the model was performed by randomly selecting 90% for the training sample and 10% for testing in a non-exhaustive 10-fold cross-validation execution. The performance metrics for the area under the ROC curve (AUC), accuracy and area under the precisionrecall curve (PR-AUC) were computed for the validation set after each execution.
The input of the model re-arranged the EEG channels by proximity for improving the information of adjacent channels, which helps convolutional neural networks to learn more effectively [67]. The original ordering of the channels and the final order are shown in Figure 3, including a plot of input before and after this re-ordering. These samples can be represented as (128 × 32) matrices and can be used to obtain EEG characteristics such as the delta, theta, alpha, beta, and gamma channels that we used in our statistical analysis. In our case, we used a basic FFT (Fast-Fourier Transform) filtering function to extract these time-domain waveforms [65].
The second output was used to train artificial neural networks (ANN), so in the ETP we normalized the values of the samples from their original values to the [0, 1] interval by using Equation (1), as normalized inputs have proven an increase on the learning rate of convolutional neural networks [66]. . (1)

Convolutional Neural Network (CNN)
In this work, we used a generic CNN architecture typically used in image analysis consisting of a sequence of two-dimensional convolutional layers followed by a fully connected layer with sigmoid activation. The same network architecture was used in each of the VDEPs, and the neural network was trained using the previously extracted epochs, given that the epochs do not consider the subject from which they were extracted, it could be considered that all the subjects were equally fed into the model. The training process of the model was performed by randomly selecting 90% for the training sample and 10% for testing in a non-exhaustive 10-fold cross-validation execution. The performance metrics for the area under the ROC curve (AUC), accuracy and area under the precision-recall curve (PR-AUC) were computed for the validation set after each execution.
The input of the model re-arranged the EEG channels by proximity for improving the information of adjacent channels, which helps convolutional neural networks to learn more effectively [67]. The original ordering of the channels and the final order are shown in Figure 3, including a plot of input before and after this re-ordering.  Figure 3. Re-arrangement of the EEG channels for the ANN analysis.

Results
We tested the normality of the distribution of measurements of the mean pow measurements across all channels (δ, θ, α, β, and γ) using the Kolmogorov-Smirnov with the Lilliefors Significance Correction. Then, we conducted Mann-Whitney U Test examine the differences on each of the mean power measurements according to categories related to each of the VDEP criteria, e.g., the differences in the Alpha ba between the Symmetrical and Asymmetrical categories for the Balance VDEP. The Ma Whitney U test (also known as the Wilcoxon rank-sum test) was chosen to test the n hypothesis that there is a probability of 0.5 that a randomly drawn observation for group is larger than a randomly drawn observation from the other.
The Kolmogorov-Smirnov test proved that the distribution of measurements of mean δ, θ, α, β, and γ bands of the EEGs provided in the DEAP were not norm distributed, with p < 0.0001 in all cases, as shown in Table A1.

Balance
The mean power measurements across all channels were statistically significan higher in the Symmetrical category than the Asymmetrical category for the Balance, w p < 0.05 in all cases (Figure 4). The details of the ranks and statistical average of the m band measurements for the Balance are listed in Table A2. The full Test Statistics Mann-Whitney U Tests conducted on the Balance are listed in Table A7.

Results
We tested the normality of the distribution of measurements of the mean power measurements across all channels (δ, θ, α, β, and γ) using the Kolmogorov-Smirnov test with the Lilliefors Significance Correction. Then, we conducted Mann-Whitney U Tests to examine the differences on each of the mean power measurements according to the categories related to each of the VDEP criteria, e.g., the differences in the Alpha band between the Symmetrical and Asymmetrical categories for the Balance VDEP. The Mann-Whitney U test (also known as the Wilcoxon rank-sum test) was chosen to test the null hypothesis that there is a probability of 0.5 that a randomly drawn observation for one group is larger than a randomly drawn observation from the other.
The Kolmogorov-Smirnov test proved that the distribution of measurements of the mean δ, θ, α, β, and γ bands of the EEGs provided in the DEAP were not normally distributed, with p < 0.0001 in all cases, as shown in Table A1.

Balance
The mean power measurements across all channels were statistically significantly higher in the Symmetrical category than the Asymmetrical category for the Balance, with p < 0.05 in all cases (Figure 4). The details of the ranks and statistical average of the mean band measurements for the Balance are listed in Table A2. The full Test Statistics for Mann-Whitney U Tests conducted on the Balance are listed in Table A7.

Colour
There were no statistically significant differences in the mean power measurements across all channels (δ, θ, α, β, and γ) between the Warm and Cold categories for the Colour, with p > 0.05 in all cases ( Figure 5). The details of the ranks and statistical average of the mean power measurements for the Colour are listed in Table A3. The full Test Statistics for Mann-Whitney U Tests conducted on the Colour are listed in Table A8.

Light
All the mean power measurements across all channels (δ, θ, α, β, and γ) were statistically significantly higher in the Bright category than the Dark category for the Light, with p < 0.05 in all cases ( Figure 6). The details of the ranks and statistical average of the mean power measurements for the Light are listed in Table A4. The full Test Statistics for Mann-Whitney U Tests conducted on the Light are listed in Table A9.

Colour
There were no statistically significant differences in the mean power measurements across all channels (δ, θ, α, β, and γ) between the Warm and Cold categories for the Colour, with p > 0.05 in all cases ( Figure 5). The details of the ranks and statistical average of the mean power measurements for the Colour are listed in Table A3. The full Test Statistics for Mann-Whitney U Tests conducted on the Colour are listed in Table A8.

Colour
There were no statistically significant differences in the mean power measurements across all channels (δ, θ, α, β, and γ) between the Warm and Cold categories for the Colour, with p > 0.05 in all cases ( Figure 5). The details of the ranks and statistical average of the mean power measurements for the Colour are listed in Table A3. The full Test Statistics for Mann-Whitney U Tests conducted on the Colour are listed in Table A8.

Light
All the mean power measurements across all channels (δ, θ, α, β, and γ) were statistically significantly higher in the Bright category than the Dark category for the Light, with p < 0.05 in all cases ( Figure 6). The details of the ranks and statistical average of the mean power measurements for the Light are listed in Table A4. The full Test Statistics for Mann-Whitney U Tests conducted on the Light are listed in Table A9.

Light
All the mean power measurements across all channels (δ, θ, α, β, and γ) were statistically significantly higher in the Bright category than the Dark category for the Light, with p < 0.05 in all cases ( Figure 6). The details of the ranks and statistical average of the mean power measurements for the Light are listed in Table A4. The full Test Statistics for Mann-Whitney U Tests conducted on the Light are listed in Table A9.

Movement
All the mean power measurements across all channels (δ, θ, α, β, and γ) were statistically significantly higher in the Fast category than the Slow category for the Movement, with p < 0.05 in all cases (Figure 7). The details of the ranks and statistical average of the mean power measurements for the Movement are listed in Table A5. The full Test Statistics for Mann-Whitney U Tests conducted on the Movement are listed in Table A10.

Convolutional Neural Networks (CNN)
All the VDEP targets were accurately predicted from the EEG signal using a simple CNN classification model trained using a non-exhaustive 10-fold cross-validation approach. The performance metrics were obtained from each training and the averages were computed. It is important to notice that we trained 40 independent models, 10 for each VDEP, which shared the same base structure (see Table A6). The prediction of Movement was the most accurate among the studied VDEPs (AUC = 0.9698, Accuracy = 0.9675, PR-AUC = 0.9569). On the other hand, the corresponding trained model struggled to predict the Colour VDEP from the EEG signal input (AUC = 0.7584, Accuracy = 0.7447,

Movement
All the mean power measurements across all channels (δ, θ, α, β, and γ) were statistically significantly higher in the Fast category than the Slow category for the Movement, with p < 0.05 in all cases (Figure 7). The details of the ranks and statistical average of the mean power measurements for the Movement are listed in Table A5. The full Test Statistics for Mann-Whitney U Tests conducted on the Movement are listed in Table A10.

Movement
All the mean power measurements across all channels (δ, θ, α, β, and γ) were statistically significantly higher in the Fast category than the Slow category for the Movement, with p < 0.05 in all cases (Figure 7). The details of the ranks and statistical average of the mean power measurements for the Movement are listed in Table A5. The full Test Statistics for Mann-Whitney U Tests conducted on the Movement are listed in Table A10.

Convolutional Neural Networks (CNN)
All the VDEP targets were accurately predicted from the EEG signal using a simple CNN classification model trained using a non-exhaustive 10-fold cross-validation approach. The performance metrics were obtained from each training and the averages were computed. It is important to notice that we trained 40 independent models, 10 for each VDEP, which shared the same base structure (see Table A6). The prediction of Movement was the most accurate among the studied VDEPs (AUC = 0.9698, Accuracy = 0.9675, PR-AUC = 0.9569). On the other hand, the corresponding trained model struggled to predict the Colour VDEP from the EEG signal input (AUC = 0.7584, Accuracy = 0.7447, Figure 7. Violin plot of the average power in each of the bands for Slow and Fast categories for the Movement VDEP, ranging from low (δ delta) to high (γ gamma) frequencies.

Convolutional Neural Networks (CNN)
All the VDEP targets were accurately predicted from the EEG signal using a simple CNN classification model trained using a non-exhaustive 10-fold cross-validation approach. The performance metrics were obtained from each training and the averages were computed. It is important to notice that we trained 40 independent models, 10 for each VDEP, which shared the same base structure (see Table A6). The prediction of Movement was the most accurate among the studied VDEPs (AUC = 0.9698, Accuracy = 0.9675, PR-AUC = 0.9569). On the other hand, the corresponding trained model struggled to predict the Colour VDEP from the EEG signal input (AUC = 0.7584, Accuracy = 0.7447, PR-AUC = 0.6940). A summary of the classification performance for each VDEP is displayed in Table 2.

Discussion
The VDEPs are responsible for capturing our attention, persuading us, informing us, and engaging a link with the visual information represented. The answer to what makes a good design and how you can create visual materials that stand out is in the proper use of VDEPs [9].
Past research has mostly focused on the VDEPs capability of causing specific emotions and how to activate the desired response from the spectator [33,68,69]. A severe limitation of such research has been its inability to boost the classification accuracy of various visual stimuli that are inferred to VDEPs and their impact on human brain activity. Another limitation of previous research has been its inability to demonstrate which cues related to the VDEPs and EEG are most correlated with human brain activity, and the difficulty to reveal the VDEPs that are involved and are conditioning how visual content is perceived.
We sought to address these limitations to understand the relationships between multimedia content itself with users' physiological responses (EEG) to such content by analysing the VDEPs and their impact on human brain activity.
The results of this study suggest that variations in the light/value (Accuracy 0.90/Loss 0.23), rhythm/movement (Accuracy 0.95/Loss 0,13) and balance (Accuracy 0.81/Loss 0.76) VDEPs in the music video sequences produce a statistically significant effect over the δ, θ, α, β and γ EEG bands, and Colour is the VDEP that has produced the least variation in human brain activity (Accuracy 0.79/Loss 0.50). The CNN model successfully predicts the VDEP of a video fragment solely by the EEG signal of the viewer.
The violin plots confirm that the distributions are practically equal between classes of the same VDEP, and the fact that the p-values are significant comes from a large number of samples. We find significant differences in some bands for some VDEPs, but these are minor as the distributions show. However, this could be expected since we are simply analysing power in classical a priori defined bands. However, the CNN results are more remarkable. The fact that we obtain high ranking values with the CNN versus the similarity of the calculated powers in those bands suggests, for future work, that the relationship between VDEPs and EEG is more complex than simple power changes in predetermined frequency bands.
The results show how human brain activity is more susceptible to producing alterations to sudden changes in the visualization of movement, light, or balance in music video sequences than the impact that a colour variation can generate in human brain activity. Reddish tones transmit heat, just as blood is hot and fire burns. The bluish brushstrokes are associated with cold and lightness [70]. These are the colours with which the mind paints, effortlessly and from childhood, water, rain, or a bubble. Different studies support the results obtained on these associations that we can consider universal [68], being directly involved in the design of VDEPs. However, divergences appear when it comes to distinguishing between smooth and rough, male, or female, soft or rigid, aggressive, or calm, for example. Moreover, people disagree more, depending on the results, when judging whether the audio-visual content is interesting, and whether they like it or not.
Brain activity may be slightly altered by changes in the design of different VDEPs such as colour, brightness, or lighting. However, it may be more severely altered when one visualizes sudden changes in movement and course of action. The possible explanation for the results obtained is that the majority of the most basic processes of perception are part of the intrinsic neuronal architecture, the human being more accustomed to this type of modifications in the action that is visualized, and within this context being more susceptible to sudden and abrupt changes in the action that we observe, distorting and altering our brain activity, which has been demonstrated in various studies [69]. Most of the difficulties that arise in studies of this typology are the particularities of the individual. The same person, depending on its perceptual state, will observe the same scene differently, by existing a very wide range of experiences and individual variances [71][72][73][74][75].
Several limitations affect this study. It is important to notice that to the best of our knowledge, there is currently no established ontology for VDEPs. Although the identification of four VDEPs has been sufficient to find and determine the physiological relationships (EEG) and the VDEPs, it would be very interesting if we had a system to organize this knowledge and allow us to construct the architecture of neural networks focused and customized to specific VDEPs, which would permit us to improve accuracy and reduce loss, instead of working with generic networks.
Therefore, we consider that further research in this area should pursue the development of ontologies that let us structure the knowledge related to design, recognition, labelling, filtering, and classification of VDEPs for the improvement and optimization for their use in information systems, expert systems, and decision support systems. Moreover, this study should be extended to research about other VDEPs combinations on brain activity. Future research should also carry out investigations using multiple VDEPs in the EEG of each individual to study the relationships and interactions between different VDEPs. Furthermore, we recommend analysing the VDEPs separately in each EEG channel to understand how the synergies between various VDEPs affects visual perception and visual attention. Finally, future investigations could focus on exploring VDEPs through other models such as generative ones to visualize the topographical distribution of the EEG channels and each VDEP.

Conclusions
This study found evidence supporting that there is a physiological link between VDEPs and human brain activity. We found that the VDEPs expressed in music video sequences are related to statistically significant differences in the average power of classical EEG bands of the viewers. Furthermore, a CNN classifier tasked to identify the VDEP class from the EEG signals achieved accuracies of 90.44%, 79.67%, 81.25%, and 95.29% of accuracy for the Light, Colour, Balance, and Movement VDEPs, respectively. The results suggest that the relationship between the VDEPs and brain activity is more complex than simple changes in the EEG band power.

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.  In the left music video frame, a composition with a predominantly symmetrical balance can be observed, while in the right music video frame, a composition with a predominantly asymmetrical balance can be appreciated.  In the left music video frame, a composition with a predominantly symmetrical balance can be observed, while in the right music video frame, a composition with a predominantly asymmetrical balance can be appreciated.   brightness of an image or photograph. In this case were selected the following visual representation settings: Values in the perception of space/logarithmic histogram. The histogram shows how the pixels are distributed, the number of pixels (X-axis) in each of the 255 levels of illumination (Y-axis), from 0 at the lower left (absolute black) to 255 at the lower right (absolute white) of the image. Pixels with the same illumination level are stacked in columns on the horizontal axis. (C) The greyscale is used to establish comparatively both the brightness value of pure colours and the degree of clarity of the corresponding gradations of this pure colour.    Clip 22 (18)(19)(20)(21)(22), (24)(25)(26)(27)(28)(29)(30)(31)(32)(33), (55)(56)(57)(58)(59)(60)