text:
\section{Introduction}
\begin{figure}[t]
    \centering
    \includegraphics[width=0.9\linewidth]{figures/Figure1.png}
    \caption{The daily new infection cases of COVID-19 for four countries \cite{covid_google}. We observe multiple pandemic waves with varying starting days and magnitudes.}
    \label{four_country}
\end{figure}
The spreading mechanism of COVID-19 \cite{kissler2020projecting} is complicated due to its dependency on disease features and social factors such as human mobility \cite{yabe2020non,qian2021scaling}, public awareness\cite{qian2020modeling}, and intervention policies \cite{desvars2020structured}. One prominent phenomenon of the complex disease spreading process is the multiple outbreak wave, which implies the periodic rebound to a large number of infection cases \cite{kaxiras2020multiple,cacciapaglia2020second,leung2020first} and is obvious in many countries such as the USA, UK, France, and Japan (Figure \ref{four_country}). Abrupt and uncertain disease outbreaks disturb individuals' daily life, government's reopening policies \cite{chang2020mobility}, medical resources managements \cite{moghadas2020projecting}, and rick assessment \cite{ye2020alpha}. It has an enormous social impact to investigate and construct such an accurate model to predict the multiple waves by fully utilizing different types of social and mobility data \cite{xiao2021c}.

Many prediction models for the first outbreak wave have been proposed to anticipate the infection and death cases \cite{shahid2020predictions,chimmula2020time,kufel2020arima,schwabe2021predicting}. One critical input for these models is the mobility data \cite{kang2020multiscale,huang2020twitter,chang2021supporting}, which describes population movements and is positively related to the disease infections \cite{xiong2020mobile}. Nevertheless, continuous tracking of human mobility dynamics shows that the mobility strength did not exhibit consistent relationships with infection cases: (1) the USA: human mobility fluctuated around 95\% of the normal period from July 1 to Dec. 1, 2020 \cite{usa_covid} which witnessed the second wave in July and the third wave in November (Figure \ref{four_country}); (2) Tokyo: the social contact index resumed to the normal level and decreased slightly after July 2020, but Tokyo experienced the second wave since then \cite{yabe2020early}. The inconsistency in the mobility strength and disease infection necessitates other data that is more representative of disease outbreak waves.

During COVID-19, many text-based methods have been proposed to aid communities such as to understand human's emotion states \cite{ju2021dr} and to answer peoples' questions \cite{yan2021multilingual}. Web search records collected by the web service provider have extensive applications such as customer behavior analysis \cite{goel2010predicting}, stock market prediction \cite{zhong2019revisiting}, and disease outbreak monitoring \cite{bento2020evidence,hisada2020surveillance}. For COVID-19, symptom-related web search records (e.g., \textit{fever}, \textit{cough}, and \textit{headache}) reflect the public's virus-induced symptoms that cannot be mined from the mobility data. Evidence in Tokyo showed that the Pearson correlation between \textit{High-Risk Users} (which are defined from web search records) and infection cases was 0.719 with a 16-day lag for the second wave \cite{yabe2020early}. It also found people's symptom-related web search frequency was smaller during the second wave than the first wave, though the number of patients were significantly higher. These results inspire us to leverage the web search data and recover human awareness decay to predict the multiwave pandemic.

In this study, we propose a multiwave infection prediction approach, with the direct application as urban district-level disease outbreak early warning. District-level disease prediction has the following three requirements: (1) comprehensive data sources, such as people movement and social responses, should be included to contain various hints that are closely related to the disease spreading; (2) spatial and temporal disease dissemination patterns of COVID-19 should be taken into consideration; (3) it captures complex dependency between infection cases and other factors. To deal with these challenges, we first define the Web search-Mobility Network (WMN) whose nodes and edges maintain web search frequency and inter-district trip information, respectively. Afterward, we propose a Social Awareness-Based Graph Neural Network (SAB-GNN) architecture upon the WMN to capture the spatio-temporal infection case dynamics in different urban districts. We train and test the model using real-world infection, human mobility, and web search data in the Tokyo area from April 2020 to May 2021, and obtain better prediction performance than baseline models. Our method has three contributions:
\begin{itemize}
\item We focus on predicting multiwave disease outbreaks that are globally prevalent but are seldom investigated. Different from a single wave prediction, multiwave prediction enables the public agencies to evaluate the long-term risk and take appropriate actions at different pandemic stages.
\item We propose the SAB-GNN by fusing historical infection, mobility, and web search data that provide sufficient evidence of potential disease outbreaks. The spatial module, temporal module, and social awareness module take separate responsibilities and jointly contribute to the final prediction.
\item The proposed method is implemented on a mega-city, Tokyo, with a period spanning more than one year across four pandemic waves. We conduct a comprehensive analysis of disease outbreaks and prediction results from different models at different time intervals, which promotes a more nuanced understanding of the disease waves.
\end{itemize}

\section{Related Work}
\textbf{Time series models:}
Existing COVID-19 time series models cover multiple types such as auto-regressive integrated moving average (ARIMA) \cite{kufel2020arima}, and long short-term memory (LSTM) \cite{chimmula2020time,jo2020condlstm}. Moreover, biologists and engineering scientists focus on the relationship between fatality rate and biochemical indicators \cite{zhou2020not}, human mobility \cite{jo2020condlstm}. When it comes to urban district-level disease infection prediction, while the inter-district connections provide crucial pathways for both human movement and disease dissemination, these models are insufficient to capture such spatial dependency between different urban districts. In fact, the spatial dependency information enables us to deal with the data scarcity issue which may occur during the pandemic season. For instance, assume that we have sufficient mobility and social media data for district $\mathcal{A}$ and deficient data for district $\mathcal{B}$, and recognize strong mobility connections between districts $\mathcal{A}$ and $\mathcal{B}$. Considering the connections between the two districts helps to fully utilize the infection information and predict the infection cases in a global manner.

\textbf{Graph neural networks:} Graph neural network (GNN) is an innovative neural network that captures the relationship between multi-hop neighborhood nodes via the message passing mechanism \cite{zhou2020graph}. In the past years, various GNN models such as graph convolutional network (GCN) \cite{kipf2016semi}, GraphSage \cite{hamilton2017inductive}, graph attention network (GAT) \cite{velivckovic2017graph}, and relational graph convolutional network (R-GCN) \cite{schlichtkrull2018modeling} were developed and applied to many fields such as neural machine translation \cite{bastings2017graph}, visual question answering \cite{narasimhan2018out}, traffic prediction \cite{yao2019revisiting,zhang2021traffic,wang2020traffic,chen2019gated,peng2020spatial}, network metric generation \cite{xue2021quantifying}, and online diagnosis \cite{wang2021online}.

Researchers have harnessed the superiority of the GNN in modeling spatial dependency to perform the disease infection case prediction. Most published GNN approaches \cite{kapoor2020examining,panagopoulos2021transfer,gao2021stan} focused on the infection prediction before July 2020 when the first global outbreak occurred using the historical infection and mobility data. The mobility data was sufficient to reflect the disease spreading patterns during the first wave thanks to its simple relationship with the infection. First, areas attracting more passengers had higher risks of experiencing rapidly increasing cases than some lonely areas at the beginning of the first wave. Second, the travel restriction policies during the first wave suppressed the mobility strength and thus decelerated the disease outbreak \cite{xiong2020mobile}.

\begin{figure*}[ht]
    \centering
    \includegraphics[width=0.8\linewidth]{figures/Figure2.png}
    \caption{The framework of Social Awareness-Based Graph Neural Networks. We first model urban districts as nodes, and propagate the web search frequency embedding using the mobility information. Then, we use a learnable social awareness matrix to recover node representations, and feed them into an LSTM sequence to predict the next $D_{2}$ days' infection cases.}
    \label{figure2}
\end{figure*}

Nevertheless, the interaction between infection and mobility evolved into a much more complicated status during the later waves because the infection cases were affected by a large variety of causes such as mask policy, the vaccination, which hindered the ability of pure mobility data to reflect the infection tendency. In addition, many studies have recognized that the available mobility data for COVID-19 infection cases prediction was limited by the period length \cite{panagopoulos2021transfer,rodriguez2021steering}, which resulted in unstable learned models. In summary, the long-term multiwave infection prediction requests alternative data sources that provide sufficient interconnections with the infection case under the dynamic environment. In this study, we turn to the novel web search data, which directly reveals human's awareness to the disease and potential symptoms, to perform the multiwave disease prediction.

\section{Preliminaries}

In this section, we first describe the mobility and web search data used as features for the prediction, and then formally define our prediction task. Consider an urban area that is divided into several pre-defined urban districts. For the mobility data, we first collect location records (longitude, latitude, and time) of mobile phone users within the urban area and extract each individual's trajectory points as a sequence. Next, we project the sequence points of each mobile phone user into the urban districts to obtain the number of daily inter-district trips. Note that inter-district trips build natural pathways to transmit disease viruses in the city so that these districts have correlated exposure to disease outbreak risks. For the web search data, we scope $n_{w}$ COVID-19 symptom-related web search words from multiple official sources. We assign each mobile phone user to the urban district where his/her home is located and then aggregate the daily web searching frequency for all users living in this district. In summary, we obtain the number of inter-district trips between every two districts, and symptom-related web search frequency in each district for each day.

Our goal is to utilize the infection, mobility, and web search data for the past $D_{1}$ days to predict new infection cases for the next $D_{2}$ days at the urban district level. We define a sequence of WMNs: $G_{t}=(V,\mathbf{E}_{t},\mathbf{H}_{t},\mathbf{I}_{t})$. Here, the subscript $t$ implies a day. $V=\{v_{1},v_{2},...,v_{n}\}$ represents $n$ urban districts. $\mathbf{E}_{t}\in \mathbb{R}^{n \times n}$ is a matrix where the ($i,j$)-entry represents the number of trips from the district $v_{i}$ to $v_{j}$ on the day $t$. We denote the daily web search frequency of residents living in the district $v_{i}$ as a vector $\mathbf{h}_{t}^{v_{i}} \in \mathbb{R}^{n_{w}}$ so that $\mathbf{H}_{t} = [\mathbf{h}_{t}^{v_{1}},\mathbf{h}_{t}^{v_{2}},...,\mathbf{h}_{t}^{v_{n}}]^{T}\in \mathbb{R}^{n \times n_{w}}$ is the node feature matrix encoding the web search frequency for all districts. $\mathbf{I}_{t}\in \mathbb{R}^{n \times 1}$ is the collection of daily new infection cases in different districts.

\textbf{Prediction task}: On the day $T$, given $G_{t}$ for all $t \in [T-D_{1}+1, T]$, predict $\mathbf{I}_{t}$ for all $t \in [T+1, T+D_{2}]$.

\section{Social Awareness-Based Graph Neural Networks}

In this section, we first present the intuition of the SAB-GNN model, and then sequentially introduce its three modules: the spatial information propagation module, the social awareness recovery module, and the temporal information passing module, and finally declare the loss function.

We deduce that the future reported district-level infection cases are jointly influenced by the existing infection cases, the number of susceptible individuals that may have been infected (which can be mined from public's symptom-related web search data), and in-person contact patterns across the city (which is reflected by the inter-district trip numbers) and use them as features. Given that the pandemic spreading is indeed a complicated temporal process embedding on the space, we propose to build temporal and spatial modules \cite{yao2018deep} to track the infection dynamics. Lastly, the existing study finds that the public's symptom-related web search frequency decreases from the first wave to the second wave, which reveals a prevalent social phenomenon that people's awareness of a hot topic gradually declines (which is referred to as \textit{social awareness decay} in this study) \cite{yabe2020early}. Based on the fact that people adapt themselves to the mask policies, travel restrictions, and routine testings and pay less attention to the COVID-19, we propose a social awareness recovery module in the SAB-GNN to estimate the actual occurrence of COVID-19-related symptoms. Taken together, we build an integrated future infection case prediction model with three modules (i.e., spatial module, awareness recovery module, and temporal module) by fusing the historical infection, mobility, and web search data.

\subsection{Spatial Module: Graph Neural Networks}
Recall that the web search frequency vector $\mathbf{h}_{t}^{v_{i}}$ reflects the number of potential infected individuals in the urban district $v_{i}$ and $\mathbf{E}_{t}$ records the daily inter-district trips. We therefore perform the convolution operation on $\mathbf{h}_{t}^{v_{i}}$ using the $\mathbf{E}_{t}$ information under the graph neural network framework to capture the disease risk propagation properties. As shown in Figure \ref{figure2}, using the symptom-related web search frequency in each urban district, we employ the one-hot encoding to initialize the representation for each urban district (i.e., each node in $G_{t}$) as the input matrix:  $\mathbf{X}_{t}^{(0)} = \mathbf{H}_{t}$. Following the GCN model \cite{kipf2016semi}, we define the node representation propagation rule between the layers $k$ and $(k+1)$ as:
\begin{equation}
    \mathbf{X}_{t}^{(k+1)} = \sigma(\tilde{\mathbf{D}_{t}}^{-\frac{1}{2}}\tilde{\mathbf{E}_{t}}\tilde{\mathbf{D}_{t}}^{-\frac{1}{2}}\mathbf{X}_{t}^{(k)}\mathbf{W}^{(k)}),
    \label{equ1}
\end{equation}
where
\begin{equation}
    \tilde{\mathbf{E}_{t}} = \mathbf{E}_{t} + \mathbf{I}_{n \times n}, \tilde{\mathbf{D}_{ii}} = \sum_{j=1}^{n}\tilde{\mathbf{E}_{ij}},
    \label{equ2}
\end{equation}
and $\mathbf{W}^{(k)}$ is a learnable weight matrix, $\mathbf{I}_{n \times n}$ is the $n$ by $n$ identity matrix, $\sigma(\cdot)$ is the activation function ReLU.

Note that we normalize the matrix $\mathbf{E}_{t}$ such that the sum of each column is equal to 1 (i.e., the sum of incoming edges on one node is 1), which is used in the existing study \cite{panagopoulos2021transfer}. In practice, it is possible to replace the spectral convolution with other GNN variants such as GAT, GraphSage. We also implement them and find quite approximate prediction performance as the GCN. The outcome of the spatial module is a matrix
\begin{equation}
    \mathbf{H}_{t}^{S}=\mathbf{X}_{t}^{(K)}=[\mathbf{x}_{t}^{v_{1}},\mathbf{x}_{t}^{v_{2}},...,\mathbf{x}_{t}^{v_{n}}]^{T},
\end{equation}
with $n$ rows that encode the web search frequency and mobility where $K$ is the number of layers.

\subsection{Social Awareness Recovery Module}
The symptom-related web search frequency is positively related to the number of actual symptom occurrences among the population, as well as the probability that a user with a symptom searches the word on the Internet. As mentioned earlier, the social awareness decay effect informs that probability of symptom-related word searching gradually decreases with time after the first COVID-19 wave. To estimate the actual symptom occurrences, we propose to multiply the observed web search representation by a monotonically increasing function with respect to the time.

Specifically, we first linearly normalize each entry of the web search record vector $\mathbf{h}_{t}^{v_i}$ to 0 and 1 by the maximal and minimal web search frequency of each word across all days in the urban district $v_{i}$, and feed them into the spatial module, and obtain $\mathbf{H}_{t}^{S}$. Next, we define a simple increasing function $r(t|i,t_{0})=e^{\lambda_{i}^{2}(t-t_{0})}$ regarding $t$ to recover the social awareness:
\begin{equation}
   \tilde{\mathbf{x}}_{t}^{v_{i}} = \mathbf{x}_{t}^{v_{i}}r(t|i,t_{0}) =  \mathbf{x}_{t}^{v_{i}}e^{\lambda_{i}^{2}(t-t_{0})},
   \label{equ3}
\end{equation}
where $\lambda_{i}^{2}$ is learnable and measures the social awareness recovery rate in $v_{i}$. $t$, $t_{0}$ represent the current day and the first day of the study period, respectively. Given that the land use, economy type, demographic characteristic discrepancy may lead to spatially varying social awareness rates, we specify district-dependent $\lambda_{i}^{2}$ to encode its unique awareness decay behavior. A large value of $\lambda_{i}^{2}$ implies that social awareness of COVID-19 for residents living in $v_{i}$ declines rapidly and we therefore adopt this large value to recover the social awareness. Note that we introduce the square in $\lambda_{i}^{2}$ to ensure that it is non-negative. Collectively speaking, the social awareness recovery module transforms $\mathbf{H}_{t}^{S}$ to $\tilde{\mathbf{H}}_{t}^{S}$ by
\begin{equation}
    \tilde{\mathbf{H}}_{t}^{S} = \mathbf{H}_{t}^{S}\circ \mathbf{M}_{t,t_{0}},
    \label{equ4}
\end{equation}
where $\mathbf{M}_{t,t_{0}}$ is the awareness recovery matrix (ARM):
\begin{equation}
    \mathbf{M}_{t,t_{0}} =
    \begin{bmatrix}
        e^{\lambda_{1}^{2}(t-t_{0})} & e^{\lambda_{1}^{2}(t-t_{0})}& ... & e^{\lambda_{1}^{2}(t-t_{0})} \\
        e^{\lambda_{2}^{2}(t-t_{0})} & e^{\lambda_{2}^{2}(t-t_{0})}& ... & e^{\lambda_{2}^{2}(t-t_{0})} \\
        \vdots & \vdots  & \ddots & \vdots\\
        e^{\lambda_{n}^{2}(t-t_{0})} & e^{\lambda_{n}^{2}(t-t_{0})}& ... & e^{\lambda_{n}^{2}(t-t_{0})} \\
    \end{bmatrix},
    \label{equ5}
\end{equation}
and $\circ$ is the Hadamard product. This transform implies that the entries in $\mathbf{H}_{t}^{S}$ are amplified to capture the actual disease risk which is underestimated due to the social awareness decay effect when $t$ is large. In the end, we perform 0-1 normalization on the infection matrix $\mathbf{I}_{t}$ to obtain $\tilde{\mathbf{I}}_{t}$, and arrive at the output of the social awareness recovery module by concatenating the web search and infection representations, which is
\begin{equation}
    \mathbf{H}_{t}^{A}=[\tilde{\mathbf{H}}_{t}^{S},\tilde{\mathbf{I}}_{t}].
    \label{equ6}
\end{equation}

\subsection{Temporal Module: Long Short-Term Memory}
To capture the temporal dependency of district-level features and infection cases, we adopt the existing LSTM model \cite{hochreiter1997long}. $\forall i \in {1,2,...,n}$, we extract the $i$-th row of matrices $\mathbf{H}_{t}^{A}, t\in [T-D_{1}+1,T]$ and feed them into an LSTM sequence (Figure \ref{figure2}). Since we have already modelled the spatial dependency in the spatial module, in the temporal module we pass the node representations from $\mathbf{H}_{t}^{A}$ separately for different nodes, and these LSTM sequences share the identical structures and parameters.

\begin{algorithm}[h]
\begin{algorithmic}[1]
    \Require
    $\{G_{t}, T-D_{1}+1 \leq t \leq T; I_{t}, T+1 \leq t \leq T+D_{2}\}_{T\in T_{train}}$, where $G_{t} = (V,\mathbf{E}_{t},\mathbf{H}_{t},\mathbf{I}_{t})$.
    \Ensure the SAB-GNN model.

    \For {each epoch}
        \For {each batch}
           \State $\mathcal{L}_{batch} \leftarrow 0$, $m=0$;
           \For {$T$ in this batch}
             \State $m$ $\leftarrow m+1$;
             \State \textbf{/* Spatial module */}
             \State Evaluate $\mathbf{X}_{t}^{(k+1)}$ using Eq. (\ref{equ1}), $T-D_{1}+1 \leq t \leq T$;
             \State Generate representations
             \State $\mathbf{H}_{t}^{S}=\mathbf{X}_{t}^{(K)}$, $T-D_{1}+1 \leq t \leq T$;
             \State \textbf{/* Social awareness recovery module */}
             \State  Update $\mathbf{H}_{t}^{S}$ with $\mathbf{M}_{t,t_{0}}$;
             \State Compute $\mathbf{H}_{t}^{A}$ using Eq. (\ref{equ6});
             \State \textbf{/* Temporal module */}
             \State Feed each row $i$ of $\mathbf{H}_{t}^{A}$ into an LSTM model and
             \State obtain the hidden states for $t=T+1,T+2,...,T+D_{2}$;
             \State Use a one-layer perceptron to transform
             \State these hidden states to $D_{2}$ predicted cases $
             \hat{I}_{t,i}$;
             \State Compute the loss $\mathcal{L}_{T}$  using Eq. (\ref{equ7});
             \State $\mathcal{L}_{batch} \leftarrow \frac{m-1}{m}\mathcal{L}_{batch}  + \frac{1}{m}\mathcal{L}_{T}$;
           \EndFor
           \State Use Adam to update the model.
        \EndFor
    \EndFor
    \end{algorithmic}
    \caption{Training algorithm of SAB-GNN}
    \label{algo:sab_gnn}
\end{algorithm}

\subsection{Loss Function}
Recall that our objective is to perform the infection case prediction for the next $D_{2}$ days, we define the loss function as the mean squared error, which is:
\begin{equation}
    \mathcal{L}_{T} = \frac{1}{D_{2}n}\sum_{t=T+1}^{T+D_{2}}\sum_{i=1}^{n}(I_{t,i}-\hat{I}_{t,i})^{2},
    \label{equ7}
\end{equation}
where $I_{t,i}$ and $\hat{I}_{t,i}$ denote the actual and predicted infection cases for the district $i$ on the day $t$. We present the training process as Algorithm 1.

\section{Experiments}
\subsection{Mobility, Web Search, and Symptom Data}
We utilize four datasets: infection cases, mobile phone location data, web search data, and symptom data in Tokyo from Jan. 6, 2020, to May 15, 2021. The mobility and web search data were owned by an \_ANONYMOUS\_COMPANY\_ with strict privacy protection regulations.
The descriptions of these data are as follows:

\begin{itemize}
    \item Infection data. We access the daily new infection cases for 23 wards of Tokyo from the Tokyo COVID-19 Task Force website \cite{infection} which is maintained by Tokyo metropolitan government.
    \item Mobility data. Table \ref{table_11} presents a sample piece of raw mobility data. The mobility data consists of a fake id (after privacy protection operations), the user's geolocation in terms of longitude and latitude at a specific time. The data collection frequency was around half an hour on average, which depends on the movements of the user: the frequency is higher if the user moves faster. In this study, the mobility data is utilized to identify Tokyo residents' home locations and mine the Mobility Matrix $\mathbf{E}_{t}$.

    \item Web search data. Table \ref{table_22} presents a sample piece of raw web search records. Each record contains the user ID, web query words, as well as web search time. The web search data is used to get COVID-19 symptom-related web search counts. Note that the \_ANONYMOUS\_COMPANY\_ maintains these data to promote users' experiences and does not share any individual-level data with other agencies.

    \item Symptom data. Based on two existing COVID-19 symptom studies \cite{sarker2020self,wang2021covid} and the COVID-19 symptom list released by the Centers for Disease Control and Prevention \cite{cdc}, we finally specify 44 symptoms, which are used to measure the counts of symptom-related web search records. The full list of these symptoms is shown in Table \ref{table_33}. Note that we translate these symptoms from English to Japanese, and then count the number of corresponding web searches.
\end{itemize}

\begin{table}[H]
\centering
\begin{tabular}{|c|c|}
\hline
\textbf{Location data}   & Sample       \\ \hline
\textbf{ID1}             & thisisfakeid \\ \hline
\textbf{Lat}             & 35.683       \\ \hline
\textbf{Lon}             & 139.763      \\ \hline
\textbf{Time (unixtime)} & 1584390800   \\ \hline
\textbf{Date}            & 20200317     \\ \hline
\end{tabular}
\caption{\label{table_11}A sample of mobility data.}
\end{table}

\begin{table}[H]
\centering
\begin{tabular}{|c|c|}
\hline
\textbf{Web search data} & Sample              \\ \hline
\textbf{ID1}             & thisisfakeid        \\ \hline
\textbf{Time}            & 2020-03-17 08:32:14 \\ \hline
\textbf{Query}           & XXX                 \\ \hline
\end{tabular}
\caption{\label{table_22}A sample of web search data.}
\end{table}

\begin{table}[h]
\begin{tabular}{|c|c|c|c|}
\hline
\textbf{} & \textbf{Symptom} & \textbf{} & \textbf{Symptom}        \\ \hline
1              & Abdominal pain             & 23             & Hot flush                         \\ \hline
2              & Ageusia                    & 24             & Hyperhidrosis                     \\ \hline
3              & Anosmia                    & 25             & Insomnia                          \\ \hline
4              & Anxiety                    & 26             & Lethargic                         \\ \hline
5              & Arthralgia                 & 27             & Loss of appetite                  \\ \hline
6              & Body ache                  & 28             & Mental health symptoms            \\ \hline
7              & Chest pain                 & 29             & Migraine                          \\ \hline
8              & Chest tightness            & 30             & Nasal dryness                     \\ \hline
9              & Chills                     & 31             & Nausea                            \\ \hline
10             & Confusion                  & 32             & Oropharyngeal pain                \\ \hline
11             & Cough                      & 33             & Pain                              \\ \hline
12             & Dehydration                & 34             & Palpitation                       \\ \hline
13             & Diarrhea                   & 35             & Pyrexia                           \\ \hline
14             & Disorientation             & 36             & Rash                              \\ \hline
15             & Dizziness                  & 37             & Rhinorrhea                        \\ \hline
16             & Dyspnea                    & 38             & Sinusitis                         \\ \hline
17             & Ear infection              & 39             & Sleep disturbance                 \\ \hline
18             & Ear pain                   & 40             & Sneezing                          \\ \hline
19             & Eye infection              & 41             & Stress                            \\ \hline
20             & Eye pain                   & 42             & Sweating                          \\ \hline
21             & Fatigue                    & 43             & URTI \\ \hline
22             & Headache                   & 44             & Vomiting                          \\ \hline
\end{tabular}
\caption{\label{table_33}Specified COVID-19 related symptoms.}
\end{table}

\subsection{Data Preprocessing}

\begin{figure}[h]
    \centering
    \includegraphics[width=0.8\linewidth]{figures/Figure1_supp.png}
    \caption{Data preprocessing framework to obtain inputs for machine-learning models.}
    \label{figure11}
\end{figure}

Recall that our prediction task requires the standard mobility matrix $\mathbf{E}_{t}$ as well as the web search matrix $\mathbf{H}_{t}$ on the day $t$, we design a data preprocessing framework to convert the raw mobility and web search data to $\mathbf{E}_{t}$ and $\mathbf{H}_{t}$ (Figure \ref{figure11}). We notice from the raw mobility data that a proportion of user IDs only appear a few times, which indicates that they might be contemporary travelers in Tokyo and may not be strongly related to the inter-city disease spreading in several months, and thus exclude their mobility data in this study. We finally identify 551,745 permanent users in the 23 special wards of Tokyo, which take around 6\% of the total population in these wards. Our mobility and web search data preprocessing procedure is as follows:
\begin{itemize}
    \item We apply the Mean shift algorithm \cite{cheng1995mean} to each permanent users' location points during the night hours (from 6:00 PM to 9:00 AM) from Jan. 6 to 31, 2020 to estimate the longitude and latitude of their homes using Java (step (1) in Figure \ref{figure11}).
    \item We extract the daily trajectories of these permanent residents using Java (step (2) in Figure \ref{figure11}) and then project the location points along the trajectories to the urban districts using the spatial join function in $\mathsf{geopandas}$ package in Python and obtain the mobility matrix $\mathbf{E}_{t}$ (step (3) in Figure \ref{figure11}). Note that we mine the daily location trajectory of each user ID and identify cross-district trips with the duration of at least 10 minutes (these trips are referred to as valid trips).
    \item Based on the specified 44 COVID-19 symptoms, we count the number of symptom searches for each permanent resident using Java (step (4) in Figure \ref{figure11}), aggregate them by urban districts (step (5) in Figure \ref{figure11}), and arrive at the web search matrix $\mathbf{H}_{t}$.
\end{itemize}

The statistics of these data are shown in Figure \ref{figure3}.
\begin{figure}[h]
    \centering
    \includegraphics[width=1.0\linewidth]{figures/Figure3.png}
    \caption{Statistics of mobility, web search frequency, and infection cases. (a) Dynamics of daily new cases and symptom-related web search frequency per user. (b) The distribution of web search frequency per user on three days. (c) Dynamics of daily new cases and inter-district trip number per user. (d) The number of inter-district trips on three days.}
    \label{figure3}
\end{figure}

\begin{table*}[ht]
\centering
\begin{tabular}{|c|c|cc|cc|cc|}
\hline
\multirow{2}{*}{\textcolor{black}{\textbf{The Third Wave}}}        & \multirow{2}{*}{Features}  & \multicolumn{2}{c|}{($D_{1}$, $D_{2}$) = (21, 7)} & \multicolumn{2}{c|}{($D_{1}, D_{2}$) = (21, 14)} & \multicolumn{2}{c|}{($D_{1}, D_{2}$) = (21, 21)}  \\ \cline{3-8}
                    &           & \multicolumn{1}{c|}{MAE}    & RMSE   & \multicolumn{1}{c|}{MAE}    & RMSE   & \multicolumn{1}{c|}{MAE}    & RMSE                              \\ \hline

HA (past all days)            & I    & 22.63                        & 26.45     & 22.53                           & 27.22        & 22.16                           & 27.65                             \\
HA (past $X$ days)  & I          & 14.15                           & 17.02     & 16.92                           & 20.72        & 19.14                           & 23.81                             \\
LSTM    & I             & 19.67                              & 24.07        & 17.23                             & 21.93      & 20.61                             & 26.61                                  \\
LSTM   & I+W        & 20.61                           & 24.89       & 20.72                           &25.91       & 19.93                             & 25.94                        \\
Seq2seq    & I          &  17.27                             & 22.24        &23.04                             & 28.73      & 16.79                             & 22.77                               \\
Seq2seq & I+W   & 15.69                           &19.83       & 21.76                           & 27.46         & 19.66                             & 25.69                                 \\
ST-GNN   & I+W+M           & 20.20                             & 25.86       & 21.53                           & 27.32       & 20.24                           & 26.36                  \\
MPNN+LSTM    & I+W+M              &  12.40                            & 17.31        & 16.30                          & 21.73        & 19.78                             & 25.59                \\ \hline
SAB-GNN-wsa    & I+W+M           & 10.75                           & 13.43       & 12.72                          & 16.33       & 15.61                           & 20.10                  \\
SAB-GNN  & I+W+M    & \textbf{8.03}                              & \textbf{10.43}                                 & \textbf{11.23}       & \textbf{14.78}          & \textbf{13.76}                     & \textbf{18.24}                                      \\ \hline
\multirow{2}{*}{\textcolor{black}{\textbf{The Fourth Wave}}}     & \multirow{2}{*}{Features}    &  \multicolumn{2}{c|}{($D_{1}, D_{2}$) = (21, 7)} & \multicolumn{2}{c|}{($D_{1}, D_{2}$) = (21, 14)} & \multicolumn{2}{c|}{($D_{1}, D_{2}$) = (21, 21)}  \\ \cline{3-8}
                               & & \multicolumn{1}{c|}{MAE}    & RMSE   & \multicolumn{1}{c|}{MAE}    & RMSE   & \multicolumn{1}{c|}{MAE}    & RMSE                              \\ \hline

HA (past all days)     &  I     & 8.76                           & 10.04     & 9.14                           & 10.59        & 9.41                           & 11.08                             \\
HA (past $X$ days)     & I      & 4.52                           & 5.76     & 5.52                           & 7.02        & 6.47                           & 8.23                             \\
LSTM & I        & 5.94                              & 7.67        & 8.20                             &  10.64      & 8.32                             & 10.95                                  \\
LSTM & I+W       & 7.43                           & 9.79       & 8.28                            &10.55       & 8.11                             & 10.63                        \\
Seq2seq & I        &  6.37                             & 8.38        &8.84                             & 11.42      & 8.45                            & 11.08                                \\
Seq2seq & I+W & 5.96                           &7.74       & 7.93                           & 9.86         & 10.43                             & 12.94                                 \\
ST-GNN    & I+W+M         & 5.46                             & 6.95       & 7.63                            & 9.97       & 10.54                           & 13.96                  \\
MPNN+LSTM    &I+W+M              &  3.34                            & 4.49        & 5.27                          & 7.97        & \textbf{4.80}                             & \textbf{6.56}                \\ \hline
SAB-GNN-wsa   &I+W+M           & 3.32                             & 4.46       & 4.63                            & 6.18       & 5.03                           & 6.77                  \\
SAB-GNN  &I+W+M    & \textbf{3.25}                             & \textbf{4.24}                                   & \textbf{4.28}       & \textbf{5.57}          & 5.25                     & 6.82                                      \\ \hline
\end{tabular}
\caption{\label{table1}Performance evaluation using past $D_{1}$ days' features to predict next $D_{2}$ days' infection ($I$: infection data; $W$: web search data; $M$: inter-district mobility data; SAB-GNN-wsa: the SAB-GNN model without the social awareness recovery).}
\end{table*}

\subsection{Experiment Setting, Evaluation Metrics, Baselines}
We conduct experiments to predict two outbreak waves, i.e., \textbf{the third wave} (from Dec. 10, 2020 to  Feb. 7, 2021) and \textbf{the fourth wave} (from March 17 to May 15, 2021). The period for each experiment covers 10 months of observations where the train/validation/test ratio is 70\%/10\%/20\%. We use the first 8 months (months 1 to 8) as training and validation data, and the last 2 months (months 9 and 10) as testing data. Note that the validation data has the size of 1 month and is evenly distributed from months 7 to 8. Since most existing disease prediction studies \cite{panagopoulos2021transfer,rodriguez2021steering,wang2020examining} predict the infection at the week-level, we design three scenarios: $(D_{1},D_{2})=(21,7),(21,14),(21,21)$ (recall that we use the past $D_{1}$ days' features to predict the future $D_{2}$ days' infection cases).

We train the model using PyTorch and Adam optimizer \cite{kingma2014adam}. We use the validation set to determine the learning rate as 0.0001, the epoch number, the batch size, and the dropout rate as 100, 8, and 0.50, respectively. The experiments run on an Intel Xeon w-2155 3.3 GHz CPU and 32 GB of RAM. Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) are utilized to evaluate the prediction performance: RMSE = $\sqrt{\frac{1}{D_{2}n}\sum_{t=T+1}^{T+D_{2}}\sum_{i=1}^{n}(I_{t,i}-\hat{I}_{t,i})^{2}}$,
MAE = $\frac{1}{D_{2}n}\sum_{t=T+1}^{T+D_{2}}\sum_{i=1}^{n}\vert I_{t,i}-\hat{I}_{t,i} \vert$.

To benchmark our model, we also implement eight existing prediction models: (1) Historical average (HA) infection cases until the day $T$; (2) Historical average of the last $D_{1}$ days; (3,4) LSTM; (5,6) Seq2seq: encode the input infection case and decode the sequence using two separate LSTMs \cite{cho2014learning}; (7) ST-GNN: an Spatio-Temporal GNN model whose inputs are past infection cases and mobility patterns \cite{kapoor2020examining}; (8) MPNN+LSTM: a message passing neural network with the LSTM \cite{panagopoulos2021transfer}.

\subsection{Comparisons with Baseline Models}

We present the prediction performance of the SAB-GNN and other baseline models for the third wave and the fourth wave (Table \ref{table1}). Our proposed SAB-GNN and SAB-GNN-wsa (\textbf{w}ithout \textbf{s}ocial \textbf{a}wareness recovery) models yield the smallest MAE and RMSE for most scenarios (5 out 6), which demonstrates our models capture the relationships between past infection, web search, mobility, and future infection. Comparing SAB-GNN and SAB-GNN-wsa, the social awareness recovery mechanism decreases the prediction error for most cases (5 out of 6), and the web search frequency has higher predictability after the awareness recovery operation.
\begin{figure}[h]
    \centering
    \includegraphics[width=1.0\linewidth]{figures/cross_1.png}
    \caption{Prediction performance of the SAB-GNN. (a,b) The relative prediction error (i.e., $|I_{real}-I_{predict}|/|I_{real}|$) during the third wave (a) and the fourth wave (b).
    (c) Predicted cases and real cases for the fourth wave when $(D_{1},D_{2})=(21,7)$.}
    \label{cross_1}
\end{figure}

\begin{figure*}[ht]
    \centering
    \includegraphics[width=1.0\linewidth]{figures/cross_2.png}
    \caption{Predicted and real cases on different urban districts of SAB-GNN for the fourth wave when $(D_{1},D_{2})=(21,7)$.}
    \label{cross_2}
\end{figure*}
In addition, results from all models reveal a general tendency that the web search and mobility features result in prediction performance improvement, even if the past infection case is already a strong feature to connect with the future infection case. Finally, we observe from the LSTM and Seq2seq that simply concatenating the web search frequency embedding with the past infection cases is unable to consistently boost the prediction, which affirms the essentiality of designing a suitable model architecture to utilize the web search data. For each model in Table 1, the running time for each scenario is within 20 minutes on the Ubuntu system with 32 GB RAM and 3.3 GHz Xeon w-2155 CPU.

\subsection{Prediction Results for Different Urban Districts}
We visualize the prediction errors and compare the predicted cases with the actual cases for each urban district in Figures \ref{cross_1}, \ref{cross_2}. As shown in Figures \ref{cross_1}ab, the relative prediction errors for most districts are homogeneously lower than 0.50 except for the central district (i.e., Chiyoda). This reveals that SAB-GNN mines the relationships between features and future infection cases in a global manner thanks to the message passing mechanism among neighborhood nodes. The reason for the relatively large prediction error in Chiyoda (i.e., District 1) is because Chiyoda is where Imperial Palace locates and has quite a few real infection cases (Figure \ref{cross_2}). Figure \ref{cross_1}c displays that the predicted daily case curve is able to capture the increasing tendency of actual cases from day 240 to day 280 and also maintains a small prediction error, which further demonstrates the power of our proposed SAB-GNN.

Figure \ref{cross_2} exhibits that for most urban districts, the model is able to anticipate the general increasing tendency of infection cases during the fourth wave. This information is especially valuable for local public agencies to make timely preparations at the beginning of a wave. Note that the prediction of District 4 (i.e., Shinjuku) is not as accurate as other districts. One potential interpolation is that Shinjuku is a commercial area with many entertainment industries and thus does not have similar infection patterns as other districts.

\subsection{Parameter Sensitivity}
The influences of model parameters in the SAB-GNN on the prediction performance are shown in Figure \ref{figure5}. Figure \ref{figure5}a displays the changes of RMSE for the fourth peak prediction with respect to varying numbers of symptom-related web search features used in the model. Note that there is a huge frequency discrepancy between different symptoms and we only feed the most frequent $k$ symptoms into our model. The result suggests that $k=8$ provides the best prediction performance. The potential interpretation is that: when $k$ is small, more symptoms contribute stronger connections with future infection cases; when $k$ is large, many weakly-related symptoms bring noises to the training and thus harm the prediction performance.

Next, we tune the numbers of layers $L_{1}, L_{2}$ in the GNN and LSTM modules (Figures \ref{figure5}bc) and conclude that $(L_{1},L_{2})=(1,2)$ yields the best prediction. It is reasonable that the simple model with a small number of parameters is preferred in this infection case prediction task whose training data is quite limited \cite{qiu2021miniseg}. Finally, we test the model in Figure \ref{figure5}d by varying $D_{1}$ and $D_{2}$ and observe the overall tendency that using more days' (i.e., $D_{1}$ is large) information to predict later fewer days' (i.e., $D_{2}$ is small) results in higher accuracy, which is consistent with the finding from the existing study \cite{panagopoulos2021transfer}.

\subsection{Ablation Study}
To confirm the effect of each module in the SAB-GNN on prediction results, we perform the ablation experiments for both the third wave and fourth wave (Figure \ref{figure6}). We remove one of the three modules in the SAB-GNN, and perform the prediction for the same time horizons as the full SAB-GNN model. We show quantitatively that the SAB-GNN model obtains lower prediction RMSE than other incomplete models, especially for the third wave, which reveals that both the spatial module, temporal module, and social awareness module contribute to the final prediction results in a positive manner. While existing studies have paid much attention to the spatial and temporal relationships between features and infection cases, we recommend introducing suitable social knowledge into the prediction, given the positive effect of the social awareness recovery mechanism.

\begin{figure}[t]
    \centering
    \includegraphics[width=1.0\linewidth]{figures/Figure6.png}
    \caption{Parameter sensitivity study.}
    \label{figure5}
\end{figure}
\begin{figure}[t]
    \centering
    \includegraphics[width=1.0\linewidth]{figures/Figure5.png}
    \caption{Ablation experiments. We implement the algorithms of SAB-GCN, SAB-GCN-wsa (without social awareness), SAB-GCN-wt (without temporal module), SAB-GCN-ws (without spatial module) for the third wave (the blue bar) and the fourth wave (the orange bar).}
    \label{figure6}
\end{figure}

\section{Conclusions}
Motivated by the multiwave outbreak of COVID-19 across the globe, we establish the SAB-GNN to predict future infection cases. Besides the historical infection and mobility data used in existing studies, our approach utilizes the novel symptom-related web search data which provides alternative evidence of future waves. More importantly, we consider the social awareness decay effect and propose the social awareness recovery module to estimate the actual infection risks. Experiments on the third and fourth peaks of Tokyo affirm that the SAB-GNN outperforms other baseline models and captures the increasing trend of pandemic waves. Our method is applicable to many countries given the wide coverage of web search data.

\bibliographystyle{ACM-Reference-Format}

\section{Introduction}
ACM's consolidated article template, introduced in 2017, provides a
consistent \LaTeX\ style for use across ACM publications, and
incorporates accessibility and metadata-extraction functionality
necessary for future Digital Library endeavors. Numerous ACM and
SIG-specific \LaTeX\ templates have been examined, and their unique
features incorporated into this single new template.

If you are new to publishing with ACM, this document is a valuable
guide to the process of preparing your work for publication. If you
have published with ACM before, this document provides insight and
instruction into more recent changes to the article template.

The ``\verb|acmart|'' document class can be used to prepare articles
for any ACM publication --- conference or journal, and for any stage
of publication, from review to final ``camera-ready'' copy, to the
author's own version, with {\itshape very} few changes to the source.

\section{Template Overview}
As noted in the introduction, the ``\verb|acmart|'' document class can
be used to prepare many different kinds of documentation --- a
double-blind initial submission of a full-length technical paper, a
two-page SIGGRAPH Emerging Technologies abstract, a ``camera-ready''
journal article, a SIGCHI Extended Abstract, and more --- all by
selecting the appropriate {\itshape template style} and {\itshape
  template parameters}.

This document will explain the major features of the document
class. For further information, the {\itshape \LaTeX\ User's Guide} is
available from
\url{https://www.acm.org/publications/proceedings-template}.

\subsection{Template Styles}

The primary parameter given to the ``\verb|acmart|'' document class is
the {\itshape template style} which corresponds to the kind of publication
or SIG publishing the work. This parameter is enclosed in square
brackets and is a part of the {\verb|documentclass|} command:
\begin{verbatim}
  \documentclass[STYLE]{acmart}
\end{verbatim}

Journals use one of three template styles. All but three ACM journals
use the {\verb|acmsmall|} template style:
\begin{itemize}
\item {\verb|acmsmall|}: The default journal template style.
\item {\verb|acmlarge|}: Used by JOCCH and TAP.
\item {\verb|acmtog|}: Used by TOG.
\end{itemize}

The majority of conference proceedings documentation will use the {\verb|acmconf|} template style.
\begin{itemize}
\item {\verb|acmconf|}: The default proceedings template style.
\item{\verb|sigchi|}: Used for SIGCHI conference articles.
\item{\verb|sigchi-a|}: Used for SIGCHI ``Extended Abstract'' articles.
\item{\verb|sigplan|}: Used for SIGPLAN conference articles.
\end{itemize}

\subsection{Template Parameters}

In addition to specifying the {\itshape template style} to be used in
formatting your work, there are a number of {\itshape template parameters}
which modify some part of the applied template style. A complete list
of these parameters can be found in the {\itshape \LaTeX\ User's Guide.}

Frequently-used parameters, or combinations of parameters, include:
\begin{itemize}
\item {\verb|anonymous,review|}: Suitable for a ``double-blind''
  conference submission. Anonymizes the work and includes line
  numbers. Use with the \verb|\acmSubmissionID| command to print the
  submission's unique ID on each page of the work.
\item{\verb|authorversion|}: Produces a version of the work suitable
  for posting by the author.
\item{\verb|screen|}: Produces colored hyperlinks.
\end{itemize}

This document uses the following string as the first command in the
source file:
\begin{verbatim}
\documentclass[sigconf]{acmart}
\end{verbatim}

\section{Modifications}

Modifying the template --- including but not limited to: adjusting
margins, typeface sizes, line spacing, paragraph and list definitions,
and the use of the \verb|\vspace| command to manually adjust the
vertical spacing between elements of your work --- is not allowed.

{\bfseries Your document will be returned to you for revision if
  modifications are discovered.}

\section{Typefaces}

The ``\verb|acmart|'' document class requires the use of the
``Libertine'' typeface family. Your \TeX\ installation should include
this set of packages. Please do not substitute other typefaces. The
``\verb|lmodern|'' and ``\verb|ltimes|'' packages should not be used,
as they will override the built-in typeface families.

\section{Title Information}

The title of your work should use capital letters appropriately -
\url{https://capitalizemytitle.com/} has useful rules for
capitalization. Use the {\verb|title|} command to define the title of
your work. If your work has a subtitle, define it with the
{\verb|subtitle|} command.  Do not insert line breaks in your title.

If your title is lengthy, you must define a short version to be used
in the page headers, to prevent overlapping text. The \verb|title|
command has a ``short title'' parameter:
\begin{verbatim}
  \title[short title]{full title}
\end{verbatim}

\section{Authors and Affiliations}

Each author must be defined separately for accurate metadata
identification. Multiple authors may share one affiliation. Authors'
names should not be abbreviated; use full first names wherever
possible. Include authors' e-mail addresses whenever possible.

Grouping authors' names or e-mail addresses, or providing an ``e-mail
alias,'' as shown below, is not acceptable:
\begin{verbatim}
  \author{Brooke Aster, David Mehldau}
  \email{dave,judy,steve@university.edu}
  \email{firstname.lastname@phillips.org}
\end{verbatim}

The \verb|authornote| and \verb|authornotemark| commands allow a note
to apply to multiple authors --- for example, if the first two authors
of an article contributed equally to the work.

If your author list is lengthy, you must define a shortened version of
the list of authors to be used in the page headers, to prevent
overlapping text. The following command should be placed just after
the last \verb|\author{}| definition:
\begin{verbatim}
  \renewcommand{\shortauthors}{McCartney, et al.}
\end{verbatim}
Omitting this command will force the use of a concatenated list of all
of the authors' names, which may result in overlapping text in the
page headers.

The article template's documentation, available at
\url{https://www.acm.org/publications/proceedings-template}, has a
complete explanation of these commands and tips for their effective
use.

Note that authors' addresses are mandatory for journal articles.

\section{Rights Information}

Authors of any work published by ACM will need to complete a rights
form. Depending on the kind of work, and the rights management choice
made by the author, this may be copyright transfer, permission,
license, or an OA (open access) agreement.

Regardless of the rights management choice, the author will receive a
copy of the completed rights form once it has been submitted. This
form contains \LaTeX\ commands that must be copied into the source
document. When the document source is compiled, these commands and
their parameters add formatted text to several areas of the final
document:
\begin{itemize}
\item the ``ACM Reference Format'' text on the first page.
\item the ``rights management'' text on the first page.
\item the conference information in the page header(s).
\end{itemize}

Rights information is unique to the work; if you are preparing several
works for an event, make sure to use the correct set of commands with
each of the works.

The ACM Reference Format text is required for all articles over one
page in length, and is optional for one-page articles (abstracts).

\section{CCS Concepts and User-Defined Keywords}

Two elements of the ``acmart'' document class provide powerful
taxonomic tools for you to help readers find your work in an online
search.

The ACM Computing Classification System ---
\url{https://www.acm.org/publications/class-2012} --- is a set of
classifiers and concepts that describe the computing
discipline. Authors can select entries from this classification
system, via \url{https://dl.acm.org/ccs/ccs.cfm}, and generate the
commands to be included in the \LaTeX\ source.

User-defined keywords are a comma-separated list of words and phrases
of the authors' choosing, providing a more flexible way of describing
the research being presented.

CCS concepts and user-defined keywords are required for for all
articles over two pages in length, and are optional for one- and
two-page articles (or abstracts).

\section{Sectioning Commands}

Your work should use standard \LaTeX\ sectioning commands:
\verb|section|, \verb|subsection|, \verb|subsubsection|, and
\verb|paragraph|. They should be numbered; do not remove the numbering
from the commands.

Simulating a sectioning command by setting the first word or words of
a paragraph in boldface or italicized text is {\bfseries not allowed.}

\section{Tables}

The ``\verb|acmart|'' document class includes the ``\verb|booktabs|''
package --- \url{https://ctan.org/pkg/booktabs} --- for preparing
high-quality tables.

Table captions are placed {\itshape above} the table.

Because tables cannot be split across pages, the best placement for
them is typically the top of the page nearest their initial cite.  To
ensure this proper ``floating'' placement of tables, use the
environment \textbf{table} to enclose the table's contents and the
table caption.  The contents of the table itself must go in the
\textbf{tabular} environment, to be aligned properly in rows and
columns, with the desired horizontal and vertical rules.  Again,
detailed instructions on \textbf{tabular} material are found in the
\textit{\LaTeX\ User's Guide}.

Immediately following this sentence is the point at which
Table~\ref{tab:freq} is included in the input file; compare the
placement of the table here with the table in the printed output of
this document.

\begin{table}
  \caption{Frequency of Special Characters}
  \label{tab:freq}
  \begin{tabular}{ccl}
    \toprule
    Non-English or Math&Frequency&Comments\\
    \midrule
    \O & 1 in 1,000& For Swedish names\\
    $\pi$ & 1 in 5& Common in math\\
    \$ & 4 in 5 & Used in business\\
    $\Psi^2_1$ & 1 in 40,000& Unexplained usage\\
  \bottomrule
\end{tabular}
\end{table}

To set a wider table, which takes up the whole width of the page's
live area, use the environment \textbf{table*} to enclose the table's
contents and the table caption.  As with a single-column table, this
wide table will ``float'' to a location deemed more
desirable. Immediately following this sentence is the point at which
Table~\ref{tab:commands} is included in the input file; again, it is
instructive to compare the placement of the table here with the table
in the printed output of this document.

\begin{table*}
  \caption{Some Typical Commands}
  \label{tab:commands}
  \begin{tabular}{ccl}
    \toprule
    Command &A Number & Comments\\
    \midrule
    \texttt{{\char'134}author} & 100& Author \\
    \texttt{{\char'134}table}& 300 & For tables\\
    \texttt{{\char'134}table*}& 400& For wider tables\\
    \bottomrule
  \end{tabular}
\end{table*}

Always use midrule to separate table header rows from data rows, and
use it only for this purpose. This enables assistive technologies to
recognise table headers and support their users in navigating tables
more easily.

\section{Math Equations}
You may want to display math equations in three distinct styles:
inline, numbered or non-numbered display.  Each of the three are
discussed in the next sections.

\subsection{Inline (In-text) Equations}
A formula that appears in the running text is called an inline or
in-text formula.  It is produced by the \textbf{math} environment,
which can be invoked with the usual
\texttt{{\char'134}begin\,\ldots{\char'134}end} construction or with
the short form \texttt{\$\,\ldots\$}. You can use any of the symbols
and structures, from $\alpha$ to $\omega$, available in
\LaTeX~\cite{Lamport:LaTeX}; this section will simply show a few
examples of in-text equations in context. Notice how this equation:
\begin{math}
  \lim_{n\rightarrow \infty}x=0
\end{math},
set here in in-line math style, looks slightly different when
set in display style.  (See next section).

\subsection{Display Equations}
A numbered display equation---one set off by vertical space from the
text and centered horizontally---is produced by the \textbf{equation}
environment. An unnumbered display equation is produced by the
\textbf{displaymath} environment.

Again, in either environment, you can use any of the symbols and
structures available in \LaTeX\@; this section will just give a couple
of examples of display equations in context.  First, consider the
equation, shown as an inline equation above:
\begin{equation}
  \lim_{n\rightarrow \infty}x=0
\end{equation}
Notice how it is formatted somewhat differently in
the \textbf{displaymath}
environment.  Now, we'll enter an unnumbered equation:
\begin{displaymath}
  \sum_{i=0}^{\infty} x + 1
\end{displaymath}
and follow it with another numbered equation:
\begin{equation}
  \sum_{i=0}^{\infty}x_i=\int_{0}^{\pi+2} f
\end{equation}
just to demonstrate \LaTeX's able handling of numbering.

\section{Figures}

The ``\verb|figure|'' environment should be used for figures. One or
more images can be placed within a figure. If your figure contains
third-party material, you must clearly identify it as such, as shown
in the example below.
\begin{figure}[h]
  \centering
  \includegraphics[width=\linewidth]{sample-franklin}
  \caption{1907 Franklin Model D roadster. Photograph by Harris \&
    Ewing, Inc. [Public domain], via Wikimedia
    Commons. (\url{https://goo.gl/VLCRBB}).}
  \Description{A woman and a girl in white dresses sit in an open car.}
\end{figure}

Your figures should contain a caption which describes the figure to
the reader.

Figure captions are placed {\itshape below} the figure.

Every figure should also have a figure description unless it is purely
decorative. These descriptions convey what's in the image to someone
who cannot see it. They are also used by search engine crawlers for
indexing images, and when images cannot be loaded.

A figure description must be unformatted plain text less than 2000
characters long (including spaces).  {\bfseries Figure descriptions
  should not repeat the figure caption – their purpose is to capture
  important information that is not already provided in the caption or
  the main text of the paper.} For figures that convey important and
complex new information, a short text description may not be
adequate. More complex alternative descriptions can be placed in an
appendix and referenced in a short figure description. For example,
provide a data table capturing the information in a bar chart, or a
structured list representing a graph.  For additional information
regarding how best to write figure descriptions and why doing this is
so important, please see
\url{https://www.acm.org/publications/taps/describing-figures/}.

\subsection{The ``Teaser Figure''}

A ``teaser figure'' is an image, or set of images in one figure, that
are placed after all author and affiliation information, and before
the body of the article, spanning the page. If you wish to have such a
figure in your article, place the command immediately before the
\verb|\maketitle| command:
\begin{verbatim}
  \begin{teaserfigure}
    \includegraphics[width=\textwidth]{sampleteaser}
    \caption{figure caption}
    \Description{figure description}
  \end{teaserfigure}
\end{verbatim}

\section{Citations and Bibliographies}

The use of \BibTeX\ for the preparation and formatting of one's
references is strongly recommended. Authors' names should be complete
--- use full first names (``Donald E. Knuth'') not initials
(``D. E. Knuth'') --- and the salient identifying features of a
reference should be included: title, year, volume, number, pages,
article DOI, etc.

The bibliography is included in your source document with these two
commands, placed just before the \verb|\end{document}| command:
\begin{verbatim}
  \bibliographystyle{ACM-Reference-Format}

\section{Introduction}
ACM's consolidated article template, introduced in 2017, provides a
consistent \LaTeX\ style for use across ACM publications, and
incorporates accessibility and metadata-extraction functionality
necessary for future Digital Library endeavors. Numerous ACM and
SIG-specific \LaTeX\ templates have been examined, and their unique
features incorporated into this single new template.

If you are new to publishing with ACM, this document is a valuable
guide to the process of preparing your work for publication. If you
have published with ACM before, this document provides insight and
instruction into more recent changes to the article template.

The ``\verb|acmart|'' document class can be used to prepare articles
for any ACM publication --- conference or journal, and for any stage
of publication, from review to final ``camera-ready'' copy, to the
author's own version, with {\itshape very} few changes to the source.

\section{Template Overview}
As noted in the introduction, the ``\verb|acmart|'' document class can
be used to prepare many different kinds of documentation --- a
double-blind initial submission of a full-length technical paper, a
two-page SIGGRAPH Emerging Technologies abstract, a ``camera-ready''
journal article, a SIGCHI Extended Abstract, and more --- all by
selecting the appropriate {\itshape template style} and {\itshape
  template parameters}.

This document will explain the major features of the document
class. For further information, the {\itshape \LaTeX\ User's Guide} is
available from
\url{https://www.acm.org/publications/proceedings-template}.

\subsection{Template Styles}

The primary parameter given to the ``\verb|acmart|'' document class is
the {\itshape template style} which corresponds to the kind of publication
or SIG publishing the work. This parameter is enclosed in square
brackets and is a part of the {\verb|documentclass|} command:
\begin{verbatim}
  \documentclass[STYLE]{acmart}
\end{verbatim}

Journals use one of three template styles. All but three ACM journals
use the {\verb|acmsmall|} template style:
\begin{itemize}
\item {\verb|acmsmall|}: The default journal template style.
\item {\verb|acmlarge|}: Used by JOCCH and TAP.
\item {\verb|acmtog|}: Used by TOG.
\end{itemize}

The majority of conference proceedings documentation will use the {\verb|acmconf|} template style.
\begin{itemize}
\item {\verb|acmconf|}: The default proceedings template style.
\item{\verb|sigchi|}: Used for SIGCHI conference articles.
\item{\verb|sigchi-a|}: Used for SIGCHI ``Extended Abstract'' articles.
\item{\verb|sigplan|}: Used for SIGPLAN conference articles.
\end{itemize}

\subsection{Template Parameters}

In addition to specifying the {\itshape template style} to be used in
formatting your work, there are a number of {\itshape template parameters}
which modify some part of the applied template style. A complete list
of these parameters can be found in the {\itshape \LaTeX\ User's Guide.}

Frequently-used parameters, or combinations of parameters, include:
\begin{itemize}
\item {\verb|anonymous,review|}: Suitable for a ``double-blind''
  conference submission. Anonymizes the work and includes line
  numbers. Use with the \verb|\acmSubmissionID| command to print the
  submission's unique ID on each page of the work.
\item{\verb|authorversion|}: Produces a version of the work suitable
  for posting by the author.
\item{\verb|screen|}: Produces colored hyperlinks.
\end{itemize}

This document uses the following string as the first command in the
source file:
\begin{verbatim}
\documentclass[sigconf]{acmart}
\end{verbatim}

\section{Modifications}

Modifying the template --- including but not limited to: adjusting
margins, typeface sizes, line spacing, paragraph and list definitions,
and the use of the \verb|\vspace| command to manually adjust the
vertical spacing between elements of your work --- is not allowed.

{\bfseries Your document will be returned to you for revision if
  modifications are discovered.}

\section{Typefaces}

The ``\verb|acmart|'' document class requires the use of the
``Libertine'' typeface family. Your \TeX\ installation should include
this set of packages. Please do not substitute other typefaces. The
``\verb|lmodern|'' and ``\verb|ltimes|'' packages should not be used,
as they will override the built-in typeface families.

\section{Title Information}

The title of your work should use capital letters appropriately -
\url{https://capitalizemytitle.com/} has useful rules for
capitalization. Use the {\verb|title|} command to define the title of
your work. If your work has a subtitle, define it with the
{\verb|subtitle|} command.  Do not insert line breaks in your title.

If your title is lengthy, you must define a short version to be used
in the page headers, to prevent overlapping text. The \verb|title|
command has a ``short title'' parameter:
\begin{verbatim}
  \title[short title]{full title}
\end{verbatim}

\section{Authors and Affiliations}

Each author must be defined separately for accurate metadata
identification. Multiple authors may share one affiliation. Authors'
names should not be abbreviated; use full first names wherever
possible. Include authors' e-mail addresses whenever possible.

Grouping authors' names or e-mail addresses, or providing an ``e-mail
alias,'' as shown below, is not acceptable:
\begin{verbatim}
  \author{Brooke Aster, David Mehldau}
  \email{dave,judy,steve@university.edu}
  \email{firstname.lastname@phillips.org}
\end{verbatim}

The \verb|authornote| and \verb|authornotemark| commands allow a note
to apply to multiple authors --- for example, if the first two authors
of an article contributed equally to the work.

If your author list is lengthy, you must define a shortened version of
the list of authors to be used in the page headers, to prevent
overlapping text. The following command should be placed just after
the last \verb|\author{}| definition:
\begin{verbatim}
  \renewcommand{\shortauthors}{McCartney, et al.}
\end{verbatim}
Omitting this command will force the use of a concatenated list of all
of the authors' names, which may result in overlapping text in the
page headers.

The article template's documentation, available at
\url{https://www.acm.org/publications/proceedings-template}, has a
complete explanation of these commands and tips for their effective
use.

Note that authors' addresses are mandatory for journal articles.

\section{Rights Information}

Authors of any work published by ACM will need to complete a rights
form. Depending on the kind of work, and the rights management choice
made by the author, this may be copyright transfer, permission,
license, or an OA (open access) agreement.

Regardless of the rights management choice, the author will receive a
copy of the completed rights form once it has been submitted. This
form contains \LaTeX\ commands that must be copied into the source
document. When the document source is compiled, these commands and
their parameters add formatted text to several areas of the final
document:
\begin{itemize}
\item the ``ACM Reference Format'' text on the first page.
\item the ``rights management'' text on the first page.
\item the conference information in the page header(s).
\end{itemize}

Rights information is unique to the work; if you are preparing several
works for an event, make sure to use the correct set of commands with
each of the works.

The ACM Reference Format text is required for all articles over one
page in length, and is optional for one-page articles (abstracts).

\section{CCS Concepts and User-Defined Keywords}

Two elements of the ``acmart'' document class provide powerful
taxonomic tools for you to help readers find your work in an online
search.

The ACM Computing Classification System ---
\url{https://www.acm.org/publications/class-2012} --- is a set of
classifiers and concepts that describe the computing
discipline. Authors can select entries from this classification
system, via \url{https://dl.acm.org/ccs/ccs.cfm}, and generate the
commands to be included in the \LaTeX\ source.

User-defined keywords are a comma-separated list of words and phrases
of the authors' choosing, providing a more flexible way of describing
the research being presented.

CCS concepts and user-defined keywords are required for for all
articles over two pages in length, and are optional for one- and
two-page articles (or abstracts).

\section{Sectioning Commands}

Your work should use standard \LaTeX\ sectioning commands:
\verb|section|, \verb|subsection|, \verb|subsubsection|, and
\verb|paragraph|. They should be numbered; do not remove the numbering
from the commands.

Simulating a sectioning command by setting the first word or words of
a paragraph in boldface or italicized text is {\bfseries not allowed.}

\section{Tables}

The ``\verb|acmart|'' document class includes the ``\verb|booktabs|''
package --- \url{https://ctan.org/pkg/booktabs} --- for preparing
high-quality tables.

Table captions are placed {\itshape above} the table.

Because tables cannot be split across pages, the best placement for
them is typically the top of the page nearest their initial cite.  To
ensure this proper ``floating'' placement of tables, use the
environment \textbf{table} to enclose the table's contents and the
table caption.  The contents of the table itself must go in the
\textbf{tabular} environment, to be aligned properly in rows and
columns, with the desired horizontal and vertical rules.  Again,
detailed instructions on \textbf{tabular} material are found in the
\textit{\LaTeX\ User's Guide}.

Immediately following this sentence is the point at which
Table~\ref{tab:freq} is included in the input file; compare the
placement of the table here with the table in the printed output of
this document.

\begin{table}
  \caption{Frequency of Special Characters}
  \label{tab:freq}
  \begin{tabular}{ccl}
    \toprule
    Non-English or Math&Frequency&Comments\\
    \midrule
    \O & 1 in 1,000& For Swedish names\\
    $\pi$ & 1 in 5& Common in math\\
    \$ & 4 in 5 & Used in business\\
    $\Psi^2_1$ & 1 in 40,000& Unexplained usage\\
  \bottomrule
\end{tabular}
\end{table}

To set a wider table, which takes up the whole width of the page's
live area, use the environment \textbf{table*} to enclose the table's
contents and the table caption.  As with a single-column table, this
wide table will ``float'' to a location deemed more
desirable. Immediately following this sentence is the point at which
Table~\ref{tab:commands} is included in the input file; again, it is
instructive to compare the placement of the table here with the table
in the printed output of this document.

\begin{table*}
  \caption{Some Typical Commands}
  \label{tab:commands}
  \begin{tabular}{ccl}
    \toprule
    Command &A Number & Comments\\
    \midrule
    \texttt{{\char'134}author} & 100& Author \\
    \texttt{{\char'134}table}& 300 & For tables\\
    \texttt{{\char'134}table*}& 400& For wider tables\\
    \bottomrule
  \end{tabular}
\end{table*}

Always use midrule to separate table header rows from data rows, and
use it only for this purpose. This enables assistive technologies to
recognise table headers and support their users in navigating tables
more easily.

\section{Math Equations}
You may want to display math equations in three distinct styles:
inline, numbered or non-numbered display.  Each of the three are
discussed in the next sections.

\subsection{Inline (In-text) Equations}
A formula that appears in the running text is called an inline or
in-text formula.  It is produced by the \textbf{math} environment,
which can be invoked with the usual
\texttt{{\char'134}begin\,\ldots{\char'134}end} construction or with
the short form \texttt{\$\,\ldots\$}. You can use any of the symbols
and structures, from $\alpha$ to $\omega$, available in
\LaTeX~\cite{Lamport:LaTeX}; this section will simply show a few
examples of in-text equations in context. Notice how this equation:
\begin{math}
  \lim_{n\rightarrow \infty}x=0
\end{math},
set here in in-line math style, looks slightly different when
set in display style.  (See next section).

\subsection{Display Equations}
A numbered display equation---one set off by vertical space from the
text and centered horizontally---is produced by the \textbf{equation}
environment. An unnumbered display equation is produced by the
\textbf{displaymath} environment.

Again, in either environment, you can use any of the symbols and
structures available in \LaTeX\@; this section will just give a couple
of examples of display equations in context.  First, consider the
equation, shown as an inline equation above:
\begin{equation}
  \lim_{n\rightarrow \infty}x=0
\end{equation}
Notice how it is formatted somewhat differently in
the \textbf{displaymath}
environment.  Now, we'll enter an unnumbered equation:
\begin{displaymath}
  \sum_{i=0}^{\infty} x + 1
\end{displaymath}
and follow it with another numbered equation:
\begin{equation}
  \sum_{i=0}^{\infty}x_i=\int_{0}^{\pi+2} f
\end{equation}
just to demonstrate \LaTeX's able handling of numbering.

\section{Figures}

The ``\verb|figure|'' environment should be used for figures. One or
more images can be placed within a figure. If your figure contains
third-party material, you must clearly identify it as such, as shown
in the example below.
\begin{figure}[h]
  \centering
  \includegraphics[width=\linewidth]{sample-franklin}
  \caption{1907 Franklin Model D roadster. Photograph by Harris \&
    Ewing, Inc. [Public domain], via Wikimedia
    Commons. (\url{https://goo.gl/VLCRBB}).}
  \Description{A woman and a girl in white dresses sit in an open car.}
\end{figure}

Your figures should contain a caption which describes the figure to
the reader.

Figure captions are placed {\itshape below} the figure.

Every figure should also have a figure description unless it is purely
decorative. These descriptions convey what's in the image to someone
who cannot see it. They are also used by search engine crawlers for
indexing images, and when images cannot be loaded.

A figure description must be unformatted plain text less than 2000
characters long (including spaces).  {\bfseries Figure descriptions
  should not repeat the figure caption – their purpose is to capture
  important information that is not already provided in the caption or
  the main text of the paper.} For figures that convey important and
complex new information, a short text description may not be
adequate. More complex alternative descriptions can be placed in an
appendix and referenced in a short figure description. For example,
provide a data table capturing the information in a bar chart, or a
structured list representing a graph.  For additional information
regarding how best to write figure descriptions and why doing this is
so important, please see
\url{https://www.acm.org/publications/taps/describing-figures/}.

\subsection{The ``Teaser Figure''}

A ``teaser figure'' is an image, or set of images in one figure, that
are placed after all author and affiliation information, and before
the body of the article, spanning the page. If you wish to have such a
figure in your article, place the command immediately before the
\verb|\maketitle| command:
\begin{verbatim}
  \begin{teaserfigure}
    \includegraphics[width=\textwidth]{sampleteaser}
    \caption{figure caption}
    \Description{figure description}
  \end{teaserfigure}
\end{verbatim}

\section{Citations and Bibliographies}

The use of \BibTeX\ for the preparation and formatting of one's
references is strongly recommended. Authors' names should be complete
--- use full first names (``Donald E. Knuth'') not initials
(``D. E. Knuth'') --- and the salient identifying features of a
reference should be included: title, year, volume, number, pages,
article DOI, etc.

The bibliography is included in your source document with these two
commands, placed just before the \verb|\end{document}| command:
\begin{verbatim}
  \bibliographystyle{ACM-Reference-Format}

\section{Introduction}
ACM's consolidated article template, introduced in 2017, provides a
consistent \LaTeX\ style for use across ACM publications, and
incorporates accessibility and metadata-extraction functionality
necessary for future Digital Library endeavors. Numerous ACM and
SIG-specific \LaTeX\ templates have been examined, and their unique
features incorporated into this single new template.

If you are new to publishing with ACM, this document is a valuable
guide to the process of preparing your work for publication. If you
have published with ACM before, this document provides insight and
instruction into more recent changes to the article template.

The ``\verb|acmart|'' document class can be used to prepare articles
for any ACM publication --- conference or journal, and for any stage
of publication, from review to final ``camera-ready'' copy, to the
author's own version, with {\itshape very} few changes to the source.

\section{Template Overview}
As noted in the introduction, the ``\verb|acmart|'' document class can
be used to prepare many different kinds of documentation --- a
double-blind initial submission of a full-length technical paper, a
two-page SIGGRAPH Emerging Technologies abstract, a ``camera-ready''
journal article, a SIGCHI Extended Abstract, and more --- all by
selecting the appropriate {\itshape template style} and {\itshape
  template parameters}.

This document will explain the major features of the document
class. For further information, the {\itshape \LaTeX\ User's Guide} is
available from
\url{https://www.acm.org/publications/proceedings-template}.

\subsection{Template Styles}

The primary parameter given to the ``\verb|acmart|'' document class is
the {\itshape template style} which corresponds to the kind of publication
or SIG publishing the work. This parameter is enclosed in square
brackets and is a part of the {\verb|documentclass|} command:
\begin{verbatim}
  \documentclass[STYLE]{acmart}
\end{verbatim}

Journals use one of three template styles. All but three ACM journals
use the {\verb|acmsmall|} template style:
\begin{itemize}
\item {\verb|acmsmall|}: The default journal template style.
\item {\verb|acmlarge|}: Used by JOCCH and TAP.
\item {\verb|acmtog|}: Used by TOG.
\end{itemize}

The majority of conference proceedings documentation will use the {\verb|acmconf|} template style.
\begin{itemize}
\item {\verb|acmconf|}: The default proceedings template style.
\item{\verb|sigchi|}: Used for SIGCHI conference articles.
\item{\verb|sigchi-a|}: Used for SIGCHI ``Extended Abstract'' articles.
\item{\verb|sigplan|}: Used for SIGPLAN conference articles.
\end{itemize}

\subsection{Template Parameters}

In addition to specifying the {\itshape template style} to be used in
formatting your work, there are a number of {\itshape template parameters}
which modify some part of the applied template style. A complete list
of these parameters can be found in the {\itshape \LaTeX\ User's Guide.}

Frequently-used parameters, or combinations of parameters, include:
\begin{itemize}
\item {\verb|anonymous,review|}: Suitable for a ``double-blind''
  conference submission. Anonymizes the work and includes line
  numbers. Use with the \verb|\acmSubmissionID| command to print the
  submission's unique ID on each page of the work.
\item{\verb|authorversion|}: Produces a version of the work suitable
  for posting by the author.
\item{\verb|screen|}: Produces colored hyperlinks.
\end{itemize}

This document uses the following string as the first command in the
source file:
\begin{verbatim}
\documentclass[sigconf]{acmart}
\end{verbatim}

\section{Modifications}

Modifying the template --- including but not limited to: adjusting
margins, typeface sizes, line spacing, paragraph and list definitions,
and the use of the \verb|\vspace| command to manually adjust the
vertical spacing between elements of your work --- is not allowed.

{\bfseries Your document will be returned to you for revision if
  modifications are discovered.}

\section{Typefaces}

The ``\verb|acmart|'' document class requires the use of the
``Libertine'' typeface family. Your \TeX\ installation should include
this set of packages. Please do not substitute other typefaces. The
``\verb|lmodern|'' and ``\verb|ltimes|'' packages should not be used,
as they will override the built-in typeface families.

\section{Title Information}

The title of your work should use capital letters appropriately -
\url{https://capitalizemytitle.com/} has useful rules for
capitalization. Use the {\verb|title|} command to define the title of
your work. If your work has a subtitle, define it with the
{\verb|subtitle|} command.  Do not insert line breaks in your title.

If your title is lengthy, you must define a short version to be used
in the page headers, to prevent overlapping text. The \verb|title|
command has a ``short title'' parameter:
\begin{verbatim}
  \title[short title]{full title}
\end{verbatim}

\section{Authors and Affiliations}

Each author must be defined separately for accurate metadata
identification. Multiple authors may share one affiliation. Authors'
names should not be abbreviated; use full first names wherever
possible. Include authors' e-mail addresses whenever possible.

Grouping authors' names or e-mail addresses, or providing an ``e-mail
alias,'' as shown below, is not acceptable:
\begin{verbatim}
  \author{Brooke Aster, David Mehldau}
  \email{dave,judy,steve@university.edu}
  \email{firstname.lastname@phillips.org}
\end{verbatim}

The \verb|authornote| and \verb|authornotemark| commands allow a note
to apply to multiple authors --- for example, if the first two authors
of an article contributed equally to the work.

If your author list is lengthy, you must define a shortened version of
the list of authors to be used in the page headers, to prevent
overlapping text. The following command should be placed just after
the last \verb|\author{}| definition:
\begin{verbatim}
  \renewcommand{\shortauthors}{McCartney, et al.}
\end{verbatim}
Omitting this command will force the use of a concatenated list of all
of the authors' names, which may result in overlapping text in the
page headers.

The article template's documentation, available at
\url{https://www.acm.org/publications/proceedings-template}, has a
complete explanation of these commands and tips for their effective
use.

Note that authors' addresses are mandatory for journal articles.

\section{Rights Information}

Authors of any work published by ACM will need to complete a rights
form. Depending on the kind of work, and the rights management choice
made by the author, this may be copyright transfer, permission,
license, or an OA (open access) agreement.

Regardless of the rights management choice, the author will receive a
copy of the completed rights form once it has been submitted. This
form contains \LaTeX\ commands that must be copied into the source
document. When the document source is compiled, these commands and
their parameters add formatted text to several areas of the final
document:
\begin{itemize}
\item the ``ACM Reference Format'' text on the first page.
\item the ``rights management'' text on the first page.
\item the conference information in the page header(s).
\end{itemize}

Rights information is unique to the work; if you are preparing several
works for an event, make sure to use the correct set of commands with
each of the works.

The ACM Reference Format text is required for all articles over one
page in length, and is optional for one-page articles (abstracts).

\section{CCS Concepts and User-Defined Keywords}

Two elements of the ``acmart'' document class provide powerful
taxonomic tools for you to help readers find your work in an online
search.

The ACM Computing Classification System ---
\url{https://www.acm.org/publications/class-2012} --- is a set of
classifiers and concepts that describe the computing
discipline. Authors can select entries from this classification
system, via \url{https://dl.acm.org/ccs/ccs.cfm}, and generate the
commands to be included in the \LaTeX\ source.

User-defined keywords are a comma-separated list of words and phrases
of the authors' choosing, providing a more flexible way of describing
the research being presented.

CCS concepts and user-defined keywords are required for for all
articles over two pages in length, and are optional for one- and
two-page articles (or abstracts).

\section{Sectioning Commands}

Your work should use standard \LaTeX\ sectioning commands:
\verb|section|, \verb|subsection|, \verb|subsubsection|, and
\verb|paragraph|. They should be numbered; do not remove the numbering
from the commands.

Simulating a sectioning command by setting the first word or words of
a paragraph in boldface or italicized text is {\bfseries not allowed.}

\section{Tables}

The ``\verb|acmart|'' document class includes the ``\verb|booktabs|''
package --- \url{https://ctan.org/pkg/booktabs} --- for preparing
high-quality tables.

Table captions are placed {\itshape above} the table.

Because tables cannot be split across pages, the best placement for
them is typically the top of the page nearest their initial cite.  To
ensure this proper ``floating'' placement of tables, use the
environment \textbf{table} to enclose the table's contents and the
table caption.  The contents of the table itself must go in the
\textbf{tabular} environment, to be aligned properly in rows and
columns, with the desired horizontal and vertical rules.  Again,
detailed instructions on \textbf{tabular} material are found in the
\textit{\LaTeX\ User's Guide}.

Immediately following this sentence is the point at which
Table~\ref{tab:freq} is included in the input file; compare the
placement of the table here with the table in the printed output of
this document.

\begin{table}
  \caption{Frequency of Special Characters}
  \label{tab:freq}
  \begin{tabular}{ccl}
    \toprule
    Non-English or Math&Frequency&Comments\\
    \midrule
    \O & 1 in 1,000& For Swedish names\\
    $\pi$ & 1 in 5& Common in math\\
    \$ & 4 in 5 & Used in business\\
    $\Psi^2_1$ & 1 in 40,000& Unexplained usage\\
  \bottomrule
\end{tabular}
\end{table}

To set a wider table, which takes up the whole width of the page's
live area, use the environment \textbf{table*} to enclose the table's
contents and the table caption.  As with a single-column table, this
wide table will ``float'' to a location deemed more
desirable. Immediately following this sentence is the point at which
Table~\ref{tab:commands} is included in the input file; again, it is
instructive to compare the placement of the table here with the table
in the printed output of this document.

\begin{table*}
  \caption{Some Typical Commands}
  \label{tab:commands}
  \begin{tabular}{ccl}
    \toprule
    Command &A Number & Comments\\
    \midrule
    \texttt{{\char'134}author} & 100& Author \\
    \texttt{{\char'134}table}& 300 & For tables\\
    \texttt{{\char'134}table*}& 400& For wider tables\\
    \bottomrule
  \end{tabular}
\end{table*}

Always use midrule to separate table header rows from data rows, and
use it only for this purpose. This enables assistive technologies to
recognise table headers and support their users in navigating tables
more easily.

\section{Math Equations}
You may want to display math equations in three distinct styles:
inline, numbered or non-numbered display.  Each of the three are
discussed in the next sections.

\subsection{Inline (In-text) Equations}
A formula that appears in the running text is called an inline or
in-text formula.  It is produced by the \textbf{math} environment,
which can be invoked with the usual
\texttt{{\char'134}begin\,\ldots{\char'134}end} construction or with
the short form \texttt{\$\,\ldots\$}. You can use any of the symbols
and structures, from $\alpha$ to $\omega$, available in
\LaTeX~\cite{Lamport:LaTeX}; this section will simply show a few
examples of in-text equations in context. Notice how this equation:
\begin{math}
  \lim_{n\rightarrow \infty}x=0
\end{math},
set here in in-line math style, looks slightly different when
set in display style.  (See next section).

\subsection{Display Equations}
A numbered display equation---one set off by vertical space from the
text and centered horizontally---is produced by the \textbf{equation}
environment. An unnumbered display equation is produced by the
\textbf{displaymath} environment.

Again, in either environment, you can use any of the symbols and
structures available in \LaTeX\@; this section will just give a couple
of examples of display equations in context.  First, consider the
equation, shown as an inline equation above:
\begin{equation}
  \lim_{n\rightarrow \infty}x=0
\end{equation}
Notice how it is formatted somewhat differently in
the \textbf{displaymath}
environment.  Now, we'll enter an unnumbered equation:
\begin{displaymath}
  \sum_{i=0}^{\infty} x + 1
\end{displaymath}
and follow it with another numbered equation:
\begin{equation}
  \sum_{i=0}^{\infty}x_i=\int_{0}^{\pi+2} f
\end{equation}
just to demonstrate \LaTeX's able handling of numbering.

\section{Figures}

The ``\verb|figure|'' environment should be used for figures. One or
more images can be placed within a figure. If your figure contains
third-party material, you must clearly identify it as such, as shown
in the example below.
\begin{figure}[h]
  \centering
  \includegraphics[width=\linewidth]{sample-franklin}
  \caption{1907 Franklin Model D roadster. Photograph by Harris \&
    Ewing, Inc. [Public domain], via Wikimedia
    Commons. (\url{https://goo.gl/VLCRBB}).}
  \Description{A woman and a girl in white dresses sit in an open car.}
\end{figure}

Your figures should contain a caption which describes the figure to
the reader.

Figure captions are placed {\itshape below} the figure.

Every figure should also have a figure description unless it is purely
decorative. These descriptions convey what's in the image to someone
who cannot see it. They are also used by search engine crawlers for
indexing images, and when images cannot be loaded.

A figure description must be unformatted plain text less than 2000
characters long (including spaces).  {\bfseries Figure descriptions
  should not repeat the figure caption – their purpose is to capture
  important information that is not already provided in the caption or
  the main text of the paper.} For figures that convey important and
complex new information, a short text description may not be
adequate. More complex alternative descriptions can be placed in an
appendix and referenced in a short figure description. For example,
provide a data table capturing the information in a bar chart, or a
structured list representing a graph.  For additional information
regarding how best to write figure descriptions and why doing this is
so important, please see
\url{https://www.acm.org/publications/taps/describing-figures/}.

\subsection{The ``Teaser Figure''}

A ``teaser figure'' is an image, or set of images in one figure, that
are placed after all author and affiliation information, and before
the body of the article, spanning the page. If you wish to have such a
figure in your article, place the command immediately before the
\verb|\maketitle| command:
\begin{verbatim}
  \begin{teaserfigure}
    \includegraphics[width=\textwidth]{sampleteaser}
    \caption{figure caption}
    \Description{figure description}
  \end{teaserfigure}
\end{verbatim}

\section{Citations and Bibliographies}

The use of \BibTeX\ for the preparation and formatting of one's
references is strongly recommended. Authors' names should be complete
--- use full first names (``Donald E. Knuth'') not initials
(``D. E. Knuth'') --- and the salient identifying features of a
reference should be included: title, year, volume, number, pages,
article DOI, etc.

The bibliography is included in your source document with these two
commands, placed just before the \verb|\end{document}| command:
\begin{verbatim}
  \bibliographystyle{ACM-Reference-Format}

redpajama_set_name: RedPajamaArXiv