\section{Introduction}
The ability to consume readily available information from the internet is alarming for both individuals and organizations. The quality of content on social media platforms has been significantly affected due to the spread of fake news, misinformation and unverifiable facts. The current tally of internet users stands at 4.66 billion\footnote{\href{https://www.internetlivestats.com/}{Internet Live Stats (as on 15-07-2021)}}~\citep{kemp2015global}; and many of these users generate, post and consume content without any regulation, in a large number of countries\footnote{\href{https://en.wikipedia.org/wiki/Internet_censorship_and_surveillance_by_country}{Internet Censorship in Countries}}. Due to the unrestricted nature of online platforms, there is a significant increase in the amount of misinformation on social media~\citep{allen2020evaluating}, especially in developing nations~\citep{badrinathan2020educative,wasserman2019exploratory}. Studies show that events such as the presidential election of the United States in 2016 were affected due to \textit{moderated fake news} campaigns~\citep{tavernise2016fake}. Shu et al.(2017)~\cite{shu2017fake} propose that \textit{fake news is intentionally written, verifiably false, and is created in a way that makes it look authentic}. Manual efforts by other online platforms such as Poynter\footnote{\href{https://www.poynter.org/}{Poynter: Online}}, FactCheck\footnote{\href{https://www.factchecker.in/}{FactCheck: Online}}, AltNews\footnote{\href{https://www.altnews.in/}{AltNews: Online}} \textit{etc.} to detect fake news, requires a lot of human effort and can prove to be cumbersome. Such manual efforts can be time-consuming, challenging, and at times, can also be ineffective as fake news can spread faster than verified claims over social media platforms.

Automatic Fake News Detection is a task that aims to mitigate the problem of misinformation with the help of evidence supported by various sources. Most of the approaches in this recently devised task aim to use the classical machine learning-based methods or the recent deep learning-based methods to help classify news items as fake or as real. Initially proposed methods for the task applied machine learning-based techniques but cited insufficient data as a major concern~\citep{vlachos-riedel-2014-fact}. Recent deep learning and ensemble approaches \citep{malon-2018-team,DBLP:journals/corr/abs-1811-04670} were proposed on the FEVER \citep{thorne-etal-2018-fever} and LIAR \citep{wang-2017-liar} datasets, and have been shown to perform very well. Studies have proposed a combination of evidence detection with textual entailment concerning the claim~\citep{vijjali-etal-2020-two}. FEVER Shared Tasks~\cite{thorne2018fact,thorne-etal-2019-fever2} have helped the automatic fact verification task gather attention towards the problem and helped generate approaches to mitigate the issues with previously proposed solutions. This study shows that our novel approach improvises over state-of-the-art approaches and helps detect fake news related to COVID-19. Our approach performs web-search for evidence collection and uses BERT-score similarity to match the unverified claim with the top-k searches. Further, we propose the use of summarization to mitigate problems with the evidence collection. We summarize the top-n selected lines from these articles and use them as evidence to support or reject the news item claim. Our experiments perform an extensive evaluation of the approach over the datasets released as a part of the CONSTRAINT-2021 Shared Task~\citep{patwa2020fighting} shared task.\\
\\
Our summarized contributions with this paper are:

\begin{itemize}
    \item We propose a novel approach to help automate the evidence collection for any fake news detection dataset.
    \item Additionally, we incorporate a summarization component that helps outperform the state-of-the-art approaches for automatic fact verification on the CONSTRAINT-2021 dataset.
\end{itemize}

\section{Related Work}
Automatic detection and classification of fake news, especially in epidemic situations like COVID-19, is a significant issue for society. Most of the recent works have identified that fake news is written intentionally and factually false~\citep{shu2017fake}. Several datasets have been released for the AI community in the field of fake news detection, such as LIAR~\citep{wang-2017-liar}, Fake News Challenge-1, and FEVER~\citep{thorne-etal-2018-fever}. Some recent techniques extract the evidence from Wikipedia to classify a claim as  \texttt{SUPPORTED, REFUTED or NOTENOUGHINFO}~\citep{thorne-etal-2018-fever}. They formulate the problem as a three-step process (i) first the top-k documents are identified based on the TF-IDF based approaches (ii) then top-k sentences are identified from the documents, and (iii) finally the textual entailment based approaches~\citep{DBLP:journals/corr/ParikhT0U16} are used to classify the claim. Team Papelo~\citep{malon-2018-team} used the Transformer-based approach for the textual entailment and selected the evidence-based on tf-idf and entities present in the title. \citet{hanselowski-etal-2018-ukp} selects the documents and sentence using the entity mentions and recognizes textual entailment using Enhanced Sequential Inference Model (ESIM) \citep{chen-etal-2017-enhanced}. Despite the several attempts, fake news detection is a challenging problem and countering fake news is a typical issue that requires continuous studies. Recently, some researchers released the datasets related to COVID-19 fake news detection. Shahi and Nandini (2020) \cite{DBLP:journals/corr/abs-2006-11343} proposed the first multi-lingual cross-domain dataset for COVID-19 that consists of 5182 fact-checked news articles from Jan-2020 to May-2020. They collected data from 92 different websites and manually classified them into 23 classes. Kar et al. \cite{DBLP:journals/corr/abs-2010-06906} also released a multi-indic-lingual dataset besides English to detect the fake news in social media tweets. They obtained 480 tweets in Bengali and 460 tweets in Hindi. In addition to tweets, they also included several features related to tweets such as retweet count, favourite count, total URL in description, URL, friend list, followers, \textit{etc.} Recently, a very relevant dataset was released by Patwa et al. \cite{patwa2020fighting} which consists of 10,700 tweets or claim collected from various sources such as Twitter, PolitiFact, Snopes, Boomlive. They experimented with various machine learning techniques like Decision Trees, Logistic Regression, SVM, Gradient Boosting DT and achieved the F1-score of 93.32. Most of the previous work on COVID-19 dataset proposed an ensemble approach of various models such as BERT, RoBERTa, XLNet, \textit{etc.}~\cite{DBLP:journals/corr/abs-2101-12027, DBLP:journals/corr/abs-2101-11954, 2021arXiv210105701S}. Chen et al.~\citep{10.1007/978-3-030-73696-5_9} trained the model with additional words such as covid-19, coronavirus, pandemic, indiafightscorona since the BERT tokenizer will split these words into separate tokens. Some works leveraged the fine-tuned models like COVID-Twitter-BERT (CT-BERT) \citep{DBLP:journals/corr/abs-2005-07503} and demonstrated a boost in performance \citep{10.1007/978-3-030-73696-5_11,DBLP:journals/corr/abs-2012-11967,DBLP:journals/corr/abs-2101-04012}. The fake news detection methods described above mainly uses the claim for the classification. Our method focuses on extracting and summarizing the evidence from the external source and uses it to classify the claim on the COVID-19 fake news detection dataset~\citep{10.1007/978-3-030-73696-5_11,patwa2020fighting}.

\begin{table}[h!]
\centering
\begin{tabular}{|l|c|c|r|} \hline
\textbf{Split} & \textbf{Real} & \textbf{Fake} & \textbf{Total} \\ \hline
Training & 3360 & 3060 & 6420  \\\hline
Validation & 1120 & 1020 & 2140 \\\hline
Test & 1120 & 1020 & 2140 \\\hline
\textbf{Total} & 5600 & 5100 & 10700 \\\hline
\end{tabular}
\caption{Dataset Statistics}
\label{table:stats}
\vspace{-0.7cm}
\end{table}

\section{Dataset}
\label{section:dataset}

For our work, we use the pre-released COVID-19 fake news dataset as a part of the CONSTRAINT-2021 shared task~\citep{patwa2020fighting}. This gold-standard manually annotated dataset comprises social media posts and articles which are related to COVID-19. Each post or tweet contains content in the English language and is classified in either of the two categories- \textbf{(1) Real:} where tweets or articles which are factually correct and verified from authentic sources, for example, ``Wearing mask can protect you from the virus. (Twitter)''; or \textbf{(2) Fake:} where tweets or posts related to COVID-19 which are factually incorrect and verified as false, for example, ``If you take Crocin thrice a day you are safe. (Facebook)''.

The authors collect fake news from two different sources- social media platforms and public fact-checking platforms. The social media posts include text from Facebook posts, Instagram posts, and Twitter posts, whereas the fact-checking websites such as PolitiFact, Snopes, and Boomlive are used to collect fact-checked news items. To further collect real news, they sample tweets from official government channels, news channels, and medical institutes. Overall, a total of 14 such sources were used to prepare this dataset.

The dataset comprises 10700 manually annotated samples and is split into (60\%) train, (20\%) validation and (20\%) test sets. We provide the exact numbers for each split/class in Table~\ref{table:stats} for clarity. The dataset is class-balanced as it contains 52.3\% samples of real posts and 47.7\% samples of fake posts. As an analysis on it, we obtained a word-cloud illustration for both real and fake samples and observed a high lexical overlap between both the classes, where words like `\textit{coronavirus}', `\textit{covid19}', `\textit{people}', `\textit{cases}', `\textit{number}', `\textit{test}', \textit{etc.} are repeatedly used in both the sets. We do not show the word cloud due to space constraints. We create and present a wordcloud for the dataset in Figure~\ref{fig:wordcloud}.

\begin{figure}[!t]
\centering
\subfigure[Worcloud of Real Posts]{
\includegraphics[width=.40\columnwidth]{Figures/real_wordcloud.png}
}
\subfigure[Worcloud of Fake Posts]{
\includegraphics[width=.40\columnwidth]{Figures/fake_wordcloud.png}
}
\caption{Wordcloud of real/fake posts in our dataset.}
\label{fig:wordcloud}
\vspace{-0.3cm}
\end{figure}

\section{Our Approach}
In this section, we provide details of our novel approach to augment the dataset with evidence from web search and the use of this evidence to complement the task of fact verification. The algorithm for our approach can be seen in Algorithm~\ref{algo}.
\begin{algorithm}[]
\DontPrintSemicolon
  \KwInput{Claim $c$, Blocked URLs $u$}
  \KwOutput{Evidence $e$}

\SetKwFunction{SearchF}{ArticleRet}
  \SetKwProg{Fn}{Function}{:}{}
  \Fn{\SearchF{$c$, $k=3$}}{

        $results \gets GoogleSearch($c$);$

        $filtered\_results \gets \emptyset$;
        \ForEach{$r_i \in results$}{%
            \If{$r_i \notin u$}{

                $s_i \gets$ Similarity($c$, $r_i$); \tcp*{Document Similarity using spacy library}
                \If{$s_i > 0.7$}{
                    $filtered\_results \gets (r_i, s_i)$
                }
            }
        }
        $filtered\_results \gets Sort(filtered\_results)[:$k$];$
        $\KwRet\ filtered\_results;$
  }
  $articles$ $\gets \SearchF{c}$;

  $e \gets \emptyset$;   \tcp*{Evidences}
  \ForEach{$a_i \in articles$}{%
        $d \gets WebsiteData(a_i.url)$;
        $sents \gets d['<h>'] + d['<p>']$;  \tcp*{Extract <p> and <h> tags from html}
        \ForEach{$s_i \in sents$}{%
            $sim \gets Similarity(c, s_i);$
            \If{$sim > 0.5$}{
                    $e$ $\gets$ ($s_i$, $sim$);
            }
        }
        $e$ $\gets Sort($e$)[:$3$];$
  }
 $\KwRet$ $e;$

\caption{Algorithm to collect the evidence from the input claim}
\label{algo}
\end{algorithm}
As discussed above, we collect this evidence and prune to top-k related news items based on semantic similarity via BERTScore~\citep{devlin-etal-2019-bert}. We also select top-n lines from each article for further building an evidence repository, as detailed below in further subsections.

\begin{figure*}[t!]
\centering
\includegraphics[width=1\textwidth]{Figures/paperimage.jpg}
\caption{Full architecture of our proposed approach.}
\label{fig:arch}
\end{figure*}

\subsection{Evidence Collection}
In the original dataset of COVID-19, evidence is not released along with the claim. We hypothesize that evidence is equally relevant to classify the claim as proposed by~\citet{thorne-etal-2018-fever}. As per our approach, given a claim or post text, we first select K relevant articles using a BERT-based sentence similarity score as detailed here; and can be seen as an architecture component in Figure~\ref{fig:arch}.

\subsubsection{Article Retrieval}
For each claim $c$, we search the claim as a query using a publicly available search API. The response returned by this API consists of \texttt{(heading, text)} pairs. We use the spacy~\cite{spacy} library to get the similarity score of response text with respect to the input claim. Based on this similarity score, we select top K results that have the similarity score greater than $0.7$\footnote{Selected with empirical evaluation and manual analysis after trying 0.5, 0.6, 0.7, 0.8 using the 'en\_nli\_roberta\_base' model for document similarity}. While selecting documents, we prune for webpages in other languages and pages which are direct links to PDF or other such non-text files. As an immediate next step, we scrape the selected web pages to obtain the matching N sentences concerning the claim as detailed here.

\subsubsection{Sentence(s) Retrieval}
In the previous step, we extract the relevant article URLs $U = (u_1, u_2, u_3)$. We employ a similar method to find the sentences within each article. For every url $u$, we first scrape the webpage and extract the text from $<h>$ and $<p>$ tags. Further, we use the same similarity score to select the top N sentences with respect to the claim. We obtain a similarity threshold of 0.5 after performing a similar empirical evaluation as mentioned in the footnote. Eventually, we concatenate the selected sentences from these articles, which act as our evidence for the claim. We would like to note that increasing the threshold significantly higher returned an empty set in some case and hence we choose a relatively lower threshold (0.5).

An example of evidence collected via our approach is shown in Table~\ref{tab:summary} where the column titled ``Evidence'' shows the output after these steps.

\begin{table*}[t]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{@{}llll@{}}
\toprule
\multicolumn{1}{c}{\textbf{Claim}} & \multicolumn{1}{c}{\textbf{Evidence}} & \multicolumn{1}{c}{\textbf{Summarization-1 (S1)}} & \multicolumn{1}{c}{\textbf{Summarization-2 (S2)}} \\ \midrule
\begin{tabular}[c]{@{}l@{}}There is no evidence that children have \\ died because of a COVID-19 vaccine. \\ No vaccine currently in development has \\ been approved for widespread public use.\\ https://t.co/9ecvMR8SAf\end{tabular} & \begin{tabular}[c]{@{}l@{}}Currently there is no coronavirus vaccine that has been\\ approved for the American public. And there is no \\ evidence that children have died because they received \\ one of the COVID-19 vaccines being developed.\\ PolitiFact found no evidence that anyone has died from\\ complications related to a trial COVID-19 vaccination.\\ There is no evidence that children have died because \\ of a COVID-19 vaccine.\end{tabular} & \begin{tabular}[c]{@{}l@{}}There is no evidence that children have died\\ because they received a COVID-19 vaccine.\\ No evidence that anyone has died from \\ complications related to a trial COVID-19.\end{tabular} & \begin{tabular}[c]{@{}l@{}}There is no evidence that children have \\ died because they received one of the \\ COVID-19 vaccines being developed.\\ PolitiFact found no evidence that anyone\\ has died from complications related to a \\ trial COVID-19 vaccination.\end{tabular} \\ \bottomrule
\end{tabular}%
}
\caption{Illustrative example of our approach pipeline shown as Claim $\xrightarrow{}$ Evidence $\xrightarrow{}$ Summarization-1 $\xrightarrow{}$ Summarization-2, where Summarization-2 is obtained after fine-tuning T5 language model, and used as \textbf{evidence input} for classification}
\label{tab:summary}
\end{table*}

\subsection{Dataset Preprocessing}
To map claims with evidence, we pre-process both the dataset and the evidence collected from external sources. Following are the details of the pre-processing steps: \textbf{(1) URL Mapping:} We observe that some posts contain URLs in a masked form, \textit{e.g.,} https://t.co/z5kk XpqkYb. Our approach extracts these URLs using a regular expression-based match and maps them to the original URL using the python `\textit{requests}' library. Any additional information from the URL is removed, and only appropriate URLs remain in the text. For example, https://t.co/z5kkXpqkYb  $\xrightarrow{}$  https://www.cdc.gov/; \textbf{(2) Special symbols:} We removed extra white-spaces, special symbols and brackets like \^ , (, ),  \{,\}; \textbf{(3) Hashtags, Emojis and Mentions:} Additionally, we remove hashtags and replace it with the token ``HASHTAG:''.

For example, \#COVID-19 becomes HASHTAG:COVID-19. Similarly, we also replace mentions ``@'' with ``MENTION:'' token. At the end, we convert emojis to their text form using the `demoji' library\footnote{\href{https://github.com/bsolomon1124/demoji
}{GitHub: Demoji}}; \textbf{(4) Lowercasing:} Eventually, we lowercase the claim and the evidence text to obtain the input data used for the next Summarization step.

\subsection{Summarization of Evidence}
Our pre-processed evidence for claims in many cases were multiple paragraphs resulting in performance degradation. Therefore, we propose the addition of a summarization component to our pipeline which utilizes state-of-the-art Text-to-Text Transfer Transformer (T5) language model~\citep{DBLP:journals/corr/abs-1910-10683} for the inherent summarization task\footnote{This language model can perform the summarization task with the help of a prefix ``summarize'' to the input text provided.}. Due to the nature of the usual summarization task input, a large body of text (full documents), we believe that our comparatively short paragraphs would be better summarized. This language model is fine-tuned for the task of summarization helps us obtain a summarized text for each piece of evidence resulting in what we call Summarization-1 or S1. An output obtained is shown in Table~\ref{tab:summary}.

\subsubsection{Fine-tuning T5 on FEVER Dataset}
As an additional experimental step, we further fine-tune the Text-to-Text Transfer Transformer (T5) model using the original FEVER dataset~\citep{thorne-etal-2018-fever}. The original T5 summarization model is trained on the CNN/Daily Mail~\cite{DBLP:conf/nips/HermannKGEKSB15} data where the input is the news article text, and the objective is to highlight summarized text as the output. The T5 is an encoder-decoder model pre-trained on a multi-task mixture of unsupervised and supervised tasks and for which each task is converted into a text-to-text format. This allows for the use of the same model, loss function, hyperparameters, \textit{etc.} across our diverse set of tasks. T5 works well on a variety of tasks out-of-the-box by prepending a different prefix to the input corresponding to each task, e.g., for the task of translation$\xrightarrow{}$ \textit{translate English to German: $<$English Sentence$>$}, for the task of summarization$\xrightarrow{}$ \textit{summarize: $<$English Text$>$}.

For our experiments, the aim is to summarize the pre-processed evidence while including the claim. Thus, we hypothesize that fine-tuning on an auxiliary dataset will improve the quality of the generated summary. For fine-tuning, we use the same hyperparameters as described in their paper to generate another model. We perform another iteration of the summarization step using this fine-tuned model to generate a parallel set of evidence and label the output as Summarization-2 or S2 as shown in Table~\ref{tab:summary}. Further, we provide the details of \textbf{as the classification task, which uses either S1 or S2 as evidence input to classify} the claims as real or fake (Figure~\ref{fig:arch}).

\begin{table*}[t!]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{|l|c|c|l|l|l|l|l|l|l|l|l|l|l|l|l|l|l|l|l|l|}
\hline
 & \multicolumn{2}{c|}{Previous Approaches} & \multicolumn{9}{c|}{\textbf{Our Approach w/ various Classification methods}} \\ \hline
\multirow{2}{*}{} & \multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}~\citet{10.1007/978-3-030-73696-5_9}\end{tabular}} & \multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}~\citet{10.1007/978-3-030-73696-5_11}\end{tabular}} & \multicolumn{3}{c|}{Logistic Regression} & \multicolumn{3}{c|}{SVM} & \multicolumn{3}{c|}{LSTM} \\ \cline{4-12}
 &  &  & \multicolumn{1}{c|}{-} & \multicolumn{1}{c|}{S1} & \multicolumn{1}{c|}{S2} & \multicolumn{1}{c|}{-} & \multicolumn{1}{c|}{S1} & \multicolumn{1}{c|}{S2} & \multicolumn{1}{c|}{-} & \multicolumn{1}{c|}{S1} & \multicolumn{1}{c|}{S2} \\ \hline
\textbf{P} & 0.9902 & 0.986 & 0.9531 & 0.9565 & 0.9701 & 0.9641 & 0.9671 & 0.9764 & 0.9589 & 0.9598 & 0.9612  \\ \hline
\textbf{R} & 0.9901 & 0.985 & 0.9531 & 0.9564 & 0.9700 & 0.9639 & 0.9668 & 0.9761 & 0.9584 & 0.9596 & 0.9612  \\ \hline
\textbf{F} & 0.9901 & 0.985 & 0.9531 & 0.9565 & \underline{0.9700} & 0.9639 & 0.9668 & \underline{0.9761} & 0.9584 & 0.9596 & \underline{0.9612}  \\ \hline
\end{tabular}%
}
\caption{Results obtained after the fake news classification task where the values for previous approaches are from the latest shared task results and the results for each iteration of our approach are shown [P (Precision), R (Recall), and F (F-Score)]. (-) $\xrightarrow{}$ No Evidence, S1 $\xrightarrow{}$ Summarization-1 as Evidence, S2 $\xrightarrow{}$ Summarization-2 as Evidence.}
\label{tab:results}
\end{table*}

\begin{table*}[t!]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{|l|l|l|l|l|l|l|l|l|l|}
\hline
 & \multicolumn{9}{c|}{\textbf{Our Approach w/ various Deep Learning Classification methods}} \\ \hline
\multirow{2}{*}{} & \multicolumn{3}{c|}{BERT$_{base}$} & \multicolumn{3}{c|}{RoBERTa$_{base}$} & \multicolumn{3}{c|}{XLNet$_{base}$} \\ \hline
 &  \multicolumn{1}{c|}{-} & \multicolumn{1}{c|}{S1} & \multicolumn{1}{c|}{S2} & \multicolumn{1}{c|}{-} & \multicolumn{1}{c|}{S1} & \multicolumn{1}{c|}{S2} & \multicolumn{1}{c|}{-} & \multicolumn{1}{c|}{S1} & \multicolumn{1}{c|}{S2} \\ \hline
\textbf{P} & 0.9612 & 0.9916 & 0.9917 & 0.9918 & 0.9929 & 0.9922 & 0.9920 & 0.9934 & 0.9947 \\ \hline
\textbf{R} & 0.9864 & 0.9888 & 0.9897 & 0.9897 & 0.9911 & 0.9916 & 0.9892 & 0.9911 & 0.9925 \\ \hline
\textbf{F} & 0.9858 & 0.9888 & \underline{0.9893} & 0.9893 & 0.9908 & \underline{\textbf{0.9908}} & 0.9892 & 0.9910 & \underline{\textbf{0.9925}} \\ \hline
\end{tabular}%
}
\caption{Results obtained after the fake news classification task where the results for each iteration of our approach with various deep learning classification methods are shown [P (Precision), R (Recall), and F (F-Score)]. (-) $\xrightarrow{}$ No Evidence, S1 $\xrightarrow{}$ Summarization-1 as Evidence, S2 $\xrightarrow{}$ Summarization-2 as Evidence.}
\label{tab:results}
\end{table*}

\section{Experiment Setup}
In this section, we discuss the experiment setup in detail. We perform the task of fake news detection as a binary classification task in a supervised setting. We choose to perform our experiments with both conventional machine learning- and deep learning- based classifiers. From the machine learning-based approaches, we choose Logistic Regression (LR) and Support Vector Machines (SVM) with the GridSearch implementation for best results over multiple hyperparameters (values of c, different kernels, \textit{etc.}) We also utilize LSTMs with various contextual language models from the deep learning methods. From the deep learning-based approaches, we use a simple LSTM implementation with pre-trained GloVE\footnote{https://nlp.stanford.edu/projects/glove/} vectors, BERT$_{base}$, RoBERTa$_{base}$, and XLNET$_{base}$ -based classifiers. Our LSTM implementation uses \textit{Adam} optimizer with a learning rate of 0.001, and 256 as the batch size. For classifiers based on BERT$_{base}$, RoBERTa$_{base}$, and XLNET$_{base}$, we use the HuggingFace implementations with a batch size of 32, L2 regularization and cross-entropy loss. The regularization parameter $\lambda$ was set to 0.1. Each classification method is iterated \textbf{(1)} without evidence (-), \textbf{(2)} with augmented summarized evidences from S1, \textbf{(3)} and then with S2, thus giving us three sets of results for each method; as shown in Table~\ref{tab:results}.

As an input to the classifier, we use the claim as-is from the dataset as described above. We have a dataset $D$ $= (x_n, y_n)_{n=1}^{N}$ comprising of $N$ training samples. Here $x_n = (c_n, e_n)$, where $c_n$ represents the claim, and $e_n$ represents evidence gathered using our approach. $X\in \mathcal{X}$ is defined on input space, and $Y\in\mathcal{Y} = \{0,1\}$ are the corresponding labels. Thus, given a claim $c$ and evidence $e$, the aim of this task is to train a classifier such that the claim $c$ is predicted as fake news or not, i.e $F_{\theta}:\mathcal{X} \rightarrow \mathcal{Y} \in\{0, 1\}$.
  \begin{equation}
    F(c, e;\theta)=
    \begin{cases}
      1, & \text{if}\ c \ is  \ the \ fake \ news \\
      0, & \text{otherwise}
    \end{cases}
  \end{equation}

where $F(c, e)$ is the function our model aims to learn over each iteration or epoch.

\section{Results and Discussion}

\begin{table*}[!ht]
\centering
\begin{tabularx}{\linewidth}{|>{\raggedright\arraybackslash}X|c|c|c|} \hline
\textbf{Text} & \textbf{XLNet} & \textbf{LR} & \textbf{SVM} \\ \hline
We always appreciate questions about the quality of our data. If you see a number that doesn't look right please file an issue at and we will investigate.  [SEP] SOURCES: github.com & \xmark & \checkmark & \checkmark\\\hline

The number of daily tests has been increasing in a steep climb. Average daily tests during the past three weeks also strongly depict the progress made in enhancement of \#COVID19 tests across the country.  [SEP] SOURCES: twitter.com/MoHFW\_INDI & \xmark & \checkmark & \checkmark \\\hline
Bill Gates said thousands of people will die with the COVID-19 vaccine  [SEP] SOURCES: & \xmark & \xmark & \xmark  \\ \hline
\end{tabularx}
\caption{Qualitative error analysis of some output cases both in terms of successes and failures of our approach.}
\label{table:error}
\end{table*}

The results for our classification task are shown in Table~\ref{tab:results}. Using our approach, we are able to marginally outperform (+0.24, F-Score) the previous state-of-the-art (SoTA) approaches for the task of fake news detection as shown in the last column (XLNet$_{base}$, S2). Even the RoBERT$_{base}$ model is able to outperform the SoTA approaches by a small margin. We present the values of our top two best models in boldface in Table~\ref{tab:results}. Although the improvement margin is small, we would like to note that the previous SoTA approaches are already performing at almost a 0.99 F-score. \textit{We executed our model run multiple times to ensure that our improvement margin is indeed truly obtained.} We also observe that RoBERTA$_{base}$, and XLNet$_{base}$ outperform the SoTA approaches (Chen et. al. / Li et. al.) even with S1 summarization component. Classical machine learning-based approaches are also shown to perform very well for this task as the scores of 0.96 can be considered to be a good performance for any classification method.

However, this is not the only key takeaway from these results. \textit{We observe that by using our novel approach, a consistent improvement is seen in the task results}. The efficacy of our approach can be seen from Table~\ref{tab:results}, as either S1 or S2 consistently outperforms all the base models (-) [no evidence] in the table.  Moreover, using our approach, we are able to gather key evidence for such a dataset where, to begin with, only claims were present with manually annotated labels. Table~\ref{table:error} illustrates the success and failure cases from XLNet, Logistic Regression and Support Vector Machines. We observe that first two cases were incorrectly predicted by the XLNet but were predicted correctly by LR and SVM. Last case was predicted incorrectly by all of the models. We believe that the absence of source in the text could be a potential reason for this failure. Our approach can gather the evidence using a fully automated method with summarization component(s) in the pipeline. The importance of this component can be gathered from manual observations of examples in the augmented dataset. We observe that summarized evidences shorten the length of the evidences, which helps the Transformer architecture-based classifiers like BERT$_{base}$, RoBERTa$_{base}$, XLNet$_{base}$ perform better. These pre-trained models have a token length limitation of 512 tokens which is easily able to capture our summarized evidence. We also manually observe that the summarization component helps reduce redundancy in the generated sentences and removes duplicates. Hence, improving the quality of evidence used as additional input helps reduce the training time. The performance of our models with the fine-tuned summarization component (S2) seems to perform better than S1, and the model without any evidence, as can be seen in Table~\ref{tab:results}.

We acknowledge that the CONSTRAINT dataset is saturated in terms of possible improvements. However, with this paper, \textit{our aim is to show the efficacy of our summarization technique which can help the evidence detection for news}. We chose this dataset at an early stage of our work, and our experiments do show that improvements can, in fact, still be shown on this dataset. Our best-performing system surpasses the state-of-the-art by 0.23\% points.

\section{Conclusion and Future Work}
In this paper, we present an automated method to collect evidence for the fake news detection task. We use our novel approach to augment the dataset, released in the CONSTRAINT-2021 Shared Task, with evidence sets collected from the web. Our method helps process these evidence sets, clean them and use them to generate summarized evidence based on two different methodologies. We use either of the summarized evidence as an additional input to the fake news classification task and perform an evaluation of our approach. We discuss the results of the classification task and conclude that our approach helps outperform the previous SoTA approaches by a small margin, however, helping generate evidence for a crucial dataset. We show that a summarization module can help collect evidence more effectively. We augment this dataset with the summarized evidence and release it along with the code and generated models for further research. We would also like to conclude that our method is generalizable; since it uses pre-trained metrics (BERTScore) and models (T5), it can be used to gather evidence for other datasets. The overall pipeline is also not very time-consuming (2 seconds per sample) once fine-tuned models are included in it. We hope our method and the resources are helpful to the NLP community.

In future, we would like to use our method to gather evidence for other fact detection/verification datasets as well. Our initial aim is to reproduce this study with other datasets and ensure that our method performs well in a real-world scenario. We would also like to apply this method and gather further evidence for existing fake news datasets, and perform our experiments to evaluate this approach over multiple exisiting datasets, including existing multilingual datasets.


\section{INTRODUCTION}
The COVID-19 pandemic has revealed the need for non-pharmaceutical interventions such as lockdowns to mitigate the spread of novel diseases, when cures and vaccines are not available. Unfortunately, while very effective, such interventions are very costly from an economic perspective. A wide variety of strategies were adopted all over the world, implementing different tradeoffs between economic and epidemic cost. In general, this class of epidemic control strategies can be classified in two main categories: \emph{suppression strategies}, which “aim to reverse epidemic growth, reducing case numbers to low levels and maintaining that situation indefinitely", or
\emph{mitigation strategies}, which “focus on slowing but not necessarily stopping epidemic spread, reducing peak healthcare demand" while
minimizing the negative effects on economic activities \cite{ferguson2020report}.

We here aim at understanding better the trade-off in terms of economic and epidemic cost  of lockdown strategies on epidemic spread within a standard susceptible-infected-recovered (SIR) model \cite{kermack1927contribution}. We study the optimal lockdown from an optimal control perspective as first analyzed in \cite{behncke2000optimal}, with an objective that takes into account the epidemic cost associated with deaths and the economic cost due to lockdowns, as suggested in \cite{alvarez2020simple,acemoglu2020multi}. The cost takes into account hospital congestion, which appeared to be a key feature of the recent pandemic, by assuming that the lethality due to the disease is an increasing affine function of the fraction of infected $i$,  leading to a quadratic mortality rate in $i$. To the best of our knowledge, no analytical solutions to optimal control problems with quadratic cost in $i$ are provided in the literature \cite{nowzari2016analysis}. While in \cite{alvarez2020simple,acemoglu2020multi} numerical solutions are provided, in \cite{kruse2020optimal} authors show that if the cost is linear in the infected, i.e., there is no hospital congestion, and convex in the intervention, the optimal policy is quasi-convex. In  \cite{miclo2020optimal} the authors consider the problem of minimizing the cost of the intervention subject to the constraint that the fraction of infected is below a certain threshold, i.e., $i(t) \le \overline{i}$ for every time $t$,  where the value of $\overline{i}$ is based on hospitals capacity. For the case of infinite time horizon $T$, the authors prove analytically that in a first phase it is optimal to let the epidemic spread, until $i(t)=\overline{i}$; then the optimal lockdown is such that the infected remain constant at the threshold, and is released when herd immunity is reached, i.e., when the fraction of people that contracted the disease is large enough to guarantee that, from there on, the number of infected people will decrease even without lockdown. While \cite{miclo2020optimal} handle hospital congestion by a hard constraint on the simultaneous fraction of infected, without any cost on the number of deaths, our formulation considers a quadratic cost in $i$ without hard constraints. A feedback control with a hard constraint of the simultaneous fraction of infected is also proposed in \cite{di2020covid}. Optimal control approaches to containment of COVID-19 pandemic using more detailed compartmental models are also studied numerically in \cite{djidjou2020optimal,kantner2020beyond}.

The contribution of our work is two-fold. We first show numerically that as $T$ increases and the trade-off between economic and epidemic cost vary, the optimal control captures the fundamental difference between suppression and mitigation, exhibiting a phase transition between solutions that avoid diffusion of the disease, and solutions that steer the system to herd immunity while maintaining the number of infected low to mitigate the negative effects of hospital congestion. Our second main contribution is to consider two simple feedback policies, where lockdown levels are selected to either stabilize the reproduction number $R(t)$ (i.e., reduce the number of secondary infections to  ``flatten the curve'') or the fraction of infected (i.e., containing the number of hospitalizations). We note that the policy stabilizing $i(t)$ resembles the optimal control in \cite{miclo2020optimal}, with the only difference that in \cite{miclo2020optimal} the threshold depends on hospitals capacity, while in our formulation the threshold is a free parameter that can be optimized to reduce the cost. We analytically compute the exact cost of these policies, and provide an upper bound to the performance gap between the two policies and the optimal control. In particular, we show that if $T$ is infinite, controlling $i(t)$ is better than controlling $R(t)$ and is close to optimal. Together with its close-to-optimal performance, the policy that controls the number of infected is in feedback form and easy to interpret, in contrast with numerical solutions of optimal control. As a corollary, our result shows that the spectral control approach  (see e.g., \cite{nowzari2016analysis, birge2020controlling}), where typically one aims at selecting the minimal intervention guaranteeing $R(t)<1$ is not optimal in the long run when i) the cost of the intervention has to be paid repeatedly in time (e.g., lockdown are different than vaccinations since if lockdown are interrupted the reproduction number will immediately go back to above 1, while after a vaccination campaign,  assuming permanent immunization, the  system will be stable forever on) and ii)  the horizon is long or the economic cost associated with the intervention is comparable with the epidemic cost. We remark that our theoretical analysis is restricted to the case of infinite time horizon and does not consider pharmaceutical interventions that may be implemented when cures and vaccines become available. Furthermore, we assume that individuals who contracted the infection become immune from there one. We emphasize that our purpose is not to suggest specific control strategies for the COVID-19 pandemic, but to provide general insights and mathematical contribution on the control of epidemics
\section{Model and problem formulation}
\label{sec:model}
To capture the effect of lockdown interventions, we consider the following simple SIR model introduced in \cite{alvarez2020simple}:
\begin{equation}\label{eq:sir}
	\begin{cases}
		\dot{s}=-\beta {(1-\theta L)^2}si\\
		\dot{i}=\beta{(1-\theta L)^2} si-\gamma i\\
		\dot{r}=\gamma i,
	\end{cases}
\end{equation}
where $s/i/r$ is the fraction of susceptible/infected/recovered agents, with initial condition $s_0/i_0/r_0$, $\beta$ is the infection rate, $\gamma$ is the recovery rate, $L \in [0,\overline{L}]$ is the percentage of population in lockdown and $\theta$ is the effectiveness of the lockdown (i.e. even if $L=1$ only $\theta$ percent of the population actually complies with the lockdown) and the dependence of $s, i, r, L$ on $t$ is omitted for convenience of notation. While the transmission mechanisms of the SARS-CoV-2 are very complicated (see e.g. \cite{sun2021transmission}), we assume that the lockdown $L$ enters quadratically in the dynamics, because people get the disease through pairwise interactions, (see \cite{alvarez2020simple,acemoglu2020multi}). When considering lockdown policies, one needs to consider not only the epidemics dynamics, but also the economic cost associated with such interventions. To this end, the cost of the lockdown policy can be divided into an \textit{economic cost} (due to loss of workforce because of the lockdown) which we model as
$ cost_{eco}(t)=L(t)$,
and an  \textit{epidemic cost} (related to number of fatalities) which following \cite{alvarez2020simple} and \cite{acemoglu2020multi} we model as
$ cost_{epi}(t)= \kappa \gamma_d(i(t))i(t),$
where $\gamma_d(i(t))i(t)$ is the mortality rate and $\kappa$ is the trade-off between economic and epidemic cost. We assume that the mortality  $\gamma_d(i)=\gamma_0+\gamma_1 i$ is increasing in $i$ to take into account hospitals congestion, as proposed in \cite{alvarez2020simple,acemoglu2020multi}.
\subsection*{Reproduction number}
The behaviour of the SIR dynamics is characterized by the reproduction number $R(t):=\beta s(t) /\gamma$, which is the number of secondary infections that an infected person produces on average. If $R(t)<1$, the epidemic is stable in the sense that the number of infected decreases indefinitely, while if $R(t)>1$ the number of infected increases until the first time $t^*$ such that $R(t^*)=1$. In presence of control, i.e., when $L(t)>0$, the infection rate and therefore the reproduction number are reduced. We let $R_L(t):=\beta(1-\theta L)^2/\gamma$ denote the \emph{controlled reproduction number} and $R(t)$ denote the \emph{uncontrolled reproduction number}.
\subsection*{Optimal control}
Let $c(t):=cost_{eco}(t)+cost_{epi}(t)$.
The objective of the planner is to find the optimal control $L^*(t)$ solving
\begin{equation}\label{eq:cost}
	\begin{aligned}
	L^* = \underset{L:[0,T] \to [0,\overline{L}]}{\arg\min} & \quad  \int_0^T c(t) dt \\
	\textup{s.t.} & \quad \textup{dynamics in }\eqref{eq:sir},
	\end{aligned}
\end{equation}
where $T$ is the time horizon,
We start by noting that, using similar techniques as in \cite{behncke2000optimal}, one can show that the optimal lockdown $L^*(t)$ solution to the optimal control problem \eqref{eq:cost} is an increasing function of the force of infection $f:=\beta s i$. It is remarkable to note that, although the stability of the system does not depend on $i$, the lockdown policy does, through $f$. Thus, even if the system is highly unstable (i.e.,
$R(t)=\beta s(t)/\gamma \gg 1$) if $i(t) \rightarrow 0$ then $L^*(t) \rightarrow 0$, which means that the optimal lockdown lets the infected grow when the number of infected is low.
Besides this characterization, an analytic solution to \eqref{eq:cost} remains open. Numerical solutions to optimal control problems such as \eqref{eq:cost} have been investigated, but typically suffer from lack of interpretability and robustness guarantees, as they are not in feedback form, i.e., the policy does not depend explicitly on the state. As an alternative, in the next section we propose two simple feedback policies based on controlling the reproduction number (which is typically considered in spectral control \cite{nowzari2016analysis}) or controlling the total number of infected (to reduce congestion of the health system).
These strategies, or their combination, have informed the response to COVID-19 of governments and public/private entities (e.g., Cornell University \cite{cornell}, USA \cite{USA} and Italy \cite{Italy}).

\section{Two feedback policies}\label{sec:heuristic}
We here describe the two feedback policies. Their performance are studied in Section \ref{sec:results}. To simplify the exposition we set $\theta=1$ and $\overline{L}=1$, but  all results can be  generalized.
\subsection{Policy A: Controlling reproduction number}
This policy aims at controlling the reproduction number, as typically considered in spectral control (see e.g. \cite{nowzari2016analysis}). Given a target reproduction number $\rho$, if the uncontrolled reproduction number $R(t)$ is greater than $\rho$, i.e. $\beta s(t) /\gamma > \rho$, a lockdown level $L_\rho(t)>0$ such that $R_L(t)=\rho$ is imposed, leading to $L_\rho$ satisfying
	\begin{equation}\label{eq:r0_const}
		R_L(t)= \frac{\beta}{\gamma}(1-L_\rho(t))^2s(t) =\rho.
	\end{equation}
If instead $R(t)\le \rho$, then $L_\rho(t)=0$.\\
   We can divide the dynamics in two phases:
	\begin{enumerate}
		\item Phase I: in $[0,t^*)$ with $t^*$ such that $s(t^*)=\gamma \rho / \beta$. In this phase $\beta  s /\gamma\ge \rho $, $L_\rho$ is as in \eqref{eq:r0_const} and the dynamics are
		\begin{equation*}
			\label{eq:part1rho}
				\dot{s}=-\rho \gamma i, \quad
				\dot{i}=\rho \gamma i -\gamma i = (\rho-1)\gamma i, \quad
				\dot{r}=\gamma i.
		\end{equation*}
		\item Phase II: in $[t^*,T]$. Since $R(t)\le\rho$, $L_\rho=0$, the dynamics is uncontrolled and given by
	\begin{equation*}
			\dot{s}=-\beta  si, \quad
			\dot{i}=\beta si -\gamma i, \quad
			\dot{r}=\gamma i.
	\end{equation*}
	\end{enumerate}
	Fig. \ref{fig:control_R0} illustrates the dynamics under Policy A.
	Notice that Phase I may not exist if $\beta  s_0 /\gamma \le \rho$, and Phase II may not exist if $\beta s(T)/\gamma>\rho$ (this scenario typically occurs when $\rho<1$ because the infected decay exponentially and the disease does not spread). Policy A is parametric in $\rho$, thus a policy-maker that adopts this policy should select $\rho^*$ to minimize the total cost (see Fig. \ref{fig:control_R0}), leading to
	\begin{equation*}\label{eq:r}
	\begin{aligned}
		\rho^*:=\arg\min_\rho & \quad \int_0^T c(t) dt \\
		s.t & \quad  \quad \eqref{eq:sir} \textup{ with } L(t)=L_\rho(t).
	\end{aligned}
	\end{equation*}
\begin{figure}
	\centering
\includegraphics[width=4.2cm]{1f.png}
\includegraphics[width=4.2cm]{cost_vs_rmax_f.png}
\caption{Policy A. \emph{Left}: Dynamics with $\rho=1.2$. Phase I ($L_\rho(t)>0$) lasts until $\beta s /\gamma = \rho$ (red line); then, the trajectory becomes uncontrolled (Phase II). Infected grows until the reproduction number becomes less than 1. The unit of time is one day. \emph{Right}: optimization over $\rho$. The parameters used in this simulation, and are $\beta=0.2$, $\gamma=1/18$, $\gamma_0=5.6 \cdot 10^{-4}$, $\gamma_1=5.6 \cdot 10^{-3}$ based on the COVID-19 pandemic, see \cite{alvarez2020simple, acemoglu2020multi}), $s_0=0.98$, $i_0=0.01$, $r_0=0.01$, $\kappa=40\cdot365$, $T=5$ years. For the used parameters $\rho^*>1$, meaning that the total cost is minimized with a mitigation policy that lets the infection spread in a controlled way. \label{fig:control_R0}
\end{figure}
\subsection{Policy B: controlling fraction of infected}
As anticipated in Section \ref{sec:model}, optimality conditions suggest that the optimal quantity to be controlled is the force of infection \cite{behncke2000optimal}. Stabilizing this quantity, i.e., imposing lockdown $L_f(t)$ such that that the $\beta(1- L_f(t))^2 s(t) i(t) = \phi$ yields the dynamics
\begin{equation*}
	\label{eq:dyn_force}
	\dot{s}=-\phi, \quad
	\dot{i}=\phi -\gamma i, \quad
	\dot{r}=\gamma i,
\end{equation*}
which stabilizes asymptotically $i(t)$ to $\phi/\gamma$. Based on this observation, instead of controlling the force of infection, we here propose a policy that stabilizes the number of infected $i(t)$ (as the optimal policy in \cite{miclo2020optimal}), whose meaning is more intuitive. Given a target value $\iota$, policy B works as follows: if $i_0<\iota$, $L_\iota(t)=0$ until the first time $\tau_1$ such that $i(\tau_1)=\iota$; if $i_0>\iota$, $L_\iota(t)=1$ until $\tau_1$; then, impose $L_\iota(t)$ in such a way that $\dot{i}(t)=0$, hence $i(t)=\iota$, until time $\tau_2$ such that herd immunity is reached, i.e., $s(\tau_2)=\gamma/\beta$; finally, the lockdown is released, i.e., $L_\iota(t)=0$ (see Fig. \ref{fig:control_i}). Given $\iota$, we divide the trajectory in three phases:
\begin{enumerate}
	\item Phase I in $[0,\tau_1)$, with $\tau_1$ being the smallest time such that $i(\tau_1)=\iota$. If $i_0 < \iota$  $L_\iota(t)=0$ and the dynamics is
	\begin{equation*}
		\dot{s}=-\beta s i, \quad
		\dot{i}=\beta s i -\gamma i, \quad
		\dot{r}=\gamma i,
\end{equation*}
If $i_0 > \iota$, $L_\iota(t)=1$, with dynamics
		\begin{equation*}
		\dot{s}=0, \quad
		\dot{i}= -\gamma i, \quad
		\dot{r}=\gamma i.
\end{equation*}
	\item Phase II in $[\tau_1,\tau_2)$, with $\tau_2$ such that $s(\tau_2)=\gamma/\beta$. In this phase $i(t)=\iota$, and the dynamics is
	\begin{equation*}
		\dot{s}=-\gamma \iota, \quad
		\dot{i}=0, \quad
		\dot{r}=\gamma \iota.
\end{equation*}
with
\begin{equation*}
	L_\iota(t) \ \text{s.t.}\ \beta s(t) (1-L_\iota(t))^2 = \gamma.
\end{equation*}
	\item Phase III in $[\tau_2,T]$: the dynamics is uncontrolled and given by
	\begin{equation*}
		\dot{s}=-\beta s i, \quad
		\dot{i}=\beta s i -\gamma i,  \quad
		\dot{r}=\gamma i.
	\end{equation*}
\end{enumerate}
Notice that in Phase III $\dot{i}(t)<0$ even without control because herd immunity is reached.
In contrast with \cite{miclo2020optimal}, where the value of $\iota$ depends on the ICU capacity and the policy-maker has to choose the optimal policy complying with the constraint $i(t)\le \iota$, here for any value of $\iota$ the policy is given, and the policy-maker should select $\iota^*$ to minimize the total cost \eqref{eq:cost} (see Fig. \ref{fig:control_i}), leading to
\begin{equation*}\label{eq:i}
	\begin{aligned}
		\iota^*:=\arg\min_\iota & \quad \int_0^T c(t) dt \\
		s.t & \quad  \quad \eqref{eq:sir} \textup{ with } L(t)=L_\iota(t).
	\end{aligned}
\end{equation*}
\begin{figure}
	\centering
	\includegraphics[width=4.2cm]{4f.png}
	\includegraphics[width=4.2cm]{cost_vs_imax_f.png}
	\caption{Policy B. \emph{Left}: Dynamics with $\iota=0.06$. In Phase I the trajectory is uncontrolled ($L(t)>0$) and the fraction of infected grows (Phase I); when $i(t)$ reaches the threshold $\iota$, lockdown $L(t)$ is imposed in such a way to maintain the fraction of infected constant (Phase II). When herd immunity is reached, the lockdown is released and infected start decreasing (Phase III). \emph{Right}: Optimization over $\iota$. The parameters are as in Fig. \ref{fig:control_R0}. \label{fig:control_i}}
\end{figure}
\section{Results}\label{sec:results}
This section is divided in two parts. In the first part, some observations on numerical solutions are presented. In the second part, we provide an upper bound on the performance gap between the feedback policies presented in Section \ref{sec:heuristic} and the optimal control \eqref{eq:cost}, and show that, some under assumptions, controlling the fraction of infected is close to optimal and outperforms the policy based on controlling the reproduction number.
\subsection{Numerical results}
\begin{figure}
	\centering
	\includegraphics[width=4.2cm]{plot_pareto_chi_N2.png}
	\includegraphics[width=4.2cm]{plot_pareto_T_N2.png}
	\caption{\emph{Left}: Economic loss, defined as $\frac{\int_0^T L(t)dt}{T} $, vs epidemic cost, i.e., fraction of population died because of the disease under the optimal control, as the trade-off between economic and epidemics cost $\kappa$ vary ($T=1.5$ yrs). \emph{Right}: same with $T$ varying, ($\kappa=20 \cdot 365$), and economic cost defined as $\frac{\int_0^T L(t)dt}{1.5 \text{yrs}}$. Other parameters are as in Fig. \ref{fig:control_R0}.}
		\label{fig:phase}
\end{figure}
Our first result is that numerical solutions of the optimal control exhibit a phase transition from suppression to mitigation, as described in \cite{ferguson2020report}, as $T$ increases and $\kappa$ decreases. In the left panel of Fig. \ref{fig:phase}, economic and epidemic cost as their trade-off $\kappa$ vary are computed, while other parameters (including $T$) are fixed. There is a clear phase transition: when $\kappa$ is small, the optimal policy lets the epidemic spread (high number of deaths, small economic cost), whereas when $\kappa$ is large optimal policy maintains the lockdown until the horizon (high economic cost and small epidemic cost). Solutions in between these two strategies are never optimal. The right plot shows that a similar phase transition occurs with $\kappa$ constant and varying $T$. Despite the economic and epidemic cost exhibiting a phase transition, we emphasize that the total cost is continuous in the parameters.
\begin{remark}
	We also observe that similar phase transition occurs for Policies A and B, namely if $\kappa$ is large and $T$ is small, the optimal reproduction number for Policy A satisfies $\rho^*<1$, implying that $i(t)$ decreases, and under Policy B the optimal $\iota^*$ is very small, whereas, if $\kappa$ is small and $T$ is large, the optimal reproduction number satisfies $\rho^*>1$, the optimal $\iota^*$ is much larger, and the epidemic cost is in general comparable with the economic cost.
\end{remark}\medskip
This confirms our intuition that solutions based on spectral control problems, that minimize statically the cost of intervention subject to the constraint $R_0<1$, are suboptimal from an optimal control perspective when $T$ is large or  $\kappa$ is not too high, since stabilizing the system indefinitely leads to a prohibitively
 high economic cost. As shown in Fig. \ref{fig:suppression_mitigation}, in the range of parameters that make the optimal control select suppression strategies, the optimal control and the feedback policies achieve similar performances, whereas when the optimal control selects mitigation, Policy B is close to optimal and outperforms Policy A. We next investigate these observations.
\subsection{Theoretical results}
\label{sec:results}

\begin{table*}	\caption{Analytic expressions of the cost under the two heuristics}
\vspace{-0.3cm}\hrule
	\begin{equation}\label{eq:t*}
			\begin{aligned}
			&C_A(\rho
			=(t^*-T_1)+\frac{\kappa \gamma_0}{\gamma} (r(\infty)-r_0)
				+\frac{\kappa \gamma_1 }{2(\rho-1)\gamma} \left(i(t^*)^2 -i(0)^2\right)
				+\kappa\gamma_1\bigg(\gamma\frac{[\ln(s(t^*))]^2-[\ln(s(\infty))]^2}{2\beta^2}-\frac{s(t^*)-s(\infty)}{\beta} +\big(1-r(t^*)-\frac{\gamma}{\beta}\ln (s(t^*))\big)\frac{r(\infty)-r(t^*)}{\gamma}\bigg),\\
			&	\textup{with}\quad  t^*=\frac{ ln\left( 1 - \frac{\rho\gamma-\beta s_0 }{\beta\nu i_0}\right)}{(\rho-1)\gamma}, \
	\nu = \frac{\rho}{\rho+1}, \ \tilde{\nu}=(s_0+\nu i_0), \
	s(t^*)=\frac{\gamma \rho}{\beta},  \ r(t^*)=1-\frac{s(t^*) + \tilde{\nu}}{\nu},\\&\qquad
	T_1=2\sqrt{\frac{\rho }{\beta \tilde{\nu}(\rho-1)^2\gamma }}  { \left(     \at\left(\sqrt{\frac{s_0}{\tilde{\nu}}}\right) - \at\left(\sqrt{\frac{ \rho\gamma}{\beta\tilde{\nu}}}\right)    \right) }, \
	r(\infty)=1-s(\infty)=1-s(t^*) e^{-\frac{\beta}{\gamma} (r(\infty)-r(t^*))}
			\end{aligned}
		\end{equation}
		\hrule
		\vspace{0.2cm}
		\begin{equation}\label{eq:cost_B}
		\begin{aligned}
		&C_B(\iota
		=\kappa\gamma_1\Bigg(\gamma\frac{[\ln s_0]^2-[\ln s(\tau_1)]^2}{2\beta^2}-\frac{s_0-s(\tau_1)}{\beta} +\big(s_0+i_0-\frac{\gamma}{\beta}\ln s_0\big)\frac{i_0+s_0-\iota-s(\tau_1)}{\gamma}+(\tau_2-\tau_1)\iota^2+\gamma\frac{[\ln \frac{\gamma}{\beta}]^2-[\ln s(\infty)]^2}{2\beta^2}-\frac{\frac{\gamma}{\beta}-s(\infty)}{\beta} +\\
		&+\big(\frac{\gamma}{\beta}+\iota-\frac{\gamma}{\beta}\ln \frac{\gamma}{\beta}\big)\frac{r(\infty)-1+\iota+\frac{\gamma}{\beta}}{\gamma}\Bigg) + \frac{1}{\iota}\bigg(\sqrt{\frac{s(\tau_1)}{\gamma}}-\sqrt{\frac{1}{\beta}}\bigg)^2+\frac{\kappa\gamma_0}{\gamma} (r(\infty)-r_0)\\
		&\textup{where } s(\tau_1) \textup{ and } r(\infty) \textup{ are the unique feasible solutions of}\\& \quad \ln\Big(\frac{s(\tau_1)}{s_0}\Big)=\frac{1}{\gamma}[s(\tau_1)-s_0+\iota-i_0], \
		r(\infty)=1-\frac{\gamma}{\beta} e^{-\frac{\beta}{\gamma} (r(\infty)-1+\iota+\frac{\gamma}{\beta})}, \ \tau_2-\tau_1=\frac{s(\tau_1)-\gamma/\beta}{\gamma \iota}, \
		s(\infty)=1-r(\infty).
	\end{aligned}
		\end{equation}
		\hrule
\end{table*}
\begin{figure}
	\centering
	\includegraphics[width=4.2cm]{prova.png}
	\includegraphics[width=4.2cm]{fig2f.png}\\
	\includegraphics[width=8cm]{table_pnas2.png}

	\caption{\emph{Left}: $T=1 yr$, optimal control (orange) and feedback policies (policy A in blue, policy B in purple) select suppression strategy.
	Performances of the three strategies are comparable, and economic cost is much larger than epidemic cost. \emph{Right}: $T=3 yrs$,
	optimal control and feedback policies select mitigation. The dynamics under policy B is similar to dynamics under the optimal control, and Policy B outperforms Policy A. In this simulation $\kappa=1.5$, and the other parameters are as in Fig. \ref{fig:control_R0}. \label{fig:suppression_mitigation}}
\end{figure}
To compare the cost achieved by the feedback policies with the optimal cost, we work under the assumption that $T=+\infty$, which has two main implications:
\begin{enumerate}
	\item there exists $t^*$ such that $s(t^*)=\gamma/\beta$ under optimal control, that is, herd immunity is reached; indeed, if it were not the case, the lockdown could never be totally released and the economic cost would explode;
	\item the fraction of infected $i(T)$ vanishes (see e.g. \cite{hethcote2000mathematics}).
\end{enumerate}
In other words, with infinite $T$ there is no phase transition between suppression and mitigation (unless $\kappa=0$), since avoiding the spread of the infection leads to an infinite economic cost and is therefore unfeasible.
\begin{remark}
We observed from simulations that above a certain value of $T$, the optimal policy does not depend on $T$. Indeed, if the epidemic reaches herd immunity, the infected naturally asymptotically decrease to $0$ and both epidemic and economic cost tend to vanish. Hence our results provide insight not only for the infinite horizon case, but for any large $T$
\end{remark}\medskip
Before establishing our main results, we introduce a technical lemma on SIR dynamics which is needed for the proofs and may be of separate interest.
\begin{lemma}[Uncontrolled dynamics]
	\label{prp:nc_square}
	Let $a \le b$, and let $L(t)=0$ for every $t \in [a,b]$.
	Then,
	\begin{equation*}
		\begin{aligned}
			\int_a^b i^2dt
			&=\gamma\frac{[\ln(s(a))]^2-[\ln(s(b))]^2}{2\beta^2}-\frac{s(a)-s(b)}{\beta} +\\ &+\big(s(a)+i(a)-\frac{\gamma}{\beta}\ln (s(a))\big)\frac{r(b)-r(a)}{\gamma}.
		\end{aligned}
		\label{eq:i^2_nc}
	\end{equation*}
\end{lemma}\medskip
\begin{proof}
	It can be shown that $s(t)+i(t)-\frac{\gamma}{\beta}s(t)$ is constant in time if $L(t)=0$, hence
	\begin{equation}
		\label{eq:constant}
		s(a)+i(a)-\frac{\gamma}{\beta}s(a) = s(b)+i(b)-\frac{\gamma}{\beta}s(b).
	\end{equation}
	By \eqref{eq:constant}, omitting the dependence on $t$,
	\begin{equation}
		\begin{aligned}
			&\int_a^b i^2dt=\int_a^b i\big(\frac{\gamma}{\beta}\ln (s)-\frac{\gamma}{\beta}\ln s(a)-s+s(a)+i(a)\big)dt\\
			=&\frac{\gamma}{\beta}\int_a^b i \ln (s) dt-\int_a^b is dt + \big(s(a)+i(a)-\frac{\gamma}{\beta}\ln s(a)\big)\int_a^b i dt.
		\end{aligned}
		\label{eq:sum_pieces}
	\end{equation}
	From $\dot{s}=-\beta s i$, the first integral gets:
	\begin{equation}
		\begin{aligned}
			&\int_a^b i \ln(s) dt= -\frac{1}{\beta}\int_a^b \frac{\dot{s}\ln(s)}{s}dt\\
			=&-\frac{1}{2\beta}\int_a^b\frac{d}{dt}[\ln(s)]^2dt=\frac{[\ln s(a)]^2-[\ln s(b)]^2}{2\beta}.
			\label{eq:piece1}
		\end{aligned}
	\end{equation}
	The second and the third integral are
	\begin{equation}
		\label{eq:piece2}
		\begin{aligned}
			&\int_a^b is dt=-\frac{1}{\beta}\int_a^b \frac{\dot{s}}{s}sdt=-\frac{1}{\beta}\int_a^b \dot{s}dt=\frac{s(a)-s(b)}{\beta},\\
			&\int_a^b i(t)dt=\int_a^b\frac{\dot{r}(t)}{\gamma}dt=\frac{r(b)-r(a)}{\gamma}.
		\end{aligned}
	\end{equation}
	The statement follows by plugging \eqref{eq:piece1} and \eqref{eq:piece2} in \eqref{eq:sum_pieces}.
\end{proof}\medskip
Let $C_{A}(\rho)$ and $C_B(\iota)$ denote the cost of policies $A$ and $B$ for a given value of $\rho$ and $\iota$, respectively. In the next two propositions we derive analytical expressions for $C_{A}(\rho)$ and $C_B(\iota)$. For the sake of readability, for the two policies we work respectively under the assumption that $\iota \ge i_0$ and $\rho > 1$, but similar expressions hold in case $\iota<i_0$ and $\rho \le 1$.
\begin{proposition}
	\label{prpA}
	Let $T = +\infty$. Then, $C_A(\rho)$ is as in \eqref{eq:t*}.
\end{proposition}
\begin{proof}
	See the Appendix.
\end{proof}
\begin{proposition}
	\label{prpB}
Let $T = +\infty$. Then, $C_{B}(\iota)$ is as in  \eqref{eq:cost_B}.
\end{proposition}
\begin{proof}
	See the Appendix.
\end{proof}
The idea for the proofs is to split the cost in three terms:
\begin{enumerate}
	\item economic cost: $\int_0^T L dt$;
	\item infected cost:
	$\int_0^T i dt  $;
	\item infected squared cost:
	$\int_0^T i^2 dt  $.
\end{enumerate}
The terms may be computed exactly by using Lemma \ref{prp:nc_square} and by using the fact that in presence of control the dynamics is linear.
In the next proposition we give a lower bound on the optimal cost, which will be used to compare Policy A and B with the optimal control. For technical reasons we restrict our analysis to policies that do not put control after herd immunity. This is a natural assumption, and numerical solutions show that this is indeed the case for most values of parameters.
	\begin{proposition}
		\label{prp:lb}
	Let $T=+\infty$, and let $t^*$ be the smallest time such that $s(t^*)\le\gamma/\beta$ (herd immunity). The cost of any control such that $L(t)=0$ for every $t \in [t^*,+\infty)$ is lower bounded by
	\begin{equation*}
		\label{eq:lower_bound}
		C^* := \underset{0\le i^* \le i_{max}}\min U(i^*),
	\end{equation*}
	where
	$
	 U(i^*)=2 {U_{12}^I(i^*)}+U_{22}^{II}(i^*)+U_{3}(i^*)
	 $
	 and
	 \begin{equation*}\footnotesize
		\begin{aligned}\footnotesize
			U_{12}^I(i^*)&= \epsilon \sqrt{\kappa\gamma_1} \left[ \frac{ -i^*+ i_0-s({t^*})+s_0}{\gamma} +\frac{1}{\beta} (\ln{s(t^*})- \ln s_0)\right],\\
		U_{22}^{II}(i^*) &=
			\frac{\gamma}{\beta}\frac{[\ln{(\gamma/\beta)}]^2-[\ln(s(\infty))]^2}{2\beta}-\frac{\gamma/\beta-s(\infty)}{\beta} +\\ &+\big(\gamma/\beta+i^*-\frac{\gamma}{\beta}\ln {\gamma/\beta}\big)\frac{i^*+\gamma/\beta-s(\infty)}{\gamma},\\
			U_3(i^*)&= \frac{\kappa\gamma_0}{\gamma}(r(\infty)-r_0),
		\end{aligned}
	\end{equation*}
	with $r(\infty)$ solution of $
		r(\infty)=1-s(t^*) e^{-\frac{\beta}{\gamma} \big(r(\infty)-1+i^*+s(t^*)\big)}$, $s(\infty)=1-r(\infty)$, $i_{max}=i_0+s_0-\frac{\gamma}{\beta}\big(1-\ln\frac{\gamma}{\beta s_0}\big)$, $\epsilon = \sqrt{27/32}$.
\end{proposition}\medskip
\begin{remark}
	Notice that $U(i^*)$ depends on $i^*$ both explicitly and through $s(\infty)$ and $r(\infty)$.
\end{remark}\medskip
\begin{proof}
	See the Appendix.
\end{proof}
Apart from constants, $U_3$ is the infected cost, and $U_{22}^{II}$ is the infected squared cost after herd immunity, computed starting from the state at time $t^*$ under the assumption that there is no control after herd immunity (so that economic cost is zero). These are computed using \eqref{eq:sir} with $L(t)=0$ and Lemma \ref{prp:nc_square}, respectively. The term $U_{12}^I$ is a lower bound for the sum of economic and infected squared cost before herd immunity. \\
Let $C_{opt}$ denote the cost of the optimal control among all the controls such that after herd immunity do not put any lockdown.
We can now derive bounds between performance of our two policies and $C_{opt}$.
\begin{theorem}
	Let $T = +\infty$. Then,
	\begin{equation*}
		C_A(\rho^*)-C_{opt} \le \min_\rho \ \eqref{eq:t*} - C^*.
	\end{equation*}
\end{theorem}
\begin{proof}
The proof follows immediately from Propositions \ref{prpA} and~\ref{prp:lb}.
\end{proof}\medskip
\begin{theorem}
Let $T = +\infty$. Then,
\begin{equation*}
	C_B(\iota^*)-C_{opt} \le \min_\iota \ \eqref{eq:cost_B} - C^*.
\end{equation*}
\end{theorem}
\begin{proof}
The proof follows immediately from Propositions \ref{prpB} and \ref{prp:lb}.
\end{proof}\medskip
In Fig. \ref{fig:plot} the exact cost of the two policies, the cost under the optimal control computed numerically, and the lower bound on the optimal cost are plotted as a function of $\kappa$. Note that controlling $i(t)$ is close to the optimal control computed numerically. The theoretical analysis allows to conclude that, for the tested parameters, controlling $i(t)$ is better than controlling $R(t)$, as the distance between cost of Policies A and B is comparable with the distance between Policy B and the lower bound $C^*$. From a computational viewpoint, note that controlling $i(t)$ instead of solving the optimal control, reduces the complexity of the problem from an optimization over a set of functions to a minimization over the parameter $\iota$.
\begin{figure}
	\centering
	\includegraphics[width=6cm]{plots_heuristic_vs_lowerbound_f2.png}
	\caption{\label{fig:plot} Exact cost of Policies A and B with optimal $\rho^*$ and $\iota^*$ (analytical), optimal cost (numerical) and lower bound to the optimal cost (analytical) as functions of $\kappa$. We noticed that $T=5$ years is large enough to approximate infinite time horizon. Indeed, after a large enough time, both economic and epidemic cost are negligible (see Fig. \ref{fig:suppression_mitigation}). Other parameters are as in Fig. \ref{fig:control_R0}.}
\end{figure}
\section{Conclusion}
In this work we consider a SIR with lockdown, and we study the optimal lockdown as solution to an optimal control problem. We model the cost as the sum of an epidemic cost, and economic cost due to lockdowns. We first show numerically that, as the time horizon and the cost of life vary, the solutions of optimal control exhibit a phase transition from suppression strategies to mitigation strategies, as described in \cite{ferguson2020report}. Then, we introduce two simple feedback policies, one stabilizing the reproduction number, and one stabilizing the fraction of infected. We compute their exact cost, and compare analytically their performance with the optimal cost. We show that in case of infinite time horizon controlling the fraction of infected is close to optimal. Future directions include but are not limited to: investigation on where the phase transition occurs; extending the analysis to the case of finite time horizon, considering different cost functions like sigmoidal functions, and extending the analysis to network SIR, as in the numerical analysis of \cite{acemoglu2020multi}.

\section*{Acknowledgements}
This research was carried on within the framework of the MIUR-funded {\it Progetto di Eccellenza} of the {\it Dipartimento di Scienze Matematiche G.L. Lagrange}, Politecnico di Torino, CUP: E11G18000350001. It received partial support from the {\it Compagnia di San Paolo} through a Joint Research Project. It was also supported by C3.ai Digital Transformation Institute award.

\bibliographystyle{IEEEtran}