text:
https://www.thehour.com/wilton/article/Newtown-fund-should-pay-for-new-school-8186311.php
Newtown fund should pay for new school
Published 6:50 am EDT, Monday, July 15, 2013
The State Bonding Commission is expected to approve funding to begin the task of replacing Sandy Hook Elementary School in Newtown, scene of the horrific slaughter of 20 children and six adults late last year at the hands of a crazed gunman.
The existing school will be leveled to make way for the new school. It will stand as a monument to those 20 children, and the six adults who died trying to protect them.
The students finished the school year at Chalk Hill Middle School, which was not in use at the time of the shooting.
The Newtown Board of Education is weighing plans for a $42 million, 70,000-square foot facility.
The Bonding Commission, which is chaired by Gov. Dannel P. Malloy, is expected to follow his wishes. The General Assembly earlier this month authorized the Bonding Commission to allocate up to $50 million for the school construction.
"It is my hope that by announcing the funding today, we can help Newtown continue with the healing process. Our thoughts are with those 20 angels and their families. Their memories will forever be close to our hearts," said Malloy.
There have been some questions raised about the state's action since usually schools are built by the municipalities in which they reside.
The question has been raised about the millions of dollars that have been donated to the town. Did it go to the families or what?
It would seem proper that any available funds be devoted to the school. This would seem in keeping with the wishes of the many donors.
There should be an accounting of the fund and where the money has been spent.
Given the state's financial condition, such generosity seems misplaced.


text:
\section{Introduction}
Modal verbs (also referred to as modal operators, modals, or modal auxiliaries \cite{ImreAttila2017ALAt}) convey important semantic information about a situation being described or the speaker's perspective related to the likelihood and/or mode of the proposition \cite{Lyons1997, Quirk1985}.
Because of the widespread use of modal verbs in our daily lives, an accurate modeling of modal verb senses from context is essential for semantic understanding. For example, as modal verbs are often used with verbs that express one's personal state or stance, such as \textit{admit, imagine}, and \textit{resist} \cite{Biber}, we can utilize them for better speaker intention identification or sentiment analysis (See our case studies in Section \ref{sec:case_study}).

In both linguistics and NLP, however, there is no unifying consensus on how to organize these words (Table \ref{tab:framework_comparison}). One reason for this indeterminacy is their lack of a straightforward definition \cite{Nuyts2005OnDM}.
Modal verbs have nuanced meanings, and their interpretation is often subjective.
If a speaker says, ``I \textit{can} go to the event today'', it can refer to their ability to go to the event.
Alternatively, if the speaker is a minor, the listener may interpret it as having permission from their parents.
As such, categorizing modal verbs requires more attention than many other linguistic features, making the task challenging even for humans (Table \ref{tab:annotation_examples} shows examples of annotator disagreement on the modal verb categorization task).

We present a new dataset, \texttt{MoVerb}\xspace, containing 4540 annotated conversational English utterances with their modal verb categories. We chose the conversational domain since spoken, casual text is more flexible and nuanced compared to language from other domains, and therefore could reap the most benefits from better modal verb classifications.
To the best of our knowledge, this study provides the first empirical comparison of two modal verb frameworks with annotated datasets, evaluating the practicality of these different theoretical frameworks.
Our study shows a clear inclination towards one of the two frameworks and quantitatively shows how humans struggle with the task.

\begin{savenotes}
\begin{table*}[]
    \footnotesize
    \centering

    \begin{tabular}{@{}P{2.6cm}P{0.6cm}P{0.6cm}P{0.6cm}P{0.6cm}P{0.6cm}P{0.6cm}}

    \toprule
    \textbfsc{Reference} & \multicolumn{6}{c}{\textbfsc{Modality Categories}}\\
    \midrule

    \citet{Kratzer1991} &  \multicolumn{6}{c}{Epistemic{\hskip 3cm}Deontic{\hskip 3cm}Circumstantial}\\\midrule
    \citet{palmer_1986} & \multicolumn{6}{c}{Epistemic{\hskip 3cm}Deontic{\hskip 3cm}Dynamic}\\\midrule

    \citet{Quirk1985} & \multicolumn{6}{c}{Possibility{\hskip 3mm}Ability{\hskip 3mm}Permission{\hskip 3mm}Necessity\footnote{Logical Necessity}{\hskip 3mm}Obligation\footnote{Obligation/Compulsion}{\hskip 3mm}Inference\footnote{Tentative Inference}{\hskip 3mm}Prediction{\hskip 3mm}Volition}\\\midrule
    \citet{baker} & \multicolumn{6}{c}{Requirement{\hskip 5mm}Permissive{\hskip 5mm}Success{\hskip 5mm}Effort{\hskip 5mm}Intention{\hskip 5mm}Ability{\hskip 5mm}Want{\hskip 5mm}Belief}\\\midrule

    \citet{Ruppenhofer} & \multicolumn{6}{c}{Epistemic{\hskip 8mm}Deontic{\hskip 8mm}Dynamic{\hskip 8mm}Optative{\hskip 8mm}Concessive{\hskip 8mm}Conditional}\\\midrule
    \vspace{-4mm}\citet{Matthewson2018ModalFF} & \multicolumn{6}{c}{\makecell{Root \\ {(Teleological\hspace{0.2cm}Deontic\hspace{0.2cm}Bouletic)} }\hspace{0.2cm}\makecell{Epistemic\\ (Inferential\hspace{0.2cm}Reportative)}}\\\midrule

    \citet{Nissim2013CrosslinguisticAO}\footnote{\citeauthor{Nissim2013CrosslinguisticAO}'s work includes more categories on different dimensions, but we only show those comparable to the others in this table} & \multicolumn{4}{c}{\makecell{Epistemic\\(committment\hspace{0.2cm}evidential)}{\hskip 5mm}\makecell{Deontic\\(manipulative\hspace{0.2cm}volition)}} & \multicolumn{2}{c}{\makecell{Dynamic\\(axiological\hspace{0.2cm}appreciative\hspace{0.2cm}apprehensional)}}\\\midrule

    \citet{Portner} & \multicolumn{2}{c}{\makecell{Epistemic\\}} & \multicolumn{2}{c}{\makecell{Priority\\(Deontic \hspace{0.2cm} Bouletic \hspace{0.2cm} Teleological)}} & \multicolumn{2}{c}{\makecell{Dynamic\\(Volitional \hspace{0.2cm} Quantificational)}}\\

    \bottomrule

    \end{tabular}
    \caption{A non-exhaustive list of past work on modality and the frameworks they use. Note that some linguists support two-tiered categorical frameworks by defining general categories that are further divided into subcategories.
    \label{tab:framework_comparison}
\end{table*}
\end{savenotes}

Our main contributions are as follows:
\begin{itemize}\setlength{\parskip}{0pt}\setlength{\itemsep}{0pt}\setlength{\partopsep}{0pt}
\item We collected \texttt{MoVerb}\xspace, an annotated conversational domain dataset containing two types of labels for modal verbs in 4540 English utterances. The dataset is split into two: the first consisting of utterances with a single final label determined by majority voting and the second consisting of utterances with complete disagreement (Table \ref{tab:annotation_examples}).
\item We found the difficulty of annotating modal verbs even based on solid theoretical frameworks. We show that high annotator disagreement can be caused by issues not necessarily related to a difference in interpretation. We suggest that the gap between practice and theory could be bridged by prioritizing the straightforwardness of annotations.

\item We found a clear performance gap between the fine-tuned classifiers trained on different frameworks of data in \texttt{MoVerb}\xspace: 82.2 F1 on Quirk and 78.3 F1 on Palmer.
Additionally, both frameworks struggled when applied across different domains.
\end{itemize}

\section{Related work}\label{sec:related_work}
\begin{table*}[t!]
\small
\centering
\begin{tabular}{V{8.5cm}ccc}
\toprule
\textbfsc{Utterances with complete agreement}  & \textbfsc{Annotator 1} & \textbfsc{Annotator 2} & \textbfsc{Annotator 3}\\
\midrule
Usually moving your body helps but it depends on her situation... i \textit{would} get a 2nd opinion! & volition & volition & volition\\
\midrule
I bought a lottery ticket and have a feeling I \textit{will} win. & prediction & prediction & prediction\\
\midrule
That is really sweet of them. \textit{Must} have been a big party. & necessity & necessity & necessity\\
\midrule
I get it.. but you know life really is too short.. i think you \textit{should} try to reach out! Do it!:) & obligation & obligation & obligation\\
\midrule
\multicolumn{4}{c}{}
\\
\toprule
\textbfsc{Utterances with complete disagreement}  & \textbfsc{Annotator 1} & \textbfsc{Annotator 2} & \textbfsc{Annotator 3}\\
\midrule
That \textit{must} have been terrible. Were you okay? & inference & necessity & possibility\\
\midrule
I am going to a drink and paint party tomorrow. It \textit{should} be pretty fun! & inference & necessity & prediction\\
\midrule
I am stressed by my blood test results that I \textit{will} have tomorrow. & ability & necessity & prediction\\
\midrule
I work remotely, I wish that you \textit{could} do something like that as well.& ability & permission & possibility\\
\bottomrule
\end{tabular}
\caption{Annotation examples from \texttt{MoVerb}\xspace for complete agreement and disagreement among the three annotators. Note that \textit{necessity} here refers to logical necessity, not social or physical necessities.}
\label{tab:annotation_examples}
\end{table*}

There are numerous linguistic studies about modal verbs and their categorization theories \cite{Quirk1985, palmerModality1990, Lyons1997, Mindt, kratzer2012, morante-sporleder-2012-modality, aarts_mcmahon_hinrichs_2021}. However, despite attempts to reconcile them \cite{reconcile_frameworks}, the widespread variation makes it unclear which theory and framework would work best for certain NLP tasks (Table \ref{tab:framework_comparison}). A dataset using multiple modal verb frameworks would help researchers experiment, but that dataset is yet to be built.
To the best of our knowledge, there is no English dataset dedicated to the comparison of modal verb labeling

Framework consistency is not the only thing lacking in modality datasets.
Sources of modality can vary from dataset to dataset as well. In a multilingual corpus focusing on modality as a whole, \citeauthor{Nissim2013CrosslinguisticAO} manually tag words and phrases representing modality. Because of the lack of emphasis on modal verbs, this dataset contains only 32 instances over 7 modal verbs: \textit{will, might, can, may, would, could,} and \textit{should} \cite{Nissim2013CrosslinguisticAO}. Although some may argue that focusing on modality as a whole is more natural, we argue that a dataset focusing on modal verbs is also necessary because of the ample complexities of modal verbs on their own, as previously mentioned.

Even datasets that do focus on modal verbs are not guaranteed to study the same set of words \cite{Ruppenhofer, marasovic-etal-2016-modal}. Additionally, modal verbs in different domains, namely conversational and academic, have quite dissimilar distributions \cite{Biber}.
In our cross-domain analysis, we utilized a dataset for subjectivity analysis in opinions and speculations from the news domain \cite{Ruppenhofer, Wiebe2005AnnotatingEO}.
\citeauthor{Ruppenhofer} do not include \textit{would} and \textit{will} in their annotations, making their dataset challenging for analyzing conversational English. \textit{Would} and \textit{will} are 1st and 3rd when we rank modal verbs by their frequencies in spoken English \cite{Mindt, Biber}.\footnote{As of August 2022, the Corpus of Contemporary American English (COCA) dataset \url{https://www.wordfrequency.info} shows \textit{will} and \textit{would} being the 47th and 49th most common words overall. For comparison, \textit{shall} is 1090th.}

We note that there is a slight difference in our annotation frameworks. \citeauthor{Ruppenhofer} create a different schema of their own, building off of work by \citet{baker} and \citet{palmer_1986}. We do not use \citeauthor{baker}'s labels since we are more interested in applying traditional linguistic theories. However, we are still able to compare results since Palmer's categories\xspace make up 97.57\% of the annotations in the \citeauthor{Ruppenhofer} dataset.

\section{Theoretical Frameworks}
\begin{table*}
\small
\centering
\begin{tabular}{l c c c ccc cc}
\toprule
&\smtextbfsc{possibility}&{\smtextbfsc{prediction}}&\smtextbfsc{inference}&\smtextbfsc{necessity}&\smtextbfsc{ability}&\smtextbfsc{volition}&\smtextbfsc{permission}&\smtextbfsc{obligation}\\
\midrule
\smtextbfsc{deontic} & 50 & 21 & 22 & 27 & 42 & 31 & 22 & 288\\
\smtextbfsc{epistemic} & 454 & 307 & 120 & 317 & 110 & 12 & 1 & 10\\
\smtextbfsc{dynamic} & 197 & 172 & 13 & 11 & 758 & 194 & 22 & 22\\
\bottomrule
\end{tabular}
\caption{The frequency distribution between Quirk's and Palmer's categories\xspace in \texttt{MoVerb}\xspace. This table shows that there is no clear mapping between the two frameworks, although there are common combinations (epistemic possibility, dynamic ability, etc.) that reveal overlapping categories. }
\label{tab:7_3_table}
\end{table*}

We use two labeling frameworks in our dataset annotations that we refer to as Quirk's categories\xspace and Palmer's categories\xspace. The labels we use are as follows

\vspace{-3mm}
\begin{itemize}\setlength{\partopsep}{0pt} \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}
    \item Quirk's categories\xspace consist of eight labels: \textit{possibility},
    \textit{ability},
    \textit{permission},
    \textit{logical necessity} (abbrev. \textit{necessity}),
    \textit{obligation/compulsion} (abbrev. \textit{obligation}),
    \textit{tentative inference} (abbrev. \textit{inference}),
    \textit{prediction}, and \textit{volition}. While each category's definitions may be inferred by their names, further explanations can be found in Figures \ref{fig:mturk_quirk_descriptions} and \ref{fig:mturk_quirk_examples} in Appendix \ref{sec:mturk_form}.
    \item Palmer's categories\xspace consist of three labels: \textit{deontic}, \textit{epistemic}, and \textit{dynamic}. A deontic modal verb influences a thought, action, or event by giving permission, expressing an obligation, or making a promise or threat. An epistemic one is concerned with matters of knowledge or belief and with the possibility of whether or not something is true. Lastly, dynamic modal verbs are related to the volition or ability of the speaker or subject, in other words, some circumstantial possibility involving an individual (Figures \ref{fig:mturk_palmer_descriptions} and \ref{fig:mturk_palmer_examples} in Appendix \ref{sec:mturk_form}).
\end{itemize}

Table \ref{tab:7_3_table} shows a contingency table for \texttt{MoVerb}\xspace. We see that there is no straightforward mapping allowing us to cleanly convert one framework to the other. However, the different distributions of one set of labels within labels of the other framework reveal which categories are similar to each other.

\section{\texttt{MoVerb}\xspace: Annotated Modal Verb Dataset}\label{sec:methodology}

\begin{table*}[t]
\small
\centering
\begin{tabular}{l@{\hskip 1mm}c@{\hskip 2mm}c@{\hskip 2mm}c@{\hskip 2mm}c@{\hskip 2mm}c@{\hskip 2mm}c@{\hskip 2mm}c@{\hskip 2mm}c@{\hskip 2mm}r}
\toprule
&\textbfsc{will}&\textbfsc{would}&\textbfsc{should}&\textbfsc{may}&\textbfsc{might}&\textbfsc{must}&\textbfsc{could}&\textbfsc{can}&\textbfsc{total}\\
\midrule
\smtextbfsc{possibility} & 50 & 61 & 7 & 128 & 324 & 0 & 119 & 96 & 785 (0.22\%)\\
\smtextbfsc{ability} & 14 & 24 & 0 & 0 & 0 & 1 & 302 & 657 & 998 (0.28\%)\\
\smtextbfsc{permission} & 2 & 4 & 4 & 19 & 1 & 0 & 10 & 12 & 52 (0.01\%)\\
\smtextbfsc{necessity} & 7 & 12 & 13 & 0 & 0 & 334 & 3 & 1 & 370 (0.1\%)\\
\smtextbfsc{obligation} & 5 & 6 & 307 & 1 & 0 & 18 & 0 & 4 & 341 (0.1\%)\\
\smtextbfsc{inference} & 6 & 42 & 45 & 2 & 11 & 73 & 1 & 1 & 181 (0.05\%)\\
\smtextbfsc{prediction} & 351 & 183 & 19 & 0 & 5 & 4 & 4 & 3 & 569 (0.16\%)\\
\smtextbfsc{volition} & 129 & 92 & 11 & 3 & 6 & 1 & 6 & 6 & 254 (0.07\%)\\
\midrule
\textbfsc{total} & 564 (16\%) & 424 (12\%) & 406 (11\%) & 153 (4\%) & 347 (10\%) & 431 (12\%) & 445 (13\%) & 780 (22\%) & \textbf{3550}\\
\toprule
\smtextbfsc{epistemic} & 283 & 269 & 78 & 99 & 232 & 479 & 118 & 161 & 1719 (42\%)\\
\smtextbfsc{deontic} & 32 & 65 & 437 & 25 & 18 & 35 & 27 & 52 & 691 (16.9\%)\\
\smtextbfsc{dynamic} & 336 & 258 & 29 & 37 & 108 & 6 & 315 & 592 & 1681 (41.1\%)\\
\midrule
\textbfsc{total} & 651 (16\%) & 592 (14\%) & 544 (13\%) & 161 (4\%) &358 (9\%) &520 (13\%) &460 (11\%) &805 (20\%) & \textbf{4091}\\
\bottomrule
\end{tabular}
\captionof{table}{The breakdown of categories in each modal verb.}
\label{tab:dataset-statistics}
\end{table*}

Table \ref{tab:dataset-statistics} shows the  statistics of our \texttt{MoVerb}\xspace dataset.
We use the eight core modal verbs in our study:  \textit{can}, \textit{could}, \textit{may}, \textit{might}, \textit{must}, \textit{will}, \textit{would}, and \textit{should}.
\textit{Shall} is also another core modal verb but is excluded from our work since there are too few instances of it in our conversational dataset.\footnote{\textit{shall} is more likely to be used in legal contexts \cite{CoatesAndLeech}, which is outside the scope of this study.}

We chose the Empathetic Dialogues dataset \cite{empathetic_dialogues} for our annotation task because of its variety of utterances in the conversational domain and wide usage in social dialogues. An utterance is defined as a speaker's output in a single turn and can potentially be one or more sentences. We extracted utterances containing only one modal verb as detected using SpaCy's POS tagger and lemmatizer \cite{spacy2}. We focused on utterances containing one modal verb for simplicity, but this did not exclude much from our dataset since only 2.4\% of the utterances had more than one modal verb.\footnote{78.8\% had none and 18.8\% had one.}

We included utterances containing more than one sentence (as long as they used only one modal verb) in order to retain as much context as possible. In this way, we separated out the first 4540 utterances containing single modal verbs, except for \textbf{may} and \textbf{might}, which we collected and used all of due to scarcity (Figure \ref{fig:bar-a} and Table \ref{tab:dataset-statistics}).

After finalizing our candidate utterances to annotate, we utilized Amazon Mechanical Turk (MTurk) to gather crowd-sourced labels for each modal verb. Three annotations were collected for each of the 4540 utterances and assigned final labels based on majority voting (Figures \ref{fig:bar-b} and \ref{fig:bar-c}). Our HIT (Human Intelligence Tasks) form is included in Appendix \ref{sec:mturk_form} (Figures \ref{fig:mturk_instructions}-\ref{fig:mturk_sentences_cat2}).
We limited our MTurk pool to Master workers (high-performing workers) living in the US with approval rates of $>98\%$.
Each worker was allowed to annotate as many HITs as they wanted
and were allowed to submit annotations for both frameworks. They were prevented from participating any further if we saw that their annotations for Quirk's categories\xspace seemed random (Appendix \ref{sec:filtering_criteria}).
Unfortunately, we could not reliably differentiate randomness from subjective differences for Palmer's categories\xspace, since each category has less stringent restrictions on which modal verbs can be assigned to them. As such, there is no clean and unsupervised method of quickly determining data quality for this framework.

\begin{figure*}
    \begin{subfigure}[t]{0.32\textwidth}
        \centering
    \includegraphics[width=\textwidth,trim={0 0 0 0cm},clip]{figures/mv3_bar.png}
        \caption{Modal verb distribution}
        \label{fig:bar-a}
    \end{subfigure}\hfill
    \begin{subfigure}[t]{0.30\textwidth}
        \includegraphics[width=\textwidth]{figures/cat13_bar.png}
        \caption{Quirk's label distribution. }
        \label{fig:bar-b}
    \end{subfigure}\hfill
    \begin{subfigure}[t]{0.30\textwidth}
        \includegraphics[width=\textwidth]{figures/cat23_bar.png}
        \caption{Palmer's label distribution.  }
        \label{fig:bar-c}
    \end{subfigure}
    \caption{Dataset statistics: (a) Modal verb distribution, (b) Quirk's categories label distribution, and (c) Palmer's categories label distribution.
    This chart only includes utterances that had a majority label.}
    \label{fig:bar}
\end{figure*}

\subsection{Post-analysis on Annotations}
\label{sec:post_analysis}

Our final annotations revealed some commonly disagreed upon categories (Figure \ref{fig:cat1_disagreements}, \ref{fig:cat2_disagreements} here and Table \ref{tab:conflicting_annots} in Appendix \ref{sec:dataset_statistics}). For some instances in Quirk's categories\xspace, it seemed that annotators were simply using certain labels interchangeably, as opposed to truly diverging on how the modal verb affected the utterance. For example, in Figure \ref{fig:cat1_disagreements}, we can see that \textit{inference} and \textit{(logical) necessity} are co-occurring in high frequencies. Utterances containing sentences like, ``You \textit{must} have been so happy'' and ``You \textit{must} have been so scared'' in the dataset often both had at least one \textit{(logical) necessity} and \textit{inference}  annotations each. While sentence length is not an infallible measure of context, the lack of correlation between it and annotator disagreement (Figure \ref{fig:context_effect} in Appendix \ref{sec:dataset_statistics}) suggests that simply having more text may not be the best method for providing context for this task.
This illustrates how frameworks well-grounded in theory can still be interpreted differently in practice. Some labels may be combined for less noise if their distinction is unnecessary for downstream tasks.

\begin{table}[t]
\small
\centering
\begin{tabular}{lcc}
\toprule
&\textbfsc{Quirk} & \textbfsc{Palmer}\\
\midrule
\textbfsc{Percent agreement} &0.58 & 0.60\\
\bottomrule
\end{tabular}
\caption{Percent agreement values for both frameworks in \texttt{MoVerb}\xspace}
\vspace{-3mm}
\label{tab:percent_agreement}
\end{table}

Another common behavior was that annotators sometimes seemed to label utterances based on what could be inferred. For example, an utterance containing a sentence like ``I \textit{may} go to the store today'' was often labeled as both \textit{ability} and \textit{possibility}. One could argue that this \textit{may} represents \textit{ability}, since it indicates that the user has the ability to go to the store today or that the information regarding the speaker's \textit{ability} is most important. However, one could also claim that the annotator is then labeling what can be inferred from the utterance, not necessarily what the modal verb semantically represents. This behavior could also be observed in Table \ref{tab:7_3_table} and Figure \ref{fig:cat2_disagreements}, where \textit{epistemic} and \textit{dynamic}, which are seen most in utterances labelled \textit{possibility} and \textit{ability}, appear commonly in conflicting annotation pairs.

All in all, annotators seemed to struggle more with using Palmer's categories, as can be speculated from the fact that the percent agreement between the two frameworks was very similar, despite Palmer's categories having significantly fewer labels (Table \ref{tab:percent_agreement}). This is not too surprising, given that Palmer's categories are more abstract and can thus be less intuitive. The unfamiliar label titles may have also added a layer of complication to the task.

\subsection{Data Subjectivity}
\label{sec:data_subjectivity}

We argue that these disagreements highlight the flexibility and ambiguity that have plagued linguists for decades and emphasize the subjectivity of modal verbs. This is because modal verb annotations highly rely on what the reader interprets as the main takeaway of what the modal verb represents.
Quirk's mappings (Table \ref{tab:quirk_categories} in Appendix \ref{sec:dataset_statistics}) were not used to limit annotator options in the MTurk form since we wanted annotators to select labels on their own with minimal input from us. The added flexibility may have led to lower inter-annotator agreement levels, which is inevitable for subjective annotations \cite{Leonardelli2021AgreeingTD, Basile2020ItsTE, Aroyo}.

\section{Experimental Results}
We like to answer the following questions using the collect \texttt{MoVerb}\xspace dataset: (1) how well \texttt{MoVerb}\xspace could be used to train the latest deep learning models for modal verb sense prediction task (Section \ref{sec:single_domain_classification}) and (2) how transferable that knowledge (trained on the conversational domain) was to other domains, namely the news opinion domain (Section \ref{sec:cross_domain_transferability}).

\paragraph{Experiment Design}\label{sec:evaluation}
For the first experiment, we split our datasets into train-test ratios of 90/10. For the second experiment focusing on transferability, we used one dataset for the training and validation data and the other for the test set. We conducted this on both \texttt{MoVerb}-Palmer\xspace$\rightarrow$ \citeauthor{Ruppenhofer} and \citeauthor{Ruppenhofer}$\rightarrow$ \texttt{MoVerb}-Palmer\xspace combinations. Additionally, since we initially surmised that the lack of \textit{will/would} examples in the \citeauthor{Ruppenhofer} dataset would cause issues, we conducted the same experiment with those modal verbs removed from \texttt{MoVerb}-Palmer\xspace to observe the effect of not including \textit{will/would} (Table \ref{tab:model_comparison} and \ref{tab:x_domain_transferability}).

For all experiments, we ran 10-fold cross-validations and used an early stopping callback that would get triggered once the F1 value stopped increasing by at least 0.01.
For learning rates, we tested among $5e-6, 1e-5$, and $2e-5$, and used the weighted F1 score for evaluation. We used the Pytorch Lightning library to train and evaluate a Transformer model with an Adam epsilon of 1e-8, and a batch size of 32. Additionally, our trainer used GPU acceleration with a GeForce RTX 3090 using the DistributedDataParallel strategy

We fine-tuned six Transformer-based models \cite{Vaswani} from Huggingface Transformers \cite{huggingface-transformer}: ALBERT\textsubscript{base} \cite{albert}, BERT (both base and large) \cite{Devlin2019BERTPO}, RoBERTa (both base and large) \cite{liu2019roberta}, and DistilBERT\textsubscript{base} \cite{distilbert}.
In all runs, the RoBERTa models showed the best test F1 scores (Tables \ref{tab:classifier_res2} and \ref{tab:classifier_res4} in Appendix \ref{sec:classification_results}).

\begin{table}[t]
\small
\centering
\begin{tabular}{lcc}
\toprule
\textbfsc{Dataset} & \textbfsc{Validation} & \textbfsc{Test}\\
\midrule
\texttt{MoVerb}-Quirk\xspace & 78.98 & 82.22\\
\texttt{MoVerb}-Quirk\xspace (w/o will/would) & 83.56 & 84.31\\
\midrule
\texttt{MoVerb}-Palmer\xspace & 77.08 & 78.36\\
\texttt{MoVerb}-Palmer\xspace (w/o will/would) &  80.62 & 80.89\\
\midrule
\citeauthor{Ruppenhofer} & 83.31 & 85.60\\
\hline
\end{tabular}
\caption{Best-performing F1 scores averaged over a 10-fold cross validation traned on various datasets. We selected the best F1 scores out of various model and learning rate combinations. For a more complete table, see Table \ref{tab:classifier_res2}.}
\label{tab:model_comparison}
\end{table}

\begin{table*}[t]
\small
\centering
\begin{tabular}{ccccc}
\toprule
\textbfsc{Rank} & \multicolumn{2}{c}{\textbfsc{\texttt{MoVerb}-Palmer\xspace}} & \multicolumn{2}{c}{\textbfsc{\citeauthor{Ruppenhofer}}}\\
& \smtextbfsc{Modal Verb} & \smtextbfsc{Label} & \smtextbfsc{Modal Verb} & \smtextbfsc{Label}\\
\midrule
1 & can (19.7\%) & epistemic (42.0\%) & can (29.5\%) & deontic (46.1\%)\\
\midrule
2 & will (15.9\%) & dynamic (41.1\%) & should (22.4\%) & epistemic (27.6\%)\\
\midrule
3 & would (14.5\%) & deontic (16.9\%) & could (19.7\%) & dynamic (26.3\%)\\
\midrule
4 & should (13.3\%) & - & must (14.8\%) & -\\
\midrule
5 &  must (12.7\%) & - & may (8.5\%) & -\\
\bottomrule
\end{tabular}
\caption{Modal verb and label distribution comparisons between \texttt{MoVerb}\xspace and \citet{Ruppenhofer}. Note that while the modal verb ranking will be the same for both frameworks in \texttt{MoVerb}\xspace, we only list a ranking of \texttt{MoVerb}-Palmer\xspace in order to compare it with \citet{Ruppenhofer}.}
\label{tab:ruppenhofer_vs_palmer}
\end{table*}

\subsection{Single-Domain Modal Verb Sense Classification }\label{sec:single_domain_classification}

From Table \ref{tab:model_comparison}, we observe that \texttt{MoVerb}\xspace can indeed be used to train Transformer-based models \cite{Vaswani} on how to label modal verbs.
The table shows that \texttt{MoVerb}-Quirk\xspace does better at training models compared to \texttt{MoVerb}-Palmer\xspace.
However, we see that performance differences between domains have a larger effect than comparisons observed between frameworks. The performance difference between \texttt{MoVerb}-Palmer\xspace and \citeauthor{Ruppenhofer}'s dataset is greater than that between \texttt{MoVerb}-Palmer\xspace and \texttt{MoVerb}-Quirk\xspace. This was even after removing \textit{will}s and \textit{would}s, since \citeauthor{Ruppenhofer} did not annotate those two modal verbs, which were common in our Disagreement subset. This greater performance difference may be attributed to the fact that news-related writing tends to be more structured than conversational data or that the \citeauthor{Ruppenhofer}'s dataset contained a higher proportion of \textit{should} and \textit{could}s, which were less likely to be disagreed upon (Table \ref{tab:ruppenhofer_vs_palmer} here and Table \ref{tab:agreement_disagreement_table_count} in Appendix \ref{sec:dataset_statistics}). However, more annotations in different domains should be collected for a further definitive conclusion.

Table \ref{tab:difficult_predictions} contains instances where the classifiers predicted incorrectly and with low confidence. Classification of these utterances is especially difficult because of the ambiguity of the modal verbs and need for more context. However, Table \ref{fig:context_effect} in Appendix \ref{sec:dataset_statistics} showed that text length was not correlated with better predictions in our dataset. This suggests that the context our model requires in \texttt{MoVerb}\xspace may need to be collected from knowledge bases or by using long context comprehension.

\subsection{Cross-Domain Transferability}
\label{sec:cross_domain_transferability}
We also applied the classifiers trained on \texttt{MoVerb}-Palmer\xspace to the \citeauthor{Ruppenhofer} news opinion domain dataset\footnote{Downloaded from\\ \url{http://ruppenhofer.de/pages/Data\%20sets.html}} in order to see how our classification model might perform in another domain (Table \ref{tab:x_domain_transferability}).
As mentioned in Section \ref{sec:related_work}, this dataset uses a slightly modified framework, adding three more labels to Palmer's categories. However, we removed them in our experiment since they only made up 3.2\% of the dataset we extracted. We also filtered out sentences with more than one modal verb in order to mirror what we use in Empathetic Dialogues \cite{empathetic_dialogues}.

\begin{table}[h]
\small
\centering
\begin{tabular}{lll}
\toprule
\textbfsc{Dataset} & \textbfsc{Val. F1} & \textbfsc{Test F1}\\
\midrule
\texttt{MoVerb}-Palmer\xspace$\rightarrow$ R\&R\xspace & 75.4 & 61.44\\
\midrule
R\&R\xspace$\rightarrow$ \texttt{MoVerb}-Palmer\xspace & 86.5 & 66.37\\
\midrule
\texttt{MoVerb}-Palmer\xspace (w/o w$^2$)$\rightarrow$ R\&R\xspace & 80.23 & 69.74\\
\midrule
R\&R\xspace$\rightarrow$ \texttt{MoVerb}-Palmer\xspace (w/o w$^2$) & 86.5 & 75.93\\
\bottomrule
\end{tabular}
\caption{Observing cross-domain transferability. We use R\&R\xspace to represent \citeauthor{Ruppenhofer} and w$^2$ to represent \textit{will/would} in the interest of space. The dataset to the left of the arrow represents the training dataset, and the one on the right represents the validation/test set.}
\label{tab:x_domain_transferability}
\end{table}

We see that our models struggled significantly when the training data and test data came from different sources (Table \ref{tab:x_domain_transferability} here and Table \ref{tab:classifier_res4} in Appendix \ref{sec:dataset_statistics}). Utterances from a conversational dataset are bound to be different from opinions extracted from news sources due to the nature of their content. We additionally ran the same experiment after removing \textit{will/would} from \texttt{MoVerb}-Palmer\xspace to see the extent to which the lack of these two labels affected the F1 scores. The scores rose significantly for both directions although did not reach performance levels observed in single-domain classification (Table \ref{tab:x_domain_transferability}). A couple difficult examples for cross-domain classification is shown in Table \ref{tab:difficult_predictions} as well.

\section{Conclusions}\label{sec:conclusion}

\begin{table*}[h!]
\centering
\small
\begin{tabular}{lV{6cm}lll}
\toprule
\textbfsc{Dataset} & \textbfsc{Sentence} & \textbfsc{Prediction} & \textbfsc{Label}\\
\midrule
\texttt{MoVerb}-Quirk\xspace & We do not have a fence but I know my dog \textit{will} stay in the yard & volition (49.36) & prediction (3/3)\\
\texttt{MoVerb}-Palmer\xspace & That stinks! Try not to be jealous though. Something else \textit{will} come your way. & dynamic (49.87) & epistemic (3/3)\\
R\&R & ``A government in which the president controls the Supreme Court, the National Assembly and the Armed Forces \textit{can} not be called a democracy, '' Soto charged. & deontic (65.7) & dynamic (N/A)\\
\texttt{MoVerb}-Palmer\xspace$\rightarrow$ R\&R & They are provided with a medical exam upon admission, and their diet ( is someone making a point about diversity here ? ) ranges from bagels and cream cheese to rice and beans -- all eaten with plastic utensils -- after which the prisoners \textit{may} clean their teeth with specially shortened brushes. & epistemic (48.95) & deontic (N/A)\\
R\&R$\rightarrow$\texttt{MoVerb}-Palmer\xspace & News that big \textit{would} be a shock to anyone! How did you both handle it? & dynamic (49.74) & epistemic (3/3)\\
\bottomrule
\end{tabular}
\caption{Difficult examples incorrectly labeled by our RoBERTa-large classifier. The numbers in the parentheses represent the classifier's confidence score for the predictions and the annotator agreement score for the labels. We use R\&R to represent \citeauthor{Ruppenhofer}. Although coincidentally our \texttt{MoVerb}\xspace instances here all have complete annotator agreement, they represent cases with ambiguous meaning and/or alternative interpretation. In the first example, we see that the model is focusing more on the dog by putting emphasis on its decision (volition) rather than its owner's prediction. In the second example, one could argue that the model ``believes'' more in the internal locus of control, where one's own actions determine an outcome (dynamic), as opposed to the external locus of control, which puts more emphasis on plain luck (epistemic). As such, instances with low prediction confidences can help shed light on alternative interpretations and hidden biases in text.}
\label{tab:difficult_predictions}
\end{table*}
We compared two linguistic frameworks by crowd-sourcing annotations for 4540 utterances. Modal verb categorization is a difficult task, even for humans, making supervised datasets a vital part of analyses. See Table \ref{tab:difficult_predictions} for examples our classifier struggled with. Our work shows that within \texttt{MoVerb}\xspace, annotators had an easier time working with Quirk's categories\xspace. Fewer disagreement relative to the number of labels led to less noise, which translated to better performance on Transformer models, both intra and cross-domain, although domain differences seemed to affect the performance more than framework chosen (Table \ref{tab:classifier_res2} in Appendix \ref{sec:classification_results}). Additionally, Quirk's categories\xspace gave us a more precise study of modal verb patterns (Figure \ref{fig:cat1_disagreements} vs. Figure \ref{fig:cat2_disagreements}) due to more specific labels. Therefore, barring cases where there is a specific reason to use Palmer's categories\xspace (i.e. expanding another dataset that uses Palmer's categories\xspace or comparing work with other studies that use it), \textbf{we recommend working with Quirk's categories\xspace for smoother dataset generation and better downstream task performance}.

Our dataset will be available to the public and we hope that it will provide helpful information and insights for other studies as well.
Each framework's dataset will be split into two subsets: those with a label majority with at least 2 annotators agreeing with each other (Agreement subset) and those where there were absolutely no agreement among annotators (Disagreement subset\footnote{However, this disagreement subset is not used in our experiments.}) (Table \ref{tab:agreement_disagreement_table_count}).
Our fine-tuned classifiers will also be released with the dataset for those who need an easy-to-use modal verb intent classifier or find that it can help performance in other tasks when combined with other resources.

\section{Limitations and Future Work}\label{sec:discussion_and_future_work}
We list several limitations to our work and future plans for handling them. Firstly, this research does not consider modality in other languages, domains, or frameworks. Our conclusions and insights can only be applied to conversational instances of languages that share the same modal verb morphology as English. Therefore, expanding our target text and incorporating more frameworks can greatly increase potential uses for our dataset and thus would be an important next step.

Secondly, our data forces a single label onto each utterance. This is beneficial for training models, but could also mean we are disregarding disagreements that could shed more light onto how people interpret modal verbs. Methods of how to annotate subjective data and handle disagreement have been explored by many \cite{Basile2020ItsTE, Basile2019, Aroyo}. We believe our dataset can be used to test these strategies that propose modifications preventing disagreement to be treated as noise. Future work could also potentially include allowing annotators to express uncertainty on given labels.

The last limitation we discuss is that this work only focuses on utterances with single modal verbs. We would need to conduct more studies to determine how generalizable our work is to longer, more complicated sentences. This may involve working on another domain since conversational text is less often long and complicated.

Other steps for advancing this work would be to use the dataset for specific NLP tasks, such as paraphrasing and inference. One way in which model verbs could be used in inference is to focus on \textit{permission} and \textit{obligation} to see social power dynamics in text (who seems to be receiving/giving permission more than average or who seems to be controlled by more social obligations). Or perhaps, one could investigate the annotations with complete disagreements and determine what caused those disagreements. Identifying what part of the sentence or context prompted certain annotations and lack of agreement would entail high degrees of natural language understanding.

\subsection*{Ethical Considerations}\label{sec:ethics}
We paid $\$1$ for 20 annotated sentences on MTurk, which translated to an average hourly wage of $\$12$. This is higher than both the federal and state minimum wage according to the Minnesota
Department of Labor and Industry. Additionally, recognizing the fact that our HITS were not easy and that annotator blocks can lead to terminated accounts, we utilized qualifications\footnote{Qualifications allow us to blacklist workers who did not reach our standards for this particular task, without jeopardizing their account status.} to prevent workers from submitting additional HITS to our project.

\subsection*{Acknowledgments}\label{sec:acknowledgments}
We thank Dr. Brian Reese and our anonymous reviewers for their valuable, constructive feedback. We would also like to extend our gratitude to the Mechanical Turk workers who made this project possible.
\section{Case Studies of Potential Applications}\label{sec:case_study}

There is some debate as to whether we should focus on modality as a whole since it can be expressed with other vocabularies not limited to modal verbs \cite{Nissim2013CrosslinguisticAO, Pyatkin2021ThePT}. However, we argue that modal verbs alone offer enough complexity, and there are still downstream NLP tasks that would benefit from better modal verb categorization.

Difficulty with modal verb understanding can cause confusion in semantic similarity tasks. Using a RoBERTa Hugging Face model \cite{huggingface-transformer} pretrained on the Microsoft Research Paraphrase Corpus (MRPC) subset of the General Language Understanding Evaluation (GLUE) dataset\footnote{textattack/roberta-base-MRPC},  we saw that given some original sentence, the model was not able to reliably identify the unlikely interpretations. For example, given the sentence, ``My parents said I \textit{can} go'', the model would flag all following three as semantically equivalent by a score of at least 0.73: ``My parents said I have the ability to go.', ``My parents said I might go.'', and ``My parents said I have permission to go''.\footnote{0.978, 0.732, and 0.988 respectively}

As another example, we generated paraphrases for the Empathetic Dialogues dataset \cite{empathetic_dialogues} using the T5 Parrot paraphraser \cite{parrot} in the Hugging Face library.\footnote{prithivida/parrot\_paraphraser\_on\_T5} This revealed that $1951$ out of 2490 (78.35\%) paraphrases created for 865 sentences\footnote{We removed utterances with multiple sentences since paraphrase models will sometimes drop a sentence in an attempt to create a "new" paraphrase.} kept their original modal verbs. This shows that being able to correctly identify and paraphrase the sense of a modal verb can greatly increase variety in paraphrasing.
\input{4-theoretical}
\input{5-dataset}
\input{6-evaluation}
\input{7-conclusion}

\bibliographystyle{acl_natbib}

\section{Experiment Design}
\subsection{Mechanical Turk}
\label{sec:mturk_form}
\vspace{0.5cm}
    \includegraphics[scale=0.45]{figures/instructions.png}
    \captionof{figure}{General instructions given to MTurk workers}
    \label{fig:mturk_instructions}
\vspace{1cm}
    \includegraphics[scale=0.45]{figures/descriptions.png}
    \captionof{figure}{Descriptions given to MTurk workers for Quirk's categories\xspace}
    \label{fig:mturk_quirk_descriptions}
\vspace{1cm}
    \includegraphics[scale=0.45]{figures/example.png}
    \captionof{figure}{Examples given to MTurk workers for Quirk's categories\xspace}
    \label{fig:mturk_quirk_examples}
\end{minipage}

\clearpage

\begin{minipage}{1\textwidth}
    \includegraphics[scale=0.49]{figures/palmer_vocab.png}
    \captionof{figure}{Descriptions given to MTurk workers for Palmer's categories\xspace}
    \label{fig:mturk_palmer_descriptions}
\vspace{1cm}
    \includegraphics[scale=0.49]{figures/palmer_examples.png}
    \captionof{figure}{Examples given to MTurk workers for Palmer's categories\xspace}
    \label{fig:mturk_palmer_examples}
\vspace{1cm}
    \includegraphics[scale=0.41]{figures/sentences.png}
    \captionof{figure}{Example sentences to annotate and the corresponding drop-down boxes for Quirk's categories}
    \label{fig:mturk_sentences}
\vspace{1cm}
    \includegraphics[scale=0.41]{figures/sentences_cat2.png}
    \captionof{figure}{Example sentences to annotate and the corresponding drop-down boxes for Palmer's categories}
    \label{fig:mturk_sentences_cat2}
\end{minipage}

\clearpage

\begin{minipage}{1\textwidth}
    \subsection{Filtering Criteria}
    \label{sec:filtering_criteria}
    Workers were only prevented from working on further HITs when we noticed issues in their annotation quality.
    The issues were detected based on their frequency of disagreement with others and deviation from Quirk's mappings, where he laid out which labels could be assigned to which modal verbs (Table \ref{tab:quirk_categories}). We set the threshold high enough to only filter out the top 1\% of whose responses consistently deviated from both their fellow annotators and Quirk's mappings so as to not bias our data. Extreme deviation from both peers and a well-established framework implies more randomness than genuine subjective differences.\\

    \begin{center}
    \small
    \begin{tabular}{cccccc}

    \toprule
    & \smtextbfsc{\makecell[t]{can/\\could}} & \smtextbfsc{\makecell[t]{may/\\might}} & \smtextbfsc{\makecell[t]{must}} & \smtextbfsc{\makecell[t]{should}} & \smtextbfsc{\makecell[t]{will/\\would}}\\

    \midrule
    \smtextbfsc{possibility} & o & o & x & x & x\\
    \smtextbfsc{ability} & o & x & x & x & x\\
    \smtextbfsc{permission} & o & o & x & x & x\\
    \smtextbfsc{necessity} & x& x & o & x & x\\
    \smtextbfsc{obligation}& x & x & o & o & x\\
    \smtextbfsc{inference} & x & x & x & o & x\\
    \smtextbfsc{prediction} & x & x & x & x & o\\
    \smtextbfsc{volition} & x & x & x & x & o\\
    \bottomrule
    \end{tabular}\\

    \captionof{table}{Label to modal verb mapping as defined by Quirk}
    \label{tab:quirk_categories}
\vspace{0.5cm}
    \begin{tabular}{lll}
    \multicolumn{3}{r}{\# of Annotators}\\
    \toprule
    \textbfsc{\# Annotations} & \textbfsc{Quirk} & \textbfsc{Palmer}\\
    \midrule
    $< 200$ & 87 & 83\\%0
    200 \textasciitilde\texttt{}400 & 6 & 3\\%1
    400 \textasciitilde\texttt{ }600 & 3 & 1\\%2
    600 \textasciitilde\texttt{ }800 & 0 & 2\\%3
    800 \textasciitilde\texttt{ }1000 & 1 & 0\\%4
    1000 \textasciitilde\texttt{ }1200 & 1 & 0\\%5
    1200 \textasciitilde\texttt{ }1400 & 0 & 1\\%6
    1400 \textasciitilde\texttt{ }1600 & 0 & 1\\%7
    1600 \textasciitilde\texttt{ }1800 & 0 & 2\\%8
    1800 \textasciitilde\texttt{ }2000 & 1 & 0\\%9
    2000 \textasciitilde\texttt{ }2200 & 0 & 0\\%10
    2200 \textasciitilde\texttt{ }2400 & 1 & 0\\%11
    2400 \textasciitilde\texttt{ }2600 & 1 & 1\\%12
    \bottomrule
    \end{tabular}
    \captionof{table}{Distribution of how many annotations were contributed by each annotator}
\label{tab:annotator_contribution_distribution}
\end{center}
\end{minipage}

\clearpage

\section{Dataset Statistics}
\label{sec:dataset_statistics}
\begin{minipage}{1\textwidth}
\begin{center}
    \small
    \begin{tabular}{llll}
    \toprule
    \multicolumn{4}{c}{\textbfsc{Quirk's categories\xspace}}\\
    \smtextbfsc{Modal Verb} & \smtextbfsc{Agreement} & \smtextbfsc{Disagreement} & \smtextbfsc{Total}\\
    \midrule
    will  &  564  &  166  &  730 \\
    would  &  424  &  280  &  704 \\
    should  &  406  &  172  &  578 \\
    may  &  153  &  26  &  179 \\
    might  &  347  &  48  &  395 \\
    must  &  431  &  112  &  543 \\
    could  &  445  &  63  &  508 \\
    can  &  780  &  121  &  901 \\
    \midrule
    total & 3550 & 988 & 4538\\
    \midrule

    \multicolumn{4}{c}{}\\
    \multicolumn{4}{c}{\textbfsc{Palmer's categories\xspace}}\\
    \smtextbfsc{Modal Verb} & \smtextbfsc{Agreement} & \smtextbfsc{Disagreement} & \smtextbfsc{Total}\\
    \midrule
    will  &  651  &  79  &  730 \\
    would  &  592  &  113  &  705 \\
    should  &  544  &  34  &  578 \\
    may  &  161  &  18  &  179 \\
    might  &  358  &  37  &  395 \\
    must  &  520  &  23  &  543 \\
    could  &  460  &  48  &  508 \\
    can  &  805  &  96  &  901 \\
    \midrule
    total & 4091 & 448 & 4539\\
    \bottomrule
    \end{tabular}
    \captionof{table}{Proportion of agreements and disagreements within the dataset. The totals do not add up to 4540 because of ``unknown'' labels, which we omitted from the table due to low count, but are included in the dataset.}
    \label{tab:agreement_disagreement_table_count}

\vspace{1cm}
    \small
    \begin{tabular}{llV{7cm}}
    \toprule
    \textbfsc{Combination} & \textbfsc{Proportion} & \textbfsc{Example Utterances}\\
    \midrule
    inference-possibility-prediction & 8.00\%. & That \textit{should} be fun. Pokemon is a great franchise. I have many of the handheld games.\\
    \midrule
    inference-necessity-prediction & 5.16\% & The odds \textit{must} be astronomical, almost like winning the lottery.\\
    \midrule
    possibility-prediction-volition & 4.45\% & Do you mean LeBron James? I was hoping he \textit{would} come to Miami!\\
    \midrule
    ability-inference-possibility & 3.54\% & Oh no. Were you able to get things sorted out? We live far away from family and I know how hard it \textit{can} be especially when there are health concerns.\\
    \midrule
    ability-possibility-prediction & 3.44\% & True, just do not like how the world is inching toward a conflict that \textit{could} spill over to a nuclear war.\\
    \midrule
    deontic-dynamic-epistemic & 92.63\% & I was disappointed by my manager when he told that I \textit{will} probably get my promotion next year(not this year)\\
    \bottomrule
    \end{tabular}
    \captionof{table}{Top conflicting annotation triplets from \texttt{MoVerb}\xspace}
    \label{tab:conflicting_annots}

\end{center}
\end{minipage}

\clearpage
\begin{minipage}{1\textwidth}
\begin{center}
    \includegraphics[scale=0.35]{figures/context_effect.png}
    \captionof{table}{We see no difference in the number of unique annotations (fewer unique annotations meaning less disagreement) and the corresponding utterance length. While this is intuitively surprising, it aligns with findings from \cite{pavlick-kwiatkowski-2019-inherent}.}
    \label{fig:context_effect}
\end{center}
\vspace{1cm}
\section{Classification results}
\label{sec:classification_results}

\end{minipage}

\clearpage
\begin{minipage}{1\textwidth}
\centering
\begin{tabular}{lllll}
\toprule
\textbfsc{Model} & \textbfsc{Learning rate} & \textbfsc{Dataset} & \textbfsc{Validation F1} & \textbfsc{Test F1}\\
\midrule
ALBERT\textsubscript{base}& 5e-6 & Quirk & 75.49 & 79.36\\
BERT\textsubscript{base}& 5e-6 & Quirk & 75.02 & 77.66\\
BERT\textsubscript{large}& 5e-6 & Quirk & 77.88 & 80.56\\
RoBERTa\textsubscript{base}& 5e-6 & Quirk & 79.21 & 80.81\\
\rowcolor{Gray}
RoBERTa\textsubscript{large}& 5e-6 & Quirk & 78.98 & 82.22\\
DistilBERT\textsubscript{base}& 5e-6 & Quirk & 78.1 & 79.19\\
\midrule
ALBERT\textsubscript{base}& 1e-5 & Quirk & 69.61 & 72.67\\
BERT\textsubscript{base}& 1e-5 & Quirk & 77.84 & 78.39\\
BERT\textsubscript{large}& 1e-5& Quirk & 77.99 & 80.23\\
RoBERTa\textsubscript{base}& 1e-5& Quirk & 78.72 & 80.53\\
\rowcolor{Gray}
RoBERTa\textsubscript{large}& 1e-5 & Quirk & 78.63 & 80.62\\
DistilBERT\textsubscript{base}& 1e-5& Quirk & 77.5 & 78\\
\midrule
ALBERT\textsubscript{base}& 2e-5 & Quirk & 70.22 & 73.18\\
BERT\textsubscript{base}& 2e-5 & Quirk & 77.74 & 78.47\\
BERT\textsubscript{large}& 2e-5 & Quirk & 77.80 & 79.19\\
\rowcolor{Gray}
RoBERTa\textsubscript{base}& 2e-5 & Quirk & 78.55 & 79.88\\
RoBERTa\textsubscript{large}& 2e-5 & Quirk & 77.42 & 79.14\\
DistilBERT\textsubscript{base}& 2e-5 & Quirk & 77.02 & 77.80\\
\hline
ALBERT\textsubscript{base}& 5e-6 & Palmer & 74.66 & 75.58\\
BERT\textsubscript{base}& 5e-6 & Palmer & 76.17 & 75.49\\
BERT\textsubscript{large}& 5e-6 & Palmer & 75.22 & 75.11\\
RoBERTa\textsubscript{base}& 5e-6 & Palmer & 76.9 & 77.51\\
\rowcolor{Gray}
RoBERTa\textsubscript{large}& 5e-6 & Palmer & 77.08 & 78.36\\
DistilBERT\textsubscript{base}& 5e-6 & Palmer & 76.37 & 74.5\\
\midrule
ALBERT\textsubscript{base} & 1e-5 & Palmer & 73.63 & 74.36\\
BERT\textsubscript{base} &1e-5 & Palmer & 74.35 & 74.02\\
BERT\textsubscript{large}& 1e-5 & Palmer & 74.27 & 74.68\\
RoBERTa\textsubscript{base}& 1e-5 & Palmer & 75.94 & 76.76\\
\rowcolor{Gray}
RoBERTa\textsubscript{large}& 1e-5& Palmer & 76.09 & 76.85\\
DistilBERT\textsubscript{base}& 1e-5 & Palmer & 74.72 & 73.6\\
\midrule
ALBERT\textsubscript{base}& 2e-5 & Palmer & 74.36 & 74.79\\
BERT\textsubscript{base}& 2e-5 & Palmer & 73.66 & 72.76\\
BERT\textsubscript{large}& 2e-5 & Palmer & 73.63 & 74.16\\
\rowcolor{Gray}
RoBERTa\textsubscript{base}& 2e-5 & Palmer & 75.46 & 76.57\\
RoBERTa\textsubscript{large}& 2e-5 & Palmer & 70.54 & 70.59\\
DistilBERT\textsubscript{base}& 2e-5 & Palmer & 74.09 & 72.81\\
\bottomrule
\end{tabular}
\captionof{table}{F1 scores for fine-tuned models trained using \texttt{MoVerb}\xspace, averaged over a 10-fold cross-validation.}
\label{tab:classifier_res2}
\end{minipage}
\clearpage
\begin{minipage}{1\textwidth}
\centering
\begin{tabular}{lllll}
\toprule
\textbfsc{Model} & \textbfsc{Learning rate} & \textbfsc{Dataset} & \textbfsc{Validation F1} & \textbfsc{Test F1}\\
\midrule
ALBERT\textsubscript{base}& 5e-6 & Palmer $\rightarrow$ R\&R\xspace & 74.26 & 47.4\\
BERT\textsubscript{base}& 5e-6 & Palmer $\rightarrow$ R\&R\xspace & 75.77 & 42.88\\
BERT\textsubscript{large}& 5e-6 & Palmer $\rightarrow$ R\&R\xspace & 75.72 & 42.29\\
RoBERTa\textsubscript{base}& 5e-6 & Palmer $\rightarrow$ R\&R\xspace & 76.89 & 52.53\\
\rowcolor{Gray}
RoBERTa\textsubscript{large}& 5e-6 & Palmer $\rightarrow$ R\&R\xspace & 76.61 & 54.78\\
DistilBERT\textsubscript{base}& 5e-6 & Palmer $\rightarrow$ R\&R\xspace & 75.74 & 47.71\\
\midrule
ALBERT\textsubscript{base}& 1e-5 & Palmer $\rightarrow$ R\&R\xspace & 71.16 & 42.09\\
BERT\textsubscript{base}& 1e-5 & Palmer $\rightarrow$ R\&R\xspace & 74.8 & 48.44\\
BERT\textsubscript{large}& 1e-5 & Palmer $\rightarrow$ R\&R\xspace & 74.57 & 50.72\\
\rowcolor{Gray}
RoBERTa\textsubscript{base}& 1e-5 & Palmer $\rightarrow$ R\&R\xspace & 75.41 & 57.99\\
RoBERTa\textsubscript{large}& 1e-5 & Palmer $\rightarrow$ R\&R\xspace & 70.47 & 57.75\\
DistilBERT\textsubscript{base}& 1e-5 & Palmer $\rightarrow$ R\&R\xspace & 74.19 & 54.58\\
\rowcolor{Gray}
\midrule
ALBERT\textsubscript{base} & 2e-5 & Palmer $\rightarrow$ R\&R\xspace & 73.64 & 52.75\\
BERT\textsubscript{base}& 2e-5 & Palmer $\rightarrow$ R\&R\xspace & 74.18 & 55.72\\
BERT\textsubscript{large}& 2e-5 & Palmer $\rightarrow$ R\&R\xspace & 74.29 & 57.4\\
\rowcolor{Gray}
RoBERTa\textsubscript{base}& 2e-5 & Palmer $\rightarrow$ R\&R\xspace & 75.4 & 61.44\\
RoBERTa\textsubscript{large}& 2e-5 & Palmer $\rightarrow$ R\&R\xspace & 70.3 & 59.1\\
DistilBERT\textsubscript{base}& 2e-5 & Palmer $\rightarrow$ R\&R\xspace & 73.7 & 57.56\\
\midrule
ALBERT\textsubscript{base} & 5e-6 & R\&R\xspace $\rightarrow$ Palmer & 83.41 & 37.08\\
BERT\textsubscript{base} & 5e-6 & R\&R\xspace $\rightarrow$ Palmer & 80.91 & 56.11\\
BERT\textsubscript{large} & 5e-6 & R\&R\xspace $\rightarrow$ Palmer & 81.35 & 52.35\\
RoBERTa\textsubscript{base}& 5e-6 & R\&R\xspace $\rightarrow$ Palmer & 85.76 & 57.15\\
\rowcolor{Gray}
RoBERTa\textsubscript{large}& 5e-6 & R\&R\xspace $\rightarrow$ Palmer & 86.5 & 66.37\\
DistilBERT\textsubscript{base} & 5e-6 & R\&R\xspace $\rightarrow$ Palmer & 82.71 & 56.36\\
\hline
ALBERT\textsubscript{base}& 1e-5 & R\&R\xspace $\rightarrow$ Palmer & 81.47 & 46.08\\
BERT\textsubscript{base}& 1e-5 & R\&R\xspace $\rightarrow$ Palmer & 81.82 & 57.23\\
BERT\textsubscript{large}& 1e-5 & R\&R\xspace $\rightarrow$ Palmer & 82.44 & 53.89\\
RoBERTa\textsubscript{base}& 1e-5 & R\&R\xspace $\rightarrow$ Palmer & 85.22 & 58.2\\
\rowcolor{Gray}
RoBERTa\textsubscript{large}& 1e-5 & R\&R\xspace $\rightarrow$ Palmer & 88.07 & 65.4\\
DistilBERT\textsubscript{base}& 1e-5 & R\&R\xspace $\rightarrow$ Palmer & 81.88 & 55\\
\midrule
ALBERT\textsubscript{base}& 2e-5 & R\&R\xspace $\rightarrow$ Palmer & 80.94 & 43.96\\
BERT\textsubscript{base}& 2e-5 & R\&R\xspace $\rightarrow$ Palmer & 82.74 & 57.13\\
BERT\textsubscript{large}& 2e-5 & R\&R\xspace $\rightarrow$ Palmer & 84.13 & 58.89\\
\rowcolor{Gray}
RoBERTa\textsubscript{base}& 2e-5 & R\&R\xspace $\rightarrow$ Palmer & 84.04 & 60.71\\
RoBERTa\textsubscript{large}& 2e-5 & R\&R\xspace $\rightarrow$ Palmer & 79.61 & 59.12\\
DistilBERT\textsubscript{base}& 2e-5 & R\&R\xspace $\rightarrow$ Palmer & 80.45 & 57.36\\
\rowcolor{Gray}
\bottomrule
\end{tabular}
\captionof{table}{Observing cross-domain transferability between Palmer's categories\xspace and Ruppenhofer and Rehbein (R\&R\xspace). We see a clear performance domination of the RoBERTa models.}
\label{tab:classifier_res4}

\end{minipage}

redpajama_set_name: RedPajamaArXiv


text:
Oregon Ducks Men's Basketball
Oregon State Beavers Football
Oregon State Beavers Men's Basketball
Judge: South Carolina GOP can cancel its 2020 primary
By MEG KINNARD
AP-US-Election-2020-Republican-Primaries, 1st Ld-Writethru
Matt Rourke, ASSOCIATED PRESS
COLUMBIA, S.C. (AP) — A judge on Wednesday upheld the South Carolina Republican Party's decision not to hold a 2020 presidential primary, a move taken by several states in erecting hurdles for the long-shot candidates challenging President Donald Trump.
In her order, Circuit Judge Jocelyn Newman wrote the law "does not give Plaintiffs a legal right to a presidential preference primary, and the Court will not substitute its own judgment for that of the General Assembly or the SCGOP."
Earlier this year, former South Carolina congressman Bob Inglis sued state Republicans, saying the party's decision to skip a primary deprives him and others "of the ability to vote for the candidate of their choice in South Carolina's famous (and particularly influential) 'First in the South' primary."
South Carolina is among several states that have canceled Republican primaries and caucuses next year, an effort that helps Trump consolidate his support as Democrats work to winnow their large candidate field. The move, taken September in South Carolina by the state party's executive committee, is not unusual for the party of the White House incumbent seeking reelection.
Challengers have emerged to Trump, including former Massachusetts Gov. Bill Weld and former Illinois congressman Joe Walsh. Mark Sanford, a former congressman and governor from South Carolina, entered the race but left after two months.
In years past, both Republicans and Democrats have cut state nominating contests when an incumbent president from their party sought a second term. In 1984, South Carolina GOP leaders opted to call off their primary as President Ronald Reagan sought reelection. In 2004, the GOP again canceled the state's primary, with leaders deciding instead to endorse President George W. Bush's reelection.
The South Carolina Democratic Party didn't hold presidential primaries in 1996 or in 2012, when Presidents Bill Clinton and Barack Obama were their incumbents.
Meg Kinnard can be reached at http://twitter.com/MegKinnardAP
Catch up on the 2020 election campaign with AP experts on our weekly politics podcast, "Ground Game."
South Carolina state government
U.S. Republican Party
People in the Story
Bob Inglis
Grants Pass Daily Courier

redpajama_set_name: RedPajamaCommonCrawl


text:
Q: Cannot Create OAuth Consumer I am trying to set up OAuth access to an API for a site running 1.9.3.2. All of the Magento instructions say:

You can register your application by selecting System > Web Services > REST - OAuth Consumers and clicking Add New in the Admin Panel.

Source
The problem with this is that the dashboard I am looking at does not have any "REST - OAuth*" options in the System menu.

I haven't been able to find any configuration anywhere to enable this or any plugin that may be conflicting and I am at a loss as to why these options are here when everything seems to indicate they should just always be in that menu.
Is there something that needs to be enabled somewhere that I am unaware of? Or something that could conflict and cause these menu items to not appear?

A: I figured out my issue. In System -> Configuration -> Advanced -> Advanced, I had to enable Mage OAuth. Then I had to go to System -> Cache Management and run Flush Magento Cache. I was unaware that settings and modules for the actual admin dashboard would also be cached and not automatically flushed when settings are updated.

redpajama_set_name: RedPajamaStackExchange


text:
Jollibee Foods Corp. CEO Ernesto Tanmantiong: 'We Are Humbled to Be Loved by So Many People Across the Globe'
The Jollibee Group faced its toughest challenge yet over the past year, but it has quickly rebounded and remains focused on its goal of becoming one of the top 5 restaurant companies in the world.
by Paul John Caña
We present this year's Businessman of the Year, Jollibee Foods Corp. President and CEO Ernesto Tanmantiong
PHOTO: Joshua Tan
Like many other industries, the hospitality and food and beverage sectors experienced tremendous losses over the past year. Reduced hours and store closures were a common sight as strict lockdowns and quarantine restrictions forced people indoors. Jollibee Foods Corporation (JFC), the country's largest food service network, was no exception, posting its first ever net loss in its 46-year history in 2020.
While the company was forced to close down a number of stores last year, it continues to chug along and opened even more stores domestically and overseas, including first-ever openings of its flagship Jollibee brand in places like Rome, London's West End, and Spain. There are clear signs of recovery as management recalibrates and continues to focus on its goal of becoming one of the top restaurant companies in the world. Clearly the 'Bee is still flying high and has no plans of slowing down.
We honor JFC and its management team led by Chairman Tony TanCaktiong and President and CEO Ernesto Tanmantiong for being a symbol of the Filipino's entrepreneurial spirit and becoming a beacon of joy and hope in a year wracked by nearly endless news of strife and grief. In this rare interview, Tanmantiong discusses the company's most challenging year ever; its efforts to assist its employees, the community, and the country amidst the pandemic; its acquisitions strategy; and what his favorite thing is to order when he pops in to visit a Jollibee store.
ESQUIRE PHILIPPINES: Can you please give us an overview of how the pandemic has affected your business? Would you say this has been the most difficult challenge you have faced as a businessman? If not, how does this compare to previous challenges?
Ernesto Tanmantiong: This ongoing pandemic has been our biggest challenge so far—perhaps even more so than the Asian financial crisis back in the late '90s.
The quick-service restaurant industry was among the hardest-hit sectors by the pandemic. Even though we all are trying hard, this crisis spared nobody. For the Jollibee Group, this meant we had to close hundreds of stores, affecting many of our people, and resulting in a loss in 2020—a first in our company's history. This number could have gone higher if not for the substantial pivots we made to adapt to the changing needs of our customers and our business transformation. I cannot emphasize enough our appreciation for the hard work of our people and our partners, and through their collective effort, we swung back to profitability by the fourth quarter of 2020 and continue to grow and expand in 2021.
ESQ: What steps did the company take to address revenue and profit losses? What measures did you adopt to cut expenses and preserve cash?
ET: We continue to run the business in more cost-efficient ways: streamlining operational processes; consolidating commissaries and services that can be shared across brands; rationalizing non-performing stores; and shifting ad budgets to more digital media, which allows better targeting at a lesser cost vs. traditional media.
More significantly, we also took steps recognizing that the pandemic brought shifts and lasting changes to consumer behavior, such as the greater use of digital, work from home, and more reliance on delivery and take-out. As part of our Business Transformation, we continued building our off-premise channels such as delivery, drive-thru, and take-out. For example, our strengthened partnership with aggregators has resulted to sales growth of two to six times, depending on the market. We have implemented call and pick-up to help boost take-out sales, and even curbside pick-up for stores that do not have a drive-thru.
We also speeded the turnaround time of our drive-thru operations to service more customers. We also opened Cloud Kitchens and Delivery and Takeout Stores, which are locations dedicated to delivery and/or takeout in lower rent areas that allow us to expand faster with less capital requirements. We also have Mobile Kitchens that can travel to different places where people can get our food without having to build a brick-and-mortar store.
Aside from these, we introduced new Family and Group Meals as people often dine at home in groups now vs before. We also introduced new products like Jollibee's Spicy Champ, Nasi Lemak Chickenjoy in Singapore and Malaysia, Greenwich's Extreme Overload Pizza, Mang Inasal's Liempo, and other products to attract certain segments. Across all channels, we continue to highlight value for money through our delicious but still affordable products, which remains highly relevant during these times.
Although much progress has already been made, we know that we are still not out of the woods yet. There is still much for us to learn and do, but with these efforts, I believe JFC will emerge stronger than ever before.
ESQ: Could you please tell us about your company's own COVID-19 response for your staff members and employees? I understand the company set aside P1 billion as an emergency response fund for its own employees. What was the thinking behind such a generous amount dedicated just for COVID relief?
ET: It was important to us to ensure our people's safety and well-being. We would not have made it this far if not for their commitment and unwavering service, and we wanted to extend as much support as we could. We allocated a P1 billion emergency response fund to provide salaries and subsidies even when the stores were closed and they couldn't work and advanced the 13th month pay from December to April to help with their financial needs. This covered all of our store work teams, including senior citizens and PWDs, and it was also extended to our partner employers to give financial support to their respective employees.
On top of the emergency fund, we have also prioritized our people's health and safety, whether they are working on-site or at home. We implemented a health and wellness program that provided our employees with PPEs, free shuttle services, free vitamins, health webinars, and access to mental health professionals to ensure their holistic well-being. We also vaccinated our employees for free under our #JabForJoy program and extended the sourcing of vaccines for their families and dependents. We take care of our employees because they are at the heart of our company.
ESQ: JFC has also actively contributed to healthcare workers, law enforcement personnel, and other frontliners during the height of lockdown and strict quarantine periods during the past year. Could you also talk about the company's efforts to help the government and public sector navigate the health crisis?
ET: Since we are in the restaurant industry, we felt we could do the most good by focusing on food accessibility, which is the main thrust of the Jollibee Group Foundation. We ramped up our FoodAID Program and since the start of the pandemic in 2020, we have donated over P300 million worth of food to thousands of frontliners and families who were severely affected by the pandemic.
Together with civic groups and the private sector, we also initiated Project Karinderya to re-establish 320 karinderyas or eateries that have closed due to the pandemic. By re-opening and training them well, we were able to improve the livelihoods of the Karinderya owners and assist other families in the community by providing them free food vouchers that they could claim in the Karinderyas, creating a synergy. To date, 6,400 family beneficiaries or approximately 32,000 individuals have benefited from this program. As with our other programs, we hope to give not just one-off assistance, but a lasting, sustainable impact on the community.
We are also working with the Inter-Agency Task Force, leveraging our experience in supply chain operations, to fast-track vaccinations to more Filipinos and help the county achieve herd immunity the soonest time possible.
ESQ: JFC has also been on an aggressive expansion phase as of late, but especially in the past year. The company has made no secret of its plans of becoming one of the world's top food companies. Where is the company now in terms of that goal? What are the next steps and what else needs to be done?
ET: We continue to dream big, and even despite the changes brought by the pandemic, our vision remains unchanged: to become one of the top five restaurant companies in the world by serving great tasting food and bringing the joy of eating to everyone. Towards this, we did not just aim for recovery, but pushed for expansion. Hence, we accelerated the growth of our international business in the midst of the pandemic, with many milestones such as our first Jollibee in Rome, London's famous West End, Wales, Maryland State and San Antonio, Texas, in the U.S., and, just a few months ago, we opened our first Jollibee in Spain.
Jollibee appeals to a more global and diverse audience and not only Filipinos, with stores in the U.K., Hong Kong, Singapore, Brunei, and Vietnam seeing more and more local customers, if not the majority. We are also encouraged by the growth of our other international brands such as Yonghe King and Hong Zhuang Yuan in China; and Smashburger, Chowking, Red Ribbon, aside from Jollibee in North America. All of them are now doing even better today versus pre-pandemic.
We target a 50-50 split in our Domestic and International sales, and with our accelerated international expansion, we are closer to that goal than ever before. In 2020, the majority of our openings happened abroad—a first in our history. This year, we are targeting to open 450 stores. We are optimistic that our existing and upcoming international stores will help us further grow as more countries reopen their economies.
ESQ: Could you walk us through the strategy behind acquisitions? How do you determine which companies would work within the JFC ecosystem and would complement JFC's own business?
ET: Acquisitions have always played an important role in our growth. We look at the track record of a prospective company and assess the company's scalability and potential for broad acceptance across a country, and even globally. We also look for partners whose vision and goals are anchored on the same set of values that guide our organization.
The Coffee Bean and Tea Leaf is by far our largest acquisition, with over 1,000 outlets across 27 countries. Our strategic intent is to be big in coffee, which is in the very lucrative beverage segment expected to significantly grow globally. This also synergizes our expertise from operating Highlands Coffee in Vietnam to produce better, higher-quality coffee beans for our customers.
We also have the Michelin-starred Tim Ho Wan as one of our brands, and we have seen long queues and jampacked restaurants in all the restaurants we've opened in the past months. We are excited to open 100 stores in Mainland China in the next five years.
Recently, we announced our plan to acquire 51 percent of Taiwan's best-rated milk tea brand Milksha, which has over 250 outlets in five countries. This acquisition allows us to compete in this fast-growing beverage category and, together with its founder, rapidly grow the brand globally. We are also very excited to introduce Milksha to Filipinos soon through Chowking.
In North America, Smashburger is doing better today even compared to pre-pandemic. In the Philippines, our goal is to be the leading developer of international restaurants, bringing in and operating brands such as Panda Express, Burger King, PHO24, and Yoshinoya.
ESQ: How do you deal with competition?
ET: Many of the changes we've gone through were always predicated on our customers—from the time we shifted to hot meals from ice cream during our early ice cream parlor beginnings to today, where we pivot our business to adapt to the changing habits brought by the pandemic. We always strive to stay in tune with our customers, from their shift to digital, reliance on delivery and take-out, and dining in larger groups with their families at home. We responded as quickly as possible.
We focus more on serving delicious food, because in the end, good taste knows no borders and boundaries. Despite the many pivots that we've done, we haven't changed our commitment in giving quality, superior tasting food.
ESQ: How do you balance ensuring the viability and sustainability of the business with corporate responsibility to contribute to the community and nation-building?
ET: As the Jollibee Group grows, we want to make sure that the communities we serve grow with us. On top of providing employment opportunities, we also leverage on our expertise as a food company to help communities through the Jollibee Group Foundation.
Our programs in the foundation are centered around access to food. The Farmer Entrepreneurship Program trains small farmers to become entrepreneurs, providing them a set up so they can directly sell to institutional buyers such as JFC, thus improving their livelihoods substantially and sustainably. To date, we have bought P300 million worth of fresh produce from the farmer groups, benefitting hundreds of farmers.
On the other hand, our school feeding program, called Busog, Lusog, Talino (BLT), aims to address hunger among grade school pupils to help them stay in school by providing nutritious daily lunch meals prepared in collaboration with the pupils' parents, the Local Government Unit, and other partners. Through the BLT program, we've served over 240,000 pupils since 2007.
ESQ: Often, your names are included in "richest people" lists like the one Forbes comes out with every year. How do you feel about these lists and what is your reaction when your names are included?
ET: It makes me more appreciative of how far we've come from our small ice cream house beginnings. It was really about working hard and dreaming big. More importantly, Jollibee Group would not be where it is today if not for the people and partners we've worked with all these years. I am grateful to everyone for sharing our vision and working hard together to achieve it.
ESQ: Please tell us about how you spent the most severe lockdown periods last year. What did you do to keep busy? Or did this become a time for you to slow down and take it easy?
ET: Pre-pandemic, I used to spend half of my time on foreign business travels. But when lockdown and quarantine restrictions were implemented, I spent most of my time attending zoom meetings and webinars on business, finance, health, and digital.
ESQ: If possible, please give us the names of books you read, TV shows and movies you watched, games you played, etc.
ET: The books that I've read during the quarantine include Better, Simpler Strategy by Felix Oberholzer-Gee and The Outsiders by William Thorndike. These books were very thought-provoking. I also read trade magazines to remain attuned with industry trends.
For movies and Netflix series, I watched For Life, The 100, Sweet Tooth, and Money Heist.
For exercise, I do yoga and use the elliptical. I also do morning and evening walks with my wife, Susan. On Sundays, we go around to visit different places in and out of the city to see how the situation is during the different levels of lockdowns.
ESQ: Do you still sometimes go inside Jollibee (or any of the other JFC brands) and order? What are your favorite items to get?
ET: Yes, of course! My favorite is still Chickenjoy and Amazing Aloha Champ from Jollibee. But I also love Chowking's Pork Chao Fan with Siomai and Wonton Mami, as well as Bola-Bola Siopao.
I also love Mang Inasal's Chicken and Pork Barbeque and Palabok. In Red Ribbon, I order Yema Caramel Cake, the Chocolate Dedication Cake, and a lot of Mamon. In Greenwich, my personal favorite is the Hawaiian Extreme Overload. I also go to Burger King and get the 4-Cheese Whopper or Mushroom Swiss King.
ESQ: What are the most important lessons you are teaching your children about business and about life?
ET: I always tell them that dreams are free. But dreaming is not enough. Dream big and put your dreams into action. Be passionate about it. If ever you fail, don't be discouraged as these mistakes are part of the experience. Recognize your shortcomings and learn from them as fast as you can.
Also, do things with excellence. My dad always used to tell us that if you are a tailor, then you have to make sure the clothes you make fits well. If you're opening a restaurant, you have to make sure that the food really tastes good. In the end, I tell my children that this is the goal: to always have the passion to excel in life, but also keep in mind to be of service and do good.
ESQ: Jollibee has become such an iconic and integral part of the Filipino's way of life. How does it feel to have built something that has gone beyond merely being a business into an enterprise that genuinely delivers joy to millions of people every day?
ET: We are inspired to do more for our people and customers, to offer more of our great-tasting food to everyone, and to create more progress for the communities where we operate.
We are humbled and honored to be among the brands that are loved by so many people across the globe. We have come a long way from our ice cream house beginnings in 1975. Back then, we were just looking for a stable source of livelihood for our family. But through everyone's hard work, we've grown into a company with over 5,800 stores across 34 countries carrying 17 brands. We know there's still much to do, and we will not stop in our mission of serving delicious food, bringing the joy of eating to everyone.
ESQ: What's a practical business tip or piece of advice you can share with SMEs or those just thinking of opening their own business?
ET: Our learning is that it is important to know your purpose and to remain steadfast towards that vision and mission. Every crisis and setback comes with a learning moment to progress myself or the company.
We also always prioritized taking care of our people even when times were difficult. Our people are at the heart of our business, and we wouldn't be able to recover as fast as we did without their hard work and passion for the company.
Part of staying true to our mission is also sticking to the fundamentals even if circumstances change. Even if we had to adapt many things, we never sacrificed the quality of our food. For Jollibee Group, our core is to serve great tasting food at good value for money; this is what propelled our success in the past, and will continue to do so in the future.
ESQ: What kind of legacy would you like to leave behind if and when the time comes for you to take a step back from the business?
ET: Even as a small ice cream parlor back in 1975, we always set ourselves to bigger and bigger dreams: from being the best in the country, and when we achieved that, to being the best in Asia, and now to being among the top five in the world. I believe that the next generation of leaders will continue to dream big, and work hard to make the dream happen.
I am happy to share that the big difference of JFC is the culture of our people and how we always have the humility to listen and learn, how we work with the spirit of family and fun, how we focus on our customers and how we do things with speed and excellence and integrity. It is our culture that moves and supercharges our strategy. When the time comes that I step back from the business, I feel confident that the best for JFC is yet to come.
Photographs by Joshua Tan • Special thanks to Edison Lacea of JFC
Jollibee Foods Corp,


text:
Pakistan's Pharmaceutical Companies Introduced Most Expensive Corona Vaccines
Khalid Talat
Ferozsons and Searle Pharma Pakistan have introduced the most expensive injection for the treatment of coronavirus. This injection is available in the market and according to the report, this is a course consisting of 11 vaccines, the cost of one vaccine is Rs. 20,000 while the total cost of this entire course is Rs. 220,000.
A doctor's prescription is required to obtain this injection, as well as a photocopy of the ID card and stamp paper, which must state that the company is not responsible for any side effects or reactions.
According to sources, it is not known whether the injection to be introduced has been approved by the Government of Pakistan or not, but it is good news for those who can afford it, and painful for the poor people because they will never be able to afford such an injection.
Related Topics:corona vaccinecorona vaccine pakistanCoronaVirus
Nexus of Terrorist Organizations in Afghanistan – Security High Alert in Pakistan
Officials from India's intelligence agency RAW and Afghanistan's intelligence agency NDS have held meetings with the TTP and other banned organizations. In which, a conspiracy has been hatched to reorganize the Tehreek-e-Taliban Pakistan (TTP) in Pakistan.
The meetings of Indian and Afghan Intelligence agencies with Taliban organizations have significant importance because Ikramullah Turabi, who was involved in the assassination of Benazir Bhutto, has been seen active in the meeting after a long time.
ALSO READ: Is the Afghan peace process under threat Due to Corona and Why India is Happy about it?
The Indian intelligence agency RAW and the NDS have established a nexus of banned organizations in Afghanistan. A meeting of banned organizations has been held in Paktia and Kantar, in which, an alliance has been formed between Jamaat-ul-Ahrar, Hizb-ul-Ahrar and banned Tehreek-e-Taliban Pakistan under the auspices of RAW and NDS.
The news agency has revealed that Ikramullah Turabi's reactivation and the presence of Indian intelligence agency RAW and Afghanistan's intelligence agency NDS in the meeting of Taliban organizations is an indication that besides sabotaging the atmosphere of peace in Pakistan, it can also affect the Afghan peace process.
The meeting of Hizb-ul-Ahrar, Jamaat-ul-Ahrar, and TTP, led by RAW and NDS, aims to plan a renewed attack on Pakistan's security forces and destroy the atmosphere of peace established after huge sacrifices.
ALSO READ: Why the Current Scenario in Afghanistan is so Complex?
It may be recalled that Ikramullah Turabi was arrested from Afghanistan in December 2013 in the Benazir Bhutto assassination case and was granted bail by the Peshawar High Court. Immediately after being granted bail by the Peshawar High Court, Ikramullah returned to Afghanistan and intensified his terrorist activities.
Young Female Doctor in Karachi Committed Suicide
A devastating incident happened in Karachi where a female doctor, Maha Ali, has committed suicide. According to the police, she locked herself in the washroom and shot herself. Dr. Maha was brought to Jinnah Hospital in critical condition and kept on a ventilator during the treatment but she died during the treatment.
کراچی کے علاقے ڈیفنس فیز 4 میں خاتون ڈاکٹر نے خود کشی کرلی#Maha #Suicide #GTVNewsHD pic.twitter.com/RCXpb5uXyO
— GTV NETWORK HD (@gtvnetworkhd) August 19, 2020
According to SP Clifton Imran Mirza, Dr. Maha was suffering from depression. Initially, the incident appeared to be a case of a domestic problem. According to police, Dr. Maha was 25 years old and was working at South City Hospital. Dr. Maha returned from the hospital just hours before she attempted suicide.
ALSO READ: Women Condition in Pakistan – Reality with Facts & Figures
کراچی : رات ڈیفینس میں خاتون نے خود کو گولی مار کر خود کشی کرلی
خاتون کی شناخت ڈاکٹر ماہا علی کے نام سے ہوئی ہے، پولیس
خاتون نے خود کو واش روم میں بند کرکے گولی ماری
ابتدائی طور پر واقعہ گھریلو ناچاقی کا شاخسانہ معلوم ہوتا ہے،ایس پی کلفٹن عمران مرزا#Karachi pic.twitter.com/q1yf47SP1b
— کراچی والے (@KarachiWalay90) August 19, 2020
According to the police, Dr. Maha tried to commit suicide in the washroom on the first floor of the house and was rushed to the hospital in critical condition. Dr. Maha's father had rented the house 15 days ago in Gizri.
Why the Govt has failed to Control the Rising Price of Sugar in the Country?
According to Ansar Abbasi, while the sugar scandal case was being investigated, the sugar mafia started threatening FIA Director General Wajid Zia over the increase in sugar prices. The sugar mafia demanded that the inquiry be stopped or else a sugar crisis would be created.
In this regard, it was reported on April 5 that the sugar mafia has warned Prime Minister Imran Khan as well as DG FIA and Chairman Inquiry Commission Wajid Zia to stop the investigation into the sugar scandal immediately, otherwise, there will be a severe shortage of sugar in the country, as a result of which the price of sugar can go up to Rs 110 per kg.
According to The News, DG FIA Wajid Zia, who was the head of the commission, was asked from the mafia to stop the investigation, or else the price of sugar would go up to Rs 110. The DG FIA informed the Prime Minister's Office of the threat he had received.
ALSO READ: Imran Khan Directed Federal Agencies to Start Crackdown Against Sugar Mafia
Despite threats and pressure, the Commission of Inquiry not only completed its work but also the Prime Minister fulfilled his promise by publicizing the report. However, the government's efforts to stem the rise in sugar prices have not been successful.
Under the action plan announced by the government to root out the sugar mafia, seven federal and provincial agencies have been tasked to take different types of criminal, regulatory and tax-related measures against those found guilty in the Sugar Commission report.
According to these orders, the politicians whose sugar mills were named in the forensic audit had to be questioned by the NAB and six other state agencies. These agencies include the NAB, FIA, SECP, FBR, Competition Commission, SBP, and provincial anti-corruption departments.
ALSO READ: Sindh High Court Directed the Government to Halt further Action against Sugar Mills
These state agencies had to send a reference to the NAB for an investigation into the total subsidy of Rs 29 billion given to the sugar industry by the provincial and federal governments in the last five years.
The FBR had to investigate the income tax and sales tax issues related to tax evasion and anonymous transactions and submit an investigation report within 90 days.
The Competition Commission was asked to investigate anti-competitive and cartelization issues in the sugar industry and submit its report within 90 days.
ALSO READ: Imran Khan's Victory over Profiteers – Billions of Rupees are Lost by the Mafia
The SBP was ordered to investigate matters related to the sale of stock, waiver of loans, and repayment of loans and exports.
The FIA and SECP were asked to investigate corporate fraud in sugar mills.
The task of provincial anti-corruption agencies was to prosecute sugar mill owners for violations of provincial laws, including paying farmers less than the decided amount and committing illegal deductions.
Institutions were given 90 days for the report, but the facts are that sugar is being sold at Rs 100 per kg in most parts of the country, while prices have risen to Rs 110 in some areas.
Education9 months ago
All Compulsory and Optional Subjects – Download Free Books/Articles for CSS/PMS//NTS/NAB Exams (Update Regularly)
80+ Most Important Essays & Articles (All in One)
Global Power Dynamics and Pakistan's Foreign Policy
The Muslim world is at a new turning point in history. The distance between Arabia and Israel seems to be...
After an illegal action of scrapping the special status of Jammu and Kashmir, India is now adamant to take the...
As we all are apprised of the rack and ruin brought about by the inclement monsoon season in the metropolitan...
Why Shouldn't Pakistan Accept Israel?
Israel is a completely illegitimate state and was created by occupying Muslim territories. The blood of millions of Muslims was...
Officials from India's intelligence agency RAW and Afghanistan's intelligence agency NDS have held meetings with the TTP and other banned...


text:
Iver Bjerkestrand, né le  à Lørenskog, est un skieur alpin norvégien.

Biographie
Actif depuis 2002 dans les courses FIS, il fait ses débuts en Coupe du monde en  à l'occasion du slalom géant de Sölden. Il marque ses premiers points en , avec une  place au super G de Kvitfjell. Deux ans plus tard, pour sa dernière course en Coupe du monde, il est  au même lieu.

En , il obtient son premier et seul podium en Coupe d'Europe au slalom géant de Trysil.

Il compte un titre national à son actif en 2009 au slalom géant.

Il se retire du sport de haut niveau en 2015.

Palmarès

Coupe du monde
 Meilleur classement général :  en 2013.
Meilleur résultat : .

Classements détaillés de Coupe du monde

Coupe d'Europe
 1 podium.

Coupe nord-américaine
 1 victoire.

Championnats de Norvège
Champion en 2009 du slalom géant.

Références

Liens externes

Naissance en juillet 1987
Naissance à Lørenskog
Skieur alpin norvégien

redpajama_set_name: RedPajamaWikipedia


Hi, everybody! New challenge time at Craft Us Crazy: Use black and/or party theme. I went with black, although this could be for a Halloween party, too.
I went crazy with the black glitter on this! Love the black cat and pumpkins (svg from SVG Cuts.com) Come join us for the new challenge at Craft Us Crazy - and check out the inspiration while you're there.
fun & festive, love all the sparkle Lori!
You just keep creating the most amazing Halloween creations Lori, love your glittery cats!
Trick or treat to you - one wonderful Halloween card full of bright pretty colors and lots of sparkle. Can't let a cat be too black now can we!! Love it Lori!
What a wonderful card my crafty friend. I LOVE the black sparkle. Looks perfect.
TOO FUN!!! LOVE the glitter!
Hahahahaha...okay, you win!! I HAD to drag my husband over and show him proof that I am NOT the only one who goes glitter batty! Hopefully you're more organized with it than I am. He's finding it in HIS car now..LOL.
I love Halloween and this screams Halloween! Love it!


Hotel | Accommodation Kirkenes: Scandic Arctic Hotel is located in the heart of Kirkenes, not far from the Russian and Finnish borders.
A melting pot of Norwegian, Finnish, Sami and Russian culture, in a wild and beautiful landscape. The hotel offers a range of rooms, including double/single rooms, rooms with parquet flooring and family rooms. All the rooms are equipped with cabel-TV, radio, mini-bar, telephone and hair-dryer. The hotel also offers a service centre with a PC and laser printer.


text:
import luigi

"""
To run:
   python simple_pipeline.py --local-scheduler
"""

class InputFile(luigi.Task):
   """
   A Task wrapping up a Target object
   """

   def output(self):
      """
      Retun the contents of /etc/passwd
      """
      return luigi.LocalTarget('/etc/passwd')

class MyTask(luigi.Task):
   """

   """

   def requires(self):
      """
      Specity the dependency of the input file
      """
      return InputFile()

   def output(self):
      """
      Specify the file to write to
      """
      return luigi.LocalTarget('/tmp/userNames.txt')

   def run(self):
      """
      Run the Task
      """

      # Open the files specified by requires and output, respectively
      ifp = self.input().open('r')
      ofp = self.output().open('w')

      # Put each username in the output file
      for line in ifp:
         ofp.write('{0}\n'.format(line.strip().split(':')[0]))

      # Don't forget to close the file
      ofp.close()

if __name__ == '__main__':
   luigi.run(main_task_cls=MyTask)

redpajama_set_name: RedPajamaGithub