\section{Introduction}

Throught this paper, let $R$ be a commutative ring with nonzero identity unless otherwise noted.

Since the prime ideals of commutative rings play a critical role in the area of commutative algebra,
various generalizaions of prime ideals have been explored in several studies.
 In \cite{DZ}, D. Zhao introduced the extension function $\delta$ of ideals and defined the $\delta$-primary ideals of rings.
Let $\mathcal{I}(R)$ be the set of all ideals of a ring $R$.
A function $\delta$ from $\mathcal{I}(R)$ to $\mathcal{I}(R)$  is said to be an expansion function or an ideal expansion, if it takes ideals to ideals with $I \subseteq \delta(I)$
and if $I\subseteq J$ for some $I, J\in \mathcal{I}(R) $, then $\delta(I)\subseteq\delta(J)$.
A $\delta$-primary ideal $I$ of a ring $R$ is an ideal satisfying that if $ab\in I$ and $a\notin I$ for some $a, b\in R$, then $b\in \delta(I)$.
In \cite{USK}, the authors introduced the concept of $n$-ideals: a proper ideal is called $n$-ideal if for any $a, b\in R$ with $ab\in I$, then $a\in \sqrt{0}$ or $b\in I=\delta_{0}(I)$.
For an ideal $I \in \mathcal{I}(R)$, $\sqrt{I}$ denots the radical of $I$, that is, $\sqrt{I}=\{r\in R| \text{ there exists } n\in \mathbb{N} \text{ such that } r^n\in I\}$.
Recently, E. Yetkin Celikel and G. Ulucak introduced the concepts of  $\delta$-$n$-ideals and quasi-$J$ ideals of a commutative ring $R$ in \cite{EYC,EG1}.
An ideal $I$ is said to be a $\delta$-$n$-ideal if for some $a, b\in R$ and $ab\in I$ with $a \notin \sqrt{0}$, then $b\in \delta(I)$.
For an ideal $I$, it is called a quasi-$J$-ideal of a ring if $ab\in I$ for some $a, b\in R$, then $a\in J(R)$ or $b\in \sqrt{I}=\delta_{1}(I)$. And this concept is a generalization of $J$-ideals appeared in \cite{KB}.

In this paper, we defined the $\delta$-$J$-ideal of a commutative ring,  that is, if $a, b\in R$ and $ab\in I\in \mathcal{I}(R)$, then $a\in J(R)$ (the Jacobson radical of $R$)   or  $b\in \delta(I)$. This definition generalizes the results in \cite{EYC,EG1}.
More precisely, on the one side, $\sqrt{0}$ is always included in $J(R)$, thus this concept generalizes the  $\delta$-$n$-ideal in \cite{EG1}.
On the other hand, $\sqrt{I}=\delta_{1}(I)$ becomes a special expansion function of $\mathcal{I}(R)$, thus the concept of $\delta$-$J$-ideals generalizes the quasi-$J$-ideal in \cite{EYC}. In addtion, we also present some properties about the operations of $\delta$-$J$-ideals  such as localizations, homomorphic images, idealization and so on.

	\section{Properties of $\delta$-$J$-ideals}
Let $R$ be a commutative ring with nonzero identity $1$,  $\mathcal{I}(R)$ be the set of all ideals of $R$, $\delta$ be an expansion function of $\mathcal{I}(R)$, and $J(R)$ be the Jacobson radical of $R$.
For more definitions and properties of idealization used in this paper, one can refer to literatures \cite{MF,DDM,JH}.

Now, let's present the definition of the $\delta$-$J$-ideal.
\begin{definition}
	Let $R$ be a ring, a proper ideal $I$ of $R$ is said to be a $\delta$-$J$-ideal, if whenever $a, b \in R$, $ab \in I$,  then $a\in J(R)$  or $b\in \delta(I)$.
\end{definition}

This definition is different to the conccepts of $\delta$-$n$-ideals and quasi-$J$-ideals, and it is more general than these two concepts.

In the following result, we presented some equivalent conditions of $\delta$-$J$-ideal of a ring in some special circumstance.
\begin{theorem}\label{th1}
	Let $I$ be a proper ideal of ring $R$. Then the following statements are equivalent:
	\begin{enumerate}[(1)]
		\item$I$ is a $\delta$-$J$-ideal of $R$.
		\item If $a \in R$ and $K$ is an ideal of $R$ with $aK$$\subseteq$$I$, then $a\in J(R)$ or $K\subseteq \delta (I)$ if $a\notin J(R)$.
		\item If $K$ and $L$ are ideals of $R$ with $KL \subseteq I$, then $K \subseteq J(R)$ or $L \subseteq \delta(I)$.
	\end{enumerate}
\end{theorem}

\begin{proof}
	(1)$\Rightarrow$(2): Suppose that $I$ is a $\delta$-$J$-ideal of $R$, $aK\subseteq I$ and $a \notin J(R)$. Then if $as \in I$, for any $s\in K$, it follows that $s\in \delta(I) $.  Furthermore, $K\subseteq \delta(I)$.

	(2)$\Rightarrow$(3): Suppose that $KL\subseteq I$ and $K\nsubseteq J(R)$. Then there exists $a \in K\setminus J(R)$. Since $aL \subseteq I$ and $a \notin J(R)$, we have $L\subseteq \delta(I)$ by $(2)$.

	(3)$\Rightarrow$(1): Suppose that $a, b \in R$ and $ab \in I$, the result follows by letting $K= \langle a \rangle $ and $L= \langle b \rangle $ in $(3)$.
\end{proof}

\begin{proposition}\label{prop1}
	Let $I$ be an ideal of $R$ with $\delta(I)\neq R$, then the following statements are equivalent:
	\begin{enumerate}[(1)]
		\item $I$ is a $\delta$-$J$-ideal of $R$.
		\item $I\subseteq J(R)$ and if whenever $a, b \in R$ with $ab \in I$, the $a\in J(I)$ or $b\in \delta(I)$.
	\end{enumerate}
\end{proposition}

\begin{proof}
	(1)$\Rightarrow$(2): Suppose that $I$ be a $\delta$-$J$-ideal of $R$. Assume that $I\nsubseteq J(R)$.
Then there exists an element $a\in R$ with $a\in I\setminus J(R)$. Since $a= a\cdot 1\in I$ and $a\notin J(R)$, it follows that $1\in \delta(I)$, which is a contradition.
Thus $I\subseteq J(R)$, and the other part in $(2)$ follows clearly since $J(R)\subseteq J(I)$.

	(2)$\Rightarrow$(1): Suppose that $ab \in I$ and $a\notin J(R)$. Since $I \subseteq J(R)$, we conclude that $J(I)\subseteq J(J(R))= J(R)$ and $a \notin J(I)$.
Thus, $b\in \delta(I)$,  $I$ is a $\delta$-$J$-ideal of $R$.

\end{proof}
Next, we characterize the ring which every proper ideal is a $\delta$-$J$-ideal.

\begin{theorem}
	Let $R$ be a ring such that $\delta(I)\neq R$ for all proper ideal $I$ of  $R$. Then the following statements are equivalent:
	\begin{enumerate}[(1)]
		\item $R$ is a quasi-local ring with maximal ideal $M= J(R)$.
		\item Every proper principal ideal is a $\delta$-$J$-ideal of $R$.
		\item Every proper ideal is a $\delta$-$J$-ideal of $R$.
	\end{enumerate}
\end{theorem}

\begin{proof}
	(1)$\Rightarrow$(2): Suppose that $(R, J(R))$ is a quasi-local ring.
Then every element of $R$ is either an unit or belongs to $J(R)$.
Let $I=\langle x\rangle$ be a principal ideal and let $a, b\in R$ such that $ab\in I$ and $a\notin J(R)$. Then $a$ is an unit and $b\in I\subseteq \delta(I)$. So $I$ is a $\delta$-$J$-ideal.

	(2)$\Rightarrow$(3): Let $I$ be a proper ideal of $R$. Suppose that $a,b \in  R$ with $ab\in I$ and $a\notin J(R)$. Put $J=\langle ab\rangle$.Then $J$ is a $\delta$-$J$-ideal by (2). Therefore, $b\in \delta(J)\subseteq \delta(I)$, $I$ is also a $\delta$-$J$-ideal.

	(3)$\Rightarrow$(1): Let $I$ be a maximal ideal of $R$. Then $I$ is a $\delta$-$J$-ideal by assumption, thus $I=J(R)$ by Proposition \ref{prop1}.
\end{proof}
The following lemma gives a relationship between $\delta$-primary ideal and $\delta$-$J$-ideal.  We also consider the connection between maximal ideals and $\delta$-$J$-ideals.

\begin{lemma}
	Let $\delta$ be an expansion of ideals of $R$. Then the followings are hold:

	\begin{enumerate}[(1)]
		\item Let $I$ be a $\delta$-primary ideal of $R$ with $\delta(I)\neq R$. Then $I$ is a $\delta$-$J$-ideal of $R$ if and only if $I\subseteq J(R)$.
		\item Let $I$ be a maximal ideal of $R$ with $\delta(I)\neq R$. Then $I$ is a $\delta$-$J$-ideal of $R$ if and only if $I=J(R)$.

	\end{enumerate}
\end{lemma}

\begin{proof}
	(1) Suppose that $I\subseteq J(R)$. Choose $a, b \in R$ with $ab\in I$ and $a\notin J(R)$, hence $a\notin I$.
Since $I$ is a $\delta$-primary ideal, we have $b\in \delta(I)$, that is,  $I$ is a $\delta$-$J$-ideal. The other side is obvious.

	(2) Suppose that $I$ is a $\delta$-$J$-ideal. Since $I$ is maximal, we have $J(R)\subseteq I$. Then $I$ is a $\delta$-primary ideal. It follows from (1) that $I\subseteq J(R)$. Thus the equality $I=J(R)$ holds. Conversely, if $I=J(R)$, then $I$ is a $\delta$-$J$-ideal.
	\end{proof}

Now, we recall the concept of ideal quotient. Let $I, J$ be ideals of $R$, their ideal quotient is $(I:J)=\{x\in R|xJ\subseteq I\}$. If $J$ is a principal ideal $\langle x \rangle$, we shall write $(I:x)$ in place of $(I:\langle x \rangle)$.

\begin{lemma}\label{lemma2}
	Let $\delta$ be an expansion of $\mathcal{I}(R)$. If $I$ is a $\delta $-$J$-ideal of $R$ such that $(\delta(I): x) \subseteq \delta ((I: x))\neq R$ for all $x \notin R\setminus \delta(I)$, then $(I: x)$ is a $\delta$-$J$-ideal.
\end{lemma}

\begin{proof}
	Suppose that $a, b\in R$ with $ab\in (I:x)$ and $a\notin J(R)$. Since $abx \in I$ and $I$ is $\delta$-$J$-ideal, we have $bx \in \delta(I)$. Thus $b\in (\delta(I): x)\subseteq \delta((I: x))$, $(I: x)$ is a $\delta$-$J$-ideal.
\end{proof}

\begin{theorem}\label{th6}
	Let $I$ be a maximal $\delta$-$J$-ideal of $R$ and $(\delta(I): x)\subseteq \delta((I: x))$, for all $x\notin R\setminus(\delta(I)\cup J(R))$. Then $I$ is a $J$-ideal.

\end{theorem}

\begin{proof}
	Suppose that $I$ is a maximal $\delta$-$J$-ideal of $R$. Let $a, b\in R$ such that $ab\in I$ and $a\notin J(R)$. Then $(I: a)$ is a $\delta$-$J$-ideal of $R$ by Lemma \ref{lemma2}. Since $I$ is a maximal $\delta$-$J$-ideal and $I\subseteq (I: a)$, then $b\in (I: a)= I$. Therefore, $I$ is a $J$-ideal of $R$.
\end{proof}

Now, we can propose the following results.
\begin{corallary}
	Let $R$ be a ring, $\delta(I)\neq R$ for all proper ideal of $R$, and $(\delta(I): x)\subseteq \delta((I: x))$ for all $x\notin R\setminus(\delta(I)\cup J(R))$. Then the following statements are equivalent:
	\begin{enumerate}[(1)]
		\item $J(R)$ is a $J$-ideal of $R$.
		\item $J(R)$ is a $\delta$-$J$-ideal  of $R$.
		\item $J(R)$ is a prime ideal of $R$.
	\end{enumerate}
\end{corallary}

\begin{proof}
	(1)$\Leftrightarrow$(3): Clearly.

	(1)$\Rightarrow$(2): Clearly.

	(2)$\Rightarrow$(1): For any $\delta$-$J$-ideal of $R$, we have $I\subseteq J(R)$. Thus $J(R)$ is the maximal $\delta$-$J$-ideal of $R$. Then $J(R)$ is a $J$-ideal by Theorem $\ref{th6}$.
\end{proof}

\begin{proposition}
	Let $\delta$ be an expansion function of  $\mathcal{I}(R)$ and $I$ be a proper ideal of $R$ with $\delta(\delta(I))= \delta(I)$. Then the followings are holds:

	\begin{enumerate}[(1)]
		\item If $I$ is a $\delta$-$J$-ideal and $a\notin J(R)$, then $\delta((I: a))= \delta(I)$.
		\item $\delta(I)$ is a $J$-ideal if and only if $\delta(I)$ is a $\delta$-$J$-ideal.
		\item If $IK = JK $ and $I, J$ are $\delta$-$J$-ideals of $R$ with $\delta(\delta(J))= \delta(J)$ and $K \cap (R-J(R)) \neq \emptyset$ for some ideal $K$ of $R$, then $\delta(I)= \delta(J)$.
		\item If $IK$ and $I$ are $\delta$-$J$-ideals of $R$ with $\delta(\delta(IK))= \delta(IK)$ and $K\cap (R-J(R))\neq \emptyset$ for some ideal $K$ of $R$, then $\delta(IK)= \delta(I)$.
	\end{enumerate}
\end{proposition}

\begin{proof}
	(1) Let $I$ be a $\delta$-$J$-ideal and $a\notin J(R)$. Note that $I\subseteq (I: a)$, so $\delta(I)\subseteq \delta((I: a))$.
 Let $x\in (I: a)$. Then $x\in \delta(I)$. Since $xa\in I$ and $a\notin J(R)$. Thus $(I: a)\subseteq \delta(I)$, we get $\delta(I: a)\subseteq \delta(\delta(I))=\delta(I)$. The equality follows.

	(2) Clearly.

	(3) Since $IK = JK\subseteq I, J$. It follows from $JK \subseteq I$ and $K\cap (R-J(R))\neq \emptyset$ that $J\subseteq \delta(I)$.
Similarly, $I\subseteq \delta(J)$.  Moreover, $\delta(\delta(I))=\delta(I)$ and $\delta(\delta(J))=\delta(J)$, Thus $\delta(I)=\delta(J)$.

	(4) Since $IK\subseteq I$, we have $\delta(IK)\subseteq \delta(I)$. We can also get $I\subseteq \delta(IK)$ since $IK\subseteq IK$.
 Note that $K \cap (R-J(R))\neq \emptyset$, thus $\delta(IK)=\delta(I)$ by assumption.

\end{proof}

\begin{proposition}
	Let $\delta $ and $\gamma$ be expansion functions of $R$ and $I$ be a proper ideal of $R$. Then:

	\begin{enumerate}[(1)]
		\item If $\delta(I)$ is a $J$-ideal of $R$, then $I$ is a $\delta$-$J$-ideal of $R$. The converse is also true if $\delta=\delta_{1}$.
		\item Suppose that $\delta(I)\subseteq \gamma(I)$ for any ideal $I$ of $R$. If $I$ is a $\delta$-$J$-ideal of $R$, then $I$ is a $\gamma$-$J$-ideal of $R$.
		\item If $\gamma(I)$ is a $\delta$-$J$-ideal of $R$, then $I$ is a $\delta\circ \gamma$-$J$-ideal of $R$.
	\end{enumerate}
\end{proposition}

\begin{proof}
	(1) Suppose that $ab\in I$ and $a\notin J(R)$ for some $a, b \in R$. Since $I\subseteq \delta(I)$ and $\delta(I)$ is a $J$-ideal, we get $b\in \delta(I)$. Thus $I$ is a $\delta$-$J$-ideal of $R$. Conversely, suppose that $ab\in \delta_{1}(I)$ with $a\notin J(R)$ for some $a, b\in R$, then $(ab)^{n}= a^{n}b^{n}\in I$ for some $n\geq 1$, and clearly $a^{n}\notin J(R)$ because $\delta_{1}(J(R))=J(R)$ (see \cite{MF}). Since $I$ is a $\delta_{1}$-$J$-ideal, we have $b^n\in \delta_{1}(I)$. Thus $b\in \delta_{1}(I)$, as required.

	(2) Clearly.

	(3) Assume that $\gamma(I)$ is a $\delta$-$J$-ideal of $R$. Let $ab\in I$ for some $a, b \in R$ and $a\notin J(R)$. Since $I \subseteq\gamma(I)$, we have $ab\in \gamma(I)$. Since $\gamma(I)$ is a $\delta$-$J$-ideal of $R$, $b\in \delta(\gamma(I))= \delta \circ \gamma(I)$.
\end{proof}

\begin{proposition}
	Let $I, J$ and $K$ be proper ideals of $R$ with $J\subseteq K\subseteq I$. If $I$ is a $\delta$-$J$-ideal of $R$, and $\delta(J)= \delta(I)$, then $K$ is a $\delta$-$J$-ideal of $R$.
\end{proposition}

\begin{proof}
Assume that $I$ is a $\delta$-$J$-ideal of $R$ and $\delta(J)= \delta(I)$. Let $ab\in K$ for some $a, b\in R$. Then by Theorem \ref{th1}, $a\in J(R)$ or $b\in \delta(K)$ since $K\subseteq I$ and $\delta(J) = \delta(I)= \delta(K)$. Thus, $K$ is a $\delta$-$J$-ideal of $R$.
\end{proof}

An expansion $\delta$ is said to be a intersection preserving function if $\delta(I)\cap\delta(J)= \delta(I\cap J)$ for any $I, J\in \mathcal{I}(R)$, see \cite{DZ} for more details.
\begin{proposition}
	Let $\delta$ be an ideal expansion which preserves intersection. Then the following statements are hold:

	\begin{enumerate}[(1)]
		\item If $I_{1}, I_{2}, \ldots, I_{n}$ are $\delta$-$J$-ideals of $R$, then $I= \bigcap_{i=1}^{n} I_{i}$ is a $\delta$-$J$-ideal of $R$.
		\item Let $I_{1}, I_{2}, \ldots, I_{n}$ be of $R$ such that $\delta(I_{i})'s$ are non-comparable prime ideals of each other of $R$. If $I= \bigcap_{i=1}^{n} I_{i}$ is a $\delta$-$J$-ideal of $R$, then $I_{i}$ is a $\delta$-$J$-ideal of $R$ for all $i = 1, 2, \cdots, n$.
	\end{enumerate}
\end{proposition}

\begin{proof}
	(1) Let $ab\in I$ and $b\notin \delta(I)$ for some $a, b \in R$. Since $\delta(I)= \bigcap_{i=1}^{n}\delta(I_{i})$, we have $b\notin \delta(I_{k})$ for some $k\in \{1, \ldots , n\}$. It follows that $a\in J(R)$. Thus $I$ is a $\delta$-$J$-ideal.

	(2) Suppose that $ab \in I_{k}$ and $a\notin J(R)$ for some $k\in \left\{1, \ldots , n\right\}$. Choose $x\in \left(\prod_{i=1,i\neq k}^{n}I_{i} \right)\setminus\delta(I_{k})$. Hence $abx\in \bigcap_{i=1}^{n}I_{i}$. Since $ \bigcap_{i=1}^{n}I_{i}$ is a $\delta$-$J$-ideal, we have $bx\in \delta( \bigcap_{i=1}^{n}I_{i})= \bigcap_{i=1}^{n}\delta(I_{i})\subseteq \delta(I_{k})$. Note that $\delta(I_{k})$ is prime, thus $b\in \delta(I_{k})$.
\end{proof}

Let $R$ and $S$ be two commutative rings and $\delta, \gamma$ be expansion functions of $\mathcal{I}(R), \mathcal{I}(S)$, respectively. If $\delta(f^{-1}(J))= f^{-1}(\gamma(J))$ for all ideal $J$ of $S$ with ring homomorphism $f : R\rightarrow S$, then we call $f$ is a $\delta\gamma$-homomorphism.

\begin{proposition}
	Let $f : R\rightarrow S$ be a  $\delta\gamma$-homomorphism, wher $\delta$ and $\gamma$ are expansion functions of $\mathcal{I}(R)$ and $\mathcal{I}(S)$, respectively. Then the following hold:
	\begin{enumerate}[(1)]
		\item Let $f$ be a monomorphism. If $J$ is a $\gamma$-$J$-ideal of $S$. Then $f^{-1}(J)$ is a $\delta$-$J$-ideal of $R$.
		\item Suppose that $f$ is an epimorphism and $I$ is a proper ideal of $R$ with $\ker f\subseteq I$. If $I$ is a $\delta$-$J$-ideal of $R$, then $f(I)$ is a $\gamma$-$J$-ideal of $S$.
	\end{enumerate}
\end{proposition}

\begin{proof}
	(1) Let $ab\in f^{-1}(J)$ for $a, b\in R$. Then $f(ab)= f(a)f(b)\in J$, which implies $f(a)\in J(S)$ or $f(b)\in \gamma(J)$.
If $f(a)\in J(S)$, then $a\in J(R)$ since $f$ is a monomorphism.
If $f(b)\in \gamma(J)$, then we have $b\in f^{-1}(\gamma(J))= \delta(f^{-1}(J))$ since $f$ is a $\delta\gamma$-homomorphism.
 Thus $f^{-1}$ is a $\delta$-$J$-ideal of $R$.

	(2) Suppose that $a, b\in S$ with $ab\in f(I)$ and $a\notin J(S)$. Since $f$ is an epimorphism, there exist $x, y \in R$ such that $a= f(x), b=f(y)$.
Thus we have $x\notin J(R)$ from $a\notin J(S)$. Since $\ker f \subseteq I$, $ab= f(xy)\in f(I)$ implies that $xy\in I$. Thus $y\in \delta(I)$ and $b=f(y)\in f(\delta(I))$. On the other hand $\gamma(f(I))=f(\delta(I))$, we have $b\in \gamma(f(I))$. Thus $f(I)$ is a $\gamma$-$J$-ideal of $S$.

\end{proof}

Let $\delta$ be an expansion function of $\mathcal{I}(R)$, $I$ be an ideal of $R$. Then the function $\delta_{q} : R\slash I \rightarrow R\slash I$ defined by $\delta_{q}(J\slash I)=\delta(J)\slash I$ for all ideals $I\subseteq J$, becomes an expansion function of $\mathcal{I}(R\slash I)$.

\begin{corallary}
	Let $\delta$ be an expansion function of $\mathcal{I}(R)$ and $J\subseteq I$ be proper ideals of $R$. Then the followings hold:
	\begin{enumerate}[(1)]
		\item If $I$ is a $\delta$-$J$-ideal of $R$, then $I\slash J$ is a $\delta_{q}$-$J$-ideal of $R\slash J$.
		\item If $I\slash J$ is a $\delta_{q}$-$J$-ideal of $R\slash J$ and $J\subseteq J(R)$, then $I$ is a $\delta$-$J$-ideal of $R$.
		\item If $I\slash J$ is a $\delta_{q}$-$J$-ideal of $R\slash J$, and $J$ is a $\delta$-$J$-ideal of $R$ where $\delta(J)\neq R$, then $I$ is a $\delta$-$J$-ideal of $R$.
		\item Let $S$ be a subring of $R$ with $S\nsubseteq I$. Then $S\cap I$ is a $\delta$-$J$-ideal of $R$.

	\end{enumerate}
\end{corallary}

\begin{proof}
	(1) Take the natural homomorphism $\pi : R\rightarrow R\slash J$. Then we have that $I\slash J$ is a $\delta_{q}$-$J$-ideal of $R\slash J$ since $\ker \pi\subseteq I$.

	(2) Let $I\slash J$ be a $\delta_{q}$-$J$-ideal of $ R\slash J$ and $J\subseteq J(R)$. Assume that $ab\in I$ and $a\notin J(R)$ for some $a, b\in R$. Then $ab+J=(a+J)(b+J)\in I\slash J$ and $a+J\notin J(R\slash J)$ by assumption, $b+J\in \delta_{q}(I\slash J)= \delta(I)\slash J$, that is $b\in \delta(I)$.

	(3) Clearly.

	(4) The result follows just from taking the injection $i : S\rightarrow R$ defined by $i(a)=a$ for every $a\in S$.
\end{proof}

A proper ideal $I$ of a ring $R$ is called a superfluous ideal of $R$ if there is no proper ideal $J$ of $R$ such that $I + J= R$.

\begin{lemma}\label{lemma2.7}
	Any $\delta$-$J$-ideal of a ring $R$ with $\delta(I)\neq R$ is superfluous.
\end{lemma}
\begin{proof}
	Let $I$ be a $\delta$-$J$-ideal fo $R$ with $\delta(I)\neq R$. Assume that there exists a proper ideal $J$ of $R$ with $I+ J =   R$. Then $1= a + b$ for some $a\in I, b\in J$ and so $1-b\in I\subseteq J(R)$. Thus $b\in J$ is an unit and we get $J= R$, a contradiction.
\end{proof}

\begin{proposition}
	Let $I$ and $J$ be $\delta$-$J$-ideals of $R$ such that $\delta(I)\neq R$, $\delta(J)\neq R$. Then $I+ J $ is a $\delta$-$J$-ideal of $R$.
\end{proposition}

\begin{proof}
	Let $I,  J$ be $\delta$-$J$-ideals of $R$ such that $\delta(I)\neq R$, $\delta(J)\neq R$. Since $I, J$ are superfluous by Lemma \ref{lemma2.7}, $I + J \neq R$. Hence $I\cap J$ is a $\delta$-$J$-ideal. Also, $I\slash (I\cap J)$ is a $\delta_{q}$-$J$-ideal of $R\slash (I\cap J)$. Now, by the isomorphism $I\slash(I\cap J) \cong (I  + J)\slash J$, we have $(I  + J)\slash J$ is a $\delta_{q}$-$J$-ideal of $R\slash J$, therefore $I+ J$ is a $\delta$-$J$-ideal of $R$.
\end{proof}

Let $S$ be a multiplicatively closed subset of $R$. Denote by $S^{-1}R$ the ring of fractions of $R$ with respect to $S$.
Note that $\delta_{S}$ is an expansion function of $\mathcal{I}(S^{-1}R)$ such that  $\delta_{S}(S^{-1}I)= S^{-1}(\delta(I))$, where $\delta$ is an expansion function of $R$. Denote $Z_{I}(R)=\{r\in R | rs \in I  \text{ for some } s \in R\setminus I\}$, where $I$ is a proper ideal of $R$.

\begin{proposition}
	Let $S$ be a multiplicatively closed subset of $R$ and $\delta$ be an expansion function of $R$ and $J(S^{-1}R)= S^{-1}J(R)$. Then
	\begin{enumerate}[(1)]
		\item If $I$ is a $\delta$-$J$-ideal of $R$ with $I\cap S  = \emptyset$, then $S^{-1}I $ is a $\delta_{S}$-$J$-ideal of $S^{-1}R$.
		\item  Let $S\cap Z_{J(R)}(R)= S\cap Z_{\delta(I)}(R)= \emptyset$. If $S^{-1}I $ is a $\delta_{S}$-$J$-ideal of $S^{-1}R$, then $I$ is a $\delta$-$J$-ideal of $R$.
	\end{enumerate}
\end{proposition}

\begin{proof}
	(1) Suppose $\frac{a}{s}\cdot \frac{b}{t} \in S^{-1}I$ and $\frac{a}{s}\notin J(S^{-1}R)$ for some $a, b \in R$ and $s, t \in S$. Then there exists $u\in S$ with $abu\in I$, thus $bu\in \delta(I)$ since $a\notin J(R)$. Hence $\frac{b}{t} = \frac{bu}{tu}\in S^{-1}(\delta(I))= \delta_{S}(S^{-1}I)$. Consequently, $S^{-1}I$ is a $\delta_{S}$-$J$-ideal of $S^{-1}R$.

	(2) Let $a, b \in R$ with $ab\in I$. Then $\frac{a}{1}\cdot \frac{b}{1}\in S^{-1}I$, It implies that either $\frac{a}{1}\in J(S^{-1}R)= S^{-1}J(R)$ or $\frac{b}{1}\in \delta_{S}(S^{-1}I)=S^{-1}\delta(I)$. So if $\frac{a}{1}\in S^{-1}J(R)$, then we can deduce that $as \in J(R)$ for some $s\in S$. It follows from $S\cap Z_{J(R)}(R)= \emptyset$ that $a\in J(R)$.
 If $\frac{b}{1}\in S^{-1}\delta(I)$, by a similar disscution, we can get $b\in \delta(I)$.
\end{proof}

Let $R(+)M$ be the idealization of an $R$-module $M$. For an expansion function $\delta$ of $R$, define $\delta_{(+)}$ as $\delta_{(+)}(I (+) N)= \delta(I)(+)M$ for some ideal $I(+)N$ of $R(+)M$. It is clear that $\delta_{(+)}$ is an expansion function of $R(+)M$.

\begin{proposition}
	Let $I$ be an ideal of a ring $R$ and $N$ be a submodule of an $R$-module $M$. Then $I$ is a $\delta$-$J$-ideal of $R$ if and only if $I(+)N$ is a $\delta_{(+)}$-$J$-ideal of $R(+)M$.
\end{proposition}

\begin{proof}
	Let $I$ be a $\delta$-$J$-ideal of $R$. Assume that $(r, m), (s, m')\in I(+)N$ and $(s, m')\notin J(R)(+)M$ for some $(r, m)(s, m')\in R(+)M$. Then $s\in \delta(I)$ since $rs\in I$ and $s\notin J(R)$. Thus $(s, m')\in \delta(I)(+)M= \delta_{(+)}(I(+)N)$.

Conversely, suppose that $I(+)N$ is a $\delta_{(+)}$-$J$-ideal of $R(+)M$. Let $r, s \in R$ with $rs\in I$ and $s\notin J(R)$. Hence, we get $(r, m)(s, m')\in I(+)N$ and clearly $(s, m')\notin J(R)(+)M$. Therefore, $(r, m)\in \delta_{(+)}(I(+)N)$, and $r\in \delta(I)$, $I$ is a $\delta$-$J$-ideal.
\end{proof}


\section{Introduction}

Numerical Reasoning over Text (NRoT) is a reading comprehension task that involves producing an answer to numerical question given a short passage as context. Unlike reading comprehension tasks that can be solved by extracting the answer verbatim from the passage, NRoT usually involves using the question to determine the correct mathematical operation(s) while also identifying the correct values from the passage to use.

Research interest in NRoT has grown with the introduction of the Discrete Reasoning Over Paragraphs (DROP) dataset~\cite{Dua2019}. The majority of DROP examples are number questions involving arithmetic, which has motivated complex models that combine symbolic and neural processing modules~\cite{andor2019, ran2019, chen2020}. The best performing DROP model utilize a symbolic arithmetic module in conjunction with a neural network and other techniques such as ensembling.

We demonstrate in this work that manually engineered partitioning of the functionality between distributed and symbol modules is unnecessary for achieving good performance. Rather, the recently introduced Text-to-Text Transfer Transformer (T5)~\cite{Raffel2019} model is able to internalize NRoT without adaptation. We take full advantage of the multitasking ability of T5 to introduce a sequential training pipeline that is low resource, amiable to experimental cycle, and even achieves good performance using smaller scale models.\footnote{All code and sample models are available at github.com/lesterpjy/numeric-t5.}

\begin{figure*}[htb!]
	\centering
	\captionsetup{font=small}
	\includegraphics[width=0.95\linewidth]{schedule.png}
	\caption{Summary of the five training pipelines. The validation and RC experiments are pre-trained sequentially on NUM, TXT, and SQuAD before fine-tuning on DROP and DROP classification. All experiments begin with pre-trained T5-Small. RC experiment 2 moves the RC training from pre-training to fine-tuning by combining SQuAD, DROP classification, and DROP with multitasking. Multitask experiment is our attempt to multitask-train on all datasets prior to fine-tuning.}
	\label{fig:schedule}
\end{figure*}

\section{T5 for Numerical Reasoning over Text}
We propose five training pipelines for NRoT using T5, each consisting of two stages: pre-training on NRoT and general reading comprehension followed by fine-tuning on DROP and a classification task derived from DROP (Figure \ref{fig:schedule}). Multitask training as described in the T5 paper is used in each stage of training: different datasets are combined with temperature scaling and a special token for identification. Unless specified otherwise, we validate on all the datasets in each respective stage. The first stage begins with a pre-trained T5-Small model~\cite{Raffel2019}. Each following stage, the model begins training on the best performing model from the previous stage.
Our first two configurations (Validation Experiments 1\&2) are designed to test the performance of selecting the best models using different validation data. We experimented with validating on the DROP dev set versus validating on the dev sets of the synthetic datasets (NUM/TXT), described in detail in Section \ref{sec:datasets}.
The next two experiments (RC Experiments 1\&2) attempt to strengthen reading comprehension by multitask training using SQuAD. Finally, we attempt multitasking on all datasets simultaneously (Multitask Experiment). Multitask Experiment is trained with validation on DROP only, instead of validation on the synthetic datasets (NUM/TXT) due to concerns with the model learning parameters for the synthetic datasets closer to fine-tuning and test time. The SQuAD dataset is introduced as an extra step for learning complicated language tasks in RC1 and RC2 for this reason. Since SQuAD is included in the single first step for pre-training in the Multitask experiment, we have deliberately avoided validating its pre-training step on the synthetic datasets. Based on original T5 paper, it is reasonable to hypothesize that multitasking without stages would be the best way to achieve optimal performance.

\label{sec:pre-exp}

\newcolumntype{e}{X}
\newcolumntype{k}{>{\raggedright\arraybackslash\hsize=.4\hsize}X}
\begin{table*}[ht]
	\centering
	\small
	\captionsetup{font=small}
		\begin{tabularx}{\textwidth}{p{2cm}|e|k|p{1.0cm}}
			\hline
			Reasoning & Passage (shorten) & Question & Answer \\ \hline\hline
			Count + Sort & Denver would retake the lead. . .yet Carolina answered as kicker John Kasay ties the game with a 39-yard field goal. . . . Carolina closed out the half with Kasay nailing a 44-yard field goal. . . . In the fourth quarter, Carolina sealed the win with Kasay’s 42-yard field goal. & Which kicker kicked the most field goals? & John Kasay \\\hline

			Subtraction & That year, his Untitled (1981), a painting. . . was sold by Robert Lehrman for 16.3 million, well above its 12 million high estimate. & How many more dollars was the Untitled (1981) painting sold for than the 12 million dollar estimation? & 4300000\\\hline
		\end{tabularx}
	\caption{\label{tbl:drop_ex} Examples of QA pairs found in DROP. The question types and distribution in DROP are subtraction (28.8\%), comparison (18.2\%), selection (19.4\%), addition (11.7\%), count (16.5\%), sort (11.7\%), coreference resolution (3.7\%), other arithmetic (3.2\%), set of spans (6.0\%), other (6.8\%). Combinations of reasoning skills are also possible.}
\end{table*}

\section{Datasets}
\label{sec:datasets}
\paragraph{DROP} Discrete Reasoning Over Paragraphs (DROP), introduced by AllenNLP in 2019~\cite{Dua2019},  is a crowdsourced, adversarially-created 96k question benchmark. The benchmark consists of four types of questions, which can be answered using the context provided. Approximately 61\% of the examples in DROP are number questions that involves arithmetic. The other types are “single-span” (32\%), “spans” (6\%), and “date” (2\%). Note that all four question types in DROP can require NRoT skills, as shown in Table \ref{tbl:drop_ex}.

\paragraph{Synthetic Data} Two synthetic datasets tailored to boost performance on DROP are developed by~\cite{Geva2020}. The Numeric dataset (NUM) consists of near 1M synthetically generated questions on seven types of numerical skills. Textual dataset (TXT) builds on NUM, and includes 2M plus synthetically generated examples.

We introduce an additional synthetic task based on the DROP dataset itself, whereby the model learns to predict the DROP question-type. While not provided at test time, we expect that explicit awareness of the question types will aid the model in knowing what reasoning strategies to use.
\paragraph{SQuAD} We investigate using SQuAD v1.1~\cite{rajpurkar2016} to improve NRoT by strengthening general reading comprehension in question and answering tasks.

\paragraph{Evaluation}
DROP employs two metrics for evaluation: an adjusted F1, and Exact-Match (EM). EM uses that same criteria as SQuAD. F1 has additional logic that invalidates all matching material within an answer when there is a numeric mismatch. Overall F1 is computed using macro-averaging over individual answers. In the presence of multiple ground truths, both EM and F1 will take a max over all computed scores.

\begin{comment}
\subsection{Preliminary Experiments}
In a set of preliminary experiments, we test fundamental hypothesis relating to the ability for T5 to internalize NRoT. Other experiments explore the use of model scale, question-type hinting, digit-by-digit tokenization, and temperature-scaled mixing at further improving our baseline performance. In question-type hinting (DROP class), T5 is trained to "classify" DROP QA pairs by the four question types.~\citet{Wallace2019} have shown that numerical computation benefits from the direct use of digits. As such we have tokenized numbers digit-by-digit (e.g. 100 → [‘1’, ‘0’, ‘0’]).

\subsection{Training Schedules}
\label{sec:schedule}
Our training pipeline consists of two stages: pre-training on NRoT and RC, and fine-tuning on DROP and DROP classification. Five experiments are designed to test multiple methodologies and hypotheses (summarized by Figure \ref{fig:schedule}). For each experiment, the model is trained to multitask on the datasets in each stage, and unless specified otherwise, validated on the all the datasets in that stage as well. The first stage begins with the T5-Small parameters, and each stage after the first, the model begins training on the best performing model from the previous stage. The first two experiments (Validation Experiments 1\&2) are designed to test the performance of saving best models by different validation data. We experimented with validating on the DROP dev set versus validating on the dev sets of the synthetic datasets tailored to boost model performance on DROP. These synthetic datasets developed by~\citet{Geva2020} consist of a 1M examples numeric dataset(NUM), and a 2M examples textual dataset(TXT). The construction of these datasets are detailed in the Appendix \ref{sec:appendix}. We hypothesize a difference in performance due to how parameters are saved during training. The next two experiments (RC Experiments) attempt to strengthen reading comprehension by building on top of previous training using SQuAD v1.1~\cite{rajpurkar2016}, as the language complexity of synthetic datasets are lower. RC1 includes SQuAD v1.1 in pre-training while RC2 tests whether including SQuAD v1.1 in the fine-tuning stage lead to better performance. Finally, we test the feasibility of multitasking on all datasets (Multitask Experiment). Based on original T5 paper, it is reasonable to hypothesize that multitasking without stages would also achieve good performance.
\end{comment}

\begin{table}[hbt]
	\captionsetup{font=small}
	\small
	\scalebox{0.86}{
		\begin{tabular}{ccccc}
			\hline & \multicolumn{2}{c}{Development} &\multicolumn{2}{c}{Test} \\
			Model & EM & $\text{F}_1$ & EM & $\text{F}_1$ \\ \hline\hline
			Baseline (T5-Small) & 41.12	& 44.64 & 41.97& 45.90 \\
			Validation Experiment 1 & 65.00	& 68.53 & - & - \\
			Validation Experiment 2 & 66.04	& 69.60 & - & - \\
			RC Experiment 1 & \textbf{66.87} & \textbf{70.31} & \textbf{67.00} & \textbf{70.83} \\
			RC Experiment 2 & 66.41	& 69.80 & - & - \\
			Multitask Experiment & 63.10	& 66.47 & - & - \\ \hline\hline
			NAQANet & 46.20 & 49.24 & 44.07 & 47.01 \\
			GenBert & 68.8 & 72.3 & 68.6 & 72.4 \\
			NumNet & 64.92 & 68.31 & 64.56 & 67.97\\
			NumNet+(RoBERTa) & 81.07 & 84.42 & 81.52 & 84.84\\
			QDGAT(RoBERTa) & 84.07 & 87.05 & 84.53 & 87.57\\
			QDGAT(ALBERT) & - & - & 87.04 & 90.10 \\

			\hline
		\end{tabular}
	}
	\caption{\label{tbl:overall} Performance summary for our baseline, training experiments, and select benchmarks. NAQANet is the best-performing model proposed in DROP’s original paper. GenBERT is a modified BERT-base model fine-tuned on the same synthetic datasets. Both NumNet and QDGAT are frameworks with separate language and numerical reasoning modules. QDGAT with an ALBERT language module is the current state-of-the-art.}
\end{table}

\begin{table}[hbt]
	\captionsetup{font=small}
	\small
	\scalebox{1}{
		\begin{tabular}{ccccc}
			\hline
			Model & Initialization &  \#Params\\ \hline
            NT5 & T5-Small & 60M\\
			GenBert & Bert-Base & 110M \\
			NumNet+(RoBERTa) & RoBERTa-Large & 355M\\
			QDGAT(RoBERTa) & RoBERTa-Large & 355M\\
			\hline
		\end{tabular}
	}
	\caption{\label{tbl:modelsize}Number of parameters used for initialization for respective models.}
\end{table}

\newcolumntype{s}{>{\hsize=.2\hsize}X}
\newcolumntype{g}{>{\hsize=1.6\hsize}X}
\newcolumntype{u}{>{\hsize=.8\hsize}X}
\begin{table*}[hbt]
	\scriptsize
	\centering
	\captionsetup{font=small}
		\begin{tabularx}{\textwidth}{ugssssssssss}
			\toprule & & \multicolumn{2}{X}{Number} & \multicolumn{2}{X}{Date}  & \multicolumn{2}{X}{Span} & \multicolumn{2}{X}{Spans} & \multicolumn{2}{X}{\textbf{Overall}}\\
			Model & Schedule & EM & F1 & EM & F1 & EM &F1 & EM &F1 & EM & F1\\ \hline\hline\addlinespace[0.1cm]
			baseline & DROP & 31.79 & 31.83 & 43.95 & 53.28 & 62.09 & 67.42& 26.98 & 55.44 & 41.12 & 44.64 \\
			\addlinespace[0.1cm]\hline\addlinespace[0.1cm]

			\multirow{4}{\hsize}{Validation Experiment 1} & DROP + NUM (validate on DROP) & 36.97 & 36.99 & 43.95 & 51.63 & 59.45 & 64.67 & 27.69 & 55.79 & 43.52 & 46.95\\
			\addlinespace[0.1cm]
			& DROP + TXT (validate on DROP) & 63.25 & 63.27 & 42.04 & 51.42 & 63.03 & 68.29 & 29.63 & 56.97 & 60.83 & 64.26\\
			\addlinespace[0.1cm]
			& DROP + DROP class & 67.03 & 67.07 & 42.68 & 51.43 & 65.19 & 70.80 & 31.57 & 58.89 & 63.95 & 67.49\\
			\addlinespace[0.1cm]
			& DROP & 68.72 & 68.78 & 43.31 & 50.36 & 65.09 & 70.62 & 32.10 & 60.05 & 65.00 & 68.53\\
			\addlinespace[0.1cm]\hline\addlinespace[0.1cm]

			\multirow{4}{\hsize}{Validation Experiment 2} & DROP + NUM (validate on NUM) & 41.37 & 41.38 & 40.76 & 48.94 & 61.31 & 66.45 & 29.81 & 58.73 & 46.86 & 50.32\\
			\addlinespace[0.1cm]
			& DROP + TXT (validate on TXT) & 63.16 & 63.18 & 44.59 & 52.76 & 63.23 & 68.56 & 29.81 & 58.85 & 60.90 & 64.42\\
			\addlinespace[0.1cm]
			& DROP + DROP class & 67.73 & 67.75 & 45.86 & 53.80 & 65.16 & 70.61 & 33.69 & 61.18 & 64.54 & 68.02\\
			\addlinespace[0.1cm]
			& DROP & 69.78 & 69.83 & 42.68 & 51.23 & 66.00 & 71.47 & 34.22 & 62.53 & 66.04 & 69.60\\
			\addlinespace[0.1cm]\hline\addlinespace[0.1cm]

			\multirow{3}{\hsize}{RC Experiment 1}& DROP + SQuAD$^\star$ & 65.61 & 65.68 & 45.22 & 55.38 & 65.94 & 71.23 & 34.39 & 62.75 & 63.52 & 67.06\\
			\addlinespace[0.1cm]
			& DROP + DROP class & 68.65 & 68.69 & 45.22 & 53.87 & 66.54 & 72.01 & 36.68 & 63.47 & 65.71 & 69.17\\
			\addlinespace[0.1cm]
			& DROP &  \textbf{70.34} & \textbf{70.39} & 45.22 & 53.85 & 66.75 & 72.35 &  \textbf{37.74} & 63.43 & \textbf{66.87} & \textbf{70.31}\\
			\addlinespace[0.1cm]\hline\addlinespace[0.1cm]

			\multirow{2}{\hsize}{RC Experiment 2}& DROP + DROP class + SQuAD$^{\star\diamond}$ & 65.90 & 65.94 & 45.22 & 54.31 & 66.48 & 71.73 & 35.80 & 62.55 & 63.95 & 67.35\\
			\addlinespace[0.1cm]
			& DROP & 69.44 & 69.47 & 45.22 & 53.17 &  \textbf{67.45} & \textbf{72.60} & 35.63 & 63.08 & 66.41 & 69.80\\
			\addlinespace[0.1cm]\hline\addlinespace[0.1cm]

			\multirow{3}{\hsize}{Multitask Experiment} & DROP + TXT + NUM + SQuAD & 56.84 & 56.86 & 42.68 & 50.44 & 64.58 & 69.72 & 33.51 & 61.78 & 57.62 & 61.04\\
			\addlinespace[0.1cm]
			& DROP + DROP class & 63.73 & 63.81 &  \textbf{49.04} & \textbf{56.28} & 65.97 & 71.24 & 36.16 & 63.65 & 62.54 & 65.99\\
			\addlinespace[0.1cm]
			& DROP & 64.43 & 64.49 & 45.86 & 52.99 & 66.48 & 71.61 & 36.51 & \textbf{63.80} & 63.10 & 66.47\\
			\addlinespace[0.1cm]

			\bottomrule
		\end{tabularx}
	\caption{\label{tbl:phase-type} The decomposed and overall EM and $\text{F}_1$ scores on different answer types in the development set of DROP for each experiment. High scores for each type are in bold. $\star$Notice that the RC experiments begin training using the weights learned in validation experiment 2. $\diamond$RC Experiment 2 fine-tune with SQuAD in addition to DROP and DROP classification.}
\end{table*}

\section{Results}

The overall results of our five training experiments are summarized in Table \ref{tbl:overall}, and decomposed in Table \ref{tbl:phase-type}.
Our best model achieves 66.8 EM and 70.3 F1 on the dev set, and a 67.0 EM and 70.8 F1 on test. Although the EM and F1 performance appears to have a degree of variance across the experiments. It is clear based on the overall model performance that RC1 and RC2 experiments are the most successful in internalizing the numerical reasoning required for performing well on DROP. While underperforming QDGAT-ALBERT, the current state-of-the-art that makes use of both neural and symbolic modules, NT5 performs well for a purely neural based method. Notably, our models use T5-Small with significantly fewer parameters than GenBERT. The encoder-decoder T5-small model has 60 million parameters, compared to the 110 million parameters of GenBERT in its encoder alone.

Overall, Table \ref{tbl:phase-type} shows that pre-training with DROP, synthetic datasets and SQuAD, and fine-tuning on DROP and DROP classification sequentially is able to significantly boost the performance on number questions, an increase of F1 from 31.83 to 70.39, while maintaining or improving performance on other types of questions. Additionally, when testing out the baseline, we found T5-Base increase F1 score over T5-Small by 11 points.

\subsection{Difference in validation dataset}
A surprising finding here is that saving models while validating on the synthetic dev sets outperforms saving models while validating on the DROP dev sets after the first stage. Specifically, this achieves a F1 score (50.32) that is 3.37 points higher (46.95) without sacrificing performance on span/spans questions. We reason that this performance gap is caused by the difference between the loss on development and DROP’s evaluation metrics, as detailed in Section \ref{sec:datasets}.

\subsection{Strengthening Reading Comprehension}

 Performance on extractive RC tasks is boosted with the addition of SQuAD v1.1 in pre-training. We further test if this performance change persist when multitask training SQuAD v1.1 together with DROP and DROP classification in the fine-tuning stage. The resulting model sees improvement across all question types at the end of the training on SQuAD v1.1. Crucially, performance on RC tasks (date, span, and spans) sees an average improvement of 3.06 points in F1 over the previous result. However, this came at the expense of minor deteriorated performance on numeric questions.

\subsection{All datasets multitasking}
Fine-tuning simultaneously on all datasets underperforms our best model by nearly 6-point on F1.

\section{Error Analysis}
\label{sec:error}
To better understand the achievement and limitations of the best model, we analyzed its errors on the dev set. In 38 of 100 errors sampled from number questions, the model has made at least one partial digit match. Of the total 86 errors on date questions, 39 questions require arithmetic calculations. In 9 of these 86 errors, the model wrongly performs numerical calculations, instead of simply extracting answers. With a sample of 100 span and spans errors, 49 of the questions contain reasoning skills not covered in the pre-training datasets. This is compared to the 43\% shown by~\citet{Dua2019}\footnote{Criteria might vary due to human evaluation}. Many of these errors can be addressed with pre-training datasets that cover more complicated calculations and reasoning skills.

\section{Related Works}
\label{sec:related}

Introduced by~\cite{Geva2020}, GenBERT is a BERT-Base model customized with specialized heads for handling discrete reasoning. It is also the main inspiration for our approach. The current state-of-the-art model on DROP, QDGAT-ALBERT, uses a directed graph attention network between a ALBERT based representation extractor and a prediction module for discrete reasoning~\cite{chen2020}. For works analyzing the mathematical reasoning ability of models over text refer to~\citet{Wallace2019, Ravichander2019}.

\section{Conclusion}
\label{sec:conclusion}

We introduced a sequential pre-training framework for numeracy with T5. Our method demonstrates strong improvements on NRoT over a baseline vanilla T5 model. Although current state of the art, QDGAT, which makes use of a hybrid of a neural and symbol modules, and human performance on DROP are better performing, our approach touts both simplicity and low resource usage, achieving good performance using only T5-small.

\bibliographystyle{acl_natbib}

\section{Appendix}
\label{sec:appendix}

In this appendix we described some of the technical details of our implementation of the NT5 framework, including parameters tuning, and the brief overview of synthetic datasets used. We used Colab Pro TPU as our computing infrastructure, and the average run time for a pipeline including all the steps is roughly 50 hours.

\subsection{Hyperparameters}
Five input prefixes (context, answer\_me, calculate, and classify\_me) are used for labeling the datasets during training. An example input from DROP is: “answer\_me: How many more men did Ming Rui have than cavalry? context: In March 1768, Ming Rui began his retreat, pursued by a Burmese army of 10,000 men and 2000 cavalry…” Note that we place the question before context in order to avoid the cutoff when the input exceeds our encoder’s max length. Our preset encoder and decoder max lengths (512 and 54, respectively) cover 100\% of context and gold answers in all datasets except for DROP (of which 3.89\% of contexts are cut off in training). Mixing temperatures are set to 1 and training steps are calculated accordingly such that one epoch covers all examples in each of the datasets. Multitask Experiment is the only exception because combining all the datasets in one causes training to extend in time beyond our resource limit. The temperature here is set to 10 and the training steps are calculated such that one epoch is roughly a full run of DROP training set.

\subsection{Learning Rate}
We experiment multiple learning rate schedules on the development set, including constant rates (1e-3, 1e-4, 1e-6) with Adam as the optimizer, and various warm-up/decay rates ([1e-6, 1e-10] and [1e-3, 1e-7] respectively). In the end, we settle on the following parameters: warm-up starting rate (1e-8), warm-up ending rate (1e-4), decay rate after warm-up (1e-3). We allocate 10\% of the total epochs to warm-up, during which the rate increases linearly by batch. Once the warm-up ends, the learning rate decays by epoch according to the following schedule:
\begin{equation}
\text{LR} = \frac{\text{LR}}{(1 + \text{decay rate}(\text{epoch} - \text{warmup epoch}))}
\end{equation}

\subsection{Synthetic Datasets}
\label{appen:syn}
Here we briefly discuss the two synthetic datasets by \citet{Geva2020}. NUM is a dataset consisting of 1M mathematical expressions synthetically generated using six different types of templates that correspond to various numeric operations. An example of the “combination” templates is “s1 f1 s2 f2 s3 f3”, which can translate to a synthetic example of “517.4 - 17484 - 10071.75 + 1013.21.” The symbol “s1” defines the sign for the first digit in the mathematical expression and “f1” defines the first digit to be a float type. Another example is the “max/min/avg” template, “o(f1, f2, f3),” where “o” symbolizes the operation of min/max/avg. The symbols are converted to their associated numbers or operations randomly together with correct answers during data generation. TXT consists of 2M Q\&A examples synthetically generated by building on top of the mathematical skills found in NUM. In addition, the data generation is based on multiple sentence, question, and context templates that are created by following the “world state” framework created by \citet{Hosseini2014} to build math word problems. The extracted templates are sentences with “world states” which consist of countable \textit{entities}, \textit{entity containers} (owners of the entities), and \textit{verbs} (which describe the changes of entities and entity containers) that can be filled. Synthetic context-question pairs with answers can then be randomly generated by filling in the extracted templates with domain-specific vocabularies that correspond to the world states.

\subsection{Temperature-Scaled Mixing}
\label{appen:multitask}
We took advantage of T5 multitasking framework to prevent T5 from unlearning language understanding or numerical reasoning throughout the training schedules. We multitask-train our model by mixing different datasets with temperature-scaled mixing, a strategy utilized in the original work of T5 \citep{Raffel2019} and \textit{How multilingual is Multilingual BERT?}\citep{Pires2019}. This scaling technique is designed to mitigate the issue of a model overfitting to the larger dataset during multitasking due to the magnitude of disparity between datasets. The formula for calculating the temperature scaling rate is as following:

\begin{equation}
r_m = min(length_m \times scale)^{\frac{1}{T}}
\end{equation}

where $T$ is the temperature used across datasets for adjusting mixing proportions, $length_m$ is the size of $dataset_m$, $scale$ modifies the scales of datasets, and $r_m$ is the unnormalized ratio for $dataset_m$. This calculated rate, $r_m$, is then normalized for the sampling ratio of each dataset. When $T = 1.0$, this approach is equivalent to examples-proportional mixing, and no adjustment is done to mitigate the disparity between dataset sizes. The proportions of datasets become closer to equal-mixing as $T$ increases. The greater the $T$ becomes, the more adjustment is done to close the disparity of sizes between datasets, effectively lowering the ratios of larger datasets in the mix.

We decide to set temperature to 1 across all training schedules (except Multiutask Experiment) because we have not found task interference or negative transfer, as described by \citet{Raffel2019}, to be an issue. This is likely due to the fact that the the large datasets (NUM/TXT) are generated by closely following the specific formats and NoRT skills found in DROP.

\bibliographystyle{acl_natbib}


\section{Introduction}

Eye-tracking data provides precise records of where humans look during reading, with millisecond-level accuracy. This type of data has recently been leveraged for uses in natural language processing: it can improve performance on a variety of downstream tasks, such as part-of-speech tagging \citep{eyetrack-pos}, dependency parsing \citep{eyetrack-depparse}, and for cognitively-inspired evaluation methods for word embeddings \citep{eyetrack-vec-eval}. Meanwhile, Transformer-based language models such as BERT \citep{bert} and RoBERTa \citep{roberta} have been applied to achieve state-of-the-art performance on many natural language tasks. The CMCL 2021 shared task aims to add to our understanding of how language models can relate to eye movement features.

In this paper, we present our submission to this shared task, which achieves third place on the leaderboard. We first explore some simple baselines using token-level features, and find that these are already somewhat competitive with the final model's performance. Next, we describe our model architecture, which is based on RoBERTa (Figure \ref{fig:roberta-linear}). We find that model ensembling offers a substantial performance gain over a single model. Finally, we augment the provided training data with the publicly available Provo eye-tracking corpus and combine them using a two-stage fine-tuning procedure; this results in a moderate performance gain. Our source code is available at \url{https://github.com/SPOClab-ca/cmcl-shared-task}.

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figs/roberta-linear.png}
    \caption{Our model consists of RoBERTa with a regression head on each token, which is a linear layer that predicts the 5 output features from the last layer's embeddings. The model is initialized from pretrained weights and fine-tuned on the task data.}
    \label{fig:roberta-linear}
\end{figure}

\section{Task description}

The shared task format is described in \citet{sharedtask}, which we will briefly summarize here. The task data consists of sentences derived from the ZuCo 1.0 and ZuCo 2.0 datasets; 800 sentences (15.7 tokens) were provided as training data and 191 sentences (3.5k tokens) were held out for evaluation. The objective is to predict five eye-tracking features for each token:

\begin{table*}[t]
\centering
\begin{tabular}{lrrrrrr}
\hline
\textbf{Model} & \textbf{nFix} & \textbf{FFD} & \textbf{GPT} & \textbf{TRT} & \textbf{fixProp} & \textbf{All (Dev)} \\ \hline
Median & 7.208 & 1.162 & 3.547 & 2.732 & 21.179 & 7.165 \\
Linear regression     & 4.590 & 0.795 & 2.995 & 1.812 & 13.552 & 4.749 \\
SVR (RBF kernel)      & 4.440 & 0.723 & 2.728 & 1.728 & 12.077 & 4.339 \\ \hline
\end{tabular}
\caption{Baseline results: `Median' is a model that always predicts the median of the training data; the linear regression and SVR models use 4 token-level surface features described in Section \ref{sec:baselines}.}
\label{tab:baselines}
\end{table*}

\begin{table*}[t]
\centering
\begin{tabular}{lrrrrrr}
\hline
\textbf{Model} & \textbf{nFix}  & \textbf{FFD} & \textbf{GPT}   & \textbf{TRT}   & \textbf{fixProp} & \textbf{All (Dev)} \\ \hline
BERT-base     & 4.289 & 0.704          & 2.645 & 1.678 & 11.155 & 4.094 \\
BERT-large    & 4.150 & 0.682          & 2.493 & 1.616 & 11.013 & 3.991 \\
RoBERTa-base   & \textbf{4.066} & \textbf{0.681}        & \textbf{2.443} & \textbf{1.570} & \textbf{10.981}  & \textbf{3.930}   \\
RoBERTa-large & 4.156 & 0.681 & 2.468 & 1.623 & 11.047 & 3.995 \\ \hline
\end{tabular}
\caption{MAE using BERT and RoBERTa models with fine-tuning.}
\label{tab:transformers}
\end{table*}

\begin{table*}[t]
\centering
\begin{tabular}{lrrrrrr}
\hline
\textbf{Model} & \textbf{nFix}  & \textbf{FFD}   & \textbf{GPT}   & \textbf{TRT} & \textbf{fixProp} & \textbf{All (Dev)} \\ \hline
Single Model  & 4.066 & 0.681 & 2.443 & 1.570          & 10.891 & 3.930          \\
Ensemble of 2 & 3.978 & 0.671 & 2.350 & 1.534          & 10.714 & 3.849          \\
Ensemble of 5 & 3.944 & 0.669 & 2.321 & \textbf{1.521} & 10.665 & 3.824 \\
Ensemble of 10 & \textbf{3.943} & \textbf{0.666} & \textbf{2.316} & 1.522        & \textbf{10.660}  & \textbf{3.821}   \\ \hline
\end{tabular}
\caption{Ensembles of RoBERTa-base model, obtained by taking a simple mean of the predictions of individual models. This improves our overall performance by about 0.09 MAE compared to a single model, but with diminishing returns past 5 models.}
\label{tab:ensembles}
\end{table*}

\begin{itemize}
    \setlength\itemsep{-0.25em}
    \item Number of fixations on the current word ({\em nFix}).
    \item First fixation duration of the word ({\em FFD}).
    \item Go-past time: the time from the first fixation of a word until the first fixation beyond it ({\em GPT}).
    \item Total reading time of all fixations of the word, including regressions ({\em TRT}).
    \item Proportion of participants that fixated on the word ({\em fixProp}).
\end{itemize}

The features are averages across multiple participants, and each feature is scaled to be in the range [0, 100]. The evaluation metric is the mean absolute error (MAE) between the predicted and ground truth values, with all features weighted equally.

Since each team is allowed only a small number of submissions, we define our own train and test split to compare our models' performance during development. We use the first 600 sentences as training data and the last 200 sentences for evaluation during development. Except for the submission results (Table \ref{tab:adaptation}), all experimental results reported in this paper are on this development train/test split.

\section{Our approach}

\subsection{Baselines}
\label{sec:baselines}

We start by implementing some simple baselines using token-level surface features. Previous research in eye tracking found that longer words and low-frequency words have higher probabilities of being fixated upon \citep{rayner}. We extract the following features for each token:

\begin{itemize}
    \setlength\itemsep{-0.25em}
    \item Length of token in characters.
    \item Log of the frequency of token in English text, retrieved using the \texttt{wordfreq}\footnote{\url{https://github.com/LuminosoInsight/wordfreq/}} library.
    \item Boolean of whether token contains any uppercase characters.
    \item Boolean of whether token contains any punctuation.
\end{itemize}

Using these features, we train linear regression and support vector regression models separately for each of the 5 output features (Table \ref{tab:baselines}). Despite the simplicity of these features, which do not use any contextual information, they already perform much better than the median baseline. This indicates that much of the variance in all 5 eye-tracking features are explained by surface-level cues.

\begin{table*}[t]
\centering
\begin{tabular}{lrrrrrrr}
\hline
\textbf{Training Data} &
  \textbf{nFix} &
  \textbf{FFD} &
  \textbf{GPT} &
  \textbf{TRT} &
  \textbf{fixProp} &
  \textbf{All (Dev)} &
  \multicolumn{1}{l}{\textbf{Submission}} \\ \hline
Task Only (Single)         & 4.066 & 0.681 & 2.443 & 1.570 & 10.891 & 3.930 & n/a   \\
Provo + Task (Single) & 3.984 & 0.713 & 2.424 & 1.556 & 10.781 & 3.892 & n/a   \\
Task Only (Ensemble)       & 3.943 & 0.666 & 2.316 & 1.522 & 10.660 & 3.821 & 3.974 \\
Provo + Task (ensemble) &
  \textbf{3.888} &
  \textbf{0.664} &
  \textbf{2.306} &
  \textbf{1.499} &
  \textbf{10.586} &
  \textbf{3.789} &
  \textbf{3.929} \\ \hline
\end{tabular}
\caption{Comparison of model trained using the provided versus two-stage fine-tuning using Provo data. The additional pretraining improved overall performance by about 0.04 MAE. Our best submission is an ensemble of 10 RoBERTa-base models with two-stage fine-tuning.}
\label{tab:adaptation}
\end{table*}

\subsection{Fine-tuning transformers}

\begin{figure*}
    \centering
    \includegraphics[width=\linewidth]{figs/corr_both.png}
    \caption{Distributions and pairwise scatterplots of the task data (left) and Provo data processed to match the mean and standard deviation of the task data (right).}
    \label{fig:provo-plots}
\end{figure*}

Our main model uses RoBERTa \citep{roberta} with a linear feedforward layer to predict the 5 output features simultaneously from the last hidden layers of each token. In cases where the original token is split into multiple RoBERTa tokens, we use the first RoBERTa token to make the prediction. The model is initialized with pretrained weights and fine-tuned on the task data to minimize the sum of mean squared errors across all 5 features.

As the task data is relatively small, we found that the model needs to be fine-tuned for 100-150 epochs to reach optimal performance, far greater than the recommended 2-4 epochs \citep{bert}. We trained the model using the AdamW optimizer \citep{adamw} with learning rates of \{1e-5, 2e-5, 5e-5, 1e-4\} and batch sizes of \{8, 16, 32\}; all other hyperparameters were left at their default settings using the HuggingFace library \citep{huggingface}.

In addition to RoBERTa, we experiment with BERT \citep{bert}; we try both the base and large versions of BERT and RoBERTa, using a similar range of hyperparameters for each (Table \ref{tab:transformers}). RoBERTa-base performed the best in our validation experiments; surprisingly, RoBERTa-large had worse performance.

\subsection{Model ensembling}

We use a simple approach to ensembling: we train multiple versions of an identical model using different random seeds and make predictions on the test data. These predictions are the averaged to obtain the final submission. In our experiments (Table \ref{tab:ensembles}), ensembling greatly improves our performance, but with diminishing returns: the MAE of the 10-model ensemble is only marginally better than the 5-model ensemble. We use ensembles of 10 models in our final submission.

\subsection{Domain adaptation from Provo}

In addition to the task data provided, we also use data from the Provo corpus \citep{provo}. This corpus contains eye-tracking data from 84 participants reading 2.6k words from a variety of text sources. The corpus also provides predictability norms and extracted syntactic and semantic features for each word, which we do not use.

We process the Provo data to be similar to the task data so that they can be combined. First, we identify the Provo features that are most similar to each of the output features: we map {\em IA\_FIXATION\_COUNT} to {\em nFix}, {\em IA\_FIRST\_FIXATION\_DURATION} to {\em FFD}, {\em IA\_REGRESSION\_PATH\_DURATION} to {\em GPT}, and {\em IA\_DWELL\_TIME} to {\em TRT}, taking the mean across all participants for each feature. For the {\em fixProp} feature, we calculate the proportion of participants where $IA\_DWELL\_TIME > 0$ for each word. Finally, we scale all five features to have the same mean and standard deviation as the task data, and verify that their distributions and pairwise scatterplots are similar (Figure \ref{fig:provo-plots}).

We use two-stage fine-tuning to combine the Provo data with the task data. In two-stage fine-tuning, the entire model is fine-tuned on an auxiliary task before fine-tuning on the target task -- this often yields a performance improvement, especially when the target task has a small amount of data \citep{intermediate-fine-tuning}. In our case, we fine-tune the RoBERTa-base model for 100 epochs on the Provo data, then fine-tune for another 150 epochs on the task data. This gave a considerable improvement on both the development and submission scores (Table \ref{tab:adaptation}). Our best final submission is an ensemble of 10 identical models trained this way with different random seeds.

\section{Conclusion and future work}

We propose a simple approach to predict eye-tracking features using the RoBERTa model customized with a per-token regression head. Our initial model uses the standard fine-tuning procedure; experiments show that the performance is further improved by model ensembling and domain adaptation by two-stage fine-tuning on an intermediate eye-tracking task. Our best model achieves third place on the leaderboard.

In future work, several avenues may be explored to further improve performance. First, we did not combine our feature engineering baseline with the RoBERTa model -- engineered features (such as frequency statistics or neurolinguistic norms) would provide the model with information not contained in RoBERTa. Second, we only experimented with a small subset of features from the Provo corpus for domain adaptation, whereas it is not actually necessary for the auxiliary fine-tuning task to match the target task. Thus, it may be possible to achieve better performance by fine-tuning on a different set of Provo features, or a different dataset entirely.

\section*{Acknowledgements}

We thank Dr Jeanne Sinclair for her helpful discussions during this project. FR is supported by a CIFAR Chair in Artificial Intelligence.