\section{Introduction}

The unzipping of a double stranded DNA (dsDNA) by a mechanical force,
which is an initial step in biological processes like DNA replication
and RNA transcription~\cite{watson_molecular_2003}, has been studied
over two decades both
theoretically~\cite{bhattacharjee_unzipping_2000,lubensky_pulling_2000,sebastian_pulling_2000,marenduzzo_phase_2001,marenduzzo_dynamical_2002,kapri_complete_2004,kumar_biomolecules_2010}
and experimentally using single molecule manipulation
techniques~\cite{bockelmann_unzipping_2002,danilowicz_dna_2003,danilowicz_measurement_2004,ritort_single-molecule_2006,hatch_measurements_2007}.
The two strands of a dsDNA, whose ends are exerted by a pulling force,
is found to unzip to two single strands if the force exceeds a critical
value. The unzipping transition, which is now well established, is a
first order phase transition. If the externally applied force is
oscillatory in nature, it unzips and rezips the two strands of the DNA
in each cycle, and the force-extension isotherm shows a hysteresis.
There have been many studies of hysteresis in unbinding and rebinding of
biomolecules in recent years because it reveals important information
about the kinetics of conformational transformations, the potential
energy landscape, and controlling the folding pathway of a single
molecule and in force sensor
studies~\cite{hatch_measurements_2007,friddle_near-equilibrium_2008,tshiprut_exploring_2009,li_real-time_2007,yasunaga_quantifying_2019}.

In recent years, the behavior of a dsDNA under a periodic force with
frequency $\omega$ and amplitude $G$ has been studied by using Brownian
dynamics (BD) or Langevin dynamics (LD) simulation of an off-lattice
coarse-grained model for short chains which are limited to a maximum
number of $N=16$ base pairs and 32
monomers~\cite{kumar_statistical_2013,mishra_dynamical_2013,mishra_scaling_2013,kumar_periodically_2016,pal_dna_2018},
and by using Monte Carlo (MC) simulations of DNA chains having 1024
monomers with $N=512$ base pairs on a ($d=1+1$)-dimensional square
lattice~\cite{kapri_hysteresis_2012,
kapri_unzipping_2014,kalyan_unzipping_2019,yadav_unzipping_2021}. Both
LD and MC simulation studies show the existence of a dynamical phase
transition, where the DNA can be taken from the zipped state to an
unzipped state via a new dynamical state. The area of the hysteresis
loop, $A_{loop}$, which represents the energy dissipated in the system,
is found to scale as $1/\omega$ at higher frequencies, whereas at low
frequencies, $A_{\mathrm{loop}}$ scales as $G^{\alpha} \omega^{\beta}$,
or $(G-G_c)^{\alpha} \omega^{\beta}$, where $G_c$ is the critical force
needed to unzip the DNA for the static force case. The values of the
exponents $\alpha$ and $\beta$ obtained in BD/LD and MC simulations are
however different. In BD/LD simulation studies on shorter DNA
hairpins~\cite{kumar_statistical_2013,mishra_dynamical_2013,mishra_scaling_2013,kumar_periodically_2016,pal_dna_2018},
a chain having $2N$ monomers, whose first $N$ monomers are complementary
to the rest half. The monomers of the chain are chosen in such a manner
that the $i$th monomer from the anchored end can bind only with the
$(N-i)$th monomer of the chain, thus mimicking a base pair of the DNA.
One end of the chain is anchored at the origin and an external time
dependent periodic force $g(t)$ is applied on the free end along $x$
direction and its distance from the origin, $x(t)$, is monitored. In MC
simulation
studies~\cite{kapri_hysteresis_2012,kapri_unzipping_2014,kalyan_unzipping_2019,yadav_unzipping_2021},
the strands of DNA are represented by two directed self-avoiding walks
(DSAWs), which do not cross each other, on a ($d=1+1$)-dimensional
square lattice. Whenever the $i$th monomers of  walks (mimicking
complementary bases) are unit distance apart, there is a gain in energy
(base-pairing). Two strands of the DNA at one end are always kept fixed
at origins $O$ and $O^{\prime}$ and the other end monomers are subjected
to a time-dependent periodic force $g(t)$ and the separation, $x(t)$,
between them is monitored. In both BD/LD and MC simulation studies, the
average force-distance isotherms obtained from the time series show
hysteresis loop whose area is studied as a function of $G$ and $\omega$.
Initial BD/LD simulation studies reported exponent values $\alpha=
\beta=1/2$~\cite{kumar_statistical_2013,mishra_dynamical_2013,mishra_scaling_2013}.
These were later modified to $\alpha=0.33$ and $\beta =
1/2$~\cite{kumar_periodically_2016}. However, a different set of
exponents, $\alpha=1$ and $\beta=1.25$, were obtained for longer
homopolymer DNA chains in MC simulation
studies~\cite{kapri_unzipping_2014}. Very recently, the DSAW model has
been extended to study the unzipping of a block copolymer DNA subjected
to a periodic force, and the same set of exponents ($\alpha=1$ and
$\beta=1.25$) were obtained~\cite{yadav_unzipping_2021}. This inspired
us to perform LD simulation studies on a longer DNA chains and
investigate the true values of the loop area exponents at low frequency
regime. Our hypothesis is that the different set of exponents obtained
in earlier BD/LD studies, as compared to MC studies, are due to the
presence of strong finite size effects because of shorter chain lengths
used in those studies.

In this paper, we study the unzipping transition of a dsDNA subjected to
a periodic force using LD simulations on a model that is very much
similar to a well established DSAW model of a DNA on a $D=1+1$ square
lattice. The later model has been studied extensively, for over two
decades, using the generating function, exact transfer matrix, and Monte
Carlo
techniques~\cite{marenduzzo_phase_2001,marenduzzo_dynamical_2002,kapri_complete_2004,kapri_unzipping_2006,kapri_randomly_2007,kapri_manipulating_2008,kapri_hysteresis_2012,kapri_unzipping_2014,kapri_unzipping_2014,kalyan_unzipping_2019,yadav_unzipping_2021}.
Furthermore, the length of the DNA, having up to 184 monomers with
$N=96$ base pairs, simulated here, for the periodic case, are six times
longer than the earlier BD/LD simulation
studies~\cite{kumar_statistical_2013,mishra_dynamical_2013,mishra_scaling_2013,kumar_periodically_2016,pal_dna_2018}.
We first consider the static force case and confirm that the model
considered in this paper indeed show first order phase transition same
as DSAW model. We also obtain the value of the critical force, $G_c$,
needed to unzip the dsDNA. Next, we consider the periodic force case,
where the force-distance isotherms show hysteresis loop whose area,
$A_{\mathrm{loop}}$, behaves non-monotonically with the frequency. We
observe that the loop area scales as $A_{\mathrm{loop}} \sim 1/\omega$
in the higher frequency regime, whereas it scales as $A_{\mathrm{loop}}
\sim (G-G_c)^{\alpha}\omega^{\beta}$ with exponent values $\alpha=1$ and
$\beta=1.25$ in the lower frequency regime. These exponents are same as
that obtained in earlier MC simulation studies on longer chain
lengths~\cite{kapri_unzipping_2014,yadav_unzipping_2021}.

The paper is organized as follows: In Sec.~\ref{sec:model}, we define
the model simulated in this paper. The results are discussed in
Sec.~\ref{sec:results} and summarized in Sec.~\ref{sec:summary}.

\section{Model} \label{sec:model}

\figOne

We model the strands of a dsDNA by beads and springs in two dimensions
(see Fig.~\ref{fig:1}). The beads of the polymer experience an excluded
volume interaction modeled by the Weeks-Chandler-Andersen (WCA)
potential of the form
\begin{equation}\label{eq:LJmm}
    U_{\mathrm{bead}}(r) = \begin{cases}
        4 \varepsilon \left[ \left( \frac{\sigma}{r} \right)^{12} -
		\left( \frac{ \sigma}{r} \right)^{6} \right] + \varepsilon
		& \text{for} \ r \le r_{\mathrm{min}} \cr
        0  & \text{for} \ r > r_{\mathrm{min}}
    \end{cases}
\end{equation}
where, $\varepsilon$ is the strength of the potential.  The cutoff
distance, $r_{\mathrm{min}} = 2^{1/6} \sigma$, is set at the minimum of
the potential. The consecutive monomers of strands are connected by the
finitely extensible nonlinear elastic (FENE)
potential~\cite{grest_molecular_1986} of the form
\begin{equation}\label{eq:FENE}
    U_{\mathrm{FENE}} (r) = - \frac{1}{2} k R_{0}^{2} \ln \left( 1 -
	\frac{r^2}{R_0^2}\right),
\end{equation}
where $k$ and $R_0$ are the spring constant and the maximum allowed
distance between the consecutive monomers, respectively. The
complementary monomers of the DNA (i.e., $i$th monomers of both the
strands) interacts with each other via standard LJ potential:
\begin{equation}\label{eq:LJbp}
    U_{\mathrm{bp}}(r) = \begin{cases}
	    4 \varepsilon_{\mathrm{p}} \left[ \left( \frac{\sigma}{r}
		\right)^{12} - \left( \frac{ \sigma}{r} \right)^{6} \right] &
		\text{for} \ r \le r_{\mathrm{c}} \cr
	    0  & \text{for} \ r > r_{\mathrm{c}},
    \end{cases}
\end{equation}
where $\varepsilon_{\mathrm{p}}$ denotes the base pair interaction
strength and $r_{\mathrm{c}} = 2.5 \sigma$ is the cutoff distance.

The strands at one end of the DNA are anchored at $O$ and $O^{\prime}$
and the strands at the free end are subjected to a time-dependent
periodic force
\begin{equation}
    \mathbf{g}(t) = G | \sin (\omega t) |,
\end{equation}
where $G$ is the amplitude and $\omega$ is the angular frequency of the
oscillating force.

To integrate the equation of motion for the monomers of the chain we use
LD algorithm with velocity-Verlet update. The equation of motion for a
monomer is given by
\begin{equation}
    m \ddot{\boldsymbol r}_i = - {\boldsymbol \nabla} U_{i} +
	{\mathbf{g}} - \zeta {\boldsymbol v}_i + {\boldsymbol \eta}_i,
\end{equation}
where $m$ is the monomer mass, $ U_i = U_{\mathrm{bead}} +
U_{\mathrm{FENE}} +  U_{\mathrm{bp}}$ is the total potential experienced
by $i$th monomer, $\zeta$ is the friction coefficient, ${\boldsymbol
v}_i$ is the monomer's velocity, and ${\boldsymbol \eta}_i$ is the
random force satisfying the fluctuation-dissipation theorem $\langle {
	\eta}_i(t){\eta}_j(t^{\prime}) \rangle = 2 \zeta k_{B} T \delta_{ij}
\delta(t - t^{\prime})$. The unit of energy, length and mass are set by
$\varepsilon$, $\sigma$, and $m$, respectively, which sets the unit of
time as $\sqrt{m \sigma^2 / \varepsilon}$. In these units, we choose
$\zeta = 1.0$, $k = 30 \varepsilon/\sigma$, $R_0 = 1.5\sigma$ and $k_B T
= 0.1 \varepsilon$. A time step of $\Delta t = 0.005$ is used in all
simulation runs. The simulations are done using LAMMPS
software~\cite{plimpton_LAMMPS_1995}.

The distance between the end monomers of the two strands is monitored as
a function of time, $x(t)$, for various force amplitudes $G$ and
frequency $\omega$. Due to the periodic nature of the applied force, the
extension $x(\mathrm{g})$ as a function of force $\mathrm{g}$ can be
obtained from the time series $x(t)$. This is then averaged over $1000$
cycles to obtain the average extension, $\langle x(\mathrm{g}) \rangle$.
For longer chains (i.e., $N=64$ and 96), the computation is very costly
in the lower frequency regime. However, it was observed that, in this
regime, the averaging over even $100$ cycles is good enough to give a
smooth $x(\mathrm{g})$ vs $\mathrm{g}$ loop. To be on a safer side, we
have used $200$ cycles for averaging. For the force amplitude $G$ and
the frequencies $\omega$ used in this work, the average extension,
$\langle x(\mathrm{g}) \rangle$, for the forward and the backward paths
is not the same and a hysteresis loop is observed. The area of the
hysteresis loop, $A_{\mathrm{loop}}$, defined as
\begin{equation}
    A_{\mathrm{loop}} = \oint \langle x (\mathrm{g}) \rangle
	d\mathrm{g},
\end{equation}
depends upon the frequency $\omega$ and the amplitude $G$ of the
periodic force and serves as a dynamical order
parameter~\cite{chakrabarti_dynamic_1999}. The area of the loop,
$A_{\mathrm{loop}}$ is obtained numerically using the trapezoidal rule
after dividing the interval $\mathrm{g} \in [0,G]$ into $10^5$ equally
spaced intervals, and interpolating the value of $\langle x(\mathrm{g})
\rangle$ at the ends of these intervals using cubic splines of GNU
Scientific Library~\cite{galassi_GSL_2009}.

\section{Results and Discussions} \label{sec:results}

\subsection{Static case}

Let us first consider the equilibrium case where the dsDNA is subjected
to a constant pulling force, i.e. $\mathrm{g}(t) = G$ and check whether
the average separation behaves similarly as that obtained from the DSAW
model.

\figTwo

In Fig.~\ref{fig:2}(a), we have plotted the average separation between
the strands of the DNA, where a constant pulling force $G$ is acting, at
various $G$ values for DNA having $N=64$, 96, 128, 192 and 256 base
pairs at $T=0.1$. For smaller values of the force, the average
separation, $\langle x \rangle$, which acts as an order parameter, is
zero showing that the two strands of the DNA are in the zipped phase. On
increasing the force value, the average separation abruptly increases at
some critical force value, $G_c$, and $\langle x \rangle \sim N$ showing
that the DNA is in the unzipped phase. The critical value of force $G_c$
can be obtained by using the finite size scaling (FSS) of the form
\begin{equation}
\label{eq:fssx}
    \langle x \rangle = N^{d} \mathcal{G} \left(  \frac{(G -
	G_c)}{N^{\phi}} \right),
\end{equation}
where $d$ and $\phi$ are critical exponents. In Fig.~\ref{fig:2}(b), we
have plotted the scaled separation $\langle x \rangle /N^d$ for DNA of
various chain lengths $N=64$, 96, 128, 192 and 256 as a function of
$(G-G_c)/N^{\phi}$. The data for various chain lengths collapse on a
scaling curve for the set of critical exponents $d=1.1 \pm 0.05$ and
$\phi = 1.0 \pm 0.05$ with critical force $G_c = 0.45 \pm 0.05$. The
exponents $d=1$ and $\phi=1$ show that, in the thermodynamic limit
(i.e., $N \to \infty)$, the scaled mean separation between the strands
behaves as
\begin{equation}
    \langle x \rangle/N \sim \left\{ \begin{array}{c c c} X_{-} &
	\text{for} & G < G_c \cr X_{+} & \text{for} & G > G_c  \end{array}
	\right. ,
\end{equation}
i.e., having two different values with a jump discontinuity at $G_c$,
implying the first order nature of the unzipping
transition~\cite{bhattacharjee_unzipping_2000,marenduzzo_phase_2001,marenduzzo_dynamical_2002,kapri_complete_2004}.

\subsection{Dynamic case}

From earlier studies, it is known that when a dsDNA is subjected to a
periodic force it can be unzipped either by keeping the amplitude $G$
fixed and changing the frequency $\omega$ or \textit{vice versa}. If $G$
is not very small, and $\omega$ is sufficiently high to avoid
equilibration of the DNA, we obtain a hysteresis loop for the average
extension $\langle x (\mathrm{g}) \rangle$, whose area,
$A_{\mathrm{loop}}$, depends on $G$ and $\omega$. In
Fig.~\ref{fig:3}(a), we have shown the behavior of $A_{\mathrm{loop}}$
as a function of $\omega$ for the DNA of length $N=64$ for $G=1.5$ at
$T=0.1$. The area of the loop increases with the frequency, reaches a
maximum and then decreases as the frequency is increased further. The
loops at four different frequencies, labeled by $P,\ Q,\ R$, and $S$ in
Fig.~\ref{fig:3} (a), are shown in Fig.~\ref{fig:3}(b). Since the force
amplitude $G=1.5$ is about three times the critical force needed to
unzip the DNA at $T=0.1$, the stationary state of the DNA is unzipped
state. At a higher frequency $\omega_P = 6.28 \times 10^{-3}$, the
applied force fluctuates very rapidly and the DNA does not get time to
respond to this change. As a result, the DNA remains in the unzipped
state, as indicated by the higher values of the average extension,
$\langle x \rangle$, with a small loop area. On decreasing the frequency
to $\omega_Q = 6.28 \times 10^{-4}$, the DNA still remains in the
unzipped phase but with slightly increase in the loop area. On
decreasing the frequency further to $\omega_R = 6.28 \times 10^{-5}$,
the DNA gets enough time to relax to the oscillating force. Therefore,
during the portion of the cycle where the instantaneous force value is
less than the critical force $G_c$, the two strands of the DNA come
together and the complementary base pairs are formed, resulting the DNA
in the zipped phase with a large hysteresis loop area. This is indicated
by the lower values of the average extension, $\langle x \rangle$ for
smaller $\mathrm{g}$ values in Fig.~\ref{fig:3}(b). On decreasing the
frequency further to $\omega_S = 6.28 \times 10^{-6}$, the two strands
have ample time to relax in the lower as well as higher values of force
thus resulting in a very small loop area in the transition region. This
loop area will eventually go to zero on decreasing the frequency
further.

\figThree

In Fig.~\ref{fig:4}, we have plotted $A_{\mathrm{loop}}$ as a function
of $\omega$ at various force amplitudes $G$ for the DNA of length
$N=64$. The figure shows that the frequency, $\omega^{*}(G)$, at which
the loop area is maximum depends on the amplitude $G$ of the oscillating
force and shift towards lower side with increasing $G$. The figure also
reveals that the height of the peak increases on increasing $G$ value.
These observations are exactly similar to the behavior seen for the
$A_{\mathrm{loop}}$ in MC simulations for the homopolymer
DNA~\cite{kapri_unzipping_2014}.

\figFour

\figFive

The loop area, $A_{\mathrm{loop}}$ as a function of frequency $\omega$
for amplitude $G=1$ and $3$, are plotted in Figs.~\ref{fig:5}(a) and
\ref{fig:5}(b), respectively, for the DNA of various chain lengths
$N=16$, 32, 64 and 96. The figure shows that, similar to the MC
simulation studies~\cite{kapri_unzipping_2014}, the peak of the area
curves shift towards the lower frequency side on increasing the chain
length. Furthermore, these plots also show that the maximum of the loop
area increases with amplitude $G$. We use FSS of the form
\begin{equation} \label{eq:ANscale}
    A_{\mathrm{loop}} = N^{d} \mathcal{A} \left( \omega N^z \right),
\end{equation}
to obtain the behavior of $A_{\mathrm{loop}}$ in the thermodynamic limit
from finite size chains. We obtain a nice collapse for exponents $d=1.0
\pm 0.01$ and $z=1.0 \pm 0.02$ for $G=1$ [Fig.~\ref{fig:5}(c)]. These
exponent values are same as that obtained in MC simulation
study~\cite{kapri_unzipping_2014}. However, for higher force amplitudes
(e.g., $G=3$) we get a reasonable collapse for a slightly higher value
$d=1.1 \pm 0.05$. Figures~\ref{fig:5}(c) and (d) show that there are
strong finite size effects and the curves for the smallest chain length
$N=16$ considered in this study do not collapse perfectly on the scaling
curve. However, the data for the higher chain lengths, $N=64$ and 96,
collapse perfectly on the scaling curve for $G=1$. For force amplitude
$G=3$, we still need to simulate higher chain lengths to obtain the
correct exponent $d=1$. This simulation is  computationally very
expensive and therefore not performed. The exponents $d=1$ and $z=1$
show that the loop area scales as $A_{\mathrm{loop}} \sim 1/\omega$ in
the high frequency regime.

\figSix

To obtain the behavior of $A_{\mathrm{loop}}$ at lower frequency regime,
we have plotted in Fig.~\ref{fig:6}(a) (in a semi-log scale) the loop
area as a function of $\omega^{\beta}(G-G_c)^{\alpha}$, where $G_c$ is
the critical force for the static force case, obtained using LD
simulations for the DNA of length $N=64$ at various force amplitudes
$G=1.0$, 1.25, 1.5, 2.0 and 3.0 at temperature $T=0.1$. We obtain an
excellent data collapse for exponent values $\alpha = 1.0 \pm 0.01$ and
$\beta = 1.25 \pm 0.02$ and critical force value $G_c(T=0.1)=0.45$
obtained for the static force case in the previous section
(Eq.~\eqref{eq:fssx}). These exponent values are exactly the same as
obtained in previous studies using MC simulations of a DSAW model of the
homopolymer DNA at $T=1$~\cite{kapri_unzipping_2014}, and the double
stranded block copolymer DNA at $T=4$~\cite{yadav_unzipping_2021}. In
the homopolymer DNA study, the $A_{\mathrm{loop}}$ was plotted against
$G^{\alpha}\omega^{\beta}$. To check the quality of collapse with newer
scaled function, we have  plotted the $A_{\mathrm{loop}}$ data, for the
chain length $N=512$, obtained in Ref.~\cite{kapri_unzipping_2014} as a
function of $\omega^{\beta}(G-G_c)^{\alpha}$ with $G_c (T=1) =0.678$ and
$\alpha=1$ and $\beta=1.25$. The plot shown in Fig.~\ref{fig:6}(b) is
found to be even better than the graph shown with function
$G^{\alpha}\omega^{\beta}$ in Ref.~\cite{kapri_unzipping_2014}.

We can use the behavior of $A_{\mathrm{loop}}$ in low and high frequency
regimes to obtain the scaling function $\mathcal{G}(\omega)$. At lower
frequencies (i.e., $\omega \to 0$), we observed that, for large $N$, the
$A_{\mathrm{loop}}$ scales as $G^{\alpha} \omega^{\beta}$, while at
higher frequencies (i.e., $\omega \to \infty$), $A_{\mathrm{loop}} \sim
1/\omega$ [from Eq.~\eqref{eq:ANscale}]. These requirements are
satisfied by the scaling function
\begin{equation} \label{eq:scalingCurve}
\mathcal{G}(\omega) = \frac{P G^{\alpha} \omega^{\beta} }{ \omega^{d +
	\beta} + Q^2},
\end{equation}
with $P$ and $Q$ as the fitting parameters. The scaling function,
$\mathcal{G}(\omega)$, for $G=1$ with exponent $d=1$, and parameters
$P=0.01$ and $Q=0.005$ obtained by data fitting, is plotted in
Fig.~\ref{fig:5}(c) by a solid line. The function fits the data
extremely well in the frequency range extended over more than four
decades. We have also plotted the same scaling function for $G=3$, with
exponent $d=1.1$ for parameters $P=0.001$ and $Q=0.002$ in
Fig.~\ref{fig:5}(d). Although the scaling function $\mathcal{G}(\omega)$
fits reasonably well in the lower frequency regime, it however deviates
with the scaled data at higher frequencies. From MC simulation
studies~\cite{kapri_unzipping_2014}, we know that for $G=3$,
$A_{\mathrm{loop}}$ exhibits an oscillatory behavior in the higher
frequency regime (visible only for longer chain lengths) and the above
scaling form is not suitable. Since the maximum chain length simulated
in this study, is still 5 times lesser than that simulated in
Ref.~\cite{kapri_unzipping_2014}, we do not see the oscillatory behavior
of $A_{\mathrm{loop}}$ at higher frequencies. The deviation of the
simulation data from the scaled curve $\mathcal{G}(\omega)$ for $G=3$ at
higher frequency regime in Fig.~\ref{fig:6}(d) may be due to this very
reason.

\section{Conclusions}\label{sec:summary}

We study the unzipping of a dsDNA subjected to a periodic force with
amplitude $G$ and frequency $\omega$ using extensive LD simulations on
longer DNA chains, having up to 184 monomers with $N=96$ base pairs,
that are six times longer than previous LD simulation
studies~\cite{kumar_statistical_2013,mishra_dynamical_2013,mishra_scaling_2013,kumar_periodically_2016,pal_dna_2018}.
We first study the static force case and obtain the equilibrium average
separation between the strands of the DNA, $\langle x (\mathrm{g})
\rangle$, as a function of force $\mathrm{g}$ at $T=0.1$. Using the FSS
of force-distance isotherms $\langle x (\mathrm{g}) \rangle$ for various
chain lengths $N=64$, 96, 128, 192 and 256, we obtain the critical force
$G_c(T=0.1) = 0.45 \pm 0.05$ needed to unzip the DNA in the
thermodynamic limit $N \to \infty$. The FSS reveals that the scaled
average separation between the strands, $\langle x \rangle/N$, has a
jump discontinuity at $G_c$ implying a first order nature of the phase
transition similar to the DSAW model
studies~\cite{bhattacharjee_unzipping_2000,marenduzzo_phase_2001,marenduzzo_dynamical_2002,kapri_complete_2004}.
When the DNA is subjected to a periodic force, the average separation
between the strands $\langle x (\mathrm{g}) \rangle$, when plotted
against $\mathrm{g}$, shows hysteresis whose area, $A_{\mathrm{loop}}$,
depends on the amplitude $G$ and the frequency $\omega$ of the
oscillating force. On decreasing the frequency, the loop area first
increases from zero, reaches a maximum value at some frequency
$\omega^{*}(G)$, which depends on the amplitude $G$, and then decreases
to zero again at lower frequencies. The FFS scaling of
Eq.~\eqref{eq:ANscale} shows that, in the thermodynamic limit, the loop
area scales as $A_{\mathrm{loop}} \sim 1/\omega$ in the higher frequency
regime. In contrast, the loop area which scales as $A_{\mathrm{loop}}
\sim (G-G_c)^{\alpha}\omega^{\beta}$ is found to have exponent values
$\alpha=1$ and $\beta=1.25$. These values are exactly the same as
obtained in earlier unzipping studies of homopolymer
DNA~\cite{kapri_unzipping_2014} and block copolymer
DNA~\cite{yadav_unzipping_2021} by a periodic force on a DSAW model
using MC simulations at two different temperatures. The fact that we
have obtained the same values for the exponents, $\alpha$ and $\beta$,
at three temperatures for two different problems, i.e., homopolymer DNA
and block copolymer DNA, where the former is studied by two different
methods, MC and LD simulations of longer chain lengths at different
temperatures strongly indicates that $\alpha=1.0$ and $\beta=1.25$ are
the true scaling exponents for the DNA unzipping problem that quantify
the decrease of $A_{loop}$ to zero at low frequencies at all
temperatures.

\section*{Acknowledgements}

I thank S. M. Bhattacharjee, A. Chaudhuri and R. Yadav for their
comments on the manuscript. I thank A. Chaudhuri for allowing me to use
his computational resources.

\input{main.bbl}

\end{document}


\section{Hamiltonian $H_{s}^{(l)}$}

The Hamiltonian $H_{s}^{(l)}$, which is the last term of Hamiltonian
$H_{\text{rxn}}$ (defined in main text), describes the low-frequency
vibrational modes of the solvent that help mediate electron transfer.
We define the former Hamiltonian as
\[
H_{s}^{(l)}=\hbar\sum_{j}\omega_{j}^{(l)}b_{j}^{\dagger}b_{j}+\hbar\sum_{j}\omega_{j}^{(l)}\sum_{X=R,P}|X\rangle\langle X|\left[\lambda_{Xj}^{(l)}(b_{j}+b_{j}^{\dagger})+(\lambda_{Xj}^{(l)})^{2}\right],
\]
where the $j$th low-frequency mode has frequency $\omega_{j}^{(l)}$
and couples to electronic state $|X\rangle$ with dimensionless strength
$\lambda_{Xj}$. The contribution of the low-frequency modes to the
reaction rate is characterized by reorganization energy $\lambda_{s}=\sum_{j}\left(\lambda_{Pj}^{(l)}-\lambda_{Rj}^{(l)}\right)^{2}\hbar\omega_{j}^{(l)}$.

\section{kinetic model \label{sec:kinetic-model}}

Here, we describe in detail the kinetic model used to simulate the
reaction. As discussed in the main text, this model only considers
states $|X,\chi\rangle$ whose vibrational-cavity component has zero
($\chi=0$) or one ($\chi=1_{q}$, where $q=1,\dots,N+1$) excitation.
The population $p_{(X,\chi)}$ of state $|X,\chi\rangle$ evolves
according to the master equation
\begin{align}
\frac{dp_{(X,\chi)}}{dt} & =-\sum_{(X',\chi')\neq(X,\chi)}k_{(X,\chi)\rightarrow(X',\chi')}p_{(X,\chi)}+\sum_{(X',\chi')\neq(X,\chi)}k_{(X',\chi')\rightarrow(X,\chi)}p_{(X',\chi')}.\label{eq:master-eq}
\end{align}
State-to-state transitions ($|X,\chi\rangle\rightarrow|X',\chi'\rangle$)
are either reactive ($X'\neq X$) or nonreactive ($X'=X$).

Rates ($k_{(X,\chi)\rightarrow(X',\chi')}$ for $X'\neq X$) of reactive
transitions are given by Eq. (\ref{eq:k_ET}) and depend on FC factor
$F_{\chi,\chi'}$, which is defined in the main text. We now evaluate
$F_{\chi,\chi'}$ for various combinations of $\chi$ and $\chi'$.
Recall the standard identity \citep{AgarwalBook}
\begin{equation}
\langle m'|D(\lambda)|m\rangle=\sqrt{\frac{m!}{(m')!}}e^{-|\lambda|^{2}/2}\lambda^{m'-m}L_{m}^{(m'-m)}(|\lambda|^{2}),\quad m'\geq m,\label{eq:id_displ}
\end{equation}
where $D(\lambda)=\exp(\lambda a^{\dagger}-\lambda^{*}a)$ is the
displacement operator corresponding to bosonic annihilation operator
$a$, $|m\rangle$ is a number state of the mode represented by $a$,
and $L_{n}^{k}(x)$ is an associated Laguerre polynomial. Using Eq.
(\ref{eq:id_displ}), we obtain
\begin{equation}
F_{\chi,\chi'}=\begin{cases}
e^{-S}, & (\chi',\chi)=(0,0),\\
e^{-S}S_{q}, & (\chi',\chi)=(1_{q},0),(0,1_{q}).\\
e^{-S}(1-S_{q})^{2}, & (\chi',\chi)=(1_{q},1_{q}).\\
e^{-S}S_{q'}S_{q}, & (\chi',\chi)=(1_{q'},1_{q}),\quad q'\neq q,
\end{cases}
\end{equation}
where $S=\sum_{q=1}^{N+1}S_{q}$ and $S_{q}=|\lambda_{Pq}-\lambda_{Rq}|^{2}$.

The nonreactive transitions in our model are of two types: (1) decay/gain
of an excitation in vibrational-cavity mode $q$ and (2) energy exchange
among dark and polariton states. An excitation in mode $q$ decays
at rate
\begin{equation}
k_{(X,1_{q})\rightarrow(X,0)}=|c_{q0}|^{2}\kappa+\left(\sum_{i=1}^{N}|c_{qi}|^{2}\right)\gamma,\label{eq:k_decay}
\end{equation}
where $\kappa$ is the decay (leakage) rate of the bare cavity and
$\gamma$ is the decay rate of all bare vibrations. Detailed balance
governs the rate of the reverse process: $k_{(X,0)\rightarrow(X,1_{q})}=k_{(X,1_{q})\rightarrow(X,0)}\exp(-\beta\hbar\omega_{q})$.
Relaxation among dark and polariton states is driven by anharmonic
coupling between bare molecular vibrations and their local chemical
environment \citep{delPino2015,Xiang2018,Xiang2019jpca}. Following
theories \citep{delPino2015,Agranovich2003,Litinskaya2004} of relaxation
dynamics for molecules under VSC, the transition from a polariton
or dark state to another has rate
\begin{align}
k_{(X,1_{q})\rightarrow(X,1_{q'})} & =2\pi\left(\sum_{i=1}^{N}|c_{q'i}|^{2}|c_{qi}|^{2}\right)\left[\Theta(-\omega)\left(n(-\omega)+1\right)J(-\omega)+\Theta(\omega)n(\omega)J(\omega)\right],\label{eq:k_redis}
\end{align}
where $q'\neq q$ and $\Theta(\omega)$ is the Heaviside step function.
The environmental modes are characterized by spectral density $J(\omega)$
and the Bose-Einstein distribution function, $n(\omega)=\left(\exp(\beta\hbar\omega)-1\right)^{-1}$.

\section{Default parameters for calculations of reaction rate \label{sec:default-parameters}}

Unless otherwise stated, calculations of reaction rates are carried
out using the parameters described in this section.

For the bare molecular vibrations, we choose a mean frequency of $\overline{\omega}_{v}=2000$
cm$^{-1}$, which is representative of experimental studies on VSC
\citep{Dunkelberger2016,Xiang2018,Lather2019,Xiang2019jpca,Hirai2020,Grafton2021}.
Since these studies do not report values of inhomogeneous broadening,
we simply take the vibrational frequencies to have a standard deviation
of $\sigma_{v}=10$ cm$^{-1}$. This choice of $\sigma_{v}$ yields
a spectral linewidth of $\approx24$ cm$^{-1}$, which is consistent
with vibrational lineshapes measured in some of the cited works \citep{Lather2019,Xiang2019jpca,Hirai2020,Grafton2021}.
Regarding the electronic degree of freedom and its coupling to other
degrees of freedom, we select parameters employed in the reaction
simulations of \citep{Du2021}: $E_{R}=0$, $E_{P}=-0.6\overline{\omega}_{v}$,
$\lambda_{R}=0$, $\lambda_{P}=1.5$, $J_{RP}=0.01\overline{\omega}_{v}$,
$\lambda_{s}=0.08\overline{\omega}_{v}$.

Next, we describe the parameters governing the relaxation of vibrational
and cavity modes. The temperature $T$ is set to 298 K. We choose
$\kappa=1$ ps$^{-1}$ as the bare cavity decay rate and $\gamma=0.01$
ps$^{-1}$ for the decay rate of all bare vibrational modes \citep{Xiang2019jpca}.
To model the relaxation among dark and polariton states {[}Eq. (\ref{eq:k_redis}){]},
we use the Ohmic spectral density $J(\omega)=\eta\omega\exp(-\omega/\omega_{\text{cut}})$,
where $\eta=2\times10^{-3}$ is the interaction strength between each
bare vibrational mode and its local chemical environment, and $\omega_{\text{cut}}=50$
cm$^{-1}$ is the cutoff frequency of the environmental modes. The
chosen spectral density resembles those in models of liquid-phase
molecular systems \citep{Chang1993,Vohringer1995,Kuhn2003}.

\section{Numerical kinetic simulations and rate calculations \label{sec:kinetic-sims_rate-calcs}}

In this section, we first describe our numerical simulations of the
reaction kinetics. We then discuss how we obtain the reaction rate
from the numerically determined reactant population versus time. To
be clear, we note that the quantities (e.g., populations, energies)
shown below pertain only to states with $\chi=0,1_{q}$ for $q=1,\dots,N+1$
(see main text).

State populations as a function of time are simulated by numerically
solving master equation (\ref{eq:master-eq}). We start by writing
the equation as $d\mathbf{p}/dt=A\mathbf{p}$, where $\mathbf{p}$
is the vector of populations and $A$ is the matrix of transition
rates. Subsequently, we apply the following standard procedure to
evaluate $\mathbf{p}(t)=\exp(At)\mathbf{p}(0)$ \citep{Frankcombe2001}.
This method employs the symmetrization of $A$ to avoid a numerically
unstable matrix inversion. First, we compute the matrix $B=MAM^{-1}$,
where $M$ is the diagonal matrix with diagonal elements $M_{(X,\chi),(X,\chi)}=f_{(X,\chi)}^{-1/2}$,
and $f_{(X,\chi)}=\exp(-\beta E_{(X,\chi)})/\sum_{(X,\chi)}\exp(-\beta E_{(X,\chi)})$.
Since the transition rates {[}Eqs. (\ref{eq:k_ET}), (\ref{eq:k_decay})-(\ref{eq:k_redis}){]}
satisfy detailed balance, $B$ is symmetric. After numerically diagonalizing
$B$, the population at time $t$ is evaluated as
\begin{equation}
\mathbf{p}(t)=M^{-1}Q\exp(Dt)Q^{\intercal}M\mathbf{p}(0),
\end{equation}
where $Q$ is a matrix whose columns are the eigenvectors of $B$,
$D$ is the diagonal matrix whose diagonal elements are the eigenvalues
corresponding to said eigenvectors, and $Q^{\intercal}=Q^{-1}$ due
to $B$ being symmetric. Because we are interested in thermally activated
reactivity, the vector of initial populations, $\mathbf{p}(0)$, is
taken to be a thermal distribution of reactant eigenstates:\begin{subequations}
\begin{align}
p_{(R,\chi)}(0) & =\frac{\exp(-\beta E_{(R,\chi)})}{\sum_{\chi}\exp(-\beta E_{(R,\chi)})},\\
p_{(P,\chi)}(0) & =0.
\end{align}
\end{subequations}We evaluate $\mathbf{p}(t)$ at $t=j\Delta t$,
where $\Delta t=0.2$ ns and $j=0,\dots,100$.

Next, the reaction rate is obtained by fitting the numerically determined
values of reactant population,
\begin{equation}
p_{R}=\sum_{\chi}p_{(R,\chi)},
\end{equation}
and their respective values of $t$ to the exponential function
\begin{equation}
p_{R}=\exp(-kt).
\end{equation}
The fitting parameter $k$ is the reaction rate. For all fits, the
adjusted $R^{2}$ values have mean 0.99999 and standard deviation
$8\times10^{-6}$. Such successful fitting reflects the reaction being
first-order \citep{AtkinsBook9th} in reactant. Here, first-order
kinetics occurs because product excited states do not accumulate sufficiently
(see $\S$\ref{sec:analytical_bare}) and the product ground state
does not revert to reactant states at a fast enough rate (due to high
activation energy).

\section{Analytical rate: bare reaction \label{sec:analytical_bare}}

A simplified kinetic model for the bare reaction is shown in Fig.
\ref{fig:rxn}(a). In this model, the populations of $|R,0\rangle$
and $|P,1_{r}\rangle$ evolve as
\begin{align}
\frac{dp_{(R,0)}}{dt} & =-k_{f}p_{(R,0)}+k_{b}p_{(P,1_{r})},\label{eq:p_R0_bare}\\
\frac{dp_{(P,1_{r})}}{dt} & =-(k_{b}+\gamma)p_{(P,1_{r})}+k_{f}p_{(R,0)},\label{eq:p_P1r_bare}
\end{align}
respectively. Since $k_{f}\ll k_{b},\gamma$ (see main text), $p_{(P,1_{r})}$
does not accumulate, and so we apply the steady-state approximation
(SSA) \citep{AtkinsBook9th} to this population: $dp_{(P,1_{r})}/dt\approx0$.
Solving the resulting equation for $p_{(P,1_{r})}$ and plugging the
solution into Eq. (\ref{eq:p_R0_bare}) leads to $dp_{(R,0)}/dt\approx-k_{\text{bare}}^{(\text{analytical})}p_{(R,0)}$.
Defined in Eq. (\ref{eq:k_bare}), $k_{\text{bare}}^{(\text{analytical})}=k_{f}\left(\frac{\gamma}{\gamma+k_{b}}\right)$
represents the net rate of reactant depletion, \emph{i.e.}, the reaction
rate.

\section{Analytical rate: VSC reaction\label{sec:analytical_VSC}}

Here, we consider a simplified kinetic model for the VSC reaction.
In this model, polaritons are decoupled from the reaction, and the
reaction proceeds through multiple reaction channels, each involving
the (de)excitation of a dark mode. Define $\mathcal{D}$ as the set
of dark modes. The populations of $|R,0\rangle$ and $|P,1_{q}\rangle$,
where $q\in\mathcal{D}$, evolve according to
\begin{align}
\frac{dp_{(R,0)}}{dt} & =-\sum_{q\in\mathcal{D}}k_{f}^{(q)}p_{(R,0)}+\sum_{q\in\mathcal{D}}k_{b}^{(q)}p_{(P,1_{q})},\label{eq:p_R0_VSC}\\
\frac{dp_{(P,1_{q})}}{dt} & =-\left(k_{b}^{(q)}+\gamma\right)p_{(P,1_{q})}+k_{f}^{(q)}p_{(R,0)},\label{eq:p_P1q_VSC}
\end{align}
respectively. In analogy to the derivation of $\S$\ref{sec:analytical_bare},
we can apply the SSA to each $p_{(P,1_{q})}$ to arrive at
\begin{equation}
\frac{dp_{(R,0)}}{dt}\approx-k_{f}\left(\sum_{q\in\mathcal{D}}|c_{qr}|^{2}\frac{\gamma}{\gamma+|c_{qr}|^{2}k_{b}}\right)p_{(R,0)},\label{eq:p_R0_VSC_2}
\end{equation}
where we have used Eq. (\ref{eq:k_f/b}). Eq (\ref{eq:p_R0_VSC_2})
is equivalent to $dp_{(R,0)}/dt\approx-k_{\text{VSC}}^{(\text{analytical})}p_{(R,0)}$,
where the reaction rate constant $k_{\text{VSC}}^{(\text{analytical})}=k_{f}\left(\sum_{q\in\mathcal{D}}|c_{qr}|^{2}\frac{\gamma}{\gamma+|c_{qr}|^{2}k_{b}}\right)$
is equivalent to that shown in Eq. (\ref{eq:k_VSC}) the main text.

\section{Thermodynamic parameters of activation}

To determine the thermodynamic parameters of activation for a given
reaction (\emph{i.e.}, set of reaction parameters excluding temperature,
$T$), we calculate the reaction rate, $k$, for $T=$ 273, 283, 288,
293, 298 K. The rates are computed using numerical kinetic simulations,
as described in $\S$\ref{sec:kinetic-sims_rate-calcs}. We then fit
the $(k,T)$ values to the Eyring-Polanyi equation \citep{Eyring1935jcp,Evans1935},
\begin{equation}
k=\frac{k_{B}T}{h}\exp\left(-\frac{\Delta H^{\ddagger}}{RT}+\frac{\Delta S^{\ddagger}}{R}\right),\label{eq:eyring-polanyi}
\end{equation}
where $R$ is the gas constant. The fitting parameters $\Delta H^{\ddagger}$
and $\Delta S^{\ddagger}$ are the enthalpy and entropy, respectively,
of activation.

For all fits, the adjusted $R^{2}$ values have mean 0.999999 and
standard deviation $2\times10^{-7}$. This excellent agreement between
the numerically obtained rates and the Eyring-Polanyi equation is
attributed to a fortuitous choice of temperature range over which
the fittings are performed. Suboptimal goodness of fit is expected
for general ranges of $T$ because the transition rates {[}Eqs. (\ref{eq:k_ET}),
(\ref{eq:k_decay})-(\ref{eq:k_redis}){]} of our kinetic model have
a different functional form (with respect to $T$) compared to Eq.
(\ref{eq:eyring-polanyi}).

\section{supplemental figures}

\begin{figure}[H]
\centering\includegraphics{rxn_kappa}

\caption{$k_{\text{VSC}}/k_{\text{bare}}$ as a function of $N$ for various
cavity leakage rates ($\kappa$). The plotted values are ratios of
rates ($k_{\text{VSC}}$, $k_{\text{bare}}$), where each rate is
an average over 5000 disorder realizations. The data corresponding
to $\kappa=1$ ps$^{-1}$ is the same as that in Fig. \ref{fig:rxn}(d). }
\end{figure}
\begin{figure}[H]
\centering\includegraphics{ratio-deloc_nS-g80}

\caption{(a) $k_{\text{VSC}}/k_{\text{bare}}$ and (b) $v_{r}$ delocalization,
as a function of cavity detuning $\delta$, for various $N$ and fixed
collective light-matter coupling strength $g\sqrt{N}=8\sigma_{v}$.
In (a), $k_{\text{VSC}}$ is an average over 5000 disorder realizations,
and $k_{\text{bare}}$ is a value used to obtain Fig. \ref{fig:rxn}(d).
As $N$ increases, the $\delta$-dependence of $k_{\text{VSC}}/k_{\text{bare}}$
becomes more similar to that of $v_{r}$ delocalization. }

\end{figure}

\bibliographystyle{apsrev4-2}


\section{Introduction}

Semantic segmentation/pixel labelling is one of the core %
tasks
in visual understanding,
which is widely used in %
scene parsing, human body parsing, and many downstream applications.
It is a per-pixel classification problem, which classifies each pixel in the image into a predefined set of categories.
In recent years, semantic segmentation methods based on deep convolution neural networks (CNNs) have made tremendous progress \cite{%
cityscpaes, %
everingham2012pascal%
},
and achieved impressive results on %
large benchmark datasets.
Note that, the common prerequisite for all these successes is the
availability of massive amounts of labeled data.

Unfortunately, labeling a large amount of samples is
very
expensive and time-consuming, especially in dense prediction problems, such as semantic segmentation.
As we need to label each pixel, %
it can be of
$60$ times more effort than
image-level annotation.
Recent research reveals that semi-supervised learning (SSL), which uses a large amount of unlabeled data in conjunction with a small amount of labeled data, is greatly beneficial to classification \cite{xie2020self, berthelot2019mixmatch, sohn2020fixmatch, berthelot2019remixmatch,
chen2020big, chen2020improved, grill2020bootstrap
}. These methods can be %
grouped
into consistency methods \cite{berthelot2019mixmatch, sohn2020fixmatch, berthelot2019remixmatch}, pseudo methods \cite{xie2020self}, representation learning \cite{%
grill2020bootstrap}.

A few works %
attempted
to apply SSL to semantic segmentation. The naive student \cite{chen2005naive} uses a large model to generate the pseudo labels with unlabeled video sequences, and apply iterative training for further improvement. They only consider the pseudo labels with the original %
input
images without perturbations. A recent work %
in
\cite{ouali2020semi} considers  adding perturbations to the %
images for semantic segmentation, but they forward images with different perturbations in different sub-decoders and enforce the consistency between sub-decoders with the main decoder. Thus, this method will inevitably increase memory consumption, and is hard to apply to high-resolution data. There are also some works paying attention to generative methods\cite{hung2018adversarial, mittal2019semi, souly2017semi}. AdvSemiSeg in \cite{hung2018adversarial} and %
the work in \cite{mittal2019semi} both %
employ
Generative Adversarial Network (GAN) and train the model with a discriminative loss over unlabeled data and a supervised loss over labeled data.

Here,
we propose an effective and efficient framework to apply SSL to semantic segmentation. We consider adding the perturbations with strong augmentation to make further use of the unlabeled data. Besides, we do not include sub-networks during training. Previous works based on consistency learning~\cite{chen2020improved,xie2019unsupervised,berthelot2019remixmatch} have shown that adding noise in the procedure of learning the pseudo labels will help improve the performance in classification problems. As semantic segmentation is a per-pixel classification problem, we also consider adding strong augmentation as perturbations. However, strong augmentations will affect the distribution of mean and variance in the batch normalization. This may be the reason why the strong augmentation was not adding to the SSL for semantic segmentation~\cite{chen2005naive}, or multiple branch networks were employed to process different perturbations~\cite{ouali2020semi}.  In order to avoid the distribution shift caused by data augmentations, we propose a simple but effective method to use a distribution-specific batch normalization (DSBN). We forward the strong augmented data and the weakly-augmented data with different batch normalization statics during training, and inference the model with the batch normalization statics calculated by the weakly-augmented data.

Furthermore, as the teacher network may not perform well under all situations, some unreliable regions may be included in the generated pseudo labels. Thus directly learning from all the pixels equally will introduce noise. Inspired by previous work~\cite{wang2019symmetric}, which selects the unreliable image-pairs, and exchanges the prediction with the learning target to avoid noise, we design a new self correction loss (\emph{SCL}) and dynamically modify the weight and learning target for each pixel for semantic segmentation.

Especially, our approach achieves state-of-the-art performance on the Cityscapes dataset \cite{cityscpaes} and Pascal VOC dataset \cite{everingham2012pascal} under a semi-supervised setting.
Our main contributions are summarized as follows.

Concretely, our contributions are three-fold:
\begin{itemize}

  \item We propose an effective and efficient semi-supervised learning framework for semantic segmentation. We consider adding strong augmentation during training without any sub-networks.
  \item  A distribution-specific batch normalization is proposed to overcome the difference of data distribution caused by strong data augmentation. Besides, we design a self-training robust loss to anti noisy labels. The loss function could correct the label data while resisting noise.
  \item We demonstrate the effectiveness of our approach with extensive and detailed experimental results, including a comparison with state-of-the-art methods, as well as an in-depth analysis of our approach with a detailed ablation study in a semi-supervised setting.
\end{itemize}

\section{Related Work}
\noindent\textbf{Semi-supervised classification.}
Most of the early researches on SSL focus on classification tasks.
Current state-of-the-art representative methods include consistency based methods \cite{berthelot2019mixmatch, sohn2020fixmatch, berthelot2019remixmatch}, Pseudo labeling \cite{xie2020self} and representation learning methods \cite{chen2020big, chen2020improved, grill2020bootstrap}.
A more detailed overview of SSL can be found in~\cite{zhu2009introduction}. The consistency based methods consider the prediction of an unlabeled image should not change significantly with little perturbations. UDA \cite{xie2019unsupervised} and ReMixMatch \cite{berthelot2019remixmatch} both use a weakly-augmented example to generate an artificial label and enforce consistency against strongly-augmented examples. FixMatch implements consistency training by applying strong (CutOut \cite{devries2017improved}, CTAugment \cite{berthelot2019remixmatch}, and RandAugment \cite{cubuk2020randaugment}) and weakly data augmentation to the inputs in both the unlabeled loss and the pseudo label generation, thus encouraging the outputs to be consistent for both augmented inputs. The pseudo labeling methods believe that the pseudo labels generated by a trained model can benefit the training of new models. Noisy student training \cite{xie2020self} is an iterative self-training method that makes use of unlabeled data with pseudo labels to improve accuracy. The correspondence among different transformations is employed for representation learning. And then a small amount of data is used to fine-tune the model.

\noindent\textbf{Semi-supervised  semantic segmentation}.  Early methods utilize a GAN model in semi-supervised segmentation in \cite{hung2018adversarial, mittal2019semi, souly2017semi}, where an adversarial loss is trained on unlabeled data. Consistency regularization based approaches \cite{ouali2020semi, french2019semi, ke2020guided, luo2020semi, luo2020semi} have also been proposed recently, and achieve promising results. They depend on the fact that unlabeled data and labeled data have the same distribution, so it expects the trained model to have consistent and reliable predictions for both the unlabeled data and the labeled data. In these works, different unlabeled images under different transformations are used as the input of the model, and consistency loss is enforced on the prediction mask of the model. Nowadays self-training pushes the performance of state of the arts \cite{chen2005naive, zoph2020rethinking}. However, these methods are very simple to use self-training and do not try to solve the noise impact caused by pseudo labels.

\noindent\textbf{Data augmentation.}
Data augmentation is an effective regularization. The basic policy of data augmentation contains random flip, random crop, etc., which are commonly used on semantic segmentation. In addition to the basic strategy, the inception-preprocess \cite{szegedy2015going} is more advanced when the input image color is randomly disturbed. Recently, AutoAugment \cite{cubuk2018autoaugment} improved the inception- preprocess for image classification, using reinforcement learning to search the optimal combination of existing strategies. RandAugment \cite{cubuk2020randaugment} proposed a significantly reduced search space which allows it to be trained on the target task with no need for a separate proxy task. However, these works are about image classification. In this paper, we try to apply strong augmentation to semantic segmentation and solve the problems in this process.

\noindent\textbf{Noisy dataset loss function.}
Some works pay attention to loss function in the noisy datasets for image classification, achieving improved results. The generalized cross-entropy \cite{zhang2018generalized} was proposed to achieve the advantages of both mean absolute error (MAE) and categorical cross-entropy losses (CCE); the CCE loss is a more general class of noise-robust loss that encompasses both of them. Inspired by the symmetry of the Kullback-Leibler divergence, the symmetric cross-entropy \cite{wang2019symmetric} was proposed by combining a noise tolerance term, namely reverse cross-entropy loss, with the standard CCE loss. Nevertheless, since semantic segmentation is a pixel-level classification task, these methods are based on classification and not suit for semantic segmentation.

\section{Methods}
\label{sec:methods}

\subsection{Semi-supervised Learning Framework}
In this section, we introduce our simple semi-supervised learning framework for semantic segmentation. Different from CCT~\cite{ouali2020semi}, which need to design auxiliary decoders for different perturbations, our method can be applied to any existing segmentation network as introduced in Algorithm \ref{algorithm}.
Given a small set of labeled training examples and a large set of unlabeled training examples, the small set of labeled training examples are exploited to train a teacher model with standard cross-entropy loss. We then use the teacher model to generate pseudo labels on unlabeled images with a test-time augmentation. In order to save disk space when processing a large set of unlabeled training examples, following Naive student \cite{chen2005naive}, we only generate hard pseudo labels, i.e. one-hot vectors. Then, we train a student model with strong augmentation to make full use of the unlabeled data. As the data distribution will be disturbed by the strong augmentation, we employ the \emph{DSBN} to resisting the influence of data distribution.
Moreover, motivated by \cite{wang2019symmetric}, for the sake of anti noisy labels, we develop a new loss function, self correction loss(\emph{SCL}). \emph{SCL} is used to anti noisy labels and anti ignore labels. Finally, following \cite{chen2005naive}, we iterate the process by putting back the student as a teacher to generate new pseudo labels and train a new student.

\begin{algorithm}[ht]
\caption{Semi-supervised Learning Framework}
\label{algorithm}
\small
\LinesNumbered
\KwIn{Labeled images \{($x_{1}$, $y_{1}$), ($x_{2}$, $y_{2}$), ..., ($x_{n}$, $y_{n}$)\} and unlabeled images \{($\widetilde{x}_{1}$, $\widetilde{x}_{2}$,
..., $\widetilde{x}_{m}$\} \\}
Optimize the teacher model with the cross-entropy loss on labeled images

Use the teacher model to generate hard pseudo labels for clean (\textit{i.e.}, not distorted) unlabeled images with multi scales and flip test. Generate semi dataset \{($\widetilde{x}_{1}$, $\widetilde{y}_{1}$), ($\widetilde{x}_{2}$, $\widetilde{y}_{2}$), ..., ($\widetilde{x}_{m}$, $\widetilde{y}_{m}$)\} \\

\While{Model performance improvement}{
\For{$i=1 ,..., n$ }{
\If{$\widetilde{x}$ $\in$ \{($\widetilde{x}_{1}$,$\widetilde{x}_{2}$,...,$\widetilde{x}_{m}$\}}{
	$\hat{x}$ $=$ SAUG($\widetilde{x}$)
}

[$y^{*}$, $\widetilde{y}^{*}$] $=$ f([$x$, $\hat{x}$])\\
loss $=$ CE($y^{*}$, $y$) $+$ SCL($\widetilde{y}^{*}$, $\widetilde{y}$) \\
loss backward \\
}
}
\end{algorithm}

\iffalse

\begin{figure}
    \centering
    \includegraphics[width=0.9\linewidth]{images/strongaug.pdf}
    \caption{Python code for Strong Augmentation based on numpy.}
    \label{fig: strong aug}
\end{figure}

\fi
\lstset{
  backgroundcolor=\color{white},
  basicstyle=\fontsize{8.5pt}{9.5pt}\fontfamily{lmtt}\selectfont,
  columns=fullflexible,
  breaklines=true,
  captionpos=b,
  commentstyle=\fontsize{9pt}{10pt}\color{codegray},
  keywordstyle=\fontsize{9pt}{10pt}\color{codegreen},
  stringstyle=\fontsize{9pt}{10pt}\color{codeblue},
  frame=tb,
  otherkeywords = {self},
}
\begin{figure}[!h]
\begin{lstlisting}[language=python]
Transforms = ['contrast gamma', 'contrast linear', 'brightness', 'brightness channel', 'equalize', 'hsv', 'invert channel', 'blur','noise gau', 'noise pos', 'Channel shuffle', 'dropout', 'coarse dropout', 'multiply', 'salt pepper', 'solarize', 'jpeg compression']

Basic_transforms = ['random scale', 'random flip', 'random crop', 'Normalization']

# data augmentation func
def saug(N):
    """ Generate a set of distortions.
        Args:
            N: Number of augmentation transformations to apply sequentially
    """
    sampled_ops = np.random.choice(Transforms, N)
    return sampled_ops + Basic_transforms
\end{lstlisting}
\vspace{-1em}
\caption{Python code for Strong Augmentation based upon numpy.}
\label{fig: strong aug}
\end{figure}

\noindent\textbf{Strong data augmentation for semantic segmentation (SAUG).}
While the augmentation strategy for supervised and semi-supervised image classification has been extensively studied \cite{berthelot2019remixmatch, cubuk2019autoaugment, cubuk2019randaugment, xie2019unsupervised, sohn2020fixmatch}, not much effort has been made yet for semantic segmentation. We extend the RandAugment \cite{cubuk2019randaugment} for semantic segmentation. As illustrated in Figure \ref{fig: strong aug} We build a pool of operations, which contains $16$ image transformation methods. In each training iteration, we randomly select a series of operations from the pool, and then combine them with standard transformations of semantic segmentation (random scale, random crop, random flip, .eg).

\begin{figure}[ht]
  \centering
    \subfigure[train phase]{\includegraphics[scale=0.5]{images/dsbn-t.pdf},
    \label{training phase}}
    \subfigure[test phase]{\includegraphics[scale=0.5]{images/dsbn-i.pdf},
    \label{test phase}}
  \caption{Distribution-Specific Batch Normalization (\emph{DSBN}). (a) represents the DSBN at the training phase. Images with strong augmentation is passed through PBN, while images with weekly augmentation are passed through BN. (b) donates the DSBN at the test phase. PBN is abandoned at the test phase.
}
	\label{fig:DSBN}
	\vspace{0.2in}
\end{figure}

\subsection{Distribution-Specific Batch Normalization}
As mentioned in some previous works \cite{rota2018place, zhao2017pyramid, chen2018encoder}, batch normalization is crucial for semantic segmentation. BN \cite{ioffe2015batch} computes the mini-batch mean ($\mu_B$) and standard deviation ($\sigma_B$), formulated in Equation \ref{equ: batch mean var}. Then, as illustrated in \ref{equ: batch norma}, BN \cite{ioffe2015batch} uses a learnable scale ($\gamma$) and shift ($\beta$) parameter to transform the normalized distribution. Meanwhile, moving mean and moving variance are updated by Equation \ref{equ: batch norm decay}. In the testing phase, BN uses the moving mean and moving variance for whitening input activation.
However, strong augmentations will destroy the distribution of natural images and lead to a domain gap between training images and testing images.
Thus, the moving mean and moving variance calculated during the training phase will harm the testing performance. To deal with the domain gap, we proposed a distribution-specific batch normalization. As illustrated in Figure~\ref{fig:DSBN}, we apply a parallel batch normalization (\emph{PBN}) initialized by the original BN. In the training phase, if data comes from pseudo labels, the \emph{PBN} is used in the forward; otherwise, the standard BN is used. In the testing phase, the \emph{PBN} is deprecated, and only the standard BN is used for normalization:
\begin{equation}
    \mu_B = \tfrac{1}{m}\sum_{i=1}^{m}x_i ; \;\;   \sigma_B =
    %
    \Bigl
    [ {\tfrac{1}{m}\sum_{ i =1 }^{m}(x_i-\mu_B)^2 + \epsilon} \Bigr ] ^{0.5},
    \label{equ: batch mean var}
\end{equation}
\begin{equation}
    y_i = \gamma \frac{(x_i - \mu_B)}{\sigma_B} + \beta,
    \label{equ: batch norma}
\end{equation}
\begin{equation}
	\begin{split}
    \mu_{t+1} = \alpha \mu_{t} + (1 - \alpha) \mu_{t+1}, \\
    \sigma_{t+1} = \alpha \sigma_{t} + (1 - \alpha) \sigma_{t+1}.
    \end{split}
    \label{equ: batch norm decay}
\end{equation}

\subsection{Self correction Loss}

Previous work~\cite{wang2019symmetric} has shown that cross-entropy loss masks the model overfit to noisy labels on some easier classes and under learning on some harder classes in image classification.
As semantic segmentation is a per-pixel classification problem, similar challenges also exist. Moreover, in the semi-supervised learning framework, the learning targets are pseudo labels, which may contain unreliable regions caused by the limited capacity of the current teacher model.

To avoid overfitting to the noise contained in the pseudo labels, we propose a Self Correction Loss (SCL) for semantic segmentation. It assigns an adaptive weight by the confidence of the network output to each pixel during the training phase. Besides, we apply a noise-robust term Reverse Cross Entropy~\cite{wang2019symmetric} to the unreliable regions. Different from previous work, which chooses the reversed learning pairs by fixed confidence from human heuristics, we dynamically change the learning targets by comparing the output of the student network and the teacher network.

As shown in Equation~\ref{equ: ASCL equation}, $i$ is the location index, $\mathbf{y_i^*}$ denotes the prediction of a pixel, $\mathbf{y_i}$ denotes the pseudo label generated by the teacher, and $w_i$ represents the dynamic weight which is the largest activation after softmax among all $c$ classes as shown in Equation~\ref{equ: ASCL equation weights}.  We use the prediction to automatically adjust the confidence during training. If the confidence is very high,
we adopt positive learning. While, if the confidence is very low, we think that the pseudo label at this pixel is unreliable. Thus, reverse learning is applied. In addition, SCL is only applied on pseudo labels:
\def {\mathbf y }{ {\mathbf y }}
\begin{equation}
	\ell_i = w_i \cdot  { {\mathbf y }_i} \log( { {\mathbf y }_i^*})+(1-w_i) \cdot
	{ {\mathbf y }_i^*} \log( { {\mathbf y }_i}),
	\label{equ: ASCL equation}
\end{equation}
\begin{equation}
\label{equ: ASCL equation weights}
	w_{i} = \max(\frac{ \exp(y_{i0}^*)}{\sum_{j=0}^{c} \exp(y_{ij}^*)} , ... ,\frac{ \exp(y_{ic}^*)}{\sum_{j=0}^{c} \exp(y_{ij}^*)}).
\end{equation}
\input{experimrnts}

\section{Conclusion}

In this work, we construct a simple semi-supervised learning framework for semantic segmentation. It employs strong augmentation with distribution-specific batch normalization in the pseudo label learning efficiently. In particular, DSBN can effectively avoid the statistical bias caused by strong augmentation. Meanwhile, self correction loss also effectively avoids the noise problem of the pseudo labels. Extensive quantitative and qualitative comparisons show that the proposed method performs favorably against recent state-of-the-art semi-supervised approaches.

\input supplementary.tex

{\small
\bibliographystyle{ieee_fullname}

\section{Experiments}
In this section, we first report the implementation details. Then we perform a series of ablation experiments, and analyze the results in detail. Finally, we report our results compared with other state-of-the-art methods.
\subsection{Implementation Details}

\noindent\textbf{Training.} Our implementation is built on Pytorch \cite{paszke2019pytorch}.
Following previous work \cite{chen2018encoder}, we use the ``poly" learning rate policy where the base learning rate is multiplied by $(1-\frac{\rm iter}{
{\rm iter}_{\max}})^{\rm power}$. The initial learning rate is set to $0.01$, and power is set to $0.9$. We train the network using mini-bath stochastic gradient descent (SGD).
We employ the crop size of $769 \times 769$ on Cityscapes with DeepLabv3Plus. More details is same as \cite{chen2018encoder}.
In the addition study, following \cite{zhao2017pyramid, yuan2020multi, chen2018encoder, lin2017refinenet, wu2019wider}, we adopt the mean intersection of union (mIoU) as the evaluation metrics.

\noindent\textbf{Data augmentation.} Following \cite{chen2018encoder}, we use mean subtraction, and apply data augmentation by random resize between 0.5 and 2 and random left-right mirror during initialization training. And in semi supervised learning phrase, we use strong augmentation for improving the performance of self training.

\noindent\textbf{Dataset.} Following \cite{chen2005naive}, We conduct the main experiments and ablation studies on the Cityscapes dataset \cite{cityscpaes}. This large-scale data set contains different stereo video sequences recorded in $50$ different urban street scenes. In addition to $20k$ weak annotation frames, there are $5,000$ high-quality pixel level annotation frames, in which, $2,975$;
$500$ and $1,525$ were used for training, validation and testing. In addition, there are $20k$ images with rough annotations, namely train-coarse. We randomly subsample %
$\nicefrac{1}{8}$,
$\nicefrac{1}{4}$, and
$\nicefrac{1}{2}$
of images in the standard training set to construct the pixel-level labeled data. The remaining images in the training set are used as unlabeled data.

At last, we report our results on the Pascal VOC 2012 dataset \cite{everingham2012pascal}, which contains 21 classes including background. The standard Pascal VOC 2012 dataset has $1,449$ images as the training set and $1,456$ images as the validation set. We construct $1,449$ images as the pixel-level labeled data. The images in the augmented set \cite{SBD} (around 9k images), are used as unlabeled data.

\begin{table*}
 \footnotesize
\begin{center}
\begin{tabular}{l c c c c c c c c c}
\hlineB{2}
Baseline &ST &SAUG& DSBN &SCL &ITER & mIoU-$\nicefrac{1}{8}$ (\%) &mIoU-$\nicefrac{1}{4}$ (\%) & mIoU-$\nicefrac{1}{2}$ (\%) & mIoU-Full (\%) \\
\hline\hline
Deeplabv3plus     & && & & & 68.9&73.4&76.9&78.7 \\
Deeplabv3plus     &$\checkmark$ && & & &71.2&76.2&78.0&79.5 \\
Deeplabv3plus     &$\checkmark$ &$\checkmark$& & & &71.3 &76.3 &78.2 &79.7 \\
Deeplabv3plus     &$\checkmark$ &$\checkmark$&$\checkmark$ & & &72.3&76.9&78.7&80.0 \\
Deeplabv3plus     &$\checkmark$ &$\checkmark$ &$\checkmark$&$\checkmark$ & & 72.8&77.4&78.7&80.4\\
Deeplabv3plus     &$\checkmark$ &$\checkmark$ &$\checkmark$&$\checkmark$ &$\checkmark$ & 74.1&77.8&78.7&80.5\\
\hlineB{2}
\end{tabular}
\end{center}
\caption{Ablation study on the proposed semi-supervised learning framework. The baseline is Xception65-based Deeplabv3plus. `ST' denotes self-training with pseudo labels without strong augmentation.`SAUG' means the strong augmentation. `DSBN' means the distribution specify batch normalization. `SCL' is the proposed self correction loss. `ITER' represents iterative training. The results are evaluated on the validation set with the single-scale input.  `mIoU-$\nicefrac{1}{n}$' means that we use  $\nicefrac{1}{n}$ labeled data and the remaining images in the training set are used as unlabeled data. `mIoU-Full' means that we use all the labeled data in the training set and the train-coarse are employed as the unlabeled dataset.}
\label{tab: ablation for ist}
\end{table*}

\subsection{Ablation Study}
In this subsection, we conduct the experiments to explore the effectiveness of each proposed module under different semi-supervised settings. First, we establish the baseline for our experiments. We evaluated DeepLabv3Plus based on Xception65 with cross entropy on the validation set. Following \cite{chen2005naive, xie2020self}, self training adopts baseline model as the teacher model and the teacher model generates pseudo labels on the unlabeled dataset, then the student model which is not smaller than the teacher model is trained on pseudo labels and original labeled images. We use the scales including $0.5$, $0.75$, $1.0$, $1.5$, $1.75$  and flip for the remaining images to generate pseudo labels. For fair comparison and quick training, the student model is the same model initialized by the teacher model. The main results are shown in Table~\ref{tab: ablation for ist}, and we will give detail discussions on each module in the following sections. All the ablation study are conducted on Cityscapes.

\subsubsection{Different semi-supervised settings}

We follow previous works~\cite{french2019semi,souly2017semi,hung2018adversarial} to divide the train set into labeled data and unlabeled data according to different proportions. We
use
$\nicefrac{1}{8}$,
$\nicefrac{1}{4}$,
and $\nicefrac{1}{2}$
of the training dataset as labeled data and other images as the unlabeled data. The evaluation results are shown in Table~\ref{tab: ablation for ist} as
mIoU-$\nicefrac{1}{8}$, mIoU-$\nicefrac{1}{4}$, mIoU-$\nicefrac{1}{2}$, respectively. From the Table \ref{tab: ablation for ist}, we can see that our algorithm can effectively improve the performance by $5.2\%$, $4.4\%$, $1.8\%$, $1.8\%$, respectively.
When the total number of labeled data and unlabeled data does not change, the semi-supervised learning framework will have a larger improvement, if we have less labeled images. In the meantime, the final results will be better if we have more high-quality labeled images. Besides, we can see that only half the amount of labeled data with the help of proposed semi-supervised learning framework can achieve the same results (mIoU of $78.7\%$) as we use all labeled training data with supervised learning (row 1 and col 10).

When the total number of available data increases, semi-supervised learning will play a more valuable role. We  use all high-quality labeled data in the training set as the labeled data and employ the train-coarse as unlabeled data to conduct the experiment, and the evaluation results are referred to as mIoU-Full in Table~\ref{tab: ablation for ist}. We can see that under a strong baseline with large amount of labeled data, if extra unlabeled data can be obtained, the performance will be further improved with the proposed semi-supervised learning framework.

In practical applications, labeled data is often limited, but unlabeled data is easy to get. To explore the impact of increasing the ratio of unlabeled data in the training set.
We fix the labeled data as $1/8$ of the training set, and increase the number of the unlabeled data. The results are shown in Figure~\ref{fig:paper_Unlabel_images_multiples}.
From Figure~\ref{fig:paper_Unlabel_images_multiples}, we can see that increasing ration of the unlabeled data to the labeled data will improve the results, but the growth trend of the performance is gradually flattening limited by the capacity of the initial teacher model trained on the fixed labeled data. The performance will even drop if too many unlabeled images are introduced during training.

\begin{figure}[t]
  \centering
    \includegraphics[scale=0.8]{images/paper_Unlabel_images_multiples.eps}
  \caption{Different ratio of unlabeled data to labeled data.
  }
	\label{fig:paper_Unlabel_images_multiples}
	\vspace{-1em}
\end{figure}

\subsubsection{Impact of the strong augmentation}

In this section, we will show the impact of the proposed strong augmentation and the DSBN. First of all, as shown in Table~\ref{tab: ablation for ist}, we compare the results of the naive self-training and the self-training combined with strong augmentation and the proposed DSBN. Although self-training can improve the performance over the baseline,
self training directly uses pseudo labels that makes the network difficult to learn and introduces noise, as discussed in Section \ref{sec:methods}. Meanwhile, in the case of only using strong augmentation, there is no significant improvement in performance.
On the contrary, our method performs favorably against naive self training with strong augmentation and DSBN. In particular, compared with naive self training, our method improves the mIoU by $1.1\%$, $0.7\%$, $0.7\%$ and $0.5\%$ based on DeepLabv3Plus. In the following sections, we will analysis the distribution of the BN statistics, and then we compare the strong augmentation with different BN settings. Finally, we provided some discussions on how does the strong augmentation work with ground truth labels and the pseudo labels.

\noindent\textbf{Visualization of the BN statistics.}
To verify our statement that the distribution of the BN will be affect by the strong augmentation, we show the statistics of BN training with strong augmentation input images and the weakly augmentation input images. The visualization results can be found in Figure~\ref{fig:bn dif}.

\begin{figure}
    \centering
     \subfigure[mean]{\includegraphics[scale=0.45]{images/mean_encoder.block20.sep_conv2.block.bn_point.main_bn.running_mean.png}}
     \subfigure[variance]{\includegraphics[scale=0.45]{images/var_encoder.bn1.main_bn.running_mean.png}}
    \caption{Distribution difference of BN statics between weakly augmentation and strong augmentation. The red line donates BN statics trained with weakly augmentation data. The blue line  donates BN statics trained with strong augmentation data.
}
    \label{fig:bn dif}
\end{figure}

The distribution of the mean and variance obtained from the whole training statistics deviates from the test set after adding strong augmentation. As shown in Figure \ref{fig:bn dif}, we calculate the distribution of the mean and the variance of the BN with the weakly augmentation and the strong augmentation. The blue line denotes the distribution of the BN with weakly augmentation while the red line represents the BN with strong augmentation. We can see that strong augmentation has changed the distribution of BN statistics, which may lead to a domain gap. We think this is the reason why strong augmentation has not brought much performance improvement as shown in Table \ref{tab: ablation for ist}.

\noindent\textbf{Impact of the DSBN.}
To show the effectiveness of proposed DSBN, we conduct experiments to compare three different BN settings as follows:
\begin{itemize}
    \item Trainable BN: the mean and the variance are updated during training with both weak augmentation data and strong augmentation data.
    \item Fixed BN: the mean and the variance are fixed during training, and initialized with the pre-trained weights from the ImageNet classification.
    \item DSBN: different mean and the variance are updated during training for weak augmentation data and strong augmentation data, separately.
\end{itemize}
The baseline model is DeepLabV3Plus trained with $1/8$ labeled data on the training set. The other $7/8$ images are used as the unlabeled dataset in the semi-supervised learning. The results are shown in Table~\ref{tab: ablation for bn}. We can see that with the proposed DSBN, the performance can be improved by $0.7\%$, which indicates the DSBN can help to solve the negative effect caused by the strong augmentation.

\begin{table}[b]
\begin{center}
\resizebox{0.6\linewidth}{!}{
\begin{tabular}{l c }
\hlineB{2}
Baseline  & mIoU (\%)  \\
\hline\hline
ST     & 71.2 \\
ST + SAUG + Trainable BN &71.3 \\
ST + SAUG + Fixed BN   &71.6 \\
ST + SAUG + DSBN & 72.3 \\
\hlineB{2}
\end{tabular}
}
\end{center}
\caption{Ablation study for different batch normalization methods.}
\label{tab: ablation for bn}
\end{table}

\noindent\textbf{Discussions.}
To further understand the impact of the strong augmentation and DSBN on different data groups, we apply strong augmentation to only ground truth labels and pseudo labels.
\begin{itemize}
    \item \textit{$\nicefrac{1}{8}$ ground truth.} We train the model with only 1/8 images of the training dataset with corresponding ground truth labels.
    \item \textit{Full-ground truth.} We train the model with all images of the training dataset with corresponding ground truth labels.
    \item \textit{$\nicefrac{7}{8}$ pseudo labels.} We train the model with 7/8 images of the training dataset with corresponding pseudo labels.
\end{itemize}

For each settings, we apply weekly augmentation, strong augmentation and DSBN during training. The experiment results are shown in Table~\ref{tab: ablation for aug}. We can see that under the fully supervised settings, the strong augmentation can not bring extra improvements. Meanwhile, when applying to the unlabeled data with pseudo labels, the mIOU improved by $0.8\%$. These observations indicate that improvements under semi-supervised settings are not come from extra image transformation types on labeled data, but benefit from make further use of unlabeled data with strong augmentations.
From Table~\ref{tab: ablation for aug}, we also find that the DSBN can contribute to the training.
\textit{In addition, %
to our knowledge,
this is the first time to apply the strong augmentation to semantic segmentation under fully-supervised setting, and %
improvement was observed.}.

\begin{table}[t]
\begin{center}
\resizebox{0.8\linewidth}{!}{
\begin{tabular}{l |c }
\hlineB{2}

\hline
Methods  & mIoU (\%)\\
\hline
\multicolumn{2}{c}{Training with ground truth labels}\\
\hline
$ %
\nicefrac{1}{8}
$-ground truth     & 68.9 \\
$ %
\nicefrac{1}{8}
$-ground truth + SAUG &68.9 \\
$ %
\nicefrac{1}{8}
$-ground truth + SAUG + DSBN & 69.5\\
\hline
Full-ground truth     & 78.7 \\
Full-ground truth + SAUG &78.7 \\
Full-ground truth + SAUG +  DSBN & 79.2\\
\hline
\multicolumn{2}{c}{Training with pseudo labels}\\
\hline
$ %
\nicefrac{7}{8}$-pseudo labels    & 70.6 \\
$ %
\nicefrac{7}{8}
$-pseudo labels + SAUG &71.4 \\
$ %
\nicefrac{7}{8}
$-pseudo labels + SAUG + DSBN &72.5 \\
\hlineB{2}
\end{tabular}
}
\end{center}
\caption{Effect of the strong augmentation on different data groups. We apply the strong augmentation to the baselines trained with only labeled data, and only unlabeled data with pseudo labels, separately.}
\label{tab: ablation for aug}
\vspace{-1em}
\end{table}

\subsubsection{Impact of the Self Correction Loss}
To further improve the performance and reduce the impact of noise, we employ a self correction loss (SCL). From Table~\ref{tab: ablation for ist}, we can see that the proposed SCL will play a more important role when the pseudo labels contain more noise. The teacher model trained with a limited labeled dataset may get less useful knowledge, and can not generalize well on unlabeled dataset. Thus, the level of the noise is higher than that training on more labeled data. When we train with
$\nicefrac{1}{8}$
or
$\nicefrac{1}{4}$
labeled images, the SCL can improve the mIoU by
$0.5\%$.

Furthermore, we show the mIoU during the training phase with and without the proposed SCL in Figure \ref{fig:scl iterations}. When training without the SCL, the result will be divergent quickly. On the contrary, the result is relatively stable.
\begin{figure}[th]
  \centering
    \includegraphics[scale=0.705]{images/paper_scl_weights.eps}
  \caption{This curve %
  shows
  the
  %
  %
  %
  validation mIoU w.r.t.\ the number of iterations during training.
  The green line represents the SCE loss \cite{wang2019symmetric}. In addition, we set the same ratio between CE and RCE. The red line represents our proposed loss with same initial weights as the SCE loss. }
	\label{fig:scl iterations}
	\vspace{0.2in}
\end{figure}
\subsubsection{Impact of the
Iterative
Training
}
Following Naive Students~\cite{chen2005naive}, we also test the impact of the
iterative
training in our semi-supervised learning framework. Table~\ref{tab: ablation for ist} has shown that the iteration training will become less useful if more labeled data can be obtained. Because the initial teacher model trained with more labeled data can already produce satisfactory pseudo labels. We show the change of the performance for the number of iterations in Figure~\ref{fig: iter_dp3}. The baseline model is DeepLabV3Plus trained with $1/8$ labeled images. We can see that more iterations
indeed
help improve the performance, but the growth trend of the performance is gradually flattening.
Besides, we find that the higher the performance of the model, the less the maximum number of iterations in Figure~\ref{fig: relation_iters}.
The baseline model is obtained by
$\nicefrac{1}{2}$,
$\nicefrac{1}{4}$,
and
$\nicefrac{1}{8}$
labeled images. A stronger baseline will benefit less from the iteration training.

\begin{figure}
  \centering
    \subfigure[]{\includegraphics[scale=0.749]{images/paper_iters_deeplabv3+.eps}
    \label{fig: iter_dp3}
    }
    \subfigure[]{\includegraphics[scale=0.749]{images/paper_iters_miou.eps}
    \label{fig: relation_iters}
    }
  \caption{Iterations for semi supervised semantic segmentation. (a) The performance of the DeepLabV3Plus under different number of iterations. (b) The maximum convergence iterations for different baseline models.}
	\label{fig:iterations for segmentation}
\end{figure}

\begin{figure*}
    \centering
    \includegraphics[width=0.97\linewidth]{images/results.pdf}
    \caption{Qualitative results on the Cityscapes dataset using 1/8 labeled samples. The proposed semi-supervised approach produces improved results compared to the baseline and the naive self training. We highlight the details in yellow boxes.}
    \label{fig: results}
    \vspace{-1em}
\end{figure*}

\subsubsection{Visualization Results}
Several visualization results are shown in Figure \ref{fig: results}. The baseline method is trained with
$\nicefrac{1}{8}$
labeled images. The ST represents the self-training trained with
$\nicefrac{7}{8}$
pseudo labels without our methods. The ours donates our framework. From Figure \ref{fig: results}, we find that the baseline and self training can not separate the objects (especially big objects like bus, truck, train, sidewalk) completely while ours corrects these errors. For pole and bicycle, our method also performs well on these small-size-object classes in the images compared to the baseline model and self training.

\subsection{Comparison with state-of-the-art Methods}
\noindent\textbf{Cityscapes.}
We conduct the comparison experiments with other state-of-the-art algorithms on the Table \ref{tab: SOTA  on different amount cityscapes}. To make a fair comparison, we also apply our proposed framework to DeepLabV2 following~\cite{hung2018adversarial}. In particular, DeepLabV2 is based on ResNet-101 initialized with the pre-trained weights on ImageNet classification. The proposed framework achieves 67.6\%, 69.3\%, 70.7\% with DeepLabV2 and 74.1\%, 77.8\%, 78.7\% with DeepLabV3Plus, respectively, which performs favorably against previous state-of-the-art methods. More details ablation studies of with DeepLabV2 can be found in the supplementary materials.

\begin{table}
\begin{center}
\resizebox{\linewidth}{!}{
\begin{tabular}{r  c c c c c}
\hlineB{2}
Baseline  &Model & $\nicefrac{1}{8} $ & $\nicefrac{1}{4} $ & $\nicefrac{1}{2} $ &Full\\
\hline\hline
AdvSemiSeg\cite{hung2018adversarial}&DeepLabV2 &58.8 & 62.3 &65.7 &66.0 \\
S4GAN + MLMT\cite{mittal2019semi} &DeepLabV2 &59.3 &61.9 &- &65.8 \\
CutMix\cite{french2019semi}&DeepLabV2 &60.3 &63.87 &- &67.7 \\
DST-CBC\cite{feng2020semi} &DeepLabV2 &60.5 &64.4 &- &66.9\\
ClassMix\cite{olsson2020classmix} &DeepLabV2 &61.4 &63.6 &66.3 &66.2 \\
ECS\cite{mendel2020semi} &DeepLabv3Plus &67.4 &70.7 &72.9 &74.8 \\
\hline
Baseline &DeepLabV2  &60.6 &66.7 &69.3 &70.1\\
Ours&DeepLabV2 &67.6 &69.3 &70.7 &70.1\\
Baseline& DeepLabV3Plus&68.9&73.4&76.9&78.7\\
Ours&DeepLabV3Plus &74.1 &77.8 &78.7 &78.7\\
\hlineB{2}
\end{tabular}
}
\end{center}
\caption{Compared with  state-of-the-art\xspace methods on the Cityscapes val set. Here %
`$\nicefrac{1}{n}$'  means
that
we use  $ \nicefrac{1}{n} $ labeled data and the remaining images in the training set are used as unlabeled data.}
\label{tab: SOTA  on different amount cityscapes}
\vspace{-1em}
\end{table}

\noindent\textbf{Pascal VOC.}
We follow DeepLabV3Plus \cite{chen2018encoder} training details with 513×513 crop size on Pascal VOC \cite{everingham2012pascal}. Table \ref{tab: SOTA in voc} lists the performance results of other state-of-the-art methods and ours. We adopt the single-scale testing on our experiments. In order to make a fair comparison, we did experiments under different backbones. Our framework achieves 75.0\% mIoU based on Resnet-101\cite{He2016Deep} and 79.3\% mIoU based on Xception-65\cite{chen2018encoder}, which outperforms the PseudoSeg \cite{zou2020pseudoseg} by 1.8\%. At the same time, we find that our results based on Xception-65 can achieve the same performance under supervision. In addition, we find that the improvement of iterative training on Pascal VOC is limited, so we do not use it here.

\begin{table}
\begin{center}
\resizebox{0.7\linewidth}{!}{
\begin{tabular}{ r  c c c c c c c}
\hlineB{2}
Method     &model &mIoU \\
\hline\hline
GANSeg\cite{souly2017semi} &VGG16 &64.1 \\
AdvSemSeg\cite{hung2018adversarial} &ResNet-101 &68.4 \\
CCT \cite{ouali2020semi} &ResNet-50 &69.4 \\
PseudoSeg \cite{zou2020pseudoseg} & ResNet-101 &73.2 \\
\hline
Ours &ResNet-101 &75.0 \\
Ours &Xception-65 &79.3 \\
Fully supervised &ResNet-101 &78.3 \\
Fully supervised &Xception-65 & 79.2 \\
\hlineB{2}
\end{tabular}
}
\end{center}
\caption{Comparison with  state-of-the-art\xspace on the Pascal VOC val set (w/ unlabeled data). We use the official training set (1.4k) as labeled data, and the augmented set (9k) as unlabeled data.
}
\label{tab: SOTA in voc}
\vspace{-1em}
\end{table}

\section*{Appendix}

\renewcommand{\thesection}{S\arabic{section}}
\section{Ablation Study of DeepLabV2}

We conduct the experiments to explore the effectiveness of each proposed module under 1/8 labeled image settings. First, we establish the baseline for our experiments. We evaluated DeepLabV2 based on ResNet-101 with cross-entropy on the validation set. Same as DeepLabV3Plus in our paper, self-training adopts the baseline model as the teacher model and the teacher model generates pseudo labels on the unlabeled dataset, then the student model which is not smaller than the teacher model is trained on pseudo labels and original labeled images. We use the scales including {0.5, 0.75, 1.0, 1.5, 1.75} and mirror for the remaining images to generate pseudo labels. For fair comparison and quick training, the student model is the same model initialized by the teacher model. The main results are shown in Table~\ref{tab: ablation for ist of deeplabv2}.

\begin{table}[ht]
\begin{center}
\resizebox{\linewidth}{!}{
\begin{tabular}{l c c c c c c}
\hlineB{3}
Baseline &ST &SAUG& DSBN &SCL &ITER & mIoU \\
\hline\hline
DeepLabV2     & & & & && 60.6\\
DeepLabV2     &$\checkmark$ & && & & 62.1 \\
DeepLabV2     &$\checkmark$ &$\checkmark$ && & &62.2 \\
DeepLabV2     &$\checkmark$ &$\checkmark$ &$\checkmark$ && & 63.8 \\
DeepLabV2     &$\checkmark$ &$\checkmark$ &$\checkmark$ &$\checkmark$& & 64.5 \\
DeepLabV2     &$\checkmark$ &$\checkmark$ &$\checkmark$ &$\checkmark$ &$\checkmark$& 67.6 \\
DeepLabV2-Full    & & & & && 70.1 \\
\hlineB{3}
\end{tabular}
 }
\end{center}
\caption{Ablation study on the proposed semi-supervised learning framework. Baselines are ResNet101-based Deeplabv2. 'ST' denotes self-training with pseudo labels without strong augmentation. SAUG means the strong augmentation. DSBN means the distribution specify batch normalization. SCL is the proposed self correction loss. ITER represents iterative training. The results are evaluated on the validation set with the single-scale input. Except for those with the full identifier, all experiments used $1/8$ labeled data and the remaining images in the training set are used as unlabeled data. *-Full means using fully supervised.}
\label{tab: ablation for ist of deeplabv2}
\end{table}

We also report detailed performance under different iterations in Figure \ref{fig:iterations for deeplabv2}. We can see that more iterations will help improve the performance, but the growth trend of the performance is gradually flattening.

\begin{figure}
  \centering
    \includegraphics[scale=0.8]{images/paper_iters_deeplabv2.eps}
  \caption{Each iteration performance based on DeepLabV2}
	\label{fig:iterations for deeplabv2}
	\vspace{0.2in}
\end{figure}

\iffalse
\begin{figure*}[ht]
\begin{center}
    \subfigure[original]{\includegraphics[scale=0.2]{images/aug/original.png}}
    \subfigure[blur]{\includegraphics[scale=0.2]{images/aug/blur.png}}
	\subfigure[brightness]{\includegraphics[scale=0.2]{images/aug/brightness.png}}
    \subfigure[channel shuffle]{\includegraphics[scale=0.2]{images/aug/channle_shuffle.png}} \\
	\subfigure[coarse dropout]{\includegraphics[scale=0.2]{images/aug/coarse_dropout.png}}
    \subfigure[contrast gamma]{\includegraphics[scale=0.2]{images/aug/contrast_gamma.png}}
	\subfigure[contrast linear]{\includegraphics[scale=0.2]{images/aug/contrast_linear.png}}
	\subfigure[dropout]{\includegraphics[scale=0.2]{images/aug/dropout.png}} \\
    \subfigure[equalize]{\includegraphics[scale=0.2]{images/aug/equalize.png}}
    \subfigure[flip]{\includegraphics[scale=0.2]{images/aug/flip_v.png}}
	\subfigure[HSV]{\includegraphics[scale=0.2]{images/aug/hsv.png}}
    \subfigure[invert channel]{\includegraphics[scale=0.2]{images/aug/invert_channel.png}} \\
    \subfigure[compression]{\includegraphics[scale=0.2]{images/aug/jpeg_compression.png}}
	\subfigure[multiply]{\includegraphics[scale=0.2]{images/aug/multiply.png}}
    \subfigure[noise gau]{\includegraphics[scale=0.2]{images/aug/noise_gau.png}}
    \subfigure[noise pos]{\includegraphics[scale=0.2]{images/aug/noise_pos.png}} \\
	\subfigure[salt pepper]{\includegraphics[scale=0.2]{images/aug/salt_pepper.png}}
    \subfigure[scale]{\includegraphics[scale=0.2]{images/aug/scale.png}}
    \subfigure[solarized]{\includegraphics[scale=0.2]{images/aug/solarize.png}}
    \subfigure[brightness channel]{\includegraphics[scale=0.2]{images/aug/brightness_channel.png}}
\end{center}
  \caption{Strong augmentation for semantic segmentation}
	\label{fig:strong augmentaion}
	\vspace{0.2in}
\end{figure*}

\renewcommand{\thesection}{S\arabic{section}}
\section{Visualization Strong Augmentation}

Several visualization augmentation images are shown in Figure \ref{fig:strong augmentaion}. From Figure \ref{fig:strong augmentaion}, we find that strong augmentation images are much different from the original image. This may cause the domain gap.

\fi