text:
The Reviewers Gaze Through Blinders
A Critique of Some Reviews of We Are Proud to Present a Presentation About the Herero of Namibia, Formerly Known as Southwest Africa, From the German Südwestafrika, Between the Years 1884-1915, by Jackie Sibblies Drury
By charles c. smith
[The following is an excerpt from charles c. smith's article "The Reviewers Gaze Through Blinders: A Critique of Some Reviews of We Are Proud to Present…". To read the full version of the text in PDF form please click HERE]
With debates swirling around cultural and racial diversity in Canadian Theatre, and having been involved in those debates for a considerable number of years, I was curious when I received a call from playwright and director Ravi Jain inviting me to see the production of We are Proud to Present at the Theatre Centre in Toronto. My curiosity grew when Ravi indicated that Franco Boni and he were inviting me to see this play and to write, not necessarily about the play but about its reviews that appeared in the Toronto press.
What a peculiar request, I thought; and rather risky. For instance, there were only 2 shows left and I might not be available to see either. Also, what if I came away with the same thoughts as the critics? And what if I simply didn't like the play at all? As well, I have to confess that a bit of a chill went down my spine when Ravi mentioned to me that the play was by an African American woman with a particular focus on anti-Black racism. Was this going to be about cultural appropriation – a Brown man focusing on Black peoples? This question was quickly resolved for me when I learned that previous productions of this play had been directed by persons of colour who are not Black and who worked closely with Drury.
In meeting with Ravi and Franco, I soon learned their interest, i.e., they were concerned about the number of reviews where White reviewers appeared to have been limited by a 'gaze' that left them short-sighted when it came to Black and White bodies on stage, in this case in a play about the work-shopping of a script that tells or rather, purposefully fails to tell, the story of what has been called the first genocide of the 20th century where the German colonial power nearly wiped out the Herero of what is now Namibia.
Needless to say, I could not resist and went with my wife (who happens to be German and an invaluable advisor and contributor to my writing) to see the performance. On entering, we were quite amused with the play's beginnings, the seeming informality of being shown to our seats by the cast, the high-school presentation style postings of material about the historical subject matter on the wall, and then the play's artfully clumsy formal start that went on to swing like a pendulum between the seemingly inept lecture about the haunting historical facts, the dynamics of which the heedless 'actors' turn out to be quite unequipped to grasp, and the 'performance' of the play in the play – a matter progressively fraught with contentious rehearsals for a play that never comes to be.
The experience of this particular piece of theatre seemed more than timely, given that I had devoted a lot of thought (and words) to the topic of 'voice and authenticity' in my recent writings on law, education and the arts.1 So, after I saw the play, it was with some wariness that I began to read the local reviews of We Are Proud To Present; especially given several similar scenarios in the world of theatre were currently fuelling an intense debate on racial dynamics on stage and about who could conceivably tell whose story – or who should be permitted to do so.
As I began reading reviews of We Are Proud To Present, I was also aware of other similar scenarios that suggest that the reviews of this play are not alone but, rather sadly, are legion. To me, this suggests a sectoral issue, one that is critical to artists who fall outside of the Eurocentric project. Some context here is important as it places this play and its reviews within a contemporary discourse that is now raging across North America, the continuing fiasco with the Oscars not being an anomaly. For example:
Award-winning actress Tonya Pinkins' rationale for leaving production of Brecht's Mother Courage;2
Ross Jackson's accusing American theatre of three very troubling practices: slotting, tokenism, and dehumanization;3
Assorted discussion of the "Depressing montage of white actors in non-white roles as exposed/highlighted in Master Of None."4
I must add quickly, before being accused of Americanizing Canadian theatre, that these debates swirl across Canadian terrain as well as shown among others in the 2014 research Figuring the Plural, an examination of ethnocultural, or ethnically/culturally specific, arts organizations in Canada and the United States.5
To return to Canadian soil and stage, there are several current examples that relate to:
Canstage and its selection of all-White directors, playwrights, choreographers and translators;6
The challenges issued by Real Theatre Canada in regard to allegations of racism in Vancouver theatre;7
The writing of choreographer Natasha Bakht on how South Asian dance forms have been reviewed;8
The contentions raised by Peter Chin in regard to a review of his new work;9
The writing of George Elliot Clarke about pervasive racism in Canadian Theatre;10
The Equity in Theatre project of the Playwrights' Guild of Canada which looked at gender equity in theatre and the intersections of gender with race and other social identities;11
The attempt by Luminato to bring an exhibit developed by a White South African on human zoos to Toronto;12
And for Quebec, playwright and director Rahul Varma has issued the following charge of carryover racism of the colonial past:
"Quebec is hit by the blackface controversy again. In an article titled La victoire des moustiques published in Véro, actor Louis Morissette attacked Radio-Canada for wanting to hire a black actor to play a black character instead of employing a blackface, i.e. a white actor in degrading make-up: blackened face, afro-wig, puffed-up lips and enlarged hips – for his year-end satire called Bye Bye 2015. Those who criticized his stand on blackface, he mockingly called them mosquitoes. Morissette is not alone; he is a product of the inherent colonial racism so present in the Francophone art world. Boycott him and his shows.13
These are just some of the recent examples that spring to mind and I'm sure I'm leaving much out here, but my intent is simply to give some sense of the veritable vortex of a controversy as context in which the reviews of We Are Proud to Present must be placed. We have to look at this as a sectoral issue and as one that is of critical importance to artists who fall outside of the Western Eurocentric project that still dominates the Canadian (and the North American) stage.
charles c. smith is a published poet, playwright and essayist. He is also the Cultural Liaison in the Dean's Office at the University of Toronto Scarborough where he lectures in the English Department/Creative Writing. (For the full bio please view the PDF below)
To continue to the full version of the text in PDF form please click HERE
1 See: Feminism, Law and Inclusion; Crisis, Conflict and Accountability: Law Enforcement and Racial Profiling in Canada; Anti-Racisim in Education: Missing in Action; Pluralism in the Arts in Canada: A Changes is Gonna Come; The Dirty War: the Making of the Myth of Black Dangerousness published respectively by Canadian Scholars Press (formerly Sumach Press) and the Canadian Centre for Policy Alternatives. Several chapters of my work on racial profiling have also been published in U. of T. Press. In addition, I had also spoken about this in keynotes and lectures on contemporary Black aesthetics at Queen's University, Concordia University and the Big Dream Conference in North Bay. These latter pieces were efforts to discuss heterogeneity and radical Black aesthetics as I have become increasingly concerned about the homogenization of arts' funder categories that lump all racialized peoples into the category they term 'visible minority'.
2 Who Loses, Who Thrives When White Creatives Tell Black Stories, January 01, 2016. See also: The New York Times, Theatre 1/1/2016
3 Blackness in Nonprofit Theater: Where Representation Becomes Marginalization, Equity, Diversity, Inclusion Project with NPQ, January 29, 2016
4 Caroline Siede, January 29, 2016, http://www.avclub.com/article/heres-depressing-montage-white-acto…4?utm_source=facebook&utm_medium+ShareTools&utm_campaign=default
5 Mina Malton et al, Art Institute of Chicago, 2014
6 See Matthew Jocelyn ducks real question on Canadian Stage diversity, Globe and Mail, January 29, 2016, J. Kelly Nestruck
7 See Allegations of systemic racism in Vancouver theatre could spark change, http://globalnews.ca/news/2143702/allegations-of-systemic-racism-in-vancouvers-theatre-scene-could-spark-change/
8 See Mere Song and Dance, in Pluralism in the Arts in Canada: A Change is Gonna Come, edited by charles c smith, Canadian Centre for Policy Alternatives
9 See Is everyone a critic: Changing Paradigms in Dance Criticism, in The Dance Current, Kathleen Smith http://www.thedancecurrent.com/feature/everyone-critic
10 See The Stage is Not White and Neither is Canada, in Pluralism in the Arts in Canada: A Change is Gonna Come, edited by charles c smith, Canadian Centre for Policy Alternatives
11 See http://eit.playwrightsguild.ca/
12 See Social Media Uproar over rumours of 'Exhibit B' Installation coming to Luminato, J. Kelly Nestruck, http://www.theglobeandmail.com/arts/theatre-and-performance/social-media-uproar-over-rumours-of-controversial-exhibit-b-installation-coming-to-luminato/article23867715/, April 9, 2015
← Older: SIDE STREETS: Amanda McCavour & Micah Adams
The Theatre Centre is excited to unveil the latest installation in our Café: Crystal Dream Homes by Amanda McCavour and Micah Adams, as part of SIDE STREETS, …
Newer: Meet our new Manager of Artist & Community Activation →
We've had an exciting start to our week yesterday, as we welcomed a new member to our team here at The Theatre Centre. We're pretty …


Do you love photos of placentas? I certainly do! Monet Moutrie, a birth photographer in Colorado, published a series of 12 placenta photos on the Huffington Post blog. These are The Placenta Blog's Favorite Five: Gorgeous!
Mother in Australia displays children's placenta photos in art exhibit.


text:
\section{Introduction}

\label{sec:introduction}

\begin{figure*}[!htb]
\centering
\includegraphics[width=1.0\textwidth]{figures/teaser_01.jpg}
\caption{\emph{Face swapping and reenactment.} Left: Source face swapped
onto target. Right: Target video used to control the expressions of the face
appearing in the source image. In both cases, our results appear in the
middle. }
\label{fig:teaser}
\end{figure*}

\IEEEPARstart{F}{ace swapping} is the visual transformation of a face from a
source image to a target image, such that, the resulting image seamlessly
replaces the face appearing in the target image as depicted in Fig.~\ref%
{fig:teaser}. \emph{Face reenactment} (aka \emph{face transfer} or \emph{%
puppeteering}) utilizes the facial movements and expression of a control
face in one video to guide the motions and deformations of a second face
appearing in another video or image (Fig.~\ref{fig:teaser}). Both tasks have
attracted significant research attention due to their applications in
entertainment~\cite%
{alexander2009creating,kemelmacher2016transfiguring,wolf2010eye}, privacy~%
\cite{blanz2004exchanging,lin2012face,mosaddegh2014photorealistic},
and training data generation and augmentation, specifically for detection of such methods~\cite{nirkin2020deepfake,masi2020two,guera2018deepfake}.
Most contemporary works proposed methods for either swapping or reenactment, but rarely both, relying on
underlying 3D face representations to transfer the face appearance. Face
shapes were estimated from the input image~\cite%
{Hassner:ICCV13:ViewFace3D,thies2016face2face,suwajanakorn2017synthesizing,masi2017rapid,nirkin2018face} or kept
fixed~\cite{hassner2015effective,nirkin2018face,masi2019face}. The 3D shape was then aligned with the input
images and used as a proxy to transfer the image appearance (swapping) or
controlling the facial expression and viewpoint (reenactment).

Deep networks were also applied to face swapping and reenactment. In
particular, Generative Adversarial Networks (GANs)~\cite%
{goodfellow2014generative} were shown to successfully generate realistic
fake faces images. Conditional GANs (cGANs)~\cite%
{mirza2014conditional,isola2017image,wang2018pix2pixHD} were used to
transform an image depicting real data from one domain to the other and
inspired multiple face reenactment schemes~\cite%
{pumarola2018ganimation,wayne2018reenactgan,sanchez2018triple}. The
DeepFakes project~\cite{DeepFakes} applied cGANs to face swapping in videos,
making swapping widely accessible to nonexperts and receiving substantial
public attention. Such methods are able to generate more realistic face
images by replacing the classical graphics pipeline by utilizing implicit 3D
face representations.

Some methods applied domain separation~in latent feature spaces \cite%
{tian2018cr,natsume2018rsgan,natsume18fsnet}, to decompose the identity
component of a face from the other traits, such as pose and expression. The
identity is encoded as the manifestation of latent feature vectors,
resulting in significant information loss and limiting the quality of the
synthesized images. Subject-specific approaches \cite%
{suwajanakorn2017synthesizing,DeepFakes,wayne2018reenactgan,kim2018deep} are
particularly trained for each subject or pair of subjects to be swapped or
reenacted. Thus, requiring significant training sets per subject, to achieve
reasonable results, limiting their potential usage. A major concern shared
by previous face synthesis schemes, particularly the 3D-based methods, is
that they all require particular care to handle partially occluded faces.

In this work, we improve upon our previous work~\cite{nirkin2019fsgan} in multiple ways. We provide a means for interpolating
between face landmarks without relying on 3D information using a face landmarks transformer network. We improve the inpainting generator by adding symmetry and face landmarks cues. We completely revise the preprocessing pipeline and add a postprocessing step, to reduce the jittering and saturation artifacts of our previous method. Finally, we show additional qualitative and quantitative experiments with a new metric for comparing expressions.

To summarize, our contributions are:
\begin{itemize}
\item A face landmarks transformer network for interpolating between face landmarks without 3D information.
\item Improved inpainting generator that utilizes symmetry and face landmarks cues.
\item A demonstration of an additional use case for the new face reenactment method for pose-only face reenactment.
\item Completely revised preprocessing and an additional postprocessing step for reducing hittering and saturation artifacts.
\item Introduction of a new metric for facial expression comparison.
\item Additional quantitative and qualitative experiments and ablation studies using new metrics.
\end{itemize}

\section{Related work}

\label{sec:related_work}

Methods for manipulating the appearance of face images, particularly for
face swapping and reenactment, were originally proposed to address privacy
concerns~\cite{blanz2004exchanging,lin2012face,mosaddegh2014photorealistic},
though they are increasingly being used for recreational~\cite%
{kemelmacher2016transfiguring} or entertainment tasks (e.g.,~\cite%
{alexander2009creating,wolf2010eye}).

\subsection{3D-based methods}

One of the earliest face swapping methods \cite{blanz2004exchanging}
required manual involvement, where a fully automatic method was
later~proposed \cite{bitouk2008face}. The Face2Face approach transferred
expressions from the source to target faces~\cite{thies2016face2face}, by
fitting a 3D morphable face model (3DMM)~\cite{blanz2002face,blanz2003face}
to both faces. The expression attributes of one face were then applied to
the other, where particular attention was given to the interior mouth
regions. The reenactment method of Suwajanakorn et al.~\cite%
{suwajanakorn2017synthesizing} synthesized the mouth area using a
reconstructed 3D model of (former president) Barack Obama, guided by face
landmarks, and using a similar approach for filling the face interior as in
Face2Face. The expression of frontal faces was manipulated by Averbuch-Elor
et al.~\cite{averbuch2017bringing} by transferring the mouth interior from
source to the target image using 2D wraps and face landmarks.

Finally, Nirkin et al.~\cite{nirkin2018face} proposed a face swapping
method, demonstrating that 3D face shape estimation is unnecessary for
realistic face swaps. Instead, they used a fixed 3D face shape as the proxy.
This approach, similar to ours, proposed a face segmentation method, though
the scheme was not end-to-end trainable and required particular attention to
occluded regions. We show our current results to be superior.

\subsection{GAN-based methods}

\begin{figure*}[!htb]
\centering
\includegraphics[width=1.0\textwidth]{figures/system_05.jpg}
\caption{Overview of the proposed FSGAN approach. (a) The recurrent
reenactment generator, $G_{r}$, and the segmentation generator, $G_{s}$. $G_{r}$
estimates the reenacted face, $F_{r}$, while $G_{s}$ estimates the face and
hair segmentation mask, $S_{t}$, of the target image, $I_{t}$. (b) The
inpainting generator, $G_{c}$, inpaints the missing parts of $\tilde{F}_{r}$
based on $S_{t}$ to estimate the completed reenacted face, $F_{c}$. (c) The
blending generator, $G_{b}$, blends $F_{c}$ and $F_{t}$, using the
segmentation mask, $S_{t}$. }
\label{fig:system}
\end{figure*}

GANs~\cite{goodfellow2014generative} were shown to generate fake images
having the same distribution as the target domain. Although successful in
generating realistic appearances, their training might be unstable, thus
limiting their applicability to generating low-resolution images. Subsequent
methods, however, improved the stability of the training process~\cite%
{mao2017least,arjovsky2017wasserstein}. Karras et al.~\cite%
{karras2017progressive} train GANs using a progressive coarse-to-fine
multiscale scheme. CycleGAN~\cite{zhu2017unpaired} proposed a cycle
consistency loss, allowing to train unsupervised generic transformations
between different domains. A cGAN with $L_{1}$ loss was applied by Isola et
al.~\cite{isola2017image} to derive the pix2pix method, and was shown to
produce realistic image synthesis results for applications such as
transforming edges to faces.

Recently, attention-based upsampling was proposed to improve
generation and dense prediction, to reduce the generation artifacts and
improve the prediction accuracy~\cite{wang2019carafe,lu2019indices}.

\subsection{Facial manipulation using GANs}

GANs were used for high resolution image-to-image translation by pix2pixHD~%
\cite{wang2018pix2pixHD}, that applied a multiscale cGAN architecture, and
a perceptual loss~\cite{johnson2016perceptual} using a pre-trained VGG
network that improved the results. GANimation~\cite{pumarola2018ganimation}
proposed a dual generator cGAN architecture conditioned on emotion action
units that generate an additional attention map. This map was used to
interpolate between the reenacted and original images to preserve the
background. The GANnotation~\cite{sanchez2018triple} deep facial reenactment
scheme was driven by face landmarks, where the images were generated
progressively using a triple consistency loss. The images were first
frontalized using landmarks before further processing.

A hybrid 3D/deep method was proposed by Kim et al.~\cite{kim2018deep}, that
render a reconstructed 3DMM of a specific subject using a classical graphic
pipeline. The rendered image is then processed by a generator network,
trained to map synthetic rendered views of each subject to photo-realistic
images. Finally, feature disentanglement was proposed for face manipulation.
RSGAN~\cite{natsume2018rsgan} disentangles the latent face and hair
representations, while FSNet~\cite{natsume18fsnet} proposed a latent space
which separates the identity and geometric attributes, such as facial pose
and expression.

Zakharov et al.~\cite{zakharov2019few} derived a few-shot face
reenactment approach comprised of two parts: a meta-learning stage where the
networks are trained using the face images of multiple subjects, and a
task-specific phase, where the network is fine-tuned using the images of a
particular person. In both stages, an encoder network learns to extract
identity cues, and a generator CNN learns to map this information alongside
the face landmarks to generate novel views.

Video dubbing~\cite{kim2019neural,thies2019neural} was applied using visual and audio
information, respectively. The first used a recurrent GAN to preserve the
expression style of the target. The second, extracted deep audio features
which were encoded as expression coefficients. To generate the final
rendering, both methods used neural rendering to translate classic
renderings into realistic ones using an additional person-specific
generator.
Another kind of video dubbing was performed by text editing~\cite{thies2019neural,fried2019text,yao2020iterative}, Fried et al.~\cite{fried2019text} by mapping input text to 3DMM pose and expression which was used to recurrently render the lower part of the face, and Yao et al.~\cite{yao2020iterative} suggested an interactive text editing using fast phoneme matching and neural rendering.

Ha et al.~\cite{ha2019marionette} suggested a face reenactment
framework using a few-shot setting with a landmark transformer that adjusts
the shape and blending components using an image attention block. Li et al.~%
\cite{li2019faceshifter} introduced a face swapping method that preserves
the attributes of the source face by integrating them with the generator in
a multi-scale scheme using attention blocks. In addition, occlusions are
handled in an unsupervised way by comparing the reconstructed and target
images.

Recently, Naruniec et al.~\cite{naruniec2020high} proposed a face swapping method at megapixel resolution using an encoder-decoder with contrast and light-preserving blending and landmark stabilization. Zakharov et al.~\cite{zakharov2020fast} introduced a one-shot neural rendering-based face reenactment method that can perform in real-time. They achieve this by first rendering a coarse image and then combining it with a warped texture image that is generated offline.

\section{Face swapping GAN}

\label{sec:FSGAN}

\begin{figure*}[ptb]
\centering
\includegraphics[width=\linewidth]{figures/architecture_02.jpg}
\caption{ \emph{Generator architectures.} (a) The global generator is based
on a residual variant of the U-Net~\protect\cite{ronneberger2015u} CNN,
using a number of bottleneck layers per resolution. We replace the simple
convolutions with bottleneck blocks (in purple), the concatenation with
summation (plus sign), and the deconvolutions with bilinear upsamplnig
following by a convolution. (b) The enhancer utilizes a submodule and a
number of bottleneck layers. The last output block (in blue) is only used in
the enhancer of the finest resolution.}
\label{fig:architecture}
\end{figure*}

We introduce the Face Swapping GAN (FSGAN) whose overview is
illustrated in Fig.~\ref{fig:system}. Let $I_{s}$ and $I_{t}$ be the source and
target face images, respectively, representing the faces $F_{s}\in I_{s}$ and $%
F_{t}\in I_{t}$. We aim to synthesize a new image based on $I_{t}$, such
that $F_{t}$ is seamlessly replaced by $F_{s}$ while retaining the same pose
and expression.

The FSGAN consists of three major components. The first, detailed in Section~%
\ref{subsec:reenactment} and shown in Fig.~\ref{fig:system}(a), consists of
the reenactment generator, $G_{r}$, and the segmentation CNN, $G_{s}$. The
input to $G_{r}$ are the heatmaps encoding the facial landmarks of $F_{t}$,
and generates the reenacted image, ${I}_{r}$, such that $F_{r}$ denotes $%
F_{s} $ having the same pose and expression as $F_{t}$. It also computes $%
S_{r}$; that is, the segmentation mask of $F_{r}$. The $G_{s}$ component
computes the segmentations mask of the face and hair of $F_{t}$.

Given the reenacted image, $I_{r}$, there might be missing face parts, as
demonstrated in Figs.~\ref{fig:system}(a) and \ref{fig:system}(b). Hence, we
apply the face inpainting network, $G_{c}$, detailed in Section~\ref%
{subsec:inpainting} using the segmentation, $S_{t}$, to inpaint the missing
parts. The final phase of the FSGAN, shown in Fig.~\ref{fig:system}(c) and
detailed in Section~\ref{subsec:Blending}, is the blending of the completed
face, $F_{c}$, into the target image, $I_{t}$, to derive the final face swapping
result.

The architecture of our face segmentation network, $G_{s}$, is based on the
U-Net~\cite{ronneberger2015u}, with bilinear interpolation for upsampling,
while all other generators---$G_{r}$, $G_{c}$, and $G_{b}$---are based on
the pix2pixHD architecture~\cite{wang2018pix2pixHD}, utilizing a
coarse-to-fine generator, and a multi-scale discriminator.

For the global generator, we use a U-Net with bottleneck blocks~\cite%
{he2016deep} instead of simple convolutions, and summation instead of
concatenation. The summation allows for smaller channel dimensions than by
using concatenation, while bottleneck blocks improve the convergence of
large generator models \cite{MarioNETte:AAAI2020}. As with the segmentation
network, we use bilinear interpolation for upsampling in both the global
generator and the enhancers. The actual number of layers differs per each
generator, and the architectures are detailed in Section~\ref%
{subsec:architecture}.

Following previous results \cite{wayne2018reenactgan}, training
subject-agnostic face reenactment is non-trivial and might fail when applied
to face images related by large poses. For that, we propose to break-down
large pose changes into smaller reenactment steps and interpolate between
the closest available source images corresponding to the target's pose.
These steps are detailed in the following sections.

\subsection{Detection and tracking}

\label{subsec:Detection-tracking}

Each video is first processed by the dual-shot face detector (DSFD)~%
\cite{li2019dsfd} that is more accurate than SFD~\cite{zhang2017s3fd}, which was used in our previous work. The detections are grouped into sequences by computing
the IoUs of the detections in successive frames, where IoUs \TEXTsymbol{>}
0.75 are grouped together in the same sequence.
The facial expressions were tracked using 2D face landmarks \cite{sun2019deep},
trained on the WLFW dataset, consisting of 98 points per face.
In our previous work, we used 2D and 3D landmarks of 68 points~\cite{bulat2017far}.

Even when using state-of-the-art face detection and landmarks
extraction schemes, the bounding boxes and landmarks point are not sub-pixel
accurate, causing inconsistencies between subsequent frames, resulting in
noticeable jittering, an apparent issue of our previous method. Temporal averaging might help reduce the jittering but
introduces a noticeable lag. For that, we extend the 1\euro\ filter~\cite%
{casiez20121}, based on the fact that humans are more sensitive to jittering added to small
motions, and more sensitive to lags, when there is a large motion.

We estimate the motion following Casiez et al. \cite{casiez20121},
but use it as per-frame weights for a 1D temporal averaging filter. The
larger the motion, the less averaging is applied. For the bounding boxes, we
apply the smoothing to the center and box dimensions separately, and for
the face landmarks we apply the smoothing to each face part separately.

\subsection{Generator architecture}

\label{subsec:architecture}

The generators $G_{r}$, $G_{c}$, and $G_{b}$ follow the pix2pixHD design, as depicted in Fig.~\ref{fig:architecture}.
The global generator, $\RIfM@\expandafter\text@\else\expandafter\mbox\fi{Global}(n_{1},n_{2},n_{3})$,
is defined with respect to the number of bottleneck blocks $n_{1},n_{2},n_{3}$ (shown in purple), corresponding to each of the three
scales. The enhancer, $\RIfM@\expandafter\text@\else\expandafter\mbox\fi{Enhancer}(G,n)$, is defined with respect to its
submodule and $n$ bottleneck blocks. Our reenactment generator and
completion generators are%
\begin{equation}
G_{r}=G_{c}=Enhancer(Global(2,2,3),2),
\end{equation}%
and the blending generator is%
\begin{equation}
G_{b}=Enhancer(Global(1,1,1),1).
\end{equation}%

\subsection{Training losses}

\label{subsec:Training-Losses}

\begin{figure*}[tbh]
\centering
\includegraphics[width=0.9\textwidth]{figures/reenactment_01.jpg}
\caption{\emph{Face reenactment.} A single iteration of our proposed face
reenactment method. The source image is concatenated with the heatmap
corresponding to the first face landmarks, $p_{1}$, and fed to the
reenactment generator, $G_{r}$, producing the first reenactment
iteration result. The next iteration uses the result of the previous
iteration instead of the source image with the next face landmarks.}
\label{fig:reenactment}
\end{figure*}

\subsubsection{Domain specific perceptual loss}

\label{subsec:Perceptual Loss}

The perceptual loss \cite{johnson2016perceptual}, that has been widely used
in recent works in face synthesis~\cite{sanchez2018triple}, outdoor scenes~%
\cite{wang2018pix2pixHD}, and super resolution~\cite{ledig2017photo}, is
applied to capture the fine details of the face texture.~A feature maps of a
pretrained VGG network is used to compare the high frequency details using a
Euclidean distance.
Following others~\cite{zhang2018unreasonable,tej2020enhancing}, who show that the effectiveness of perceptual loss can be improved by fine-tuning on specific domains, we propose to train the VGG-19 CNN \cite{simonyan2014very} used for perceptual loss using multiple face
recognition and attribute classification datasets, instead of a generic
ImageNet-based\ CNN. This allows to fully capture the distinct details
inherent in face images. Let $F_{i}\in
\mathbb{R}
^{C_{i}\times H_{i}\times W_{i}}$ be the feature map of the $i$-th layer of
the VGG-19 network, the perceptual loss is thus given by%
\begin{equation}
\mathcal{L}_{perc}(x,y)=\sum_{i=1}^{n}\frac{1}{C_{i}H_{i}W_{i}}\left\Vert
F_{i}(x)-F_{i}(y)\right\Vert _{1}
\end{equation}
where $C_i$ is the number of channels, and $H_i$ and $W_i$ are the height and width dimensions.

\subsubsection{Reconstruction loss}

\label{subsubsec:Reconstruction-Loss}

While the perceptual loss, as in Section \ref{subsec:Perceptual Loss},
captures the high frequencies well, generators trained using only that loss,
often produce images having inaccurate colors, corresponding to the
erroneous reconstruction of the low-frequency image content. Hence, we also
applied a pixelwise $L_{1}$ loss to the generator%
\begin{equation}
\mathcal{L}_{pixel}(x,y)=\Vert x-y\Vert _{1},
\end{equation}%
and the overall loss is given by
\begin{equation}
\mathcal{L}_{rec}(x,y)=\lambda _{perc}\mathcal{L}_{perc}(x,y)+\lambda
_{pixel}\mathcal{L}_{pixel}(x,y)  \label{equ:total loss}
\end{equation}

The loss in Eq. \ref{equ:total loss} was applied to all image generators.

\subsubsection{Adversarial loss}

\label{subsubsec:Adversarial-Loss}

To further improve the realism of the synthesized images, we applied the
pix2pixHD adversarial objective \cite{wang2018pix2pixHD}, which utilizes a
multi-scale discriminator comprising of multiple discriminators, $D_{1},D_{2},...,D_{n}$,
each operating on a different image resolution. For
a generator, $G$, and a multi-scale discriminator, $D$, our adversarial loss is
\begin{equation}
\mathcal{L}_{adv}(G,D)=\min_{G}\max_{D_{1},\dots D_{n}}\sum_{i=1}^{n}%
\mathcal{L}_{GAN}(G,D_{i}),
\end{equation}%
where
\begin{align}
\mathcal{L}_{GAN}(G,D)=& \mathbb{E}_{(x,y)}[\log D(x,y)]  \notag \\
& +\mathbb{E}_{x}[\log (1-D(x,G(x)))].
\end{align}

\subsection{Face Segmentation}

\label{subsec:segmentation}

The goal of our face segmentation is to distinguish between the
internal part of the face and the background, including the rest of the
head. In practice, we found that adding information regarding the hair
improves the segmentation accuracy. Given an image , $I\in R^{3\times H\times
W}$, we define the face segmentation generator, $G_{s}:R^{3\times
H\times W}\rightarrow R^{3\times H\times W}$. The output
segmentation mask predicts three classes: background, face, and hair.
$G_s$ was trained using the cross-entropy loss.
To improve stability and accuracy, the training images are augmented by
random rotations of
$\left[ \mathbf{-30{{}^\circ},30{{}^\circ}}\right]$, random color jittering (brightness, contrast,
saturation, and hue), random horizontal flip, and random gaussian blur with
a kernel radius of 5 and standard deviation of 1.1.

\subsection{Face reenactment}

\label{subsec:reenactment}

As in~\cite{nirkin2019fsgan}, the purpose of the face reenactment generator, $G_r$, is to align a \textit{source} face according to the pose and expression of a \textit{target} face, without subject-specific training. Similarly to~\cite{nirkin2019fsgan}, significant poses are decomposed into smaller, more manageable steps, in which $G_r$ is applied iteratively, but instead of relying on 3D landmarks, we propose a landmarks transformer network to perform the interpolation between the landmarks in each step, with more details on this in Section~\ref{subsubsec:landmarks_transformer}. A single reenactment
iteration is depicted in Fig~\ref{fig:reenactment}.

Additionally, the newly proposed $G_r$ benefit from improved preprocessing, which allows for extracting higher quality cues from the same training videos, a more accurate landmarks extraction method~\cite{sun2019deep}, and improved heatmap encoding method that is discussed in Section~\ref{subsubsec:landmarks_heatmaps}.

We use an appearance map of the source face to identify and utilize
the nearest available views, as detailed in Section~\ref%
{subsec:FaceViewInterpolation}. Given an image, $I\in R^{3\times H\times W}$,
 and a heatmap representation, $H(p)\in R^{98\times H\times W}$,
 of the facial landmarks, $p\in R^{98\times 2}$, we define the face reenactment generator
\begin{equation}
\mathbf{\ }G_{r}:\left\{ \mathbb{R}^{3\times H\times W},\mathbb{R}^{98\times
H\times W}\right\} \rightarrow R^{3\times H\times W}.
\end{equation}%
Let $p_{s},p_{t}\in R^{98\times 2}$ and $\theta
_{s},\theta _{t}\in R^{3}$, be the 2D landmarks and Euler angles of %
$F_{s}$ and $F_{t}$, respectively. Utilizing $n$
reenactment iterations, we generate $n-1$ intermediate 2D
landmarks, $p_{1},\dots ,p_{n-1}$:
\begin{align}
& p_{i}=T\left( p_{s},\left( 1-\frac{i}{n}\right) \theta _{s}+\frac{i}{n}\theta _{t}\right) \qquad \RIfM@\expandafter\text@\else\expandafter\mbox\fi{for }i=1,\dots ,n-1, \\
& p_{n}=p_{t}.  \notag
\end{align}

where $T$ is the landmark transformer discussed in the next section. The recursive reenactment output of each iteration is thus given by
\begin{align}
& I_{r_{i}}=G_{r}(I_{r_{i-1}};H(p_{i})),1\leq i\leq n \\
& I_{r_{0}}=I_{s}.  \notag
\end{align}

\subsubsection{Landmarks transformer}
\label{subsubsec:landmarks_transformer}

To generate the intermediate face landmarks we need to rotate the source landmarks by the delta angle. In our previous work \cite{nirkin2019fsgan}, 3D landmarks were used with geometric transformations, in this work we rely on 2D landmarks and pose estimation, and perform the transformation using a CNN. The 2D landmarks are
detected as in Sun et al. \cite{sun2019deep}, while the pose is estimated
using landmark free pose estimation method~\cite{ruiz2018fine,chang2017faceposenet}.

Given the landmarks, $p_{s}\in R^{98\times 2}$, in the source image, and the head pose, $\theta_{t}\in R^{3}$, in the target image, we aim to estimate the 2D
landmarks, $p_{t}$, in the target image. Thus, we propose the
landmarks transformer, $T:\left\{ p_{1},\theta _{1}\right\} \rightarrow p_{2}
$. The network is comprised of 12 linear layers, each followed by
batch normalization and ReLU activation layers except for the final layer.
It is trained using sequences of faces, where we randomly select a pair of
face images, $F_{s}$ and $F_{t}$, and minimize the mean
square error between the predicted landmarks, $T(p_{s_{i}},\theta_{t_{i}})$, and the ground truth target landmarks, $p_{t_{i}}$:%
\begin{equation}
\Vert x-y\Vert _{1}
\mathcal{L}(T)=\frac{1}{n}\sum_{i=1}^{n}\Vert T(p_{s_{i}},\theta_{t_{i}})-p_{t_{i}}\Vert_{2}^{2}.
\end{equation}

\subsubsection{Landmarks heatmaps}
\label{subsubsec:landmarks_heatmaps}

\begin{figure*}[t]
\centering
\includegraphics[clip,trim=0mm 3mm 0mm 0mm,
width=.85\textwidth]{figures/view_interpolation_01.jpg}\vspace{-3mm}
\caption{ \emph{Face view interpolation.} (a) Shows an example of an
appearance map of the source subject (Donald Trump). The green dots
represent different views of the source subject, the blue lines represent
the Delaunay Triangulation of those views, and the red X marks the location
of the current target's pose. (b) The interpolated views associated with the
vertices of the selected triangle (represented by the yellow dots). (c) The
reenactment result and the current target image. \protect\vspace{-4mm} }
\label{fig:interpolation}
\end{figure*}

Let $\mathbf{A}\in R^{C\times H\times W}$ be the output
layer of the face landmarks network~\cite{sun2019deep}, where $C$
is number of landmark points.
In our previous work, the face landmarks were encoded by taking the points of maximum activation in each channel in $A$, and were decoded by splatting the points on the heatmap followed by a Guassian blur filter. Both operation are not sub-pixel accurate. To further reduce jittering and improve temporal consistency, we replace those two operations by sub-pixel accurate encoding and decoding.

We filter-out the lower values in $%
\mathbf{A}$ corresponding to non-landmarks by normalizing $\mathbf{%
A}$ to $[0,1]$ and zeroing $\mathbf{A<0.5}$. A
landmark point, $p_{c}$, encoded by the channel, $C$, in $%
\mathbf{A}$, is given by%
\begin{equation}
p_{c}=\sum_{x,y}\mathbf{A}(C,y,x)\cdot (x+0.5,y+0.5),
\end{equation}%
After encoding the face landmarks we perform temporal
smoothing as in Section~\ref{subsec:Detection-tracking}. The heatmap layer
corresponding to $p$ is given by
\begin{equation}
\widetilde{H}(p)_{c,y,x}=1-\frac{1}{\sqrt{2}}\left\Vert
p_{c}-(x+0.5,y+0.5)\right\Vert _{2},
\end{equation}%
and the support of a heatmap point is given by
\begin{equation}
H(p)=\max \left( \frac{\widetilde{H}(p)^{8}-t}{1-t},0\right) ,
\end{equation}%
where $t=0.8$ was used in all our experiments.

\subsubsection{Training}

\label{subsubsec:reenactment_training}

Denote by $I_{s},I_{t}$, the source and target images
denoting the same subject, where both images are randomly selected from the
same video sequence. Indicated by $I_{r_{n}}$, the reenactment result
after $n$ iterations, and $I_{r}$, the reenactment results without
intermediate iterations. $\widetilde{I}$ is image $I$,
but with its background removed. Thus, the loss of the generator, $G_{r}$, is
\begin{align}
\mathcal{L}(G_{r})=& \lambda _{stepwise}\mathcal{L}_{rec}(\widetilde{I}%
_{r_{n}},\widetilde{I}_{t})+\lambda _{rec}\mathcal{L}_{rec}(\widetilde{I}%
_{r},\widetilde{I}_{t})  \notag \\
& +\lambda _{adv}\mathcal{L}_{adv},
\end{align}%
where $L_{rec}$ is given in Eq. \ref{equ:total loss}.

\subsection{Face view interpolation}

\label{subsec:FaceViewInterpolation}

Following the common computer graphics pipeline~\cite{hughes2014computer},
in which textured mesh polygons are projected onto a plane and seamlessly
rendered, we propose a novel scheme for iterative interpolation between face
views. This step is an essential phase of our method, as it enables us to
utilize the entire source video sequence, without training our model on a
particular video frame, thus allowing our approach to be subject agnostic.

Given a set of source subject images, $\left\{ \mathbf{I}_{s_{1}},\dots ,%
\mathbf{I}_{s_{n}}\right\}$, and Euler angles, $\left\{ \mathbf{e}_{1},\dots
,\mathbf{e}_{n}\right\}$, of the corresponding faces, $\left\{ \mathbf{F}%
_{s_{1}},\dots ,\mathbf{F}_{s_{n}}\right\}$, we construct the appearance map
of the source subject illustrated in Fig.~\ref{fig:interpolation}(a). The
appearance map embeds the head poses in a triangulated plane, allowing to
follow continuous paths of head poses.

We start by projecting the Euler angles, $\left\{ \mathbf{e}_{1},\dots ,%
\mathbf{e}_{n}\right\}$, to a plane by dropping the roll angle. Using a k-d
tree data structure~\cite{hughes2014computer}, we prune points in the
angular domain that are too close to each other. We retain points for which
the corresponding Euler angles have a roll angle closer to zero, and remove
motion blurred images. Using the remaining points, $\left\{ x_{1},\dots
,x_{m}\right\}$, and the four boundary points, $y_{i}\in \lbrack
-75,75]\times \lbrack -75,75]$, we build a mesh $M$ in the angular domain
using a Delaunay Triangulation.

For a query Euler angle $e_{t}$, of a face, $F_{t}$, and its corresponding
projected point, $x_{t}$, we find the triangle, $T\in M$, that contains $x_{t}$%
. Let $x_{i_{1}},x_{i_{2}},x_{i_{3}}$ be the vertices of $T$ and $%
I_{s_{i_{1}}},I_{s_{i_{2}}},I_{s_{i_{3}}}$ be the corresponding face views.
We calculate the barycentric coordinates, $\lambda _{1},\lambda _{2},\lambda
_{3}$, of $x_{t}$, with respect to $x_{i_{1}},x_{i_{2}},x_{i_{3}}$. The
interpolation result, $I_{r}$, is then given by

\begin{equation}
I_{r}=\sum_{k=1}^{3}\lambda _{k}G_{r}(I_{s_{i_{k}}};H(\mathbf{p}_{t})),
\end{equation}
where $\mathbf{p}_{t}$ are the 2D landmarks of $F_{t}$. If any vertices of
the triangle are boundary points, we exclude them from the interpolation and
normalize the weights, $\lambda _{i}$, to sum to one.

A face view query is illustrated in Figs.~\ref{fig:interpolation}(b) and~\ref%
{fig:interpolation}(c). To improve the interpolation accuracy, we used a
horizontal flip to fill in views when the appearance map is one-sided with
respect to the yaw dimension, and generate artificial views using $G_{r}$
when the appearance map is too sparse.

\subsection{Face inpainting}

\label{subsec:inpainting}

A common challenge in face swapping is partial occlusions of the input face
image, $F_{s}$. Such parts of $F_{s}$ might be missing and thus cannot be
rendered on the target face, $F_{t}$. Nirkin et al.~\cite{nirkin2018face}
used the segmentation of $F_{s}$ and $F_{t}$ to remove occluded parts and
only rendered (and swapped) the face parts in the intersection of the masks.
Pixels in $F_{t}$ were used to render the facial regions occluded in $%
F_{s} $.

For significant occlusions and when the textures of $F_{s}$ and $F_{t}$
differ notably, the resulting face swap might have noticeable artifacts. To
overcome this, we propose to apply the face inpainting generator, $G_{c}$,
shown in Fig.~\ref{fig:system}(b), to render $F_{s}$ such
that the resulting face rendering, $I_c$, will cover the entire
segmentation mask, $S_{t}$ (of $F_{t}$), thus, resolving any occlusion issues.

This is a particular inpainting task in which the expression of the
occluded face is given, and the inpainted object is symmetric. These
significant cues were not utilized in our previous work \cite%
{nirkin2019fsgan}. To this end, in addition to the reenactment image and
segmentation mask, we also feed the inpainting generator with the target
face landmarks and the horizontal flip of the reenactment image.

Given an image, $I\in R^{3\times H\times W}$, landmarks
points, $p\in R^{98\times 2}$, and a segmentation mask, $S\in
R^{3\times H\times W}$, let $\tilde{I}$ be the input image
with its background removed, $\tilde{I}^{\prime }$ the same image
horizontally flipped, $H(p)\in R^{98\times H\times W}$ the heatmap
representation of $p$, and $\tilde{S}\in
R^{H\times W}$ a binary face mask derived from $S$. We
define the inpainting generator
\begin{equation}
G_{c}:\left\{\tilde{I},\tilde{I}^{\prime },H(p),\tilde{S}\right\} \rightarrow \mathbb{R}^{3 \times H \times W}.
\end{equation}

\subsubsection{Training}

\label{subsubsec:inpainting_training}

Given the reenactment result, $I_{r}$, its corresponding
segmentation, $S_{r}$, the target image with its background removed, %
$\tilde{I}_{t}$, and its corresponding face landmarks, $p_{t}$%
, we augment $S_{r}$ by simulating common face occlusions
due to hair, by randomly removing ellipse-shaped parts, in various sizes and
aspect ratios from the border of $S_{r}$. Let $\tilde{I}_{r}$
be $I_{r}$ with its background removed using the
augmented version of $S_{r}$, and $I_{c}$ the result of
applying $G_{c}$ to $\tilde{I}_{r}$ with the additional
input as described in the previous section. We define the inpainting
generator loss
\begin{equation}
\mathcal{L}(G_{c})=\lambda _{rec}\mathcal{L}_{rec}(I_{c},\tilde{I}%
_{t})+\lambda _{adv}\mathcal{L}_{adv},
\end{equation}%
where $L_{rec}$ and $L_{adv}$ are the
reconstruction and adversarial losses as in Section \ref%
{subsubsec:Reconstruction-Loss} and \ref{subsubsec:Adversarial-Loss},
respectively.

\subsection{Face blending}

\label{subsec:Blending}

The final phase of our face swapping scheme is the blending of the completed
face, $F_{c}$, with the target face, $F_{t}$, as depicted in Fig. \ref%
{fig:system}(c). The blending accounts for different skin tones and lighting
conditions, and is inspired by previous works that applied the Poisson
equation to inpainting \cite{yeh2017semantic} and blending \cite{wu2017gp}.
For that, we propose a novel Poisson blending loss.

Let $I_{t}$ be the target image, $I_{r}^{t}$ the image of the reenacted face
transferred onto the target image, and $S_{t}$ the segmentation mask marking
the transferred pixels. Following Perez et al.~\cite{perez2003poisson} we define the
Poisson optimization%
\begin{multline}
P(I_{t};I_{r}^{t};S_{t}))=\arg \min_{f}\Vert {\Greekmath 0272} f-{\Greekmath 0272} I_{r}^{t}\Vert
_{2}^{2}\newline
\RIfM@\expandafter\text@\else\expandafter\mbox\fi{ }  \label{equ:Poisson} \\
s.t.\RIfM@\expandafter\text@\else\expandafter\mbox\fi{ }f(i,j)=I_{t}(i,j),\RIfM@\expandafter\text@\else\expandafter\mbox\fi{ }\forall \RIfM@\expandafter\text@\else\expandafter\mbox\fi{ }S_{t}(i,j)=0,
\end{multline}%
where ${\Greekmath 0272} \left( \cdot \right) $ is the gradient operator. We combine
the Poisson optimization above with the perceptual loss, thus deriving the
Poisson blending loss $\mathcal{L}(G_{b})$
\begin{equation}
\mathcal{L}(G_{b})=\lambda _{rec}\mathcal{L}%
_{rec}(G_{b}(I_{t};I_{r}^{t};S_{t}),P(I_{t};I_{r}^{t};S_{t}))+\lambda _{adv}%
\mathcal{L}_{adv}.
\end{equation}

\subsection{Subject-specific finetuning}

\label{subsec:finetuning}

The face reenactment generator, $G_r$, pretrained on a
large amount of subjects, can be finetuned to a specific subject using as a
little as 400 iterations and the same training configuration as in Section~%
\ref{sub:training_details}. After finetuning $G_r$, is used to
render a specific subject in any pose and expression. Finetuning allows to
improve the quality of the rendering and reproducing subtle face details
such as the appearance of the teeth. This formulation of our method is
quantitatively explored in the Experiments results section. Unless
explicitly stated, we do not finetune the reenactment generator in our
experiments.

\subsection{Post-processing}

\label{subsec:post_processing}

The raw result of our face swapping pipeline, which was the final result in our previous method, exhibits a slight
saturation relative to the original target image, therefore directly pasting
the generated image into the full target image might result in noticeable
bounding box borders. Hence, we apply soft erosion to the target
segmentation mask, $S_{t}$, resulting in a seamless transition from
"face" to "background". The final cropped face swapping result, $%
I_{b}^{^{\prime }}$, is then given by
\begin{equation*}
I_{b}^{^{\prime }}=I_{b}\cdot S_{t}+I_{t}\cdot (1-S_{t}).
\end{equation*}%
$I_{b}^{^{\prime }}$ is then seamlessly integrated into the full
target image by resizing it to the appropriate size and copying it to the
location of the detected bounding box.

\section{Datasets and training}

\label{sec:datasets_and_processing}

\subsection{Datasets and processing}

We used the video sequences of the IJB-C dataset~\cite{maze2018iarpa} to
train the generator, $G_{r}$, from which we automatically extracted the frames
containing particular subjects. IJB-C contains 11,000 face videos of which we
used 5,500 that were in high definition. Similar to the frame pruning
approach described in Section~\ref{subsec:FaceViewInterpolation}, we prune the
face views that are too close together as well as motion-blurred frames.

We apply the segmentation CNN $G_{s}$, to the frames and prune the frames
for which less than 15\% of the pixels in the face bounding box were
classified as face pixels. Dlib's face verification\footnote{%
Available: \url{http://dlib.net/}} was applied to group frames according to
the depicted subjects' identity, and limit the number of frames per subject
to 100, by choosing the frames whose 2D landmarks have the maximal variance.
In each training iteration, we choose the frames, $I_{s}$ and $I_{t}$, from
two randomly chosen subjects.

The VGG-19 CNNs were trained using perceptual losses on the VGGFace2 dataset~%
\cite{cao2018vggface2} for face recognition and the CelebA~\cite%
{liu2018large} dataset for face attribute classification. The VGGFace2
dataset contains 3.3M images of 9,131 identities, whereas CelebA
contains 202,599 images. The segmentation CNN, $G_{s}$, was trained using
the dataset from~\cite{nirkin2018face}, consisting of ${\sim }10K$ face
images labeled with face segmentations, and the LFW Parts Labels Dataset~%
\cite{kae2013augmenting} with ${\sim }3K$ images labeled for both face and
hair segmentations, from which we removed the neck part using facial
landmarks.

We used additional 1K images and corresponding hair segmentations from the
Figaro dataset~\cite{svanera2016figaro}. Finally, FaceForensics++~\cite%
{roessler2019faceforensics++} provides 1000 videos, from which they
generated 1000 synthetic videos on random pairs using DeepFakes~\cite%
{DeepFakes} and Face2Face~\cite{thies2016face2face}.

\subsection{Training details}

\label{sub:training_details}

The proposed generators were trained from scratch, where the weights were
initialized randomly using a normal distribution. We used the Adam optimizer~%
\cite{kingma2014adam} ($\beta _{1}=0.5,\beta _{2}=0.999$) and a learning
rate of $0.0002$ that was reduced by half every ten epochs. The following
parameters were used for all the generators: $\lambda _{perc}=1,\lambda
_{pixel}=0.1,\lambda _{adv}=0.001,\lambda _{seg}=0.1,\lambda
_{rec}=1,\lambda _{stepwise}=1$, where $\lambda _{reenactment}$ is linearly
increased from 0 to 1 during the training process. All of our networks were
trained using eight NVIDIA Tesla V100 GPUs, where the training of the $G_{s}$
generator required six hours to converge, while the rest of the networks
converged in two days. All our networks except for $G_{s}$, were trained
using a progressive multi-scale approach, starting with a resolution of 128$%
\times $128 up to 256$\times $256. The reenactment inference rate is ${\sim }%
30$fps, and ${\sim }10$fps for face swapping, both using a single reenactment iteration on a single NVIDIA Tesla V100 GPU.

\section{Experimental results}

We performed extensive qualitative and quantitative trials to experimentally
verify the proposed scheme and compare our method to contemporary face
swapping methods: DeepFakes~\cite{DeepFakes} and Nirkin et al.~\cite%
{nirkin2018face}, and the Face2Face reenactment scheme~\cite%
{thies2016face2face}. We conduct all our experiment using FaceForensics++~%
\cite{roessler2019faceforensics++} videos, running our method on the same
pairs they used. Moreover, we provide an ablation study to exemplify the
importance of each component in our pipeline.

\subsection{Quantitative experiments metrics}

\label{subsec:quant_experiments_metrics}

\begin{figure*}[t]
\centering
\includegraphics[width=1.0\linewidth]{figures/reenactment_qual_02.jpg}
\caption{ Qualitative face reenactment results. Row 1: The source face for
reenactment. Row 2: Our reenactment results (without background removal).
Row 3: The target face from which to transfer the pose and expression.
\protect\vspace{-4mm} }
\label{fig:face_reenactment_qualitative}
\end{figure*}

We applied multiple metrics and corresponding quantitative tests to
experimentally evaluate our face swapping scheme in general, and the
identity preservation, in particular, while retaining the same pose and
expression as the target subject. For that, we used the following metrics:

\vspace{2mm}\noindent {\textbf{Identity verification (id)}.} To measure the
identity preservation of our method, we first extract face embeddings using Arcface~\cite{deng2018arcface}, and then compare them using cosine similarity:
\begin{equation}
\RIfM@\expandafter\text@\else\expandafter\mbox\fi{similarity}=\langle \frac{a}{\Vert a \Vert _{2}}, \frac{b}{\Vert b \Vert _{2}} \rangle
\end{equation}%
where $a$ and $b$ are the face embeddings.
This face verification network is a different network than the one used for face-specific perceptual loss, and it was trained on a different dataset.

\vspace{2mm}\noindent {\textbf{Fr\'{e}chet Inception Distance (FID)}.} To
asses the visual quality of our generated images we adopt the FID metric~%
\cite{heusel2017gans} which is more stable than previous quality metrics
such as the structural similarity index method (SSIM) or the Inception score~%
\cite{salimans2016improved}.

\vspace{2mm}\noindent {\textbf{$\mathcal{L}_{1}$ distance}.} A quality
measurement that focuses on the low frequencies of the images, validating
the color preservation in the generated image. For this metric we compare
images with color values in the range $[0,1]$.

\vspace{2mm}\noindent {\textbf{Euler angles}.} To compare the face poses we
calculate the Euclidean distance between the Euler angles of the generated
face and those of the target face, using the method from~\cite{ruiz2018fine}%
. This distance is quantified in degrees.

\vspace{2mm}\noindent {\textbf{Face landmarks}.} Similarly to the Euler
angles metric, we measure the accuracy of the facial expression and shape by
calculating the Euclidean distance between the face landmarks of the
generated face and the target face. This distance is measured in pixels.

\vspace{2mm}\noindent {\textbf{Facial expression comparison (FEC)}. }

To further validate the accuracy of the generated facial expressions, we add the FEC metric. This is an alternative to the expression classification evaluation used for the same purpose by Chang et al.~\cite{chang2018expnet}, that used a limited number of expressions. The FEC metric is directly trained to compare facial expressions and mimics the way humans perceive expressions.

To this end, we train an expression embedding network using the FEC dataset~\cite%
{vemulapalli2019compact}, consisting of 156K images and 500K human-annotated
triplets, such that each triplet contains a pair of expressions that are
mutually more similar to the third expression. Following Vemulapalli et al.~\cite%
{vemulapalli2019compact} we use the triplet loss~\cite{schroff2015facenet}
to train an embedding network to compute a 16-dimensional expression
embedding.

\subsection{Face reenactment results}

\label{sub:face_reenactment_results}

\subsubsection{Qualitative face reenactment results}

\label{subsub:qualitative_face_reenactment_results}

We start by depicting qualitative face reenactment results in Fig.~\ref%
{fig:face_reenactment_qualitative}. We show raw face reenactment results, of
varying ethnicities, poses, and expressions without background removal.
In the left-most column we demonstrate reenacting a face
that differs significantly from the training set, exemplifying that our
method performs well when applied to faces from completely
different domains. The second column from the left demonstrates our method's
ability to cope with rare facial expressions that are not present in the
training set, and extreme iris position differences.

To exemplify the importance of the iterative reenactment method, we illustrate
in Fig.~\ref{fig:reenactment_limitations} the reenactment of the same
subject for both small and large angular differences. For large angular
differences, the identity and texture are better preserved using multiple
iterations than by using a single one.

\subsubsection{Pose only reenactment}

\label{subsub:pose_only_reenactment}

\begin{figure*}[t]
\centering
\includegraphics[width=1.0\linewidth]{figures/pose_reenactment_01.jpg}
\caption{Pose reenactment. (a) Example appearance map of an input
subject (Natalie Portman). The red dashes represent the interpolation path
and the orange Xs mark the pose of the views presented on the right. (b) The
first three rows are input queries from the appearance map. The reenactment
result is displayed in the last row.}
\label{fig:pose_reenactment}
\end{figure*}

Using the reenactment generator, $G_r$, and the landmarks
transformer, $T$, we can reenact a source face without a target
face, using only Euler angles to guide the pose. For that, we apply the face
view interpolation in Section~\ref{subsec:FaceViewInterpolation}, where the
Euler angles were varied smoothly by a user using a mouse or keyboard. For
each Euler angle, the appearance map (as in Section \ref%
{subsec:FaceViewInterpolation}) of the input subject is queried for three
face views, which are then rotated to the same tilt angle as the current
Euler angle. The views are then interpolated the same way as in Section~\ref%
{subsec:FaceViewInterpolation}. Such a reenactment is shown in Fig.~\ref%
{fig:pose_reenactment}.

\subsubsection{Expression-only reenactment comparison}

\label{subsub:expression_only_reenactment}

\begin{figure}[t]
\includegraphics[width=1.0\linewidth]{figures/exp_reenactment_qual_01.jpg}
\caption{\emph{Qualitative expression-only reenactment comparison.}
The expression of the face in the left column is transferred to the face in
the second column using Face2Face~\protect\cite{thies2016face2face} (third
column), FSGAN~\protect\cite{FaceSwap} (4th column), and our method (last
column).} \protect\vspace{-4mm}
\label{fig:face2face}\centering
\end{figure}

We compare our method to the contemporary Face2Face~\cite%
{thies2016face2face} and FSGAN~\cite{nirkin2019fsgan} methods in the
expression-only reenactment problem, and the results are shown in Fig.~\ref%
{fig:face2face} and Fig.~\ref{fig:additional_face2face}. For this problem we use a slightly different formulation:
Given a pair of faces, $F_{s}\in I_{s}$ and $F_{t}\in I_{t},$ we aim to
transfer the expression from $I_{s}$ to $I_{t}$. Hence, we modify the
corresponding 2D landmarks of $F_{t}$ by swapping in the mouth points of the
2D landmarks of $F_{s},$ similar to the generation of the intermediate
landmarks in Section~\ref{subsec:reenactment}. The reenactment result is
then given by $G_{r}(I_{t};H(\hat{p}_{t}))$ where $\hat{p}_{t}$ are the
modified landmarks.

\subsection{Face inpainting ablation results}

\label{subsec:inpainting_results}

\begin{table*}[tbp]
\centering{\
\resizebox{1.0\linewidth}{!}{
 \begin{tabular}{lcccccc}
 \toprule
 Method & FID $\downarrow$ & id $\uparrow$ & $\mathcal{L}_1$ $\downarrow$ & euler $\downarrow$ & landmarks $\downarrow$ & FEC $\downarrow$ \\ [0.5ex]
 \hline
 FSGAN~\cite{nirkin2019fsgan} & 2.64 & 0.36 $\pm$ 0.07 & 0.46 $\pm$ 0.01 & 1.77 $\pm$ 0.92       & 34.8 $\pm$ 15.0 & 0.15 $\pm$ 0.09 \\
 $G_c$ without landmarks      & 0.74 & \textbf{0.64 $\pm$ 0.11} & 0.31 $\pm$ 0.02 & 1.20 $\pm$ 0.75       & 19.4 $\pm$ 7.5  & \textbf{0.11 $\pm$ 0.09} \\
 $G_c$ with landmarks & \textbf{0.62} & \textbf{0.64 $\pm$ 0.10} & \textbf{0.16 $\pm$ 0.01} & {\bf 1.04 $\pm$ 0.52} & {\bf 17.9 $\pm$ 3.8} & 0.12 $\pm$ 0.07 \\
 \bottomrule
 \end{tabular}

}
\caption{\emph{Inpainting ablation.} We compare our previous inpainting results ~\protect\cite{nirkin2019fsgan} with two variations of the proposed face inpainting method, the first (middle row) without using
face landmarks and the seconds (bottom row) with face landmarks alongside a horizontally flipped replica of the source image.}
\label{tab:inpainting_ablation}
\end{table*}

\begin{figure*}[t]
\centering
\includegraphics[width=1.0\linewidth]{figures/inpainting_02.jpg}
\caption{\emph{\ Face inpainting qualitative results.} Top row:
Input real images with random ellipses removed from the faces. Seconds row:
Inpainting results without facial landmarks. Third row: Inpainting results
with facial landmarks. Bottom row: The full input image.}
\label{fig:inpainting_qualitative}
\end{figure*}

To exemplify the effectiveness of our proposed improvements of $G_c$,
we perform an ablation study. In
this study we remove random ellipses from faces in the first 100 frames of
the first 500 videos in FaceForensics++, similar to Section~\ref%
{subsubsec:inpainting_training}, inpaint the face using the different
inpainting methods and compare the inpainting results to the original faces.

Qualitative examples of the ablation study are shown in Fig.~\ref%
{fig:inpainting_qualitative}. It follows that our \textit{new} FSGAN
architecture for $G_{c}$ produces higher quality results than the
\textit{previous} FSGAN\cite{nirkin2019fsgan}. Moreover, the FSGAN
formulation that receives the face landmarks as an additional input better
reconstructs the facial expressions and shapes of the input faces.
Quantitative results are shown in Table~\ref{tab:inpainting_ablation} using
the metrics detailed in Section~\ref{subsec:quant_experiments_metrics}.

As apparent from the table, both new FSGAN formulations significantly outperform the
previous FSGAN \cite{nirkin2019fsgan} in all metrics. Our $G_{c}$
formulation that also receives the face landmarks and the horizontal flip of
the image, outperforms the variant without those additions on all metrics,
but for the id metric, for which both variants achieve similar scores.

\subsection{Face swapping results}

\label{sub:face_swapping_results}

\subsubsection{Qualitative face swapping results}

\label{subsub:qualitative_face_swapping_results}

\begin{figure*}[t]
\centering
\includegraphics[width=1.0\linewidth]{figures/face_swapping_qual_02.jpg}
\caption{ \emph{Qualitative face swapping results on~\protect\cite%
{roessler2019faceforensics++}.} The 3rd and 4th columns shows the
results of the 3DMM-based methods FaceSwap~\protect\cite{FaceSwap} and
Nirkin et al.~\protect\cite{nirkin2018face}, the 5th column shows DeepFakes~%
\protect\cite{DeepFakes} results, a direct mapping method, which was trained
specifically on each of those videos. The 6th column shows FSGAN~%
\protect\cite{nirkin2019fsgan} results and the last column shows the results
of our method, both on images of faces they have not seen before. \protect%
\vspace{-4mm} }
\label{fig:face_swap_qualitative}
\end{figure*}

Qualitative results of our face swapping method applied to the
FaceForensics++ videos \emph{without training our model on any of them }are
shown in Fig.~\ref{fig:face_swap_qualitative}. We purposefully chose
examples showing different poses and expressions, face shapes, and hair
occlusions. Nirkin et al.~\cite{nirkin2018face} is an image-to-image face
swapping method and therefore to be applied to videos it requires a selection strategy for the source frames. Thus, to be fair in our comparison, for each frame in the
target video, we select the source frame having the most similar pose, as face swapping becomes more difficult the larger the pose discrepancy.
To compare FSGAN in a video-to-video scenario such as this, we use our face
view interpolation, as described in Section~\ref%
{subsec:FaceViewInterpolation}. Additional qualitative comparison is presented in Fig.~\ref{fig:additional_comparison}.

\subsubsection{Quantitative face swapping results}

\label{subsubsec:face_swapping_quant}

\begin{table*}[t]
\centering{\
\resizebox{1.0\linewidth}{!}{
 \begin{tabular}{lcccccc}
 \toprule
 Method & FID $\downarrow$ & id $\uparrow$ & $\mathcal{L}_1$ $\downarrow$ & euler $\downarrow$ & landmarks $\downarrow$ & FEC $\downarrow$ \\ [0.5ex]
 \hline
 Nirkin et al.~\cite{nirkin2018face} & 0.67 & 0.45 $\pm$ 0.06 & 0.38 $\pm$ 0.03 & 3.28 $\pm$ 1.34 & 42.1 $\pm$ 9.19 & 0.37 $\pm$ 0.18 \\
 FaceSwap~\cite{FaceSwap} & \textbf{0.37} & 0.38 $\pm$ 0.05 & 0.44 $\pm$ 0.02     & 2.43 $\pm$ 0.85 & 36.9 $\pm$ 7.35 & 0.34 $\pm$ 0.17 \\
 DeepFakes~\cite{DeepFakes} & 0.38 & 0.45 $\pm$ 0.04 & 0.44 $\pm$ 0.02   & 4.05 $\pm$ 1.13 & 53.1 $\pm$ 8.32 & 0.45 $\pm$ 0.14 \\
 FSGAN~\cite{nirkin2019fsgan} & 0.46 & 0.36 $\pm$ 0.06 & 0.25 $\pm$ 0.01 & 2.37 $\pm$ 1.29 & 30.8 $\pm$ 11.2 & 0.34 $\pm$ 0.14 \\
 FaceShifter~\cite{li2019faceshifter} & 0.40 & \textbf{0.60 $\pm$ 0.03} & 0.42 $\pm$ 0.11 & 2.33 $\pm$ 0.8 & 45.51 $\pm$ 11.36 & 0.29 $\pm$ 0.11 \\
 Ours (fine-tuned) & 0.50 & 0.44 $\pm$ 0.04  & 0.22 $\pm$ 0.01          & 2.40 $\pm$ 0.77 & 28.7 $\pm$ 5.76 & 0.30 $\pm$ 0.11 \\
 Ours              & 0.53 & 0.37 $\pm$ 0.05 & \textbf{0.21 $\pm$ 0.0}  & \textbf{2.07 $\pm$ 0.68} & \textbf{21.6 $\pm$ 5.25} & \textbf{0.28 $\pm$ 0.10} \\
 \bottomrule
 \end{tabular}

}
\caption{Quantitative face swapping comparison. We compare to
previous face swapping methods and to a fine-tuned formulation of our method
trained using the FaceForensics++~\protect\cite{roessler2019faceforensics++}.}
\label{tab:face_swapping_quant}
\end{table*}

We also applied quantitative tests to compare our entire face
swapping pipeline with previous face swapping schemes. For that, we used the
same videos as in Section~\ref{subsec:inpainting_results}, and the metrics
detailed in Section~\ref{subsec:quant_experiments_metrics}. We compute the
mean and standard deviation, averaged across the videos, for all the metrics
except for the FID metric that is computed globally. The metrics $L_{1}$,
Euler, landmarks, and FEC, for each frame, are compared to the
corresponding frame in the target video, to test color, shape, pose, and
expression preservation of the target face.
The id metric for each frame, is compared against random source frames to quantify how well the identity is retained. Finally, the FID metric is used to compare the texture distributions of the source and generated faces.

We compare with five previous face swapping methods: Nirkin et al.~%
\cite{nirkin2018face}, FaceSwap~\cite{FaceSwap}, DeepFakes~\cite{DeepFakes}, FSGAN~\cite{nirkin2019fsgan}, and FaceShifter~\cite{li2019faceshifter}. The first column of Table~\ref%
{tab:face_swapping_quant} shows that the FaceSwap approach achieves the best
FID score as it directly warps the texture from the source image. Our method
achieves a slightly better FID score as explained in Section~\ref%
{subsec:limitations}. Our fine-tuned version attains an improved FID score
as it is further trained using the source images.
The second column shows that FaceShifter, a recent state-of--of-the--the-art method, best preserves the identity of the source subject. Our method achieves a lower identity score, yet retains color, pose, and expression significantly better than previous methods, as shown the four rightmost metrics in Table~\ref{tab:face_swapping_quant}. As can be seen from the results, there is a trade-off between how much a method can preserve the source identity and faithfully maintain other important attributes such as: color, pose, and expression.

\subsection{Ablation study on pipeline components}

\begin{figure*}[tbh]
\centering
\includegraphics[width=1.0\linewidth]{figures/ablation_qual_01.jpg}\vspace{%
-2mm} \vspace{-4mm}
\caption{Ablation study on the pipeline components. As shown in columns 3
and 5, without the completion network, $G_{c}$, the transferred face won't
cover the entire target face leaving obvious artifacts. In columns 3 and 4,
we can see that without the blending network, $G_{b}$, the skin color and
lighting conditions of the transferred face are inconsistent with its new
context.}
\label{fig:ablation_qualitative}
\end{figure*}

\begin{table*}[tbh]
\centering{\
\resizebox{1.0\linewidth}{!}{
\begin{tabular}{lcccccc}
\toprule
Method & FID $\downarrow$ & id $\uparrow$ & $\mathcal{L}_1$ $\downarrow$ & euler $\downarrow$ & landmarks $\downarrow$ & FEC $\downarrow$ \\ [0.5ex]
\hline
FSGAN $(G_r)$        & 0.72          & 0.44 $\pm$ 0.06 & 0.26 $\pm$ 0.01 & 2.93 $\pm$ 1.11 & 32.9 $\pm$ 8.46 & 0.36 $\pm$ 0.13 \\
Ours $(G_r)$         & 0.71          & \textbf{0.45} $\pm$ 0.07 & 0.22 $\pm$ 0.01          & 2.93 $\pm$ 0.91 & 31.8 $\pm$ 5.66          & 0.30 $\pm$ 0.11 \\
Ours $(G_r+G_c)$     & 0.67          & 0.41 $\pm$ 0.06 & 0.21 $\pm$ 0.01          & 2.57 $\pm$ 0.83 & 23.2 $\pm$ 4.95          & 0.29 $\pm$ 0.10 \\
Ours $(G_r+G_b)$     & 0.54          & 0.41 $\pm$ 0.06 & \textbf{0.20 $\pm$ 0.01} & 2.40 $\pm$ 0.79 & 28.0 $\pm$ 5.17          & 0.29 $\pm$ 0.11 \\
Ours $(G_r+G_c+G_b)$ & \textbf{0.53} & 0.37 $\pm$ 0.06 & 0.21 $\pm$ 0.0  & \textbf{2.07 $\pm$ 0.68} & \textbf{21.6 $\pm$ 5.25} & \textbf{0.28 $\pm$ 0.10} \\
\bottomrule
\end{tabular}
} }
\caption{Quantitative face swapping results on FaceForensics++~\protect\cite%
{roessler2019faceforensics++} for our ablation study on the components of
FSGAN.}
\label{tab:ablation_quant}
\end{table*}

To demonstrate the necessity of each component in our pipeline, we conducted
an ablation study on four different configurations of our method: $G_{r}$
only, $G_{r}$ with $G_{c}$, $G_{r}$ with $G_{b}$, and the entire pipeline.
The segmentation network, $G_{s}$, is used in all configurations. Qualitative
examples of our ablation study are depicted in Fig.~\ref%
{fig:ablation_qualitative}.

The quantitative results of our ablation study are shown in Table ~\ref%
{tab:ablation_quant}.
Comparing our $G_r$ to our previous reenactment generator (first row), demonstrates an improvement in all metrics, indicating that the landmarks method described in Section~\ref{subsubsec:landmarks_transformer} is superior to the one in FSGAN~\cite{nirkin2019fsgan}.
Our entire pipeline achieves the best FID
score as shown in the first column, indicating that the texture of the
source face is best preserved when none of the steps are omitted. The id
scores (in the second column) show that the source subject identity is
preserved across all pipeline networks. The $L_{1}$ scores (in
the third column) confirms that the appearance of the target face are best
reconstructed when the blending generator, $G_{b}$, is applied,
indicating the importance of the final blending step. Finally, as
demonstrated by the last three columns, the pose and expression of the
target face are best preserved by our entire pipeline.

\section{Conclusions and Future work}

\subsection{Limitations}

\label{subsec:limitations}

\begin{figure}[tbh]
\centering
\includegraphics[width=1.0\linewidth]{figures/limitations_03.jpg}
\caption{ Reenactment limitations. Top left image transformed into each of
the images in Row 1 (using the same subject for clarity). Row 2: Reenactment
with one iteration. Row3: Three iterations. \protect\vspace{-4mm} }
\label{fig:reenactment_limitations}
\end{figure}

Figure~\ref{fig:reenactment_limitations} shows our reenactment results for
different facial yaw angles. Evidently, the larger the angular differences,
the more the identity and texture quality degrade. Moreover, applying too
many iterations of the face reenactment generator also blurs the texture. As
opposed to 3DMM based methods such as Face2Face~\cite{thies2016face2face},
which warp the texture directly from the image, our method is limited to the
image resolution of the training data. Another limitation arises from using
a sparse landmark tracking method that does not fully capture the complexity of the facial expressions.

In the video-to-video scenario, the proposed face view interpolation method assume an homogeneous image quality and facial expressions. If those assumptions do not hold, it might lead to artifacts in the interpolation results. Finally, there is a trade-off between how much our method can preserve the source identity while maintaining the attributes of the target subject such as: skin color, pose, and expression.

\subsection{Discussion}

Our method eliminates laborious, subject-specific data collection and model
training, making face swapping and reenactment accessible to non-experts. We
feel strongly that it is of \emph{paramount importance} to publish such
technologies, to drive the development of technical counter
measures for detecting such forgeries, as well as compel law makers to set
clear policies for addressing their implications. Suppressing the
publication of such methods would not stop their development, but rather
make them available to select a few and potentially blindside policy makers if
it is misused.

\subsection{Future work}

We have identified that the human factor is a major limitation of our
method. It is imposed as we rely on face and face landmarks detection, which
are trained using human labeling. Face landmarks in particular, are an
abstraction originally derived for humans, which filter-out significant
important information that could be otherwise utilized by the CNN. Future
research should avoid using methods that were trained using human labeling.
This would allow to achieve subpixel accurate face tracking and
segmentation, and improve image quality and resolution. Another direction would gain accuracy while reducing runtime by skipping face and landmark detection altogether, instead leveraging recent methods for direct, image to pose, face pose estimation~\cite{albiero2021img2pose} and facial expression deformation modeling~\cite{chang2019deep}. Previous reenactment
methods mainly focused on specific domains such as faces or bodies,
reenactment methods could potentially be generalized to be applied to
multiple domains.

Utilizing only one frame at a time is suboptimal as it does not make use of all the information available for the specific subject. Additionally, temporal information can help improve the tracking of the face pose and expression. Future methods should be able to take all this available information into consideration.

\begin{figure*}[tbp]
\centering
\includegraphics[width=%
\linewidth]{figures/additional_face_swapping_qual_01.jpg}
\caption{ Additional qualitative face swapping comparison to previous
methods on FaceForensics++~\protect\cite{roessler2019faceforensics++}.}
\label{fig:additional_comparison}
\end{figure*}

\begin{figure}[tbp]
\centering
\includegraphics[width=1.0%
\linewidth]{figures/additional_exp_reenactment_qual_01.jpg}
\caption{Additional qualitative expression-only reenactment
comparison on FaceForensics++~\protect\cite{roessler2019faceforensics++}.}
\label{fig:additional_face2face}
\end{figure}

\bibliographystyle{IEEEtran}

\section*{Abstract (Not appropriate in this style!)}%
   \else \small
   \begin{center}{\bf Abstract\vspace{-.5em}\vspace{\z@}}\end{center}%
   \quotation
   \fi
  }%
 }{%
 }%
\@ifundefined{endabstract}{\def\endabstract
  {\if@twocolumn\else\endquotation\fi}}{}%
\@ifundefined{maketitle}{\def\maketitle#1{}}{}%
\@ifundefined{affiliation}{\def\affiliation#1{}}{}%
\@ifundefined{proof}{\def\proof{\noindent{\bfseries Proof. }}}{}%
\@ifundefined{endproof}{\def\endproof{\mbox{\ \rule{.1in}{.1in}}}}{}%
\@ifundefined{newfield}{\def\newfield#1#2{}}{}%
\@ifundefined{chapter}{\def\chapter#1{\par(Chapter head:)#1\par }%
 \newcount\c@chapter}{}%
\@ifundefined{part}{\def\part#1{\par(Part head:)#1\par }}{}%
\@ifundefined{section}{\def\section#1{\par(Section head:)#1\par }}{}%
\@ifundefined{subsection}{\def\subsection#1%
 {\par(Subsection head:)#1\par }}{}%
\@ifundefined{subsubsection}{\def\subsubsection#1%
 {\par(Subsubsection head:)#1\par }}{}%
\@ifundefined{paragraph}{\def\paragraph#1%
 {\par(Subsubsubsection head:)#1\par }}{}%
\@ifundefined{subparagraph}{\def\subparagraph#1%
 {\par(Subsubsubsubsection head:)#1\par }}{}%
\@ifundefined{therefore}{\def\therefore{}}{}%
\@ifundefined{backepsilon}{\def\backepsilon{}}{}%
\@ifundefined{yen}{\def\yen{\hbox{\rm\rlap=Y}}}{}%
\@ifundefined{registered}{%
   \def\registered{\relax\ifmmode{}\r@gistered
                    \else$\m@th\r@gistered$\fi}%
 \def\r@gistered{^{\ooalign
  {\hfil\raise.07ex\hbox{$\scriptstyle\rm\RIfM@\expandafter\text@\else\expandafter\mbox\fi{R}$}\hfil\crcr
  \mathhexbox20D}}}}{}%
\@ifundefined{Eth}{\def\Eth{}}{}%
\@ifundefined{eth}{\def\eth{}}{}%
\@ifundefined{Thorn}{\def\Thorn{}}{}%
\@ifundefined{thorn}{\def\thorn{}}{}%
\def\TEXTsymbol#1{\mbox{$#1$}}%
\@ifundefined{degree}{\def\degree{{}^{\circ}}}{}%
\newdimen\theight
\@ifundefined{Column}{\def\Column{%
 \vadjust{\setbox\z@=\hbox{\scriptsize\quad\quad tcol}%
  \theight=\ht\z@\advance\theight by \dp\z@\advance\theight by \lineskip
  \kern -\theight \vbox to \theight{%
   \rightline{\rlap{\box\z@}}%
   \vss
   }%
  }%
 }}{}%
\@ifundefined{qed}{\def\qed{%
 \ifhmode\unskip\nobreak\fi\ifmmode\ifinner\else\hskip5\p@\fi\fi
 \hbox{\hskip5\p@\vrule width4\p@ height6\p@ depth1.5\p@\hskip\p@}%
 }}{}%
\@ifundefined{cents}{\def\cents{\hbox{\rm\rlap c/}}}{}%
\@ifundefined{tciLaplace}{\def\tciLaplace{\ensuremath{\mathcal{L}}}}{}%
\@ifundefined{tciFourier}{\def\tciFourier{\ensuremath{\mathcal{F}}}}{}%
\@ifundefined{textcurrency}{\def\textcurrency{\hbox{\rm\rlap xo}}}{}%
\@ifundefined{texteuro}{\def\texteuro{\hbox{\rm\rlap C=}}}{}%
\@ifundefined{euro}{\def\euro{\hbox{\rm\rlap C=}}}{}%
\@ifundefined{textfranc}{\def\textfranc{\hbox{\rm\rlap-F}}}{}%
\@ifundefined{textlira}{\def\textlira{\hbox{\rm\rlap L=}}}{}%
\@ifundefined{textpeseta}{\def\textpeseta{\hbox{\rm P\negthinspace s}}}{}%
\@ifundefined{miss}{\def\miss{\hbox{\vrule height2\p@ width 2\p@ depth\z@}}}{}%
\@ifundefined{vvert}{\def\vvert{\Vert}}{
\@ifundefined{tcol}{\def\tcol#1{{\baselineskip=6\p@ \vcenter{#1}} \Column}}{}%
\@ifundefined{dB}{\def\dB{\hbox{{}}}}{
\@ifundefined{mB}{\def\mB#1{\hbox{$#1$}}}{
\@ifundefined{nB}{\def\nB#1{\hbox{#1}}}{
\@ifundefined{note}{\def\note{$^{\dag}}}{}%
\defLaTeX2e{LaTeX2e}
\ifx\fmtnameLaTeX2e
  \DeclareOldFontCommand{\rm}{\normalfont\rmfamily}{\mathrm}
  \DeclareOldFontCommand{\sf}{\normalfont\sffamily}{\mathsf}
  \DeclareOldFontCommand{\tt}{\normalfont\ttfamily}{\mathtt}
  \DeclareOldFontCommand{\bf}{\normalfont\bfseries}{\mathbf}
  \DeclareOldFontCommand{\it}{\normalfont\itshape}{\mathit}
  \DeclareOldFontCommand{\sl}{\normalfont\slshape}{\@nomath\sl}
  \DeclareOldFontCommand{\sc}{\normalfont\scshape}{\@nomath\sc}
\fi

\def\alpha{{\Greekmath 010B}}%
\def\beta{{\Greekmath 010C}}%
\def\gamma{{\Greekmath 010D}}%
\def\delta{{\Greekmath 010E}}%
\def\epsilon{{\Greekmath 010F}}%
\def\zeta{{\Greekmath 0110}}%
\def\eta{{\Greekmath 0111}}%
\def\theta{{\Greekmath 0112}}%
\def\iota{{\Greekmath 0113}}%
\def\kappa{{\Greekmath 0114}}%
\def\lambda{{\Greekmath 0115}}%
\def\mu{{\Greekmath 0116}}%
\def\nu{{\Greekmath 0117}}%
\def\xi{{\Greekmath 0118}}%
\def\pi{{\Greekmath 0119}}%
\def\rho{{\Greekmath 011A}}%
\def\sigma{{\Greekmath 011B}}%
\def\tau{{\Greekmath 011C}}%
\def\upsilon{{\Greekmath 011D}}%
\def\phi{{\Greekmath 011E}}%
\def\chi{{\Greekmath 011F}}%
\def\psi{{\Greekmath 0120}}%
\def\omega{{\Greekmath 0121}}%
\def\varepsilon{{\Greekmath 0122}}%
\def\vartheta{{\Greekmath 0123}}%
\def\varpi{{\Greekmath 0124}}%
\def\varrho{{\Greekmath 0125}}%
\def\varsigma{{\Greekmath 0126}}%
\def\varphi{{\Greekmath 0127}}%

\def{\Greekmath 0272}{{\Greekmath 0272}}
\def\FindBoldGroup{%
   {\setbox0=\hbox{$\mathbf{x\global\edef\theboldgroup{\the\mathgroup}}$}}%
}

\def\Greekmath#1#2#3#4{%
    \if@compatibility
        \ifnum\mathgroup=\symbold
           \mathchoice{\mbox{\boldmath$\displaystyle\mathchar"#1#2#3#4$}}%
                      {\mbox{\boldmath$\textstyle\mathchar"#1#2#3#4$}}%
                      {\mbox{\boldmath$\scriptstyle\mathchar"#1#2#3#4$}}%
                      {\mbox{\boldmath$\scriptscriptstyle\mathchar"#1#2#3#4$}}%
        \else
           \mathchar"#1#2#3#
        \fi
    \else
        \FindBoldGroup
        \ifnum\mathgroup=\theboldgroup
           \mathchoice{\mbox{\boldmath$\displaystyle\mathchar"#1#2#3#4$}}%
                      {\mbox{\boldmath$\textstyle\mathchar"#1#2#3#4$}}%
                      {\mbox{\boldmath$\scriptstyle\mathchar"#1#2#3#4$}}%
                      {\mbox{\boldmath$\scriptscriptstyle\mathchar"#1#2#3#4$}}%
        \else
           \mathchar"#1#2#3#
        \fi
	  \fi}

\newif\ifGreekBold  \GreekBoldfalse
\let\SAVEPBF=\pbf
\def\pbf{\GreekBoldtrue\SAVEPBF}%

\@ifundefined{theorem}{\newtheorem{theorem}{Theorem}}{}
\@ifundefined{lemma}{\newtheorem{lemma}[theorem]{Lemma}}{}
\@ifundefined{corollary}{\newtheorem{corollary}[theorem]{Corollary}}{}
\@ifundefined{conjecture}{\newtheorem{conjecture}[theorem]{Conjecture}}{}
\@ifundefined{proposition}{\newtheorem{proposition}[theorem]{Proposition}}{}
\@ifundefined{axiom}{\newtheorem{axiom}{Axiom}}{}
\@ifundefined{remark}{\newtheorem{remark}{Remark}}{}
\@ifundefined{example}{\newtheorem{example}{Example}}{}
\@ifundefined{exercise}{\newtheorem{exercise}{Exercise}}{}
\@ifundefined{definition}{\newtheorem{definition}{Definition}}{}

\@ifundefined{mathletters}{%

  \newcounter{equationnumber}
  \def\mathletters{%
     \addtocounter{equation}{1}
     \edef\@currentlabel{\arabic{equation}}%
     \setcounter{equationnumber}{\c@equation}
     \setcounter{equation}{0}%
     \edef\arabic{equation}{\@currentlabel\noexpand\alph{equation}}%
  }
  \def\endmathletters{%
     \setcounter{equation}{\value{equationnumber}}%
  }
}{}

\@ifundefined{BibTeX}{%
    \def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
                 T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}}{}%
\@ifundefined{AmS}%
    {\def\AmS{{\protect\usefont{OMS}{cmsy}{m}{n}%
                A\kern-.1667em\lower.5ex\hbox{M}\kern-.125emS}}}{}%
\@ifundefined{AmSTeX}{\def\AmSTeX{\protect\AmS-\protect\TeX\@}}{}%

\def\@@eqncr{\let\@tempa\relax
    \ifcase\@eqcnt \def\@tempa{& & &}\or \def\@tempa{& &}%
      \else \def\@tempa{&}\fi
     \@tempa
     \if@eqnsw
        \iftag@
           \@taggnum
        \else
           \@eqnnum\stepcounter{equation}%
        \fi
     \fi
     \global\@ifnextchar*{\@tagstar}{\@tag}@false
     \global\@eqnswtrue
     \global\@eqcnt\z@\cr}

\def\@ifnextchar*{\@TCItagstar}{\@TCItag}{\@ifnextchar*{\@TCItagstar}{\@TCItag}}
\def\@TCItag#1{%
    \global\@ifnextchar*{\@tagstar}{\@tag}@true
    \global\def\@taggnum{(#1)}%
    \global\def\@currentlabel{#1}}
\def\@TCItagstar*#1{%
    \global\@ifnextchar*{\@tagstar}{\@tag}@true
    \global\def\@taggnum{#1}%
    \global\def\@currentlabel{#1}}
\def\QATOP#1#2{{#1 \atop #2}}%
\def\QTATOP#1#2{{\textstyle {#1 \atop #2}}}%
\def\QDATOP#1#2{{\displaystyle {#1 \atop #2}}}%
\def\QABOVE#1#2#3{{#2 \above#1 #3}}%
\def\QTABOVE#1#2#3{{\textstyle {#2 \above#1 #3}}}%
\def\QDABOVE#1#2#3{{\displaystyle {#2 \above#1 #3}}}%
\def\QOVERD#1#2#3#4{{#3 \overwithdelims#1#2 #4}}%
\def\QTOVERD#1#2#3#4{{\textstyle {#3 \overwithdelims#1#2 #4}}}%
\def\QDOVERD#1#2#3#4{{\displaystyle {#3 \overwithdelims#1#2 #4}}}%
\def\QATOPD#1#2#3#4{{#3 \atopwithdelims#1#2 #4}}%
\def\QTATOPD#1#2#3#4{{\textstyle {#3 \atopwithdelims#1#2 #4}}}%
\def\QDATOPD#1#2#3#4{{\displaystyle {#3 \atopwithdelims#1#2 #4}}}%
\def\QABOVED#1#2#3#4#5{{#4 \abovewithdelims#1#2#3 #5}}%
\def\QTABOVED#1#2#3#4#5{{\textstyle
   {#4 \abovewithdelims#1#2#3 #5}}}%
\def\QDABOVED#1#2#3#4#5{{\displaystyle
   {#4 \abovewithdelims#1#2#3 #5}}}%

\def\tint{\msi@int\textstyle\int}%
\def\tiint{\msi@int\textstyle\iint}%
\def\tiiint{\msi@int\textstyle\iiint}%
\def\tiiiint{\msi@int\textstyle\iiiint}%
\def\tidotsint{\msi@int\textstyle\idotsint}%
\def\toint{\msi@int\textstyle\oint}%

\def\tsum{\mathop{\textstyle \sum }}%
\def\tprod{\mathop{\textstyle \prod }}%
\def\tbigcap{\mathop{\textstyle \bigcap }}%
\def\tbigwedge{\mathop{\textstyle \bigwedge }}%
\def\tbigoplus{\mathop{\textstyle \bigoplus }}%
\def\tbigodot{\mathop{\textstyle \bigodot }}%
\def\tbigsqcup{\mathop{\textstyle \bigsqcup }}%
\def\tcoprod{\mathop{\textstyle \coprod }}%
\def\tbigcup{\mathop{\textstyle \bigcup }}%
\def\tbigvee{\mathop{\textstyle \bigvee }}%
\def\tbigotimes{\mathop{\textstyle \bigotimes }}%
\def\tbiguplus{\mathop{\textstyle \biguplus }}%

\newtoks\temptoksa
\newtoks\temptoksb
\newtoks\temptoksc

\def\msi@int#1#2{%
 \def\@temp{{#1#2\the\temptoksc_{\the\temptoksa}^{\the\temptoksb}}
 \futurelet\@nextcs
 \@int
}

\def\@int{%
   \ifx\@nextcs\limits
      \typeout{Found limits}%
      \temptoksc={\limits}%
	  \let\@next\@intgobble%
   \else\ifx\@nextcs\nolimits
      \typeout{Found nolimits}%
      \temptoksc={\nolimits}%
	  \let\@next\@intgobble%
   \else
      \typeout{Did not find limits or no limits}%
      \temptoksc={}%
      \let\@next\msi@limits%
   \fi\fi
   \@next
}%

\def\@intgobble#1{%
   \typeout{arg is #1}%
   \msi@limits
}

\def\msi@limits{%
   \temptoksa={}%
   \temptoksb={}%
   \@ifnextchar_{\@limitsa}{\@limitsb}%
}

\def\@limitsa_#1{%
   \temptoksa={#1}%
   \@ifnextchar^{\@limitsc}{\@temp}%
}

\def\@limitsb{%
   \@ifnextchar^{\@limitsc}{\@temp}%
}

\def\@limitsc^#1{%
   \temptoksb={#1}%
   \@ifnextchar_{\@limitsd}{\@temp
}

\def\@limitsd_#1{%
   \temptoksa={#1}%
   \@temp
}

\def\dint{\msi@int\displaystyle\int}%
\def\diint{\msi@int\displaystyle\iint}%
\def\diiint{\msi@int\displaystyle\iiint}%
\def\diiiint{\msi@int\displaystyle\iiiint}%
\def\didotsint{\msi@int\displaystyle\idotsint}%
\def\doint{\msi@int\displaystyle\oint}%

\def\dsum{\mathop{\displaystyle \sum }}%
\def\dprod{\mathop{\displaystyle \prod }}%
\def\dbigcap{\mathop{\displaystyle \bigcap }}%
\def\dbigwedge{\mathop{\displaystyle \bigwedge }}%
\def\dbigoplus{\mathop{\displaystyle \bigoplus }}%
\def\dbigodot{\mathop{\displaystyle \bigodot }}%
\def\dbigsqcup{\mathop{\displaystyle \bigsqcup }}%
\def\dcoprod{\mathop{\displaystyle \coprod }}%
\def\dbigcup{\mathop{\displaystyle \bigcup }}%
\def\dbigvee{\mathop{\displaystyle \bigvee }}%
\def\dbigotimes{\mathop{\displaystyle \bigotimes }}%
\def\dbiguplus{\mathop{\displaystyle \biguplus }}%

\if@compatibility\else

  \RequirePackage{amsmath}
\fi

\def\makeatother\endinput{\makeatother\endinput}

\bgroup
\ifx\ds@amstex\relax
   \message{amstex already loaded}\aftergroup\makeatother\endinput
\else
   \@ifpackageloaded{amsmath}%
      {\if@compatibility\message{amsmath already loaded}\fi\aftergroup\makeatother\endinput}
      {}
   \@ifpackageloaded{amstex}%
      {\if@compatibility\message{amstex already loaded}\fi\aftergroup\makeatother\endinput}
      {}
   \@ifpackageloaded{amsgen}%
      {\if@compatibility\message{amsgen already loaded}\fi\aftergroup\makeatother\endinput}
      {}
\fi
\egroup

\typeout{TCILATEX defining AMS-like constructs in LaTeX 2.09 COMPATIBILITY MODE}
\let\DOTSI\relax
\def\RIfM@{\relax\ifmmode}%
\def\FN@{\futurelet\next}%
\newcount\intno@
\def\iint{\DOTSI\intno@\tw@\FN@\ints@}%
\def\iiint{\DOTSI\intno@\thr@@\FN@\ints@}%
\def\iiiint{\DOTSI\intno@4 \FN@\ints@}%
\def\idotsint{\DOTSI\intno@\z@\FN@\ints@}%
\def\ints@{\findlimits@\ints@@}%
\newif\iflimtoken@
\newif\iflimits@
\def\findlimits@{\limtoken@true\ifx\next\limits\limits@true
 \else\ifx\next\nolimits\limits@false\else
 \limtoken@false\ifx\ilimits@\nolimits\limits@false\else
 \ifinner\limits@false\else\limits@true\fi\fi\fi\fi}%
\def\multint@{\int\ifnum\intno@=\z@\intdots@
 \else\intkern@\fi
 \ifnum\intno@>\tw@\int\intkern@\fi
 \ifnum\intno@>\thr@@\int\intkern@\fi
 \int
\def\multintlimits@{\intop\ifnum\intno@=\z@\intdots@\else\intkern@\fi
 \ifnum\intno@>\tw@\intop\intkern@\fi
 \ifnum\intno@>\thr@@\intop\intkern@\fi\intop}%
\def\intic@{%
    \mathchoice{\hskip.5em}{\hskip.4em}{\hskip.4em}{\hskip.4em}}%
\def\negintic@{\mathchoice
 {\hskip-.5em}{\hskip-.4em}{\hskip-.4em}{\hskip-.4em}}%
\def\ints@@{\iflimtoken@
 \def\ints@@@{\iflimits@\negintic@
   \mathop{\intic@\multintlimits@}\limits
  \else\multint@\nolimits\fi
  \eat@
 \else
 \def\ints@@@{\iflimits@\negintic@
  \mathop{\intic@\multintlimits@}\limits\else
  \multint@\nolimits\fi}\fi\ints@@@}%
\def\intkern@{\mathchoice{\!\!\!}{\!\!}{\!\!}{\!\!}}%
\def\plaincdots@{\mathinner{\cdotp\cdotp\cdotp}}%
\def\intdots@{\mathchoice{\plaincdots@}%
 {{\cdotp}\mkern1.5mu{\cdotp}\mkern1.5mu{\cdotp}}%
 {{\cdotp}\mkern1mu{\cdotp}\mkern1mu{\cdotp}}%
 {{\cdotp}\mkern1mu{\cdotp}\mkern1mu{\cdotp}}}%
\def\RIfM@{\relax\protect\ifmmode}
\def\RIfM@\expandafter\text@\else\expandafter\mbox\fi{\RIfM@\expandafter\RIfM@\expandafter\text@\else\expandafter\mbox\fi@\else\expandafter\mbox\fi}
\let\nfss@text\RIfM@\expandafter\text@\else\expandafter\mbox\fi
\def\RIfM@\expandafter\text@\else\expandafter\mbox\fi@#1{\mathchoice
   {\textdef@\displaystyle\f@size{#1}}%
   {\textdef@\textstyle\tf@size{\firstchoice@false #1}}%
   {\textdef@\textstyle\sf@size{\firstchoice@false #1}}%
   {\textdef@\textstyle \ssf@size{\firstchoice@false #1}}%
   \glb@settings}

\def\textdef@#1#2#3{\hbox{{%
                    \everymath{#1}%
                    \let\f@size#2\selectfont
                    #3}}}
\newif\iffirstchoice@
\firstchoice@true
\def\Let@{\relax\iffalse{\fi\let\\=\cr\iffalse}\fi}%
\def\vspace@{\def\vspace##1{\crcr\noalign{\vskip##1\relax}}}%
\def\multilimits@{\bgroup\vspace@\Let@
 \baselineskip\fontdimen10 \scriptfont\tw@
 \advance\baselineskip\fontdimen12 \scriptfont\tw@
 \lineskip\thr@@\fontdimen8 \scriptfont\thr@@
 \lineskiplimit\lineskip
 \vbox\bgroup\ialign\bgroup\hfil$\m@th\scriptstyle{##}$\hfil\crcr}%
\def\Sb{_\multilimits@}%
\def\endSb{\crcr\egroup\egroup\egroup}%
\def\Sp{^\multilimits@}%
\let\endSp\endSb
\newdimen\ex@
\ex@.2326ex
\def\rightarrowfill@#1{$#1\m@th\mathord-\mkern-6mu\cleaders
 \hbox{$#1\mkern-2mu\mathord-\mkern-2mu$}\hfill
 \mkern-6mu\mathord\rightarrow$}%
\def\leftarrowfill@#1{$#1\m@th\mathord\leftarrow\mkern-6mu\cleaders
 \hbox{$#1\mkern-2mu\mathord-\mkern-2mu$}\hfill\mkern-6mu\mathord-$}%
\def\leftrightarrowfill@#1{$#1\m@th\mathord\leftarrow
\mkern-6mu\cleaders
 \hbox{$#1\mkern-2mu\mathord-\mkern-2mu$}\hfill
 \mkern-6mu\mathord\rightarrow$}%
\def\overrightarrow{\mathpalette\overrightarrow@}%
\def\overrightarrow@#1#2{\vbox{\ialign{##\crcr\rightarrowfill@#1\crcr
 \noalign{\kern-\ex@\nointerlineskip}$\m@th\hfil#1#2\hfil$\crcr}}}%
\let\overarrow\overrightarrow
\def\overleftarrow{\mathpalette\overleftarrow@}%
\def\overleftarrow@#1#2{\vbox{\ialign{##\crcr\leftarrowfill@#1\crcr
 \noalign{\kern-\ex@\nointerlineskip}$\m@th\hfil#1#2\hfil$\crcr}}}%
\def\overleftrightarrow{\mathpalette\overleftrightarrow@}%
\def\overleftrightarrow@#1#2{\vbox{\ialign{##\crcr
   \leftrightarrowfill@#1\crcr
 \noalign{\kern-\ex@\nointerlineskip}$\m@th\hfil#1#2\hfil$\crcr}}}%
\def\underrightarrow{\mathpalette\underrightarrow@}%
\def\underrightarrow@#1#2{\vtop{\ialign{##\crcr$\m@th\hfil#1#2\hfil
  $\crcr\noalign{\nointerlineskip}\rightarrowfill@#1\crcr}}}%
\let\underarrow\underrightarrow
\def\underleftarrow{\mathpalette\underleftarrow@}%
\def\underleftarrow@#1#2{\vtop{\ialign{##\crcr$\m@th\hfil#1#2\hfil
  $\crcr\noalign{\nointerlineskip}\leftarrowfill@#1\crcr}}}%
\def\underleftrightarrow{\mathpalette\underleftrightarrow@}%
\def\underleftrightarrow@#1#2{\vtop{\ialign{##\crcr$\m@th
  \hfil#1#2\hfil$\crcr
 \noalign{\nointerlineskip}\leftrightarrowfill@#1\crcr}}}%

\def\qopnamewl@#1{\mathop{\operator@font#1}\nlimits@}
\let\nlimits@\displaylimits
\def\setboxz@h{\setbox\z@\hbox}

\def\varlim@#1#2{\mathop{\vtop{\ialign{##\crcr
 \hfil$#1\m@th\operator@font lim$\hfil\crcr
 \noalign{\nointerlineskip}#2#1\crcr
 \noalign{\nointerlineskip\kern-\ex@}\crcr}}}}

 \def\rightarrowfill@#1{\m@th\setboxz@h{$#1-$}\ht\z@\z@
  $#1\copy\z@\mkern-6mu\cleaders
  \hbox{$#1\mkern-2mu\box\z@\mkern-2mu$}\hfill
  \mkern-6mu\mathord\rightarrow$}
\def\leftarrowfill@#1{\m@th\setboxz@h{$#1-$}\ht\z@\z@
  $#1\mathord\leftarrow\mkern-6mu\cleaders
  \hbox{$#1\mkern-2mu\copy\z@\mkern-2mu$}\hfill
  \mkern-6mu\box\z@$}

\def\qopnamewl@{proj\,lim}{\qopnamewl@{proj\,lim}}
\def\qopnamewl@{inj\,lim}{\qopnamewl@{inj\,lim}}
\def\mathpalette\varlim@\rightarrowfill@{\mathpalette\varlim@\rightarrowfill@}
\def\mathpalette\varlim@\leftarrowfill@{\mathpalette\varlim@\leftarrowfill@}
\def\mathpalette\varliminf@{}{\mathpalette\mathpalette\varliminf@{}@{}}
\def\mathpalette\varliminf@{}@#1{\mathop{\underline{\vrule\@depth.2\ex@\@width\z@
   \hbox{$#1\m@th\operator@font lim$}}}}
\def\mathpalette\varlimsup@{}{\mathpalette\mathpalette\varlimsup@{}@{}}
\def\mathpalette\varlimsup@{}@#1{\mathop{\overline
  {\hbox{$#1\m@th\operator@font lim$}}}}

\def\stackunder#1#2{\mathrel{\mathop{#2}\limits_{#1}}}%
\begingroup \catcode `|=0 \catcode `[= 1
\catcode`]=2 \catcode `\{=12 \catcode `\}=12
\catcode`\\=12
|gdef|@alignverbatim#1\end{align}[#1|end[align]]
|gdef|@salignverbatim#1\end{align*}[#1|end[align*]]

|gdef|@alignatverbatim#1\end{alignat}[#1|end[alignat]]
|gdef|@salignatverbatim#1\end{alignat*}[#1|end[alignat*]]

|gdef|@xalignatverbatim#1\end{xalignat}[#1|end[xalignat]]
|gdef|@sxalignatverbatim#1\end{xalignat*}[#1|end[xalignat*]]

|gdef|@gatherverbatim#1\end{gather}[#1|end[gather]]
|gdef|@sgatherverbatim#1\end{gather*}[#1|end[gather*]]

|gdef|@gatherverbatim#1\end{gather}[#1|end[gather]]
|gdef|@sgatherverbatim#1\end{gather*}[#1|end[gather*]]

|gdef|@multilineverbatim#1\end{multiline}[#1|end[multiline]]
|gdef|@smultilineverbatim#1\end{multiline*}[#1|end[multiline*]]

|gdef|@arraxverbatim#1\end{arrax}[#1|end[arrax]]
|gdef|@sarraxverbatim#1\end{arrax*}[#1|end[arrax*]]

|gdef|@tabulaxverbatim#1\end{tabulax}[#1|end[tabulax]]
|gdef|@stabulaxverbatim#1\end{tabulax*}[#1|end[tabulax*]]

|endgroup

\def\align{\@verbatim \frenchspacing\@vobeyspaces \@alignverbatim
You are using the "align" environment in a style in which it is not defined.}
\let\endalign=\endtrivlist

\@namedef{align*}{\@verbatim\@salignverbatim
You are using the "align*" environment in a style in which it is not defined.}
\expandafter\let\csname endalign*\endcsname =\endtrivlist

\def\alignat{\@verbatim \frenchspacing\@vobeyspaces \@alignatverbatim
You are using the "alignat" environment in a style in which it is not defined.}
\let\endalignat=\endtrivlist

\@namedef{alignat*}{\@verbatim\@salignatverbatim
You are using the "alignat*" environment in a style in which it is not defined.}
\expandafter\let\csname endalignat*\endcsname =\endtrivlist

\def\xalignat{\@verbatim \frenchspacing\@vobeyspaces \@xalignatverbatim
You are using the "xalignat" environment in a style in which it is not defined.}
\let\endxalignat=\endtrivlist

\@namedef{xalignat*}{\@verbatim\@sxalignatverbatim
You are using the "xalignat*" environment in a style in which it is not defined.}
\expandafter\let\csname endxalignat*\endcsname =\endtrivlist

\def\gather{\@verbatim \frenchspacing\@vobeyspaces \@gatherverbatim
You are using the "gather" environment in a style in which it is not defined.}
\let\endgather=\endtrivlist

\@namedef{gather*}{\@verbatim\@sgatherverbatim
You are using the "gather*" environment in a style in which it is not defined.}
\expandafter\let\csname endgather*\endcsname =\endtrivlist

\def\multiline{\@verbatim \frenchspacing\@vobeyspaces \@multilineverbatim
You are using the "multiline" environment in a style in which it is not defined.}
\let\endmultiline=\endtrivlist

\@namedef{multiline*}{\@verbatim\@smultilineverbatim
You are using the "multiline*" environment in a style in which it is not defined.}
\expandafter\let\csname endmultiline*\endcsname =\endtrivlist

\def\arrax{\@verbatim \frenchspacing\@vobeyspaces \@arraxverbatim
You are using a type of "array" construct that is only allowed in AmS-LaTeX.}
\let\endarrax=\endtrivlist

\def\tabulax{\@verbatim \frenchspacing\@vobeyspaces \@tabulaxverbatim
You are using a type of "tabular" construct that is only allowed in AmS-LaTeX.}
\let\endtabulax=\endtrivlist

\@namedef{arrax*}{\@verbatim\@sarraxverbatim
You are using a type of "array*" construct that is only allowed in AmS-LaTeX.}
\expandafter\let\csname endarrax*\endcsname =\endtrivlist

\@namedef{tabulax*}{\@verbatim\@stabulaxverbatim
You are using a type of "tabular*" construct that is only allowed in AmS-LaTeX.}
\expandafter\let\csname endtabulax*\endcsname =\endtrivlist

 \def\endequation{%
     \ifmmode\ifinner
      \iftag@
        \addtocounter{equation}{-1}
        $\hfil
           \displaywidth\linewidth\@taggnum\egroup \endtrivlist
        \global\@ifnextchar*{\@tagstar}{\@tag}@false
        \global\@ignoretrue
      \else
        $\hfil
           \displaywidth\linewidth\@eqnnum\egroup \endtrivlist
        \global\@ifnextchar*{\@tagstar}{\@tag}@false
        \global\@ignoretrue
      \fi
     \else
      \iftag@
        \addtocounter{equation}{-1}
        \eqno \hbox{\@taggnum}
        \global\@ifnextchar*{\@tagstar}{\@tag}@false%
        $$\global\@ignoretrue
      \else
        \eqno \hbox{\@eqnnum
        $$\global\@ignoretrue
      \fi
     \fi\fi
 }

 \newif\iftag@ \@ifnextchar*{\@tagstar}{\@tag}@false

 \def\@ifnextchar*{\@TCItagstar}{\@TCItag}{\@ifnextchar*{\@TCItagstar}{\@TCItag}}
 \def\@TCItag#1{%
     \global\@ifnextchar*{\@tagstar}{\@tag}@true
     \global\def\@taggnum{(#1)}%
     \global\def\@currentlabel{#1}}
 \def\@TCItagstar*#1{%
     \global\@ifnextchar*{\@tagstar}{\@tag}@true
     \global\def\@taggnum{#1}%
     \global\def\@currentlabel{#1}}

  \@ifundefined{tag}{
     \def\@ifnextchar*{\@tagstar}{\@tag}{\@ifnextchar*{\@tagstar}{\@tag}}
     \def\@tag#1{%
         \global\@ifnextchar*{\@tagstar}{\@tag}@true
         \global\def\@taggnum{(#1)}}
     \def\@tagstar*#1{%
         \global\@ifnextchar*{\@tagstar}{\@tag}@true
         \global\def\@taggnum{#1}}
  }{}

\def\tfrac#1#2{{\textstyle {#1 \over #2}}}%
\def\dfrac#1#2{{\displaystyle {#1 \over #2}}}%
\def\binom#1#2{{#1 \choose #2}}%
\def\tbinom#1#2{{\textstyle {#1 \choose #2}}}%
\def\dbinom#1#2{{\displaystyle {#1 \choose #2}}}%

\makeatother
\endinput

redpajama_set_name: RedPajamaArXiv


You can stay for up to six months as a tourist, although you're not allowed to legally take any sort of paid job. You can certainly look for one, though.
You can take a points-based test to determine your eligibility; points are granted based on your educational experience, literacy, job field, language ability, age, and adaptability. The minimum number of points needed to qualify for immigration is currently 67.
Note if you are headed to Quebec, you'll also have some extra paperwork and hurdles. Check the Quebec Immigration website for the latest.
Perhaps the easiest method of moving north is to take a position with a Canadian company which will normally sponsor you and fill out the necessary paperwork. Basically, they have to vouch that there no other Canadian citizen who is as well-qualified to take the open position as you.
Once you immigrate to Canada, you will be given permanent residency (PR) status, which enables you to apply for Canadian social benefits (but does not make you a citizen). You need to renew your PR card every five years, unless you choose to become a Canadian Citizen, which you may do after four years.


text:
Salvatore es el disco debut de Barbazul, editado en 2003.

Músicos

 Ricardo Dimaría - Bajista, cantante
 Javier Romero - Guitarrista
 Matías Viñolo - Baterista

Canciones

 Muller (el rengo)
 Como el perdón
 Pan árabe
 Mr. Ruff
 Otra vez Aquiles
 Cuenta en Berna
 Kursk
 ¿Quién ríe acá?
 Nueva chance
 Radio París
 Turka

Enlaces externos
 Página oficial de Barbazul

Álbumes de 2003
Álbumes de Barbazul
Álbumes de rock en español
Álbumes de rock de Argentina

redpajama_set_name: RedPajamaWikipedia