  Style transfer methods produce a transferred image which is a rendering of a
content image in the manner of a style image. We seek to understand how to
improve style transfer.
  To do so requires quantitative evaluation procedures, but the current
evaluation is qualitative, mostly involving user studies. We describe a novel
quantitative evaluation procedure. Our procedure relies on two statistics: the
Effectiveness (E) statistic measures the extent that a given style has been
transferred to the target, and the Coherence (C) statistic measures the extent
to which the original image's content is preserved. Our statistics are
calibrated to human preference: targets with larger values of E (resp C) will
reliably be preferred by human subjects in comparisons of style (resp.
content).
  We use these statistics to investigate the relative performance of a number
of Neural Style Transfer(NST) methods, revealing several intriguing properties.
Admissible methods lie on a Pareto frontier (i.e. improving E reduces C or vice
versa). Three methods are admissible: Universal style transfer produces very
good C but weak E; modifying the optimization used for Gatys' loss produces a
method with strong E and strong C; and a modified cross-layer method has
slightly better E at strong cost in C. While the histogram loss improves the E
statistics of Gatys' method, it does not make the method admissible.
Surprisingly, style weights have relatively little effect in improving EC
scores, and most variability in the transfer is explained by the style itself
(meaning experimenters can be misguided by selecting styles).

Computer Vision and Pattern Recognition