text:
\section{Introduction}
\label{intro}
Atom interferometry is one of the most promising candidates for ultra-accurate measurements of gravito-inertial forces \cite{dubetsky:2007}, with both fundamental \cite{dimopoulos:2007,dimopoulos:042003,Ertmer:2008,Wolf:2008} and practical (navigation or geodesy) applications. Atom interferometry is most often performed by applying successive \emph{coherent} beam-\-splitting and -\-recombining processes separated by an interrogation time $T$ to a set of particles \cite{borde:1989}. Understanding matter waves interferences phenomena follows from the analogy with optical interferometry \cite{Storey:1994,Borde:2001}: the incoming wave is separated into two wavepackets by a first beam-splitter; each wave then propagates during a time $T$ along a different path and accumulates a different phase; the two wavepackets are finally recombined by a last beam-splitter. To observe the interferences, one measures the two output-channels complementary probability amplitudes which are sine functions of the accumulated phase difference $\Delta \phi$. This phase difference increases with the paths length, i.e. with the time $T$ between the beam-splitting pulses.

\label{sec:1}

\begin{figure}
\begin{center}
\resizebox{0.96\columnwidth}{!}{%
\includegraphics{Fig1}}
\end{center}
\setlength{\unitlength}{\linewidth}

\caption{
Top: The atom interferometer assembled in the Airbus. The main
rack on the left houses the laser sources and the control electronics. The rack
on the front right contains the uninterruptable power-supply, the electrical panel and the high-power laser part. The rack on the back right hosts atom-optics part of the
experiment. Bottom: the architecture of the atom interferometer.
}
\label{fig:photos}
\end{figure}

When used as inertial sensors \cite{Kasevich:1991/2,Gustavson:1997},  the atoms are usually left free to evolve during the interrogation time $T$ so that the interferometer is only sensitive to gravito-inertial effects. In particular, one avoids residual trapping fields that would induce inhomogeneities or fluctuations and would affect the atomic signal. The interrogation time $T$ is consequently limited by, on the one hand, the free expansion of the atomic cloud, and, on the other  hand, the free fall of the atomic cloud. The limitation of expansion is alleviated by the use of ultracold gases \cite{Leanhardt:2003,lecoq:2006}, but, due to free-fall distance, long-interrogation-time experiments require tall vacuum chambers \cite{Clairon:1991}. Laboratory experiments are typically limited to about 300 ms of free fall with a 1 m-tall apparatus if the atoms are simply released, or more by launching them upward as in atomic fountains. This can be increased on much larger apparatuses : a 10 m-high atom interferometer is currently under construction at Stanford \cite{dimopoulos:042003}, giving access to 1.4 seconds of interrogation time. Free-fall heights of more than 100 m, corresponding to durations of about 5 seconds are also available in a drop tower (ZARM Bremen, Germany  \cite{vogel:2006}). Another solution consists in  performing the experiments in microgravity achieved during parabolic flights provided by an aeroplane, as for the PHARAO prototype \cite{laurent:1998}.  In the I.C.E. (\textit{Interf\'erometrie Coh\'erente pour l'Espace}) collaboration \cite{nyman:2006,varoquaux:2007} that we present here, we are conducting cold-atom interferometry experiments in such an airplane (the A-300 0-G Airbus of NOVESPACE), which carries out ballistic flights. Microgravity is obtained via 20 seconds-long parabolas by steering the plane to cancel drag and follow free fall. The residual acceleration is on the order of $10^{-2}$g. With 90 parabolas per flight session, we have access to 30 minutes total of reduced gravity. In this letter, we present a first validation of our 0-g setup by obtaining Ramsey fringes with copropagating Raman transitions during parabolas, the interrogation times being longer than those we could obtain on Earth with the same configuration.

Transferring a laboratory-bound cold atoms interferometer into an automated experiment suitable for microgravity use poses many technical challenges \cite{vogel:2006,nyman:2006,KAnemann:2007hl,yu:2004}. We assembled a prototype atomic source suitable for inertial-sensing in an airplane from the I.C.E. collaboration components \cite{nyman:2006} (see Fig. \ref{fig:photos}). The atom interferometer is made of 4 elements: a vacuum chamber with optics; lasers sources for cooling and coherent manipulation of atoms; a stable oscillator (in our case a hyperfrequency source at about 6.8 GHz \cite{nyman:2006}) which is a frequency reference for the Raman lasers; and an autonomous real-time controller for the experimental sequence and data calculations.  For the interferometric measurement, we prepare clouds of cold $^{87}$Rb in a Magneto-Optical Trap (MOT) and release them for interrogation during their free fall.

Moving away from extended-cavity-laser-diode-based systems, as developed in the PHARAO project \cite{laurent:2006}, we have designed
laser sources at $780$ nm for cooling and coherently manipulating the atoms that rely on telecom technologies and second harmonic generation \cite{Bruner:1998,Thompson:2003}. This allows to use fiber-optics components and offers a reliable, robust and compact system, quite insensitive to the environmental perturbations encountered in the airplane. These novel laser sources are very similar to the ones described in
details in \cite{lienhart:2007} so we limit here to outlining the successful design. A first reference DFB 1560 nm pigtailed laser diode (linewidth $\sim$ 1 MHz) is frequency doubled in a PPLN waveguide and locked on a $^{85}$Rb transition through a saturated absorption setup (see Fig. \ref{fig:laser}). A slave DFB 1560 nm pigtailed diode, similar to the first one, is  locked to the first laser at a frequency difference monitored through the beat note signal, as measured by a fibered fast photodiode. The frequency offset can be adjusted so that the slave DFB is red detuned from the resonance of the $F=2\rightarrow F=3$ transition of $^{87}$Rb with a detuning ranging from 0 to 1.1 GHz. A 1560 nm fibered phase modulator is then used to generate two sidebands $\sim$ 6.8 GHz apart. One of these sidebands acts either as the repumping laser during the cooling phase, or as the second Raman laser during coherent manipulation of the atoms, depending on the applied frequency.

The microwave reference has been simplified compared to \cite{nyman:2006} in order to make it more reliable in the plane environment. It's based on a direct multiplication of a 10 MHz quartz oscillator to 6.8 GHz without any intermediate oscillator or phase lock loop. The ultra-stable quartz has been chosen to be a good compromise to achieve low phase noise at low and high frequencies simultaneously (see Fig. \ref{fig:spectre}), as in \cite{laurent:1998}. The multiplication is done in three steps: a first multiplication by 10 to 100 MHz (commercial Wenzel system), then multiplication by 2 and finally to 6.8 GHz by a comb generator (non-linear transmission line, Wenzel model 7100). Two direct digital synthesis (DDS) are used to adjust the cooling/repumping frequency difference and the Raman beams frequency difference respectively.

\label{sec:2}

\begin{figure}
\begin{center}
\resizebox{0.96\columnwidth}{!}{%
\includegraphics{Fig2}}
\end{center}
\setlength{\unitlength}{\linewidth}

\caption{Spectral density of the phase noise of the quartz recorded by comparison with other ultra-stable quartz oscillators phase-locked on H-Maser of the SYRTE.}
\label{fig:spectre}
\end{figure}

After amplification through a 5W Erbium-Doped Fiber Amplifier (EDFA), the slave laser is frequency doubled in free space with a double-pass in a 4 cm bulk PPLN crystal. We typically obtain $\sim$ 300 mW at 780 nm. A 80 MHz Acousto-Optical Modulator (AOM) is used to switch between the MOT configuration (in which the non diffracted order of the AOM is used) and the Raman configuration (in which the first diffraction order is used, see Fig. \ref{fig:laser}). The use of the first order of the AOM for the Raman beam enables to create ultra-short pulses of light (10 $\mu$s typically). Additional mechanical shutters ensure a total extinction of the beams. Two optical fibers finally bring the MOT and the Raman beams to the science chamber.

\begin{figure}[b]
\begin{center}
\resizebox{0.96\columnwidth}{!}{
\includegraphics{Fig3.eps}
\end{center}
\caption{
Diagram of the laser system. The master laser (linewidth of $1$ MHz) is
a monolithic semiconductor element: a $1560$ nm Distributed
Feed-Back (DFB) fiber laser is frequency doubled in a PPLN waveguide; the resulting
$780$ nm light is then sent into a saturated-absorption
spectroscopy setup for frequency locking on a $^{85}$Rb transition; the slave is a $80$ mW DFB laser diode at $1560$ nm and is frequency-locked
on the master laser by measuring the frequency of their beat-note recorded on a fibered fast photodiode. Frequency control of the lasers is achieved via feedback to their supply current. After amplification through a 5W-Erbium-Doped Fiber
Amplifier (EDFA), the slave laser is frequency doubled in free space with a bulk PPLN crystal; we obtain about
0.3 W at $780$ nm.}
\label{fig:laser}
\end{figure}
The fibers deliver the light to the vacuum-chamber module \cite{varoquaux:2007}. The MOT fiber is sent to a 1-to-3 fiber beam-splitter \footnote{From Sch\"after und Kirchhoff : \texttt{http://www.sukhamburg.de/}}
which delivers three beams which are then retroreflected and produce the MOT. The circularly-polarized Raman beam has a 1 inch diameter  and is aligned with the horizontal plane. A 300 mG horizontal magnetic field is aligned with the Raman beam to raise the Zeeman degeneracy  of the hyperfine sub-levels. The intensity of the lasers can be up to 20 times the saturation intensity of rubidium, which
allows for short Raman pulses with weak velocity selection. The Raman detuning is about 700 MHz. The effective Rabi pulsation $\Omega_{\rm eff}$ is about $2\pi\times12.5$ kHz. Finally, a magnetic shield around the science chamber prevents from changes of the Earth's magnetic field directions during parabolas \cite{varoquaux:2007}.

\begin{figure}
\begin{center}
\resizebox{0.96\columnwidth}{!}{%
\includegraphics{Fig4}}
\end{center}
\setlength{\unitlength}{\linewidth}
\caption{
Up: inside the atom-optics rack (the vacuum chamber and the free-space
optics). Down : 3D schematics showing the beams configuration and the interferometer sequence.
a) Atom cooling and trapping (MOT, the horizontal retroreflected MOT beam is not shown for clarity), b) First Raman interrogation pulse colinear to the horizontal retroreflected MOT beam, c) second Raman interrogation pulse and d) detection with the MOT beams. The atomic cloud is represented as falling under gravity.}
\label{fig:schematics}
\end{figure}

The science chamber in which we operate our atom interferometer is shown in Fig. \ref{fig:schematics}. We load about 10$^9$ atoms in the MOT from a Rubidium vapor in 500 ms. We release the atoms from the MOT and further cool them down below 100 $\mu$K during a brief phase of optical molasses. Then, we prepare the atoms in the lower hyperfine state $F=1$ using optical pumping. After the extinction of the MOT beams, we shine the atoms with two Raman light pulses separated by a time $T$. The duration $\tau$ of these pulses is chosen such that $\Omega_{\rm eff}\times\tau=\frac{\pi}{2}$ (splitting of the matter wave). The Raman lasers are copropagating so that a nearly zero momentum is transfered during the Raman transition. In this configuration, the two successive $\frac{\pi}{2}$ pulses enable to record optically induced Ramsey fringes that are the signature of the matter waves interferences between the two interferometer paths \cite{Antoine:2003/2,Borde:2002}. After the Raman pulses, the MOT beams are switched on at resonance and a photodiode monitors the fluorescence which is proportional to the number of atoms. During a few milliseconds, the microwave source is first turned off (absence of the repumping laser) to record the number $N_2$ of atoms in $F=2$. Second, we switch on the microwave source (presence of the repumping sideband) and the photodiode detects the total number of atoms  $N$. Plotting the ratio $N_2/N$ with respect to the frequency of the Raman transitions sideband, we thus obtain Ramsey fringes, corresponding to the proportion of atoms having undergone coherent transfer between the two states $|F=1, m_F=0\rangle$ and $|F=2, m_F=0\rangle$ (see Fig. \ref{fig:fringe}).

\begin{figure*}
\begin{center}
\resizebox{1.9\columnwidth}{!}{%
\includegraphics{Fig5.eps}}
\end{center}
\caption{Comparison of fringes with or without gravity.
In the presence of gravity (upper curve and pictures), the atoms, when released from the trap
fall down, and eventually exit the Raman pulse region and the recapture volume. The atomic signal then drops and no fringes are visible. In the absence of gravity (lower curve and pictures), even for longer times, the atoms stay in the
Raman pulse region and recapture area. The detection limit is eventually set by the temperature induced expansion of the atomic cloud. }
\label{fig:fringe}
\end{figure*}

The total duration of the sequence is about $T+10$ ms from the end of the molasses phase until the detection. In the lab, i.e. in a 1-g environment, we can typically detect atoms until $T=20$ ms. Above this limit, the free fall of the atoms is too important and the atoms exit both the area of the Raman beams and the detection volume, as shown in the schematics of figure \ref{fig:fringe}. However, $T$ can be much longer during parabolas where the residual acceleration is on the order of $10^{-2}$ g.  Figure \ref{fig:fringe} clearly illustrates the advantage of such a reduced gravity environment: with $T=40$ ms, we have recorded fringes which could not be observed on ground; we could operate at a largest pulse time interval of $T=75$ ms (see Fig. \ref{fig:fringelong}) which represents a fringe period of $1/T\sim$ 14 Hz. We are prone to believe that the main limitation for longer values of $T$ is due to the cloud temperature: the spatial extension of the cloud increases, and the Rabi frequency is then not the same for all the atoms, depending on the laser intensity at their position. It can explain the decrease of the fringes amplitude when T is longer. This effect is enhanced by the residual acceleration -  $4\times10^{-2}$g during about 100 ms leads to motion amplitudes of 2 mm which is enough to reduce the Raman beams efficiency. Atoms also exit the detection area, and this makes the signal-to-noise ratio drop.

The vibration noise in the plane do not enable us to use the Raman beam in a velocity selective configuration yet \cite{Kasevich:1991}, and thus we could not render this interferometer sensitive to inertial effects. Consequently, the residual acceleration noise has been measured with accelerometers locally anchored to the experimental apparatus. This noise corresponds to large residual accelerations ($\sim$ 10 cm/s$^2$) that will doppler shift the resonance and thus hinder it. Different techniques can be used to reduce the influence of these spurious accelerations: active stabilization of the retroreflecting mirror \cite{Hensley:1999}, post-corrections or feed-forward from an accelerometer signal on the Raman phase \cite{LeGouet:2008},  or combination of vibrations measurements by a seismometer and the measured transition probabilities \cite{Merlet:2009}. With the use of an appropriate vibration isolation, it will allow interrogation of the freely-falling atoms during several  seconds, and reach high-precision not yet achieved with ground based atom inertial sensors. When implemented in space, with a residual noise lower than $10^{-6}$ g, the sensibility can reach that of the best spaced-based accelerometers\cite{Odyssey}.

To conclude, we have successfully tested a cold atom light pulse interferometer in
aircraft parabolic flights. Our preliminary results show that laboratory experiments can be adapted for this new
experimental platform and used to develop the future generation of air/spaceborn atom inertial sensors. Our experimental set-up offers an unprecedented platform for development of future fundamental physics instruments to test General Relativity of gravitation.
Unlike orbital platforms, development cycles on ground-based facilities (either in a plane or in a drop tower) can be  short enough to offer rapid technological evolution for these future sensors.  In the future, high-precision drag-free space-born applications will require further progress to achieve longer interrogation times using ultra-low velocity atoms. New-generation of degenerate atomic source design are currently under study for that purpose \cite{vogel:2006,nyman:2006}.
\begin{figure}
\begin{center}
\resizebox{.8\columnwidth}{!}{%
\includegraphics{Fig6.eps}}
\end{center}
\caption{
Optical Ramsey fringes obtained in micro gravity for $T=50$ ms and $T=75$ ms, (corresponding to respectively 18 mm and 35 mm of free fall under gravity). The x-axis is the frequency of the Raman transition sideband delivered by the hyperfrequency source and the y-axis represents the non-normalized ratio $N_2/N$.}
\label{fig:fringelong}
\end{figure}

\section*{Acknowledgments}

The I.C.E. collaboration is funded by the Centre National d'Etudes Spatiales, as are GS, and RG. This work is also supported by the European Space Agency under the "Space Atom Interferometry" program. Further support comes from the RTRA "triangle de la physique" and the European Union STREP consortium FINAQS. Laboratoire Charles Fabry, ONERA and SYRTE are all members of IFRAF\footnote{Institut Francilien de Recherche en Atomes Froids : \texttt{http://213.251.135.217/ifraf/}}.

\bibliographystyle{epj}


text:
\part*{\bf\Large\center SUPPLEMENTARY MATERIAL\\ Giant gap in the magnon excitations of the quasi-1D chain compound Sr$_3$NiIrO$_6$}

\begin{center}

{W. Wu$^{1}$}
{D.T. Adroja$^{2,3}$}
{S. Toth$^{4,5}$}
{S. Rayaprol$^{6}$}
{E. V. Sampathkumaran$^{7}$}

\vspace{0.5cm}

$^1${\it Department of Physics and Astronomy and London Centre for Nanotechnology, University College London, Gower Street, London WC1E 6BT, UK}

$^2${\it ISIS Facility, STFC, Rutherford Appleton Laboratory, Chilton, Oxfordshire OX11 0QX, UK}

$^3${\it Highly Correlated Matter Research Group, Physics Department, University of Johannesburg, \\PO Box 524, Auckland Park 2006, South Africa}

$^4${\it Laboratory for Neutron Scattering, Paul Scherrer Institut (PSI), CH-5232 Villigen, Switzerland}

$^5${\it Laboratory for Quantum Magnetism, ICMP, Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland}

$^6${\it UGC-DAE CSR, Mumbai Center, R-5 Shed, BARC, Trombay, Mumbai 400085, India}

$^7${\it Tata Institute of Fundamental Research, Homi Bhabha Road, Colaba, Mumbai 400005, India}
\end{center}

\twocolumngrid

\section{Synthesis and sample characterization}

The polycrystalline sample of Sr$_3$NiIrO$_6$\ ($\sim$6 g) was prepared by solid-state reactions in an Ar atmosphere at 1200 $^\circ$C. Stoichiometric powder mixtures of NiO  (Alfa Aesar, 99.999\% purity), IrO$_2$ (Umicore) and SrCO$_3$ (Alfa Aesar, 99.99\% purity) were annealed for 24 hours. The checking of the phase purity of the end product and the determination of unit cell parameters were carried out using an x-ray powder diffractometer (XRD) with Cu-K$\alpha$1 radiation at room temperature. Furthermore the quality of the sample was checked using neutron diffraction measurement at 100 K on the WISH diffractometer at ISIS facility of the Rutherford Appleton Laboratory (RAL) in the UK. The magnetization and heat capacity measurements were carried out using commercial MPMS and PPMS systems (Quantum Design) respectively in the temperature range of 2 and 300 K.

\begin{ruledtabular}
\begin{table}[!htb]
	\centering
	\caption{Refined crystallographic parameters of Sr$_3$NiIrO$_6$\ using the space group $R\overline{3}c$ at 2 K from neutron diffraction data \cite{Lefrancois2014asm}.}
	\label{tab:str}
	\begin{tabular}{ccccc}
    \multicolumn{3}{l}{Unit cell dimensions:}&\multicolumn{2}{l}{$a=b=9.6089(4)$ \AA}\\
    \multicolumn{3}{l}{                     }&\multicolumn{2}{l}{$c=11.1678(4)$ \AA}\\
    \multicolumn{3}{l}{Angles:}&\multicolumn{2}{l}{$\alpha=\beta=90^\circ$,$\gamma=120^\circ$}\\
    \\
    Atom  &	Site&	$x$	     &	$y$	        &	$z$				\\
    \hline
    Sr    & $18e$  & 0.3636(3) & 0 &  1/4 \\
    Ni    & $6a$   & 0         & 0 &  1/4 \\
    Ir    & $6b$   & 0         & 0 &    0 \\
 	O     & $36f$  & 0.1828(3) & 0.0247(4) & 0.1140(2) \\
	\end{tabular}
\end{table}
\end{ruledtabular}
\begin{ruledtabular}
\begin{table}[!htb]
	\centering
	\caption{Selected distances between atoms in Sr$_3$NiIrO$_6$.}
	\label{tab:dist}
	\begin{tabular}{ccc}
    Bond & \multicolumn{2}{c}{Bond length (\AA)} \\
         & Intra-chain    & Inter-chain           \\
    \hline
    Ir-Ir & 5.584 & 5.852 \\
    Ni-Ni & 5.584 & 5.852 \\
    Ni-Ir & 2.792 & 5.625 \\
    \hline
    Sr-O  & \multicolumn{2}{c}{2.477} \\
    Ir-O  & \multicolumn{2}{c}{2.014} \\
    Ni-O  & \multicolumn{2}{c}{2.183} \\
    O-O   & \multicolumn{2}{c}{2.710} \\
    Sr-Sr & \multicolumn{2}{c}{3.206} \\
    Sr-Ni & \multicolumn{2}{c}{3.494} \\
	\end{tabular}
\end{table}
\end{ruledtabular}

\begin{ruledtabular}
\begin{table}[!htb]
	\centering
	\caption{Bond angles on selected atoms in Sr$_3$NiIrO$_6$.}
	\label{tab:bond}
	\begin{tabular}{cc}
    Atoms & Angles ($^\circ$) \\
    \hline
    Ir-O-Ni & 80.59$^\circ$ \\
    Ir-O-Ir & 106.90$^\circ$ \\
    Ni-O-Ni & 110.80$^\circ$ \\
	\end{tabular}
\end{table}
\end{ruledtabular}
Our XRD analysis shows that the Sr$_3$NiIrO$_6$\ sample was single phase and crystallized in the space group $R\overline{3}c$ (space group No.: 167) with $Z=6$ formula units per unit cell. The lattice parameters at $T=100$ K were determined from the neutron diffraction experiment \cite{Lefrancois2014sm}, values are shown in Table \ref{tab:str}, which are consistent with previous reports \cite{Nguyen1995sm,Vajenine1996sm,Flahaut2003sm,Mikhailova2012sm}. The different bond lengths and inter atomic distances are shown in Tab.\ \ref{tab:dist} and selected bond angles are given in Tab.\ \ref{tab:bond}. The crystal structure consists of chains aligned along the $c$-axis, formed by alternating face-sharing NiO$_6$ trigonal prisms and IrO$_6$ octahedra. The chains are arranged on a triangular lattice in the $ab$-plane. There are two Ir ions in the primitive unit at (0,0,0) and (0,0,1/2), and two Ni atom at (0,0,1/4) and (0,0,3/4). Taking into account the rhombohedral-lattice translations $t_1=(2/3,1/3,1/3)$, $t_2=(1/3,2/3,2/3)$, and identity, there are six Ir and six Ni sites in the conventional hexagonal unit-cell. The aforementioned symmetry leads to a shift of $1/6c$ between neighboring magnetic atoms along the chains.

\section{Magnetization and Heat capacity}

\begin{figure}[!htb]
    \centering
	\includegraphics[width = \linewidth]{S1.pdf}
	\caption{(Color online)  Magnetic susceptibility measured in zero-field cooled (red triangles) and field-cooled (black circles) conditions using 0.1 T field. The inset shows the inverse susceptibility versus temperature. The solid line denotes the linear Curie-Weiss fit between 150 and 300 K.}
	\label{fig:chi}
\end{figure}

Figure \ref{fig:chi} shows the magnetic susceptibility ($\chi$) of Sr$_3$NiIrO$_6$ as a function of temperature in zero-field-cooled (ZFC) and field-cooled (FC) conditions measured in an applied magnetic field of 0.1 T. Although both the ZFC and FC susceptibilities arise sharply below $T=70$ K, these exhibit considerably different behavior below $T=20$ K. While the ZFC signal sharply drops below 20 K, the FC susceptibility is almost constant below 20 K. Similar behavior was previously reported \cite{Flahaut2003asm,Mikhailova2012asm} and also observed in single crystals of Sr$_3$NiIrO$_6$\ in external field parallel to the $c$-axis \cite{Lefrancois2014asm}. Furthermore, the spin-chain compound Sr$_3$NiRhO$_6$ ($T_N=65$ K) also exhibits very similar behavior in the ZFC and FC susceptibility \cite{Stitzer2002sm,Rayaprol2004sm,Mohapatra2007sm}. On the other hand, the susceptibilities of Sr$_3$ZnIrO$_6$ ($T_N=19$ K) \cite{Nguyen1995asm}\cite{Vajenine1996sm,Niazi2002sm,McClarty2014asm} and Sr$_3$ZnRhO$_6$ ($T_N=16$ K) \cite{Hillier2011sm,Rayaprol2004sm,Mohapatra2007sm} do not show any difference between ZFC and FC susceptibility below $T_N$. It is a general trend for the spin-chain systems of this family having two different magnetic atoms alternating along the chain to exhibit considerable different behavior in the temperature dependent susceptibility for ZFC and FC. The inverse susceptibility of Sr$_3$NiIrO$_6$ exhibits a Curie-Weiss (CW) behavior between 200 and 300 K with an effective total moment of $3.54(5)\mu_B$ per formula-unit and paramagnetic CW temperature of $-22.1(3)$ K for ZFC data (see inset of Fig.\ \ref{fig:chi}). However these values should be taken as rough estimates, due to the different temperature dependence of the paramagnetic susceptibility of Ir and Ni ions. The negative sign of the CW temperature indicates dominant antiferromagnetic interactions. Furthermore, $\chi$ exhibits a broad maximum below 100 K, indicating the low-dimensional nature (or due to frustration) of the magnetic interactions in Sr$_3$NiIrO$_6$.
The measured ac-susceptibility has a peak in the real part ($\chi'$) at 20 K \cite{Lefrancois2014asm}, which shows strong frequency dependence (i.e. peak position increases with frequency) and the time dependent magnetization follows logarithmic decay at 20 K. These results show the importance of magnetic frustration due to the triangular nature of the crystal structure.

\begin{figure}[!htb]
    \centering
	\includegraphics[width = \linewidth]{S4.pdf}
	\caption{(Color online) Heat capacity versus temperature of Sr$_3$NiIrO$_6$\ . The inset shows the heat capacity vs temperature plot in an expanded scale.}
	\label{fig:heat}
\end{figure}

We have also measured the heat capacity of Sr$_3$NiIrO$_6$\ between 2 K and 300 K (see Figure \ref{fig:heat}) and haven't found any clear signature of magnetic phase transition down to 2 K, even though the neutron diffraction study clearly reveals the present of magnetic Bragg peaks below 70 K \cite{Lefrancois2014asm}, confirming AFM magnetic order below 70 K. On the other hand, the neutron diffraction data did not show any clear change in the magnetic structure between 2 K and 70 K. It should be noted that the heat capacity of Sr$_3$NiRhO$_6$ \cite{Rayaprol2004sm,Adroja2014sm}, did not reveal any magnetic phase transition either, but those of Sr$_3$ZnIrO$_6$ \cite{Niazi2002sm,McClarty2014asm} and Sr$_3$ZnRhO$_6$ \cite{Hillier2011sm} exhibit a clear $\lambda$-type anomaly at the magnetic ordering temperature. This might indicate that the absence of the $\lambda$-type anomaly in Sr$_3$NiIrO$_6$\ is strongly related to the low-dimensional nature in magnetic structure.
The observed value of the heat capacity for Sr$_3$NiIrO$_6$\ at 300 K is 254 J$\cdot$mol$^{-1}$K$^{-1}$, which is in a good agreement with that expected from the lattice contribution based on Dulong and Petit law \cite{Ashcroft1976sm} (for n-atom molecules $Cp\sim 3nR=274.4$ J$\cdot$mol$^{-1}$K$^{-1}$), showing that magnetic correlations are completely lost at room temperature.

\begin{figure}[!htb]
    \centering
	\includegraphics[width = \linewidth]{S2.pdf}
	\caption{(a) Scattering intensity as a function of momentum transfer of Sr$_3$NiIrO$_6$\ integrated in energy between 30 and 38meV measured at 5 K. (b) Magnetic scattering intensity only, obtained by subtracting phonon background. The solid lines show the magnetic form factor for Ni$^{2+}$ (blue line) and Ir$^{4+}$ (green line) scaled to the data.}
	\label{fig:ff}
\end{figure}

\begin{figure}[!htb]
    \centering
	\includegraphics[width = \linewidth]{S3.pdf}
	\caption{One of the possible magnetic structure of Sr$_3$NiIrO$_6$\ stabilized at low temperature in a commensurate phase with propagation vector $k=(0,0,1)$. Green and orange lines denote intrachain and interchain couplings respectively. Structure of symmetry $P\overline{3}c'1$  corresponding to global phase  $\varphi=0$. For global phase $\varphi=\pi/6$ there is a possibility to have zero moment on Ni and Ir for the Ni-Ir chain at $(x,y,z)$ \cite{Lefrancois2014asm}.}
	\label{fig:Js}
\end{figure}

\section{Spin Wave calculations}

We propose a simple magnetic Hamiltonian, where we take into account only interactions between nearest neighbor magnetic atoms along the $c$-axis and single ion anisotropy of the Ni$^{2+}$ ions. This model gives a perfect fit to the inelastic neutron scattering (INS) powder data, this also justifies that the interchain interactions can be neglected. Due to the rhombohedral-translations every Ni-Ir chain are equivalent to a single chain with two atoms. We denote the Ir and Ni atoms with sublattice $a$ and $b$ respectively. We restrict our spin Hamiltonian using the crystallographic space group. The point group symmetry $D_3$ of the Ni atom allows only a single anisotropy parameter along the c-axis denoted by $D_z$. The allowed matrix elements of the Ni-Ir bond is determined by the point group symmetry of the center of the bond that is $C_3$. This allows anisotropic exchange, with different $J_z$ and $J_{xy}$ values, also Dzyaloshinskii-Moriya interaction is allowed parallel to the $c$-axis that we neglect here. With the above assumptions we propose the following effective spin Hamiltonian:
\begin{align}
\mathcal{H}  &=\displaystyle\sum_i J_z \hat{s}_{a,i}^z \hat{s}_{b,i}^z + J_{xy}\left(\hat{s}_{a,i}^x \hat{s}_{b,i}^x + \hat{s}_{a,i}^y \hat{s}_{b,i}^y \right)+ \nonumber \\
             +&\sum_i J_z \hat{s}_{a,i}^z \hat{s}_{b,i+1}^z + J_{xy}\left( \hat{s}_{a,i}^x \hat{s}_{b,i+1}^x + \hat{s}_{a,i}^y \hat{s}_{b,i+1}^y \right)+ \nonumber \\
             +&\sum_i D_z \left(\hat{s}_{b,i}^z\right)^2.
             \label{eq1}
\end{align}

The first two sums describe the anisotropic exchange interaction on both the Ni and Ir sites, while the last term describes the single-ion anisotropy of the Ni sites. In the observed magnetic structure the spins point along the $z$-axis AF order along the chain. The classical ground state of Eq.\ \ref{eq1} is identical to this if $J_z$ is positive and $|J_z|>|J_{xy}|$ for zero anisotropy. Assuming the AF ground state, we calculated the spin wave dispersion using the Holstein-Primakoff transformation \cite{Holstein1940sm} to reduce the Hamiltonian to non-interacting bosons (magnons) and applied the Bogoliubov transformation to diagonalize the quadratic form \cite{Bogoliubov1947sm}. Since there are two sublattices, we obtained two $\omega_{1,2}(Q)$ spin wave modes dispersing along the chain direction:
\begin{widetext}
\begin{align}
\omega_{1,2}(Q) = \left| \pm \sqrt{D_z^2s_b^2+2D_zJ_zs_b(s_a+s_b)+Jz^2(s_a+s_b)^2-2J_{xy}^2s_as_b(1+\cos(Q\cdot c/2))} + D_zs_b+J_z(s_a-s_b) \right|.
\end{align}
\end{widetext}
where $Q$ is the momentum of the magnon in \AA$^{-1}$ units, $c$ is the lattice parameter. In order to fit the measured spectrum, we calculated the bottom and top energies of the spin wave band ($\omega_{b}$ and $\omega_{t}$). If  $D_z<\omega_{t}/4$ we got:
\begin{align}
J_z =& \omega_{t}/2 \\
J_{xy}^2 =& D_z(\omega_{t}+\omega_{b})+1/4\omega_{t}^2-1/2\omega_{b}^2-1/4\omega_{b}\omega_{t}, \nonumber
\end{align}
while if $D_z>\omega_{t}/4$ we got:
\begin{align}
J_z   =& \omega_{t}-2D_z \\
J_{xy}^2 =& -2D_z(\omega_{t}+\omega_{b})+\omega_{t}^2-1/2\omega_{b}^2+1/2\omega_{b}\omega_{t}. \nonumber
\end{align}

If we determine the bottom and the top of the spin wave band from the INS data, there is only a single free parameter $D_z$ to be fitted. By calculating the powder spin wave spectrum by averaging out sufficiently enough random sample orientations for a series of $D_z$ values, we could determine $D_z$ by a direct comparison of the simulation and data.

An alternative fitting of the data is possible if we assume that we only observe the lower spin wave band. Thus there is an upper branch that we missed either due to its small INS cross section within our measured momentum transfer and energy range or it was above our maximum measured incident neutron energy ($E_i=500$ meV). Detection of spin waves at high energies is much harder due to the cutoff of the INS intensity at higher $Q$ values because of the magnetic form factor. We simulated this scenario within our spin wave model. The highest possible energy of the upper band is 75 meV assuming negative $D_z$, this belongs to $D_z=0$ meV, $J_z=37.5$ meV and $J_{xy}=20$ meV. In this case the gap is induced only by the difference between $J_z$ and $J_{xy}$. However in this case the lower band in the Q-integrated cuts would be symmetric as a function of energy, which is not consistent with the data, where the inelastic peak shows a shoulder higher energies. In addition, the Q-dependence of the lower magnon band would show the Ni form factor mainly, which doesn't fit the data either, where the upper shoulder has clearly different Q-dependence than the peak intensity at 35 meV, see Fig.\ 4(c-d) in the manuscript. Thus we can exclude that there is an upper undetected spin wave branch with great confidence.

redpajama_set_name: RedPajamaArXiv


text:
\section{Introduction}
Environmental lighting models aim to approximate realistic lighting effects in a scene. Existing state-of-the-art methods \cite{gardner2017learning,kanamori2019relighting,pandey2021total,sun2019single} use HDR spherical panoramas for lighting. In many real-world applications, however, only LDR LFOV images are available. Therefore, it is important to estimate HDR panorama illumination from an LDR LFOV image, aided with the capability of lighting editing to flexibly control lighting conditions. Lighting estimation and editing have a wide range of applications in mixed reality such as object insertion and relighting in virtual meetings and games. For example, given an LDR LFOV background, a person can be inserted into the background and realistically relit by controllable panorama illumination.

There are three main challenging problems in lighting estimation.
First, only a partial scene is observed in a limited FOV image. The other unobserved part of a scene is ambiguous and could significantly affect the illumination distribution in a full scene due to ambiguous lighting sources with different numbers, shapes, and directions.
Second, only a partial range of lighting is observed in typical LDR images. Physical lighting has a wide range of intensity levels in real scenes, ranging from shadows to bright light sources. However, in LDR images, underexposed areas are clipped to 0, and overexposed areas are clipped to 1. Out-of-range lighting cannot be observed in LDR images.
Third, lighting estimation of unobserved views depends on intrinsic attributes of observed objects (e.g., material and 3D geometry), but these annotations are often unavailable in real-world applications.

Existing methods on illumination estimation provide partial solutions to illumination estimation, which could be roughly classified into two groups, i.e., regression-based estimation methods \cite{gardner2017learning,gardner2019deep,garon2019fast,zhan2021sparse,hold2017deep,legendre2019deeplight} and generation-based estimation methods \cite{zhan2020emlight,song2019neural,srinivasan2020lighthouse}.
In the first group, researchers propose to directly regress panorama lighting information from a given LDR LFOV image. The main difficulty of this pipeline is to parameterize lighting representations as target labels for regression, including light representation (e.g., number, shape, directions and intensity) \cite{gardner2019deep}, Spherical Harmonics \cite{garon2019fast}, and wavelet transformation \cite{zhan2021sparse}. However, these lighting labels mainly focus on strong lighting sources and thus lose the details of a scene. \textcolor{black}{When a scene contains reflective materials (e.g., glass, mirror, and metal), this pipeline will fail to provide a desired rendered visual effect. In image-based rendering, the HDR map can be also applied as background.} Moreover, regression-based models often suffer from unstable training of neural networks due to partial observed view, a wide range of lighting, unknown geometry, and material of objects. For example, EMlight \cite{zhan2020emlight} claimed that it is necessary to train a regression model by gradually increasing training examples to avoid training collapse.
Second, generation-based estimation methods aim at generating full HDR panoramas from an LDR LFOV image. They typically decompose illumination estimation as several sub-tasks, such as LFOV-to-panorama completion, LDR-to-HDR regression \cite{zhan2020emlight}, and 3D geometry prediction \cite{song2019neural}. \textcolor{black}{These methods either require additional annotations or multiple optimization stages.}

Our proposed method belongs to the second group. Unlike the existing \sloppy{generation-based} estimation methods, we propose a coupled dual-StyleGAN panorama synthesis network (\textbf{StyleLight}) that solves LFOV-to-panorama and LDR-to-HDR problems in a unified framework and does not require any 3D geometry annotation. Specifically, StyleLight is a coupled dual-StylGAN that maps noises to both HDR and LDR panoramas. The LDR and HDR panorama synthesis share a similar generator. At the output of the generator, a tone mapping is adopted for the translation between HDR and LDR. Considering the fact that an HDR panorama consists of extremely different weak lighting distribution and strong lighting distribution, we use two discriminators to distinguish real/fake HDR and LDR panorama distributions, respectively. Given an LDR LFOV image, we propose a focal-masked GAN inversion method to find its latent code through the LDR panorama synthesis branch and then generate its corresponding HDR panorama as illumination maps through the HDR synthesis branch. Moreover, we propose a structure-preserved GAN inversion method for lighting editing with a trained StyleLight model to flexibly control the environment lighting of panoramas, such as turning on/off lights, opening/closing doors and windows, making our method well-suited for various real-world applications.

The main contributions of this paper can be summarized as follows:
\textbf{1)} We propose a coupled dual-styleGAN synthesis network (StyleLight) that integrates HDR and LDR panorama synthesis in a unified framework. During inference, we propose a focal-masked GAN inversion method to solve both LFOV-to-panorama and LDR-to-HDR generation.
\textbf{2)} We propose a structure-preserved GAN inversion method for lighting editing with the trained StyleLight model to flexibly control panorama lighting.
\textbf{3)} Extensive experiments demonstrate the superiority and the effectiveness of our proposed lighting estimation method over state-of-the-art methods on indoor HDR panoramas and show promising applications with our proposed lighting editing.

\section{Related Work}
Lighting estimation is one of the long-standing problems in computer vision and graphics. Early work \cite{barron2013intrinsic,wu2011high,valgaerts2012lightweight} typically simplifies the models by considering fixed number of lights \cite{lopez2010compositing}, annotated light positions \cite{karsch2011rendering} and geometry \cite{karsch2011rendering,karsch2011rendering,lombardi2015reflectance,karsch2011rendering,maier2017intrinsic3d,maier2017intrinsic3d}. Recent lighting estimation methods relax such assumptions and focus on more challenging lighting estimation.
We briefly review the work most relevant to this paper.

\noindent
\textbf{Lighting estimation.} Existing lighting estimation methods can be roughly classified into two groups, i.e., regression-based methods and generation-based methods. Regression-based models \cite{gardner2019deep,legendre2019deeplight,garon2019fast,zhan2021sparse,zhan2020emlight,hold2017deep} seek for effective lighting representations of HDR panoramas as labels for lighting regression. These methods take an LDR LFOV image as input and predict lighting labels, such as direct lighting representation (e.g., numbers, shapes, directions, and intensity) \cite{gardner2019deep}, Spherical Harmonics (SH) \cite{garon2019fast}, and wavelet transformation \cite{zhan2021sparse}. For example,
Gardner \textit{et al.} \cite{gardner2017learning} trained a lighting classifier on an annotated LDR panorama dataset to predict the location of lighting sources of LDR LFOV images and fine-tuned the prediction of lighting intensity on a small HDR panorama dataset.
Gardner \textit{et al.} \cite{gardner2019deep} regressed the illumination conditions such as the light source positions in 3D, areas, intensities, and colors.
LeGendre \textit{et al.} \cite{legendre2019deeplight} designed a joint camera-sphere device to collect LDR LFOV images that include three reflective spheres, and regressed HDR lighting from an LDR LFOV image.
Garon \textit{et al.} \cite{garon2019fast} regressed the 5th-order SH coefficients for the lighting at a certain location given a single image and a 2D location in an image. Regression-based models ignore the low-intensity lighting details and thus fail to provide a desired rendered quality for mirror materials. Besides, they often suffer from unstable training of neural networks due to the high dynamic range of lighting.

Generation-based methods \cite{gardner2017learning,song2019neural,srinivasan2020lighthouse} focus on decomposing illumination estimation as several sub-tasks, such as LFOV-to-panorama completion and LDR-to-HDR regression.
Song \textit{et al.} \cite{song2019neural} decomposed illuminations prediction into several simpler differentiable sub-tasks including geometry estimation, scene completion, and LDR-to-HDR estimation. Zhan \textit{et al.} \cite{zhan2020emlight} decomposed the illumination map into spherical light distribution, light intensity and the ambient term for illumination regression. This aforementioned paradigm either decomposes illumination estimation as several sub-tasks or collects expensive annotations for training.

\noindent
\textbf{Image editing via GAN inversion.} Image editing is a task that modifies a target attribution of a given image while preserving other details \cite{wang2021high,harkonen2020ganspace,li2021transforming,shen2020interpreting,roich2021pivotal,zhuang2021enjoy,tewari2020stylerig,shen2021closed}. Recent state-of-the-art image editing methods focus on generative adversarial network (GAN) inversion techniques to control the latent space of GANs for image editing.
For example, Ganspace \cite{harkonen2020ganspace} used Principal Component Analysis (PCA) to identify important latent directions and create interpretable controls for image attributes, such as viewpoint, aging, lighting, and time of day.
InterfaceGAN \cite{shen2020interpreting} proposed to edit the latent code $\mathbf{z}$ with $\mathbf{z}_{edit} = \mathbf{z}+\alpha \mathbf{n}$ for each semantic attribute where $\mathbf{n}$ is a normal vector of a linear SVM's hyper-plane.
SeFa \cite{shen2021closed} proposed a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
StyleFlow \cite{abdal2021styleflow} formulated conditional exploration as an instance of conditional continuous normalizing flows in the GAN latent space conditioned by attribute features.
These image editing methods focus on controlling the latent code to control attributes of objects. However, in our lighting editing setting, our goal is to edit a light source that is located at a small area of a panorama given a bounding box, which is different from the typical image editing.

\section{Our Approach}

\subsection{Overview of the Proposed Framework}
\label{subsec:overview}
\begin{figure*}[t]
  \centering
  \includegraphics[width=\linewidth]{framework4.pdf}
  \caption{Overview of the proposed framework. \textbf{Left:} Lighting estimation. We first propose a coupled dual-StyleGAN that generates HDR and LDR panoramas from noises. Given an LDR LFOV image, we map it into a panorama. We introduce a focal-masked GAN inversion method to find its latent code through the LDR branch and compute the HDR panorama through the HDR branch. \textbf{Right:} Lighting editing. With the trained dual-StyleGAN, we perform lighting editing to control panoramas lighting.}\label{fig:overview}
\end{figure*}
An overview of our proposed framework is depicted in \textbf{Fig. \ref{fig:overview}}. The framework consists of lighting estimation and lighting editing. The goal of lighting estimation is to predict HDR panorama from an LDR LFOV image. To achieve this goal, we introduce a novel coupled dual-StyleGAN synthesis network (StyleLight) that learns to synthesize both HDR and LDR panoramas from noises with a shared generator and two specific discriminators. With the trained StyleLight model, we propose a focal-masked GAN inversion method to find the latent code of an LDR LFOV image and predict the HDR panorama as an illumination map (Section \ref{subsec:lighting_estimation}). To control various lighting conditions of the target HDR panorama, we introduce a structure-preserved lighting editing method to solve this problem (Section \ref{subsec:light_edit}). To the best of our knowledge, this is the first attempt to perform lighting editing on HDR panoramas for controlling the scene lighting.

\subsection{Lighting Estimation}
\label{subsec:lighting_estimation}
Generation-based lighting estimation methods typically decompose illumination estimation into several sub-tasks, such as LFOV-to-panorama completion and LDR-to-HDR regression. These methods optimize sub-tasks independently and could lead to sub-optimal estimation results. Moreover, these methods can be hardly scalable to lighting editing. Different from these methods, we propose a coupled dual-StyleGAN that integrates LFOV-to-panorama, LDR-to-HDR, and lighting editing into a unified framework.

\begin{figure*}[ht]
  \centering
  \includegraphics[width=\linewidth]{plot_pixel_distribution2_1.pdf}
  \caption{Pixel distributions of two HDR panoramas. The first and third histograms cover the full range of pixel values. The second and fourth histograms visualize high-intensity values of the first and third histograms. An HDR panorama consists of two components, i.e., low-intensity pixels (blue, from 0 to 1, denoted as LDR) and high-intensity pixels that are out of range LDR pixels (red, larger than 1, denoted as HDR$^{-}$). }\label{fig:pixel_distribution}
\end{figure*}

\noindent
\textbf{Coupled dual-StyleGAN for joint LDR and HDR panorama synthesis.} HDR panoramas are more difficult to synthesize than LDR panoramas due to the complex characteristics of distributions. \textbf{Figure \ref{fig:pixel_distribution}} shows the distributions of two HDR panorama examples (others are similar). An HDR panorama consists of two components, i.e., low-intensity pixels (blue, denoted as LDR, from 0 to 1) and high-intensity pixels that are out of range LDR pixels (red, denoted as HDR$^{-}$, larger than 1). The distributions suggest three key points. First, LDR distributions are similar to a Gaussian distribution. Most of the pixels are LDR pixels and they are limited in the narrow range. This suggests that LDR distributions are easy to generate. Second, HDR$^{-}$ distributions are dissimilar and cover a few pixels in the high dynamic range (much larger than 1). This implies that HDR$^{-}$ distributions are difficult to generate and could lead to unstable training of HDR synthesis. If we only use an HDR discriminator to distinguish entire HDR panoramas, \textcolor{black}{the dynamic HDR$^{-}$ pixels (e.g., $>$100) largely affect the discriminator and other regions (0$\sim$1) are often ignored through convolution. $D_{'}$ can be regarded as a regularization term to preserve image semantics in LDR regions.} \textcolor{black}{We analyze the distributions of 1401 training images. We fit a Gaussian distribution for both LDR and HDR$^-$ of each image, respectively. We find that most of LDR distributions (99.86\% LDR images) follow Gaussian distributions and most of HDR$^-$ distributions (70.38\% HDR$^-$ images) fail to converge to Gaussian distributions.} Note that LDR panoramas play a key role in the synthesis from LDR LFOV images to HDR panoramas in the next section. We expect HDR and LDR images to be well synthesized.
It is noteworthy that although HDR$^{-}$ distributions are difficult to generate, HDR$^{-}$ and LDR have a strong semantic relation. LDR pixels provide the location and shape contextual information of lighting sources while HDR pixels provide intensity information.

Motivated by the fact that LDR panoramas are easy to generate and they provide location and shape contextual information of lighting for HDR synthesis, we introduce a coupled dual-StyleGAN for joint HDR and LDR panorama synthesis. With the help of LDR panorama synthesis, dual-StyleGAN is able to generate accurate HDR panoramas. Specifically, the dual-StyleGAN takes a noise vector $\bm{z}\in z$ as input and maps the noise ${\bm{z}}$ into a intermediate latent space $\omega$ that is much less entangled than the $z$ space, as analyzed in StyleGAN2 \cite{karras2020analyzing}. The mapping function $\bm{w}=f({\bm{z}})$ consists of eight fully-connected layers with activation layers. The latent code $\bm{w}\in \omega$ is then fed into the generator $G$. After that, two transformations $\psi(\cdot)$ and $\phi(\cdot)$ are used to transform the output of $G$ into HDR and LDR panoramas, respectively. That is, $x_{\mathrm{hdr}}=\psi(G({\bm{w}}))$ and $x_{\mathrm{ldr}}=\phi(G({\bm{w}}))$, as illustrated in \textbf{Fig. \ref{fig:overview}}. \textcolor{black}{The HDR output $\psi(\cdot)$ is an inverse tone-mapping plus clamping (positive pixels) and the LDR output $\phi(\cdot)$ is clampped from 0 to 1. To ease the stable training, we pre-process the the HDR training panoramas by $x^{'}=\alpha x^{1/{\gamma}}$, where $\gamma=2.4$ and $\alpha$ is set to 0.5 divided by 50-th percentile of all pixel values. We use $clamp(x^{'},0,+\infty))$ for HDR and $clamp(x^{'},0,1)$ for LDR. During inference, we perform inverse tone-mapping $clamp(x^{'},0,+\infty))$ into $x$. Other tone-mappings are also suitable for our framework.} The idea behind the shared generator network is that HDR and LDR panoramas share the same contextual semantics. Formally, the proposed coupled dual-StyleGAN is formulated as
\begin{equation}
\label{eq:hdr_ldr}
\begin{aligned}
 \mathop {\min }\limits_{G} \mathop {\max }\limits_{D,D^{'}}\mathcal{J}(G,D,D^{'}) &= E_{x \backsim P_{data}(x)}[\log(D(x))] \\
   &+E_{x \backsim P^{'}_{data}(x)}[\log(D^{'}(x))]\\
   &+E_{w\backsim P_{\bm{w}}}[\log(1-D(\psi(G(\bm{w}))))]\\
   &+E_{w\backsim P_{\bm{w}}}[\log(1-D^{'}(\phi(G(\bm{w}))))],\\
\end{aligned}
\end{equation}
where $D(\cdot)$ and $D^{'}(\cdot)$ denote two discriminators for HDR synthesis and LDR synthesis, respectively. \textcolor{black}{In Eq. \ref{eq:hdr_ldr}, we optimize $G$ by minimizing the third and fourth terms. We optimize $D$ and $D^{'}$ simultaneously by maximizing the probability of predicting real and fake panoramas.} Note that we do not use the same discriminator because the distributions of HDR and LDR panoramas are different, as illustrated in \textbf{Fig. \ref{fig:pixel_distribution}}.

\noindent
\textbf{Focal-Masked GAN inversion for LDR-to-HDR transformation and LFOV-to-panorama completion.} After obtaining the coupled dual-StyleGAN, we can perform LDR-to-HDR transformation and LFOV-to-panorama completion. In the coupled dual-StyleGAN, HDR synthesis and LDR synthesis share the same generator $G$ and thus guarantee pixel-level alignment between the LDR-to-HDR transformation. Thanks to the coupled dual-StyleGAN that integrates HDR synthesis and LDR synthesis into a unified framework, we can find a shared latent code of an HDR panorama and an LDR LFOV image in the $\omega$ space, as illustrated in \textbf{Fig. \ref{fig:overview}}(left). Given an LDR LFOV image, we map it into a spherical panorama as a masked panorama. We then extend a GAN inversion method \cite{roich2021pivotal} into a masked GAN inversion to tailor for our coupled dual-StyleGAN. We find the latent code of the masked panorama in the $\omega$ space. We forward the latent code through the HDR branch to obtain an HDR panorama. Let $x_{\mathrm{lfov}}^{\mathrm{ldr}}$ denote a masked panorama of the LDR LFOV image and $\bm{n}$ denote noise maps of all layers. We first optimize latent code $\bm{w}\in \omega$ such that synthesized LDR LFOV matches the target LDR LFOV as much as possible, which can be formulated as
\begin{equation}
\label{eq:inversion}
  \bm{w}_{*}, \bm{n}_{*}=\mathop{\mathrm{argmin}}\limits_{\bm{w},\bm{n}}  \mathcal{L}_{\mathrm{LPIPS}}(M\odot\phi(G(\bm{w},\bm{n};\theta)),x_{\mathrm{lfov}}^{\mathrm{ldr}})+ \lambda_{\bm{n}}\mathcal{L}_{\bm{n}}(\bm{n}),
\end{equation}
where $\mathcal{L}_{\mathrm{LPIPS}}$ represents the VGGNet based Perceptual loss \cite{johnson2016perceptual} for distance measure and $\mathcal{L}_{\bm{n}}(\bm{n})$ represents a noise regularization term to constrain noise maps $\bm{n}$ and $\lambda_{\bm{n}}$ controls its importance. We re-use the symbol $G$ with latent code $\bm{w}\in \omega$, noise maps $\bm{n}$ and weights $\theta$ as input. The variable $M$ denotes a mask that preserves the pixels corresponding to the LFOV image $x_{\mathrm{lfov}}^{\mathrm{ldr}}$. Because it is hard to directly reconstruct the target LDR LFOV image by optimizing $\bm{w}$ and $\bm{n}$, we then fine-tune the weights $\theta$ of G by
\begin{equation}
 \begin{aligned}
 \label{eq:gan_finetue}
 \mathcal{L}_{G}(\theta)=&  \mathcal{L}_{\mathrm{LPIPS}}(M\odot\phi(G(\bm{w}_{*},\bm{n}_{*};\theta)),x_{\mathrm{lfov}}^{\mathrm{ldr}})\\
 &+\lambda_{L2}^{R} \mathcal{L}_{L2}(M\odot\phi(G(\bm{w}_{*},\bm{n}_{*};\theta)),x_{\mathrm{lfov}}^{\mathrm{ldr}}),\\
 \end{aligned}
\end{equation}
 where $\mathcal{L}_{G}$ represents the similarity loss between a masked generated LDR image and the LFOV HDR image $x_{\mathrm{lfov}}^{\mathrm{ldr}}$. We use the Mean-Squared-Error loss and the VGGNet based Perceptual loss for distance measure. To further improve visual quality, we also use a regularization term, which is given by
 \begin{equation}
 \begin{aligned}
\label{eq:gan_regularization}
 \mathcal{L}_{R}(\theta)&=  \mathcal{L}_{\mathrm{LPIPS}}(\phi(G(\bm{w}_r,\bm{n}_{*};\theta)),\phi(G(\bm{w}_r,\bm{n}_{*};\theta_0)))\\&+ \lambda_{L2}^{R'}\mathcal{L}_{L2}^{R}(\phi(G(\bm{w}_r,\bm{n}_{*};\theta)),\phi(G(\bm{w}_r,\bm{n}_{*};\theta_0))),\\
 \end{aligned}
\end{equation}
where $\bm{w}_r=\bm{w}_{*}+\alpha(\bm{w}_z-\bm{w}_{*})/||\bm{w}_z-\bm{w}_{*}||_{2}$ and $\bm{w}_z$ is a latent code that is generated by a random $\bm{z}$. The $\theta_0$ is the old weights of G before finetuning. The $\alpha$ is a interpolation parameter. Finally, we combine Equations \ref{eq:gan_finetue} and \ref{eq:gan_regularization}, and fine-tune $G$ by
 \begin{equation}
\label{eq:final_loss}
 \theta_{*}=  \mathop{\mathrm{argmin}}\limits_{\theta}\mathcal{L}_{G}(\theta)+\eta\mathcal{L}_{R}(\theta).
\end{equation}

Simply extending the GAN inversion method \cite{roich2021pivotal} into a masked GAN inversion for the coupled dual-StyleGAN cannot guarantee that the pixels of lights in LDR panoramas will be transformed into high-intensity HDR$^{-}$. Strong lighting sources are the main factors for rendering environment, but high-intensity pixels are much less than low-intensity pixels, as shown in \textbf{{Fig. \ref{fig:pixel_distribution}}}. Without any constraint, the model tends to fit the major LDR pixels and ignore the few HDR$^{-}$ pixels. Therefore, we propose a focal-masked to highlight the strong lighting sources. In our setting, we highlight top 10\% strongest pixels. Given an LDR LFOV image $x_{\mathrm{lfov}}^{\mathrm{ldr}}$, we compute the top 10\% strongest pixels and obtain a focal mask, which denoted as $FOCAL(x_{\mathrm{lfov}}^{\mathrm{ldr}})$. We re-write Equation \ref{eq:inversion} as
\begin{equation}
\label{eq:inversion_new}
\begin{aligned}
  \bm{w}_{*}, \bm{n}_{*}&=\mathop{\mathrm{argmin}}\limits_{\bm{w},\bm{n}}  \mathcal{L}_{\mathrm{LPIPS}}(M\odot\phi(G(\bm{w},\bm{n};\theta)),x_{\mathrm{lfov}}^{\mathrm{ldr}})+ \lambda_{\bm{n}}\mathcal{L}_{\bm{n}}(\bm{n})\\
  &+\beta_{L2}\mathcal{L}_{L2}(FOCAL(\phi(G(\bm{w},\bm{n};\theta))),FOCAL(x_{\mathrm{lfov}}^{\mathrm{ldr}})).\\
\end{aligned}
\end{equation}
Note that $FOCAL(\phi(G(\bm{w},\bm{n};\theta)))$ is consistent with $FOCAL(x_{\mathrm{lfov}}^{\mathrm{ldr}})$ in Equation \ref{eq:inversion_new}. Similarly, we re-write Equation \ref{eq:gan_finetue}  as
\begin{equation}
\label{eq:gan_finetune_new}
\begin{aligned}
   \mathcal{L}_{G}(\theta)&=  \mathcal{L}_{\mathrm{LPIPS}}(M\odot\phi(G(\bm{w}_{*},\bm{n}_{*};\theta)),x_{\mathrm{lfov}}^{\mathrm{ldr}})\\
   &+\lambda_{L2}^{R} \mathcal{L}_{L2}(M\odot\phi(G(\bm{w}_{*},\bm{n}_{*};\theta)),x_{\mathrm{lfov}}^{\mathrm{ldr}})\\
   &+\beta_{L2}\mathcal{L}_{L2}(FOCAL(\phi(G(\bm{w}_{*},\bm{n};\theta))),FOCAL(\bm{w}_{\mathrm{lfov}}^{\mathrm{ldr}})).\\
\end{aligned}
\end{equation}

\subsection{Lighting Editing}
\label{subsec:light_edit}
Lighting estimation provides a way to replicate real-world lighting conditions of a scene from a single LDR LFOV image and can be used to re-render inserted objects. In some occasions, however, the estimated lighting conditions do not meet our design/modeling requirements. For example, we would like to turn on an indoor light and ``turn off" the light from windows at night given daytime HDR panoramas. Although light sources are located in a local area, it affects the global panorama lighting. Existing approaches do not consider this problem. Here, we introduce a lighting editing method to control panorama lighting through exploiting the powerful StyleLight.

In general, lighting editing includes three operations, i.e, adding a new light, removing a light, and controlling lighting intensity. In essence, the first two operations can be accomplished by the third one.  In light estimation, we obtain a latent code $w_0$ that recovers an HDR panorama from an LDR LFOV image. Here we assume that $\bm{w}_0$ is known. Our goal is to find a editable direction of $\bm{w}_0$ to adjust the lighting intensity such that the lighting intensity at a candidate  location changes while the geometry structure of the scene is preserved. Given a bounding box of a candidate lighting location as a mask $M$, we introduce a structure-preserved lighting editing loss to find the editable direction of $\bm{w}$, which is formulated as
\begin{equation}
\label{eq:inversion_light}
\begin{aligned}
  \bm{w}_{*}, \bm{n}_{*}&=\mathop{\mathrm{argmin}}\limits_{\bm{w},\bm{n}}  \mathcal{L}_{\mathrm{LPIPS}}((1-M)\odot \phi(G(\bm{w},\bm{n};\theta)),(1-M)\odot\phi(G(\bm{w}_0,\bm{n};\theta))\\
  &+\mathcal{L}_{L2}((1-M)\odot \phi(G(\bm{w},\bm{n})),(1-M)\odot\phi(G(\bm{w}_0,\bm{n};\theta))\\
  &+ \frac{\delta}{N}\sum M\odot\phi(G(\bm{w},\bm{n};\theta)),\\
\end{aligned}
\end{equation}
where $\delta$ denotes a lighting adjusting factor. When $\delta$ is positive, the lighting intensity decreases; when $\delta$ is negative, the lighting intensity increases. $N$ is the number of masked pixels. \textcolor{black}{The third term aims to adjust lighting intensity given a bounding box, and the first and second terms preserve other regions.}

\section{Experiments}
\label{sec:exper}
\textbf{Datasets and experimental settings.} We evaluate our StyleLight model on the Laval Indoor HDR dataset \cite{gardner2017learning} that contains 2,100 HDR panoramas. Following the train/test split of \cite{gardner2017learning}, we use 1,719 panoramas for training and 289 panoramas for testing. For the test set, we crop one HDR LFOV image from each panorama and transform it into an LDR LFOV image by tone mapping \cite{zhan2020emlight}. Since lighting estimation is a challenging task, some compared methods use extra annotations. For example, Gardner2017 \cite{gardner2017learning} used annotated lighting sources of SUN360 \cite{xiao2012recognizing} to learn to predict the locations and shapes of lighting sources. Gardner2019 \cite{gardner2019deep} manually annotated the Laval Indoor dataset using EnvyDepth \cite{banterle2013envydepth} and used per-pixel depth maps to improve lighting estimation.

\noindent
\textbf{Implementation details.}
We use StyleGAN2 \cite{karras2020analyzing} as the backbone. The output of the generator is a 256$\times$512 image. We resize it into 128$\times$256 for evaluation, which is consistent with other methods. The resolution of feature maps begins with a $4\times 8$ constant map and increases by $\times 2$ upsampling each block, yielding $256\times 512$ patch-based feature maps. In the full training of StyleLight, 4000k real images (kimg) are shown to the discriminators. We train StyleLight with 8 v100 GPUs for 7 hours. The batch size is set to 32. We use Adam optimizer for the generators and two discriminators with a learning rate of 0.0025 and set $\beta_1=0$,  $\beta_2=0.99$.
\begin{wrapfigure}{r}{0.4\textwidth}
  \centering
    \includegraphics[width=0.4\textwidth]{sphere.pdf}
  \caption{Three spheres with different materials: mirrored silver, matte silver, and diffuse grey.}\label{fig:sphere}
\end{wrapfigure}

\noindent
\textbf{Evaluation metrics.} Similar to Gardner2019 \cite{gardner2019deep} and EMlighting \cite{zhan2020emlight}, we use three spheres with different materials for evaluation, including mirror, matte silver, and gray diffuse, as illustrated in \textbf{Fig. \ref{fig:sphere}}. We render the three spheres by ground-truth HDR panoramas and the estimated illumination maps. Similar to \cite{zhan2020emlight}, we use Root Mean Square Error (RMSE), scale-invariant RMSE (si-RMSE), and Angular Error for evaluation. RMSE and si-RMSE measure pixel-level errors of lighting intensity. Angular Error measures the ratio of R, G, and B components. Different from \cite{zhan2020emlight}, we compute the metrics on the three rendered spheres and ignore the area of the rendered black background, which is invariant to the area of the background. Because there are no public codes for evaluation, we design our own hyper-parameters for the three spheres. We also use Fr\'{e}chet Inception Distance (FID)~\cite{heusel2017gans} to measure the distribution similarity between the ground-truth and the estimated illumination maps.

\subsection{Quantitative Evaluation}
\label{subsec:quantitative}
We compare StyleLight with three representative state-of-the-art illumination estimation methods, including a generation-based method \cite{zhan2020emlight} and two regression-based methods \cite{gardner2017learning,gardner2019deep}. Note that Gardner2017 \cite{gardner2017learning} used extra lighting annotations and Gardner2019 \cite{gardner2019deep} used extra depth annotations for training. StyleLight and EMlight \cite{zhan2020emlight} models do not require any extra annotation. For each method, we render three typical spheres (\textbf{Fig. \ref{fig:sphere}}) by the predicted illumination maps and ground-truth HDR panoramas on the test set and evaluate RMSE, si-RMSE, and Angular Error. We also evaluate FID to measure the distribution distance of ground-truth and predicted illumination maps.

As shown in \textbf{Table \ref{tab:Comparisons}}, our StyleLight outperforms the compared methods on all metrics and materials. Among them, Gardner2017 and Gardner2019 focus on strong lighting sources and ignore low-intensity lighting, so they fail to achieve a desired RMSE and si-RMSE on the mirror sphere and do not preserve components of colors. EMlight first regresses a Gaussian map and fuses it with the LFOV image for illumination generation. The drawback is that the multi-step optimization could lead to sub-optimal performance. Since EMlight contains regression learning, it is difficult to train. EMlight\footnote{https://github.com/fnzhan/Illumination-Estimation} claimed that the model has to be trained on subsets of 100, 1000, 2500,... and the full set gradually and could collapse occasionally during training. We also see that our StyleLight largely outperforms other methods on FID, indicating that our predicted lighting distribution is closest to ground truth on a wide range of lighting. The results also suggest that StyleLight can synthesize plausible scenes and can achieve a promising result when rendering mirror materials. Our StyleLight also provides controllable lighting editing.

\begin{table*}[!htp]
\caption{Comparison with previous work. Gardner2017 and Gardner2019 are regression-based methods. EMlight is a generation-based method. Annotation denotes required manual annotations. The evaluation metrics include Angular Error, RMSE, si-RMSE and FID. M, S, and D denote mirror, matte silver, and diffuse material spheres, respectively. $\downarrow$ denotes that smaller is better.}\label{tab:Comparisons}
\begin{center}
\scalebox{0.9}{
\begin{tabular}{c|ccc|ccc|ccc|ccc}
\hline
\multirow{2}{2.5cm}{\centering\textbf{Methods}} &\multicolumn{3}{c|}{\textbf{Gardner2017 \cite{gardner2017learning}}} &\multicolumn{3}{c|}{\textbf{Gardner2019 \cite{gardner2019deep}}} &\multicolumn{3}{c|}{\textbf{EMlight \cite{zhan2020emlight}}}&\multicolumn{3}{c}{\textbf{StyleLight}}\\
\cline{2-13}
                            ~&M &S &D &M &S &D &M &S &D &M &S &D\\
\hline
Annotation & \multicolumn{3}{c|}{lighting sources} &\multicolumn{3}{c|}{depth}&\multicolumn{3}{c|}{No}&\multicolumn{3}{c}{No}\\
\hline
Angular Error$\downarrow$ & 6.68   &6.03 &5.47  &4.63 	&3.91	&3.24 &4.67 &3.47	&2.59 &\textbf{4.30}  	&\textbf{2.96}	&\textbf{2.41}\\
RMSE$\downarrow$ & 0.70    &0.38 &0.19  &0.85	&0.49	&- &0.59  	&0.33	&0.17 &\textbf{0.56} 	&\textbf{0.30}	&\textbf{0.15} \\
si-RMSE$\downarrow$& 0.65   &0.35  & 0.15  &0.70		&0.40	&0.14 &0.58	&0.31	& 0.12&\textbf{0.55}	&\textbf{0.29}	&\textbf{0.11}\\
\hline
\hline
FID$\downarrow$ & \multicolumn{3}{c|}{307.5} &\multicolumn{3}{c|}{344.1}&\multicolumn{3}{c|}{263.8}&\multicolumn{3}{c}{\textbf{137.7}}\\
\hline

\end{tabular}}
\end{center}
\end{table*}

\subsection{Qualitative Evaluation}
\label{subsec:qualitative}
In this section, we provide visual results to show the effectiveness of the proposed StyleLight model, including visual comparison with state-of-the-art lighting estimation methods and lighting editing.

\begin{figure*}[t]
  \centering
  \includegraphics[width=\linewidth]{vis_pano_comparison.pdf}
  \caption{Visual comparisons of our method with several SOTA methods.}\label{fig:vis_comparision}
\end{figure*}

\begin{figure*}[ht]
  \centering
  \includegraphics[width=\linewidth]{render_comparison2.pdf}
  \caption{Visual comparisons on object insertion and lighting. For each input LDR LFOV image, the four different methods estimate illumination maps (at the top-left of LFOV images) for rendering newly inserted objects. }\label{fig:object_insertion}
\end{figure*}

\noindent
\textbf{Visual comparison with the state-of-the-art lighting estimation methods.}
As depicted in \textbf{Fig. \ref{fig:vis_comparision}}, the predicted illumination maps by StyleLight are similar to the ground truth, and the proposed method obtains high-quality panoramas. Compared with the ground truth, Gardner2017 and Gardner2019 highlight strong lighting sources, which are not suitable to render mirror materials. EMlight considers both low-intensity and high-intensity lighting, but it obtains blurred images and it is difficult to train because of the unstable regression. Our StyleLight can synthesize panoramas of better quality, which can be further used for mirror rendering and lighting editing.

\noindent
\textbf{Visual comparison with the state of the art on object insertion and lighting.} We further compare StyleLight with state-of-the-art methods on object insertion and lighting. \textcolor{black}{The predicted HDR panorama is a sphere. The object is placed at the center of the panorama and a warping operator \cite{gardner2017learning} is used when inserted objects are not at the center of a panorama. Finally, image-based rendering is applied. The object insertion technique is similar to EMlight.} The four different methods estimate illumination maps for rendering newly inserted objects. As shown in \textbf{Fig. \ref{fig:object_insertion}}, compared with nearby objects in LFOV images, the inserted objects by Gardner2017 and Gardner2019 tend to lose low-intensity lighting information and generate inaccurate shadows. Thanks to the unique design of coupled dual-StyleGAN that integrates both HDR and LDR panoramas into a unified framework, StyleLight achieves a better visual outputs.

\noindent
\textbf{Lighting editing with StyleLight.} As shown in \textbf{Fig. \ref{fig:lighting_editing}}, given a panorama (the first column), we can remove lights in the second column and add new lights in the third column with the method presented in Sec.~\ref{subsec:light_edit}.

\begin{figure*}[ht]
  \centering
  \includegraphics[width=0.9\linewidth]{lighting_editing2.pdf}
  \caption{Lighting editing with StyleLight. For each original panorama (Row 1), StyleLight removes a light (red box, Row 2) and then adds a new light (green box, Row 3). }\label{fig:lighting_editing}
\end{figure*}

\noindent
\textbf{Effectiveness of the Coupled Dual-StyleGAN structure.}
We conduct an ablation study to analyze the effectiveness of the coupled dual-StyleGAN framework. As shown in \textbf{Fig. \ref{fig:hdrldr_vs_hdr_only}}, the first row shows the quality of StyleLight that uses two discriminators to learn LDR and HDR distributions. The second row shows the LDR panorama quality by using one discriminator to train HDR synthesis (denoted as HDR-only) and the LDR panoramas are computed by tone mapping. It is observed that LDR panoramas from the `HDR-only' suffer from severe distortion. This ablation demonstrates the effectiveness of the coupled dual-StyleGAN structure.
\begin{figure*}[ht]
  \centering
  \includegraphics[width=0.85\linewidth]{StyleLight_vs_HDRonly_iters.pdf}
  \caption{Effectiveness of the coupled dual-StyleGAN structure. The dual-StyleGAN generates both HDR and LDR panoramas. HDR-only directly generates HDR panoramas. }\label{fig:hdrldr_vs_hdr_only}
\end{figure*}

\begin{table}[ht]
\caption{Effectiveness of the focal-masked loss function. M, S, and D denote mirror, matte silver, and diffuse material spheres, respectively. $\downarrow$ denotes that smaller is better.}\label{tab:focal_masked}
 \centering
 \begin{tabular}{c|c|ccc|ccc}
 \hline
\multirow{2}{2.5cm}{\centering \textbf{ {metrics}}}&\textbf{baseline} [\textcolor{green}{20}]&\multicolumn{3}{c|}{\textbf{masked}}&\multicolumn{3}{c}{\textbf{focal-masked}}\\
\cline{2-8}
 ~& N/A & M & S & D & M & S & D \\
 \hline
   RMSE$\downarrow$& N/A & 0.57 & 0.31 & 0.16 & \textbf{0.56} & \textbf{0.30} & \textbf{0.15}\\
 si-RMSE$\downarrow$&  N/A & 0.56 & 0.30 & 0.12 & \textbf{0.55} & \textbf{0.29} & \textbf{0.11}\\
 \hline
 \end{tabular}
 \end{table}

\begin{wrapfigure}{r}{0.5\textwidth}
  \begin{center}
    \includegraphics[width=0.48\textwidth]{user_study.pdf}
  \end{center}
  \caption{User preference.}\label{fig:user_study}
\end{wrapfigure}

\noindent
\textcolor{black}{\textbf{Effectiveness of the focal-masked loss function.} We conduct an ablation study for the new loss in Eq. \ref{eq:gan_finetune_new}. We analyze RMSE and si-RMSE as shown in Table \ref{tab:focal_masked}. The existing GAN inversion is unavailable to LFOV images. We first develop a masked version and then extend it to a focal-masked version. The table shows the improvement of the proposed focal-masked loss function.}

\noindent
\textbf{User Study.} We conduct a user study to analyze predicted illumination maps.  \textbf{Fig. \ref{fig:user_study_example}} shows two examples of the user study. \textcolor{black}{For each example, we insert an object into an image. We render the inserted object with illumination maps predicted by four methods}. Users were asked to pick the images that were more closer to the images rendered by ground-truth HDR panorama. We rendered different objects under 10 scenes, and 100 users participated in the study. Results are shown in \textbf{Fig. \ref{fig:user_study}}. We can observe that 52.1\% prefer our rendered results, which significantly outperform other methods.

\begin{figure*}[ht]
  \centering
  \includegraphics[width=\linewidth]{user_study_example.pdf}
  \caption{For each row, the inserted objects are rendered by HDR illumination maps estimated by four methods. Which one do you think is more closer to the object rendered by the ground-truth HDR illumination map? \\\begin{turn}{180}Second row: (A)Gardner 2019, (B)Ours, (C)Gardner 2017, (D)EMlight\end{turn}
  \\\begin{turn}{180}
First row: (A)EMlight, (B)Gardner 2017, (C)Ours, (D)Gardner 2019;\end{turn}}\label{fig:user_study_example}
\end{figure*}

\section{Conclusion}
\label{sec:conclusion}
We have presented a coupled dual-StyleGAN panorama synthesis network ({StyleLight}) for lighting estimation and editing. StyleLight offers high-quality lighting estimation that generates the full indoor panorama lighting from a single  LFOV image. With the trained StyleLight, we propose a structure-preserved lighting editing method that enables flexible lighting editing. Extensive experiments demonstrate that StyleLight achieves superior performance over state-of-the-art methods on indoor lighting estimation and also enables promising lighting editing on indoor HDR panoramas.

\noindent
\textcolor{black}{\textbf{Limitation.} StyleLight only considers indoor scenes in this paper. When applied to outdoor scenes, StyleLight needs a new set of hyper-parameters because lighting sources (e.g., area and intensity) are different between outdoor scenes and indoor scenes. An intuitive idea is to develop a generalizable method that adaptively tailors for various scenes, which is left to future research.}

\noindent
\textbf{Acknowledgement.} This work is supported by NTU NAP, MOE AcRF Tier 2 (T2EP20221-0033), and under the RIE2020 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s).

\clearpage
\bibliographystyle{splncs04}