\section{Introduction}
\vspace{-3mm}

The  European Space Agency (ESA) Laser Interferometer Space Antenna (LISA) is the first mission designed for gravitational wave detection in space in the frequency range 2$\times$10$^{-4}$ - 10$^{-1}$ Hz \citep{2017arXiv170200786A}.
LISA consists of three identical spacecraft (S/C) placed at the corners of an equilater triangle of 2.5 million km of side length corresponding to the arm of the interferometer. The LISA satellite constellation will be inclined at an angle of 60 degrees on the ecliptic plane trailing Earth in its orbit around the Sun at fifty million kilometer distance.
The S/C triangular formation will rotate yearly clockwise with the center of mass of the formation remaining on the ecliptic. Free-falling cubic test masses (TMs)  of gold and platinum constitute the  mirrors of the interferometer. LISA is scheduled for launch in 2035 at the maximum of the solar cycle 26 during a positive polarity epoch of the global solar magnetic field (GSMF; when the magnetic field lines exit from the Sun North Pole).

The  LISA Pathfinder (LPF)  \citep{{Antonucci:2012:12/124014},{PhysRevLett.116.231101},{2018PhRvL.120f1101A}} mission designed for the testing of the LISA instrumentation  was launched on December 3, 2015 at 4:04 UTC from the near equatorial cosmodrome in Kourou (French Guiana). The LPF S/C entered its final Lissajous orbit around the Sun-Earth first Lagrangian libration point (L1) at the end of January 2016.
The S/C orbit was quasi-elliptic with minor and major axes of $5\times 10^{5}\,\mathrm{km}$ and $8\times 10^{5}\,\mathrm{km}$, respectively. On July 18, 2017, LPF received its last command.
The Pathfinder mission of LISA hosted a diagnostics subsystem of instruments \citep{2009CQGra..26i4005C} to carry out a continuous monitoring of 1) the magnetic field near the TMs, 2) the overall high-energy particle flux incident on the S/C \citep{{2017JPhCS.840a2037G},{apj1},{10.1093/mnras/staa830}} and 3) the temperature stability and gradient \citep{10.1093/mnras/stz1017}.
As a result, important lessons were learned about spurious forces induced on the TMs by magnetic field variations and high-energy particles. The LPF platform magnetometers were located within the S/C hull to monitor  the magnetic field  generated  by wiring and instruments
while science magnetometers are typically placed outside the hull, mounted on extensible booms. As a result, science magnetometers increase mission total costs \citep{Nishio_2007} and mass balance.
To pursue mission cost reduction strategy, space agencies are carrying out many efforts to enable scientists to infer  the interplanetary magnetic field (IMF) data from platform magnetometers present in a wide variety of space missions like, {\it e.g.}, RHESSI \citep{2002SoPh..210....3L}, TRACE\footnote{https://directory.eoportal.org/web/eoportal/satellite-missions/t/trace}, PICARD\footnote{https://directory.eoportal.org/web/eoportal/satellite-missions/p/picard} and IRIS\footnote{https://directory.eoportal.org/web/eoportal/satellite-missions/i/iris}.

Previous works on magnetic field observations gathered on board the LPF S/C is reported in \citet{Mateos2015Thesis} and \citet{10.1093/mnras/staa830}.
In the present paper we illustrate an empirical method to disentangle the LPF on-board generated magnetic field from the IMF component.
The same approach may be considered for LISA and other missions for which the on-board dominant magnetic field would not result correlated with the variations of the IMF. The survey of the transit of interplanetary structures would contribute to both mission magnetic noise evaluation and space weather science investigations.

In addition to the magnetic noise, a further source of spurious Coulomb forces on the LISA TMs is associated with the charging process due to particles of galactic and solar origin with energies $>$ 100 MeV(/n) penetrating or interacting in  approximately 15 g cm$^{-2}$ of material surrounding the interferometer mirrors.
A particle detector (PD) was hosted on board the LPF satellite to estimate these forces.
The PD consisted of two $\sim 300$ $\mu$m thick silicon wafers
of  1.40 x 1.05 cm$^2$ area,   separated by 2 cm  and placed in
a telescopic arrangement. For particles with  energies $>$ 100 MeV n$^{-1}$ with an isotropic incidence on each silicon layer,
the instrument geometrical factor  was  found  energy independent and equal to  9 cm$^2$ sr for a total of 17 cm$^2$ sr.
In coincidence mode (particles traversing both silicon wafers), the geometrical factor was  about one tenth
of this  value. The silicon wafers were placed inside a shielding copper  box of 6.4 mm thickness meant to  stop particles with energies smaller than 70 MeV
n$^{-1}$. This conservative choice was made in order not to underestimate the overall flux of particles charging the TMs. The PD  allowed for the counting of
 protons and helium nuclei
 traversing  each silicon layer and for the measurement of ionization energy losses of particles in coincidence mode.
The maximum allowed particle  counting rate
was 6500 counts s$^{-1}$ on both silicon wafers, corresponding to an event integrated  proton fluence of   10$^8$
protons cm$^{-2}$ at energies $>$ 100 MeV.
In coincidence mode up to 5000 energy deposits per second could be stored on the on board computer.
Hourly integrated cosmic-ray data gathered with LPF presented 1\% statistical uncertainty  thus allowing us for the study of the galactic cosmic-ray (GCR) short-term variations with periodicities typical of the LISA band of sensitivity \citep{{apj1},{apj2},{paper3}}.
Unfortunately, no solar energetic particle (SEP)  events  with  fluences  overcoming that of GCRs  above a few tens of MeV n$^{-1}$ were observed during the LPF mission operations.
Measurements gathered with LPF \citep{2017PhRvL.118q1101A} in the spring 2016 indicated that the net charging of the TMs generated by cosmic rays was of tens of charges per second while the charging noise was of the order of one thousand charges per second. Monte Carlo simulations resulted in agreement with the measured net TM charging while the estimated noise appeared three-four times smaller than observations \citep{2015CQGra..32c5001G}. In addition, the simulations of the TM charging noise returned 10730 e s$^{-1}$ and 68000 e s$^{-1}$ during SEP events characterized by proton fluences of 4.2$\times$10$^7$ cm$^{-2}$ and 10$^9$ cm$^{-2}$ above 30 MeV, respectively \citep{ara05,2005CQGra..22S.319V}.
Recently, we have shown that the mismatch between simulated and observed charging noise was associated with the lack of propagation of a few eV electrons in the simulations \citep{{2020ApSS..51245734V},{2021CQGra..38d5013G},{2021CQGra..38n5005V}}.
For reliable TM charging estimates during the LISA mission operations we will benefit of improved simulation toolkits but it will be also of primary importance to follow the evolution of SEP events and GCR variations on board each S/C of the constellation.
The ESA Next Generation Radiation Monitor (NGRM; \citeauthor{ngrm}, \citeyear{ngrm}) is supposed to be adopted to this purpose. The NGRM consists of an electron detector and a proton unit. This instrument is optimized for particle measurements in harsh radiation environments and on board the LISA S/C would play a precious role for SEP event short-term forecasting and monitoring. The number of expected SEP events during the LISA mission operations is estimated here with the Nymmik model \citep{{Nymmik1999a},{Nymmik1999b}} as a function of event fluence on the basis of the predicted solar activity during the solar cycle 26 \citep{sing19}. LISA will allow us to carry out for the first time observations of SEPs at about one and twenty degrees  in heliolongitude that separate the three LISA S/C and LISA from near-Earth detectors and neutron monitors, respectively.
Unfortunately, the NGRM presents a small geometrical factor for particles with energies larger than 200 MeV  thus impeding to follow the dynamics of short-term variations of the particle flux of galactic origin in the LISA band of sensitivity with a precision similar to that of LPF.
It would be more than recommended to add a dedicated cosmic-ray detector to the NGRM on board each LISA S/C.
The possible contribution that LISA shall give to Space Weather science investigations has been illustrated in the roadmap of the Italian Space Agency \citep{refId0}.

This manuscript is organized as follows: the LPF magnetometer characteristics are described in Section~II.
Details of the magnetic environment within the LPF S/C are provided in Section~III.
In Section~IV the LPF magnetic field empirical modeling is described. In Section~V the LPF IMF data are compared to those gathered contemporaneously with the Wind mission.
In Section~VI the expected flux of cosmic rays and the occurrence and intensity of SEP events during the time the LISA mission will remain into orbit are estimated. Finally in Section VII we discuss the LISA contribution to Space Weather science investigations with the NGRM and, possibly, with a GCR detector.

\section{THE LISA PATHFINDER MAGNETOMETERS}
\label{section_LPFMAGS}

The Lisa Technology Package (LTP) was placed at the centre of the LPF S/C platform \citep{Anza_2005} with four Billingsley TFM100G4-S 3-axis fluxgate magnetometers (MX, MY, PX and PY) as it is shown in Figure~\ref{fig_lpf_four_mags_map} \citep{Mateos2015Thesis}.
Fluxgate magnetometers are considered optimum for space applications since they provide highly precise magnetic field measurements and coping with several design constraints ({\it e.g.}, mass, cost limitation, robustness and low-power consumption).
Each magnetometer was made of three distinct $1$-axis magnetic sensors aligned to form a Cartesian tern. Each sensor was provided with a primary inner drive consisting of a high permalloy magnetic core material and a secondary sensing (pickup) coil. The signal sensed by the pickup coil is amplified and demodulated \citep{Mateos2015Thesis}. The obtained constant part of the output signal is, then, integrated and sent back to the pickup coil as feedback of the control loop with the purpose of widening the measurement range.
When the sensor was in operation, a magnetic field  was produced by an exciting coil that periodically saturated the core in opposite directions.
The magnetometers were properly placed at about $19\ \mathrm{cm}$ distance from each TM \citep{2007JPhCS..66a2003A}, and arranged in a cross-shaped configuration with two orthogonal arms of $\simeq 74\,\mathrm{cm}$ length \citep{2009CQGra..26i4005C,2010JPhCS.228a2038D}.
The magnetometer sensing axes were aligned with the LTP reference frame {\red $\pmb{\mathrm{R}}_\mathrm{LTP}$} (Fig.~\ref{fig_lpf_four_mags_map}). The characteristics of the magnetometers are summarized in Table~\ref{tab0}.
\begin{figure}[t!]

	\hspace{-0.3cm}
	\centering
	\includegraphics[width=7.5cm,height=7cm]{./images/image_mod6f.jpeg}
	\caption{Sketch of the position of the four 3-axis magnetometers placed on board LPF (MX, MY, PX and PY) with respect to the gravitational reference sensors (GRS1 and GRS2) enclosing the two LPF TMs. The magnetometers were arranged in a cross-shaped configuration. MX and PX were placed at the edges of the arm joining the TM centers ($x$ direction) where the negative (M) and the positive (P) sides are meant with respect to the middle point between the two GRSs, respectively. Similarly, MY and PY were placed at the edges of the second arm ($y$ direction) where M and P have the same meaning as for the $x$-axis. The $z$-axis was chosen to complete the right-handed tern of the LTP reference frame, $\pmb{\mathrm{R}}_\mathrm{LTP}$.}
	\label{fig_lpf_four_mags_map}
\end{figure}
\begin{table}[tbh]
	\centering
	\resizebox{6.5cm}{!}{
		\begin{tabular}{lc}\hline\hline
			Parameter & Value \\ \hline
			Field range & $\pm 60\ \mathrm{\mu T}$ \\
			Temperature coefficient & $1.2\, \mathrm{nT\, K^{-1}}$ \\
			Noise density & $<\! 100\ \mathrm{pT\, Hz^{-1/2}}\, @\, 1 \mathrm{Hz}$ \\
			Linearity & $0.015\%$ \\
			Sensitivity & $166.7\ \mathrm{\mu V\, nT^{-1}}$ \\
			Offset voltage & $25\ \mathrm{mV}$ ($150\ \mathrm{nT}$)\\
			Operating temperature & $-55\ \mathrm{^{\circ} C}$ to $+85\ \mathrm{^{\circ} C}$ \\
			Susceptibility & $\pm 20\,\mathrm{nT}$ with $0.5\ \mathrm{mT}$ \\\hline\hline
		\end{tabular}
	}
	\caption{LPF magnetometer characteristics \citep{Mateos2015Thesis}.}
	\label{tab0}
\end{table}

The magnetic subsystem, including the four magnetometers and the acquisition electronics, was tested before the mission launch at the Universidad Politecnica de Madrid \citep{Mateos2015Thesis}. The calibration of the magnetometers  was carried out with a 3-D Helmholtz coil \citep{6740059}.
By denoting with $\mathbf{B}^\mathrm{raw}_{m}[t]$ the raw magnetic field measurements of each magnetometer:
\begin{eqnarray}
\label{eq_Bcomponents}
\displaystyle \mathbf{B}^\mathrm{raw}_{m}[t] &=& \mathrm{B}^\mathrm{raw}_x [t] \, \hat{x} + \mathrm{B}^\mathrm{raw}_y [t] \, \hat{y} + \mathrm{B}^\mathrm{raw}_z [t] \, \hat{z},
\end{eqnarray}
where $m$ ranges from $1$ to $4$ to indicate the four magnetometers MX, MY, PX and PY, respectively (Fig.~\ref{fig_lpf_four_mags_map}).
This calibration procedure allowed us to determine for each magnetic field component of individual instruments the parameters associated with   gain,  mis-alignment and  bias.
The parameters relative to the gain and to the axis mis-alignment fill two    $3\times 3$ matrices: $\mathbf{s}_m$ and  $\mathbf{T}_m$, respectively.
Finally, the measurement offset is represented by a $3\times 1$ vector $\mathbf{b}_m$ as indicated below:
\begin{eqnarray}
\label{eq_BINS_to_BMAG}
\displaystyle
\mathbf{B}^\mathrm{MAG}_{m}[t] &=& \mathbf{s}_{m} \, \mathbf{T}_{m}\, \mathbf{B}^\mathrm{raw}_{m}[t] + \mathbf{b}_m,
\end{eqnarray}
where $\mathbf{B}^\mathrm{MAG}_{m}[t]$ indicates the calibrated magnetic field measurements.

\section{LPF SPACECRAFT MAGNETIC FIELD}

\label{section_magnetic_monitoring}
The LPF magnetometer calibrated datasets were sampled at 2$\times$10$^{-1}$ Hz and downsampled at $3\times 10^{-4}\,\mathrm{Hz}$. Aliasing was prevented by applying a first-order bidirectional low-pass filter.
The intensity of the magnetic field measured by MX, MY, PX and PY magnetometers during the LPF mission are shown in Figure~\ref{fig_lpf_four_mags}.
Observations varied by almost a factor of two between $700\,\mathrm{nT}$ and about $1300\,\mathrm{nT}$.
The MX, MY and PX magnetometer recorded magnetic field values ($\mathbf{B}_\mathrm{MX}^\mathrm{MAG}$, $\mathbf{B}_\mathrm{MY}^\mathrm{MAG}$ and $\mathbf{B}_\mathrm{PX}^\mathrm{MAG}$, respectively) ranging between $800\,\mathrm{nT}$ and $1300\,\mathrm{nT}$  with sudden variations between different magnetic field intensities.
Conversely, the PY magnetometer recorded stable values of the magnetic field around $700\,\mathrm{nT}$ during the whole mission lifetime ($\mathbf{B}_\mathrm{PY}^\mathrm{MAG}$).
Magnetic field patterns and sudden changes were not ascribable to the IMF that was observed to present typical values in the $1-25\, \mathrm{nT}$ range \citep{2019ApJ...874..167A}.
In general, the magnetometers hosted on board the LPF S/C  sensed a larger noise along the three cartesian axes with respect to the Magnetic Field Investigation (MFI) instrument hosted on board the Wind mission also orbiting around L1 \citep{1995SSRv...71..207L} during the same period of LPF. Despite that, the approach presented here allows us to disentangle the IMF from the magnetic field generated on board the LPF S/C.
\begin{figure}[t!]
    \centering

	\hspace{-0.3cm}
	\includegraphics[width=8.5cm,height=8cm]{./images/LPF_fig2_20190429b_200_MOD.jpeg}\\
	\caption{Magnetic field intensities recorded by the four LPF 3-axis fluxgate magnetometers during the time the S/C was orbiting around the Lagrange point L1 from  February 16, 2016 through June 30, 2017. Datasets were sampled at $2\times 10^{-1}\,\mathrm{Hz}$ and downsampled at $3\times 10^{-4}\,\mathrm{Hz}$.
	Time is indicated in days of mission (DoM)  after the LPF launch (4:04 UTC, December 3, 2015).}
	\label{fig_lpf_four_mags}
\end{figure}
The power spectral densities (PSDs) of magnetic field measurements carried out with LPF during the whole mission operation period are reported in Figure~\ref{fig_lpf_four_mags_components_PSD}.
In the same figure the PSDs of the IMF measurements gathered with MFI are reported for comparison.
It can be observed that the onboard noise overcomes that of interplanetary origin at low frequencies while it appears consistent with it above $10^{-5}\,\mathbf{Hz}$.
\begin{figure}[thp]
   \centering

	\hspace{-1.3cm}
	\includegraphics[width=8.5cm,height=8cm]{./images/Figure_makePSDplot_20200907T001505_300_MOD2.jpeg}
	\includegraphics[width=8.5cm,height=8cm]{./images/Figure_makePSDplot_20200907T001605_300_MOD2.jpeg}\\
	\includegraphics[width=8.5cm,height=8cm]{./images/Figure_makePSDplot_20200907T001704_300_MOD2.jpeg}\\
	\caption{PSDs of LPF and Wind 3-axis magnetometer simultaneous measurements (from February 20, 2016 through  June 21, 2017). All measurements were reported in the $\pmb{\mathrm{R}}_\mathrm{LTP}$ reference frame for proper comparison.}
	\label{fig_lpf_four_mags_components_PSD}
\end{figure}

\section{PARAMETERIZATION OF THE MAGNETIC FIELD MEASUREMENTS GATHERED WITH LPF}
\label{section_model_parameterization}

The timeseries of the calibrated magnetic field  measurements gathered on-board LPF have been parameterized as follows:
\begin{eqnarray}
\label{eq_B_MAG_1}
\mathbf{B}^\mathrm{LPF}_{m}[t] =  \mathbf{B}^\mathrm{EW}_{m}[t] + \mathbf{B}^\mathrm{ED}_{m}[t] + \mathbf{B}^\mathrm{M}_{m}[t]   + \mathbf{B}^\mathrm{IMF}_{m}[t] + \mathbf{v}^c_m [t] \ .
\end{eqnarray}
In the above equation the magnetic field measured on board the LPF S/C is considered to consist of different contributions generated by the main currents flowing through wiring and electronics in the satellite ($\mathbf{B}^\mathrm{EW}_{m}[t]$ and $\mathbf{B}^\mathrm{ED}_{m}[t]$, respectively), by the magnetized materials ($\mathbf{B}^\mathrm{M}_{m}[t]$) and by the IMF ($\mathbf{B}^\mathrm{IMF}_{m}[t]$). Finally, $\mathbf{v}^c_m [t]$ represents the residue.
In the case the considered contributions to the LPF magnetic field observations are not overcome by other neglected components, the residue
is expected to be negligible and mainly associated with the measurement noise.

No measurement correction due to temperature variations were considered.
As a matter of fact, for more than 500 days after the mission launch, the temperature remained nearly constant around a typical value of $297.15\pm0.75\,\mathrm{K}$ in the LPF S/C \citep{2019MNRAS.486.3368A}.
This temperature variation corresponded to a magnetic field  offset smaller  than $0.9\,\mathrm{nT}$ (Tab.~\ref{tab0}).
Seldom during the S/C commissioning phase  and in particular for a period of 20 days after April 29, 2017 (DoM 513), the temperature was purposely maintained between $281\,\mathrm{K}$ and $285\,\mathrm{K}$. Magnetic field measurements gathered during these  intervals of time are shown in the following but excluded from the analysis.

\subsection{Magnetic field generated by wiring and electronics on board the LPF spacecraft}
\label{subs_a}

The main contribution to the  magnetic field measured  with the  MX, MY, PX and PY magnetometers in the LPF S/C is associated with currents of $10-20\,\mathrm{A}$ flowing through the main satellite and solar array buses ($I_{bus}$ and $I_{sar}$, respectively). One secondary bus ($I_{aux,A}$) distributing a current smaller than $1\,\mathrm{A}$ was also found to play an important role.
Figure~\ref{fig_lpf_channels} shows the timeseries of the currents flowing in the buses indicated above during the mission operations.
\begin{figure}[htbp]
  \centering
  \includegraphics[width=8.5cm,height=8.5cm]{./images/LPF_fig3_20190429b_300_MOD2_3busses.jpeg}\\
  \caption{LPF main bus current ($I_{bus}$), solar array bus current ($I_{sar}$) and auxiliary power supply A current ($I_{aux,A}$) during the LPF mission operations from February 16, 2016 through June 30, 2017. At DoM near 300 an isolated overload is observed. Data in the corresponding interval of time have not been included in this analysis.}
  \label{fig_lpf_channels}
\end{figure}
As an example, the correlated trend of the magnetic field measured by the MY magnetometer along the LTP $y$-axis and the main LPF current bus trend is shown in Figure~\ref{fig_ibus_mag}.
The IMF intensity shows both hourly variability and sudden jumps associated with the bus current changes.
\begin{figure}[htbp]
  \centering
  \includegraphics[width=8.5cm,height=8cm]{./images/Figure_BvsIBUS_MYy20200926T235755_300_MOD.jpeg}\\
  \caption{LPF  magnetic field measured by the MY magnetometer along the LTP $y$-axis (blue) and main bus current (red). The two physical quantities appear correlated and show common sudden changes.}
  \label{fig_ibus_mag}
\end{figure}

The magnetic field components generated by the circuitry in each measurement point  are observed along the axes of the $m$-th magnetometer in the $\pmb{\mathrm{R}}_\mathrm{LTP}$ reference frame. %
At the magnetometer locations, each current bus produces a magnetic field proportional to the current flowing through the circuit that can be estimated with  the Biot-Savart equation. As a result, the sensed magnetic field change is linearly proportional to the current change in the circuitry.

These considerations apply to the magnetic field generated by both wiring
and electronics.
\iffalse
\dgreen
Orbiting around L1, the S/C was constantly illuminated by the Sun and the solar array on board LPF provided a continuous supply of power.
This arrangement allowed to reduce the number of needed battery cells thus lowering the on-board weight and offered the possibility of using an on-board unregulated power conditioning and distribution unit (PCDU) \citep{2008ESASP.661E..45S}.
The LPF PCDU maximizes the power generation and the power storage while handling power distribution to S/C subsystems.
\fi
With respect to the magnetic field generated by the electronics, LPF hosted an unregulated power conditioning and distribution unit (PCDU) \citep{2008ESASP.661E..45S} which showed a nearly constant power conversion efficiency, $\eta$, of 92.5\% as it is shown in Figure~\ref{fig_pcdu_efficiency}.
The main supply bus voltage was stabilized very efficiently at $V_\mathrm{bus} = 27.903\pm 0.002\, \mathrm{V}$ by a Li-Ion  battery  which, due to a constant solar illumination, was handled at a fix charge/discharge point.
Sudden changes of the power conversion efficiency and current variations were observed during electronics configuration switches only.
Table~\ref{tab1} reports the complex changes of electronics configurations during the mission operations.
The S/C was driven by the drag-free attitude control system (DFACS) or by the disturbance reduction system (DRS).
Both these attitude control systems were used to command with forces and torques the S/C and the two proof TMs \citep{PhysRevLett.116.231101}.
The DFACS controlled a set of cold-gas thrusters \citep{MorrisEdwardsThrusters2013} and the electronics for the TM electrostatic actuation.
The DRS utilized a distinct control scheme implemented by a dedicated on-board computer with electronics and a second set of $8$ colloidal micro-newton thrusters (CMNTs) \citep{2018PhRvD..98j2005A}.

During time periods indicated by “DFACS+”, the DRS related subsystems were left switched on, even when the actuation was assigned to the DFACS.
For instance, this configuration was applied during the months of July and August 2016.
Analogously, the periods indicated by ``DRS+'' were those associated with actuation assigned to the DRS with the DFACS  powered on.

\begin{figure*}[htbp]
  \centering
  \includegraphics[width=17cm,height=6.0cm]{./images/PCDU_PowerTransferEfficiencyNew2_20200901T210610_300_PRINT3_MOD3.jpeg}\\
  \caption{PCDU power transfer efficiency, $\eta[t]$.}
  \label{fig_pcdu_efficiency}
\end{figure*}

The timespans were selected on the basis of the electronic switches, the current bus trends and the power transfer efficiency $\eta[t]$ in order to select proper magnetic field  pattern changes (Fig.~\ref{fig_lpf_four_mags}).
\begin{table}[tbhp]
	\centering
	\resizebox{8.5cm}{!}{
		\begin{tabular}{|c|c|c|c|c|}\hline
			Timespan & Confs & Start Time & Stop Time & Duration\\
			 & &{[UTC]} & {[UTC]} & {[Days]}\\ \hline \hline
			1  & DFACS & 2016-02-20, 22:20 & 2016-05-04, 23:45 & 74\\
			2  & DFACS & 2016-05-11, 16:35 & 2016-06-27, 06:00 & 46\\
			3  & DRS   & 2016-07-12, 09:55 & 2016-09-23, 00:05 & 72\\
			4  & DRS   & 2016-09-26, 10:00 & 2016-10-21, 16:40 & 25\\
			5  & DFACS+& 2016-10-31, 16:35 & 2016-12-07, 03:35 & 36\\
			6  & DFACS+& 2016-12-08, 13:45 & 2017-01-13, 10:00 & 35\\
			7  & DFACS+& 2017-01-17, 06:30 & 2017-01-22, 23:45 & 5\\
			8  & DFACS+& 2017-01-25, 00:45 & 2017-02-01, 11:45 & 7\\
			9  & DFACS+& 2017-02-02, 14:15 & 2017-02-10, 19:55 & 8\\
			10 & DFACS+& 2017-02-13, 17:35 & 2017-03-14, 06:00 & 28\\
			11 & DRS+  & 2017-03-18, 10:00 & 2017-04-05, 10:50 & 18\\
			12 & DFACS+& 2017-04-05, 12:55 & 2017-04-12, 04:05 & 6\\
			13 & DRS+  & 2017-04-13, 02:15 & 2017-04-29, 07:30 & 16\\
			14 & DFACS+& 2017-04-30, 00:05 & 2017-05-23, 07:30 & 23\\
			15 & DFACS+& 2017-05-24, 20:25 & 2017-06-21, 20:00 & 27\\\hline
		\end{tabular}}
	\caption{LPF electronics configuration during different time intervals.
	}
	\label{tab1}
\end{table}

In conclusion, according to the Biot-Savart law and the superposition principle \citep{yu2013}, the magnetic field intensity produced at each magnetometer position ($m$) by wiring and electronics is empirically parameterized as indicated in the following equation:
\begin{eqnarray}\nonumber
\label{eq_BH_BBS}
\displaystyle \mathbf{B}^\mathrm{EM}_{m}[t] &=& \mathbf{B}^\mathrm{EW}_{m}[t] + \mathbf{B}^\mathrm{ED}_{m}[t] =\\\nonumber
&=& \sum_{j=1}^{3} \frac{\mu_0}{4 \pi} \int_{\mathcal{C}_j} \frac{I_j [t]}{r_{m,j}^2} d\mathbf{l}_j \times \mathbf{u}_{m,j} = \\
&=& \sum_{j=1}^{3}\, \mathbf{a}_{m,j}\,  I_j [t]
\end{eqnarray}
where $d\mathbf{l}_j$ represents the infinitesimal current element along the $\mathcal{C}_j$ integration path associated with the $j$-th bus; $r_{m,j}$ indicates the distance of each $d\mathbf{l}_j$ current segment of each current bus from the $m$-th position of each magnetometer and $\mathbf{u}_{m,j}$ is the unit vector associated with $\mathbf{r}_{m,j}$ indicating the distance from the bus element to the point where the field is calculated. The $\mathbf{a}_{m,j}$ parameters allow to account for the magnetic field generated by the $j$-th bus current, $I_j [t]$. These parameters are estimated in Section~V.

\subsection{LPF  magnetic field generated by magnetized materials}
\label{subs_b}

Two distinct independent micro propulsion systems were hosted on board the LPF S/C: cold-gas micro propulsion thrusters, as inherited from the Gaia mission \citep{2011AcAau..69..822S}, and $8$ CMNTs developed by Busek and managed by NASA at the Jet Propulsion Laboratory \citep{ZiemerEtAl2006} which were part of the DRS.
The DRS included four subsystems: the Integrated Avionics Unit (IAU), two clusters of four CMNTs each, the Dynamic Control Software (DCS) and the Flight Software (FSW).
CMNTs were commanded by applying an electric potential to a charged liquid in order to emit a stream of
droplets for the thrust. During the mission, a major contribution to the magnetic field disturbance was found to be strongly correlated with the CMNT fuel mass ejection (Fig.~\ref{fig_lpf__fuelmass_mx_comparison}).
\begin{figure}[htbp]
  \centering
  \includegraphics[width=8.5cm,height=8cm]{./images/immagineRicostruitaCMNTvsMXx_MOD.jpeg}\\
  \caption{Total CMNT fuel mass ejected and on-board induction magnetic field timeseries $\mathbf{B}^\mathrm{MAG}_{\mathrm{MX},x}[t]$ (in blue and red, respectively) recorded by the MX magnetometer along the LTP $x$-axis.}
  \label{fig_lpf__fuelmass_mx_comparison}
\end{figure}
A  magnetized assembly pushes out the propellant by moving out from the magnetometers and generating a slowly decreasing magnetic field \citep{10.1093/mnras/staa830}. This decrease is mostly sensed by the MX and PX magnetometers along the LTP $x$-axis.
On the basis of the correlated behavior between the on-board magnetic field variation and the fuel mass ejection timeseries, the contribution to the on-board magnetic field arising from the CMNT tank moving assemblies is modeled as follows:
\begin{eqnarray}
\label{eq_thrusters}
\mathbf{B}_m^\mathrm{M} [t] = \mathbf{l}_{m}\ M[t]
\end{eqnarray}
where $M[t] = \sum_{i=1}^{8} {C\!M\!N\!T}_i [t]$ accounts for the fuel mass loss obtained by summing up  the fuel mass quantity ejected from the CMNTs (Fig.~\ref{fig_lpf__fuelmass_mx_comparison}). In Equation~\ref{eq_thrusters}, $\mathbf{l}_{m}$ is a parameter tuned via numerical fitting (see Section~\ref{section_bayesian_parameter_estimation}) and expressed in units of $\mathrm{nT\, kg^{-1}}$.

\section{Interplanetary magnetic field monitoring with LISA Pathfinder}

\subsection{Data analysis}
\label{section_bayesian_parameter_estimation}

The magnetic field measured on board the LPF S/C expressed in Equation~\ref{eq_B_MAG_1} can be now parameterized as follows:
\begin{eqnarray}
\label{eq_B_MAG_5}\nonumber
\mathbf{B}^\mathrm{LPF}_{m}[t] &=& \mathbf{B}^\mathrm{EM}_{m}[t] + \mathbf{B}^\mathrm{M}_{m}[t] + \mathbf{B}^\mathrm{IMF}_{m}[t] + \mathbf{v}^c_m [t] \\
&\cong&  \sum_{j=1}^{3} \mathbf{a}_{j,m} I_j[t] + \mathbf{l}_{m}\ M[t] + \mathbf{B}^\mathrm{IMF}_{m}[t] + \mathbf{v}^c_m [t]\ .
\end{eqnarray}

From the above equation, it is possible to estimate the IMF component in the time domain.
The same equation can be written in the frequency domain in order  to remove the colored noise from the data. In the frequency domain each term is indicated with the {\it tilde} symbol and the IMF can be expressed as indicated below:
\begin{eqnarray}
\label{eq_BIMFmodel_freq_applied}
\mathbf{\tilde{B}}^\mathrm{IMF}_{m}[f] &\cong& \mathbf{\tilde{B}}^\mathrm{LPF}_{m}[f] - \sum_{j=1}^{3} \mathbf{a}_{j,m} \tilde{I}_j[f] - \mathbf{l}_{m}\ \tilde{M}[f] + \mathbf{\tilde{v}}^c_m [f]\,
\end{eqnarray}
where $\mathbf{\tilde{v}}^c_m [f]$ is expected to be the overall noise amplitude in case all the main components of the magnetic field have been taken into account.
The LPF Data Analysis toolbox (LTPDA, publicly available at \url{https://www.elisascience.org/ltpda/}) is a toolbox implemented in Matlab (\url{http://www.mathworks.com}) and was specifically developed for the analysis of the LPF mission data \citep{Antonucci_2011}.
A Markov Chain Monte Carlo (MCMC) algorithm was used to estimate the posterior distribution of the model parameters on the basis of the data timeseries provided in input \citep{PhysRevD.90.042003}.
The parameter distribution  $\mathrm{\hat{\theta}}$ was obtained with an iterative least square error estimate after the PSDs of $\tilde{I}_j[f]$ and $\tilde{M}[f]$
were calculated from the
equivalent  timeseries in the time domain.
By indicating with $\theta_m$ the column vector of the input parameters:
{
\begin{eqnarray}
\label{eq_THETA}
\pmb{\mathrm{\theta}}_m = \left[\mathbf{a}_{m,j} \, , \, \mathbf{l}_m \right]^T
\end{eqnarray}
}
the IMF has been estimated by minimizing the residue among collected data and empirical modelization of the LPF S/C on-board magnetic field, as follows:
\begin{eqnarray}
\label{eq_THETA_ESTI}
\pmb{\mathrm{\hat{\theta}}}_m &\simeq& \argmin_{\theta_{m}} \left| \mathbf{\tilde{B}}^\mathrm{MAG}_{m}(\theta_{m})[f] - \mathbf{\tilde{B}}^\mathrm{LPF}_{m}(\theta_{m})[f] \right|\, .
\end{eqnarray}

Separate sets of parameters were estimated for each magnetometer and for each time segment listed in Table~\ref{tab1}.
The fitting program returned  the IMF intensity values in the range $1-25\,\mathrm{nT}$ in line with the observations gathered by the Wind experiment during the LPF mission
lifetime.
Parameter estimates and corresponding uncertainties are reported in Tables~\ref{tab:evalparmag1}, \ref{tab:evalparmag2}, \ref{tab:evalparmag3} and~\ref{tab:evalparmag4}.
As it was recalled in Section~\ref{section_model_parameterization}, the parameters associated with TS13 and TS14 timespans were ignored because of an abrupt fall of the temperature on board the S/C.
\input{tabelle_v7e.input}

\subsection{Comparison of LISA Pathfinder and Wind interplanetary magnetic field data}
In order to test the reliability of our work, the IMF inferred from LPF magnetometer measurements were compared with contemporaneous observations collected by the MFI detector on board the Wind S/C. The orbits of the two S/C are shown in Figure~\ref{fig_lpf_ace_wind_orbits} during the timespans TS1 (from February 20 to May 4, 2016) and TS3 (from July 12 to September 23, 2016), as an example. In particular, we focused on these two timespans  since they were characterized by the longest data taking periods during which the DFACS and DRS controls were actuating the S/C, respectively. The average distance between LPF and Wind S/C was of $1.12 \times 10^6\, \mathrm{km}$ and the IMF is not expected to vary significantly over these length scales \citep{1990pihl.book..183M}.
In Figures~\ref{fig_lpf__TD_TS1_components} and~\ref{fig_lpf__TD_TS3_components} we have compared the contemporaneous LPF IMF estimates and Wind observations gathered during the mentioned intervals of time.
The LPF estimates are in fairly good agreement with the Wind data along each one of the three Cartesian components. It is pointed out that the Wind MFI measurements were rotated in the LTP reference frame, $\pmb{\mathrm{R}}_\mathrm{LTP}$.
Some residual flickering is observed in Figure~\ref{fig_lpf__TD_TS3_components} during the TS3 timespan for the MX magnetometer along the LTP $x$-axis. This  noise is believed to be associated with  a residual contribution of the magnetized moving assemblies (see Section~\ref{subs_b}, \citeauthor{10.1093/mnras/staa830}, \citeyear{10.1093/mnras/staa830}).

In Figure~\ref{fig_lpf__PSD} are reported the PSDs obtained by processing the LPF IMF timeseries presented in Figures~\ref{fig_lpf__TD_TS1_components} and~\ref{fig_lpf__TD_TS3_components}.
The $\mathrm{PSD}$ of $\mathbf{B}^\mathrm{IMF}_\mathrm{m}-\mathbf{B}^\mathrm{IMF}_\mathrm{Wind}$ is shown in yellow in the same figure obtained by subtracting the IMF measurements carried out by MFI (in green) from the LPF IMF data (in red).
By comparing the PSDs of the calibrated LPF magnetic field measurements (in blue) with the PSDs of the IMF measured on board Wind (in green), it is possible to notice that the measurements gathered on board LPF are affected by a larger noise at all frequencies.
The noise difference between LPF estimated IMF and the IMF measured by the Wind MFI goes down 10\% below $4\times10^{-5}\,\mathrm{Hz}$, where the noise of the IMF becomes increasingly dominant.

\begin{figure}[htbp]
\centering
	\includegraphics[width=7cm,height=6.5cm]{./images/orbita3D_20200805T011739_MOD2.jpeg}\\
	\includegraphics[width=7cm,height=6.5cm]{./images/orbita3D_20200805T011910_MOD2.jpeg}
	\caption{LPF and Wind orbits during the timespans indicated with TS1 and TS3.}
	\label{fig_lpf_ace_wind_orbits}
\end{figure}

\begin{figure*}[htbp]
\vspace{-2cm}
  \centering
	\includegraphics[width=16cm,height=24cm]{./images/Folder/makeTDplot_B3IMF_TS1_MOD2.jpeg}
	\caption{Estimated LPF IMF timeseries ($\mathbf{B}^\mathrm{IMF}_\mathrm{MX}$, $\mathbf{B}^\mathrm{IMF}_\mathrm{MY}$, $\mathbf{B}^\mathrm{IMF}_\mathrm{PX}$ and $\mathbf{B}^\mathrm{IMF}_\mathrm{PY}$) and Wind MFI measurements in the $\pmb{\mathrm{R}}_\mathrm{LTP}$ reference frame for the TS1 time interval (see Table~\ref{tab1}).}
	\label{fig_lpf__TD_TS1_components}
\end{figure*}

\begin{figure*}[htbp]
\vspace{-2cm}
  \centering
	\includegraphics[width=16cm,height=24cm]{./images/Folder/makeTDplot_B3IMF_TS3_MOD2.jpeg}
	\caption{Same as Figure~\ref{fig_lpf__TD_TS1_components} for the TS3 time interval.
	}
	\label{fig_lpf__TD_TS3_components}
\end{figure*}

\begin{figure*}[htbp]
  \centering
\vspace{-3.0cm}
	\includegraphics[width=13cm,height=25cm]{./images/Figure_TS1and3_Braw_Bimf_Wimf_Bres_v2_MOD4.jpeg}

	\caption{PSDs of the LPF magnetometer measurements ($\mathbf{B}^\mathrm{MAG}_\mathrm{MX}$, $\mathbf{B}^\mathrm{MAG}_\mathrm{MY}$, $\mathbf{B}^\mathrm{MAG}_\mathrm{PX}$, $\mathbf{B}^\mathrm{MAG}_\mathrm{PY}$) and of the estimated IMF. Datasets refer to TS1 (left panel) and to TS3 (right panel) time intervals (Tab.~\ref{tab1}). The PSDs obtained by subtracting the Wind data from each LPF IMF measurement timeseries are also provided.
	}
	\label{fig_lpf__PSD}
\end{figure*}

The empirical approach adopted in the present work to separate the IMF component from on-board platform magnetic field  measurements  has been applied to the entire LPF datasets.
The IMF timeseries $\mathbf{B}^\mathrm{IMF}_\mathrm{MX}[t]$, $\mathbf{B}^\mathrm{IMF}_\mathrm{MY}[t]$, $\mathbf{B}^\mathrm{IMF}_\mathrm{PX}[t]$ and $\mathbf{B}^\mathrm{IMF}_\mathrm{PY}[t]$ obtained by processing the four magnetometer datasets are reported in Figure~\ref{fig_lpf_IMF_comparison} in blue, red, green and black colors, respectively. In the same figure, the LPF IMF estimates are compared to contemporaneous measurements carried out with the MFI detector (in yellow) hosted on board the Wind S/C (NASA CDAWeb, \url{https://directory.eoportal.org/web/eoportal/satellite-missions/}).
Data gaps are associated with magnetic experiments, electronics functional tests, faults and anomalies.
\begin{figure*}[hbtp]
  \centering
	\includegraphics[width=16cm]{./images/FiguraBimf_overall_v5.jpeg}
	\caption{IMF measurement timeseries obtained with the four LPF magnetometers ($\mathbf{B}^\mathrm{IMF}_\mathrm{MX}$, $\mathbf{B}^\mathrm{IMF}_\mathrm{MY}$, $\mathbf{B}^\mathrm{IMF}_\mathrm{PX}$ and $\mathbf{B}^\mathrm{IMF}_\mathrm{PY}$ data appear in blue, red, green, and black, respectively).
	Simultaneous Wind magnetic measurements, $\mathbf{B}^\mathrm{IMF}_\mathrm{Wind}$, are also shown in yellow for comparison. Timespans TS13 and TS14 (Tab.~\ref{tab1}) are ignored as discussed in Section~\ref{section_magnetic_monitoring} (hatched area).
	The $x$-axis indicates the mission lifetime in DOM.}
	\label{fig_lpf_IMF_comparison}
\end{figure*}
A good agreement is found between Wind MFI and LPF IMF estimates during the whole mission duration.

In Figure~\ref{fig_lpf__wind2lpf} the time interval ranging from July 15 through August 4, 2016 is considered in particular.
\begin{figure}[htbp]
  \centering
	\includegraphics[width=8.5cm,height=8.5cm]{./images/EventoGargiuloX20190501T011810_300_TUTTE_2.jpeg}\\
	\caption{Comparison between LPF and Wind IMF measurements in the $\pmb{\mathrm{R}}_\mathrm{GSE}$ between July 15 and August 4, 2016.
	}
	\label{fig_lpf__wind2lpf}
\end{figure}
In this figure, the IMF components inferred from the LPF on-board measurements are shown shown in the Global Solar Ecliptic reference frame ($\pmb{\mathrm{R}}_\mathrm{GSE}$) in order to compare our results with the Wind IMF contemporaneous measurements. The
LTP $z$-axis of the LPF magnetometer reference frame was aligned with the GSE $x$-axis during the mission lifetime, while the $\pmb{\mathrm{R}}_\mathrm{LTP}$ $xy$-plane rotated around the LTP $z$-axis with a six month period.
Between August 2 and August 3, 2016 the LPF S/C crossed a magnetic cloud.
It is worthwhile to recall that a magnetic cloud is a coherent plasma structure characterized by a smooth rotation of the IMF and an enhanced magnetic field intensity with respect to the average value observed in the solar wind while temperature and plasma beta parameters ({\it i.e.} the ratio of the plasma pressure to the magnetic pressure) present lower values with respect to an undisturbed interplanetary medium \citep{1990GMS....58..373B}. The LPF IMF data are compared to the magnetic field estimated by applying the Grad-Shafranov reconstruction technique based on the Wind S/C observations  \citep{Hu_Sonnerup_2002,Hu2017,2020ApJ...901...21B}.
The comparison between the IMF components inferred from the GS reconstruction and the LPF observations is shown in Figure~\ref{fig_gargiulo}. The LPF IMF estimates are computed by averaging the data of the four magnetometers and are reported by considering a sampling time of one minute. Again, a good agreement is observed and this comparison allows for an independent validation of the results of our empirical approach  within a few percent.
\begin{figure*}[htbp]
  \centering
    \includegraphics[width=9.0cm,height=8.5cm]{./images/gargiulo_gradshafranov_imfRecon_mod3.jpeg}
	\caption{
	LPF IMF (black line) and IMF along the LPF orbit obtained with a Grad-Shafranov reconstruction based on Wind data of the magnetic cloud crossed by the S/C on August 2-3, 2016 (open blue circles). Each LPF IMF component was obtained by averaging the measurements of the four on-board magnetometers.}
	\label{fig_gargiulo}
\end{figure*}

A similar work could be carried out for LISA that will also host platform magnetometers in order to monitor the transit of interplanetary magnetic structures.
It is worthwhile to point out that contrarily to LPF, LISA  will not benefit of other experiments gathering interplanetary magnetic field, plasma data and particle monitoring along its orbit.

\section{Galactic and solar energetic particle events during the LISA mission operations}

\subsection{Galactic cosmic-ray energy spectrum during the solar cycle 26}

The galactic cosmic-ray (GCR) flux in the inner heliosphere appears energy, time, space, and charge dependent.
The overall GCR flux was observed to vary by a factor of four near Earth during the last three solar cycles when the majority of data to study the long-term variations of particles of galactic origin were gathered in space \citep{2021A&A...656A..15G}.
Since LISA will orbit the Sun at 1 AU,  both near-Earth GCR and SEP observations  can be used to carry out long-term predictions of the overall particle flux  during the mission operations by properly taking into account  the solar activity expected  during the solar cycle 26 \citep{sing19}. The predicted sunspot number, the most widely used proxy of the solar activity during the solar cycle 26, is reported in Figure~\ref{fig_sunspot_number_prediction} and Table~\ref{tab2}.

\begin{figure}[hbt!]
  \centering
\hspace{-0.3cm}
\centering
\includegraphics[width=9.cm,height=9.cm]{sc26.pdf}
\caption{Sunspot number prediction for the solar cycle 26 \citep{sing19}.}
\label{fig_sunspot_number_prediction}
\end{figure}

\begin{table}[tbh]
        \centering
        \resizebox{3.2cm}{!}{
                \begin{tabular}{lcc}\hline\hline
                        Year & SSN & SEP \\ \hline
                        2035 & 68 & 4.7 \\
                        2036 & 76 & 5.3 \\
                        2037 & 77 & 5.3 \\
                        2038 & 74 & 5.1 \\
                        2039 & 63 & 4.4 \\
                        2040 & 42 & 2.9 \\
                        2041 & 32 & 2.2 \\
                        2042 & 21 & 1.5 \\
                        2043 & 11 & 0.8 \\
                        2044 & 1  & 0.07 \\\hline\hline
                \end{tabular}
        }
        \caption{Expected sunspot number (SSN) and SEP events during the LISA mission with a fluence larger than the estimated GCR background.}
        \label{tab2}
\end{table}

The past solar cycle 24 has been the weakest of the last hundred years. According to Singh and Barghawa \citep{sing19}, the next two solar cycles are expected to be even weaker. However, it is worthwhile to point out that the initial phase of the solar cycle 25 appears more intense than these predictions and other authors indicate an intensity similar to that of the solar cycle 23 \citep{Diego_Storini_Laurenza_2010,McIntoshEtAl2020,Diego_and_Laurenza_2021}. Quasi-eleven and quasi twenty-two year periodicities are observed in the GCR intensity associated with the solar activity and the GSMF polarity reversal, respectively.
The force-field model by Gleeson and Axford \citep{glax68} is adopted here for the prediction of GCR long-term modulation while the Nymmik model \citep{{Nymmik1999a},{Nymmik1999b}} is considered to estimate the occurrence of SEP events  with fluence larger than  10$^6$ protons cm$^{-2}$ above 30 MeV during the LISA mission. The Gleeson and Axford model allows us to estimate the GCR flux in the inner heliosphere by assuming a time independent interstellar spectrum and a solar modulation parameter $\phi$ describing the effect of the solar activity modulation (see \citeauthor{apj1}, \citeyear{apj1}, where the details of our approach are illustrated).
The sunspot number is correlated with the solar modulation parameter.
After the LISA launch in 2035, the maximum expected number of  sunspots is about 80 and, consequently, the solar modulation parameter may possibly vary between 600 MV/c and 700 MV/c, as in September 2015 for instance (see \url{https://cosmicrays.oulu.fi/phi/Phi_mon.txt} handled by the University of Oulu, Finland). This estimate appears consistent with the solar modulation values inferred from \citet{refsitof}.

\noindent The proton energy spectrum above 70 MeV corresponding to the above solar modulation was obtained with the Gleeson and Axford model (bottom dot-dashed line in Fig.~\ref{fig_min_and_max_proton_energy_spectra}) parameterized according to \citet{papini96} and \citet{apj1} as follows:
\begin{equation}
F(E)= 18000 \times (E+1.5)^{-3.66}\ E^{0.87}  \ \ \ {\rm particles/(m^2\ sr\ s\ GeV),}
\label{equation2}
\end{equation}
where E is the proton kinetic energy in GeV. The proton flux is considered in the figure since protons constitute 90\% of the overall cosmic-ray bulk.
We verified the reliability of this approach for the LISA Pathfinder mission in 2016-2017, by comparing our cosmic-ray proton flux predictions to observations carried out above 450 MeV n$^{-1}$ by the AMS-02 experiment during the same period on the Space Station (see \citeauthor{aguilar18}, \citeyear{aguilar18}).
Our predicted GCR integral proton flux at the LPF launch in December 2015 was found only 10\% higher than the AMS-02 data \citep{grim19}.

\subsection{Estimates of solar energetic particle event occurrence during the LISA mission}
Nymmik \citep{{Nymmik1999a},{Nymmik1999b}} has found that the SEP fluence  distribution  follows a power-law trend with an exponential cutoff for increasing values of the fluence. This model applies to proton fluences ranging between 10$^6$ and 10$^{11}$ protons cm$^{-2}$ above 30 MeV, where 10$^6$ protons cm$^{-2}$ represents the fluence that approximately corresponds to the background of GCRs near solar minimum. The Nymmik model was developed on the basis of SEP event measurements carried out with the IMP-7 and IMP-8 S/C during the solar cycles 20-22 and from proton fluxes estimated with radionuclide observations in lunar rocks generated in the last few million years.
The method is illustrated in the following \citep{StoriniEtAl2008}. The yearly number of SEP events, $<n>$ is assessed on the basis of sunspot number predictions, $<N>$:

\begin{equation}
 <n>=0.0694<N> \label{ee1}.
\end{equation}
\noindent In order to estimate the number of events per interval of fluence, $f$, a  normalization constant C must be then set by integrating the
equation:
\begin{equation}
dn=C f^{-1.41} e^{-f/f_c} df  \label{ee2},
\end{equation}
\noindent and by equaling it to $<n>$. In the above equation
$f_c$ is assumed equal to 4$\times$10$^{9}$.
It is pointed out that 65\% of the events are expected to belong in the fluence range 10$^6$-10$^7$ protons cm$^{-2}$.

In \citet{Grimani:2012ce} the expected number of SEP events satisfying the Nymmik model was compared to observations carried out between 1986 and 2004. Model and data were found in agreement  within a factor of two.
The expected  number of  SEP events  during the time the LISA mission will remain in space, possibly between 2035 and 2044, is reported in Table~\ref{tab2}.

As a worst case scenario, the occurrence of a SEP event with  10$^9$ protons cm$^{-2}$ fluence in the next years should be also considered to estimate the LISA TM charging. The typical rate of occurrence of these events is one every sixty years. The last one, characterized by particle acceleration above 100 MeV n$^{-1}$, was observed on February 23, 1956 \citep{2008AdSpR..41..926V}.
The proton fluxes observed during the evolution of SEP events of different intensities are shown in Figure~\ref{fig_min_and_max_proton_energy_spectra} in comparison to the minimum expected galactic flux over the LISA mission duration.

\section{Next Generation Radiation Monitor}
The ESA NGRM is small ($<$1 dm$^{3}$), light ($<$ 1 kg) and characterized  by a low power consumption ($<$ 1 W).
This detector is optimized for low-energy particle measurements  in harsh radiation environments, typical of SEP events and of particles magnetically trapped in the Earth and other planetary magnetospheres.
The NGRM consists of two units. The proton unit allows for the measurement of the proton flux between 2 MeV and 200 MeV. In Figure~\ref{fig_min_and_max_proton_energy_spectra} proton energy spectra of particles of galactic and solar origin are compared. Protons with energies smaller than 200 MeV constitute a minor percentage of particles of galactic origin (continuous lines) at both solar minimum (upper curve) and maximum (lower curve).
As it was shown in the previous section, the bottom dot-dashed curve represents the minimum flux of galactic protons expected during the LISA mission.
In the same figure the proton energy spectra observed during the evolution of SEP events of different intensities appear softer than the GCR energy spectrum and, consequently, the particle flux below 200 MeV is representative of the total solar proton flux. As a result, the NGRM is an optimum monitor of the solar particle flux.
Linear energy transfer of ions can be also measured with this detector.
The electron unit is meant to monitor electrons in the energy range 100 keV-7 MeV.

In Figure~\ref{fig_solar_galactic_electron_fluxes} solar (top dashed and dotted curves) and galactic  (solid, dot-dashed and bottom dashed lines) electron fluxes are compared \citep{{Grimani_2009},{gri09}}. A sudden increase of the  electron flux at MeV energies is a precursor of a solar energetic proton event due to particle velocity dispersion. Solar electron fluxes showing two different spectral indices indicate impulsive SEP events associated with proton acceleration below 50 MeV. Conversely, a single spectral index characterizes the solar electron flux associated with gradual events with proton acceleration above 50 MeV. In \citet{Grimani_2009} and references therein it was shown that the intensity of the solar electron flux in the MeV range is similar to the proton flux intensity in the 50 MeV range. The NGRM has excellent SEP event short-term forecasting and measurement capabilities. LISA will allow us to carry out the first multispacecraft SEP event observations at a few million kilometers with twin detectors.
The LISA data will be also compared to observations gathered near Earth at 50 million kilometers.
Conversely, the GCR flux short-term variations will not be nicely monitored with NGRM.

The geometrical factor of the NGRM for GCR measurements is of about 0.01 cm$^2$, approximately hundred times smaller than that of the detector hosted on board LPF.
As a matter of fact, the NGRM will not allow us to monitor the dynamics of non-recurrent GCR short-term variations of typical durations of three-four days and grossly only recurrent GCR short-term variations that were found to last an average of 9 days with LPF \citep{paper3}.

\begin{figure}[t!]
\hspace{-0.3cm}
\centering
\includegraphics[width=9.cm,height=9.cm]{protheNGRM.pdf}
\caption{Minimum and maximum proton energy spectra of galactic origin observed near Earth (top and bottom continuous lines, respectively). The bottom dot-dashed line represents the minimum proton flux expected during the LISA mission. Dashed lines (December 14, 2006; \citet{2011ApJ...742..102A}), dotted lines (May 7, 1978; \citeauthor{1984AdSpR...4b.117M}, \citeyear{1984AdSpR...4b.117M}) and top dot-dashed lines (February 23, 1956; \citeauthor{2008AdSpR..41..926V}, \citeyear{2008AdSpR..41..926V}) represent solar proton fluxes observed during the evolution of SEP events of different intensities. The vertical black line indicates the maximum energy of the NGRM proton spectrum measurement capability.}
\label{fig_min_and_max_proton_energy_spectra}
\end{figure}

The classification of interplanetary processes associated with GCR short-term variations observed with LPF are reported in \citet{apj1,apj2}.
The evolution of each GCR short-term depression is unique and depends on the energy differential flux of GCR particles at the transit of subsequent interplanetary
structures.
Consequently, the average monthly GCR flux observations are not representative of the GCR flux at shorter timescales.
Moreover, it is of fundamental importance to maintain separated the effects of cosmic-ray long-term modulations from short-term variations, due to a different dependence on the particle energies for the estimate of the LISA and LPF TM charging.
Only low statistical uncertainty data gathered hourly in space  above a few tens of MeV n$^{-1}$ would allow us to disentangle the role of long-term and short-term solar modulation of the GCR flux in the LISA sensitivity band.

The evolution of individual recurrent and transient GCR variations studied with data gathered on board the LPF S/C were compared to contemporaneous Earth neutron monitor observations. This comparison revealed  that very poor clues can be obtained on the physics of GCR flux short-term
variations in space from  ground data affected by low-energy particle flux attenuation in the atmosphere and by the geomagnetic cut-off.
For instance, recurrent GCR flux depressions of intensities $<$ 5\% detected on board LPF were not observed on  neutron monitors by revealing that only particles with energies smaller than neutron monitor effective energies ($>$ 10 GeV; see for details \citeauthor{apj2}, \citeyear{apj2}) were involved in these processes. It is worthwhile to recall that particle count rates observed
with neutron monitors vary proportionally to the cosmic-ray flux at energies larger than the effective energy  \citep{gil17}, thus providing a direct measurement of the GCR flux at the top of the atmosphere above 10 GeV.
While GCR recurrent short-term variations observed in space were associated with particle modulation below a few GeV within 1\% uncertainty, Forbush decreases (sudden drops of the  GCR flux observed at the transit of ICMEs, \citeauthor{1937PhRv...51.1108F}, \citeyear{{1937PhRv...51.1108F},{1954JGR....59..525F},{1958JGR....63..651F}}) were characterized by a GCR particle flux modulation at energies above 10 GeV.
Three Forbush decreases were observed with LPF and neutron monitors on July 20 and August 2, 2016 and on May 27, 2017.
The event depression amplitudes were found of 5.5\%, 9\% and 7\% in space and
of less than  2-3\% in neutron monitor observations \citep{apj2}. In conclusion, also neutron monitor observations will not allow us to properly monitor the LISA TM charging during galactic cosmic-ray flux short-term variations. This may be useful, for instance, if TM glitches of unknown origin will be observed. As a result, a dedicated cosmic-ray detector with characteristics similar to those of the LPF PD would be more than recommended for LISA to improve the monitoring of the transit of interplanetary magnetic structures and high-speed streams.

\begin{figure}[t!]
\hspace{-0.3cm}
\centering
\includegraphics[width=9.cm,height=9.cm]{intelenegpol_sol.pdf}
\caption{Solar electron fluxes associated with impulsive (upper dashed line) and  gradual (dotted line) SEP events \citep{Grimani_2009}. The continuous line, the dot-dashed line and the bottom dashed line represent minimum and maximum interplanetary and galactic electron fluxes observed during a negative polarity period (A$<$0) of the GSMF (when the solar magnetic field lines enter the North pole of the Sun) during which the flux of negative particles is maximum with respect to periods of  positive GSMF polarity and similar conditions of solar activity \citep{Grimani_2009}. The vertical continuous line indicates the maximum electron energy measured by the NGRM. It can be observed that below 7 MeV the solar electron flux overcomes by several orders of magnitude the interplanetary and galactic components allowing for SEP event forecasting.}
\label{fig_solar_galactic_electron_fluxes}
\end{figure}

\section{Conclusions}
Spurious forces acting on the TMs of the future LISA and LISA-like interferometers for gravitational wave detection in space will be monitored and mitigated in order to improve the mission performance. Magnetometers and particle detectors will be hosted on board the LISA S/C to monitor the variations of the magnetic field and the TM charging process. Precious clues were obtained about the interplanetary medium impact in limiting the LISA performance with the LPF mission flown in 2016-2017 around the L1 Lagrange point to test the LISA technology.
The LPF magnetometers measured a magnetic field varying from $700$ to $1200\,\mathrm{nT}$ near the TMs while the IMF values in L1 ranged between $1$ and $25\,\mathrm{nT}$. The on-board generated magnetic field components were observed to vary by about $150\,\mathrm{nT}$ on timescales of months and of $20-30\,\mathrm{nT}$ daily.
An empirical approach was considered in the present work to disentangle the IMF from the on-board originated magnetic field by processing the LPF platform magnetometer measurements.
An agreement within a few percent was found between the LPF estimated IMF components and those provided by the dedicated MFI instrument on board the Wind S/C also orbiting around L1.
A similar approach could be considered for the future gravitational wave experiments carried out in space.
Provided the availability of the whole housekeeping dataset, an empirical magnetic field noise model can be produced when a S/C is already in-flight, even in the case that platform magnetometers are immersed in a strong varying and direction dependent on-board generated magnetic field.
Solar energetic particle event short-term forecasting and evolution monitoring will be also carried out with the NGRM on board the LISA S/C. An additional particle detector dedicated to galactic  cosmic-ray variation observations should be also included in the set of detectors meant for the mission diagnostics.
LISA and the second generation LISA-like interferometers
will also naturally play the role of multipoint observatories for SEP events and interplanetary large-scale structure monitoring with magnetometers used in combination with dedicated GCR detectors.

\begin{acknowledgements}

      The LISA Pathfinder mission is part of the space-science program of the European Space Agency.
      The Italian contribution has been supported by the Agenzia Spaziale Italiana (ASI) and the Istituto Nazionale di Fisica Nucleare (INFN).
      The LISA Pathfinder magnetometer data are available at the LISA Pathfinder Legacy Catalog (\url{http://lpf.esac.esa.int/lpfsa/}).
      Data from Wind experiment were collected from NASA CDAWeb (\url{https://cdaweb.sci.gsfc.nasa.gov/index.html/}).

\end{acknowledgements}


\section{Experiments}
\label{sec:exp}
In this section, to demonstrate the efficacy of our training framework, we carry out experiments on five action recognition datasets, including Kinetics-400~\cite{kay2017kinetics}, Something-Something-v2~\cite{goyal2017something}, Moments-in-Time~\cite{monfort2019moments}, Activitynet~\cite{caba2015activitynet} and Kinetics-700~\cite{carreira2019short}.
The action recognition task is defined to be a classification task given a trimmed video clip.
Unlike previous works~\cite{likhosherstov2021polyvit,zhang2021co}, we do not initialize our model using ImageNet~\cite{deng2009imagenet} since it consumes more computation. Please refer to the supplementary material for detailed comparison between train from scratch recipe and from ImageNet.
In the experiments, we aim to showcase
that our method can achieve significant performance improvement with minimal computation overhead compared to baselins.

\subsection{Experimental Setup}
\noindent\textbf{Datasets.}
We evaluate our method on five datasets.
Kinetics-400~\cite{kay2017kinetics} (K400) consists of about 240K training videos and 20K validation videos in 400 human action classes.
The videos are about 10 seconds long.
Kinetics-700~\cite{carreira2019short} (K700) extends the action classes to 700 with 545K training and 35K validation videos.
The Something-Something-v2 (SSv2)~\cite{goyal2017something} dataset contains person-object interactions, which emphasizes temporal modeling.
SSv2 includes 168K videos for training and 24K videos for evaluation on 174 action classes.
The Moments-in-Time (MiT) dataset is one of the largest action dataset with 727K training and 30k validation videos.
MiT videos are mostly short 3-second clips.
The ActivityNet dataset~\cite{caba2015activitynet} (ActNet) originally contains untrimmed videos with temporal annotations of 200 action classes.
We cut the annotated segments of the videos into 10-second long clips and split the dataset into 107K training and 16K testing.
Following previous works~\cite{feichtenhofer2019slowfast,zhang2021co}, we follow the standard dataset split and report top-1/top-5 classification accuracy on the test split for all datasets.
We conduct two sets of experiments, namely, ``K400, MiT,  SSv2, ActNet'', and ``K700, MiT, SSv2, ActNet''.

\textbf{Implementation.}
Our backbone model utilizes MViTv2 as described in Section~\ref{sec:mvit}.
Our models are trained from scratch with random initialization, without using any pre-training (same as in ~\cite{feichtenhofer2019slowfast} and different from previous works~\cite{zhang2021co,likhosherstov2021polyvit} that require large-scale image dataset pre-training like ImageNet-21K~\cite{deng2009imagenet} or JFT-3B~\cite{DBLP:journals/corr/abs-2106-04560}).
We follow standard dataset splits as previous works~\cite{li2021improved,feichtenhofer2019slowfast,yan2022multiview}.
See more details in the supplementary material.
\begin{comment}
For both sets of experiments, we start our training from the corresponding Kinetics-trained model checkpoints to reduce training time.
The Kinetics pre-training follows ``train from scratch'' protocol of MViTs~\cite{fan2021multiscale,li2021improved}.
We use a learning rate of 2e-4 and cosine learning rate decay.
We use synchronized SGD with 128 V100/A100 GPUs to train for 50 epochs with a batch size of 512, with a linear warm-up from 2e-6 for the first 10 epochs.
Each experiment takes 2-4 days to finish.
We use a weight decay of 0.05 and the adamw~\cite{loshchilov2017decoupled} optimizer is used.
We use 32x3 video frames (number of frames times sampling rate) as inputs except for SSv2.
For SSv2, following the MViT paper~\cite{fan2021multiscale}, we uniformly sample frames from the whole videos instead of getting fix-length clips.
For all three datasets, we use random $224\times224$ crops unless mentioned otherwise and horizontal flipping from a video clip (except for videos from SSv2), which is randomly sampled from the full-length video and resized to a shorter edge side of randomly sampled in [256, 320] pixels.
We utilize Mixup~\cite{zhang2017mixup} and Rand Augment~\cite{cubuk2020randaugment} for data augmentation during training.
Following previous works~\cite{feichtenhofer2019slowfast,fan2021multiscale}, we sample $3\times3$ (unless mentioned otherwise) clips for each video during testing: we uniformly sample 3 clips for the temporal domain and 3 spatial crops of size $224\times224$ after the shorter edge size are resized to 256 pixels.
We average the softmax scores across all clips for final prediction.
We utilize 2-layer MLPs with 2048 hidden size and a GELU activation for the expander and the classification heads.
A single-layer linear projection head is added for each pair of datasets during training, and they are discarded during inference (the original classification head outputs are used as the final prediction.)
\end{comment}

\textbf{Baselines.}
PolyViT~\cite{likhosherstov2021polyvit} utilizes multi-task learning on image, video and audio datasets to improve vision transformer performance. The backbone they used are based on ViT-ViViT~\cite{arnab2021vivit}.
Similarly, VATT~\cite{akbari2021vatt} utilizes additional multi-modal data for self-supversied pretraining and finetunes on downstream datasets. The backbone network is based on ViT~\cite{dosovitskiy2020image}.
CoVER~\cite{zhang2021co} is a recently proposed co-training method that includes training with images and videos simultaneously. Their model backbone is based on TimeSFormer~\cite{bertasius2021space}.
We also compare our method with other recent models trained using large-scale image datasets.
See Table~\ref{tab:k400} and Table~\ref{tab:k700} for the full list.

\subsection{Main Results}

\input{table/k400}

\input{table/k700}

We summarize our method's performance in Table~\ref{tab:k400} and Table~\ref{tab:k700}.
We train our model jointly on MiT, SSv2, ActNet and two versions of the Kinetics datasets.

We first compare our method with the original MViTv2 backbone in Table~\ref{tab:k400}.
``MViTv2 w/ abs. pos.'' means MViTv2 model with absolute positional embedding, which is taken from Table A.6 of MViTv2 paper~\cite{li2021improved} and it is (almost) the same as our model implementation.
We can not achieve the same accuracy with the same recipe as MViTv2, which may be due to differences in the Kinetics dataset (missing some videos, etc. See supplementary material for full dataset statistics).
PolyViT~\cite{likhosherstov2021polyvit} is trained jointly with multiple image, audio and video datasets.
We list the larger ones.
We train our baseline model on the training set of each dataset to investigate the baseline performance.
As we see, after adding robust joint training proposed in this paper, performance on each dataset has increased by 2.1\%, 3.1\%, 1.9\% and 5.9\% on K400, MiT, SSv2, ActivityNet, respectively in terms of top-1 accuracy.
Note that our method achieves such improvement withtout large-scale image pre-training and additional inference computational cost.

We then compare our method with state-of-the-art on these datasets.
We train a higher resolution model with larger spatial inputs (312p) and achieves better performance compared to recent multi-dataset training methods, CoVER~\cite{zhang2021co} and PolyVit~\cite{likhosherstov2021polyvit}, on Kinetics-400, and significantly better on MiT and SSv2, as shown in Table~\ref{tab:k400}.
Note that our model does not use any image training datasets, and our model computation cost is only a fraction of the baselines.
We also show that our performance boost does not come from the additional training dataset of ActivityNet in Table~\ref{tab:ablation}.

Our method also achieves competitive results compared to state-of-the-art models trained with large-scale image dataset (ImageNet-21K~\cite{deng2009imagenet}).
Compared to a recent method, MTV-B~\cite{yan2022multiview}, our method is able to achieve significantly better top-1 accuracy across Kinetics-400, MiT, SSv2 by 0.8\%, 1.4\%, 0.8\%, respectively, at half of the computation cost and without large-scale pre-training.
Note that our model is parameter-efficient, while multiple MTV-B models need to be trained and tested on these datasets separately.
Our method can achieve better performance with a deeper base backbone or larger resolution inputs but we have not tested due to limitation of computation resources.

We then compare our method on the Kinetics-700, MiT, SSv2 and ActivityNet training with baselines.
Our parameter-efficient model can achieve better performance than MTV-B~\cite{yan2022multiview} at one-fifth of the computation cost.
With a larger resolution model at 312p, we achieves significantly better performance than the baseline across Kinetics-400, MiT, SSv2 by 2.2\%, 4.9\%, 3.4\%, respectively.

\subsection{Ablation Experiments}
In this section, we perform ablation studies on the K400 set.
To understand how action models can benefit from our training method,
we explore the following questions (results are shown in
Table~\ref{tab:ablation}):

\input{table/ablation}

\noindent \textbf{Does our proposed robust loss help?}
We compare our model training with vanilla multi-dataset training, where multiple classification heads are attached to the same backbone and the model is trained simply with cross-entropy loss.
The vanilla model is trained from a K400 checkpoint as ours.
As shown in Table~\ref{tab:ablation}, we try training the vanilla model with both the same training schedule as ours and a 4x longer schedule.
As we see, there is a significant gap between the overall performance of the vanilla model and ours, validating the efficacy of our proposed method.
Also, longer training schedule does not lead to better performance on some datasets, including SSv2, suggesting vanilla multi-dataset training is unstable.
In terms of performance on ActivityNet, we observe that both training methods achieve good results, which might be because ActivityNet classes are highly overlapped with Kinetics-400 (65 out of 200).

\noindent \textbf{How important is the informative loss?}
We then experiment with removing the informative loss (Section~\ref{sec:robust}) during multi-dataset training.
It seems that the feature embedding of the model collapse and the model is not trained at all.
We further investigate why ``w/o informative Loss'' completely fails but ``Vanilla'' seems to work by running an experiment of "w/o informative Loss \& w/o projection add", which means we remove the projected logits addition in Eq. 5 and directly compute classification loss on the projected logits.
Therefore we can consider this run as adding additional projection branches to the vanilla architecture. The results are slightly better than "Vanilla" on K400 and much better on MiT/SSv2. It indicates that adding projected logits to the original branch without informative loss would prevent the model from converging (the total loss does not go down).

\noindent \textbf{How important is the projection loss?}
We then experiment with removing the projection heads (Section~\ref{sec:robust}) during multi-dataset training.
The model is trained with the original cross-entropy loss and the informative loss.
As shown in Table~\ref{tab:ablation}, the performance on MiT and SSv2 suffers by a large margin, indicating that the projection design helps boost training by better utilizing multi-dataset information.

\input{table/projection}
\noindent \textbf{What does the cross-dataset projection layer learn?}
We analyze the cross-dataset projection weights of the K400/312p model and list top 5 concepts for each pair of datasets in Table~\ref{tab:proj}.
We make two observations. First, the top projections are visually similar actions, which confirms our intuition that there are intrinsic relations in the datasets that the model can mine to improve performance. For example, “bending metal” in K400 and “bending” in MIT, “parkour” in K400 and “Capoeira” in Activitynet. Interestingly, “Wiping something off of something” in SSv2 and “cleaning windows” in K400.
Second, the action with the same name may not have the highest weights. In “mit to kinetics”, the “sneezing” action ranks 5th in the projection weights, suggesting that there might be discrepancies of the same concept in different datasets.
These observations are interesting and one may compare the learned weights with textual semantic relations (like those in ConceptNet). We leave this to future work.

\noindent \textbf{Does the additional ActivityNet data help?}
In previous methods like CoVER and PolyViT, the ActivityNet dataset has not been used.
In this experiment, we investigate the important of the ActivityNet dataset by removing it from the training set.
From Table~\ref{tab:ablation}, we can see that the performance across all datasets drop by a small margin, indicating our superior results compared to CoVER (see Table~\ref{tab:k400} and Table~\ref{tab:k700}) come from the proposed robust training paradigm rather than the additional data.

\subsection{Discussion}
\label{sec:discussion}
By multi-dataset training transformers on various datasets, we obtain competitive results on multiple action datasets, without large-scale image datasets pre-training.
Our method, \fancyname, is parameter-efficient and does not require hyper-parameter tuning.
Current limitations of our experiments are that we have not tried co-training with image datasets such as ImageNet-21K~\cite{deng2009imagenet}.
Hence we do not know how much performance gain that would entail.
We plan to explore this in future work.
In addition, we have not tried training larger model with FLOPs on par with state-of-the-art or other backbone architectures (e.g. CNNs) due to limitation of our computational resources.
Hence we are not sure how our algorithm would behave with these models.
We have not explicitly explored how temporal modeling could benefit from multi-action-dataset training, which we leave for future work.
Although our model is trained on multiple datasets, potential dataset biases can still cause negative societal impact in real-world deployment, as the datasets we have do not fully represent all aspects of human actions.
\section{Introduction}

Human vision can recognize video actions efficiently despite the variations of scenes and domains.
Convolutional neural networks (CNNs) \cite{cnn-lecun-eccv10,cnn-dtran-iccv15,cnn-Carreira-Zisserman-cvpr17,DBLP:journals/corr/SigurdssonDFG16,DBLP:conf/nips/FeichtenhoferPW16,liang2020spatial}
effectively
exploit
the power of modern computational devices and employ spatial-temporal filters to recognize actions, which
considerably outperform
traditional models such as oriented filtering in space-time (HOG3D) \cite{DBLP:conf/bmvc/KlaserMS08}.
However, due to the high variations in space-time, the
state-of-the-art accuracy of action recognition is still far from being
satisfactory, compared with the success of 2D CNNs in image recognition \cite{he2016deep}.

Recently, vision transformers
such as
ViT \cite{dosovitskiy2020image}, and MViT \cite{fan2021multiscale} that are based on the self-attention~\cite{vaswani2017attention} mechanism
were
proposed to tackle the problems of image and video recognition,
and achieved impressive performance.
Instead of modeling pixels
as
CNNs, transformers apply attentions on top of visual tokens.
The inductive bias of translation
invariance
in CNNs makes it require less training data than
attention-based transformers in general.
In contrast,
transformer has the advantage that it
can better leverage  `big data', leading to improved accuracy than CNNs.
We have
witnessed
a rapid growth in video datasets \cite{kay2017kinetics} in recent years, which would make up for the shortcomings of data-hungry transformers.
The video data has not only grown in quantity from hundreds to millions of videos~\cite{monfort2019moments},  but also evolved from simple actions such as handshaking to complicated daily activities from the Kinetics-700 dataset~\cite{carreira2019short}.
Meanwhile, transformers combined with low-level convolutional operations have been proposed \cite{fan2021multiscale} to further improve the
efficiency and accuracy.

Due to the data-hungry nature of transformers, most transformer-based models for action recognition requires large-scale pre-training with image datasets such as ImageNet-21K~\cite{deng2009imagenet} and JFT-3B~\cite{DBLP:journals/corr/abs-2106-04560} to achieve good performance.
This pre-training and fine-tuning training paradigm is time-consuming and it is not parameter-efficient, meaning that for each action dataset, a new model need to be trained end-to-end.
Different from large image datasets such as ImageNet-21K that covers a wide range of object classes, currently the most diverse action dataset, Kinetics-700, only contains 700 classes.
Each action dataset may be also limited to a certain topic or camera views.
For example, Moments-in-Time~\cite{monfort2019moments} only contains short actions that happen in three  seconds and Something-Something-v2~\cite{goyal2017something} focuses on close-up camera view of person-object interactions.
These dataset biases might hinder  models trained on a single dataset to generalize and be used in practical applications.
These challenges in action datasets make learning a general-purpose action model very difficult.
An ideal model should be able to cover a wide range of action classes meanwhile keeping the computation cost low.
However, \textit{simply combining all these datasets to train a joint model does not lead to good performance} \cite{likhosherstov2021polyvit}.
In previous work~\cite{zhang2021co},
the authors
have shown the benefit of training a joint model using multiple action datasets but their method requires large-scale image datasets such as ImageNet-21K~\cite{deng2009imagenet} and JFT-3B~\cite{DBLP:journals/corr/abs-2106-04560}, which is not available to the research community.

\begin{figure}[t!]
	\centering
		\includegraphics[width=1.051\textwidth]

  {figures/fig1a-crop.pdf}
	\caption{Overview of our multi-dataset training framework. We propose to utilize the intrinsic relations between classes across different action datasets. As we can see, the two video examples from Kinetics and Moments-in-Time datasets, respectively, show that samples from these two classes can be used to train both classification heads.
	The videos from multiple action datasets are input to the MViTv2 (Section~\ref{sec:mvit}) backbone and the model is trained jointly.
	The informative loss is applied to maximize the information content of the embedding from the backbone and the projection loss is applied to learn the intrinsic relations (Section~\ref{sec:robust}).
	}
	\label{fig:overview}
\end{figure}

In this paper,
we propose a general training paradigm for \textbf{Multi}-dataset \textbf{Train}ing of robust action recognition models, \fancyname.
Our method is designed to learn robust and informative feature representations in a principled
approach, using the informative loss for regularization.
We do not assume the availability of large-scale image datasets pre-training (although one can certainly take advantage of that).
Since there are intrinsic relations between different classes across different action datasets (See Fig.~\ref{fig:overview} for examples of similar classes from two datasets), we propose a projection loss to mine such relations such that the whole network is trained to avoid over-fitting to certain dataset biases.
Finally, all proposed loss terms are weighted using learned parameters. Thus, no hyper-parameter tuning is needed.
Our empirical findings as shown in Table~\ref{tab:k400} indicate that our robust training method can consistently improve model backbone performance across multiple datasets.
We show that our model can achieve competitive results compared to state-of-the-art methods, even without large-scale image dataset pre-training, and with a lower  computational cost.

The main contributions of this paper are thus three-fold:
\begin{itemize}
\itemsep 0cm
\item
To our knowledge,
this is the first work to introduce informative representation regularization into multi-dataset training for improving action recognition.
\item We propose an effective
approach
to mine intrinsic class relations in multi-dataset training by introducing the projection loss.
\item Our method requires
negligible computation
overhead during training and no additional computation during inference to the backbone network. Extensive experiments on
various
datasets suggest our method can
 consistently
improve performance.
\end{itemize}
\section{Method}

Our method is
built upon the backbone of
the improved Multi-scale Vision Transformers (MViTv2)~\cite{li2021improved,fan2021multiscale}.
Note that
our approach
works with any action recognition backbones.
Given videos from multiple datasets during training, the model backbone takes the video frames and produces feature embeddings for each video.
The same number of Multi-layer Perceptron (MLP) as the datasets are constructed as model heads to predict action classes for each dataset.
To facilitate robust cross-dataset training, we propose two loss terms, namely, \textit{the informative loss and projection loss}.
The informative loss aims to maximize the embeddings' representative power.
The projection loss, with the help of multiple cross-dataset projection layers,  guides the model to learn intrinsic relations between classes of different dataset, hence the model heads can be trained jointly.
See Fig.~\ref{fig:overview} for an overview of our framework.
In this section, we first briefly describe the MViTv2 backbone design, and then present our proposed robust cross-dataset training paradigm.

\begin{comment}
\begin{figure}[t!]
	\centering
		\includegraphics[width=0.47\textwidth]{figures/mvitv2_block.drawio.png}
	\caption{The MViTv2 Block. The residual connection for pooled query tensor (red arrow) and the residual 3D convolution operation outside the Multi-head Pooling Attention block are additions to the MViTv1~\cite{fan2021multiscale} design. The linear layer in the residual connection of the MLP block is only needed when the output embedding dimension is different.
	Compared to the MViTv2 paper~\cite{li2021improved}, we do not use the decomposed relative embedding.
	}
	\label{fig:mvit}
\end{figure}

\end{comment}

\subsection{The MViTv2 Backbone}
\label{sec:mvit}
Our model is based on the improved multi-scale vision transformers (MViTv2)~\cite{fan2021multiscale,li2021improved}, which learns a hierarchy from dense (in space) and simple (in channels) to coarse and complex features.
The series of work of vision transformers~\cite{dosovitskiy2020image} (ViTs) follows the basic self-attention architecture~\cite{vaswani2017attention} originally proposed for machine translation.
\begin{comment}
In contrast to natural language which can be directly tokenized into words, given the input video $\mathbf{V} \in \mathbb{R}^{T \times H \times W \times 3}$, ViTs extract tokens by splitting the video into
$N = \lfloor T/t \rfloor \times \lfloor H/h \rfloor \times \lfloor W/w \rfloor$
non-overlapping patches, $\{ \mathbf{v_1}, \cdots, \mathbf{v_N} \in \mathbb{R}^{t \times h \times w} \}$.
Each patch is then projected into a patch embedding by a 3D convolution operator $E$.
All patch embeddings are then concatenated into a sequence, and separate learnable spatial-temporal positional embeddings $\mathbf{p_s}, \mathbf{p_t}$ are also
added to this sequence.
The patch embedding process is denoted by:
\begin{equation}
    \mathbf{X_0} = [ \mathbf{E}\mathbf{v_1} \cdots  \mathbf{E}\mathbf{v_N}] + P(\mathbf{p_s}, \mathbf{p_t}) \in \mathbb{R}^{N \times d_{p}}
\end{equation}
The $P$ function extends the separate position embedding into the length of the sequence by repeating at the same spatial or temporal location.
$d_{p}$ is the dimension of the patch embedding.
\end{comment}
The key component of the MViTv1 model~\cite{fan2021multiscale} is the Multi Head Pooling Attention (MHPA), which pools the sequence of latent tensors to reduce the spatial or temporal dimension of the feature representations.
In MViTv2~\cite{li2021improved}, a residual connection in MHPA for the pooled query tensor and a decomposed relative position embedding~\footnote{We did not implement this part as the code were not available at the time of writing (March 2022).} are added.
In this paper, we use 3D convolution as the pooling operation.
Please refer to supplemental material for a visualization of the MViTv2 block.
Each MViTv2 block consists of a multi-head pooling attention layer (MHPA) and a multi-layer perceptron (MLP), and the residual connections are built in each layer.
The feature of each MViTv2 block is computed by:
\begin{equation}
\begin{split}
    X_1 &= {\rm MHPA}(LN(X)) + Pool(X) \\
    Block(X) &= {\rm MLP}(LN(X_1)) + X_1,
\end{split}
\end{equation}
where $X$ is the input tensor to each block.
Multiple MViTv2 blocks are grouped into stages to reduce the spatial dimension while increase the channel dimension.
The full backbone architecture
is listed in supplementary material.

\textbf{Classification head.}
For the action recognition problem, the model produces C-class classification logits by first averaging the feature tensor from the last stage along the spatial-temporal dimensions (we do not use the [CLASS] token in our transformer implementation), denoted as $\mathbf{z} \in \mathbb{R}^{d}$.
A linear classification layer is then applied on the averaged feature tensor to produce the final output, $\mathbf{y} = \mathbf{W_{\rm out}}\mathbf{z} \in \mathbb{R}^{C}$.

\begin{comment}
\textbf{Pre-training and finetuning.}
In the standard training paradigm for action recognition, models are pretrained using image datasets (ImageNet~\cite{deng2009imagenet} or large-scale datasets like JFT-3B~\cite{DBLP:journals/corr/abs-2106-04560}) and then finetune on the target action recognition dataset.
For CNN-based backbones, model weight inflation~\cite{kay2017kinetics} is utilized to adapt the model trained on 2D image data to 3D video input.
For transformer-based backbones, as the inputs are tokenized into a sequence, to adapt the model from image pretraining, the positional embeddings are interpolated to account for the additional temporal dimension before the finetuning.
\end{comment}

\textbf{Multi-dataset training paradigm.}
In general, to facilitate multi-dataset training of $K$ datasets, the same number of classification heads are appended to the feature embeddings.
The $k$-th dataset classification output is defined as
$\mathbf{Y}_k = h_k(\mathbf{Z}; \mathbf{W}_{k}) \in \mathbb{R}^{B \times C}$,
where $h_k$
can
be a linear layer or a MLP and $\mathbf{W}_{k}$ is the layer parameter.

\subsection{  {\textbf MultiTrain}: Robust Multi-dataset Training}
\label{sec:robust}

Our training process fully leverages different action recognition datasets by enforcing an \textbf{informative loss} to maximize the expressiveness of the feature embedding and a \textbf{projection loss} for each dataset that mines the intrinsic relations between classes across other datasets.
We then use uncertainty to weight different loss terms without the need for any hyper-parameters.

\textbf{Informative loss.} Inspired by the recently proposed VICReg~\cite{bardes2021vicreg} and Barlow Twins~\cite{zbontar2021barlow} methods  for self-supervised learning in image recognition, we propose to utilize an informative loss function with two terms, \textbf{variance and covariance}, to maximize the expressiveness of each variable of the embedding.
This loss is applied to each mini-batch, without the need for batch-wise nor feature-wise normalization.
Given the feature embeddings of the mini-batch, $\mathbf{Z} \in \mathbb{R}^{B \times d}$, an expander (implemented as a two-layer MLP) maps the representations into an embedding space for the informative loss to be computed, denoted as $\mathbf{Z'} \in \mathbb{R}^{B \times d}$.
The \textbf{variance loss} is computed using a hinge function and the standard deviation of each dimension of the embeddings by:
\begin{equation}
   \mathcal{L}^v = \frac{1}{d}\sum^{d}_{j=1} \text{max}\Bigl(
        0, 1 - \sqrt{\frac{\sum(\mathbf{Z'}_{ij} - \bar{\mathbf{Z'}}_{:j})}{d-1} + \epsilon}
   \Bigr),
\end{equation}
where $:$ is a tensor slicing operation that extracts all elements from a dimension, and $\bar{\mathbf{Z'}}_{:j}$ is the mean over the mini-batch for $j$-th dimension. $\epsilon$ is a small scalar preventing numerical instabilities.
With random sampling videos across multiple datasets for each batch, this criterion encourages the variance of each dimension in the embedding to be close to $1$, preventing embedding collapse~\cite{zbontar2021barlow}.
The \textbf{covariance loss} $c(\mathbf{Z'})$ is defined as:
\begin{equation}
\begin{split}
    C(\mathbf{Z'}) &= \frac{1}{n - 1}\sum^{n}_{i=1} (\mathbf{Z'}_i - \bar{\mathbf{Z'}})(\mathbf{Z'}_i - \bar{\mathbf{Z'}})
    ^{
    T
    } \;,\;
    \text{where} \; \bar{\mathbf{Z'}} = \frac{1}{n}\sum^{n}_{i=1}\bar{\mathbf{Z'}}_{i} \\
    \mathcal{L}^c &= \frac{1}{d}\sum_{i \neq j}[C(\mathbf{Z'})]^2_{i,j}
\end{split}
\end{equation}
Inspired by VICReg~\cite{bardes2021vicreg} and Barlow Twins~\cite{zbontar2021barlow}, we first compute the covariance matrix of the feature embeddings in the batch, $C(\mathbf{Z'})$, and then define the covariance term $\mathcal{L}^c$ as the sum of the squared off-diagonal coefficients of $C(\mathbf{Z'})$, scaled by a factor of $1/d$.

\textbf{Projection Loss.} In previous works~\cite{zhang2021co,likhosherstov2021polyvit}, the intrinsic relations between classes from across different datasets have been mostly ignored during training.
We believe that samples in one dataset can be utilized to train the classification head of other datasets. As shown in Fig.~\ref{fig:overview}, the ``Clean and jerk'' video sample from Kinetics can be considered as a positive sample for ``Weightlifting'' in Moments-in-Time as well (but not vice versa).
Based on this intuition, we propose to add a directed projection layer for each pair of datasets for the model to learn such intrinsic relations.
One can also initialize the projection using prior knowledge but it is out-of-scope for this paper.
Given the output from the $k$-th dataset classification output, the projected classification output is defined as:
\begin{equation}
    \mathbf{Y'}_k = \mathbf{Y}_{k} + \sum^{K-1}_{i \neq k} \mathbf{W}^{proj}_{ik}\mathbf{Y}_{i} \in \mathbb{R}^{C_k},
\end{equation}
where $C_k$ is the number of classes for the $k$-th dataset and $\mathbf{W}^{proj}_{ik}$ is the learned directed class projection weights from $i$-th to $k$-th dataset.
In this paper we only consider a linear projection function.
We then use the ground truth labels of the $k$-th dataset to compute standarad cross-entropy loss:
\begin{equation}
    \mathcal{L}_k = - \sum^{C_k}_{c=1} \hat{\mathbf{Y}}_{k,c} \log(\mathbf{Y'}_{k,c}),
\end{equation}
where $\hat{\mathbf{Y}}_{k,c}$ is the ground truth label for the $c$-th class from the $k$-th dataset.

\textbf{Training.} We jointly optimize the informative loss and the projection loss during multi-dataset training.
To avoid tuning loss weights of different terms, we borrow the weighting scheme from multi-task learning~\cite{kendall2018multi} and define the overall objective function as:
\begin{equation}

    %
    %
    \mathcal{L}(\sigma) = \mathcal{L}^v + \mathcal{L}^c + \sum^{K}_{k=1} \frac{1}{2\sigma_k^2}\mathcal{L}_k+\log\sigma_k,
\end{equation}
where $\sigma$ is a vector of learnable parameters of size $K$ (the number of datasets) for each projection loss term. This avoids the need to manually tune loss weights for different datasets.

\section{Related Work}
We review some work that is closest to ours.

\noindent\textbf{CNNs and Vision Transformers.}
CNNs work as the standard backbones throughout computer vision tasks for image and video. Various effective convolutional neural architectures have been raised to improve the precision and efficiency (\textit{e.g.}, VGG \cite{simonyan2014very}, ResNet \cite{he2016deep} and DenseNet \cite{huang2017densely}). Although CNNs are still the primary models for computer vision, the Vision Transformers
have already
shown their enormous potential.
Vision Transformer (ViT \cite{dosovitskiy2020image}) directly applies the architecture of Transformer on image classification and gets encouraging performance. ViT and its variants~\cite{arnab2021vivit,liang2022stargazer,bertasius2021space,fan2021multiscale,liu2021video,yan2022multiview}
achieve outstanding results in both image and video processing in recent years.

\noindent\textbf{Action Recognition$/$Classification.}
The research of action recognition has advanced with both new datasets and new models.
One of the largest modern benchmarks for action recognition is the Kinetics dataset \cite{kay2017kinetics}.
The Kinetics dataset proposes a
large
benchmark with more categories and more videos (\textit{e.g.}, 400 categories 160,000 clips in  \cite{kay2017kinetics} and 700 categories in ~\cite{carreira2019short}) as
more challenging
benchmarks  compared to previous datasets like UCF-101~\cite{ucf101}.
The Moments-in-Time~\cite{monfort2019moments} (MiT) dataset provides a million short video clips that cover 305 action categories.
Note that it is
infeasible
for
Kinetics and MiT datasets
to
cover
all the possible actions in all possible scales.
For example, surveillance actions~\cite{chen2019minding,chang2019mmvg} are missing in the two datasets.
Many new approaches ~\cite{tran2018closer,zhao2018trajectory,lin2019tsm,feichtenhofer2019slowfast,yang2020temporal,liang2020spatial,chen2019minding,chang2019mmvg} have been carried out on these datasets, of which the SlowFast network \cite{feichtenhofer2019slowfast} and MViT \cite{fan2021multiscale} obtain
promising
performance.
We can see that the trend of action recognition in the last two decades is to collect
larger
datasets (\textit{e.g.}, Kinetics)
and
build
models with a larger capacity.

\noindent\textbf{Multi-dataset Co-Training.}
Previously, multi-dataset co-training has been explored in the image domain such as detection~\cite{zhou2022simple,wang2019towards} and segmentation~\cite{lambert2020mseg}.
Several works~\cite{munro2020multi,chen2019temporal, Song_2021_CVPR,hinami2018multimodal}
were proposed
to combine multiple video datasets for training.
Larger datasets
often deliver
better results. Combining
multiple
datasets to boost data size, and
improve the final performance \cite{ghadiyaram2019large}, and the simultaneous use of multiple datasets
is
also likely to
alleviate
the damaging impact of dataset bias. OmniSource \cite{duan2020omni} utilizes
web images as part of the training dataset to expand the diversity of the training data to reduce dataset bias.
VATT~\cite{akbari2021vatt}
uses
additional multi-modal data for self-supversied pretraining and finetunes on downstream datasets.
CoVeR~\cite{zhang2021co} combines image and video training even during the finetuning stage and reports significant performance boost compared to single-dataset training.
PolyViT~\cite{likhosherstov2021polyvit} further extends to training with image, video and audio datasets using different sampling procedures.
In this paper, we propose a simple yet effective way (no multi-stage training, no complex dataset-wise sampling and hyper-parameter tuning) for multi-action-dataset training, without the use of any image or additional data from other modality.

\noindent\textbf{Video Domain Generalization (VDG).}
Our work is also related but different from video domain generalization~\cite{zhou2022domain}.
The key distinction is that our goal is to train a single model on multiple related tasks (multiple action datasets) such that the model performs well on the same set of tasks, whereas VDG aims to generalize a model to unseen out-of-distribution target domain~\cite{yao2021videodg,chen2019temporal,choi2020shuffle,da2022dual,tzeng2017adversarial}.
These models still suffer from problem of parameter-inefficiency, meaning that separate models are needed for different target datasets.

\section{Conclusion}
In this paper, we present \fancyname, a robust multi-dataset training approach that maximizes information content of representation and learns intrinsic relations between individual datasets.
Our method can train parameter-efficient models that perform well across multiple datasets.

{
\bibliographystyle{plainnat.bst}