text:
\section{Introduction}
Inverse problems are ubiquitous and crucial to solve in scientific endeavors and our daily life. For example, astrophysicists use radio electromagnetic data to image galaxies and black holes~\citep{hogbom1974aperture,akiyama2019first}. Geoscientists use seismic recordings to reveal the internal structures of Earth~\citep{tarantola1984inversion,tromp2005seismic,virieux2009overview}. Biomedical engineers and doctors use X-ray projections, ultrasound measurements, and magnetic resonance data to reconstruct images of human tissues and organs~\citep{lauterbur1973image,gemmeke20073d,lustig2007sparse}.

Solving an inverse problem starts with the definition of a forward mapping from parameters \({\mathbf{m}}\) to data \({\mathbf{d}}\), which we formally write as
\begin{equation}
	{\mathbf{d}} = f({\mathbf{m}}) + \bm{\epsilon},
\end{equation}
where \(f\) stands for a forward model that usually describes some physical process, \(\bm{\epsilon}\) denotes noise, \({\mathbf{d}}\) the observed data, and \({\mathbf{m}}\) the parameters to be estimated. The forward model can be either linear or nonlinear and either explicit or implicitly defined by solving partial differential equations (PDEs). This study considers three representative inverse problems: \textbf{Compressive Sensing MRI}, \textbf{Deblurring}, and \textbf{Eikonal (traveltime) Tomography}, which have important applications in medical science, geoscience, and astronomy. The details of each problem and its forward model are in \appref{append:background_forward_models}.

The forward problem maps \({\mathbf{m}}\) to \({\mathbf{d}}\), while the inverse problem estimates \({\mathbf{m}}\) given \({\mathbf{d}}\). Unfortunately, inverse problems are generally under-determined with infinitely many compatible solutions and intrinsically ill-posed because of the nature of the physical system. Worse still, the observed data are usually noisy, and the assumed forward model might be inaccurate, exacerbating the ill-posedness. These challenges require using regularization to inject \textit{a priori} knowledge into inversion processes to obtain plausible and high-fidelity results. Therefore, an inverse problem is usually posed as an optimization problem:
\begin{equation}
  \argmin_{{\mathbf{m}}} ~(1/2) \left\|{\mathbf{d}} - f \left({\mathbf{m}}\right) \right\|^2_2 + \mathcal{R}({\mathbf{m}}),
  \label{eqn:inv_reg}
\end{equation}
where \(\mathcal{R}({\mathbf{m}})\) is the regularization term. Beyond traditional regularization methods such as the Tikhonov regularization and Total Variation (TV) regularization, deep generative models (DGM), such as VAEs~\citep{kingma2013auto}, GANs~\citep{goodfellow2014generative}, and normalizing flows~\citep{dinh2014nice,dinh2016density,kingma2016improving,papamakarios2017masked,marinescu2020bayesian}, have shown great potential for regularizing inverse problems~\citep{bora2017compressed,van2018compressed,hand2018phase,ongie2020deep,asim2020invertible,mosser2020stochastic,li2021multiparameter,siahkoohi2021preconditioned,pmlr-v139-whang21a,cheng2022inout,daras2021intermediate,daras2022score}. Such deep generative models directly learn from training data distributions and are a powerful and versatile prior. They map latent vectors \({\mathbf{z}}\) to outputs \({\mathbf{m}}\) distributed according to an \textit{a priori} distribution: \( {\mathbf{m}} = g({\mathbf{z}}) \sim p_{\texttt{target}}, {\mathbf{z}} \sim \mathcal{N}(\mathbf{0}, {\mathbf{I}}) \), for example. The framework of DGM-regularized inversion~\citep{bora2017compressed} is
\begin{equation}
  \argmin_{{\mathbf{z}}} ~(1/2) \left\|{\mathbf{d}} - f \circ g\left({\mathbf{z}}\right) \right\|^2_2 + \mathcal{R^\prime}({\mathbf{z}}),
\label{eqn-augmented_inversion}
\end{equation}
where the deep generative model \(g\), whose layers are frozen, reparameterizes the original variable \({\mathbf{m}}\), acting as a hard constraint. Instead of optimizing for \({\mathbf{m}}\), we now estimate the latent variable \({\mathbf{z}}\) and retrieve the inverted \({\mathbf{m}}\) by forward mappings. Since the latent distribution is usually a standard Gaussian, the new (optional) regularization term \(\mathcal{R^\prime}({\mathbf{z}})\) can be chosen as \(\beta \|{\mathbf{z}}\|^2_2\) for GANs and VAEs, where \(\beta\) is a weighting factor. See \appref{append:glow_inv_formulation} for more details on a similar formulation for normalizing flows. Since the optimal \(\beta\) depends on the problem and data, tuning \(\beta\) is highly subjective and costly.

However, this formulation of DGM-regularized inversion still leads to unsatisfactory results if the data are noisy or the forward model is inaccurate, as shown in \figref{fig:figs_intro}, even if we fine-tune the weighting parameter \(\beta\).
\begin{figure}[t!]
	\centering
	\includegraphics[width=0.8\textwidth]{figures/visual_motivation.pdf}
	\caption{The comparison of images generated by a deep generative model (DGM) -- Glow using latent tensors that deviate from a spherical Gaussian distribution and those after corresponding corrections. The visual effects highlight the necessity of keeping the latent tensor within such a distribution during inversion. The second column shows the characteristics of deviated latent tensors: (a) histogram: i.i.d.~ components but the distribution is skewed; (b) histogram: i.i.d.~ components but the distribution is heavy-tailed; (c) latent tensor image: non-i.i.d.~ entries. The first column shows the corresponding outputs of a Glow network. The third column shows latent tensors corrected by (a) the Yeo-Johnson layer (YJ), (b) the Lambert $W\times F_X$ layer (LB), and (c) the full set of our Gaussianization layers (G layers). Those corrected latent tensors map to realistic images shown in the fourth column. All latent tensors have a norm of 0.7$\sqrt{\text{vec dim}}$ because of the Gaussian Annulus Theorem (\appref{append:gaussian_typical_set}) and the fact that Glow works best with a temperature smaller than one (see \figref{fig:various_radii}). Additional examples for \textbf{StyleGAN2} and \textbf{Stable Diffusion}~\citep{Rombach_2022_CVPR} can be found in \appref{append:add_latent_deviations}.}
	\label{fig:effects_deviate_glow}
\end{figure}

To analyze this problem, first recall that a well-trained DGM has a latent space (usually) defined on a standard Gaussian distribution. In other words, a DGM either only sees standard Gaussian latent tensors during training (\textit{e.g.}, GANs) or learns to establish 1-1 mappings between training examples and typical samples from the standard Gaussian distribution (\textit{e.g.}, normalizing flows, \appref{append:KL_duality}). As a result, the generator may map out-of-distribution latent vectors to unrealistic results. We show in \figref{fig:effects_deviate_glow} the visual effects of several types of deviations of latent vectors from a spherical Gaussian (with a temperature of 0.7). It can be seen that (1) the latent tensor should have i.i.d.~ entries, and (2) these entries should be distributed as a 1D standard Gaussian in order to generate plausible images. Since the traditional DGM-regularized inversion lacks such Gaussianity constraint, we conjecture\footnote{Out-of-distribution/typicality tests are very challenging for high-dimensional data~\citep{rabanser2019failing,nalisnick2018do}. Also, since latent tensors are Gaussian/Gaussian-like, it is hard to conduct dimension reduction for such tests.} that the latent tensor deviates from the desired high-dimensional standard Gaussian distribution during inversion, leading to poor results. Our observations and reasoning motivate us to propose a set of differentiable Gaussianization layers to reparameterize and Gaussianize the latent vectors of deep generative models (\textit{e.g.}, StyleGAN2~\citep{karras2020analyzing} and Glow~\citep{kingma2018glow}) for inverse problems.

\section{Method}
\label{sec:method}
\subsection{Overview - reparameterization using Gaussianization layers}
\label{sec:reparam_gaussian_layers}
Our solution is based on a necessary condition of the standard Gaussian prior on latent tensors. Let us define a \textit{partition} of a latent tensor \({\mathbf{z}} \in \mathbb{R}^n\) as the collection of non-overlapping patches \(P({\mathbf{z}}) = \{{\mathbf{z}}_{i}\}_{i = 1,\cdots,N}\), where the patches \({\mathbf{z}}_i \in \mathbb{R}^D\) are of the same dimension and can be assembled as \({\mathbf{z}}\), \textit{i.e.}, \(n = N\times D\). If the latent tensor \({\mathbf{z}} \sim \mathcal{N}(\bm{0}, {\mathbf{I}})\), then for all \({\mathbf{z}}_i\) from any partition \(P({\mathbf{z}})\), we have \({\mathbf{z}}_i \sim \mathcal{N}(\bm{0}, {\mathbf{I}})\).

Note that \({\mathbf{z}}\) is a symbolic representation of the latent tensor. In a specific DGM, \({\mathbf{z}}\) can be either a 2D/3D tensor or a list of such tensors corresponding to a multi-scale architecture (\appref{append:reparam_schemes}). Even though there are numerous partition schemes, such as random grouping of components, we choose the simplest: partitioning \({\mathbf{z}}\) based on neighboring components, which works well in practice.

To constrain \({\mathbf{z}}_i \sim \mathcal{N}(\bm{0}, {\mathbf{I}})\) during inversion, we reparameterize \({\mathbf{z}}_i\) by constructing a mapping \(h: {\mathbf{v}}_i \rightarrow {\mathbf{z}}_i\), such that \({\mathbf{z}}_i \sim \mathcal{N}(\bm{0}, {\mathbf{I}})\). The new variables \({\mathbf{v}}_i\) are of the same dimension as \({\mathbf{z}}_i\). Suppose that we have constructed the patch-level mapping \(h\), we can obtain a mapping \(h^\dagger\) at the tensor level, such that \({\mathbf{z}} = h^\dagger({\mathbf{v}})\), where \({\mathbf{v}}\) is assembled by \({\mathbf{v}}_i\) in the same way as \({\mathbf{z}}\) by \({\mathbf{z}}_i\). For example, \(h^\dagger = \mathtt{diag}(h, \cdots, h)\) if patches are extracted from neighboring components and are concatenated into a vectorized \({\mathbf{v}}\). Hence, the original DGM-regularized inversion~\ref{eqn-augmented_inversion} becomes
   \begin{equation}
      \argmin_{{\mathbf{v}}} ~(1/2) \left\|{\mathbf{d}} - f \circ g \circ   h^\dagger({\mathbf{v}})   \right\|^2_2.
	  \label{eqn:reparameterized_augmented_inversion}
   \end{equation}
Since we are imposing a constraint through reparameterization, there is no need to include the regularization term \(\mathcal{R^\prime}({\mathbf{z}})\). The new formulation is still an unconstrained optimization problem, enabling us to use highly efficient unconstrained optimizers, such as L-BFGS~\citep{nocedal2006numerical} and ADAM~\citep{kingma2015adam}.

The remaining critical piece is to construct \(h\), and it leads to our Gaussianization layers. First, we translate the constraint of \({\mathbf{z}}_i = h({\mathbf{v}}_i) \sim \mathcal{N}(\bm{0}, {\mathbf{I}})\) into the following optimization problem:
\begin{equation}
  \argmin_{h} D_{\mathrm{KL}}\left( p_{Z}\left( h\left({\mathbf{v}}_i\right) \right)\Vert \mathcal{N}(\bm{0}, {\mathbf{I}}) \right),
\end{equation}
where \(p_Z\) is the probability density function (PDF) of \({\mathbf{z}}_i\). Second, we adopt the framework proposed by precursor works of normalizing flows on Gaussianization~\citep{chen2000gaussianization,laparra2011iterative} to solve this optimization problem. The KL-divergence can be decomposed as the sum of the multi-information \(I({\mathbf{z}}_i)\) and the marginal negentropy \(J_m({\mathbf{z}}_i)\)~\citep{chen2000gaussianization,meng2020gaussianization}:
   \begin{equation}
   		D_{\mathrm{KL}} \left(p_Z\left({\mathbf{z}}_i \right) \Vert \, \mathcal{N}\left(\bm{0}, \mathbf{I}\right)\right) = I({\mathbf{z}}_i) + J_m({\mathbf{z}}_i),
		\label{eqn:KL_div}
   \end{equation}
where
   \begin{equation}
   		I({\mathbf{z}}_i) = D_{\mathrm{KL}} \left( p_Z\left( {\mathbf{z}}_i \right) \Big\Vert \prod_j^D p_j(z_i^{(j)}) \right), \text{ and  } J_m({\mathbf{z}}_i) = \sum_{j=1}^D D_{\mathrm{KL}} \left( p_j(z_i^{(j)} ) \Big\Vert \mathcal{N}(0, 1) \right).
       \label{eqn:1D_gaussianization}
   	\end{equation}
Here \(z_i^{(j)}\) denotes the \(j\)-th component of patch vector \({\mathbf{z}}_i\), and \(p_j\) stands for the marginal PDF for that component. The multi-information \(I({\mathbf{z}}_i)\) quantifies the independence of the components of \({\mathbf{z}}_i\), while the marginal negentropy \(J_m({\mathbf{z}}_i)\) describes how close each component is to a 1D standard Gaussian. With this decomposition, the optimization procedure depends on the facts: (1) the KL divergence and a standard Gaussian in \eqref{eqn:KL_div} are invariant to an orthogonal transformation, and (2) the multi-information term is invariant to a component-wise invertible differentiable transformation (\appref{append:KL-MI-proofs}). As a result, we perform Gaussianization in two steps:

1. Minimize the multi-information \(I({\mathbf{z}}_i) \). This is done by an orthogonal transformation that keeps the overall KL divergence the same but increases the negentropy \(J_m({\mathbf{z}}_i)\). We achieve this by using our independent component analysis (ICA) layer. ICA is the optimal choice since it maximizes non-Gaussianity so that the subsequent marginal Gaussianization step removes it and results in a large decrease in \(D_{\mathrm{KL}}\left( p_{Z}\left( h\left({\mathbf{v}}_i\right) \right)\Vert \mathcal{N}(\bm{0}, {\mathbf{I}}) \right)\).

2. Minimize the marginal negentropy \(J_m({\mathbf{z}}_i)\) by component-wise operations that perform 1D Gaussianization of marginal distributions \(p_j, {}_{j=1,\cdots,D}\). The multi-information does not change under component-wise invertible operations. Therefore, the overall KL divergence between \({\mathbf{z}}_i\) and the Gaussian distribution decreases.

The Gaussianization steps are well-aligned with the motivating example (\figref{fig:effects_deviate_glow}). To constrain DGM outputs to be plausible, one should make components within latent tensor patches independent (or destroy the patterns) (\figref{fig:effects_deviate_glow}(c)) and shape the 1D distribution as close as possible to Gaussian (\figref{fig:effects_deviate_glow}(a)(b)).

We will see that \(h\) is parameterized by an orthogonal matrix and two scalar parameters in 1D Gaussianization layers. Unlike conventional neural network layers, the input-data-dependent layers are not defined by learning from a dataset ahead of time but by solving certain optimization problems on the job (\figref{fig:opt_layer}). Special care should be taken to implement the gradient computation correctly and ensure that they pass the finite-difference convergence test (\appref{append:opt_layer_backward}). As an overview, the composition of our proposed layers is:
\begin{equation*}
  {{\mathbf{v}} \boldsymbol{\rightarrow} \textbf{Whitening} \boldsymbol{\rightarrow} \textbf{ICA} \boldsymbol{\rightarrow} \textbf{Yeo-Johnson} \boldsymbol{\rightarrow} \textbf{Lambert }W\times F_X \boldsymbol{\rightarrow} \textbf{Standardization} \boldsymbol{\rightarrow} {\mathbf{z}}},
\end{equation*}
where whitening and ICA belong to the first step and the rest belong to the second step. We will discuss in \appref{append:additional_discussions} some possible simplifications of the layers in practice after our ablation studies.

The overall proposed inversion process (one iteration) is illustrated in \figref{fig:inversion_illustration}.
\begin{figure}[htpb]
	\centering
			\includegraphics[width=0.8\textwidth]{figures/Inversion_illustrate.pdf}
	\caption{Illustration of the proposed inversion process. Gradient computation in the Gaussianization layers is enabled by the implicit function theorem and automatic differentiation (\appref{append:opt_layer_backward}). We use the L-BFGS optimizer~\citep{nocedal2006numerical} to update \({\mathbf{v}}\).}
	\label{fig:inversion_illustration}
\end{figure}

\subsection{Reducing multi-information -- ICA layer}
\label{sec:ica_layer}
The orthogonal matrix \({\bm{W}}\) is constructed by the independent component analysis (ICA). The input is the patch vectors \(\{{\mathbf{v}}_{i}\vert {}_{i = 1,\cdots,N}\}\). The ICA algorithm first computes the input-dependent orthogonal matrix \({\bm{W}}\) and then computes \({\mathbf{p}}_i = {\bm{W}}^\top {\mathbf{v}}_i, {}_{i = 1,\cdots,N}\) as the output. The orthogonal matrix \({\bm{W}}\) makes the entries of each \({\mathbf{p}}_i\) independent random variables.

We use the FastICA algorithm~\citep{hyvarinen1999fast,hyvarinen2000independent}, which employs a fixed-point algorithm to maximize a contrast function \(\Phi\) (\textit{e.g.}, the logcosh function) for our ICA layer to reduce multi-information.

The FastICA algorithm typically requires that the data are pre-whitened. We adopt the ZCA whitening method or the iterative whitening method introduced in \citet{hyvarinen1999fast} (\appref{append:G_layers_details}).
With the whitened data, we compute \({\bm{W}}\) using a damped fixed-point iteration scheme:
\begin{equation}
  {\bm{W}} = \frac{1}{N} \left[\alpha {\bm{V}} \phi\left({\bm{W}}^\top {\bm{V}}\right)^\top - {\bm{W}} \mathtt{diag}\left( \phi'\left( {\bm{W}}^\top {\bm{V}} \right) \mathbf{1} \right)\right],
  \label{eqn:mat_form_ica}
\end{equation}
where \(\mathbf{1}\) is an all-one vector, the column vectors of \({\bm{V}}\) are \(\{{\mathbf{v}}_{i}\vert {}_{i = 1,\cdots,N}\}\), \(\phi(\cdot) = \Phi'(\cdot)\), \(\alpha\in (0,1)\), and we use \(\alpha=0.8\) throughout our experiments. To save computation time, we only perform a maximum of 10 iterations. The details of the whole algorithm can be found in \appref{append:G_layers_details}.

We set the initial \({\bm{W}}\) as an identity matrix. If the input vectors are already standard Gaussian (in-distribution), the computed \({\bm{W}}\) will still be an identity matrix, which maps the input to the same output. In practice, the empirical distribution from finite samples is not a standard Gaussian, so \({\bm{W}}\) is not an identity matrix but another orthogonal matrix, which still maps standard Gaussian input vectors to standard Gaussian vectors as output. For this reason, we sample starting \(\{{\mathbf{v}}_{i}\vert {}_{i = 1,\cdots,N}\}\) from the standard Gaussian distribution to start inversions with plausible outputs.

\subsection{Reducing marginal negentropy}
For 1D Gaussianization, we choose a combination of the Yeo-Johnson transformation that reduces skewness and the Lambert \(W\times F_X\) transformation that reduces heavy-tailedness. Both are layers based on optimization problems with only one parameter, which is cheap to compute and is easy to back-propagate the gradient. \Eqref{eqn:1D_gaussianization} requires us to perform such 1D transformations for each component of the random vectors. In other words, we need to solve the same optimization problem for \(D\) times, which imposes a substantial computational burden. Instead, we empirically find it acceptable to share the same optimization-generated parameter across all components. In other words, we perform only a single 1D Gaussianization, treating all entry values in the latent vector as the data simultaneously.

\paragraph{Power transformation layer}
We propose to use the power transformation or Yeo-Johnson transformation~\citep{yeo2000new} to reduce the skewness of distributions. As shown in \figref{fig:lambert_yeojohnson_activation}(a), the form of the Yeo-Johnson activation function depends on parameter \(\lambda\). If \(\lambda = 1\), the mapping is an identity mapping. If \(\lambda > 1\), the activation function is convex, compressing the left tail and extending the right tail, reducing the left-skewness. If \(\lambda < 1\), the activation function is concave, which oppositely reduces the right-skewness.
We refer the readers to \appref{append:G_layers_details} for details.
\begin{wrapfigure}{R}{0.48\textwidth}
	\centering
		\setlength{\tabcolsep}{0.1cm}
		\begin{tabular}{ll}
			\small{(a)} & \small{(b)} \\
			\includegraphics[width=0.22\textwidth]{./figures/yeo_johnson.jpg} & \includegraphics[width=0.22\textwidth]{./figures/lambertw_tail.jpg}
		\end{tabular}
	\caption{The nonlinear activation functions from (a) the power transformation (Yeo-Johnson) layer and (b) the Lambert \(W \times F_X\) layer.}
	\label{fig:lambert_yeojohnson_activation}
\end{wrapfigure}

\paragraph{Lambert \(W\times F_X\) layer}
Due to noise and inaccurate forward models, we observe that the distribution of latent vector values tends to be shaped as a heavy-tailed distribution during the inversion process. To reduce the heavy-tailedness, we adopt the Lambert \(W\times F_X\) method detailed in~\citet{goerg2015lambert}.

We use the parameterized Lambert \(W\times F_X\) distribution family to approximate a heavy-tailed input and solve an optimization to estimate an optimal parameter \(\delta\) (\appref{append:G_layers_details}), with which the inverse transformation maps the heavy-tailed distribution towards a Gaussian.

\Figref{fig:lambert_yeojohnson_activation}(b) shows that the Lambert \(W\times F_X\) layer acts as a nonlinear squashing function. As \(\delta\) increases, it compresses more the large values and reduces the heavy-tailedness. Intuitively, the Lambert \(W\times F_X\) layer can also be interpreted as an intelligent way of imposing constraints on the range of values instead of a simple box constraint. We refer the readers to \appref{append:G_layers_details} for more details about the optimization problem and implementation.

\paragraph{Standardization with temperature} Since the Lambert \(W\times F_X\) layer output may not necessarily have a zero mean and a unit (or a prescribed) variance, we standardize the output using
\begin{equation}
	{\bm{z}} = ({\bm{x}} - \mathbb{E}[{\bm{x}}])/\sqrt{\mathrm{Var}({\bm{x}})} *\gamma,
\end{equation}
where \(\gamma\) is the temperature parameter suggested in \citet{kingma2018glow}.

\section{Related work}
\label{sec:related_work}
\paragraph{End-to-end NNs for inverse problems} There are vast amounts of end-to-end neural networks designed for inverse problems, using CNNs~\citep{chen2017low,jin2017deep,sriram2020end}, GANs~\citep{mardani2018deep,lugmayr2020srflow,wang2018esrgan}, invertible networks~\citep{ardizzone2018analyzing}, and diffusion models~\citep{kawar2022denoising}. The general idea is simple: train a neural network that directly maps observed data to estimated parameters. Even though such methods seem effective in a few applications, DGM-regularized inversion with our Gaussianization layers is preferable for the following reasons. First, the forward modeling can be so expensive computationally that it is infeasible to collect a decent training datasets for some applications. For example, one large-scale fluid mechanics or wave propagation simulation can take hours, if not days. Second, the relationship between parameters and data can be highly nonlinear, and multiple solutions may exist. An end-to-end network may map data to interpolated solutions that are not realistic. In comparison, our method can start from different initializations or even employ sampling techniques to address this issue. Third, the configuration of data collection can change from experiment to experiment. It is unbearable, for example, to re-train the network each time we change the number and locations of sensors. In contrast, not only can our method deal with this situation, but it can even use the same DGM for different forward models, as we can see in the compressive sensing MRI and eikonal tomography examples. Almost all end-to-end methods are only applied to linear inverse problems, whereas our Gaussianization layers are also effective in nonlinear problems.

\paragraph{Other techniques to improve DGM-regularized inversion}
In high-dimensional space, the probability mass of a standard Gaussian distribution concentrates within the so-called Gaussian typical set (\appref{append:gaussian_typical_set}). To be in the Gaussian typical set, one necessary but not sufficient condition is to be within an annulus area around a high-dimensional sphere (\appref{append:gaussian_typical_set}). Utilizing this geometric property, DGM-regularized inversion methods like \citet{bojanowski2017optimizing} and \citet{liang2021flow} force updated latent vectors to stay on the sphere. This strategy is closely related to spherical interpolation~\citep{white2016sampling}. We call this strategy the \textbf{spherical constraint} for inversion. In the original StyleGAN2 paper, the authors also noticed that in image projection tasks, the noise maps tend to have leakage from signals~\citep{karras2020analyzing} -- the same phenomenon we discussed. They proposed a multi-scale noise regularization term (\textbf{NoiseRlg}) to penalize spatial correlation in noise maps. We extend the same technique to our inverse problems for comparison. Note that we use a whitening layer before the ICA layer. The \textbf{whitening layer} can be used alone, similar to \citet{huang2018decorrelated} and \citet{siarohin2018whitening}, whose performance will be reported in ablation studies. Also, \citet{wulff2020improving} observed that for StyleGAN2, the Leaky ReLU function can ``Gaussianize" latent vectors in the W space. \textbf{CSGM-w}~\citep{kelkar2021prior} utilizes this \textit{a priori} knowledge to improve DGM-regularized compressive sensing problems. To further compare with the Gaussianizaion layers, we in addition propose an alternative idea that reparameterizes latent vectors using learnable orthogonal matrices (Cayley parameterization) and fixed latent vectors, which is closely related to the work of orthogonal over-parameterized training~\citep{liu2021orthogonal} (\appref{append:ortho_param} ). Recently, score-based generative models start to show promise for solving inverse problems~\citep{jalal2021robust,song2021solving}. However, they have been mainly applied to linear inverse problems and challenged by noisy data~\citep{kawar2021snips}. Besides, scored-based methods might not work for certain physics-based inverse problems, since Gaussian noise parameters may break the physics simulation engine\footnote{For example, in elastic waveform inversion, initializing material properties using Gaussian noise may create unrealistic scenarios where the P-wave speed is lower than the S-wave speed at some spatial locations.}.

\section{Experiments}
We consider three representative inversion problems for testing: compressive sensing MRI, image deblurring, and eikonal traveltime tomography. For MRI and eikonal tomography, we used synthetic brain images as inversion targets and used the pre-trained StyleGAN2 weights from \citet{kelkar2021prior} (trained on data from the databases of fastMRI~\citep{zbontar2018fastmri,knoll2020fastmri}, TCIA-GBM~\citep{scarpace2016radiology}, and OASIS-3~\citep{lamontagne2019oasis}) for regularization. We used the test split of the CelebA-HQ dataset~\citep{karras2017progressive} for deblurring, and the DGM is a Glow network trained on the training split. We refer readers to \appref{append:datasets_training} for details on datasets and training.

We tested each parameter configuration in each inversion on 100 images (25 in the eikonal tomography due to its expensive forward modeling). Since the deep generative models are highly nonlinear, the results may get stuck in local minima. Thus, we started inversion using three different randomly initialized latent tensors for each of the 100 or 25 images, picked the best value among the three for each metric, and reported the mean and standard deviation of those metrics, except for CSGM-w, TV, and NoiseRlg, where the initialization is fixed. The metrics we used were PSNR, SSIM~\citep{wang2004image}, and an additional LPIPS~\citep{zhang2018perceptual} for the CelebA-HQ data. We used the LBFGS~\citep{nocedal2006numerical} optimizer in all experiments except TV, noise regularization, and CSGM-w, which use FISTA~\citep{beck2009fast} or ADAM~\citep{kingma2015adam}. The temperature was set to 1.0 for StyleGAN2 and 0.7 for Glow.
\subsection{Compressive sensing MRI using StyleGAN2}
The mathematical model of compressive sensing MRI is
\begin{equation}
  {\mathbf{d}} = {\bm{A}} {\mathbf{m}} + \bm{\epsilon},
  \label{eqn:mri_eq}
\end{equation}
where ${\bm{A}} \in \mathbb{C}^{M\times N}$ is the sensing matrix, which consists of FFT and subsampling in the k-space (frequency domain). \eqref{eqn:mri_eq} is an under-determined system, and we use \(\text{Accl}=N/M\) to denote the acceleration ratio. We also added i.i.d.~ complex Gaussian noise with a signal-to-noise ratio (SNR) of 20 dB or 10 dB to the measured data. See \appref{append:MRI} for more background information.

\tabref{tab:MRI_results_comparison} summarizes the comparison of results from total variation regularization (TV), noise regularization (NoiseRlg)~\citep{karras2020analyzing}, spherical constraint/reparameterization: \({\bm{z}} = {\bm{v}} / \|{\bm{v}}\|_2 * \sqrt{\texttt{dim}({\bm{v}})}\), CSGM-w~\citep{kelkar2021prior}, our proposed orthogonal reparameterization (Orthogonal), and our proposed Gaussianization layers (G layers). \Figref{fig:comparison_mri} shows examples of inversion results. In the base case where Accl=8x and SNR=20 dB, the Gaussianization layers gives the best scores. Their advantage gets more significant when data SNR decreases to 10 dB. We also notice that scores from all methods are greatly improved if we make the system better determined (\textit{i.e.}, Accl=2x). Also, the performance of TV, spherical constraint, and Gaussianization layers are similar in this scenario. We conclude that our proposed Gaussianization layers are effective and more robust than other methods.

\begingroup
\setlength{\tabcolsep}{1.2pt}
\begin{figure}[htpb]
    \centering%
    \newcommand{0.1}{0.11}
    \newcommand{\imgTrue}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_mri/tv_accl8x_snr20.0/#1/True.jpg}}
    \newcommand{\imgTV}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_mri/tv_accl8x_snr20.0/#1/Inv.jpg}}
    \newcommand{\imgSP}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_mri/sphere_accl8x_snr20.0/#1/Inv.jpg}}
    \newcommand{\imgCSGM}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_mri/csgm_accl8x_snr20.0/#1/Inv.jpg}}
    \newcommand{\imgNoiseRlg}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_mri/noisergl_accl8x_snr20.0/#1/Inv.jpg}}
    \newcommand{\imgOrtho}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_mri/ortho_accl8x_snr20.0/#1/Inv.jpg}}
    \newcommand{\imgGauss}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_mri/gaus_accl8x_snr20.0/#1/Inv.jpg}}
	\begin{tabular}{ccccccc}
	\small{True} & \small{TV} & \small{Spherical} & \small{CSGM-w} & \small{NoiseRlg} & \small{Orthogonal} & \small{G layers}\\
	\imgTrue{mri_110} & \imgTV{mri_110} &\imgSP{mri_110}& \imgCSGM{mri_110}
        & \imgNoiseRlg{mri_110} & \imgOrtho{mri_110} & \imgGauss{mri_110} \\
	\imgTrue{mri_104} & \imgTV{mri_104} &\imgSP{mri_104}& \imgCSGM{mri_104}
        & \imgNoiseRlg{mri_104} & \imgOrtho{mri_104} & \imgGauss{mri_104} \\
	\imgTrue{mri_106} & \imgTV{mri_106} &\imgSP{mri_106}& \imgCSGM{mri_106}
        & \imgNoiseRlg{mri_106} & \imgOrtho{mri_106} & \imgGauss{mri_106} \\
	\end{tabular}
	\caption{Comparison of compressive sensing MRI inversion results (Accl=8x, SNR=20 dB).}
	\label{fig:comparison_mri}
\end{figure}
\endgroup

\begingroup
\setlength{\extrarowheight}{1pt}
\begin{table}[htpb]
\caption{Comparison of compressive sensing MRI results.}
\label{tab:MRI_results_comparison}
\centering
\small
\begin{tabular}{l@{~~~~}c@{~~~~~}c@{~~~~~}c@{~~~~~}c@{~~~~~}c@{~~~~~}c@{~~~~~}}
 \toprule
  \multirow{2}{3em}{Method}
            & \multicolumn{2}{c}{Accl = 8x, SNR = 20 dB} & \multicolumn{2}{c}{Accl = 8x, SNR = 10 dB}
            & \multicolumn{2}{c}{Accl = 2x, SNR = 20 dB} \\
            \cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7}
            & PSNR$\uparrow$ & SSIM$\uparrow$ & PSNR$\uparrow$ & SSIM$\uparrow$ & PSNR$\uparrow$ & SSIM$\uparrow$ \\
 \hline
 TV         & 20.20\tiny{$\pm$1.33} & 0.60\tiny{$\pm$0.063}
            & 19.78\tiny{$\pm$1.32} & 0.57\tiny{$\pm$0.058}
            & \best{32.92\tiny{$\pm$1.36}} & \best{0.91\tiny{$\pm$0.024}} \\

 Spherical  & 26.93\tiny{$\pm$6.57} & 0.79\tiny{$\pm$0.159}
            & 22.78\tiny{$\pm$5.38} & 0.61\tiny{$\pm$0.246}
            & 32.90\tiny{$\pm$3.50} & 0.90\tiny{$\pm$0.061} \\

 CSGM-w     & 20.19\tiny{$\pm$2.11} & 0.57\tiny{$\pm$0.096}
            & 19.98\tiny{$\pm$1.94} & 0.55\tiny{$\pm$0.091}
            & 27.53\tiny{$\pm$2.98} & 0.74\tiny{$\pm$0.085} \\

 NoiseRgl   & 21.61\tiny{$\pm$2.27} & 0.50\tiny{$\pm$0.073}
            & 18.09\tiny{$\pm$1.05} & 0.27\tiny{$\pm$0.036}
            & 28.04\tiny{$\pm$1.46} & 0.66\tiny{$\pm$0.037} \\

 Orthogonal & 26.14\tiny{$\pm$5.79} & 0.78\tiny{$\pm$0.134}
            & 25.10\tiny{$\pm$5.08} & 0.77\tiny{$\pm$0.132}
            & 29.29\tiny{$\pm$5.32} & 0.84\tiny{$\pm$0.108} \\

 G layers   & \best{27.99\tiny{$\pm$5.70}} & \best{0.83\tiny{$\pm$0.128}}
            & \best{25.48\tiny{$\pm$4.76}} & \best{0.78\tiny{$\pm$0.149}}
            & 32.41\tiny{$\pm$4.60} & 0.90\tiny{$\pm$0.079} \\
\bottomrule
\end{tabular}
\end{table}
\endgroup

\paragraph{Ablation study}
\tabref{tab:ablation_mri} summarizes the ablation study on the components of the Gaussianization layers. We kept the standardization layer on for all cases. The conclusions are as follows: 1. The ICA layer is the most significant part in improving result scores; 2. The whitening/ZCA layer alone is not effective; 3. The Yeo-Johnson (YJ) and the Lambert \(W\times F_X\) (Lambt) layers are more effective when the noise level is higher (\textit{e.g.}, SNR=10 dB vs. 20 dB). Their performance seems data-dependent: when SNR=20 dB, YJ seems more effective, while Lambt gives the best scores when SNR=10 dB. But the overall improvement from these two layers is marginal. In practice, one may only use ICA and one of the 1D Gaussianization layers. Additionally, we tested the effect of patch size on the z (style) vectors (\appref{append:additional_exp}). We find that the largest possible patch size gives the best results.
\begingroup
\setlength{\extrarowheight}{1pt}
\begin{table}[htpb]
\caption{Ablation study of the Gaussianization layers.}
\label{tab:ablation_mri}
\centering
\small
\begin{tabular}{l@{~~}l@{~~~}l@{~~~~}c@{~~~~~}c@{~~~~~}c@{~~~~~}c@{~~~~~}c@{~~~~~}c@{~~~~~}}
 \toprule
  \multicolumn{3}{c}{\multirow{2}{3em}{Method}}
            & \multicolumn{2}{c}{Accl = 8x, SNR = 20 dB} & \multicolumn{2}{c}{Accl = 8x, SNR = 10 dB} \\
            \cmidrule(r){4-5} \cmidrule(r){6-7}
    &   &   & PSNR$\uparrow$ & SSIM$\uparrow$ & PSNR$\uparrow$ & SSIM$\uparrow$ \\
 \hline
  ICA (\ding{55}),\,& YJ (\ding{55}),& Lambt (\ding{55})
            & 26.93\tiny{$\pm$6.40} & 0.787\tiny{$\pm$0.153}
            & 22.98\tiny{$\pm$5.54} & 0.604\tiny{$\pm$0.252} \\

  ICA (\ding{55}),\,& YJ (\checkmark),& Lambt (\ding{55})
            & 26.92\tiny{$\pm$6.42} & 0.787\tiny{$\pm$0.156}
            & 23.14\tiny{$\pm$5.44} & 0.622\tiny{$\pm$0.245} \\

  ICA (\ding{55}),\,& YJ (\ding{55}),& Lambt (\checkmark)
            & 25.33\tiny{$\pm$5.89} & 0.743\tiny{$\pm$0.163}
            & 23.58\tiny{$\pm$5.19} & 0.695\tiny{$\pm$0.189} \\

  ZCA,\,& YJ (\ding{55}),& Lambt (\ding{55})
            & 27.08\tiny{$\pm$6.52} & 0.786\tiny{$\pm$0.157}
            & 22.94\tiny{$\pm$5.50} & 0.623\tiny{$\pm$0.235} \\

  ICA (\checkmark),\,& YJ (\ding{55}),& Lambt (\ding{55})
            & 27.91\tiny{$\pm$5.77} & 0.824\tiny{$\pm$0.129}
            & 25.22\tiny{$\pm$4.76} & 0.770\tiny{$\pm$0.154} \\

  ICA (\checkmark),\,& YJ (\checkmark),& Lambt (\ding{55})
            & \best{27.99\tiny{$\pm$5.70}} & \best{0.831\tiny{$\pm$0.128}}
            & 25.48\tiny{$\pm$4.76} & \best{0.779\tiny{$\pm$0.149}} \\

  ICA (\checkmark),\,& YJ (\ding{55}),& Lambt (\checkmark)
            & 27.21\tiny{$\pm$5.74} & 0.816\tiny{$\pm$0.125}
            & \best{25.57\tiny{$\pm$4.98}} & \best{0.779\tiny{$\pm$0.148}} \\

  ICA (\checkmark),\,& YJ (\checkmark),& Lambt (\checkmark)
            & 27.37\tiny{$\pm$5.90} & 0.819\tiny{$\pm$0.135}
            & 25.09\tiny{$\pm$4.91} & 0.771\tiny{$\pm$0.144} \\

\bottomrule
\end{tabular}
\end{table}
\endgroup

\subsection{Image deblurring using Glow}
The mathematical model for deblurring is
\begin{equation}
  {\mathbf{d}} = {\bm{H}} \ast {\mathbf{m}} + \bm{\epsilon},
\end{equation}
where \({\bm{H}}\) is a smoothing filter and \(\ast\) denotes convolution. We used a Gaussian smoothing filter with a standard deviation of 3, and added noise \(\bm{\epsilon}\sim \mathcal{N}(\mathbf{0}, 50^2 {\mathbf{I}})\) to the observed data. Although the system may not be under-determined, the high-frequency information is lost due to low-pass filtering; hence this is also an ill-posed problem. Though we only tested on facial images, deblurring has wide applications in astronomy and geophysics. See \appref{append:deblurring} for more background information.
\begingroup
\setlength{\tabcolsep}{1pt}
\begin{figure}[htpb]
    \centering%
    \newcommand{0.1}{0.09}
    \newcommand{\imgTrue}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_deblur/gaus_p2/#1/True.jpg}}
    \newcommand{\imgSmooth}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_deblur/gaus_p2/#1/Obs.jpg}}
    \newcommand{\imgSph}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_deblur/spherical/#1/Inv.jpg}}
    \newcommand{\imgOrt}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_deblur/ortho/#1/Inv.jpg}}
    \newcommand{\imgGau}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_deblur/gaus_p1/#1/Inv.jpg}}
 \begin{tabular}{l@{~}|@{~}l}

    \begin{tabular}{c c c c c}
    \scriptsize{Original} & \scriptsize{Observed} & \scriptsize{Spherical} & \scriptsize{Orthogonal} & \scriptsize{G layers}\\
    \imgTrue{186521} & \imgSmooth{186521} & \imgSph{186521} &\imgOrt{186521} & \imgGau{186521} \\
    \imgTrue{185120} & \imgSmooth{185120} & \imgSph{185120} &\imgOrt{185120} & \imgGau{185120} \\
    \end{tabular}
    &
    \begin{tabular}{c c c c c}
    \scriptsize{Original} & \scriptsize{Observed} & \scriptsize{Spherical} & \scriptsize{Orthogonal} & \scriptsize{G layers}\\
    \imgTrue{194792} & \imgSmooth{194792} & \imgSph{194792} &\imgOrt{194792} & \imgGau{194792}\\
    \imgTrue{185989} & \imgSmooth{185989} & \imgSph{185989} &\imgOrt{185989} & \imgGau{185989}\\
    \end{tabular}

\end{tabular}
	\caption{Comparison of deblurring results from different methods.}
	\label{fig:glow_deblur_celeba-hq}
\end{figure}
\endgroup

\tabref{tab:deblurring_results} and \figref{fig:glow_deblur_celeba-hq} show that the Gaussianization layers are also effective in Glow, better than using the spherical constraint or the orthogonal reparameterization. We also demonstrate the efficacy of Gaussianization layers when the forward model is inaccurate, in which case the induced error in data is not Gaussian (\appref{append:additional_exp}).

\paragraph{Ablation study}
\tabref{tab:deblurring_results} also shows that the two parameterization schemes (\appref{append:reparam_schemes}) for Glow using the Gaussianization layers have similar performance. Besides, we report the ablation study on the components of the Gaussianizaiton layers for Glow (\appref{append:additional_exp}).

\subsection{Eikonal tomography using StyleGAN2}
In acoustic wave imaging (e.g., ultrasound tomography), we excite waves using sparsely distributed sources one at a time at the boundary of the object. Then we reconstruct its internal structures (the spatial distribution of wave speed) given the first-arrival travel time recorded on the boundary. The following eikonal equation approximately describes the shortest travel time \(T({\bm{x}};{\bm{x}}_s)\) that the acoustic wave emerging from the source location \({\bm{x}}_s\) takes to reach location \({\bm{x}}\) inside the target object~\citep{yilmaz2001seismic}:
\begin{equation}
	|\nabla T\left( {\bm{x}}; {\bm{x}}_s \right)| = 1/ c\left( {\bm{x}} \right), \quad T({\bm{x}}_s;{\bm{x}}_s) = 0,
\end{equation}
where \(c({\bm{x}})\) is the wave propagation speed at each location. Both this eikonal PDE and the implicitly defined forward mapping \(c({\bm{x}})\rightarrow T({\bm{x}})\) are nonlinear, and there has been little research on DGM-regularized inverse problems with such nonlinear characteristics. The inverse problem is severely ill-posed, which is equivalent to a curved-ray tomography problem. See \appref{append:eikonal_tomography} for more background information. We added noise to the recorded travel time using the following formula: \(T_{\text{noisy}}({\bm{x}}_r;{\bm{x}}_s) = T({\bm{x}}_r;{\bm{x}}_s) (1 + \epsilon) \), where \(\epsilon \sim \mathcal{N}(0,0.001^2)\) and \({\bm{x}}_r\) denotes any receiver location. In other words, longer traveltime corresponds to larger uncertainties.

We show that the Gaussianization layers outperformed other methods in this tomography task in \tabref{tab:eikonal_results} and \figref{fig:comparison_tomo}. Note that this is a statistical conclusion. We also report an example where the spherical constraint works better than the Gaussianization layers (bottom right).

\begin{table}[htpb]
\centering
\makebox[0pt][c]{\parbox{\textwidth}{%
    \begin{minipage}[t]{0.5\textwidth}\centering
      \setlength{\extrarowheight}{1pt}
      \caption{Deblurring results using Glow.}
      \label{tab:deblurring_results}
      \vspace{0.2em}
      \centering
      \small

      \begin{tabular}{l@{~~~}|c@{~~~}c@{~~~}c@{~~~}}
      \toprule
        Method      & LPIPS$\downarrow$ & PSNR$\uparrow$ & SSIM$\uparrow$ \\
      \hline

      Spherical  & 0.17\tiny{$\pm$0.06} & 21.78\tiny{$\pm$1.14} & 0.580\tiny{$\pm$0.066} \\

      Orthogonal   & 0.16\tiny{$\pm$0.06} & 21.91\tiny{$\pm$1.31} & 0.583\tiny{$\pm$0.063} \\

      G layers P1  & 0.13\tiny{$\pm$0.05} & 22.40\tiny{$\pm$1.34} & 0.583\tiny{$\pm$0.069} \\

      G layers P2  & \best{0.13\tiny{$\pm$0.05}} & \best{22.47\tiny{$\pm$1.27}} & \best{0.590\tiny{$\pm$0.064}} \\
      \bottomrule
      \end{tabular}

    \end{minipage}
    \hfill
    \begin{minipage}[t]{0.5\textwidth}\centering
      \setlength{\extrarowheight}{1pt}
      \caption{Eikonal tomography using StyleGAN2}
      \label{tab:eikonal_results}
      \vspace{0.2em}
      \centering
      \small
      \begin{tabular}{l@{~~~}|c@{~~~}c@{~~~}}
      \toprule
        Method      & PSNR$\uparrow$ & SSIM$\uparrow$ \\
      \hline

      TV  & 20.66\tiny{$\pm$1.21} & 0.543\tiny{$\pm$0.073} \\
      Spherical  & 22.19\tiny{$\pm$4.05} & 0.686\tiny{$\pm$0.144} \\
      G layers   & \best{24.80\tiny{$\pm$2.55}} & \best{0.803\tiny{$\pm$0.093}} \\
      \bottomrule
      \end{tabular}
    \end{minipage}
}}
\end{table}

\begingroup
\setlength{\tabcolsep}{1.2pt}
\begin{figure}[htpb]
    \centering%
    \newcommand{0.1}{0.11}
    \newcommand{\imgTrue}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_mri/tv_accl8x_snr20.0/#1/True.jpg}}
    \newcommand{\imgTV}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_tomo/tv_noisestd0.001_stride36_tv_beta10.0/#1/Inv.jpg}}
    \newcommand{\imgSP}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_tomo/sphere_noisestd0.001_stride36/#1/Inv.jpg}}
    \newcommand{\imgGauss}[1]{\includegraphics[width=0.1\linewidth]{figures/figures_inv_tomo/gaus_noisestd0.001_stride36_1_0_1_1/#1/Inv.jpg}}
  \begin{tabular}{c | c}

	\begin{tabular}{cccc}
	\small{True} & \small{TV} & \small{Spherical} & \small{G layers}\\
	\imgTrue{mri_104} & \imgTV{mri_104} &\imgSP{mri_104} & \imgGauss{mri_104} \\
	\imgTrue{mri_129} & \imgTV{mri_129} &\imgSP{mri_129} & \imgGauss{mri_129} \\
	\end{tabular}
  &
	\begin{tabular}{cccc}
	\small{True} & \small{TV} & \small{Spherical} & \small{G layers}\\
	\imgTrue{mri_123} & \imgTV{mri_123} &\imgSP{mri_123} & \imgGauss{mri_123} \\
	\imgTrue{mri_101} & \imgTV{mri_101} &\imgSP{mri_101} & \imgGauss{mri_101} \\
	\end{tabular} \\

\end{tabular}

	\caption{Comparison of eikonal tomography results from different methods.}
	\label{fig:comparison_tomo}
\end{figure}
\endgroup

\section{Discussion and conclusions}
\label{sec:conclusions}
We provide insights on the experiment results and further discuss this work's limitations, computational cost, and broader impact in \appref{append:additional_discussions}.

In summary, we have identified a critical problem in DGM-regularized inversion: the latent tensor can deviate from a typical example from the desired high-dimensional standard Gaussian distribution, leading to unsatisfactory inversion results. To address the problem, we have introduced the differentiable Gaussianization layers that reparameterize and Gaussianize latent tensors so that the inversion results remain plausible. In general, our method has achieved the best scores in terms of mean values and standard deviations compared with other methods, demonstrating our method's advantages and high performance in terms of accuracy and consistency. (Regarding the standard deviation of scores, we only compare with other methods with competitive mean scores, such as orthogonal reparameterization and spherical constraint.) Our proposed layers are plug-and-play, require minimal parameter tuning, and can be applied to various deep generative models and inverse problems.

redpajama_set_name: RedPajamaArXiv


Event Driven Network Automation is a term used to describe what large scale NetOps teams are doing to scale, deploy and manage networking infrastructure.
YAML data formatting and Jinja2 templating with Python glueing and executing.
Ansible/YAML and Netconf/API for configuration, execution operations.
BGPMon looks like it could be used to check up on changes in a Routed Core with BGP based Leaf-Spine Clos Fabric.
Zero Touch Provisioning – ZTP is best suited for quickly bringing up new devices.
An Orchestration-style GUI layer custom made for every domain in the network would definitely be required as well for various aspects of NetOps.
There can be human driven network automation but there can also be event driven network automation which can be termed as 'closed loop' with rule based actions defined by humans.
The events driven, closed loop, rule-based-actions execution layer would then be managed by humans. This layer would be evolving and to manage it there would be a requirement of necessary data structuring and scripting skills in addition to being mindful of what the impact is on the network layer (DC or WAN, both).


text:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>eu.agilejava</groupId>
    <artifactId>snoopee-micro</artifactId>
    <version>2.0.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>SnoopEE Micro</name>
    <description>SnoopEE MicroProfile Registration Client</description>

    <parent>
        <groupId>eu.agilejava</groupId>
        <artifactId>snoopee-discovery</artifactId>
        <version>2.0.0-SNAPSHOT</version>
    </parent>

    <dependencies>

        <dependency>
            <groupId>eu.agilejava</groupId>
            <artifactId>snoopee-client</artifactId>
            <version>2.0.0-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>org.eclipse.microprofile</groupId>
            <artifactId>microprofile</artifactId>
            <type>pom</type>
            <scope>provided</scope>
        </dependency>

    </dependencies>

</project>

redpajama_set_name: RedPajamaGithub


WOW!!!! You want Country!!! This Home is for YOU! Nestled in the Evans area ,1 mile from the school. Minutes to fishing or hunting. Next to Clear Creek WMA . 8 BEAUTIFUL Acres with a Nice 3 Bedroom and 2 Bath home. Lots of room to GROW. Make it your For Ever Home or Home away from Homes. Plus, Seller is Offering a $2000.00 Flooring allowance!! Call Robie Graham @ 337-404-8488 or Text 110285 to 35620 today !!!


text:
Q: Flex-4 Beta2 : Background image stopped working After upgrading a project from Flex4 Beta1 to Beta2, I've found that the background-image style is no longer supported on Halo components.
Eg:
<mx:VBox xmlns:mx="http://www.adobe.com/2006/mxml"
         backgroundImage="@Embed(source='myImage.png')" />

Note that jira issue SDK-23050 points out the following :

backgroundImage is not supported for Spark skins.

But this is not using a Spark skin, just a standard Halo component.
Anyone got any ideas?

A: <s:BorderContainer x="100" y="100" width="200" height="200"

                   backgroundImage="@Embed(source='image.jpg)" >
</s:BorderContainer>

Use the code above. Any spark:BorderContainer have the option backgroundImage, either other backgroundImage options. Add the image you want to your source and put for image.jpg your image's name with its extension. do not forget single quotation marks.

A: You have to create a skin class now. Use a VGroup and specify a custom made skin.

A: MX components in Flex 4 allow you to use Spark skins, and Adobe uses Spark skins by default so that MX and Spark components in the same application look like they belong together. Obviously, Spark skins support different functionality than Halo skins, and now you're expected to tweak the skins if you want them to look different (except for some simple color styles). Considering how creative designers can be, this is a much better approach than using only CSS because it's way more flexible, but it catches people familiar with the Flex 3 workflow by surprise.
It's not too hard to copy the default skin and add a background to it. However, you should also be able to use the old Halo skin to get the backgroundImage support, if that's what you prefer. I believe you want to tell your VBox to use mx.skins.halo.HaloBorder.

A: I ended up changing backgroundImage to backgroundSkin, and that solved the problem.

redpajama_set_name: RedPajamaStackExchange


text:
Face & Beard
Bradley Mountain
Crud Sweden
Iron & Resin
Mizo
Triumph&Disaster
Headware
Benzak
Iron and Resin
Fuel Motorcycles
Freenote
The Revive Club
Triumph & Disaster
50% Sale50% Sale
Summer SaleSummer Sale
Why I Started The Revive Club
Stolen iMagination
Now reading: Keeping The Kid Alive
Keeping The Kid Alive
As a kid, I dreamed of being The Bush Tucker Man. I used to think he had the best job in the world, and now at the age of forty three, I still think that. The adventures of this man are like a portal that still now allow me to climb in, and be transported back thirty years.
So partly this post is about wanting to share with you something that might also transport you back....but also the broader issue of keeping that childlike sense of hope and dreams alive as you get older. The willing suspension of disbelief, the disregard for trivial things such as practicalities and details.....but that burning flame of imagination that makes you feel like a kid again....and just maybe pushes you to take a little adventure of your own.
'The Bush Tucker Man' refers to a retired Major in the Australian army called Les Hiddins - Major Les Hiddins. Les was always something of an expert in the flora and fauna of the Australian bush - and it was this interest and knowledge that led to him being given the mother of all jobs by the army. Les was tasked with the job of travelling the Australian outback, researching how to survive on the land. Drawing on the ancient and deep knowledge of the Aboriginal communities, Les explores the outback in his Army issue Land Rover.
In the late 80s, Les was given his own TV series in Australia called 'The Bush Tucker Man'. It follows him around on his adventures, as he explores the land, camping each night, and explaining how to survive eating the food that can be found on the land - hence the name for the show. 'Tucker' is an Australian word for food for those of you scratching your heads at this point.
Years back I ordered the DVD box set from Australia. The quality of the recordings is loooooow....but for me it's part of its charm. I mean doesn't that say something - that an old pixelated video can stir your soul like that? I reckon if it was in 4k quality, it wouldn't quite have the same charm. The accompanying music through the shows is just perfect - evocative and uplifting. Les is a no-nonsense, say-what-you-mean Aussie and his enthusiasm, dry wit and love of his environment is both addictive and infectious. I feel like I've been with him on all these adventures.
For those of you who may be interested Les wears his signature Akubra hat that he's basically reshaped to give it that unique slant or 'bash' as Les refers to it. Apparently the shape he gave to his hat was a style very common to cattle men in northern Australia - sort of a style unique to them. It's the 'Sombrero' model hat that Akubra make, which in its original shape, looks quite different. Aside from his trusty army fatigues, he always wears a sturdy pair of Australian RM Williams boots.
There are a bunch of videos on YouTube you can see to give you a taste including this one where he meets up with Bushcraft legend Ray Mears and talks about how he got into the job of cataloguing Australia's bush tucker. But there are some things I personally don't mind paying for. Maybe that's the kid in me telling me that Les gets some of the money in return for his passions.You can order the box set from the ABC Network directly in Australia here or from Amazon in the UK here.
Enjoy and I hope you manage to find a little space in your imagination and dreams for the adventures of Les Hiddins, the Bush Tucker Man.
Written by Jonathan Maher
The Harrington
© The Revive Club

redpajama_set_name: RedPajamaCommonCrawl


text:
Tearing Strips
1628, Fire Season, Fertility Week
During some downtime in Dunstop, Varanis attempts to tear a strip off Berra for using the wrong name. Session 13 (Family's Always Embarrassing) and shortly after Darn Mending
Later that night, Varanis signals to Berra that she wants a quiet word.
Berra is playing a game of swords and boards against herself, on a game-board etched into one of the tables. She abandons it to drift over to Varanis.
"You called me Varanis earlier," the Vingan murmurs. "We agreed to be a little less obvious, remember?"
Berra makes a sulky face at odds with her next words. "I know, yeah. I'm sorry."
Varanis arches a brow. "Why does your face say something different from your words?"
"Uh, because it's dark in here?" Berra suggests. Maybe she's a bit tipsy. She seems to be more than usually aggressive.
"Oh really?" Berra's belligerence is rapidly being matched by Varanis'.
Berra stares. "Yeah?" she suggests, in a way that says she is happy to take this outside, even.1I am finding myself wondering if Varanis fumbled an insight roll… 2000, baby. And Berra's good at looking sulky, as has been established in other RP.
"Yeah," Varanis replies.3Dice roll to determine mood/response: Air – 12/91 "Should we go outside to talk about it?"
Berra blinks. "Sure," she says. "If you're… yeah, let's." She stalks casually towards the door.
Varanis glares. "Out back. We don't need to put on a performance for every passer-by."
Berra glances around, rolls out her shoulders comfortably, heads that way.
She turns on her heel to stalk out the back door of the inn, brushing past the cook on the way by.
Berra ends up trailing behind, giving Varanis plenty of room.
Outside, the air is cool, with a hint of rain. The stable is behind the inn's main building and a single torch lights the small open space between.
Berra checks out the area, even glancing at the roof. "I messed up. And you're under a lot of stress." There is a patronising tone in her voice.
Varanis wheels and with a single step, puts herself almost toe to toe with the smaller woman. "Then why, by Orlanth's blue balls, are you trying to start something with me?" she growls.
"You're a fine one to talk," Berra says with something like scorn. "You're the one who asked me out here!"
"You may not have said the words aloud, but the challenge was there. Am I suppose to ignore that?"
Staring up at Varanis, Berra puts her hands on her hips. "Yeah? Are you saying I wouldn't tell you the truth?"4Success on Air.
"You rather conveniently didn't say anything, did you? You just stared it at me while you said you were sorry. I'm sure you are sorry – you wouldn't say it otherwise. But you are also mad about it." The intensity of her anger falls rapidly, as Varanis speaks, as though she's beginning to listen to herself. "Ah fuck. If you said you were sorry, you meant that. And that means that whatever I read in your face wasn't about you challenging me." She steps back, giving Berra room and runs a shaky hand through her mess of plaits. "Fuck," she says again. "Berra, I'm sorry. I don't know what came over me."5Rolled on INT*4 after Berra's statement and passed that.
The light from the torch falls on Berra's face – it shows worry and concern, and fading anger, gone almost as soon as it can be seen. "Kay. Wanna start again?" she asks in a small voice, careful.
Varanis draws a shuddering breath. "I need to leave here. I can't stand being this close to the two them. And we're heading deeper into Lunar territory and somehow I have get get everyone there and then back to Boldhome alive. We're hunting something so evil that even some of our worst enemies fear it. And we're doing it with targets on our backs. And I have to trust her and I don't and I just want to get away from her before I do something stupid like demand to talk to her again." The words spew forth, lacking the clarity and organization of one of Varanis' speeches. Her emotions have her twisted up in knots.
"We'll let Irillo know we have to move on tomorrow," Berra says. "In case that's not his plan. But you don't have to trust her. You have to trust her belief in her plans. She don't want you dead – she wants to make peace so she can be rich and get fatter."
"And him?"
"Like he matters to her. He wants to be in charge here – betraying you doesn't make that happen, because the Bat's still a threat, right? You're not enough of a bribe to make them acknowledge him if they can just send it this way and be done with it."
"Can you call me Ranie?" Varanis asks, returning to the start of things and sidestepping her other concerns.
Berra takes a deep breath. "I don't know. I …" She trails off. "It feels wrong to me. Maybe less so since I heard Mirava say it, but it's not what I know you as."
"It's still my name. Have you never heard Xenofos use it?"
Berra adds, miserably, "And I keep forgetting. I've tried not to call you anything, but it's really tough and I'll try to call you Ranie but I don't know if I'll always get it right. I just forget."
Varanis runs a hand through her hair again, this time catching on the plaits and making a fist in it all. She takes slow breaths and as she releases it a third time, her hand slides free and her body visibly relaxes. "Maybe I shouldn't have asked it of you," she admits. "It's likely the songs won't have made it too deeply into Tarsh and the other provinces. Our names aren't exactly uncommon either. We'll be fine." She sounds as though she is trying to convince herself.
Berra looks down, examining the ground, her hands, her sandaled feet. "There's really only one person who cares," she says, "And the rest, Irillo will give gifts to." That is, bribe in ways that will not bring unwanted attention. Small bribes, of the sort he hands out as 'taxes'.
Wryly, Varanis murmurs, "I'm going to end up owing him my first-b…" She stops. "Gah. That's not funny anymore."
"It's fine. I'll fight him for her. He has no idea how to keep a weapon inline and I don't need money." Berra tries a smile. "Anyhow, you're not the sponsor of the caravan here." She glances around, but even though nobody seems to be close, says no names.
"You think Irillo would do this for her?"
"Er, he's been complaining about the cost all the way here. He's going to get most of it back, and the rest is for Harmony." Berra shrugs. "He'll owe us all, we'll owe him. That's how people come together."
"Let's discuss the issue of names on the road with the others," Varanis says at last. "Whatever we decide, we need to be consistent."
"We can just keep me and you at different ends of the caravan," Berra says. "That'll help. I'm sorry. It's me. It's not even about lying, or at least Truth, I think. Just something in me knows you as I knew you when I met you, and keeps using it… you got a rank? Like, 'soldier'?"
Varanis snorts. "And yet you can call me Vareena when it suits." Then she shakes her head. "It's fine. We'll be fine, somehow. I'm sorry I tried to take my anger out on you. Something in me was certain you were trying to provoke me."
"If I've called you that, it's not often," Berra says, and sighs. "And it's because my Lord does. I pick up bits. Odd bits from odd people." With two fingers she traces the angles or a rather wobbly Water Rune in the air. "But yeah. Let's make sure we get away as soon as possible."
"Before I do something stupid," Varanis agrees, staring into the dark above them. Rooftops or palace, doesn't really matter. Both choices would be bad ones. For an awkward moment, it looks like the Esrolian woman is contemplating hugging her friend, but if that's the case, she changed her mind and the moment passes. "Let's go back inside. I should try to rest before my guard shift."
Berra says, "You can take it off, if you want, and sleep outside his door." Then she face-palms. "Sorry. You're not Humakti. Maybe you can't. I don't know."
"I need to greet Yelm anyway. But, if we have enough people, then yes. I'll sleep near my ward and be up in time for Yelm. Maybe if I poke him awake when I rise, we can get on the road sooner."
"I'm not saying I don't sometimes get up loudly," Berra replies, but she is already turning to go, with some destination or other on her mind.
I am finding myself wondering if Varanis fumbled an insight roll…
000, baby. And Berra's good at looking sulky, as has been established in other RP.
Dice roll to determine mood/response: Air – 12/91
Success on Air.
Rolled on INT*4 after Berra's statement and passed that.
Posted in Berra Logs, Varanis Logs
Tagged with 1628, Berra, Dunstop, Fire, Varanis
Previous post: Darn Mending Next post: Climbing the Walls


text:
Editorial: Graduation advice from our crusty newsroom
Opinion Opinion | May 24, 2013
Will Cook
It's graduation season, that time of year when our valley's seniors embark on a new life, whether it's by attending college, getting a job, traveling, you name it.
Graduates also will be hearing a lot of advice over the next few months, so the news staff at The Aspen Times has chimed in with its own.
In no particular order, the following are some suggestions from Times staffers on how to tackle life in the post-high school world.
• "Pursue a major and career that trips your trigger. Don't pursue a path just because someone else thinks it would be best for you. You've got the rest of your life to collect bad advice." — Scott Condon, class of 1980, Osage (Iowa) High School
• "Don't go to college. Believe it or not, there are more enjoyable, rewarding and even educational things you can do, and even if you live on the French Riviera for four years, it will cost less than four years of a private university. And think of all the people who became successful without attending college: LeBron James, Taylor Swift, Moses, George W. Bush." — Stewart Oksenhorn, class of 1981, Livingston (N.J.) High School
• "Remember to have fun, and I don't just mean by partying: Travel, find a new hobby, learn how to cook. Life's about to get a whole lot busier than it was even in high school, and it's important that you make time for yourself." — Jill Beathard, class of 2007, Argyle (Texas) High School
• "Don't go into journalism." — Janet Urquhart, class of 1978, Muskego (Wis.) High School
• "Think hard and long before committing to four years of college. College does not mean a guaranteed future and might just be a precursor to a series of mediocre jobs. Go to work doing something you like, mooch off your family for a while, and save money. Consider starting a business in an area that sparks your passion, and be your own boss. The last thing you want to do is spend your life working for the man. Stay true to yourself." — Andre Salvail, class of 1983, Loyola College Prep, Shreveport, La.
• "Don't let school get in the way of your education." — Dan Thomas, class of 1992, Blacksburg (Virginia) High School
• "Don't forget your phone charger," and "The times of greatest change are the times of greatest opportunity." (advice by way of Emerson) — Dale Strode, class of 1969, Glenwood Springs High School
• "Don't let success go to your head or failure go to your heart." (The only meaningful thing I've ever read on Facebook.) — Rick Carroll, class of 1987, Captain Shreve High School, Shreveport, La.
• "Travel as often as you can and to as many places as you can. The things you will learn in the real world far outweigh — and are far more fun than — things you will learn in the classroom or sitting at a desk." — Jeanne McGovern, class of 1986, Los Amigos High School, Fountain Valley, Calif.
• "If you're not a lottery winner or a trust-funder, your high-minded liberal idealism will almost guarantee that you won't get that house on Red Mountain you've had your eye on for the past four years." — Evan Gibbard, class of 1997, Basalt High School
Pine Creek Cookhouse
Sous Chef & Line Cooks HOUSING AVAILABLE
Seeking experienced sous chef and line cooks. HOUSING AVAILABLE FOR IMMEDIATE MOVE IN. COMPETITIVE PAY IN A STUNNING ENVIROMENT.
Lineworker
Full-time position available for Apprentice Lineworker to Journey Lineworker. Responsible for the construction, repair, & maintenance of Holy Cross Energy's…


text:
The Basics of a Fire Sprinkler System: A Complete Guide
Did you know? Fire sprinkler systems helped control 97% of home fires, according to the NFPA (National Fire Protection Association).
Hence, installing a fire sprinkler system keeps your family safe from home fires. It immediately activates after a kitchen fire occurs and makes fire damage cleanup easier.
But how exactly does a fire sprinkler system work? This post outlines the basics of a fire sprinkler system.
The Basics of a Fire Sprinkler System
How does a fire sprinkler system work for your home? Here are the basics of a fire sprinkler system:
1.The Water Source
First of all, you need to understand where the water comes from. A sprinkler system begins with the water supply and ends at the sprinkler head.
The water source can be a water tank, a municipal water system, or a lake. The local government usually determines what type of water supply your house or building needs.
For instance, a fire sprinkler system for a cabin near the lake can use the water from the lake. On the other hand, an apartment building in a city center can use the city's water system.
2.The Pipes
The next essential component of a sprinkler system is the piping from the water supply to the riser.
Installers put the pipes underground. However, they might put them above the ground at times. Either way, the piping should be restrained. It's because a sudden force of water can rip it apart.
Installers usually reinforce bends using a mechanical joint restraint, concrete thrust blocks, tie rods, or a combination of the three.
If the piping's done underground, experts factor in corrosion. However, sacrificial anode bags can delay the corrosion process. In addition, installers incorporate a backflow system into the piping to prevent contaminating the primary water source.
3.The Sprinkler Riser
The sprinkler riser refers to a vertical pipe. It connects to and brings the water from the piping to the overhead. The riser involves several parts, including a check valve, control valve, test drain, water flow switch, and many pressure gauges.
The control valve turns off the sprinkler system once the fire has been extinguished. On the other hand, the check valve prevents water from returning to the underground system.
The water flow switch signals the fire alarm system when water moves through the system. Meanwhile, the test drain measures the pressure and flow from the system without actually setting off any heads. Lastly, pressure gauges show the working pressure of the sprinkler system.
4.The Sprinkler Head
The head releases water when the system detects fire. Each sprinkler head features a trigger mechanism, which opens to release water.
Home fire sprinkler systems usually have conventional heads. They release water onto the ceiling. On the other hand, pendant or upright sprinklers release water straight down. They work well for places with high ceilings.
Additionally, every sprinkler head activates independently. Only sprinkler systems above the fire will maximize water pressure over the fire to reduce it.
Types of Fire Sprinkler Systems
Fire sprinkler systems, just like fire extinguishers, come in many types. Here are the main types of fire sprinkler systems:
1.Wet Pipe Fire Sprinkler System
Wet pipes are the most common type of fire sprinkler. In this system, water constantly fills the pipes.
The fusible link or glass bulb will break when the temperature gets hot enough. Since the system already has water, water quickly flows out of the sprinkler head.
Contrary to popular belief, not all sprinkler heads will release water simultaneously. Only those that detect heat will release water.
Additionally, wet pipe sprinkler systems are the most cost-effective and reliable. For these reasons, many homes and commercial buildings use them.
However, some places may not need wet systems. If the environment is too cold, the water inside the sprinkler will freeze. So, cold areas shouldn't use wet pipe fire sprinklers.
2.Dry Pipe Fire Sprinkler System
Dry pipe systems work like wet pipe systems. The only difference is that water doesn't fill the pipes.
Instead, it stores water behind a dry pipe valve away from the sprinklers. Like a wet pipe system, the dry pipe's glass bulb or fusible link breaks when the temperature increases.
But in this case, water won't immediately come out because the pipes don't have it. Instead, air comes out of the sprinkler head. This decreases the pressure, causing the dry pipe valve to open and water to fill the system.
Then, water comes out of the sprinkler head. Since water won't come out immediately, dry pipe fire sprinklers should have a specific size, which shortens the time for water to come out.
You'll find dry pipe sprinkler systems in cold places where the water inside pipes might freeze.
3.Pre-Action Fire Sprinkler System
Something should happen first for a pre-action fire sprinkler system to work. The trigger depends on the system: non-interlock, single interlock, or double interlock.
A non-interlock system involves detection devices or automatic sprinklers. The system will only work if detection devices or automatic sprinklers activate.
Meanwhile, a single interlock system relies on detection devices. It means they won't work if the devices don't detect heat or smoke.
On the other hand, a double interlock system involves both detection devices and automatic sprinklers. It means they'll only work if detection devices and automatic sprinklers activate.
Like dry pipe systems, pre-action pipes don't hold water. Instead, they contain compressed nitrogen or regular air.
4.Deluge Fire Sprinkler System
A deluge fire sprinkler system features open sprinkler heads and unpressurized dry piping. The system directly connects to a water supply. Additionally, a deluge valve releases the water to all open sprinkler heads when activated by heat or smoke.
Like a pre-action fire sprinkler system, deluge pipes don't hold water. The deluge valve will keep water from filling the system until it detects smoke or heat.
Once the detection system gets activated, water fills the pipes and flows to the open sprinkler heads. While deluge fire sprinkler systems don't suit homes, they work best in high-hazard places such as aircraft hangers.
How Much Does a Fire Sprinkler System Cost?
Residential fire sprinklers usually cost $1 per square foot. Commercial fire sprinkler systems can go up to $2 per square foot. Hence, sprinklers for a three-bedroom house cost about $2,000.
Plus, fire sprinkler systems don't require a lot of maintenance. They only need to be inspected twice a year.
Sprinklers don't just save lives; they also save you money in the long run. Fire sprinkler systems control or put out 97% of fires.
They're great investments that will last for many years. Sprinklers also keep your family safe and at ease.
What are the 4 required components of a sprinkler system?
The 4 required components of a sprinkler system are the water source, pipe, riser, and sprinkler head.
How does a fire sprinkler system work?
A fire sprinkler works by detecting heat or smoke. When triggered, the glass bulb or fusible link will break. Water will then come out of the sprinkler heads, reducing the fire.
What are 4 types of sprinkler systems?
The 4 types of fire sprinkler systems are wet, dry, pre-action, and deluge. Wet sprinkler systems contain water in the pipes, while dry systems store water at a location away from the pipes. On the other hand, pre-action sprinkler systems need to be triggered by specific actions. In contrast, deluge systems release water from all sprinkler heads.
What are 3 design considerations for an automatic fire sprinkler system?
The 3 design considerations for an automatic fire sprinkler system are:
The water supply.
The kind of fire sprinkler system.
The hazard protection the building needs.
What is the standard pressure for fire sprinkler?
The standard pressure for a fire sprinkler is less than 175 psi (12.1 bar) for those installed aboveground and 150 psi (10.4 bar) underground.
What triggers a fire sprinkler to go off?
Heat or smoke triggers a fire sprinkler to go off. A sprinkler gets activated when the temperature rises to fire-induced levels (usually between 155 to 165 degrees Fahrenheit) or when it detects enough smoke.
What are NFPA 13 sprinklers?
An NFPA 13R sprinkler is a residential sprinkler designed for low-rise homes. The standard produces a sprinkler system that controls home fires and protects against injury and death in buildings with multiple families.
What is the red thing in a fire sprinkler?
Most sprinkler heads have a small bulb with red liquid inside. The bulb prevents water from accidentally coming out of the sprinkler. The heat from a fire expands the liquid. The bulb will burst and release water when the pressure becomes too high.
Knowing the basics of a fire sprinkler system helps you choose the right one for your home. Along with a kitchen fire suppression system, fire sprinklers keep you and your family safe.
But it's always better to be safe than sorry. Although it is impossible to foresee where or when a fire will start, you can take steps to get ready if it does.
Assembling a fire kit with an emergency fire blanket, fire spray, fire protection gloves, and smoke mask helps. Go to Prepared Hero and shop for fire safety tools now!
How to Fireproof Your Home the Right Way
Whether you're building a new home, renovating your house, or simply want to fireproof your home, we're here to help....
Top 20 Fire Hazards at Home
According to the National Fire Protection Association (NFPA), a home fire occurs every 93 seconds in the US. On top...
How to Build the Best Family Bug Out Bag
A complete family bug out bag helps you weather challenges during emergencies like blizzards or fires that will put you...
When Is Tornado Season in the US? How to Keep Your Family Safe
According to the National Oceanic and Atmospheric Administration (NOAA), about 1,000 tornadoes occur across the United States annually. These tornadoes...
EXPLORE THE BEST IN SAFETY
Shop by best selling products
Hero Fire Spray
Just aim & spray
Stops fire instantly
HOUSEHOLD ESSENTIAL
Emergency Fire Blanket
No messy cleanup
Reusable & never expires!
Hero Survival Bag
Prevent hypothermia
Lightweight & portable
View Best Sellers


text:
Retiring head of SF police union leaves huge legacy
On his last day, the POA staff tried to keep it light. Delagnes walked to the Hall of Justice to be processed out.
"Thirty five years broken down to turning in your gun and your badge and saying goodbye," he said.
Delagnes is leaving behind quite a legacy. When he entered the POA as vice president in 1992, San Francisco was the 92nd best compensated police department in California.
Now, San Francisco officers are among the best paid in the country. Starting pay is $82,000 and police here retire at 90 percent of their salary.
And there's a growing respect for cops in the city. "Our last poll showed a 72 percent positive rating for police officers and that's the highest I've ever seen for any police department in the country, so we're doing something right," Delagnes said.
As POA president, Delagnes was like a bulldog -- tough and unrelenting when it came to protecting his family of cops.
"I say things that shock people but I think the reputation I built as a person who will tell you the truth and not sugar coat everything," he said.
Over the years, mayors and city officials asked privately for Delagnes's advice and support. But Delagnes also had his critics.
Public defender Jeff Adachi fought him over pension reform and police corruption cases. "When he said something, it was going to be big time wrestling. He was just going to throw himself at you and everything he had."
But even adversaries like Adachi admit Delagnes did his job well. "I doubt very much that we'll see another one like him in the years to come," he said.
In the hallway on his way out, Delagnes passed a picture of his recruit class taken in June 1978. "That will be the proudest moment of my life -- being a San Francisco police officer," he said.


text:
package com.xqbase.java.tree;

/**
 * A tree position interface for the tree nodes.
 *
 * @author Tony he
 */
public interface Position<E> {

    /**
     * Return the object stored at this position.
     */
    E element();
}


Sherlock Holmes and the Case of the Jersey Lily is an American play, written by Katie Forgette and directed by William Loudermilk, performed by the Edison Community College Stagelight Players at the Edison Theatre (Piqua, OH, USA) from 28 october to 6 november 2011, starring Darin Keim as Sherlock Holmes and Ernie Lawson as Dr. Watson.
© Photos : Edison Community College.
Mrs. McGlynn / Irma Tory : ?
This page was last edited on 2 November 2018, at 15:27.


Neate mused on Saturday that Mustangs QB Michael Faulds is probably the OUA's nominee for the Hec Crighton Trophy. Then he wondered whether Faulds or his Queen's counterpart Dan Brannagan actually deserved the nomination. Then he concluded you should double-check the numbers due to all the variables in the competition each player faced.
First things first, let's not have an apples-to-paleontology comparison. Let's throw out Queen's vs. Waterloo and Western vs. Toronto and Queen's vs. Western,* so that we have common opponents for both quarterbacks. Then let's ignore York because ... well, it's York. Anyone could pass for 400 yards.
* You might be wondering why we're ignoring their Oct. 17 head-to-head matchup; remember, we want to compare them against the same opponents. Faulds played against Queen's; Brannagan never did.
We see an early advantage on yardage for Faulds. (Even rushing yards, too: 178 vs. 29.) And even with the extra attempts, he's quite a bit ahead of Brannagan in adjusted net yards per attempt (ANY/A, the basis for these rankings), with 8.9 vs. 7.4.
In other words, even after we take away nearly two games from Faulds, he still ends up with 182 more yards and two more TDs. And the ANY/A advantage grows: 9.8 for Faulds, only 7.6 for the Queen'sman.
For my money, the debate ends there.
If you don't want to punish Brannagan for missing some time, it doesn't matter: we gave both players the same playing time against the same opponents and Faulds still came out on top. Granted, Faulds' receivers and offensive line and so on could have helped him out more than Brannagan's did, but without much playing time from other Western QBs, it's tough to figure that out.
Although both players had pretty good seasons, as best as I can tell for now, one QB deserves the nomination. And it's not the one wearing yellow.
I should point out that this works just as well for any same-conference quarterbacks (or any two players at the same position), in case someone has a similar question about two other players somewhere else in the country. Faulds/Brannagan just came up because of the passing records.
The sad part for Brannagan is that he got hurt in the McMaster game near the end of the first half on a play that was unnecessary. He rolled right and took off and having a chance to go out of bounds instead drove himself directly headfirst into the Mac defender. Larry Moko of the Hamilton Spectator has an excellent article that followed the game. Saw Brannagan from several feet away as he went off to the dressing room and he looked fairly good and I believe Queen's held him back for that game and others more as precaution for future post season.
The discussion of who is more deserving of Hec Creighton totally academic in nature. Neither likely to play pro ball after graduation and both probably more interested in the "final dance" for Yates and Laval/Montreal two headed monster subsequently just to qualify for Vanier.
Ultimately it also brings to the front of whether the OUA should have an "A" and "B" conference with top 5 from previous year playing against each other and top 4 qualifying for playoffs. The top two of the B conference would then qualify for post season play and the 5th seed of "A" slipping down to the "B" conference the following year.
That's a really nice piece, good work.
I understand why you discounted the Queen's vs Western game but I think that is an important game to consider if one of these two individuals is to be the OUA nominee. Head to head, Brannagan beat Faulds.
I concur with the last comment. I think Fitzbiggon has had an amazing year on a team that has shown hints of being a very good football team but for the most part has been medicore.
How about considering a player like Gary Ross at Mt A? He is playing on a terrible team and they have had to play SMU 2x, X 2x and Laval and this guy has played great all season long? I could only imagine what he could have done had he played in the "Stat Padding Division OUA" playing even worse teams the Mt A?
Quick, name the last player who won playing on an 0-8 team. Ross is a fabulous player, if they had a "long list" of 10, he'd be on it, but his best chance was in '07 and '08 when Mount A was competitive.
Sorry, if Mount A or Acadia took Toronto, York, or Windsor's place in the OUA schedule, they're not in the playoffs.
Make me wonder why last years HEC winner is not the leading CIS passer of all time given he has played for the best team in Canada for the last 5 years? Could it be that he could not build his statistics up playing against watered down teams in the OUA?
Fitzgibbon got almost half his rushing yards (352 of 736) and receiving yards (185 of 400) vs. guess who, York and Toronto. He's a good player, but that's a little hard to ignore.
Riva was spotted vs. those teams (14 of 606 rushing yards, 94 of 401 yards). He did his damage in the other 6 games, though granted, he did have more than 200 total yards and 5 TDs vs. Windsor, which Guelph did not play.
That being said, Fitzgibbon is a superb player and the exact kind who would benefit if we did pick 10 semi-finalists from across the country. The current system pretty much ensures the recipient will be a QB or running back from a team that was near the top of its conference. Guelph is 3-5, so that tears it. However, if it was thrown more wide open, his school could highlight him from the start of the season and he'd have a better shot.
Realistically, for the OUA nom it's Faulds.
@ Anon. 11:08: You need to stay healthy to have a shot at a career mark. Benoit Groulx missed more than half the regular season in 2007. That hurt his chances. He has a chance at a third Vanier Cup and he won a Hec, so cry not for him!
Suggesting Faulds only did this because of the watered-down OUA is pretty ignorant. The old record was set just last season by a Quebec conference player, Matt Connell from McGill!
Please keep in mind, "most yards" does not equal "best now," let alone "best ever." It's just a record. Chris Flynn is probably the best CIS quarterback of all time.
I'm new to all this but running up stats on a team out of the playoffs is I think different than playing them week 1 to 4 when there is hope of a playoff birth. Just seems unfair that one QB who missed 1.5 games but beat the other guy head-to-head is not being considered because the others stats are slightly better.
If I'm not mistaken, Western played both Toronto and York. Please don't insult Waterloo by saying they are close to a Toronto. Waterloo is a far better program than Toronto this season.
But who said this stuff is always fair or always right.
I understand the concept of a head-to-head matchup, but does everyone realize that this only works as a narrative device for the game story and not as a basis for comparison? Maybe if we're talking about Faulds the running back and Brannagan the linebacker, sure.
@Anon 4:50 - The only non-playoff team I considered here, the Lancers, were 2-0 (over Toronto and York, ahem) when they played Western and 2-2 when they played Queen's. If losing to Waterloo takes you out of the playoff race, then that's a pretty sad playoff race.
Let's throw out Queen's vs. Waterloo and Western vs. Toronto and Queen's vs. Western, so that we have common opponents for both quarterbacks. Then let's ignore York because ... well, it's York.
Please don't insult me by commenting on my article without actually reading it.
keep in mind as 8 time Stanley Cup winner Serge Savard said, "statistics are for losers". Not sold on Brannagan but he did win the big one.
We now how far stats got the Gaels the last few yrs. see my post in the roundup. As for Flynn he didn't win a Vanier but he came close and was dominant.
Waterloo didn't have No. 1 pass defence. Ottawa and Laurier were each more efficient.
Chris Flynn should have won 4 HEC's as his best season was his first but back then you could not be a rookie and win it, Flynn is by far the best QB to ever play in the CIS!
A head-to-head matchup may not be the basis for comparison, however, not only did Brannagan win that game, he also out threw Faulds by over 100 yards in that game. Also, Fitzgibbon, or Ross, or anybody on a poor team should not be considered for an award titled "most VALUABLE player", a player holds much less value when he can't lead his team to any, or many, wins.


Alta Equipment Company is seeking a full-time rental shop diesel service technician / mechanic with the classiciation of Rental Shop Technician B for our Detroit, MI facility.This is a union position and will be a part of Operating Engineers Local 324.
A Rental Shop Technician B is qualified to successfully, efficiently, and accurately perform rental check-in's, drain, refill, and change filters of various fluid compartments, to inspect and lubricate various equipment as required by their assigned service department, as well as perform general technician duties. A Rental Shop Technician B will service aerial platforms, compact equipment, small utility rental equipment (light towers, tools, etc.), and the complete line of rental equipment offered by the Company.


Explicitly import fromPromise operator e.g.
Wow, nice shot. :) Exactly what I needed!
Notwithstanding, if so taught, your business technology mentor will settle on choices for your sake dependent on strong experience and comprehension of your destinations.
The three best colleges in Singapore incorporate the National University of Singapore (NUS),HOSPITALITY MUT Nanyang Technological University, and Singapore Management University.
Numerous organizations list a technology as one of their upper hands. Is this substantial? At times indeed, however In many cases no.
Nice blog, your work is great and I hope for some more nice posts. Continue writing such a nice blog. pkr jain vatika It's hard to discover fantastic composition like yours nowadays. And also here I got new ideas and guidelines. Thank you so much for your great sharing.


text:
Home Perspectives Should 13RW ended in season one?
Should 13RW ended in season one?
By Aloysius Lee -
Photo Courtesy of Flora Carr. When is 13 Reasons Why Season 3 released on Netflix? Cast, trailer and spoilers
A popular Netflix series named 13 Reasons Why has come out with a new season. 13 Reasons Why is one of my all-time favorite Netflix series since I started watching Netflix. The series is a teen drama television series based on the novel Thirteen Reasons Why by Jay Asher. It talks about how high-school teenagers deal with high-school drama, gossip, bullying, and sexual assaults. Starring Dylan Minnette as Clay Jensen and Katherine Langford as Hannah Baker, the series follows high school students who live in the aftermath of their classmate's death.
In the first season, the story begins with Clay receiving a set of tapes from Hannah two weeks after she took her own life. In the tapes, Hannah unfolds all thirteen reasons why she took her own life. Everyone who was involved in leading Hannah to take her own life was on the tapes and was requested to listen to them and pass them on to the next person. No doubt, it was a huge success for the series as it became a big hit on Netflix.
Soon, the second season was released and it went on unfolding even more secrets between the secrets. Hannah's parents went to a civil legal battle with the high school Hannah was in, which was called Liberty High. Everyone who was mention in the tapes – both Hannah's friends and family members – was involved in the case. Everyone who was present in court unfolded some secrets that were not told in the first season; the series goes on and ends up with Hannah's parents breaking up and Tyler, one of the characters, attempting to shoot up the school during Spring Fling.
The latest season picked up with how Clay and friends tried to help Tyler get better after he attempted to shoot up the school while finding out that one of Liberty High's former students, Bryce Walker, was murdered. Everyone was once again under savior pressure and tension because the investigation of Bryce's death will expose some of their darkest secrets. The series then took a turn from spreading awareness of bullying and sexual assaults in season one and two to a murder mystery that a group of teenagers are trying to solve. It's a similar kind of storyline that we see in Riverdale.
So, some series should have ended with the very first season and I believe 13 Reasons Why is one of them. It has kind of lost the message the series was trying to send, and it is just not right. Do not get me wrong, I absolutely love the first season, I think it was a very good series spreading positive awareness of the struggle's teenagers are facing in high school nowadays. Encouraging the people who need or who are struggling to reach out for help. I love the first season, but I believe the first season is where it should have ended.
Previous articleNFL Week 6 Games to Watch and Score Predictions
Next articleAre We In Control?
Aloysius Lee


You all know how much I love cowgirl boots. That is one of the reasons I love getting to partner with Durango. They have some of the most beautiful boots that are beyond comfortable. Growing up in the Central Valley I have worn my fair share of boots and always love getting to wear my Durango's. During the summer months I normally would pair my boots with a dress or jeans. However the day I shot these photos, there was a breeze so leggings seemed to be appropriate. I honestly love the look it gave this whole outfit.
I have had the shirt in today's post for a while now. You may have remembered the maroon one from this post. Well I liked that shirt so much, I bought it in this color too. It is flowy and yet so flattering at the same time. I am a huge fan of Knox Rose from Target. That line has a variety of country approved clothing. Whether it's a fringe cover up or a lose fitting blouse, it is so stylish and affordable. I knew this top would be the perfect addition to a pair of cowgirl boots. I wanted to show off my boots, so I threw on my favorite black leggings. This outfit was so easy to put together. It looks great and is so classy. Now we can't have a country outfit without a glass of my moms sweet tea. Side note: I'm absolutely addicted to her sweet tea. I have had to cut back on it because it isn't the healthiest, but man it is good. I grew up on it. My mom poured me a glass in one of her mason jars and we were set to go shoot some pictures.
I hope you all have a wonderful Tuesday! God bless!
So you'll notice that the image above isn't retouched, the lighting isn't bright and it's not arranged how I normally would have if I were posting it on Instagram. Well, this is a real life post and that requires a real life photo of my lunch while sitting in a cafe.
Have you ever sat in a cafe and listened to the conversations around you? Now I don't mean this in a creepy way, that isn't my intention or the purpose of this post. However, every now and then I love to just sit in a cafe, by myself, completely removed from technology and just be still. While in the pursuit of peace and quiet, I always tend to over hear conversations around me. Some of them are light hearted, girlfriends catching up, talking about their kids. Some are more burdening on the heart, a friend opening up to her other friend about how hard her divorce has been. No matter the place, time of day or type of cuisine, I always hear some pretty interesting conversations. Sitting in a cafe can be very eye opening and humbling.
Why is Michelle writing a blog post all about other people's business? I'm not, the purpose of this post is to help you stop and realize just how much people are truly dealing with. We get so caught up in our Instagram feed, our Facebook friends and we forget about those people who are sitting in front of us and around us. I am a firm believer that there is always someone out there less fortunate or in a tougher situation than you. In the moment I forget that things could always be worse. Then I walk into a cafe to spend time with my own thoughts and sure enough, I hear someone talking about something that would absolutely break me if I was having to deal with it.
I always love how God knows exactly what I need to hear, or overhear. These last two weeks have been brutal to say the least. I guess my vacation to the East coast was the first real vacation I took where I had a ton of responsibilities to come home to. I guess it was my first "adult," vacation. I have been in a zone, no lie. I haven't been as patient, as graceful and as kind as I normally am. Although subconsciously I know i'm not being the person I want to be, it is ridiculously hard to pull yourself out of it. When you are in the midst of the grind, you tend to only focus on that.
I hope you all have a wonderful day and remember to start small, maybe today it is 5 minutes, maybe tomorrow it is 10. Take the time to notice people in real life and not just the ones filling up your timeline on Facebook.