text:
\section{\label{introduction} Introduction}

In the transonic speed regime, a strong shock wave is formed over the upper surface of the aircraft wing, causing a sharp increase in wave drag and aerodynamic instability. Therefore, the aerodynamic performance of airfoil should be evaluated in the extensive range of operating conditions, which requires significant computational resources. A practical approach to reduce the computational burden is to use a regression model as a cost-efficient alternative to high-fidelity flow analysis \cite{yang2022design}. Numerous regression models have been developed, such as cubic spline interpolation \cite{mckinley1998cubic}, the radial basis function \cite{gutmann2001radial}, and Gaussian processes \cite{rasmussen2003gaussian, schulz2018tutorial} (or Kriging \cite{jeong2005efficient}); however, these models are only suitable for estimating scalar quantities such as lift coefficients. Consequently, if the quantities of interest are high-dimensional vectors, such as pressure or the velocity field, the computational complexity becomes prohibitive owing to ``the curse of dimensionality.''

Data-driven reduced-order modeling (ROM) has recently been recognized as an effective solution to this problem. ROM treats a high-fidelity computer simulation as a ``black-box function'' and creates simplified models in a data-driven manner without any modification of the governing equation. The main aim of this approach is to reduce the degrees of freedom of high-dimensional data using the dimensionality reduction (DR) technique (also known as representation learning or manifold learning), which finds the low-dimensional latent representation of the data. Through this technique, the dimensionality of the data can be significantly reduced to a level that is suitable for training the regression models effectively.

One of the most widely used DR techniques is proper orthogonal decomposition (POD) \cite{sirovich1987turbulence}, which is also referred to as principal component analysis or the Karhunen-Lo\`eve theorem. Given the training data, the POD method extracts a set of orthogonal bases (typically referred to as the mode), which minimizes the projection error. Subsequently, by truncating the non-dominant POD modes, the original high-dimensional data can be represented in a low-dimensional subspace. However, the data are represented only by the linear combination of POD modes; therefore, POD-based ROM suffers from performance degradation in nonlinear problems \cite{lucia2002domain}.

Several approaches have been proposed to overcome this limitation, such as dividing the problem into multiple subproblems \cite{amsallem2012nonlinear, dupuis2018surrogate, kanglocal, lucia2002domain, xiao2019domain}, improving the capacity of the regression model \cite{fossati2013multiparameter, hesthaven2018non}, and performing adaptive sampling \cite{guenot2013adaptive}. Owing to the remarkable performance of deep neural networks (DNNs) in many engineering problems (to name a few, computer vision, natural language processes, and bioinformatics), DNN-based DR has recently attracted significant attention for its potential to improve ROM in highly nonlinear problems.

The most commonly used DNN-based DR technique is the autoencoder (AE), which learns the latent space of high-dimensional data in an unsupervised manner \cite{kramer1991nonlinear}. The nonlinear activation function between its hidden layers enables it to learn a more compact latent representation than the POD technique. These characteristics significantly improve the reconstruction performance of ROM, which has been validated for various nonlinear problems in other studies \cite{milano2002neural, eivazi2020deep, zhang2021machine}. In particular, convolutional neural networks (CNNs), which are neural network architectures that are specialized for processing grid-pattern data, have been introduced to further improve the accuracy of ROM \cite{hasegawa2020machine, wang2021flow, mucke2021reduced, wu2021reduced, maulik2021reduced, gruber2022comparison, kadeethum2022non}. Although AE-based ROM has exhibited promising performance, the major limitation still lies in its physically uninterpretable latent space because AE is a black-box model that only reduces the reconstruction error; interpretable information, such as the orthogonality and energy rank in POD, cannot be obtained through AE.

Recently, the introduction of a variational autoencoder (VAE) and its improved understanding has been providing a clue to this problem. In contrast to AE which only minimizes reconstruction error, VAE regularizes latent space by also minimizing Kullback-Leibler (KL) divergence \cite{kingma2013auto}. The balance between the reconstruction error and KL-divergence can be further adjusted by the hyperparameter $\beta$, which is referred to as $\beta$-VAE \cite{higgins2016beta}. The well-regularized latent space in $\beta$-VAE is known to have two important characteristics. First, the latent space in $\beta$-VAE is trained to be a disentangled representation, where a single latent variable (LV) encodes only one generative factor of the original dataset. This ensures independence between each LV as the POD extracts orthogonal bases. Second, the KL-divergence term tends to penalize the activeness of LVs, owing to strong assumptions about their posterior distribution \cite{burda2015importance, burgess2018understanding}. Therefore, the size of the latent vector required for reconstruction is reduced \cite{eivazi2022towards}. In summary, the KL-divergence in $\beta$-VAE enables its latent space to learn disentangled and compressed features than that of the AE model.

There are a few studies related to $\beta$-VAE in the field of fluid dynamics. Wang et al. \cite{wang2021flow} applied $\beta$-VAE to ROM for transonic flow problems and verified its superior performance to POD. However, their study mainly focused on the implementation of $\beta$-VAE and lacked an analysis of the interpretability of its LVs. Eivazi et al. \cite{eivazi2022towards} proposed a method to quantitatively measure the orthogonality and energy rank  of LVs for turbulent flows. Unfortunately, the relationship between LVs and physical parameters was not sufficiently investigated, because their main focus was the nonlinear mode decomposition of time-dependent flows. Moreover, their energy-ranking method requires separate post-processing of the reconstructed data after model training, and the energy of LVs are indirectly calculated in the output space rather than in the latent space. Accordingly, the approach is less related to KL-divergence which is the main cause of the inactiveness of LVs.

This study proposes a physically interpretable ROM using $\beta$-VAE. The proposed ROM can measure interpretable quantities of LV, such as rank and independence. In particular, the rank of LVs is measured by KL-divergence, which does not require post-processing of the reconstructed data and can be extended to any physical/non-physical problems. This approach is verified both qualitatively and quantitatively using various techniques including Sobol sensitivity analysis and latent traversal plots. Moreover, the physical meaning of the LVs is thoroughly investigated, and it is confirmed that $\beta$-VAE can extract ``physics-aware'' LVs that solely represent the physical parameters from which the training dataset is generated, i.e., Mach number and angle of attack in this study. To the best of our knowledge, this work is the first observation of physics-aware LVs in the field of fluid dynamics, and we believe that they have considerable potential to improve the interpretability of ROM. Finally, the effects of the physics-awareness of LVs on the accuracy of ROM are analyzed in terms of both regression and prediction error, which confirms the potential to reduce the computational cost of ROM by excluding the physics-unaware LVs.

The rest of this paper is organized as follows. In Sec. \ref{sec:ROM}, the DNN-based DR techniques for data-driven ROM are described in detail. In Sec. \ref{sec:num_exp}, the setup of the numerical experiment is presented. In Sec. \ref{sec:result}, the interpretability of $\beta$-VAE and its effect on ROM are thoroughly investigated. And finally, in Sec. \ref{sec:conclustion}, the summary and conclusions of this study are provided.

\section{\label{sec:ROM} Data-driven reduced-order modeling}
A high-fidelity computer simulation can be regarded as an input–output vector function $\mathcal{F}: \mathbb{R}^p \rightarrow \mathbb{R}^d$, where $p$ and $d$ are the dimensions of the input and output vectors, respectively. In general, the dimension of the output vector in a computational fluid dynamics (CFD) simulation is extremely large ($d>10^5$), owing to the dense computational grid required to capture accurate physical phenomena in the simulation. This characteristic prohibitively increases the training time of the surrogate model; therefore, the data-driven ROM approach aims to alleviate this problem by adopting the DR technique.

As shown in Fig. \ref{fig:flowchart_rom}, the overall process of the data-driven ROM can be divided into the offline and online stages. In the offline stage, the training dataset is generated through the design of experiments, high-fidelity CFD simulation, and pre-processing. Then, the DR model is trained in an unsupervised manner, and high-dimensional data are encoded into a low-dimensional latent space. Finally, the regression models are trained to correlate the input parameters with the LVs. In the online stage, LVs are predicted from the trained regression models for an unknown set of input parameters. High-dimensional data are then reconstructed by decoding the predicted LVs. Generally, most of the computational time is spent in the offline stage, because not only the dataset should be collected from a high-fidelity computer model, but also the DR and regression models should be trained in this process. However, the online stage does not require data-generation and model-training time; therefore, high-dimensional data can be efficiently reproduced in this stage.

The present study adopted AE and $\beta$-VAE as DR techniques and Gaussian process regression (GPR) as the regression model. The rest of this section primarily focuses on the description of AE and $\beta$-VAE; for readers interested in the theoretical background of GPR, please refer to references \cite{rasmussen2003gaussian, schulz2018tutorial}.

\begin{figure*}[ht]
\begin{center}
\subfloat[\label{fig:offline}]{\includegraphics[width=6cm]{figures/rom_flowchart_offline.png}}
\quad \quad
\subfloat[\label{fig:online}]{\includegraphics[width=6cm]{figures/rom_flowchart_online.png}}
\caption{Flowchart of data-driven reduced-order modeling: (a) offline stage, (b) online stage.}
\label{fig:flowchart_rom}
\end{center}
\end{figure*}

\subsection{\label{sec:AE} Autoencoder (AE)}
The AE model is a widely used deep-learning-based DR technique. The key to the AE is to output exactly what is inputted. The structure of an AE consists of two parts: an encoder and a decoder. The input of the AE, $\textbf{x}$ (which is distributed independently and identically), is entered into the encoder and exits as the compressed LVs $\textbf{z}$. Then, $\textbf{z}$ enters the decoder and exits as reconstructed data $\tilde{\textbf{x}}$. The encoder and decoder consist of a multi-layer perceptron (MLP), which makes it possible to model nonlinearity in the reduction and reconstruction processes. Because the objective of training the AE model is to reconstruct $\tilde{\textbf{x}}$, which is similar to the original data $\textbf{x}$, the loss function is defined by Eq. \ref{eq:MSE}. Note that although the loss function of the AE is defined by the mean square error (MSE) in this study, any other error metrics, such as the binary cross entropy between $\textbf{x}$ and $\tilde{\textbf{x}}$, can be used depending on the properties of the data.

\begin{equation}\
\label{eq:MSE}
\mathcal{L}_{AE} = \frac{1}{N}\sum\limits_{i=1}^N{(\textbf{x}_i-\tilde{\textbf{x}}_i)^2},
\end{equation}
where $N$ denotes the dataset size.

However, this AE model has a clear limitation: there is no training algorithm that guarantees a regularized latent space. Herein, the expression ``regularized latent space'' means that the latent space is trained to be smooth and continuous; thus, when inputs $\textbf{x}_1$ and $\textbf{x}_2$ are similar (or close), their corresponding mapped latent space, $\textbf{z}_1$ and $\textbf{z}_2$, should also be similar (or close) \cite{ghosh2019variational}. In a similar context, an unregularized latent space indicates that although the inputs are sufficiently similar, their mappings to the latent space may be significantly different. As the original input $\textbf{x}$ becomes condensed layer-by-layer through the encoder of the AE model, its representation becomes increasingly abstract \cite{vincent2008extracting}. Therefore, the output of the encoder $\textbf{z}$ (also the input of the decoder) is not guaranteed to be regularized in the training procedure of the AE. This explains why the AE model cannot be used as a generative model: the regularity between the trained latent space and reconstructed data is abstract and therefore uninterpretable. From a ROM perspective, this aspect is also important. If the smoothness of LVs is not ensured, the correlation between physical parameters and LVs is expected to be nonlinear. Thus, the performance of the regression model may be degraded, which results in a poor predictive accuracy of ROM. Therefore, since there is no technique to assure regularized latent space in the training algorithm of AE, applying this model to ROM can degrade the physical interpretability and accuracy of the ROM technique.

\subsection{\label{sec:VAE} Beta-Variational autoencoder ($\beta$-VAE)}

Many studies have focused on the limitations of this unregularized latent space trained by an AE and have applied VAE in their framework. The structure of the VAE is similar to that of the AE. One major difference is that VAE stochastically extracts the latent space $\textbf{z}$ via random sampling, whereas the AE model obtains its latent space deterministically.

The mathematical formulae for the VAE model are presented below (for further details, please refer to \cite{kingma2013auto}). We consider reducing the original dataset $\textbf{X} = \left\{\textbf{x}_{i} \right\}_{N}^{i=1}$ to the latent space $\textbf{z}$ using the VAE model. In the inference of $\textbf{z}$ from $\textbf{X}$, variational inference is adopted, owing to the intractability of the posterior $p_\mathbf{\theta}(\textbf{z}|\textbf{x})$, where $\mathbf{\theta}$ is a parameter of the VAE model (to be more specific, it is intractable, owing to the likelihood of $p_\mathbf{\theta}(\textbf{z})$). Therefore, instead of the intractable posterior $p_\mathbf{\theta}(\textbf{z}|\textbf{x})$, the VAE is trained to obtain a substitute for the posterior $q_\mathbf{\phi}(\textbf{z}|\textbf{x})$ (here, $\mathbf{\phi}$ is a variational parameter). Then, the log-likelihood of $p_\mathbf{\theta}(\textbf{x})$ can be expressed as (a detailed derivation is provided in \cite{yang2022inverse}):

\begin{equation}\
\label{eq:loglike}
\log(p_\mathbf{\theta}(\textbf{x}))=\int{\log{(\frac{p_\mathbf{\theta}(\textbf{x},\textbf{z})}{q_\mathbf{\phi}(\textbf{z}|\textbf{x})})}q_\mathbf{\phi}(\textbf{z}|\textbf{x})d\textbf{z}}+
\int{\log{(\frac{q_\mathbf{\phi}(\textbf{z}|\textbf{x})}{p_\mathbf{\theta}(\textbf{z}|\textbf{x})})}q_\mathbf{\phi}(\textbf{z}|\textbf{x})d\textbf{z}},
\end{equation}

where the second term on the right-hand side (RHS) is the KL divergence of $q_\mathbf{\phi}(\textbf{z}|\textbf{x})$ from $p_\mathbf{\theta}(\textbf{z}|\textbf{x})$, which is always nonnegative according to its definition (KL-divergence is a measurement of the statistical distance between two probability distributions). Therefore, the first term on the RHS becomes the lower bound of the log-likelihood. Thus, the problem of maximizing the log-likelihood becomes the problem of maximizing the lower bound. This lower bound can be expressed as (again, a detailed derivation is provided in \cite{yang2022inverse}):

\begin{equation}\
\label{eq:ELBO}
\int{\log{(\frac{p_\mathbf{\theta}(\textbf{x},\textbf{z})}{q_\mathbf{\phi}(\textbf{z}|\textbf{x})})}q_\mathbf{\phi}(\textbf{z}|\textbf{x})d\textbf{z}}=
\mathbb{E}_{q_\mathbf{\phi}(\textbf{z}|\textbf{x})}{[\log{p_\mathbf{\theta}(\textbf{x}|\textbf{z})}]}-
\int{\log{(\frac{q_\mathbf{\phi}(\textbf{z}|\textbf{x})}{p_\mathbf{\theta}(\textbf{z})})}q_\mathbf{\phi}(\textbf{z}|\textbf{x})dz},
\end{equation}

where the first and second terms on the RHS are the reconstruction error and KL divergence of $q_\mathbf{\phi}(\textbf{z}|\textbf{x})$ from $p_\mathbf{\theta}(\textbf{z})$ ($KL(q_\mathbf{\phi}(\textbf{z}|\textbf{x})||p_\mathbf{\theta}(\textbf{z}))$), respectively. However, owing to the existence of $q_\mathbf{\phi}(z|x)$ in the reconstruction error, the backpropagation process cannot be performed, and calculating the gradient of the reconstruction error with respect to $\mathbf{\phi}$ is problematic owing to the posterior $q_\mathbf{\phi}(\textbf{z}|\textbf{x})$. Therefore, a ``reparameterization trick'' is adopted, which allows for back-propagation through the sampling process. The concept behind this trick is to sample LV $\textbf{z}$ from the auxiliary noise variable $\epsilon \sim N(0,1^2)$ (this random sampling makes the latent space in the VAE stochastically determined). Then, the $k$th LV ($z_k$) is assumed to follow the distribution below:

\begin{equation}\
\label{eq:latent_code}
z_k = \mu_k + \sigma_k \odot \epsilon.
\end{equation}

Therefore, the KL divergence in Eq. \ref{eq:ELBO} can be re-written as below when the posterior $q_\mathbf{\phi}(\textbf{z}|\textbf{x})$ and prior $p_\mathbf{\theta}(\textbf{z})$ are assumed to follow the Gaussian distribution $N(\mathbf{\mu},\mathbf{\sigma}^2)$ and $N(0,\textbf{I}^2)$, respectively.

\begin{equation}\
\label{eq:KLwithrepar}
KL(q_\mathbf{\phi}(\textbf{z}|\textbf{x})||p_\mathbf{\theta}(\textbf{z})) = \frac{1}{2}\sum\limits_{k=1}^d{(\sigma_k^2+\mu_k^2-(\log(\sigma_k^2)+1))},
\end{equation}

where $\mu_k$ and $\sigma_k$ represent the mean and standard deviation used for the reparameterization of $z_k$, respectively, and $d$ denotes the dimension of the latent space. Herein, as the posterior approximation $q_\mathbf{\phi}(\textbf{z}|\textbf{x})$ approaches the prior $p_\mathbf{\theta}(\textbf{z})$, the KL divergence decreases. Finally, the loss function used to train the VAE model can be rewritten as in Eq. \ref{eq:VAEloss}, which consists of the reconstruction error (MSE term) and regularizer (KL-divergence term). The KL-divergence term in the loss function of VAE is used to regularize the latent space to be trained; therefore, it is also called the regularization loss, which induces a sparser latent space \cite{kingma2013auto, doersch2016tutorial, higgins2016beta, pati2021attribute} (just as the L1 regularization term makes the model sparse in the Lasso regression \cite{tibshirani1996regression}).

\begin{equation}\
\label{eq:VAEloss}
\begin{aligned}
\mathcal{L}_{VAE}
&=\mathcal{L}_{AE} + KL(q_\mathbf{\phi}(\textbf{z}|\textbf{x})||p_\mathbf{\theta}(\textbf{z}))\\
&=\frac{1}{N}\sum\limits_{i=1}^N{(\textbf{x}_i-\tilde{\textbf{x}}_i)^2} + \frac{1}{2}\sum\limits_{k=1}^d{(\sigma_k^2+\mu_k^2-(\log(\sigma_k^2)+1))}.
\end{aligned}
\end{equation}

Higgins et al. \cite{higgins2016beta} focused on the trade-off relationship between the reconstruction error and KL divergence in the loss function of the VAE. As the reconstruction error decreases, the reconstruction accuracy increases, but the degree of regularization (or information capacity) in the latent space decreases. To tune the balance between the two performances, Higgins et al. \cite{higgins2016beta} proposed the $\beta$-VAE, which can control the importance of each loss term (reconstruction and KL-divergence) using the adjustable hyperparameter $\beta$. The $\beta$-VAE is a simple modification of the VAE: they have exactly the same structures but slightly different loss functions. The loss function of the $\beta$-VAE is as follows:

\begin{equation}\
\label{eq:bvae_loss}
\mathcal{L}_{\beta{\text -}VAE} =\frac{1}{N}\sum\limits_{i=1}^N{(\textbf{x}_i-\tilde{\textbf{x}}_i)^2}
+\beta KL(q_\mathbf{\phi}(\textbf{z}|\textbf{x})||p_\mathbf{\theta}(\textbf{z})).
\end{equation}

The only difference in the loss functions between the VAE and $\beta$-VAE is whether the KL divergence term is weighted by hyperparameter $\beta$. By introducing this hyperparameter, Higgins et al. \cite{higgins2016beta} achieved quantitative and qualitative improvements in the disentanglement performance over the traditional VAE model.

\section{\label{sec:num_exp} Numerical experiments}

\subsection{\label{sec:problem_def} Data preparation}

A two-dimensional (2D) transonic flow problem, which is not only practical for evaluating the airfoil performance, but is also a benchmark problems that is widely used in previous ROM studies \cite{kanglocal, dupuis2018surrogate, lucia2002domain, wang2021flow}, is examined in this study. The transonic flow field is generated by the KFLOW finite-volume-based CFD solver \cite{park2004implementation, hong2022enhanced}. Specifically, the Reynolds-averaged Navier-Stokes equation is solved coupled with the Spalart-Allmaras turbulence model. An RAE 2822 airfoil is selected as the baseline geometry, and a structured O-grid with a size of 131,072 ($512\times256$) is generated; the corresponding grid is shown in Fig. \ref{fig:airfoil_grid}. The Reynolds number is fixed at $6.5\times 10^6$, and two physical parameters are chosen: Mach number (Ma) and angle of attack (AoA). The parameter space of each variable is set to [0.5, 0.8] and [0, 3], respectively. The Latin hypercube design is used to generate 500 samples points in 2D parameter space. A flow analysis is then conducted, and the resultant velocity fields and pressure are normalized by far-field conditions. As the variations of flow properties far from the airfoil are negligible, only half of the entire grid near the airfoil surface ($512\times128$) is used to train and test the model. Consequently, a total dataset with a shape of $500 \times 3 \times 512 \times 128$ is obtained (dataset size $\times$ flow field components $\times$ x-direction grids $\times$ y-direction grids), where the flow-field components are the x-velocity, y-velocity, and pressure. Finally, the dataset size of 500 is split into a ratio of 9:1; thus, the size of the training dataset is 450, and that of the test dataset is 50.

\begin{figure}[htb!]
    \centering
    \includegraphics[width=0.7\linewidth]{figures/airfoil_grid.png}
    \caption{Computational grid used for the flow analysis; structured O-grid with a size of $512\times256$.}
    \label{fig:airfoil_grid}
\end{figure}

\subsection{\label{sec:param_set_lr} Training details}

\begin{figure*}[htb!]
	\centering
		\includegraphics[width=1.\textwidth]{figures/CVAE_arch.png}
	\caption{Structures of the AE and VAE/$\beta$-VAE.}
	\label{fig:VAE_arch}
\end{figure*}

\begin{table}[htb!]

\caption{Details of the blocks and layers of VAE/$\beta$-VAE used in this study.}\label{tab:NN_name}

\setlength{\tabcolsep}{0pt}
\begin{tabular*}{\textwidth}{@{\extracolsep{\fill}\quad}lccccccccc}

\hline \hline
\textbf{Name}         & \textbf{Layer type}   & \textbf{Filter} & \textbf{Kernel} & \textbf{Stride} & \textbf{Activation} & \textbf{Batch Norm.} \\ \hline
\multirow{2}{*}{Up}   & Max-polling     & --               & 2 $\times$ 2           & --               &         --            & --                            \\
                      & Convolution     & 64              & 3 $\times$ 3           & 1               & LeakyReLU           & o                            \\\hline
\multirow{2}{*}{Down} & Upsampling      & --               & 2 $\times$ 2           & --               &         --           & --                            \\
                      & Convolution     & 64              & 3 $\times$ 3           & 1               & LeakyReLU           & o                            \\\hline
Conv1\_in             & Convolution     & 64              & 1 $\times$ 1           & 1               & LeakyReLU           & o                            \\\hline
Conv1\_out            & Convolution     & 3               & 1 $\times$ 1           & 1               & --                   & --                            \\\hline
FC                   & Fully Connected & --               & --               & --               & --                   & --                            \\   \hline \hline
\end{tabular*}
\end{table}

\begin{table}[htb!]
\caption{Network structure of the VAE/$\beta$-VAE used in this study.}\label{tab:NN_structure}
\setlength{\tabcolsep}{0pt}
\begin{tabular*}{\textwidth}{@{\extracolsep{\fill}}cccccc}

\hline \hline
\multicolumn{2}{c}{\textbf{Encoder}} & \multicolumn{2}{c}{\textbf{Bottleneck}} & \multicolumn{2}{c}{\textbf{Decoder}}  \\ \hline

\textbf{Layer} & \textbf{Output size} & \textbf{Layer} & \textbf{Output size} & \textbf{Layer} & \textbf{Output size} \\ \hline

Input          & 3 $\times$ 512 $\times$ 128        & Flatten        & 256           & Up1            & 64 $\times$ 8 $\times$ 2           \\
Conv1\_in      & 64 $\times$ 512 $\times$ 128       & FC1: $\mu$    & 16            & Up2            & 64 $\times$ 16 $\times$ 4          \\
Down1          & 64 $\times$ 256 $\times$ 64        & FC2: $\sigma$ & 16            & Up3            & 64 $\times$ 32 $\times$ 8          \\
Down2          & 64 $\times$ 128 $\times$ 32        & Resampling     & 16            & Up4            & 64 $\times$ 64 $\times$ 16         \\
Down3          & 64 $\times$ 64 $\times$ 16         & FC3            & 256           & Up5            & 64 $\times$ 128 $\times$ 32        \\
Down4          & 64 $\times$ 32 $\times$ 8          & Unflatten      & 64 $\times$ 4 $\times$ 1    & Up6            & 64 $\times$ 256 $\times$ 64        \\
Down5          & 64 $\times$ 16 $\times$ 4          & --             & --            & Up7            & 64 $\times$ 512 $\times$ 128       \\
Down6          & 64 $\times$ 8 $\times$ 2           & --             & --            & Conv1\_out     & 3 $\times$ 512 $\times$ 128        \\
Down7          & 64 $\times$ 4 $\times$ 1           & --             & --            & Output         & 3 $\times$ 512 $\times$ 128        \\  \hline \hline

\end{tabular*}
\end{table}

In this study, all the models mentioned in Sec. \ref{sec:ROM}, AE, VAE, and $\beta$-VAE, are trained to investigate their differences in terms of DR for transonic flow. In particular, several $\beta$-VAE models are trained ($\beta\in[10, 20, 30, 40, 50, 100, 150, 200, 500, 750, 1000, 2000, 3000, 4000]$) to investigate the effects of the $\beta$ value (technically speaking, the VAE can be regarded as the $\beta$-VAE when $\beta$ has a value of 1). The dimension of the latent space is set to 16, which is considered sufficient for encoding the training data generated from the 2D parameter domain (Ma and AoA). To verify this dimension selection with respect to the linear-based DR technique, POD is conducted, and it is confirmed that the chosen dimension conserves 99.18$\%$ of the energy.

An appropriate encoder structure should be selected to effectively reduce the input data, which has dimensions of 3$\times$512$\times$128, to the dimension of 16. Similarly, an appropriate decoder structure should be selected to effectively reconstruct a latent space with a dimension of 16 to that of 3$\times$512$\times$128. To determine their suitable structures, hyperparameter tuning based on a grid search was conducted. Finally, it was confirmed that the MSE reconstruction error, which represents the overall accuracy of the trained model, does not strongly depend on whether batch normalization, max-pooling, and up-sampling are applied. However, their application significantly affects the smoothness of the reconstructed flow field. If an artificial discontinuity (rather than a discontinuity that reflects a physical phenomenon, such as a shock wave) is observed in the reconstructed flow field, the flow field cannot be considered realistic. In this context, these points are important for predicting realistic flow fields using CNN-based deep learning models. Finally, the architectures of the selected models, which are considered to be sufficient in terms of MSE error and the smoothness of the reconstructed flow field, are shown in Tables \ref{tab:NN_name} and \ref{tab:NN_structure} and Fig. \ref{fig:VAE_arch} (the only difference between the structure of the AE and VAE/$\beta$-VAE is the bottleneck, and the structures of the VAE and $\beta$-VAE are exactly the same).

The Adam optimizer is adopted to train these selected models. The initial learning rate is set to $10^{-3}$, and it decays at a rate of 0.1 every 1000 epochs. The maximum epoch should be selected carefully because a model that does not fully converge can make significantly different predictions than a converged model \cite{yang2022comment}. For all models (AE, VAE, and $\beta$-VAE), the maximum number of epochs is set to 3000 because additional epochs did not result in a significant change, and this value guaranteed sufficient convergence in terms of the loss function. A mini-batch size of 50 is selected. Therefore, nine iterations (because the size of the training dataset is 450) are performed per epoch. Using a Tesla P100-PCIE-16GB GPU, the average training time for each model is calculated to be approximately 3 h.

\section{\label{sec:result} Result and discussion}

\subsection{\label{sec:reconstruction} Training results}
First, the resultant loss functions of the various models are shown in Fig. \ref{fig:loss_tradeoff}. More specifically, the loss function is decomposed into the MSE and KL divergence (as in Eq. \ref{eq:bvae_loss}) to confirm their trade-off relationship. The MSE and KL divergence of the VAE/$\beta$-VAE models (note that the VAE is a special case of the $\beta$-VAE when $\beta$ equals 1) are indicated by blue and red symbols, respectively. The $\beta$ and KL divergence values cannot be defined in the AE model; therefore, only the MSE loss of the AE model is indicated separately by a dashed line. Herein, a clear trade-off relationship between the two components (MSE and KL divergence) can be confirmed, as mentioned by Higgins et al. \cite{higgins2016beta}. As $\beta$ increases, KL divergence decreases, while MSE increases. This is consistent with the concept of the $\beta$-VAE, which suppresses KL divergence by increasing $\beta$. In particular, when $\beta>1000$, the MSE increases significantly; therefore, an accurate reconstruction of the flow field is no longer possible.

\begin{figure}[htb!]
	\centering
		\includegraphics[width=.7\textwidth]{figures/loss_tradeoff.png}
	\caption{MSE and KL divergence of the trained VAE/$\beta$-VAE models.}
	\label{fig:loss_tradeoff}
\end{figure}

Second, the reconstructed pressure flow fields of the three test cases (which are not used during the training process) are shown in Fig. \ref{fig:recon_compar} to compare the trained models more intuitively and visually. The selected test cases are as follows: test case 1 is in the absence of a shock wave ($Ma=0.61$ and $AoA=2.12^{\circ}$), test case 2 is in the presence of a weak shock wave ($Ma=0.72$ and $AoA=1.68^{\circ}$), and test case 3 is in the presence of a strong shock wave ($Ma=0.78$ and $AoA=1.57^{\circ}$). Five models are compared, including the AE, VAE, 30-VAE ($\beta$-VAE with $\beta=30$), 100-VAE  ($\beta$-VAE with $\beta=100$), and 1000-VAE ($\beta$-VAE with $\beta=1000$). In Fig. \ref{fig:recon_compar}, it can be confirmed that the reconstructed pressure contours for all three cases and all models do not exhibit any significant difference, which indicates that all models have been trained with sufficient accuracy to predict the flow fields.

\begin{figure*}[htb!]
	\centering
		\includegraphics[width=1.\textwidth]{figures/recon_compar_cut.png}
	\caption{Reconstructed pressure fields of the trained models (AE, VAE, 30-VAE, 100-VAE, and 1000-VAE).}
	\label{fig:recon_compar}
\end{figure*}

\subsection{\label{sec:activness} Activeness/Inactiveness of LVs}

The main purpose of the $\beta$-VAE is to encourage the model to learn the most efficient representation of the dataset \cite{higgins2016beta}. That is, the $\beta$-VAE enables the model to learn disentangled features by pushing a larger penalty on the KL divergence loss term. One of the interesting phenomena in the VAE, also identified by Burda et al. and Sønderby et al., is that some of the LVs in the VAE can become inactive during the training process \cite{burda2015importance, sonderby2016ladder}. Burda et al. argued that this phenomenon stems from the property of the loss function in the VAE rather than optimization issues. The loss function of the VAE model is almost the same as that of the $\beta$-VAE (both consist of a MSE and KL divergence); thus, it can be easily inferred that the same phenomenon will occur in the $\beta$-VAE. In fact, Eivazi et al. discovered that as $\beta$ increases (more penalty on the KL divergence term), the number of inactive LVs increases \cite{eivazi2022towards}. Therefore, it can be concluded that some LVs become inactive owing to the KL divergence term, which is incorporated into the loss function of the VAE/$\beta$-VAE to facilitate the disentanglement of the extracted features.

In this context, it is crucial to determine the rank of the LVs according to their degree of activeness to determine whether they are activated or inactivated. From the perspective of interpretability, ranking LVs can also be useful for analyzing the dominant patterns of complex flows. In this regard, Eivazi et al.\cite{eivazi2022towards} proposed a strategy to rank LVs by quantifying the contribution of LVs to the reconstruction. However, this approach requires a separate post-processing of the reconstruction after training the $\beta$-VAE, and the rank of LVs are indirectly evaluated in the output space rather than in the latent space. Consequently, the approach is less related to KL-divergence which is the main cause of the inactiveness of LVs. To address this problem, we investigate the method that is directly related to latent space and does not require any post-processing of the reconstruction.

As the first step in this approach, the standard deviations of the LVs ($\sigma_{z}$) from the training dataset are shown in Fig. \ref{fig:LV_std}. When $\sigma_{z}$ is high for a specific LV, it indicates that the corresponding LV is activated during the training process. In contrast, when $\sigma_{z}$ is low, it can be understood that the corresponding LV remains inactive during training (a scenario in which the value of the LV does not change though various data is given as input). In Fig. \ref{fig:LV_std}, it can be confirmed that both the AE and VAE models do not have any inactive LVs (all $\sigma_{z}$ values are larger than 0.5). However, a completely different trend is observed in the 30-VAE model: only four LVs (LV index 6, 8, 11, and 15) are activated. As $\beta$ increases, this trend becomes more prominent. In the 100-VAE model, only three (LV index 8, 11, and 15) are activated, and in the 1000-VAE model, two (LV index 8 and 11) are activated.

\begin{figure}[htb!]
    \centering
	    \includegraphics[width=.65\textwidth]{figures/LV_std_cut.png}
	\caption{Standard deviations of LVs from the training dataset (AE, VAE, 30-VAE, 100-VAE, and 1000-VAE models are compared).}
	\label{fig:LV_std}
\end{figure}

From the results obtained thus far, it may seem intuitive to examine the activeness of LVs via the standard deviation $\sigma_{z}$. However, this criterion is not directly related to the KL-divergence term, which is the primary reason for its inactiveness \cite{burda2015importance,eivazi2022towards}. Therefore, in this study, the activeness/inactiveness of the LV is also analyzed based on the KL divergence, and the results are shown in Fig. \ref{fig:KL_ANOVA}. Interestingly, compared to Fig. \ref{fig:LV_std}, it can be verified that LVs with a high $\sigma_{z}$ also have a high KL divergence (similarly, LVs with a low $\sigma_{z}$ also have a low KL divergence). For example, when the LVs are sorted by $\sigma_{z}$ and the KL divergence in the AE model, the top three LVs in both cases are LV14, LV9, and LV15. Similarly, for both cases in the 1000-VAE model, the top two LVs (there are only two LVs with significant $\sigma_{z}$/KL divergence) are LV8 and LV11. Therefore, it can be concluded that the activeness of the LV in terms of $\sigma_{z}$ and the KL divergence of the LV have a close relationship (which is also supported by the fact that the inactivated LVs have both $\sigma_{z}$ and KL-divergence values of zero).

\begin{figure}[htb!]
    \centering
	    \includegraphics[width=.65\textwidth]{figures/KL_ANOVA_cut.png}
	\caption{KL divergence and Sobol results with respect to LVs from the training dataset (AE, VAE, 30-VAE, 100-VAE, and 1000-VAE models are compared).}
	\label{fig:KL_ANOVA}
\end{figure}

In contrast to the AE and VAE models, notable trends are observed for the 30-VAE, 100-VAE, and 1000-VAE models. Each model has only four, three, and two activated LVs, respectively, both in terms of $\sigma_{z}$ and the KL divergence, for example, only LV6, LV8, LV11, and LV15 are activated for the 30-VAE. These results are consistent with those of Eivazi et al., who discovered that the number of activated LVs decreases as beta increases in the $\beta$-VAE model \cite{eivazi2022towards}. This phenomenon can be interpreted as increasing the weight factor on the KL divergence ($\beta$), which is responsible for regularization, encourages a sparser latent space \cite{kingma2013auto, doersch2016tutorial, higgins2016beta} (this sparsification effect of the KL divergence can be considered similar to that of the L1 regularization term in the Lasso regression \cite{tibshirani1996regression}).
Considering the fact that the regularization effect of KL divergence causes inactiveness in latent space, this study proposes an approach for ranking the activeness of LVs according to KL divergence.

In fact, a similar approach was applied by Sønderby et al., who judged whether the LV is activated via the KL divergence \cite{sonderby2016ladder}. However, the relationship between the activeness of the LV and KL divergence was not described clearly; therefore, this study also attempts to clarify this relationship. Further investigations are conducted to check whether the LV judged to be active in terms of the KL divergence actually has a greater effect on the reconstruction flow field than the inactive LV (conversely, to check whether LVs judged to be inactive in terms of the KL divergence have a smaller effect on the reconstructed field). Sobol sensitivity analysis is utilized for this investigation. A total 18,432 latent vectors are sampled using the Saltelli sampler in the range of $[\mu_{k}-2\sigma_{k}, \mu_{k}+2\sigma_{k}]$, and their corresponding reconstructed flow fields are obtained through the decoder parts of the AE and VAE/$\beta$-VAE models. Sobol analysis should be conducted on the scalar output; therefore, a set of 3$\times$512$\times$128 pixels of the flow fields is converted to a scalar value before the analysis (any scalar value representing the main characteristics of the flow field can be used for this purpose, and the sum of all pixel components is used in this study). Fig. \ref{fig:KL_ANOVA} shows the calculated first-order Sobol indices. It can be confirmed that the higher the KL divergence, the larger the value of the Sobol indices in the VAE/$\beta$-VAE models (because the KL divergence does not affect the AE model, the model does not exhibit a clear trend with the Sobol indices). These results demonstrate the validity of the KL divergence as a criterion for quantifying the activeness of LVs. In summary, the KL divergence is used to rank the activeness of each LV in this study, and the justification of this decision-making process is successfully shown through intuitive but quantitative investigations by comparing the KL divergence ranking with $\sigma_{z}$ from the training dataset (Fig. \ref{fig:LV_std}) and the Sobol indices (Fig. \ref{fig:KL_ANOVA}).

\subsection{\label{sec:independence} Independence of LVs}

The previous section (Sec. \ref{sec:activness}) introduced a method to sort activated LVs using the KL divergence. This study attempts to exploit these activated LVs by interpreting them as compressed information to represent the training dataset. In this context, the final goal is to determine which physical features these variables actually contain. Before examining the physical meaning of the activated LVs, the independence of each LV is investigated. The KL-divergence penalty does not simply force some portion of the latent space to be inactive; its original purpose is to promote independence and, therefore, the interpretability of LVs \cite{higgins2016beta}. Eivazi et al. measured the independence of LVs using a Pearson correlation matrix \cite{eivazi2022towards}. The same approach is applied in this study to investigate the orthogonality (or independence) of the LVs in all trained models. In Fig. \ref{fig:heatmap}, the absolute values of the components in the Pearson correlation matrix are shown from the AE to 1000-VAE models. Herein, it can be verified that their values decrease as $\beta$ increases, which indicates that the LVs gradually become independent of each other. These results practically prove that the KL-divergence term encourages the independence of each LV (the fact that the absolute correlations are largest in the AE model also supports this). The Pearson correlation coefficients ($r$) of the top two variables ranked by the KL divergence and their $p$-values are calculated and summarized in Table \ref{tab:pval} to scrutinize the independence of LVs in more depth. Again, clear trends are observed, in which LVs become uncorrelated as $\beta$ increases. As $\beta$ increases, $r$ decreases, thereby indicating that the top two variables become increasingly uncorrelated, and the $p$-value increases, which indicates that their correlation becomes increasingly statistically insignificant.

\begin{figure}[htb!]
    \centering
	    \includegraphics[width=.7\textwidth]{figures/heatmap_cut.png}
	\caption{Absolute values of the components in the Pearson correlation matrix for LVs (AE, VAE, 30-VAE, 100-VAE, and 1000-VAE models are compared).}
	\label{fig:heatmap}
\end{figure}

\begin{table}[]
\centering
\caption{Pearson correlation coefficient ($r$) and its $p$-value for top two variables ranked based on KL divergence (AE, VAE, 30-VAE, 100-VAE, and 1000-VAE models are compared)}
\begin{tabular}{ccc}
\hline \hline
Models & \begin{tabular}[c]{@{}c@{}} $r$ \end{tabular} & $p$-value \\ \hline
AE       & { 0.520} & { 1.53e-32} \\
VAE      & { 0.195} & { 3.04e-05} \\
30-VAE   & { -0.023} & { 6.27e-01} \\
100-VAE  & { -0.016} & { 7.34e-01} \\
1000-VAE & { 0.012} & { 7.98e-01} \\ \hline \hline
\end{tabular}
\label{tab:pval}
\end{table}

Eivazi et al. also computed the determinant of the correlation matrix to measure the degree of correlation within the entire set of LVs (when the determinant has a value of 0/1, it indicates that these variables are completely correlated/uncorrelated) \cite{eivazi2022towards}. However, the calculation of the determinant of the matrix shown in Fig. \ref{fig:heatmap} indicates the independence of the all LVs (16 LVs in this study), which can be meaningless in the $\beta$-VAE models. The number of activated LVs is significantly smaller than 16; thus, analyzing all LVs may be pointless. Therefore, the independence of various LV combinations is further analyzed. First, possible combinations of two to seven LVs are obtained (e.g., the number of combinations consisting of four LVs is ${}_{14} C_{4}$, and each combination comprises a 4$\times$4 correlation matrix). The determinants of all these combinations are then calculated, and their statistics are summarized in Fig. \ref{fig:combi}. It can be confirmed that for any number of LVs used in a combination, the $\beta$-VAE models have significantly higher determinants than the AE and VAE models. In summary, the results presented in Fig. \ref{fig:heatmap} and \ref{fig:combi} consistently show that the $\beta$-VAE successfully learns more uncorrelated LVs compared to the AE and VAE models, owing to the KL divergence, which forces the LVs to be independent. These independent (or uncorrelated) LVs are expected to have interpretable (or disentangled) physical features, which will be discussed in Sec. \ref{sec:Disen}.

\begin{figure}[htb!]
    \centering
	    \includegraphics[width=.9\textwidth]{figures/combi_cut.png}
	\caption{Determinants of Pearson correlation matrices for combinations of 2 to 7 LVs (AE, VAE, 30-VAE, 100-VAE, and 1000-VAE models are compared).}
	\label{fig:combi}
\end{figure}

Thus far, independence within the trained latent space has been investigated in a quantitative manner, but this approach does not provide straightforward information on what these variables independently contain. Accordingly, each LV is visualized via a traversal of itself, which is the most widely adopted approach in the field of representation learning (e.g., latent traversal plots of celebA, 3D chairs, and 2D shape dataset in the study by Higgins et al. \cite{higgins2016beta}). The latent traversal plot shows the traversal of a single LV with other fixed LVs. Therefore, it is important to visually understand the features of only one LV of interest without the intervention of other variables. As an example of the application of this plot in an engineering field, Yang et al. utilized it to effectively visualize the various surface-pressure distributions on an airfoil, which were generated by a VAE model decoder \cite{yang2022inverse}. In this study, since the generated output by the AE and VAE/$\beta$-VAE models is a flow field, a latent traversal plot is applied to identify the physical features of the flow field contained in the LV. Fig. \ref{fig:lat_trav} shows the pressure flow fields for two extreme LVs. One is the most dominant LV, which is ranked first by KL divergence, and the other is ranked last; therefore, it is the most trivial. The noteworthy point in this figure is that in the AE model, the pressure field changes abruptly as the most dominant LV changes. When the most trivial LV changes, the field changes gradually, and the change is not as significant as that of the most dominant LV case. Because the LV inactiveness does not occur in the AE model (owing to the lack of the KL divergence term in the loss function), all the variables are activated and contain uninterpretable (or entangled) physical features. However, in the case of the VAE and 1000-VAE models, the most dominant LVs also cause substantial changes, but the most trivial LVs cause indistinguishable variations in the flow field. This can be regarded as a virtue of the KL divergence in the $\beta$-VAE model. It forces inactiveness in the latent space, therefore, only activated LVs contain meaningful and independent physical features. In particular, when $\beta$ is large, this effect becomes stronger, as observed in the sweep of the most dominant LV in the 1000-VAE model \cite{eivazi2022towards}: this LV can be interpreted as containing information on the occurrence of the shock wave, and the variation of the flow field it arouses is the largest compared to other LVs.

\begin{figure}[htb!]
    \centering
	    \includegraphics[width=1.\textwidth]{figures/lat_trav.png}
	\caption{Latent traversal plots of pressure flow fields for two extreme LVs: first (most dominant) and last (most trivial) variables ranked by KL divergence (AE, VAE, and 1000-VAE models are compared).}
	\label{fig:lat_trav}
\end{figure}

\subsection{\label{sec:Disen} Physics-awareness of LVs}

In Sec. \ref{sec:activness}, the activeness/inactiveness of the VAE/$\beta$-VAE in the latent space was discussed. Then, Sec. \ref{sec:independence} investigated the independence of the physical features contained in the LVs of the $\beta$-VAE. In this section, the LVs that satisfy both activeness and independence are scrutinized because they are expected to contain the physics-aware information of the dataset. Thus, a physics-oriented investigation is conducted to understand the physical meaning that each LV contains.

First, we examine the relationship between the physical parameters (whose distribution is shown in Fig. \ref{fig:ma_aoa_a}) and the top two LVs (ranked by KL divergence) of the training dataset. Accordingly, in Fig. \ref{fig:ma_aoa_b}, the distribution of the training dataset with respect to those two variables is visualized using a plot where the x-axis is the most dominant LV (or $1^{st}$ dominant LV), and the y-axis is the second dominant LV (or $2^{nd}$ dominant LV). For the AE, VAE, and 1000-VAE models, the plots of the first column are colored according to the Ma value, and the second column is colored according to the AoA value. In the AE model, there are no noticeable trends for either Ma or AoA. In contrast, in the VAE model, the plot in the first column exhibits a trend that is not clear, but still noticeable: as the $2^{nd}$ dominant LV increases, Ma increases. Similarly, in the plot of the second column, as the $1^{st}$ dominant LV increases, AoA also tends to increase. However, the increase in Ma cannot be explained by the increase in the $2^{nd}$ LV alone, as shown by the color contour of Ma (left column). When explaining the physical parameter of Ma, there is some influence from the $1^{st}$ LV; however, this influence is weaker than that of the $2^{nd}$ LV. Similarly, it cannot be concluded that the $2^{nd}$ LV alone is responsible for the physical parameter AoA. These ambiguous correlations between the two LVs and two physical parameters become disentangled in the 1000-VAE model. Clear correlations between the LVs and physical parameters are observed in both the left and right plots of the 1000-VAE model. Unlike the VAE, when describing a physical parameter, the role of each LV is decoupled; thus, each parameter can be explained with only one LV. The behavior of Ma and AoA can be explained by only the $1^{st}$ and $2^{nd}$ dominant LVs, respectively (for the sake of brevity, $LV_{Ma}$ refers to the $1^{st}$ dominant LV, and $LV_{AoA}$ refers to the $2^{nd}$ dominant LV from now on, which implies that $LV_{Ma}$ and $LV_{AoA}$ are the LVs responsible for Ma and AoA, respectively). In Fig. \ref{fig:ma_aoa_b}, the trajectories of the boundary data are also shown (the boundary data are extracted from the boundary of the physical parameter space, as in Fig. \ref{fig:ma_aoa_a}). In the AE model, it can be confirmed that the latent space cannot be closed by only two dominant LVs because the trajectory of the boundary data does not cover the entire training dataset. This suggests that additional LVs must be considered to represent the trained latent space in an AE model. However, in the 1000-VAE model, by using only two activated LVs, the latent space of the entire training dataset is exactly closed by the boundary data trajectory. This observation supports the validity of our method for ranking LVs based on KL divergence because the latent space of the 1000-VAE model can be sufficiently represented by only the two LVs activated in terms of the KL divergence ($LV_{Ma}$ and $LV_{AoA}$). Again, all the observed results in Fig. \ref{fig:ma_aoa} can be regarded as a consequence of the KL divergence in the $\beta$-VAE.

The visual analysis in the previous paragraph is qualitative; therefore, a quantitative analysis is performed to verify whether $LV_{Ma}$ or $LV_{AoA}$ affects Ma or AoA independently in the 1000-VAE model. Single-variable linear regression (LR) models are trained to implement this analysis (the relationships between physical parameters and LVs appear linear in the qualitative analysis; therefore, the LR model is considered to be sufficient to represent their relationship). Specifically, one LR model is trained with the input variable as $LV_{Ma}$ and the output variable as Ma (which can be expressed as Ma=$f(LV_{Ma})$), and the other is trained with the input variable as $LV_{AoA}$ and the output variable as AoA (the training dataset is the same as that in Sec. \ref{sec:problem_def}, which has a size of 450). If $LV_{Ma}$ and Ma (or $LV_{AoA}$ and AoA) have a linear relationship, the fitted LR models perform well with the test dataset (the test dataset is the same as that in Sec. \ref{sec:problem_def}, which has a size of 50). The results of these two LR models are shown in Fig. \ref{fig:lin_reg}. In each subplot, the equation of the trained LR model between the LV and its corresponding physical parameter is also included. For the LR model of $LV_{Ma}$ and Ma, the coefficient of determination ($R^{2}$) that is calculated using the test dataset is 0.950, and the $R^{2}$ value of the LR model of $LV_{AoA}$ and AoA is calculated to be 0.969. Both models exhibit a high generalized performance, which indicates that each physical parameter can be represented by the linear relationship of only one LV with sufficient accuracy. In addition, we train two additional LR models with both LVs as input features: one with Ma as the output (Ma=$f(LV_{Ma},LV_{AoA})$) and the other with AoA as the output (AoA=$f(LV_{Ma},LV_{AoA})$). If Ma is expressed using both LVs, the $R^{2}$ value of the LR model Ma=$f(LV_{Ma},LV_{AoA})$ is significantly higher than that of the LR model Ma=$f(LV_{Ma})$ (the same principle applies to AoA). The trained LR models with both LVs as inputs are described by Eq. \ref{eq:bi_lr}, and there are two notable points. First, the coefficient of one LV is negligible compared to that of the other LV considering that $LV_{Ma}$ and $LV_{AoA}$ have similar scales (most of them are in the range of [-2,2] for the training dataset; see the x-axis and y-axis ranges of the 1000-VAE model in Fig. \ref{fig:ma_aoa_b}). For example, when approximating Ma, the coefficient of $LV_{AoA}$ (which is -0.00094672) is negligible compared to the coefficient of $LV_{Ma}$ (which is 0.08448135). It can be understood that for 0.1 increments of Ma, $LV_{Ma}$ requires 1.18 increments, whereas $LV_{AoA}$ requires 105.63 increments. This indicates that Ma can be expressed only by $LV_{Ma}$ while ignoring $LV_{AoA}$, which is consistent with the previous qualitative analysis. The second interesting point is that the values of $R^{2}$ do not change compared to those of the single-variable LR models. For the regression of Ma, the incorporation of $LV_{AoA}$ into the input feature does not change $R^{2}$. In the case of AoA, when $LV_{Ma}$ is included, only a 0.001 increase in $R^{2}$ is observed. These negligible improvements in the $R^{2}$ values also suggest that the two active LVs represent Ma and AoA in a disentangled (or independent) manner.

\begin{equation}\
\label{eq:bi_lr}
\begin{bmatrix}
Ma \\
AoA
\end{bmatrix} =
\begin{bmatrix}
0.08448135 & -0.00094672\\
-0.01486785 & 0.8536322
\end{bmatrix}
\begin{bmatrix}
\text{$LV_{Ma}$} \\
\text{$LV_{AoA}$}
\end{bmatrix} +
\begin{bmatrix}
0.6516843 \\
1.5056742
\end{bmatrix}.
\end{equation}

\begin{figure*}[p]
\begin{center}
\subfloat[\label{fig:ma_aoa_a}]{\includegraphics[width=7cm]{figures/ma_aoa_a_cut.png}}
\quad \quad
\subfloat[\label{fig:ma_aoa_b}]{\includegraphics[width=\textwidth]{figures/ma_aoa_b_cut.png}}
\caption{Investigation of physical features contained in the top two LVs: (a) distributions of training dataset and boundary data for Ma and AoA, and (b) distributions of training dataset for first and second LVs (the left figures are colored according to Ma, and the right figures are colored according to AoA).}
\label{fig:ma_aoa}
\end{center}
\end{figure*}

\begin{figure*}[htb!]
\begin{center}
\subfloat[\label{fig:lin_reg_a}]{\includegraphics[width=6cm]{figures/lin_reg_a_cut.png}}
\subfloat[\label{fig:lin_reg_b}]{\includegraphics[width=6cm]{figures/lin_reg_b_cut.png}}
\caption{The results of the single variable linear regression: (a) the results of Ma=$f(LV_{Ma})$, and (b) the results of AoA=$f(LV_{AoA})$.}
\label{fig:lin_reg}
\end{center}
\end{figure*}

The physical meanings of $LV_{Ma}$ and $LV_{AoA}$ are verified both qualitatively and quantitatively. Next, the latent traversal plots of airfoil surface pressure distributions are shown in Fig. \ref{fig:cp_ma_aoa} to check the influence of each variable on the flow field more intuitively. During the traversal of $LV_{Ma}$ (Fig. \ref{fig:cp_ma_aoa_a}), the presence or absence of the shock wave on the upper surface changes: when the LV is in the range of $\mu_{k}-3\sigma_{k}$ to $\mu_{k}$, there is no shock wave, but when the LV is increased to $\mu_{k}+1.5\sigma_{k}$ and $\mu_{k}+3\sigma_{k}$, the surface pressure distributions denote the occurrence of a shock wave, which indicates that $LV_{Ma}$ corresponds to Ma. In the traversal of $LV_{AoA}$ (Fig. \ref{fig:cp_ma_aoa_b}), a variation in the magnitude of the suction peak at the leading edge is observed, which indicates that $LV_{AoA}$ corresponds to AoA. In summary, a visual analysis of the actual effects of $LV_{Ma}$ and $LV_{AoA}$ on the pressure distributions shown in Fig. \ref{fig:cp_ma_aoa} also leads to the consistent conclusion that they correspond to Ma and AoA, respectively.

\begin{figure*}[htbp!]
\begin{center}
\subfloat[\label{fig:cp_ma_aoa_a}]{\includegraphics[width=6cm]{figures/cp_ma_aoa_a_cut.png}}
\subfloat[\label{fig:cp_ma_aoa_b}]{\includegraphics[width=6cm]{figures/cp_ma_aoa_b_cut.png}}
\caption{Latent traversal plots of airfoil surface-pressure distributions in the 1000-VAE: (a) traversal of $LV_{Ma}$, and (b) traversal of $LV_{AoA}$.}
\label{fig:cp_ma_aoa}
\end{center}
\end{figure*}

The overall process, from the generation of the dataset to the extraction of physics-aware LVs, is shown schematically in Fig. \ref{fig:doe_to_disen}. First, the initial dataset is selected based on the design of the experiments (Latin hypercube sampling in this study) with respect to two physical parameters: Ma and AoA. The generated dataset is then calculated using a flow simulation tool, and the resultant flow-field dataset is trained using the 1000-VAE model. Finally, the trained latent space is inspected using the KL-divergence ranking criterion, and two activated variables are derived. Through various approaches (qualitative and quantitative), it was confirmed that each LV corresponds to Ma and AoA, which are the physical parameters utilized directly in the training-dataset-generation process.

\begin{figure}[htb!]
    \centering
	    \includegraphics[width=.9\textwidth]{figures/doe_to_disen_cut.png}
	\caption{Schematic showing the process from the initial generation of the dataset through physical parameters (Ma and AoA) to their re-extraction by the 1000-VAE model in the form of activated LVs ($LV_{Ma}$ and $LV_{AoA}$).}
	\label{fig:doe_to_disen}
\end{figure}

\subsection{\label{sec:regression} Effects of physics-awareness of LV on ROM}

The previous sections scrutinized the physical meaning of each LV through both qualitative and quantitative analyses and determined the physics-aware LVs that are solely responsible for physical parameters. In this section, the effects of these physics-aware LVs are investigated with respect to ROM performance. This is necessary because the computational expenses required to train regression models can be effectively reduced if physics-unaware LVs can be ignored.

Fig. \ref{fig:/MSE_reg_active} shows the MSE of the regression models (referring to $MSE_{reg}$) in the AE and VAE/$\beta$-VAE models (as in Fig. \ref{fig:flowchart_rom}, these regression models predict LVs in the latent space from the physical parameters, that is, Ma and AoA in this study). Each point represents the regression model of each LV, and the symbol o/x indicates whether the LV is activated or not.
The results show that the $MSE_{reg}$ values of physics-unaware (inactivated) LVs are considerably higher than those of physics-aware LVs. This is because the correlations between physics-unaware LVs and physical parameters are too trivial for regression models to be trained with high accuracy. Fig. \ref{fig:RS_1000VAE} supports this; the figure shows that the response surface of the physics-unaware (inactivated) LV is considerably noisier than that of the physics-aware LV. Another interesting point shown in Fig. \ref{fig:/MSE_reg_active} is that the $MSE_{reg}$ values of the physics-aware LVs are generally decreasing as $\beta$ increases. This implies that the tight coupling between the activated LVs and physical parameters is advantageous in terms of regression performance in the ROM process.

\begin{figure}[htb!]
    \centering
	    \includegraphics[width=.65\textwidth]{figures/MSE_reg_activeness.png}
	\caption{MSE of the regression models (AE, VAE, 30-VAE, 100-VAE, and 1000-VAE models are compared)}
	\label{fig:/MSE_reg_active}
\end{figure}

\begin{figure*}[htb!]
\begin{center}
\subfloat[\label{fig:RS_1000VAE_D}]{\includegraphics[width=6cm]{figures/RS_1000VAE_D.png}}
\quad \quad
\subfloat[\label{fig:RS_1000VAE_T}]{\includegraphics[width=6cm]{figures/RS_1000VAE_T.png}}
\caption{Comparison of the response surface of a physics-aware LV and physics-unaware LV in the 1000-VAE: (a) physics-aware LV (active), (b) physics-unaware LV (inactive).}
\label{fig:RS_1000VAE}
\end{center}
\end{figure*}

We examine the effect of each LV on the accuracy of ROM by calculating the MSE between the ground truth and the prediction of ROM with the exclusion of $k^{th}$ LV (please note that, as in Fig. \ref{fig:flowchart_rom}, the prediction of ROM means obtaining the flow field from unknown physical parameters through the regression model and decoder part of the AE and VAE/$\beta$-VAE models). The MSE is denoted as $MSE_{pred, -k}$, where the minus sign indicates the exclusion of LV. Specifically, when the $k^{th}$ LV is selected to be excluded from this analysis, its value is assumed to be $\mu_{k}$, whereas the values of the other LVs are predicted from the regression models. Fig. \ref{fig:/MSE_pred_loo} demonstrates the resultant $MSE_{pred, -k}$ values, where the x-axis indicates the ranking of the LVs sorted according to KL divergence. An important observation is that the effect of the LV on $MSE_{pred, -k}$ decreases as the activeness ranking of the LV moves backward. By confirming that an LV ranked higher by KL divergence has a greater effect on ROM accuracy, it can be concluded that the approach of ranking LVs based on their KL divergence is also valid in terms of ROM-prediction performance. Moreover, we have verified that inactive LVs, such as those ranked from 5 to 16 in the 30-VAE, from 4 to 16 in the 100-VAE, and from 3 to 16 in the 1000-VAE, have negligible effects on $MSE_{pred, -k}$ (they are constant in these ranges). This implies that there is no need to train the regression models of physics-unaware LVs with respect to the prediction accuracy of ROM.

Fig. \ref{fig:/MSE_pred_model_compa} compares the $MSE_{pred}$ of ROM based on the AE and VAE/$\beta$-VAE models, where all physics-unaware LVs are excluded. The labels in Fig. \ref{fig:/MSE_pred_model_compa} represent the DR model and the number of physics-aware LVs used for prediction. For example, 1000-VAE-2LV denotes the 1000-VAE-based ROM prediction with only two activated LVs. Although a considerably smaller number of LVs are used in the $\beta$-VAEs than the AE, the prediction accuracy of the $\beta$-VAE models is comparable to that of the AE. It is also noteworthy that the accuracy of AE-based ROM cannot be guaranteed, even if only one LV is excluded (AE-15LV in Fig. \ref{fig:/MSE_pred_model_compa}), which indicates that all LVs of the AE model are entangled.

To summarize the results thus far, physically interpretable LVs can be obtained by the $\beta$-VAE with the proper hyperparameter $\beta$ ($\beta=1000$ in this study). These LVs, which are called physics-aware LVs, can be identified by the KL-divergence criteria proposed in this study. It was confirmed that physics-aware LVs are disentangled from each other and strongly correlated with physical parameters, which are the generating factors of the training dataset. By exploiting this, the computational cost to train the regression models can be reduced by excluding physics-unaware LVs. The overall procedure for the physics-aware ROM proposed in this study is summarized in Algorithm \ref{alg:pa_rom}.

\begin{figure}[htb!]
    \centering
    \includegraphics[width=0.65\textwidth]{figures/MSE_pred_loo.png}
    \caption{MSE of ROM prediction with the exclusion of $k^{th}$ LV (AE, VAE, 30-VAE, 100-VAE, and 1000-VAE models are compared).}
    \label{fig:/MSE_pred_loo}
\end{figure}
\begin{figure}[htb!]
    \centering
    \includegraphics[width=0.5\textwidth]{figures/MSE_pred_model_compa.png}
    \caption{MSE of ROM prediction with the exclusion of physics-unaware LVs.}
    \label{fig:/MSE_pred_model_compa}
\end{figure}

\begin{algorithm}[htb!]
\caption{Physics-aware ROM via $\beta$-VAE}\label{alg:pa_rom}
\begin{algorithmic}

\State (1) Prepare dataset from the physical phenomenon.
\State (2) Train $\beta$-VAE with the proper selection of $\beta$ through grid search algorithm.
\State (3) Identify the physics-aware LVs based on KL divergence and freeze the physics-unaware LVs as their respective mean values.
\State (4) Train regression models to correlate the physics-aware LVs with physical parameters.
\State (5) Obtain physically interpretable results via physics-aware ROM.

\end{algorithmic}
\end{algorithm}

\section{\label{sec:conclustion} Conclusion}
This study proposed physics-aware ROM using $\beta$-VAE, enabling the physical interpretation of latent space. Our approach can measure the rank and independence of LVs, which are useful information for analyzing the dominant patterns of complex physical system. The key contributions of this study can be summarized as follows:

\begin{enumerate}
    \item The effects of hyperparameter $\beta$ on the activeness of the LVs were investigated; With an increasing $\beta$ value, the number of active LVs decreases, enabling more compact latent representation. Additionally, the activeness of LVs is measured by KL-divergence in the latent space of $\beta$-VAE. This approach does not require separate post-processing of reconstructed data, and can be extended to any physical/non-physical problems. It was verified that the KL-divergence has a consistent trend with other measures, such as the standard deviations or Sobol indices of LVs.

    \item The impacts of hyperparameter $\beta$ on the independence of LVs were scrutinized; With increasing $\beta$, LVs become more independent of each other. This trend is verified both qualitatively and quantitatively using latent traverse plots and Pearson correlation coefficients.

    \item Physics-aware LVs that contain interpretable physical features were successfully identified. These LVs can solely represent the physical parameters that are the generative factor of training dataset, i.e., the Mach number and angle of attack in this study. Each LV independently represents the distinct aerodynamic features, such as the existence of shock wave and the magnitude of suction peak. To the best of authors' knowledge, this is the first observation of physics-aware LVs in the fields of fluid dynamics.

    \item By analyzing the effect of physics-aware LVs on the accuracy of ROM, the potential to reduce the computational cost of ROM was verified. Physics-aware LVs have much higher accuracy for regression than physics-unaware LVs because physics-unaware LVs are not sufficiently meaningful for the regression model to be trained. Thus, physics-unaware LVs have a negligible effect on the performance of ROM, and can be excluded to reduce the computational cost.

\end{enumerate}

These contributions allow the physical meaning of LVs to be easily identified, which has not been achieved in previous DNN-based ROM studies. Given that the presented approach is data-driven, it can be easily extended to other physical problems, such as hydraulic, structural, and molecular dynamics. For future work, a more comprehensive investigation of physics-aware LVs will be conducted, such as the effect of the distribution of the training dataset or their ability to discriminate the redundant physical parameters.

\section*{Declaration of Competing Interest}
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

\section*{Acknowledgement}
This work was supported by a National Research Foundation of
Korea grant funded by the Ministry of Science and ICT (NRF-2017R1A5A1015311).

\section*{Data availability statement}
The data that support the findings of this study are available from the corresponding author upon reasonable request.

\nocite{*}

redpajama_set_name: RedPajamaArXiv


text:
Петровское  — деревня Охотинского сельского поселения Мышкинского района Ярославской области.

Деревня расположена между правым берегом Волги и федеральной автомобильной трассой . К северо-востоку от деревни на трассе стоит деревня Курзино, а к юго-востоку также на трассе — деревня Порхачи. Вокруг этих трёх деревень одно большое поле, окружённое лесом .

На 1 января 2007 года в деревне Петровское числилось 8 постоянных жителей . Деревню обслуживает почтовое отделение, находящееся в селе Охотино .

Примечания

Населённые пункты Мышкинского района

redpajama_set_name: RedPajamaWikipedia


text:
Tonight: C. Diab album launch + Eva Bowan / Lucy Claire Free ENTRY SHOW @Birthdays
(Tuesday) October 25, 2016 - 19:30
Birthdays Dalston - 33-35 Stoke Newington Road (Map)
Injazero Records and Nice Weather for Airstrikes present:
C. Diab (ALBUM LAUNCH SHOW)
+ Eva Bowan + Lucy Claire
at Birthdays
33-35 Stoke Newington Rd, London N16 8BJ
730pm.
http://www.injazerorecords.com
http://niceweatherforairstrikes.co.uk
Use DICE - Best Gigs, No Booking Fees for your tickets and receive a FREE poster print (*T&Cs apply).
C. Diab
http://injazerorecords.com/
"Patient, hauntingly beautiful"
XLR8R
"Emotionally devastating...nothing short of extraordinary." Disorder Magazine
A resident of the spectacular wilderness of Canadian island town Port Hardy (located on the northern tip of Vancouver Island), solo bowed guitar player / trumpet player / tape manipulator C. Diab creates in his music something comparable to the remote and otherworldly beauty of his surroundings.
However, there is a lack of clear musical references to his LP No Perfect Wave. In its singularity and intersection of niche geographies - both musical and physical - the record lands only momentarily on comparison points. Listeners of Colin Stetson, for example, will enjoy C. Diab's virtuosic instrumental technique; fans of Tim Hecker and Richard Skelton will find interest in the glacial slow-motion textures; here is a soul-bearing and intimate tenderness in the same way as Arthur Russell; John Fahey and Robbie Basho scholars will appreciate his nods to the dark underworld of North American folk music. But, as with the best records, the result is much greater than these parts. The truth more distinctive. This is partly due to C. Diab's playing style. He describes his work as 'Cascadian guitar music'. Where Appalachian Guitar Music has a well-known musical identity, C. Diab's elegant cello-bowed guitar compositions have a quality just as rare and powerful, captured with meditative and swooping grace on No Perfect Wave. As longing and expressive as solo cello, droning as leftfield ambient music, moving as film score and cathartic as religious song. He names the style in reference to the biogeographical (but not political) territory of the Cascade mountain region (Northwestern USA / Southwestern Canada), which is - just as C. Diab's music - mountainous, moody, mesmeric, and filled with breathtaking majesty.
C. Diab - Caton Diab, born 1986, Ottawa, Canada - is a friend and collaborator of acclaimed vocalist / composer Ian William Craig (FatCat / 130701 Records), whose engineering work on No Perfect Wave lead Injazero Records to C. Diab's music. The record marks not only C. Diab's first album proper, but also Injazero's first physical album, released worldwide on 180g vinyl and digital formats on October 7th 2016.
Eva Bowan
http://evabowan.com
"Dense, ambient tracks ideal for late-night immersion, where thick waves of analogue synths roll over distant rhythms."
'Alien, absorbing and gorgeous."
Eva Bowan is a Polish born Brighton (UK) based multidisciplinary artist working in the fields of Music, Graphic Design, Photography, Video and Illustration.
Her interest lies in combining analogue and digital techniques, both in visual and sonic art, leading to exciting fusions of organic textures with electronic elements and glitch. She creates captivating surreal worlds which are a mixture of dreams, hidden fantasies, observing the nature and overanalysing the world around and within.
Her sound can be described as a unique combination of electronica, ambient and industrial pop. Recently she supported: James Holden, Swans, Tim Hecker, Gazelle Twin, Rosie Lowe, Silver Apples, Rival Consoles, Umberto, Noveller and more.
Lucy Claire
http://www.lucy-claire.com
Classical or not, one thing is indisputable – this music is beautiful. Listen on headphones and get lost in it.
Dani Charlton @ Amazing Radio
A swell of strings that leaves you feeling suspended in a sea of melodies.
Lucy Claire is a remarkable composer and producer, whose music fuses classical, cinematic and world influences. Her bittersweet piano motifs and melancholy string melodies, loaded with atmospheric soundscapes, along with off kilter harmonies and deep and compelling structures, have often been compared to music by composers such as Nils Frahm and Johann Johannsson as well as post rock artists like Mogwai.
*Only 30 print posters are available, offered on a first-come-first-served basis.
(Tuesday) October 25, 2016
Birthdays Dalston 33-35 Stoke Newington Road London, United Kingdom
ALTE SAU
Helsinki Zürich
Nice Weather For Airstrikes Records

redpajama_set_name: RedPajamaCommonCrawl


text:
La specialità dei 1000 metri maschili di short track degli VIII Giochi asiatici invernali si è svolta il 22 febbraio 2017 all'Arena del Ghiaccio Makomanai nel distretto di Minami-ku a Sapporo, in Giappone.

Risultati
Legenda
ADV — Avanzamento
PEN — Penalità
q — Qualifificato per il tempo

Batterie

Gruppo 1

Gruppo 2

Gruppo 3

Gruppo 4

Gruppo 5

Gruppo 6

Gruppo 7

Gruppo 8

Quarti di finale

Gruppo 1

Gruppo 2

Gruppo 3

Gruppo 4

Semifinali

Gruppo 1

Gruppo 2

Finali

Finale B

Finale A

 Il Sudcoreano Lee Jung-su si è classificato 3° con il tempo di 1.24,169, ma la medaglia di bronzo è stata attribuita al giapponese Keita Watanabe arrivato 4º perché il regolamento consentiva l'attribuzione di massimo due medaglie per ogni comitato olimpico partecipante e la Corea del Sud aveva già ottenuto l'oro e l'argento.

Short track agli VIII Giochi asiatici invernali


text:
NOKIA 3 Posts on XDA
HMD Global rolls out stable Android 10 update for the Nokia 3.1 Plus
Joe Fedewa May 13, 2020
The Nokia 3.1 Plus launched in 2018 with Android Oreo as part of the Android One program. A few months later, it was updated to Android 9 Pie, and now the device is receiving Android 10. This marks the second major OS update the Nokia 3.1 Plus has received in two years. Android 10 for the...
[Update 2: New Roadmap] HMD Global reveals Android 10 update roadmap for its Nokia smartphones, teases cheap 5G phone for 2020
Arol Wright March 11, 2020
The latest version of Android, formerly known as Android Q, will be called Android 10 as part of Google's newest efforts to give Android a fresher look. That's it, no fluff, no dessert name---just Android 10. And while this may make sense from a business perspective, it is still not sitting well with Android enthusiasts....
HMD Global will continue to support its first-gen Nokia phones for another year
Aamir Siddiqui August 15, 2019
Back in February 2017, HMD Global announced the re-entry of the Nokia brand with the international launch of the Nokia 3, Nokia 5 and Nokia 6 smartphones. This was followed up by the Nokia 8 in August 2017 and the Nokia 2 in October 2017. HMD Global executives mentioned in an interview that OEM skins...
Android 9 Pie is rolling out to the 2017 Nokia 3
Doug Lynch June 3, 2019
With all of the talk about changes happening in Android Q, some people forget that OEMs are still working on updating their smartphones to Android Pie. A little over a year ago, HMD Global announced that all Nokia smartphones would receive this update. This was a bold move as some companies are notorious for not...
Nokia 3.1 is HMD Global's latest smartphone to get Android Pie
Doug Lynch March 15, 2019
It's interesting to see which companies within the smartphone industry are able to handle OTA updates better than others. It would seem logical to think that companies making billions in revenue each year would be on top, but that hasn't been the case over the last few years. Instead, it's the smaller companies that have...
HMD Global unveils the Android Pie update roadmap for Nokia smartphones
George Burduli January 23, 2019
If we have to choose one thing to give credit to HMD Global for, I'd pick their commitment to software updates. They always tend to release the latest and greatest flavor of Android for their devices, especially in a timely manner. Thankfully, they use an almost vanilla version of Android, which makes it easier for...
Nokia 3 is now receiving the Android 8.1 Oreo update
George Burduli December 21, 2018
After they licensed the Nokia brand for use in smartphones, HMD Global has taken the industry by storm. The main reason for their success is their large portfolio of smartphones in the low and mid-range price categories, decent hardware, excellent software support, and of course, brand recognition. The Nokia 3 is one of HMD Global's...
Nokia 3.1 Android 8.1 Oreo update begins rolling out in India
Joe Fedewa September 14, 2018
The Nokia 3.1 was announced back in May, but it took a couple of months before it launched in India. The Nokia 3.1 was originally announced with Android 8.1, but in India, it launched with Android 8.0. Thankfully, Android 8.1 is starting to roll out to the Nokia 3.1 in India. HDM Global said the device...
[Update: 5.1 in the UK] Nokia 2.1, Nokia 3.1 3GB RAM variant, and Nokia 5.1 now available in India
Idrees Patel September 5, 2018
HMD Global officially announced the Nokia 2.1, Nokia 3.1, and the Nokia 5.1 in May. The phones are the successors to the Nokia 2, Nokia 3, and the Nokia 5 respectively. They slot in above the Nokia 1 Android Go entry-level phone and below the Nokia 6.1. The company's smartphone lineup gets a bit more...
Nokia 3.1 launched in India with 5.2″ 18:9 display and MediaTek MT6750 SoC
Idrees Patel July 19, 2018
HMD Global launched the Nokia 8 Sirocco, Nokia 7 Plus, the Nokia 6.1, and the entry-level Nokia 1 in India in April. In May, the company launched the Nokia 5.1, Nokia 3.1, and the Nokia 2.1 budget Android One phones. The Nokia 3.1 went up for purchase in the US a few weeks ago, and...
Nokia 3.1 now available in US for $159 from Best Buy & Amazon
Doug Lynch July 2, 2018
Since acquiring the licensing rights to sell Nokia-branded smartphones we have seen HMD Global achieve great success in the market. While some OEMs opt for heavy customizations in their software, HMD Global has opted to join the Android One program for all of their devices and as such have been able to quickly roll out...
Nokia Camera with Pro Camera mode ported to Nokia 3, Nokia 5, Nokia 6, and more
Earlier this year we learned that HMD Global was bringing the well-known Nokia Pro Camera mode to the Nokia Camera application for their Zeiss optics branded Android smartphones. This was first announced for the Nokia 7 Plus and Nokia 8 Sirocco and has since been released for the Nokia 7 and the Nokia 8. The company...
Galaxy S21 Ultra, 72 hours later: 5 takeaways after using Samsung's best phone


Although Loreta holds a Bachelor of Elementary Education from Urdaneta City University in The Philippines, she has always enjoyed working with food. In 2011, Loreta graduated from City & Guilds with a Diploma in Patisserie and followed that up with a Certificate in Asian Pastry Making from BITC. In 2017, Loreta graduated from Kaplan Higher Education Institute with a Diploma in Commerce (focusing on Hospitality & Tourism Management). We are pleased to welcome her to Palate Sensations as one of our Chef Instructors focusing on classes for domestic helpers, Asian Pastry, Thai Cuisine and supporting our Executive Chef in conducting classes. She is also in charge of the management of our kitchen.


text:
Defense Verdict Obtained in Slip and Fall Accident at Starbucks in Lawsuit Venued in Alameda County, California with Injury Claims of Traumatic Brain Injury, Tinnitus and Loss of Balance
On February 23, 2015, Plaintiff Liduina "Lee" Teles was visiting Berkeley, CA from Tewksbury, MA, to tour the UC Berkeley campus with her granddaughter. At around 12 pm, she went to the Starbucks at 2128 Oxford Street, Berkeley, and placed an order for tea. Plaintiff waited for her drink and then walked to the short counter "hand-off plane" to pick up her tea and slipped on coffee that was on the floor adjacent to the hand-off plane. Plaintiff alleges that as she fell she hit the right back of her head on the edge of the counter and then landed on her right side, hitting her elbow and hip.
The coffee was spilled by a male customer as he was picking up his drink from the hand-off plane moments before the plaintiff slipped. The male customer was trying to mop up the spilled coffee on the hand-off plane when he was noticed by the Barista who immediately approached and began to assist the male customer in cleaning up the spill while simultaneously verbally warning the other customers in the area. The Barista could see that some of the coffee had dripped onto the floor and called to his shift manager to take care of the spill in the customer area as the Barista could not access it from behind the bar.
After the Barista believed he had adequately warned all of the customers of the spill in the area of the hand-off plane, he turned to the cold beverage station to complete a drink, and when he turned back he saw the plaintiff being helped up off the floor. Almost simultaneously, the shift manager arrived to clean the spill and began to assist the plaintiff. Plaintiff was asked if she was okay and she said that she was all right but was concerned that there was no wet floor sign.
The Barista estimated that 30 seconds had elapsed from the time he had noticed the spill by the male customer until he saw the plaintiff being helped up off the floor. He also testified that 10 seconds elapsed from the time he turned away from the spill to complete the drink until he saw the plaintiff being helped off the floor. The store's CCTV recorded and confirmed that 90 seconds lapsed from the time that the plaintiff walked out of the camera view after ordering her tea to when the plaintiff is seen leaving the store with her tea in hand and in no apparent physical distress. The location of the slip and fall was not covered by a CCTV camera.
Shortly after leaving the store, Plaintiff returned to make a report and is recorded on the CCTV entering and approaching the cash register and motioning to the shift supervisor to get his attention and pointing to her right forearm and elbow but seemingly in no acute distress. Plaintiff and her daughter then completed an incident report form.
Plaintiff went with her family to the Emergency Department at Alta Bates Hospital in Berkeley and complained that she had struck the back of her head in a fall and had pain in her right elbow and hip among other complaints. The medical records confirmed that both the attending nurse and treating doctor did not observe any signs of trauma to the plaintiff's head. Plaintiff was diagnosed with a head injury, back pain and elbow contusion and instructed to follow-up with her primary care physician back in MA.
On July 6, 2015, the plaintiff was transported to the Emergency Department at a local hospital in MA after feeling dizzy while walking on the treadmill at her gym. The plaintiff was diagnosed with vertigo, nausea and vomiting. The plaintiff was admitted to hospital and remained there for four days. Plaintiff was ultimately diagnosed with vestibular neuritis. Significantly, during the entire time the plaintiff was hospitalized for the vertigo incident, she did not tell any of her treating or consulting doctors about the head injury she allegedly suffered at the Starbucks store in Berkeley five months prior. During the litigation, the plaintiff stipulated that her vestibular neuritis condition was not caused by the fall at Starbucks in Berkeley.
Plaintiff alleged that Starbucks was negligent in that it failed to adequately protect and warn her from the coffee spill that it knew was on the floor. Plaintiff called a retail industry expert, Alex Balian, who testified that the Barista's efforts to verbally warn the customers of the spill were not sufficient and reasonable in the circumstances and therefore constituted negligence. Plaintiff argued that the case was about public safety and holding Starbucks responsible for failing to protect its customers.
Plaintiff argued that she suffered a traumatic brain injury as a result of falling and hitting her head. She claimed that the blow to her head changed her life completely and she was no longer the same person and couldn't enjoy her life and family as she had before the fall. Plaintiff claimed that immediately after hitting her head she had a loud "whooshing" sound in her head that had been with her every minute since the fall. Plaintiff also claimed that she now had a problem with her balance and with walking since the accident. Plaintiff called a medical expert who was certified in "brain injury medicine" and who testified that all of the plaintiff's ongoing symptoms and residuals were as a result of suffering a mild traumatic brain injury in the fall at Starbucks and that the plaintiff's ongoing balance problems were not related to her subsequent vertigo incident.
Starbucks argued that the actions of its Barista in verbally warning the customers about the spill was both adequate and reasonable in the circumstances and met its legal obligation in protecting the plaintiff. Additionally, Starbucks argued that the unknown male customer was a cause of the accident and that the plaintiff should have seen the male customer cleaning up the spill and was therefore aware of the spill prior to slipping and falling. Starbucks called a neurologist who testified that even if the plaintiff hit her head in the fall and suffered a concussion, which by the now accepted definition is a mild traumatic brain injury, any symptoms from it resolved within months and that the plaintiff had no ongoing symptoms related to the fall at Starbucks. Starbucks also called an ENT who testified that if the plaintiff had tinnitus then it was more likely as a result of normal hearing loss. It was significant to him that the plaintiff did not make a subjective complaint of the noise to the initial treating doctors.
Plaintiff asked the jury to award her $650,800 in damages in closing argument. Starbucks counsel and DRI members Paul Caleo and Katrina Durek asked the jury to render a defense verdict. Starbucks served a pre-trial statutory offer of $45,001. The plaintiff's lowest demand pre-trial was $150,000.
After a nine day jury trial in Oakland, California, the jury returned a defense verdict after deliberating for three hours. The jury found that Starbucks was not negligent (9/3) and not negligent in the use and maintenance of its property (9/3). As the prevailing party, Starbucks filed a cost bill of just under $100,000.
Starbucks obtained a significant pre-trial ruling when it successfully excluded the opinion of the plaintiff's retail industry expert, Alex Balian, as to what the industry standard was regarding how to warn customers of a known spill. Mr. Balian testified that the industry standard was that the employee had to physically stand over and become a barrier to the spill and that verbally warning customers was insufficient. Starbucks was successful in excluding this opinion on the basis that this was the witness' personal opinion and not based on any industry literature.
Additionally, Starbucks successfully argued that there was sufficient evidence for the unknown male customer to be on the verdict form as "fault of others" so that it was open to the jury to apportion fault to him as a cause of the plaintiff's fall. Over plaintiff's objections, the verdict form allowed the jury to apportion fault to both the unknown male customer and to the plaintiff herself.
Paul Caleo is a partner at Burnham Brown and one of the firm's premier trial lawyers who has extensive experience in complex tort, personal injury and large loss cases involving claims of products liability, premises liability, government and public entity defense, construction site accidents and trucking/motor carrier accidents. He routinely represents retail corporations of all sizes in a wide variety of cases including wrongful death, serious personal injuries, traumatic brain injury (TBI), loss prevention and retail theft cases, and injuries caused by the criminal acts of third parties, in addition to prosecuting and defending contractual indemnity claims and breach of retail lease contract claims
Katrina R. Durek is an associate at Burnham Brown who counsels and represents businesses of varying sizes, including multi-national retailers, restaurant franchises, and small businesses. Ms. Durek is a trial attorney who handles a variety of matters including premises liability, products liability, contract disputes and employment matters. Ms. Durek represents parties in all phases of litigation, including investigation, law and motion, mediation and trial.
© 2020 Burnham Brown All Rights Reserved.


Here is a super legit recipe to make great and reliable Pizza Dough at home. That specific version makes for 3 large pizzas at the end, with a 70% hydration.
In a big bowl, add 365 grams of water.
Dissolve 16 grams of salt.
Add 50 grams of flour and mix it well.
Add 1 gram of fresh yeast or half a gram of instant yeast.
Gradually add the remaining 470 grams of flour.
Knead the dough for about 20 minutes or until smooth.
Cover and let it rise for 2 hours at room temperature.
Divide it in three parts and form dough balls.
Let them rise in an airtight container for 6 hours.
Bravo you just made my super legit pizza dough.
I'm a huge Home pizza lover. I'll definitely give this a try!
Avec cette recette et un four électrique domestique, à quelle température et combien de temps suggèrerais-tu de laisser cuire une pizza type margherita s'il te plaît ?
Olive oil. It needs some extra virgin olive oil. It won't taste right without it.
I'm also curious about the no oil situation. I must try this! I also like to dust the bottom of the pizza dough with corn flour just before adding toppings and baking.
When is the video for this coming out? Come on!
Also, much respect for adding in an automatic calculator to adjust the values depending on the number of servings, why is this not on all online recipes???
You made my day by giving wonderful blog how to make the pizza base. And Who doesn't love pizza? It's the ultimate comfort food, isn't it?
Could you also add oven temp and bake time ?
I can not help myself, … I read even your website in your French accent in my head :D.
Tank you for this calculator. Makes making pizza tonight much easier :). Btw. I used to make Youtube Videos as well, but since there are 2 Kids and 3 Jobs it is hard to find the time for this passion. You live my dream. Greetings!
i wanted to make 5 pizzas, but allthough i put exactly the amounts from the calculator in the dough it was super liquid and suuuuuuper sticky. what did i do wrong? ive had to add at least 300 gramms of flour to make it a shapeable dough.
Bug Report: When increasing the number of pizzas, the measurement of Instant Yeast remains fixed at "or 0.5g instant yeast" and does not increase like the rest of the ingredients do.
Apart from straight errors in measurement, I can think of one thing: the content moisture in flour might vary. So if the moisture content is high, less water may be needed.


As an instructional designer, I know that the best assessment methods are not always the easiest to grade. Take research papers, which, broken up into an outline, rough draft, and final version, provide 2 distinct opportunities for instructor feedback to correct issues before students submit the final, high-stakes version. But it also triples the amount of grading on the instructor's part (I have been there myself). It seems that with similar assessment methods, there is an inverse relationship between the quality of the student learning experience, and the time it takes to grade.
While there are mitigation strategies such as holistic and single-point rubrics to assist with grading formative assessments, there's nothing quite so attractive as the ubiquitous multiple-choice exam. Many textbook publishers already include them when adopting their textbook, and other than the initial setup of dates, durations, and other settings, they require zero input from the instructor (including grading). There's a reason multiple-choice exams are so popular.
While many online quizzes (especially many of those available as supplemental instructor resources) focus on low-level factual recall, multiple-choice items may be written at the higher application, analysis, synthesis, or evaluation levels. Such items often involve some sort of scenario aimed at promoting learning transfer from one context to another. Additional strategies might require students to view a chart/graph and select the most accurate interpretation from among several alternatives or even to collaborate with classmates in selecting the best justification statement for why a given answer is correct prior to individually submitting their quizzes.
Writing these types of test questions is not easy and will take some effort. But you get two awesome benefits: the ability to have your exams automatically graded (who wants to spend time grading?), and you can gauge students' true understanding of the material, rather than simply what they can cram and memorize the night before.