text:
\section{Introduction}\label{sec:introduction}

\IEEEPARstart{L}{ight} field (LF) cameras record both intensity and direction of light rays, and enable many applications such as refocusing \cite{jayaweera2020multi,wang2018selective}, depth estimation \cite{EPINET,CAE,PS-RF,OACC-Net}, virtual reality \cite{overbeck2018system,yu2017light} and neural rendering \cite{sitzmann2021light,attal2022learning,suhail2022light,wang2022r2l}. Since high-resolution (HR) LF images are beneficial to various applications but are generally obtained at an expensive cost, it is necessary to reconstruct HR LF images from low-resolution (LR) LF images, i.e., to achieve LF image super-resolution (SR).

In the past decade, deep neural networks (DNNs) have been successfully applied to image SR and achieved significant progress \cite{yang2019deep,wang2020survey,li2021survey}. In the area of LF image SR, many networks \cite{LFCNN15,LFCNN17,LFNet,LFSSR,HDDRNet,resLF,ATO,LF-InterNet,LF-DFnet,MEG-Net,ZSLFSR,AFR,DistgLF,DPT,LFT} were developed to achieve continuously improved SR accuracy. However, existing LF image SR methods focus on designing advanced network architectures to fully utilize both spatial and angular information, but ignore the generalization capability to different degradations. In these methods, a single fixed downsampling kernel (e.g., bicubic downsampling) is used to generate LR LF images for training. Since the real-world degradation varies with different imaging devices (e.g., Lytro or RayTrix cameras) and shot conditions (e.g., scene depth, focal length, illuminance), existing LF image SR methods suffer from a notable performance drop on real LF images.

 \begin{figure}
 \centering
 \includegraphics[width=8.5cm]{Figs/Thumbnail.pdf}
 \vspace{-0.1cm}
 \caption{Visual results achieved by DASR \cite{DASR}, DistgSSR \cite{DistgLF} and our method on real LF images for $4\times$SR. Scenes \textit{ISO\_Chart} from the EPFL dataset \cite{EPFL} and \textit{general\_11} from the STFlytro dataset  \cite{STFlytro} are used for comparison.}
 \label{fig:thumbnail}
 \vspace{-0.3cm}
 \end{figure}

Inspired by recent success in single image SR, in this paper, we propose the first method to handle multiple degradations for LF image SR. In our method, we first develop a practical degradation model that can synthesize LR LF images with various blurs and noise levels, and then propose a degradation-adaptive network (LF-DAnet) to achieve LF image SR under multiple degradations. To incorporate the degradation prior into the SR process, we design a degradation-adaptive convolution (DA-Conv) whose weights are dynamically generated according to the degradation representation. By integrating the proposed DA-Conv with the disentangling mechanism \cite{DistgLF}, our LF-DAnet can fully incorporate spatial and angular information while adapting to different degradations. As shown in Fig.~\ref{fig:thumbnail}, compared with DistgSSR \cite{DistgLF} and DASR \cite{DASR}, our method achieves better performance on real LF images and generates images with more clear details and fewer artifacts.

 The contributions of this work are summarized as follows:

 \begin{itemize}
 \item We make the first attempt to handle the real-world LF image SR problem by developing a practical LF degradation model with various blurs and noise levels. By handling this multi-degraded LF image SR task, we address the importance of degradation adaptation capability of DNNs beyond the previous advanced network designs.
 \item We propose an LF-DAnet to deal with multiple degradations for LF image SR. Extensive ablation studies and model analyses validate the effectiveness of our method.
 \item Our method achieves state-of-the-art performance on both synthetic and real-world degradations, which not only provides a simple yet strong baseline for multi-degraded LF image SR, but also takes a step towards practical real-world LF image SR.
\end{itemize}

 The rest of this paper is organized as follows. In Section~\ref{sec:RelatedWork}, we briefly review the related works. In Section~\ref{sec:DegradationModel}, we describe our degradation model for LF image SR. In Section~\ref{sec:network}, we introduce the details and design thoughts of our LF-DAnet. Experimental results are presented in Section \ref{sec:Experiment}. Finally, we conclude this paper in Section \ref{sec:Conclusion}.

\section{Related Work}\label{sec:RelatedWork}
In this section, we briefly review several major works for DNN-based single image SR and LF image SR.

\vspace{-0.2cm}
\subsection{Single Image Super-Resolution}
The goal of single image SR is to reconstruct an HR image from its LR version. According to different degradation settings, existing single image SR methods can be roughly categorized to single degradation based methods and multiple degradation based methods.

Early works on DNN-based single image SR are generally developed on a single and fixed degradation (e.g., bicubic downsampling). Dong et al. \cite{SRCNN14} firstly applied convolution neural networks to image SR and developed a three-layer network named SRCNN. Although SRCNN is shallow and lightweight, it outperforms many traditional SR methods \cite{timofte2013anchored,timofte2014a+,yang2010image,kim2010single}. Since then, deep networks have dominated the SR area and achieved continuously improved accuracy with large models and complex architectures. Kim et al. \cite{VDSR} applied global residual learning strategy to image SR and developed a twenty-layer network called VDSR. Lim et al. \cite{EDSR} proposed an enhanced deep SR (EDSR) network by using both global and local residual connections. Zhang et al. \cite{RDN} combined residual learning with dense connection to build a residual dense network with more than 100 layers. Subsequently, Zhang et al. \cite{RCAN} developed a very deep network in a residual-in-residual architecture to achieve competitive SR accuracy. More recently, attention mechanism \cite{RNAN,DNLN,zhang2021context} and Transformer architectures \cite{SwinIR,lu2022transformer} have been extensively studied to achieve state-of-the-art SR performance.

Although the aforementioned methods have achieved continuously improved SR performance, they are tailored to a single fixed degradation and will suffer from a significant performance drop when the degradation differs from the assumed one. Consequently, many methods have been proposed to achieve image SR with multiple various degradations \cite{blindsr-survey}. Zhang et al. \cite{SRMD} proposed an SRMD network where the degradation map was concatenated with the LR image as the input of the DNN. Subsequently, Xu et al. \cite{UDVD} applied dynamic convolutions to achieve better SR performance than SRMD. In \cite{USRnet}, an unfolding SR network was developed to handle different degradations by alternately solving a data sub-problem and a prior sub-problem. Gu et al. \cite{IKC} proposed an iterative kernel correction method to correct the estimated degradation by observing previous SR results. More recently, Wang et al. \cite{DASR} achieves degradation representation learning in a contrastive manner and developed a degradation-aware SR network (i.e., DASR) to achieve state-of-the-art performance on multi-degraded image SR.

\vspace{-0.3cm}
\subsection{Light Field Image Super-Resolution}
The goal of LF image SR is to super-resolve each SAI of an LF. A straight-forward scheme to achieve LF image SR is applying single image SR methods to each SAI independently. However, this scheme cannot achieve a good performance since the complementary angular information among different views is not considered. Consequently, existing LF image SR methods focus on designing advanced network architectures to fully use both spatial and angular information.

Yoon et al. \cite{LFCNN15} proposed the first DNN-based method called LFCNN to enhance both spatial and angular resolution of an LF. In their method, SAIs are first super-resolved using SRCNN \cite{SRCNN14}, and then finetuned in pairs or quads to incorporate angular information. Wang et al. \cite{LFNet} proposed a bidirectional recurrent network for LF image SR, in which the angular information in adjacent horizontal and vertical views was incorporated in a recurrent manner. Zhang et al. \cite{resLF} proposed a multi-branch residual network to incorporate the multi-directional epipolar geometry prior for LF image SR. In their subsequent work MEG-Net \cite{MEG-Net}, the SR performance was further improved by applying 3D convolutions to SAI stacks of different angular directions. Jin et al. \cite{ATO} developed an all-to-one method for LF image SR, and performed structural consistency regularization to preserve the LF parallax structure. Wang et al. \cite{LF-InterNet} developed an LF-InterNet to repetitively interact spatial and angular information for LF image SR, and then generalized the spatial-angular interaction mechanism to the disentangling mechanism \cite{DistgLF} to achieve state-of-the-art SR accuracy.

More recently, Wang et al. \cite{LF-DFnet} used deformable convolutions \cite{deformable,deformableV2} to address the disparity problem in LF image SR. Cheng et al. \cite{ZSLFSR} proposed a zero-shot learning scheme to handle the domain gap among different LF datasets. Liang et al. \cite{LFT} proposed a Transformer-based LF image SR network, in which a spatial Transformer and an angular Transformer were designed to model long range spatial dependencies and angular correlation, respectively. Wang et al. \cite{DPT} proposed a detail-preserving Transformer to exploit non-local context information and preserve details for LF image SR.

Although remarkable progress have been achieved in LF image SR, existing methods only focus on the advanced network design but ignored the generalization capability to multiple degradations. In this paper, we bridge the gap between single image SR and LF image SR by developing the first LF image SR pipeline with multiple degradations, and investigate the degradation adaption capability of SR networks.

\section{LF Image Degradation Model}\label{sec:DegradationModel}
 In this section, we introduce our LF image degradation model. We use the two-plane model \cite{levoy1996light} to parameterize LFs. That is, without considering the channel dimension, an LF can be formulated as a 4D tensor $\mathcal{L} \in \mathbb{R}^{U\times V \times H\times W}$, where $U$ and $V$ represent angular dimensions, and $H$ and $W$ represent spatial dimensions. Since this paper focuses on enhancing the spatial resolution of an LF, we use the SAI representation in \cite{DistgLF} to describe our method. That is, an LF can be considered as a $U\times V$ array of SAIs, and each SAI has a spatial size of $H\times W$.

 Here, we followed \cite{SRMD,IKC,DASR} to learn an LR-to-HR mapping in the RGB channel space, i.e., the input LR SAI of view $(u, v)$ can be denoted as $\mathcal{I}^{\text{lr}}_{u,v} \in \mathbb{R}^{H\times W\times3}$ and the corresponding HR SAI to be reconstructed can be denoted as $\mathcal{I}^{\text{hr}}_{u,v} \in \mathbb{R}^{\alpha H\times \alpha W\times3}$, where $\alpha$ denotes the upsampling factor.
 Our degradation process can be formulated as
 \begin{equation}\label{eq:degrad}
     \mathcal{I}^{\text{lr}}_{u,v} = (\mathcal{I}^{\text{hr}}_{u,v} \otimes {k}_{u,v}){\downarrow}_{\alpha} + \mathcal{N}_{u,v},
 \end{equation}
 where $k_{u,v} \in \mathbb{R}^{21 \times 21}$ and $\mathcal{N}_{u,v} \in \mathbb{R}^{H\times W\times3}$ represent the blur kernel and additional noise of view $(u,v)$, respectively. $\otimes$ represents the convolution operation, and ${\downarrow}_{\alpha}$ represents downsampling by a factor $\alpha$. In the following text, we introduce the details of the three key components (i.e., blur kernel, noise, and downsamping) of our degradation model, and discuss the motivations behind these designs by comparing our work to existing works.

\vspace{-0.3cm}
 \subsection{Blur Kernel}
 We follow existing works \cite{SRMD,IKC} to use the isotropic Gaussian kernel parameterized by kernel width to synthesize blurring LF images. Note that, although anisotropic kernels (e.g., anisotropic Gaussian blur and motion blur) are also used in recent single image SR methods \cite{DASR,FKP,MADNet,DPSR,USRnet} for degradation modeling, we do not consider these blur kernels in our method because under LF structures, the rotation angle of the anisotropic Gaussian kernel and the trajetory of the motion blur of each SAI should be different but correlated. The formulation of these anisotropic blur kernels depends on the 6D pose changing of LF cameras, and belongs to the LF deblurring task \cite{srinivasan2017light,lee2018joint,mohan2018divide}. As demonstrated in Section~\ref{sec:comparison-real}, based on the isotropic Gaussian blur assumption, our method can achieve promising SR performance on real LF images.

\vspace{-0.3cm}
 \subsection{Noise}
 Real-world LF images (especially those captured by Lytro cameras) generally have large noise. Directly super-resolving noisy LF images without performing noise reduction can result in visually unpleasant artifacts (see Section~\ref{sec:mismatch-real}). In this paper, we consider the simple channel-independent additive white Gaussian noise in our degradation process. Each element in the noise tensor $\mathcal{N} \in \mathbb{R}^{H \times W \times 3}$ is a random variable with a mean value of $0$ and an adjustable standard deviation (i.e., noise level). It is demonstrated in Section~\ref{sec:mismatch-real} that, when the noise term is considered in our degradation model, the super-resolved images are more smooth and clean with less noise residual and ringing artifacts.

\vspace{-0.3cm}
 \subsection{Downsampling}
 We adopt the widely-used bicubic downsampling as the only downsampling approach in our method. In this way, our degradation model can be degeneralized to a standard bicubic downsampling degradation when the kernel width and noise level equal to zero. Note that, different from blur kernel and noise level which can vary in the training phase, the downsampling approach is assumed to be fixed.

 \begin{figure*}[t]
 \centering
 \includegraphics[width=17.5cm]{Figs/DAnet.pdf}

 \caption{An overview of our LF-DAnet.}\label{fig:DAnet}
 \vspace{-0.1cm}
 \end{figure*}

\vspace{-0.1cm}
 \subsection{Discussion}
 \subsubsection{Compared to Existing LF image SR Methods}
  Compared to existing LF image SR methods \cite{resLF,ATO,LF-InterNet,LF-DFnet,MEG-Net,DistgLF,DPT,LFT} which use the bicubic downsampling approach to produce LR LF images, our method adopts a more practical degradation model (i.e., Eq.~\ref{eq:degrad}) since the blur kernel and noise level in our model can be randomly set in the training phase to enlarge the degradation space. It is shown in Section~\ref{sec:comparison-real} that our LF-DAnet trained with this degradation model can achieve a promising SR performance on real LF images, which demonstrates that our proposed degradation model can well cover the real-world degradations of LF images.

 \subsubsection{Compared to More Complex Synthetic Degradations}
  It is also worth noting that several recent works for single image SR \cite{BSRGAN,RealESRGAN} designed very complex degradation models to train deep networks for real-world SR. In these methods, various kinds of blur, noise, and downsampling schemes were considered, and the order of these degradation elements (also including JPEG compression) were randomly shuffled to cover any possible real-world degradations. Although these methods \cite{BSRGAN,RealESRGAN} achieve favorable visual performance on real-world images, we do not consider designing such a complex degradation model in this paper because of the following three reasons. \textbf{\textit{First}}, single images are generally captured by various cameras and transmitted multiple times on internet, and thus go through complex and high-order degradations \cite{RealESRGAN}. In contrast, LF images are captured by a few kinds of imaging devices (e.g., Lytro or RayTrix), and saved to specific file formats that do not go through JPEG compression. Consequently, the degradation space of LF images is smaller than that of single images. \textbf{\textit{Second}}, abundant high-quality HR images and diverse scenarios are required to train a network to fit such complex degradations. Networks in \cite{BSRGAN,RealESRGAN} were trained on multiple large-scale single image datasets \cite{DIV2K,Flickr2K,wang2018recovering,ma2016waterloo} with thousands of high-quality HR images. In contrast, publicly available high-quality LF datasets are limited in amount, spatial resolution and scene diversity. Consequently, it is difficult for an LF image SR network to learn such complex degradations with insufficient training samples. \textbf{\textit{Third}}, as the first work to address LF image SR with multiple degradedations, we aim to demonstrate the importance of degradation adaption capability of SR networks, and propose a simple but effective solution to this problem. Consequently, we do not make our degradation model over-complex.

 \section{Network Architecture}\label{sec:network}

 \subsection{Overview}
 Based on the degradation model in Eq.~\ref{eq:degrad}, we develop a degradation-adaptive network (LF-DAnet) that can super-resolve LF images with various degradations. An overview of our LF-DAnet is shown in Fig.~\ref{fig:DAnet}(a). Given an array of LR SAIs and their corresponding degradations (i.e., kernel width and noise level), our LF-DAnet sequentially performs kernel prior embedding, degradation-adaptive feature extraction, and upsampling. Following \cite{DistgLF}, we build our network by cascading four residual groups. In each residual group, a degradation-adaptive block (DA-Block) is designed to adapt features according to the degradations, and four disentangling blocks (Distg-Blocks) are used to achieve spatial-angular information incorporation. The final output of our network is an array of HR LF images. Note that, since most LF image SR methods \cite{resLF,LFSSR,ATO,LF-InterNet,LF-DFnet,DistgLF,LFT} use SAIs distributed in a square array as their inputs, in this paper, we follow these methods and set $U=V=A$, where $A$ denotes the angular resolution. In the following subsections, we will introduce the details of our network design.

 \subsection{Kernel Prior Embedding}
 Handling image SR with multiple degradations is more challenging than handling that with bicubic downsampling only, since the solution space of the former one is much larger than the latter one. In such case, incorporating kernel priors into the SR process can constrain the solution space to a mainfold and thus reduce the ill-posedness of the SR process \cite{FKP}. Since only isotropic Gaussian kernel (with different kernel widths) is considered in our method, we designed a kernel prior embedding (KPE) module to fully incorporate the kernel prior into the SR process.

 In the KPE module, the isotropic Gaussian kernel $k \in \mathbb{R}^{21 \times 21}$ is first reconstructed according to the input kernel width (i.e., the only undetermined coefficient). The reconstructed kernel is then stretched into a 1D tensor $\mathbf{v}_{k} \in \mathbb{R}^{441 \times 1}$ and fed to a multi-layer perception (MLP) unit with five fully-connected (FC) layers to learn the internal characteristics. The output of the MLP is a compact blur representation with reduced dimensionality, i.e., $\mathbf{v}_{blur} \in \mathbb{R}^{15 \times 1}$. Finally, the generated blur representation is concatenated with the noise level to produce the final degradation representation $\mathbf{v}_{dg} \in \mathbb{R}^{16 \times 1}$. It is demonstrated in Section~\ref{sec:ablation} that the proposed KPE module is beneficial to the SR performance.

 \subsection{Degradation-Adaptive Block}\label{sec:DA-Block}

 Degradation-adaptive block (DA-Block) is designed to adapt image features based on the given degradation. To achieve this goal, a simple and straightforward scheme is to concatenate degradation representation with image features and fuse them via convolutions \cite{SRMD,UDVD}. However, as demonstrated in several recent works \cite{IKC,DASR}, directly convolving image features with degradation representations can cause interference since there is a domain gap between these two kinds of representations. Motivated by the fact that images with different degradations are generated by convolving the original high-quality image using isotropic Gaussian kernel with different kernel widths, in this paper, we design a degradation-adaptive convolution (DA-Conv) whose kernels are dynamically generated according to the input degradation representation.

 Specifically, in each DA-Block, the degradation representation $\mathbf{v}_{dg}$ is firstly fed to two FC layers to produce a convolutional kernel $\mathbf{w}$. Then, the input feature $\mathcal{F}_{input}$ is processed with DA-Conv (using $\mathbf{w}$) and another $1\times 1$ convolution to generate $\mathcal{F}^{spa}_{adapt}$.
 Note that, we follow \cite{DASR} to design our DA-Conv as a depth-wise dynamic convolution, and use a channel attention layer to re-weight the output features based on the statistics of both image feature and degradation representation.  Specifically, the degradation representation $\mathbf{v}_{dg}$ is passed to another two FC layers and a \textit{Sigmoid} activation layer to generate channel-wise modulation coefficients, which are then used to rescale different channels of $\mathcal{F}^{spa}_{adapt}$, resulting in $\mathcal{F}_{adapt}$. Finally, $\mathcal{F}_{adapt}$ is summed up with $\mathcal{F}_{input}$ and $\mathcal{F}^{spa}_{adapt}$ to produce the output of our DA-Block. It is demonstrated in Section~\ref{sec:ablation} that our method benefits from DA-Conv and degradation-adaptive channel attention to achieve flexible adaption to various degradations.

  \begin{figure}[t]
 \centering
 \includegraphics[width=8.5cm]{Figs/DistgBlock.pdf}

 \caption{The architecture of our modified Distg-Block.}\label{fig:DistgBlock}
 \vspace{-0.3cm}
 \end{figure}

 \subsection{Disentangling Block}
 Although the proposed DA-Block can handle input images with multiple degradations, it processes image features of different views separately without considering the inter-view correlation. Since information both within a single view and among different views is beneficial to SR performance, in this paper, we modify the disentangling block (Distg-Block) \cite{DistgLF} to incorporate multi-dimensional information for LF image SR.

 Different from the Distg-Block in \cite{DistgLF} where a series of specifically designed convolutions (i.e., spatial, angular and epipolar feature extractors) are applied to a single macro-pixel image (MacPI) feature, in this paper, we organize LF features into different shapes and apply plain convolutions to the reshaped features. Our modified approach is equivalent to the original design but is more simple and generic. Specifically, considering both batch and channel dimensions, the input feature of our Distg-Block can be denoted by $\mathcal{F}^{6D}_{in} \in \mathbb{R}^{B\times U\times V \times C \times H\times W}$, where $B$, $C$, $H$, $W$ represent batch, channel, height and width, respectively, and $U$$=$$V$$=$$A$ represent angular resolution. As shown in Fig.~\ref{fig:DistgBlock}, our Distg-Block has a spatial branch, an angular branch and two EPI branches (i.e., horizontal and vertical). In each branch, the input feature is reshaped into a 4D feature and then convolved by several 2D convolutions to achieve intra and inter view information incorporation.
In the spatial branch, $\mathcal{F}^{6D}_{in}$ is organized into $\mathcal{F}^{spa}_{in} \in \mathbb{R}^{BUV \times C \times H\times W}$, and two $3\times3$ convolutions (with Leaky ReLU activation) are used to exploit the spatial context information within each view.
In the angular branch, $\mathcal{F}^{6D}_{in}$ is organized into $\mathcal{F}^{ang}_{in} \in \mathbb{R}^{BHW \times C \times U\times V}$, and an $A$$\times$$A$ convolution is used to produce an intermediate feature $\mathcal{F}^{ang}_{temp} \in \mathbb{R}^{BHW \times C \times 1\times 1}$. Then, a 1$\times$1 convolution and 2D pixel shuffling layer are applied to $\mathcal{F}^{ang}_{temp}$ to produce the output. In the angular branch, the inter-view complementary information is incorporated.
In the EPI branch (taking horizontal EPI branch as an example), $\mathcal{F}_{in}$ is reshaped to $\mathcal{F}^{epi}_{in} \in \mathbb{R}^{BHU \times C \times V\times W}$, and a $1$$\times$$A$ convolution is used to produce an intermediate feature $\mathcal{F}^{epi}_{temp} \in \mathbb{R}^{BHU \times C \times 1\times W}$. Afterward, a 1$\times$1 convolution and 1D pixel shuffling layer are applied to $\mathcal{F}^{epi}_{temp}$ to produce the output feature. In the EPI branch, the spatial-angular correlation has been well exploited and the disparity issue has been well handled \cite{DistgLF}.

By adopting the Distg-Blocks, our method can incorporate the beneficial spatial and angular information from the input LF to achieve state-of-the-art SR performance. The effectiveness of the Distg-Block for multi-degraded LF image SR is validated in Section~\ref{sec:ablation}.

\begin{table*}[t]
\renewcommand\arraystretch{1.02}
		\caption{PSNR and SSIM results achieved by different methods on the HCInew \cite{HCInew}, HCIold \cite{HCIold} and STFgantry \cite{STFgantry} datasets under synthetic degradations (with different blur kernel widths and noise levels). Note that, the degradation degenerates to the bicubic downsampling degradation when kernel width and noise level equal to 0. Best results are in \textbf{bold faces}.}\label{tab:quantitative}
		\vspace{-0.2cm}
		\begin{center}
			\footnotesize
			\setlength{\tabcolsep}{1mm}{
				\begin{tabular}{|l|c|ccc|ccc|ccc|}
					\hline
					{Method} & {Kernel} &			\multicolumn{3}{c|}{HCInew}  &
					\multicolumn{3}{c|}{HCIold} & \multicolumn{3}{c|}{STFgantry}
					\tabularnewline
					\hline
                    \hline
					\multicolumn{2}{|l|}{\multirow{1}{*}{Noise level}}
					& $0$ & $15$ & $50$
					& $0$ & $15$ & $50$
					& $0$ & $15$ & $50$
					\tabularnewline
					\hline
                    \hline
					Bicubic & \multirow{8}{*}{$0$}
					& 27.71$/$0.852 & 25.90$/$0.789 & 19.53$/$0.492
					& 32.58$/$0.934 & 28.55$/$0.857 & 20.05$/$0.501
					& 26.09$/$0.845 & 24.68$/$0.789 & 19.18$/$0.516
					\tabularnewline
					DistgSSR \cite{DistgLF} &
					&  {31.38}$/$\textbf{0.922} & 24.88$/$0.722 & 15.59$/$0.284
					& 37.56$/$0.973 & 26.17$/$0.751 & 15.43$/$0.256
					& 31.66$/$0.953 & 24.37$/$0.754 & 15.53$/$0.319
					\tabularnewline
					LFT \cite{LFT} &
					& \textbf{31.43}$/${0.921} & 24.99$/$0.729 & 15.89$/$0.279
					& \textbf{37.63}$/$\textbf{0.974} & 26.48$/$0.765 & 15.96$/$0.258
					& \textbf{31.80}$/$\textbf{0.954} & 24.39$/$0.758 & 15.74$/$0.317
					\tabularnewline
					SRMD \cite{SRMD} &
					& 29.55$/$0.886 & 27.88$/$0.851 & 25.37$/$0.806
					& 35.04$/$0.953 & 31.56$/$0.919 & 28.26$/$0.883
					& 28.85$/$0.911 & 26.73$/$0.869 & 23.60$/$0.795
					\tabularnewline
					DASR \cite{DASR} &
					& 29.31$/$0.886 & 27.78$/$0.852 & 24.10$/$0.785
					& 34.54$/$0.950 & 31.45$/$0.919 & 22.70$/$0.829
					& 26.99$/$0.897 & 26.07$/$0.866 & 21.92$/$0.768
					\tabularnewline
					BSRNet \cite{BSRGAN} &
					& 28.42$/$0.865 & 24.98$/$0.831 & 19.32$/$0.748
					& 32.73$/$0.933 & 28.22$/$0.895 & 17.97$/$0.748
					& 26.55$/$0.880 & 22.55$/$0.829 & 17.46$/$0.714
					\tabularnewline
					Real-ESRNet \cite{RealESRGAN} &
					& 28.05$/$0.862 & 26.99$/$0.839 & 23.65$/$0.789
					& 31.80$/$0.931 & 30.11$/$0.905 & 24.14$/$0.842
					& 24.78$/$0.871 & 24.51$/$0.850 & 19.45$/$0.754
					\tabularnewline
					LF-DAnet (ours) &
					& 30.43$/$0.907 & \textbf{29.55}$/$\textbf{0.886} & \textbf{28.23}$/$\textbf{0.859}
					& 36.44$/$0.967 & \textbf{34.63}$/$\textbf{0.951} & \textbf{32.42}$/$\textbf{0.929}
					& 29.77$/$0.932 & \textbf{28.62}$/$\textbf{0.912} & \textbf{26.99}$/$\textbf{0.878}
					\tabularnewline
					\hline
                    \hline
					Bicubic & \multirow{8}{*}{$1.5$}
					& 27.02$/$0.836 & 25.42$/$0.773 & 19.41$/$0.478
					& 31.63$/$0.923 & 28.16$/$0.846 & 19.99$/$0.491
					& 25.15$/$0.821 & 24.00$/$0.764 & 18.96$/$0.493
					\tabularnewline
					DistgSSR \cite{DistgLF} &
					& 28.60$/$0.876 & 24.46$/$0.699 & 15.60$/$0.273
					& 33.64$/$0.949 & 25.97$/$0.739 & 15.43$/$0.251
					& 27.16$/$0.883 & 23.59$/$0.714 & 15.57$/$0.302
					\tabularnewline
					LFT \cite{LFT} &
					& 28.57$/$0.875 & 24.60$/$0.708 & 15.89$/$0.268
					& 33.62$/$0.949 & 26.25$/$0.753 & 15.96$/$0.252
					& 27.13$/$0.882 & 23.69$/$0.721 & 15.70$/$0.295
					\tabularnewline
					SRMD \cite{SRMD} &
					& 29.58$/$0.886 & 27.39$/$0.840 & 25.01$/$0.798
					& 35.00$/$0.953 & 31.02$/$0.912 & 27.94$/$0.879
					& 28.87$/$0.910 & 26.05$/$0.851 & 23.06$/$0.776
					\tabularnewline
					DASR \cite{DASR} &
					& 29.46$/$0.884 & 27.34$/$0.840 & 24.09$/$0.781
					& 34.87$/$0.952 & 30.95$/$0.911 & 23.44$/$0.831
					& 27.83$/$0.902 & 25.84$/$0.850 & 21.95$/$0.755
					\tabularnewline
					BSRNet \cite{BSRGAN} &
					& 28.38$/$0.861 & 24.79$/$0.824 & 19.36$/$0.746
					& 32.77$/$0.932 & 28.11$/$0.892 & 18.00$/$0.749
					& 26.67$/$0.877 & 22.34$/$0.815 & 17.39$/$0.706
					\tabularnewline
					Real-ESRNet \cite{RealESRGAN} &
					& 28.17$/$0.862 & 26.68$/$0.830 & 23.50$/$0.783
					& 32.11$/$0.932 & 29.85$/$0.900 & 24.13$/$0.840
					& 25.18$/$0.872 & 24.30$/$0.834 & 19.41$/$0.741
					\tabularnewline
					LF-DAnet (ours) &
					& \textbf{30.15}$/$\textbf{0.900} & \textbf{28.98}$/$\textbf{0.872} & \textbf{27.65}$/$\textbf{0.845}
					& \textbf{36.10}$/$\textbf{0.963} & \textbf{33.87}$/$\textbf{0.942} & \textbf{31.81}$/$\textbf{0.920}
					& \textbf{29.47}$/$\textbf{0.924} & \textbf{27.91}$/$\textbf{0.894} & \textbf{26.25}$/$\textbf{0.857}
					\tabularnewline
					\hline
                    \hline
					Bicubic & \multirow{8}{*}{$3$}
					& 25.52$/$0.803 & 24.32$/$0.741 & 19.09$/$0.454
					& 29.59$/$0.898 & 27.12$/$0.822 & 19.82$/$0.476
					& 23.21$/$0.766 & 22.45$/$0.711 & 18.41$/$0.450
					\tabularnewline
					DistgSSR \cite{DistgLF} &
					& 25.79$/$0.811 & 23.30$/$0.656 & 15.47$/$0.254
					& 29.92$/$0.904 & 25.19$/$0.710 & 15.38$/$0.241
					& 23.55$/$0.780 & 21.83$/$0.639 & 15.32$/$0.265
					\tabularnewline
					LFT \cite{LFT} &
					& 25.73$/$0.810 & 23.41$/$0.665 & 15.75$/$0.248
					& 29.83$/$0.904 & 25.42$/$0.724 & 15.91$/$0.242
					& 23.47$/$0.779 & 21.89$/$0.647 & 15.43$/$0.257
					\tabularnewline
					SRMD \cite{SRMD} &
					& 29.20$/$0.876 & 26.32$/$0.816 & 24.30$/$0.782
					& 34.39$/$0.948 & 29.87$/$0.896 & 27.36$/$0.871
					& 28.29$/$0.898 & 24.51$/$0.807 & 22.08$/$0.742
					\tabularnewline
					DASR \cite{DASR} &
					& 28.62$/$0.867 & 26.26$/$0.815 & 23.70$/$0.767
					& 33.72$/$0.942 & 29.82$/$0.896 & 23.75$/$0.829
					& 27.71$/$0.887 & 24.48$/$0.807 & 21.50$/$0.723
					\tabularnewline
					BSRNet \cite{BSRGAN} &
					& 27.60$/$0.843 & 24.13$/$0.804 & 19.33$/$0.740
					& 31.96$/$0.921 & 27.72$/$0.883 & 18.05$/$0.750
					& 26.05$/$0.849 & 21.62$/$0.776 & 17.23$/$0.691
					\tabularnewline
					Real-ESRNet \cite{RealESRGAN} &
					& 27.33$/$0.845 & 25.67$/$0.807 & 23.04$/$0.769
					& 31.45$/$0.919 & 29.11$/$0.889 & 23.93$/$0.835
					& 25.24$/$0.856 & 23.35$/$0.789 & 19.14$/$0.712
					\tabularnewline
					LF-DAnet (ours) &
					& \textbf{29.43}$/$\textbf{0.884} & \textbf{27.76}$/$\textbf{0.845} & \textbf{26.54}$/$\textbf{0.821}
					& \textbf{35.10}$/$\textbf{0.955} & \textbf{32.34}$/$\textbf{0.924} & \textbf{30.54}$/$\textbf{0.904}
					& \textbf{28.51}$/$\textbf{0.904} & \textbf{26.22}$/$\textbf{0.853} & \textbf{24.72}$/$\textbf{0.814}
					\tabularnewline
					\hline
                    \hline
					Bicubic & \multirow{8}{*}{$4.5$}
					& 24.36$/$0.779 & 23.41$/$0.718 & 18.79$/$0.438
					& 28.05$/$0.879 & 26.19$/$0.803 & 19.63$/$0.465
					& 21.80$/$0.725 & 21.26$/$0.672 & 17.90$/$0.420
					\tabularnewline
					DistgSSR \cite{DistgLF} &
					& 24.38$/$0.781 & 22.48$/$0.631 & 15.33$/$0.242
					& 28.08$/$0.880 & 24.50$/$0.690 & 15.31$/$0.235
					& 21.83$/$0.728 & 20.67$/$0.595 & 15.04$/$0.243
					\tabularnewline
					LFT \cite{LFT} &
					& 24.39$/$0.781 & 22.57$/$0.640 & 15.61$/$0.237
					& 28.08$/$0.880 & 24.71$/$0.705 & 15.84$/$0.235
					& 21.84$/$0.728 & 20.72$/$0.602 & 15.14$/$0.233
					\tabularnewline
					SRMD \cite{SRMD} &
					& 26.32$/$0.818 & 25.09$/$0.792 & 23.65$/$0.769
					& 30.62$/$0.908 & 28.61$/$0.882 & 26.66$/$0.864
					& 24.34$/$0.780 & 22.80$/$0.753 & 21.28$/$0.716
					\tabularnewline
					DASR \cite{DASR} &
					& 25.34$/$0.799 & 24.89$/$0.788 & 23.11$/$0.755
					& 29.33$/$0.895 & 28.39$/$0.880 & 23.94$/$0.827
					& 22.99$/$0.761 & 22.65$/$0.749 & 20.76$/$0.697
					\tabularnewline
					BSRNet \cite{BSRGAN} &
					& 26.31$/$0.816 & 23.40$/$0.784 & 19.26$/$0.734
					& 30.35$/$0.902 & 27.10$/$0.874 & 18.03$/$0.749
					& 24.23$/$0.795 & 20.83$/$0.738 & 17.06$/$0.680
					\tabularnewline
					Real-ESRNet \cite{RealESRGAN} &
					& 26.28$/$0.816 & 24.69$/$0.787 & 22.55$/$0.758
					& 30.04$/$0.900 & 28.08$/$0.878 & 23.66$/$0.830
					& 23.97$/$0.810 & 22.17$/$0.743 & 18.90$/$0.693
					\tabularnewline
					LF-DAnet (ours) &
					& \textbf{28.00}$/$\textbf{0.854} & \textbf{26.55}$/$\textbf{0.820} & \textbf{25.58}$/$\textbf{0.801}
					& \textbf{33.39}$/$\textbf{0.937} & \textbf{30.88}$/$\textbf{0.906} & \textbf{29.45}$/$\textbf{0.890}
					& \textbf{26.59}$/$\textbf{0.860} & \textbf{24.64}$/$\textbf{0.808} & \textbf{23.30}$/$\textbf{0.771}
					\tabularnewline
					\hline
			\end{tabular}}
\vspace{-0.05cm}
		\end{center}
\leftline{~~Note: 1) For the methods in \cite{BSRGAN,RealESRGAN}, we used the models trained with a pixel-wise L1 loss (i.e., BSRNet and Real-ESRNet) for comparison since they}
 \leftline{~~can achieve higher PSNR and SSIM values as compared to their GAN-based version (i.e., BSRGAN and Real-ESRGAN). 2) SRMD and our LF-DAnet}
 \leftline{~~are non-blind SR methods while DASR, BSRNet and Real-ESRNet are blind SR methods.}

	\end{table*}

\subsection{Discussion}
 Recent single image SR methods \cite{IKC,DASR,FKP,MADNet} generally adopt the blind SR settings, i.e., the ``groundtruth'' degradation is unknown for the SR networks. That is because, compared to non-blind SR methods \cite{SRMD,UDVD,USRnet,DPSR} where the degradation is also required as the input, blind SR is more practical since the real-world degradation is generally difficult to obtain.

 However, as the first work to address the multi-degraded LF image SR problem, we adopt the non-blind SR settings as in \cite{SRMD}, and take both degraded LF images and their degradations (blur kernel width and noise level) as inputs of our LF-DAnet. Reasons are in three folds.
 \textbf{\textit{First}}, performing non-blind SR helps us to better investigate the impact of input degradations to the SR performance, which has not been studied in LF image SR. Since the kernel width and noise level are independently fed to our LF-DAnet, performing non-blind SR can help us decouple different degradation elements and investigate their influence respectively, as demonstrated in Section~\ref{sec:mismatch}.
 \textbf{\textit{Second}}, performing non-blind SR helps us to explore the upper bound of blind SR. As the first work to handle multi-degraded LF image SR, one of the major contributions of this paper is to break the limitation of single fixed degradation and show the great potential and practical values of multi-degraded LF image SR. To this end, non-blind SR is purer and more suitable than blind SR.
\textbf{\textit{Third}}, since the proposed degradation model has only two under-determined coefficients, we can easily find a proper input degradation by observing the super-resolved images to correct the input degradations, or adopting a grid search strategy \cite{SRMD} to traverse kernel widths and noise levels in a reasonable range, as described in Section~\ref{sec:mismatch-real} .

\begin{figure*}[t]
 \centering
 \includegraphics[width=17.5cm]{Figs/Visual-Synthetic.pdf}
 \vspace{-0.1cm}
 \caption{Visual results achieved by different methods on synthetically degraded LFs ($\textit{kernel width}=1.5$, $\textit{noise level}=15$) for 4$\times$SR. The super-resolved center view images and horizontal EPIs are shown. The PSNR and SSIM scores on the presented scenes are reported below the zoom-in regions.}\label{fig:visual-syn}
 \vspace{-0.1cm}
 \end{figure*}

\section{Experiments}\label{sec:Experiment}
In this section, we first introduce the datasets and implementation details, then compare our network to several state-of-the-art SR methods. Finally, we conduct ablation studies to investigate our design choices and further analyze the impact of the input kernel widths and noise levels.

\subsection{Datasets and Implementation Details}
Our method was trained and validated on synthetically degraded LFs generated according to Eq.~\ref{eq:degrad}, and further tested on real LFs captured by Lytro Illum and Raytrix cameras. For training and validation, three public LF datasets including HCInew \cite{HCInew}, HCIold \cite{HCIold} and STFgantry \cite{STFgantry} were adopted. The division of training and validation set was kept identical to that in \cite{LF-DFnet,DistgLF,LFT,DPT}. To test the generalization capability of our method to real-world degradations,  three public LF datasets (i.e., EPFL \cite{EPFL}, INRIA \cite{INRIA} and STFlytro \cite{STFlytro}) developed with Lytro cameras and a dataset \cite{Raytrix} developed with a Raytrix camera were used as our test sets. Totally 39, 8 and 26 scenes were used for training, validation and test in this paper, respectively.

The LFs in the HCInew \cite{HCInew}, HCIold \cite{HCIold}, STFgantry \cite{STFgantry}, EPFL \cite{EPFL}, INRIA \cite{INRIA} and STFlytro \cite{STFlytro} datasets have an angular resolution of 9$\times$9, and the LFs in the Raytrix dataset \cite{Raytrix} have an angular resolution of 5$\times$5. For LFs with an angular resolution of 9$\times$9, we followed the existing works \cite{LF-InterNet, LF-DFnet, LFT, DPT, DistgLF} to use the central 5$\times$5 SAIs in our experiments. In the training phase, we cropped HR SAIs into patches of size 152$\times$152 with a stride of 32, and used the proposed degradation model to synthesize LR SAI patches of size 38$\times$38\footnote{Following \cite{BSRGAN,RealESRGAN}, we only consider 4$\times$SR in this paper.}. We followed \cite{IKC,DASR} to set the window size of the isotropic Gaussian kernel to 21$\times$21, and followed \cite{SRMD} to randomly sample the kernel width and noise level from range $[0, 4]$ and $[0, 75]$, respectively. Note that, to avoid boundary effect caused by Gaussian filtering, only central 128$\times$128 region of the HR patches and their corresponding 32$\times$32 LR patches were used for training. We performed random horizontal flipping, vertical flipping, 90-degree rotation and RGB channel shuffling to augment the training data by 48 times. Note that, the spatial and angular dimension need to be flipped or rotated jointly to maintain LF structures.

Our network was trained using the L1 loss and optimized using the Adam method \cite{Adam} with $\beta_1$=0.9, $\beta_2$=0.999 and a batch size of 8. Our LF-DAnet was implemented in PyTorch on a PC with two NVidia RTX 2080Ti GPUs. The learning rate was initially set to $2\times10^{-4}$ and decreased by a factor of 0.5 for every 3$\times$10$^4$ iterations. The training was stopped after 10$^5$ iterations.

Following \cite{SRMD,IKC,DASR}, we used PSNR and SSIM calculated on the RGB channel images as quantitative metrics for validation. To obtain the metric score (e.g., PSNR) for a dataset with $M$ scenes (each scene has an angular resolution of $A \times A$), we first calculated the metric on $A\times A$ SAIs on each scene separately, then obtained the score for each scene by averaging its $A^2$ scores, and finally obtained the score for this dataset by averaging the scores of all $M$ scenes.

\subsection{Comparisons with State-of-the-art Methods}
  In this subsection, we compare our method to several state-of-the-art SR methods. Since our LF-DAnet is the first method to handle LF image SR with multiple degradations, we compare it with the following existing methods:
  \begin{itemize}
  \item \textbf{DistgSSR} \cite{DistgLF} and \textbf{LFT} \cite{LFT}: two top-performing LF image SR methods developed on the bicubic downsampling degradation;
  \item \textbf{SRMD} \cite{SRMD}: a popular non-blind single image SR method developed on isotropic Gaussian blur and Gaussian noise degradation;
  \item \textbf{DASR} \cite{DASR}: a state-of-the-art blind single image SR method developed on anisotropic Gaussian blur and Gaussian noise degradation;
  \item \textbf{BSRGAN} \cite{BSRGAN} and \textbf{Real-ESRGAN} \cite{RealESRGAN}: two recent real-world single image SR methods developed on the complex synthetic degradations.
  \end{itemize}
  Besides the aforementioned compared methods, we also include bicubic upsampling method to produce baseline results.

 \begin{figure*}[t]
 \centering
 \includegraphics[width=18cm]{Figs/Visual-Real.pdf}
 \vspace{-0.1cm}
 \caption{Visual results achieved by different methods on real LFs captured by Lytro Illum cameras for 4$\times$SR. Scenes \textit{Hublais} from the INRIA dataset \cite{INRIA} and \textit{general\_11} from the STFlytro dataset \cite{STFlytro} are used as example scenes for comparison. The super-resolved center view images and horizontal EPIs are shown. For SRMD and our method, the input blur kernel width and noise level are set to 2 and 30, respectively. Groundtruth HR images are unavailable in this case.}\label{fig:visual-real}
 \vspace{-0.1cm}
 \end{figure*}

 \begin{figure*}[t]
 \centering
 \includegraphics[width=17.5cm]{Figs/Visual-Raytrix.pdf}
 \vspace{-0.1cm}
 \caption{Visual results achieved by different methods on real LFs captured by a Raytrix camera for 4$\times$SR. Scenes \textit{boxer} from the dataset in \cite{Raytrix} is used as an example scene for comparison. The super-resolved center view images and horizontal EPIs are shown. For SRMD and our method, the input blur kernel width and noise level are set to 4 and 60, respectively. Groundtruth HR images are unavailable in this case.}\label{fig:visual-raytrix}
 \vspace{-0.1cm}
 \end{figure*}

 \subsubsection{Results on Synthetically Degraded LFs}\label{sec:comparison-syn}
  Table~\ref{tab:quantitative} shows the quantitative PSNR and SSIM results achieved by different methods under synthetic degradations with different blur and noise levels. It can be observed that DistgSSR and LFT produce the top-2 highest PSNR and SSIM results under the bicubic downsampling degradation (i.e., $\textit{kernel width}=0$, $\textit{noise level}=0$), but suffer from significant performance drop when the kernel width and noise level are larger than zero. This demonstrates that existing LF image SR methods trained on the noise-free bicubic downsampling degradation cannot generalize well to other degradations.

  SRMD and DASR achieves much better performance than DistgSSR and LFT on blurry and noisy scenes since these two methods are designed for multi-degraded image SR. Note that, SRMD is benefited from the input groundtruth degradation and thus slightly outperforms DASR. It can be also observed that the PSNR and SSIM values produced by BSRNet  and Real-ESRNet are lower than SRMD and DASR. That is because, the degradation space in BSRNet  and Real-ESRNet are much larger, so that the capability of these two methods in handling specific degradations is less powerful. It is worth noting that these single image SR methods only use spatial context information within single views for SR but overlook the correlations among different views, resulting in inferior SR performance and the angular inconsistency issue (see Section~\ref{sec:AngCons}).

 Compared to these state-of-the-art single and LF image SR methods, our LF-DAnet can simultaneously incorporate the complementary angular information and adapt to different degradations, and thus achieves the best PSNR and SSIM results on both in-distribution (\textit{kernel width}$=$$0,1.5,3$) and out-of-distribution (\textit{kernel width}$=$$4.5$) degradations except for the noise-free bicubic downsampling one.
 The benefits of angular information and degradation adaption are further analyzed in Section~\ref{sec:ablation}. Figure~\ref{fig:visual-syn} shows the visual results produced by different methods with blur kernel width and noise level being set to 1.5 and 15, respectively. It can be observed that our LF-DAnet can recover faithful details from the blurry and noisy input LFs.

\subsubsection{Results on Real LFs}\label{sec:comparison-real}
We test the practical values of different SR methods by directly applying them to LFs captured by Lytro and Raytrix cameras. Since the groundtruth HR images of the input LFs are unavailable, we compare the visual results produced by different methods in Figs.~\ref{fig:visual-real} and \ref{fig:visual-raytrix}. It can be observed that the image quality of the input LFs is low since the bicubicly upsampled images are blurry and noisy. DistgSSR and LFT augment the input noise and produce results with artifacts (see Fig.~\ref{fig:visual-real}) or blurring details (see Fig.~\ref{fig:visual-raytrix}). This demonstrates that methods developed on the fixed bicubic downsampling degradation cannot handle real-world degradations and thus have limited practical values.

Although SRMD, DASR, BSRGAN and Real-ESRGAN are specifically designed to handle image SR with multiple degradations, these methods do not consider inter-view correlation and ignore the beneficial angular information. Consequently, when handling the highly ill-posed real-world LF image SR problem, these single image SR methods suffer from noise residual (e.g., DASR), over-smoothness (e.g., SRMD), and color distortion (e.g., Real-ESRGAN) problems.

Compared to existing methods, our LF-DAnet achieves the best SR performance on real LFs, i.e., the results produced by our method have finer details (e.g., the words and characters in scene \textit{general\_11}) and less artifacts. This demonstrates that our LF-DAnet trained on the proposed degradation model can effectively handle real LF image SR problem. Readers are recommended to watch the videos at \url{https://github.com/YingqianWang/LF-DAnet/blob/main/demo_videos.md} to view more visual results of our LF-DAnet on real LFs.

\begin{table}
\caption{Comparisons of the number of parameters (\#Param.), FLOPs and running time for 4$\times$SR. Note that, FLOPs and running time are calculated on an input LF with an angular resolution of 5$\times$5 and a spatial resolution of 32$\times$32. PSNR and SSIM scores are averaged over 9 degradations (\textit{kernel width}$=$(0, 1.5, 3), \textit{noise level}$=$(0, 15, 50)) in Table~\ref{tab:quantitative}. Best results are in \textbf{bold faces}.}\label{tab:efficiency}
\vspace{-0.1cm}
\centering
\scriptsize
\renewcommand\arraystretch{1.1}
\setlength{\tabcolsep}{0.7mm}
\begin{tabular}{|l|c|c|c|c|c|c|}
\hline
  &  \multirow{2}{*}{\#Param.}  & \multirow{2}{*}{FLOPs} & \multirow{2}{*}{Time} & \multicolumn{3}{c|}{PSNR$/$SSIM} \\
  \cline{5-7}
    & & & & HCInew & HCIold & STFgantry \\
\hline
SRMD	 \cite{SRMD}     & 1.50M & 39.76G  & 0.070s & 27.18$/$0.838 & 31.16$/$0.913 & 25.78$/$0.840 \\
DASR	 \cite{DASR}     & 5.80M & 82.03G  & 0.051s & 26.74$/$0.831 & 29.47$/$0.896 & 24.92$/$0.829 \\
BSRNet \cite{BSRGAN}	  & 16.70M & 459.6G  & 0.119s & 24.03$/$0.807 & 26.17$/$0.856 & 21.98$/$0.793 \\
Real-ESRNet	\cite{RealESRGAN}  & 16.70M & 459.6G  & 0.119s & 25.92$/$0.821 & 28.52$/$0.888 & 22.82$/$0.809 \\
DistgSSR \cite{DistgLF} & 3.53M & 65.41G  & \textbf{0.037s} & 22.79$/$0.611 & 24.97$/$0.642 & 22.06$/$0.623 \\
LFT  \cite{LFT}  & \textbf{1.11M} & \textbf{29.45G}  & 0.070s & 22.92$/$0.612 & 25.23$/$0.647 & 22.14$/$0.623 \\
LF-DAnet (ours) & 3.80M & 65.93G  & 0.039s & \textbf{28.75}$/$\textbf{0.869} & \textbf{33.69}$/$\textbf{0.939} & \textbf{27.61}$/$\textbf{0.885} \\
\hline
\end{tabular}
\vspace{-0.2cm}
\end{table}

\subsubsection{Angular Consistency}\label{sec:AngCons}
Since LF image SR methods are required to preserve the LF parallax structure and generate angular-consistent HR LF images, we evaluate the angular consistency of different SR methods by visualizing their EPI slices. As shown below the zoom-in regions in Figs.~\ref{fig:visual-syn}, \ref{fig:visual-real} and \ref{fig:visual-raytrix}, our LF-DAnet can generate more straight and clear line patterns than other SR methods on both synthetic and real-world degradations, which demonstrates that the LF parallax structure is well preserved by our method. Readers can refer to the video at \url{https://wyqdatabase.s3.us-west-1.amazonaws.com/LF-DAnet\_compare.mp4} for a visual comparison of angular consistency.

\begin{table*}
\caption{
PSNR values achieved by LF-DAnet and its variants for $4\times$SR.}\label{tab:ablation}
\centering
\scriptsize
\renewcommand\arraystretch{1.1}
\setlength{\tabcolsep}{1.8mm}
\begin{tabular}{|c|c|c|c|c|c|ccc|ccc|ccc|c|}
\hline
\multirow{2}*{Model} & \multirow{2}*{DA-Conv} & \multirow{2}*{DA-CA} & \multirow{2}*{KPE} & \multirow{2}*{Ang} & \multirow{2}*{\#Params.} & \multicolumn{3}{c|}{Noise = 0} & \multicolumn{3}{c|}{Noise = 15} & \multicolumn{3}{c|}{Noise = 50} & \multirow{2}*{Average} \\
\cline{7-15}
  & & & &  & & $\sigma_b$=0 & $\sigma_b$=1.5 & $\sigma_b$=3 & $\sigma_b$=0 & $\sigma_b$=1.5 & $\sigma_b$=3 & $\sigma_b$=0 & $\sigma_b$=1.5 & $\sigma_b$=3 & \\
\hline
Model 1  & & & & & 3.94M &
 27.87 & 27.72 & 26.97 & 26.40 & 25.71 & 24.21 & 23.40 & 22.85 & 21.87 & 25.22 (-2.39)\\
  \hline
Model 2 & & $\checkmark$ & & $\checkmark$ & 3.77M &
 28.94 & 28.28 & 27.21 & 28.14 & 27.31 & 25.82 & 26.82 & 25.99 & 24.48 & 26.00 (-1.61) \\
  \hline
 Model 3 & $\checkmark$ & $\checkmark$ & $\checkmark$ &  & 4.01M &
 27.95 & 28.00 & 27.49 & 26.43 & 25.79 & 24.35 & 23.43 & 22.92 & 21.90 & 25.36 (-2.25)\\
  \hline
  Model 4 & $\checkmark$ & $\checkmark$ & & $\checkmark$ & 3.77M &
 29.36 & 28.91 & 27.90 & 28.56 & 27.77 & 26.20 & 26.95 & 26.14 & 24.70 & 27.39 (-0.21) \\
   \hline
 Model 5 & $\checkmark$ & & $\checkmark$ & $\checkmark$ & 3.79M &
 29.15 & 29.29 & 28.25 & 28.33 & 27.85 & 26.19 & 26.84 & 26.16 & 24.63 & 27.41 (-0.20) \\
   \hline

\textbf{LF-DAnet} & $\checkmark$ & $\checkmark$ & $\checkmark$ & $\checkmark$ & 3.80M &
 \textbf{29.77} & \textbf{29.47} & \textbf{28.51} & \textbf{28.62} & \textbf{27.91} & \textbf{26.22} & \textbf{26.99} & \textbf{26.25} & \textbf{24.72} & \textbf{27.61 (+0.00)} \\
\hline
\end{tabular}
\end{table*}

\subsubsection{Efficiency}\label{sec:efficiency}
We compare our LF-DAnet to existing SR methods in terms of the number of parameters, FLOPs and running time. As shown in Table~\ref{tab:efficiency}, our LF-DAnet has a moderate model size which is slightly larger than DistgSSR due to the additional kernel prior embedding branch and the degradation-adaptive blocks. Note that, these additional 0.27M parameters only result in a 0.52G and 0.002s increase in FLOPs and running time, respectively. Compared to DASR, BSRNet (i.e., BSRGAN) and Real-ESRNet (i.e., Real-ESRGAN), our method has significantly smaller model size, lower FLOPs and shorter running time. These results demonstrate the efficiency of our method.

\subsection{Ablation Study}\label{sec:ablation}
In this subsection, we investigate the effectiveness of our proposed modules and design choices by comparing our LF-DAnet with the following variants.
\begin{itemize}
\item \textbf{Model 1}: We introduce a baseline model by removing the DA-Block and the angular and EPI branches in the Distg-Block. \textit{Consequently, this variant is equivalent to a plain single image SR network that neither performs degradation adaptation nor incorporates angular information.} Note that, we increase the number of convolution layers in this variant to make its model size not smaller than our LF-DAnet.
\item \textbf{Model 2}: We investigate the effectiveness of our DA-Conv by replacing it with a depth-wise 3$\times$3 convolution and a vanilla 3$\times$3 convolution. Distg-Block is maintained in this variant to incorporate angular information. Note that, the KPE module is also removed since vanilla convolutions do not take degradation as their input. \textit{This variant can be considered as an LF image SR method without degradation adaptation (e.g., DistgSSR) retrained on our proposed degradation model.}
\item \textbf{Model 3}: In this variant, we remove the angular and EPI branches in the Distg-Block and adopt the same strategy as in Model 1 to make the model size of this variant not smaller than our LF-DAnet.  \textit{Since this model only incorporates intra-view information to achieve degradation-adaptive SR, it can be considered as a non-blind single image SR method}, and the benefits of the angular information to multi-degraded LF image SR can be validated.
\item \textbf{Model 4}: We modify the KPE module in this variant to investigate the effectiveness of kernel prior embedding. Specifically, we do not perform isotropic Gaussian kernel reconstruction but directly fed the blur kernel width to a five-layer MLP to generate the blur degradation representation. Consequently, the isotropic Gaussian kernel prior cannot be incorporated by this variant.
\item \textbf{Model 5}: In this variant, we remove the channel attention layer from the DA-Block to investigate the benefits of degradation-adaptive channel-wise modulation.
\end{itemize}

\begin{figure*}[t]
 \centering
 \includegraphics[width=18cm]{Figs/visual-kernel.pdf}
 \vspace{-0.1cm}
 \caption{Kernel visualization of our degradation-adaptive convolutions (DA-Convs) with different input blur and noise levels.}\label{fig:visual-kernel}
 \vspace{-0.2cm}
 \end{figure*}

\begin{figure*}[t]
 \centering
 \includegraphics[width=18cm]{Figs/mismatch.pdf}
 \vspace{-0.1cm}
 \caption{Visualization of the performance variation of our method with mismatched degradations. Sub-figures (a)-(d) show the PSNR values achieved with mismatched blur kernel widths under different noise levels. Sub-figures (e)-(h) show the PSNR values achieved with mismatched noise level under different blurs. Sub-figures (i)-(l) show the PSNR values achieved with simultaneously mismatched blurs and noise levels under four representative degradation settings (marked by white cross). }\label{fig:mismatch}
 \vspace{-0.2cm}
 \end{figure*}

\subsubsection{Degradation-Adaptive Convolution}
 As the core component of our LF-DAnet, DA-Conv can adapt image features to the given degradation and thus enhances the capability to handle different degradations. As shown in Table~\ref{tab:ablation}, without using DA-Conv, Model 2 suffers from a 1.61 dB decrease in average PSNR as compared to LF-DAnet. This is because, different degradations have different spatial characteristics (as analyzed in Section~\ref{sec:DA-Block}) and cannot be well handled via fixed convolution kernels. In contrast, our DA-Conv dynamically generates convolutional kernels conditioned on the input degradation to adaptively recover the degraded image features, and thus achieves higher PSNR values on a wide range of synthetic degradations. Moreover, we visualize the kernels of our DA-Convs with different input blur and noise levels. As shown in Fig.~\ref{fig:visual-kernel}, all the four DA-Convs learn different kernel patterns for different input degradations, and the kernel intensity also varies at different network stages. The above quantitative and visualization results demonstrate the effectiveness of our DA-Conv.

\subsubsection{Angular Information}
 The major difference between our LF-DAnet and non-blind single image SR methods (e.g., SRMD) is the incorporation of the angular information. As shown in Table~\ref{tab:ablation}, when the angular information is not used (i.e., Model 3), the average PSNR value suffers from a 2.25 dB drop. This performance gap is also consistent with the gap between SRMD and our method in Table~\ref{tab:efficiency}. This clearly demonstrates that the complementary inter-view correlation is crucial for multi-degraded LF image SR.

\subsubsection{Kernel Prior Embedding}
 It can be observed in Table~\ref{tab:efficiency} that Model 4 without kernel prior embedding suffers from a 0.21 dB decrease in PSNR as compared to our LF-DAnet. That is because, without KPE, our network has to search for the best degradation kernel to recover the degraded image features. Since we adopt the isotropic Gaussian kernel as the blur kernel for synthetic degradation, KPE can help our network to reduce the searching space and thus facilitates our network to learn more accurate kernel representations.

\subsubsection{Degradation-Adaptive Channel Attention}
 As shown in Table~\ref{tab:efficiency}, when the degradation-adaptive channel attention is removed, Model 5 suffers from a 0.20 dB decrease in average PSNR as compared to LF-DAnet. This demonstrates the effectiveness of degradation-adaptive channel-wise modulation. Since our DA-Conv can only adapt to different degradations in the spatial dimension, DA-CA can be used as a complementary part of DA-Conv to enhance its degradation adaptation capability.
It is also worth noting that our DA-CA only introduces 0.01M increase in model size, and has a negligible influence on the model efficiency.

\begin{figure*}[t]
 \centering
 \includegraphics[width=18cm]{Figs/vary_blur_noise.pdf}

 \caption{Visual results achieved by our method on real LF images with different blur kernel width and noise levels. Three scenes from the EPFL \cite{EPFL}, STFlytro \cite{STFlytro} and Raytrix \cite{Raytrix} datasets are used as examples for illustration.}\label{fig:vary-blur-noise}

 \end{figure*}

\subsection{Degradation Mismatch Analyses}\label{sec:mismatch}
 In this subsection, we first analyze the performance variation of our method with mismatched input and groundtruth synthetic degradations. Then, we apply our LF-DAnet to real LF images and analyze its SR performance with various input blur kernel widths and noise levels.

 \subsubsection{Synthetic Degradation}
 Since our LF-DAnet is a non-blind SR method, it requires to take the blur kernel and noise level as its input. In the aforementioned experiments with synthetic degradations, we directly use the groundtruth degradation as the input degradations of our network. To investigate the performance of our method when the input degradation mismatches with the groundtruth one, we conduct the following experiments. \textbf{\textit{First}}, we investigate the performance variation of our LF-DAnet with mismatched blur kernel widths by traversing the groundtruth kernel width $B_{gt}$ and the input kernel width $B_{in}$ from 0 to 3 with a step of 0.3. Figures.~\ref{fig:mismatch}(a)-\ref{fig:mismatch}(d) visualize the PSNR values achieved by our method (averaged on the validation scenes) under four different noise levels (i.e., $N_{gt}=0, 15, 30, 50$). \textbf{\textit{Second}}, we investigate the performance variation of our LF-DAnet with mismatched noise levels by traversing the groundtruth noise level $N_{gt}$ and the input noise level $N_{in}$ from 0 to 50 with a step of 5. Figures.~\ref{fig:mismatch}(e)-\ref{fig:mismatch}(h) visualize the PSNR values achieved by our method (averaged on the validation scenes) under four different blur kernel widths (i.e., $B_{gt}=0, 1, 2, 3$). \textbf{\textit{Third}}, we investigate the performance variation of our LF-DAnet with simultaneously mismatched blur kernel and noise level by traversing the input blur kernel width $B_{in}$ (from 0 to 3 with a step of 0.3) and the input noise level $N_{in}$ (from 0 to 50 with a step of 5). Figures.~\ref{fig:mismatch}(i)-\ref{fig:mismatch}(l) visualize the PSNR values achieved by our method (averaged on the validation scenes) under four representative degradation settings including $\left(B_{gt}, N_{gt}\right)=(0,0), (3,0), (1.5,15), \text{and} (0,30)$.

 From Fig.~\ref{fig:mismatch}, we can draw the following conclusions:
 \begin{itemize}
    \item Best SR performance can be achieved when the input degradation matches the groundtruth one.
    \item The performance variation caused by blur mismatch is more significant than that caused by noise mismatch.
    \item When $B_{in}\neq B_{gt}$,  $B_{in}>B_{gt}$ can lead to much more significant performance degradation than $B_{in}<B_{gt}$.
    \item As the noise level increases, the PSNR variation caused by the blur kernel mismatch is reduced.
 \end{itemize}

 \subsubsection{Real-World Degradation}\label{sec:mismatch-real}
 To investigate the influence of the input kernel widths and noise levels to the SR performance under real-world degradations, we directly apply our LF-DAnet to the LFs captured by Lytro and Raytrix cameras, and traverse the input blur kernel width (from 0 to 3 with a step of 1) and the input noise level (from 0 to 60 with a step of 15). Since both the groundtruth HR images and their degradations are unavailable, we evaluate the performance of our method by visually comparing its SR results. Figure~\ref{fig:vary-blur-noise} shows the $4\times$ SR results achieved by our method with varied input degradations, from which we can obtain the following conclusions.
  \begin{itemize}
    \item A large input kernel width can enhance the local contrast and sharpens edges and textures,  but an over-large kernel width will introduce ringing artifacts to the result images.
    \item A large input noise level can enhance the local smoothness and helps to alleviate the artifacts, but an over-large input noise level will make the result images blurring.
    \item Our LF-DAnet can achieve better SR performance on Lytro LFs by setting kernel width and noise level to 2 and 30, respectively, and can achieve better SR performance on Raytrix LFs by setting kernel width and noise level to 4 and 60, respectively.
 \end{itemize}
Readers can further refer to our interactive online demo at \url{https://zhengyuliang24.github.io/LF-DAnet-Page} to view the influence of input degradations to the SR results.

\section{Conclusion and Discussion} \label{sec:Conclusion}
 In this paper, we proposed the first work to handle LF image SR with multiple degradations. We developed an LF degradation model that considers both blur and noise at multiple levels, and proposed an LF-DAnet that can adapt to different degradations for LF image SR. Experimental results show that our method not only achieves superior performance under a wide range of synthetic degradations, but also produces visually pleasant and angular consistent SR results on real LF images. Through extensive ablation studies and model analyses, we validated the effectiveness of our designs and obtained a series of insightful observations.

 It is worth noting that, although our LF-DAnet achieves significantly improved performance than existing methods on real-world LF image SR, it is sensitive to the input degradations and requires accurate degradation estimation. When the input blur kernel widths and noise levels mismatch with the real ones, our method will produce images with artifacts or over-smoothness. In the future, we will further study the challenging blind LF image SR problem, and try to design a more practical method for real-world LF image SR. We believe that our LF-DAnet will serve as a fundamental work and can inspire more researchers to focus on real-world LF image SR.

\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


text:
LEAPER (Leveraging endogenous ADAR for programmable editing of RNA) is a genetic engineering technique in molecular biology by which RNA can be edited. The technique relies on engineered strands of RNA to recruit native ADAR enzymes to swap out different compounds in RNA. Developed by researchers at Peking University in 2019, the technique, some have claimed, is more efficient than the CRISPR gene editing technique. Initial studies have claimed that editing efiiciencies of up to 80% can be achieved.

Synopsis

As opposed to DNA gene editing techniques (e.g., using CRISPR-Cas proteins to make modifications directly to a defective gene), LEAPER targets editing messenger RNA (mRNA) for the same gene which is transcribed into a protein. Post-transcriptional RNA modification typically involves the strategy of converting adenosine-to-inosine (A-to-I) since inosine (I) demonstrably mimics guanosine (G) during translation into a protein. A-to-I editing is catalyzed by adenosine deaminase acting on RNA (ADAR) enzymes, whose substrates are double-stranded RNAs. Three human ADAR genes have been identified with ADAR1 (official symbol ADAR) and ADAR2 (ADARB1) proteins developed activity profiles. LEAPER achieves this targeted RNA editing through the use of short engineered ADAR-recruiting RNAs ().  consist of endogenous ADAR1 proteins with several RNA binding domains (RBDs) fused with a peptide, CRISPR-Cas13b protein, and a guide RNA (gRNA) between 100 and 150 nt in length for high editing efficiency designed to recruit the chimeric ADAR protein to a target site.

This results in a change in which protein is synthesized during translation.

History
The technique was discovered by a team of researchers at Peking University in Beijing, China. The discovery was announced in the journal Nature Biotechnology in July 2019.

Applications
Chinese researchers have utilized LEAPER to restore functional enzyme activity in cells from patients with Hurler syndrome. They have claimed that LEAPER could have the potential to treat almost half of all known hereditary disorders.

Highly specific editing efficiencies of up to 80% can be achieved when LEAPER editing using arRNA151 is delivered via a plasmid or viral vector or as a synthetic oligonucleotide, though this efficiency varied significantly across cell types.  Based on these preliminary results, LEAPER may have the most therapeutic promise with no production of functional protein but if a partial restoration of protein expression would provide therapeutic benefit. For example, in human cells with defective α-L-iduronidase (IDUA) expression in cells from patients with IDUA-defective Hurler syndrome, LEAPER resulted in a W53X truncation mutant of p53 being edited using arRNA151 to achieve a "normal" p53 translation and functional p53-mediated transcriptional responses.

Comparison to CRISPR

LEAPER is analogous to CRISPR Cas-13 in that it targets RNA before proteins are synthesized. However, LEAPER is simpler and more efficient as it only requires , rather than Cas and a guide RNA. According to the researchers who developed LEAPER, it has the potential to be easier and more precise than any CRISPR technique.

LEAPER also eliminates health concerns and technical barriers arising from the introduction of exogenous proteins.

It has also been called more ethical as it does not change DNA and thus does not result in heritable changes, unlike methods using CRISPR Cas-9.

See also
 CRISPR gene editing
 Gene knockout
 NgAgo
 Prime editing

References

Genetic engineering
Genome editing
Biotechnology
RNA
Biochemistry

redpajama_set_name: RedPajamaWikipedia


text:
Q: Encrypt in Java Android and Decrypt in C# Unity First off, I want to be able to encrypt some data in one of my Java Class and write it in a text file on the phone. Then my Unity class in c# read it, decrypt it and the data can be used.
At the moment, my java class can encrypt and decrypt his own data. My C# can do the same as well. Problem is, my c# code can't decrypt what java previously encrypted. I'm 100% sure they have the same key (printed a log so compare and it's the same). There seems to be something different between my encryption in java and c#.
Here is the error I'm getting when trying to decrypt in c# what was previously crypted by java:
    03-22 13:32:57.034 14264 14351 E Unity   : CryptographicException: Bad PKCS7 padding. Invalid length 197.
    03-22 13:32:57.034 14264 14351 E Unity   :   at Mono.Security.Cryptography.SymmetricTransform.ThrowBadPaddingException (PaddingMode padding, Int32 length, Int32 position) [0x00000] in <filename unknown>:0
    03-22 13:32:57.034 14264 14351 E Unity   :   at Mono.Security.Cryptography.SymmetricTransform.FinalDecrypt (System.Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount) [0x00000] in <filename unknown>:0
    03-22 13:32:57.034 14264 14351 E Unity   :   at Mono.Security.Cryptography.SymmetricTransform.TransformFinalBlock (System.Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount) [0x00000] in <filename unknown>:0
    03-22 13:32:57.034 14264 14351 E Unity   :   at System.Security.Cryptography.RijndaelManagedTransform.TransformFinalBlock (System.Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount) [0x00000] in <filename unknown>:0
    03-22 13:32:57.034 14264 14351 E Unity   :   at GoogleReader.Decrypt (System.String text) [0x00000] in <filename unknown>:0

JAVA CODE :
public static String key;

public static String Crypt(String text)
{
    try
    {
        // Get the Key
        if(com.UQAC.OceanEmpire.UnityPlayerActivity.myInstance != null){
            key = Base64.encodeToString(com.UQAC.OceanEmpire.UnityPlayerActivity.myInstance.key.getEncoded(),Base64.DEFAULT);
            com.UQAC.OceanEmpire.UnityPlayerActivity.myInstance.SendMessageToUnity(key);
        } else {
            return "";
        }
        Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING");

        byte[] iv = { 1, 2, 3, 4, 5, 6, 6, 5, 4, 3, 2, 1, 7, 7, 7, 7 };
        IvParameterSpec ivspec = new IvParameterSpec(iv);

        Key secretKey = new SecretKeySpec(Base64.decode(key,Base64.DEFAULT), "AES");
        cipher.init(Cipher.ENCRYPT_MODE, secretKey, ivspec);

        return Base64.encodeToString(cipher.doFinal(text.getBytes()),Base64.DEFAULT);
    }
    catch (Exception e)
    {
        System.out.println("[Exception]:"+e.getMessage());
    }
    return null;
}

public static String Decrypt(String text)
{
    try
    {
        Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING");

        byte[] iv = { 1, 2, 3, 4, 5, 6, 6, 5, 4, 3, 2, 1, 7, 7, 7, 7 };
        IvParameterSpec ivspec = new IvParameterSpec(iv);

        Key SecretKey = new SecretKeySpec(Base64.decode(key,Base64.DEFAULT), "AES");
        cipher.init(Cipher.DECRYPT_MODE, SecretKey, ivspec);

        byte DecodedMessage[] = Base64.decode(text, Base64.DEFAULT);
        return new String(cipher.doFinal(DecodedMessage));
    }
    catch (Exception e)
    {
        System.out.println("[Exception]:"+e.getMessage());
    }
    return null;
}

C# CODE :
public static string keyStr;

public static string Decrypt(string text)
    {
        RijndaelManaged aes = new RijndaelManaged();
        aes.BlockSize = 128;
        aes.KeySize = 256;
        aes.Mode = CipherMode.ECB;
        //aes.Padding = PaddingMode.None;

        byte[] keyArr = Convert.FromBase64String(keyStr);
        byte[] KeyArrBytes32Value = new byte[32];
        Array.Copy(keyArr, KeyArrBytes32Value, Math.Min(KeyArrBytes32Value.Length, keyArr.Length));

        byte[] ivArr = { 1, 2, 3, 4, 5, 6, 6, 5, 4, 3, 2, 1, 7, 7, 7, 7 };
        byte[] IVBytes16Value = new byte[16];
        Array.Copy(ivArr, IVBytes16Value, Math.Min(ivArr.Length, IVBytes16Value.Length));

        aes.Key = KeyArrBytes32Value;
        aes.IV = IVBytes16Value;

        ICryptoTransform decrypto = aes.CreateDecryptor();

        byte[] encryptedBytes = Convert.FromBase64CharArray(text.ToCharArray(), 0, text.Length);

        byte[] decryptedData = decrypto.TransformFinalBlock(encryptedBytes, 0, encryptedBytes.Length);

        return Encoding.UTF8.GetString(decryptedData);
    }

    public static string Encrypt(string text)
    {
        RijndaelManaged aes = new RijndaelManaged();
        aes.BlockSize = 128;
        aes.KeySize = 256;
        aes.Mode = CipherMode.ECB;

        byte[] keyArr = Convert.FromBase64String(keyStr);
        byte[] KeyArrBytes32Value = new byte[32];
        Array.Copy(keyArr, KeyArrBytes32Value, Math.Min(KeyArrBytes32Value.Length, keyArr.Length));

        byte[] ivArr = { 1, 2, 3, 4, 5, 6, 6, 5, 4, 3, 2, 1, 7, 7, 7, 7 };
        byte[] IVBytes16Value = new byte[16];
        Array.Copy(ivArr, IVBytes16Value, Math.Min(ivArr.Length, IVBytes16Value.Length));

        aes.Key = KeyArrBytes32Value;
        aes.IV = IVBytes16Value;

        ICryptoTransform encrypto = aes.CreateEncryptor();

        byte[] plainTextByte = Encoding.UTF8.GetBytes(text);
        byte[] CipherText = encrypto.TransformFinalBlock(plainTextByte, 0, plainTextByte.Length);
        return Convert.ToBase64String(CipherText);
    }

Test to encrypt and decrypt a string in C#
Original String : Hello World
Crypted : 7Zmd4yvxgR6Mg0nUDQumBA==
Decrypted : Hello World

Test to encrypt and decrypt a string in Java
Original String : Hello World
Crypted : zQjSpJqU8YkHhMDHw8wuTQ==
Decrypted : Hello World

The Key during those test :
FuVf/CNYkHdyBqejq3eoHQ==

As you can see, they have the same key but encrypted the data differently. I can't seem to figure out what I'm doing wrong.

A: Found the solution ! Everything works.
After following some hints, here is the fix:

*

*Made sure everything was in ECB. In the Java insted of having this:
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING");
I simply put this in both crypt and decrypt functions :
Cipher cipher = Cipher.getInstance("AES");

*In C#, the key size and IV size need to depend on the actual original Key and the IV array (not an arbitrary number like 32 or 16, need to be the actualy size of the key)

*Insted of writing a string to the file, I now write directly the byte as they are directly after I encrypted them. This made it easier for the C# side to just straight up get everything and decrypt it.
So the major problem was to write strings in the file insted of bytes. I don't know why this caused an error (error : invalid block size).
Warning: Using ECB mode is insecure, in some circumstances, it is trivial to retrieve the plaintext.  Don't use this code in production or in any scenario where security is required.

A: Your C# code is in ECB mode but your Java code is in CBC mode.
Did you actually write the code you have or just copy and paste it?  Looks like the latter.  Don't copy and paste security code.  You should be writing it with intent and understanding.  Not "oh look here I'll throw this together and hope it works".
Also, the way you handle your IV is very insecure.  Never use a fixed IV.  Under the right circumstances it can completely reveal your plaintext.  Use a randomly generated IV for every encryption operation.

redpajama_set_name: RedPajamaStackExchange


text:
The Xometry app works best with JavaScript enabled!
Injection molding done your way:Great prices & engineering support on simple and complex jobs
CapabilitiesIndustriesResourcesSuppliesBecome a Partner
Xometry stores cookies on your computer to provide more personalized services to you, both on this website and through other media. By using this website, you consent to the cookies we use and our Privacy Policy.
BlogMachining and Fabrication
CNC Machining Trends in 2020
While 2019 delivered challenges to CNC machining shops, 2020 holds much potential for technological advancement. Turnkey parts, complex multi-axis manufacturing, smaller batch sizes, competitive lead times, and an embrace of the web are among the top trends shaping CNC machine shops for 2020 and beyond.
By Greg Paulsen
Answering the Most Frequently Asked Questions About the Xometry Advance Card
4.5 Ways to Protect Your Employees on the Shop Floor From COVID-19
Manufacturing Tools of the Trade: Machines to Buy in 2020
Custom manufacturing in the United States has been under significant pressure to grow over the last several years. Although the pricing of equipment overall has been more accessible, the development of a skilled workforce has been slower. Traditionally, the machinist workforce is the lifeblood of a shop, where generations of workers build a legacy through talent and lifetime clients. However, with global competition and demand for lower prices, there has been a strain to retain talent while also achieving customer goals. In spite of these challenges—or perhaps because of them—the market is growing and advanced manufacturing is on the cusp of a renaissance.
Much in the same way that Eliyahu Goldratt's book, The Goal: A Process of Ongoing Improvement, discusses, the custom manufacturing industry has innovated in ways to eliminate bottlenecks, increase throughput, and, of course, make money. What do these innovations look like? Below are some key points of advancement, which meet modern needs while maintaining a bottom line. Customers demand turnkey parts, complex multi-axis manufacturing, smaller batch sizes, competitive lead times, and of course, an embrace of the web. Here are the top trends shaping CNC machine shops for 2020 and beyond.
Enhanced Machining Equipment
Machines Are a Platform
Very often, the machine in the shop is seen as a means to an end: a program goes in, the material is fixtured, tools cut per computer numeric controlled pathing, the process is repeated in different operations, the part is sent to inspection, and the next project begins. The truth is that a CNC machine is a platform that can streamline each one of these steps through integrations both physically and digitally. By treating a modern machine as a platform, different features can be augmented and upgraded on both the physical side, such as integrated tool measurement, and the software side, adding high levels of customization and performance monitoring. The following advancements work to enhance CNC milling by saving time, reducing setups, and sometimes catching an error before the operator says "oops!"
CAM is the Machines' Digital Twin
Computer-aided manufacturing (CAM) software, like MasterCam, is used to program the manufacturing of a part by inserting a 3D computer-aided design (CAD) model into a virtual setup. Toolsets are added, and each machining operation is programmed and simulated. Machine tool pathing is often as simple as defining the orientation and machine parameters and letting the software compute the optimized runs. New tool pathing software can lead to double wins by reducing cutting times and mitigating wear on the endmill.
Over the last several years, CAM software has been elevated by incorporating features like advanced collision detection as standard. In the past, this software could be prohibitively expensive for a shop to license as an independent add-on. With multi-axis machining becoming more commonplace, the risk of collision increases significantly. Improved CAM software now mitigates this risk as well as improves part machining itself by taking advantage of simulation. These advancements in CAM are bridging years of workforce training by building in would-be tribal knowledge into the software.
Microservices have also helped augment CAM software in the machining industry. The company Provencut, for example, has built a platform that allows machinists to access tried-and-true "recipes," or instructions that incorporate the machine platform, tool specifications, and the optimum pathing. These snippets can be downloaded on demand and added to other code to construct a machining operation with little risk. With the reduced risk to the cutting, machinists can produce more parts with less downtime.
Provencut has built a library of CNC recipes that effectively allows for drag-and-drop programming for part features. (Source: Provencut)
Methods like model-based definition (MBD) and part modeling information (PMI) are also becoming more powerful for communicating design intent without the requirement of a 2D drawing. Information like tolerances, GD&T, surface finishes, and materials can be embedded in the digital model. With increased integration between CAD and CAM, MBD will become an industry standard to increase communication and reduce risk in custom manufacturing. Software like Autodesk'sFusion 360has seamlessly integrated CAD and CAM, allowing shop floor operators and engineers to work within the same platform.\
Machines that Train the Workforce
Haas Automation, as well as other equipment manufacturers, are installing 3D CAD viewers, and even CAD/CAM packages within their machine platforms. The display on the machine controller has progressively become larger, becoming the primary interface as well as a visual guide for setups and operation progression. To address errors during machining, Haas has pre-installed troubleshooting videos directly on its machine for instant operator access.
Haas machining centers have controllers which can contain instructional images and videos to mitigate mistakes, train operators, and reduce downtime. (Source: Haascnc.com)
Integrating intelligent software like CAM and other tools within traditional machining hardware controllers help offset workforce training and development challenges. Although this equipment can sometimes be more expensive, with recurring licenses for the embedded software, it can significantly increase what an operator can do as well as mitigate risks from operator error. This leads to efficiencies like increasing throughput and decreasing workforce barriers. Amada's press brakes for sheet metal include built-in tools and feedback to ensure operators are using the right dies and tools per operation through a digital control interface.
To put the cherry on top, newer operating platforms, like those for CNC machines, lathes, brakes, and cutters, have been upgraded with faster processing speed. This increased speed allows for more seamless inputs and feedback for operators.
More Axes To Reduce Machining Cost
Beyond software and CAD/CAM integrations, there have also been improvements to machining hardware and equipment, making them more capable and more affordable than ever before. Multi-axis machining centers such as 4- or 5-axis have entered competitive marketplaces, enabling ownership or use of high-quality machines at a lower price. These machines can be dedicated, with pre-installed rotary axes, or upgraded using interchangeable rotary tables or trunnions. For example, a 3-axis machining center can be upgraded with 5-axis capabilities by adding a trunnion that adds tilt and rotation of the workpiece. Automated tool changers can also come as a stock item or upgrade, with increases in the count of tool holders allowing for more operations with fewer setups due to tool capacity.
The benefit of adding axes to a machining center is a reduction in setups. For each setup, custom fixturing may require additional operations and materials. With the reduction of setups, the labor per part is decreased. This reduces both cost and risk as measured by the number of steps per operation which could have operator error and, therefore, increased scrap. Improvements in machining CAM software have further empowered 5th axis machining centers by optimizing guiding curves for tool pathing, improving material removal using live contouring. Utilizing additional axes is also becoming common on other platforms beyond mills, such as wire EDM and laser, plasma, and waterjet cutting to create more features in fewer operations.
A live tooling lathe with a 4+ axis machine, allowing for multiple operations to be completed beyond axial/concentric features, often completing full parts in a single operation. (Source: Xometry)
Better Tooling and Fixturing
Purchasing Drills, Mills, Inserts and more through eCommerce Sites
Like a value-added reseller (VAR) for CAD software, machine tooling representatives often add significant insight and application expertise when identifying what tooling to purchase to match the material, feature sets, and machine capabilities. Over the past several years, and moving into 2020, online resources have become available to augment some of this expertise with eCommerce platforms. Tool supply websites like Xometry Supplies offer direct-from-manufacturer selections and pricing with simple searching and filtering. This can include raw material, like 6061-T6 aluminum, as well as a variety of drills, mills, and indexable tools with inserts. Adding eCommerce to a traditionally consultation-based buying experience reduces bottlenecks in sourcing and procurement, particularly for reorders and stocking.
eCommerce options for difficult-to-source tooling have become popular resources for shop purchases. (Source: Xometry Supplies)
Other resources for online sourcing and purchasing of tools include industrial vending machines like AutoCrib and MSC Direct. These reduce lost time due to tool breakage and re-order. These machines are cabinets that are typically connected to the internet and can track the inventory of items. Based on the needs of the machine shop, these cabinets can automatically re-order stock of commodity bits and tools once a minimum quantity is reached. The combination of vending platforms with accessible tooling, regardless of the configuration, empowers CNC machine shops to find the right tools for the job.
Universal Fixturing on the Rise
Vacuum fixtures are typically used only as they're required for projects that are difficult to secure—or easy to deform—with a standard vise. A vacuum fixture, or vacuum plate, includes an embedded valve where a hose can draw suction and a grooved surface accompanied by gaskets or O-rings to create even negative pressure around a part or blank. This pressure can hold the part in place even its when being machined. Workholding suppliers like Pierson have created universal bases called SmartVac that can be custom configured for multiple operations without a custom build per part. Vacuum fixtures like these are extremely powerful for initial machining operations since they offer a turnkey alternative to a vise. Many of these universal base styles can also be easily augmented by adding custom top plates for repeat work at a fraction of the cost versus making a full fixture from scratch.
A universal vacuum plate can often live on a workcenter full-time and serve multiple operations without part-specific custom fixtures (Source: NYCCNC).
Another workholding solution comes from Matrix Innovations, who has created universal soft jaws. These specialized sets of jaws have pins that adjust individually to contours and can be locked once in place for repeated operations. Both of these solutions, vacuum tables and reusable soft jaws, reduce the parts needed to be custom manufactured per operation, allowing the machinist to focus on value-added processes that can be delivered and invoiced.
Robots and Cobots
Robots have worked on factory floors in various ways for decades, and typically have best served automation for high-volume production. This can include using robots to move parts in process or control end effectors which can weld, rivet, paint, etc. Once programmed, an industrial robot works autonomously and is caged for safety in the workspace.
Collaborative robots, better known as cobots, have been more prevalent in smaller shop floors in the last few years, and are rising in popularity moving through 2020. The difference between an industrial robot and a cobot is that cobots are designed to operate alongside human workers. Cobots also typically have more intuitive means of programming, including being "taught" their intended operation through manual movements. With cobot units, such as those provided by Universal Robots, an operator can train a cobot by holding and adjusting its arm and end effector through its intended operation, which the cobot will remember and repeat. By simplifying the programming, a cobot can be versatile, with applications for both production as well as non-production environments like job shops and high-mix manufacturing environments.
The rise of cobots in CNC machine shops can make workers more productive without significant infrastructural changes and intensive training. This technology, along with improvements in tooling options and fixturing, can improve daily production output while reducing human error, which then improves overall shop deliverables.
A collaborative robot, or cobot, can work alongside machinists and operators to increase productivity. (Source: Universal Robots)
Sensors, Feedback Loops, and the IIOT
The book, The Goal, stresses the necessity of identifying the primary bottleneck in operation and determining a way to eliminate it or increase its throughput. In the novel, the bottleneck is easily identified by identifying where production parts are stacking up in the process. Once identified, another machine is introduced in that process step to run parallel jobs and better streamline the workflow.
This approach is useful for production, but with higher demands in custom low-volume manufacturing, it is often difficult to truly understand critical bottlenecks without good data. This is where novel sensors, feedback systems, and predictive software help bridge these awareness gaps by aggregating data over various jobs to separate signals from noise on the shop floor.
Machine Monitoring Combines Hardware and Software
Manufacturing, even in small shops, is adopting the industrial internet of things, or IIOT, which utilizes a software dashboard to aggregate and report on data coming from various operations. MTConnect has built a series of hardware integrations for machines that monitor and output machine data, which can be used to schedule work more effectively. MTConnect's hardware is installed on machining centers or can use integrated sensors, which are becoming more standard with new equipment.
The company Amper has a novel alternative to MTConnect's retrofits. Amper has an IIOT device that wraps around the shop equipment's power cable and monitors electric current. It then uses artificial intelligence (AI) and machine learning to build a signature library for that machine. These signatures could be when this machine is idle or when it moving, and when it is cutting material.
Once this data is gathered, trends can be identified by floor managers and company leads. This information, along with software-augmented suggestions, can help better apply resources to operations that strategically increase overall output. More importantly, the effect of shop floor decisions can be measured and improved upon.
Another important trend in machine monitoring is the improvement of preventative maintenance, which means using predictive models and data to ensure downtime is scheduled and not a surprise. This is historically difficult without IIOT-enabled feedback loops, such as in a generational shop where machines are from various OEMs and were purchased over time.
Tool Presetters and In-Machine Measurement
Tool presetters measure the cutting faces of a tool and ensure the machine program's assumption of tool length and diameter is accurate to reality. Presetters can be stand-alone units or integrated within the machine platform. The benefit of having a presetter installed within a CNC machine is that the tool can automatically check tools during changes. For example, a presetter may identify broken or worn endmills and drills during a change and prevent upstream rejections. Tool presetters can provide feedback and in-situ correction for most new machining CAM software and platforms.
Presetters, combined with machine probe kits like those from Renishaw, significantly mitigate risks during operation. These sensors provide concrete, in-situ feedback for what the CAM program has predicted digitally. Vertical and horizontal machining units with CMM touch-trigger probes can double-check manual indication of part setups and provide minor corrections to maintain repeatable, consistent quality over multiple cycles. Probes can also provide accurate measurements of features, and savvy operators will utilize them to check for features during multi-operation jobs to ensure the part and program are matched appropriately. This use is extremely valuable for shops that have separate programmers and operators over multiple shifts to reduce scrap.
While in-situ IIOT feedback loops are finding a common home in larger shops, even smaller machine shops are now considering digital probes, wifi connections, and embedded machine monitoring a standard for their equipment. In 2020 and beyond, more integration will happen across digital and physical machining platforms to build better decision-making tools for high-throughput machining businesses.
Specialized CMM probes are safe for machine tool changers and the shop floor to assist accurate indication, in-process inspection, and operational checks. (Source: Renishaw)
3D Printing Is Helping Machining Businesses
There is a high chance that a 3D printed part or tool has found its way to your shop floor. 3D printing, also known as additive manufacturing, allows for custom shapes to be created with only a CAD file. Due to a lack of setup requirements, and a robust list of materials and processes available for printing, 3D printers have become assistive technologies for machining production.
The Right Parts for Custom Needs
3D printed plastic parts are typically not as strong as machined components. However, they have a distinct advantage of being able to hold complex shapes and geometries with little associated costs and lead times. A common use for 3D prints in shops is custom fixturing. For example, a complex part made on a live tooling lathe may be difficult to hold and index for operations like installing threaded inserts, pressing in pins, or aligning for laser marking. With 3D printed jigs and fixtures, an operator can easily create cradle-like structures to hold work consistently in the place of performance, ensuring repeatability in a product. 3D printed components also commonly act as drill guides, edge protectors, and other tools which can benefit from custom, purpose-designed geometries. Because making a 3D printed part does not occupy a CNC machine, it is an ideal parallel process for machine shop production for specific tools.
Depending on the application, 3D printed components can also be deliverable units for prototyping or low-volume production. Fused deposition modeling (FDM or FFF) can be easily sourced via online 3D printing services or built in-house using desktop 3D printers like Ultimaker or Markforged platforms. FDM creates parts by melting a filament of plastic, such as ABS or PC, and extruding it in the shape of the provided CAD design. Other industrial offerings like selective laser sintering (SLS), multi jet fusion (HP MJF), direct metal laser sintering (DMLS), and digital light synthesis (Carbon DLS) offer more robust end-use application materials ranging from polymers to metals.
A 3D printable jig (blue) designed to hold a part in place (translucent gray) for consistent marking. (Source: Xometry)
Manufacturing as a Service and Online Marketplaces
Many of the tools and trends previously described work on the guidance of eliminating bottlenecks, reducing labor and operations, and enhancing the workforce through better software. All of these solutions aim to increase quality deliverables per day as they are processed through the machine shop. However, there is a trend which is a consistent challenge amongst small business manufacturing companies: securing work.
Connecting buyers with machining suppliers can be a challenge due to mismatches with capabilities, capacity, and location. 2020 is a defining year for Manufacturing as a Service (Maas). MaaS platforms like Xometry act as both a storefront for buyers as well as a matchmaker for manufacturing shops. For the buyers, a MaaS platform provides the ease of a single vendor, allowing multiple manufacturing projects to be channeled through a digital supply chain manager. Work on a MaaS platform is distributed to capable manufacturing partners, who can secure it on demand. In many ways, MaaS is the gig economy response to custom manufacturing. Due to the aggregate power of a MaaS platform, CNC machine shops and other suppliers can use it both as a primary business channel or an ad-hoc capacity filler. A MaaS service benefits buyers with its large, elastic capacity that leverages the strengths of each manufacturer on its network. More machine shops are joining MaaS networks and going through steps to onboard as a manufacturing partner to gain access to online job boards.
MaaS platforms add significant benefit to suppliers as well, by complementing in-house services and extending shop capabilities without investment. Machine shops have the ability to off-load work to qualified manufacturers without losing days to RFQs and negotiations. Complementing services in the MaaS ecosystem include tooling and material providers like Xometry Supplies and finishing services consolidate and simplify resource procurement on active jobs, regardless of if it was internal or accepted through a MaaS job board.
The Future of Machining in 2020 and Beyond
Many of the CNC machining trends for 2020 and beyond are the increasing prevalence of tools and technologies that help shops adapt to a highly competitive, global manufacturing marketplace. The buyers, procurers, engineers, and supply chain managers who are purchasing manufactured components have an objective to secure quality suppliers at low costs and with timely deliveries. With the evolution of the internet, all systems are becoming interconnected, from the digital probes monitoring workpieces to MaaS platforms that bring in new work.
Machine shops are building more flexible workflows, with quicker responses to lower volume work and tighter manufacturing deadlines on production projects. Internal processes and planning software is improving throughput, increasing daily revenue and discovering more capacity even on legacy equipment. In a market that increasingly demands shorter lead times and cheaper prices, novel new technologies, including those in additive manufacturing, are equipping the machining workforce with the necessary tools to take on the challenges of a global marketplace.
Greg Paulsen
They call me the Director of Application Engineering at Xometry. This means I not only get to produce great design-for-manufacturing content, but also consult on a variety of custom manufacturing projects using CNC machining, additive manufacturing, sheet metal, urethane casting, and injection molding. If you have a question, I'm your guy.
Xometry Go Green
7529 Standish Place
7940 Cessna Avenue
Suite 300 East
446 East High Street
800 Winchester Rd
117 Wright Industrial Cove
Ada-Lovelace-Str. 9
85521 Ottobrunn, Germany
+49 32 22 109 8186
www.xometry.de
LBS OF C02 OFFSET
© 2021 Xometry, All Rights Reserved

redpajama_set_name: RedPajamaCommonCrawl