text:
\section{Introduction} \label{section:introduction}

Salient object detection (SOD) imitates the human visual attention mechanism to search attentive areas or objects from a scene and precisely segment them. As a basic computer vision task, SOD has been applied in many other tasks such as object tracking~\cite{tracking}, image caption~\cite{Imagecaption}, video object segmentation~\cite{MSAPS} and human matting~\cite{SIM}.

In recent years, with the development of depth sensors, the RGB-D SOD task has become a very striking branch in SOD. Depth map has illumination invariance and color independence, which provides RGB image with important geometrical and discriminative information.
Most existing RGB-D SOD methods are depth-based, that is, take an RGB image and a depth map as inputs. They usually adopt a two-stream structure~\cite{BBSNet,HDFNet,UCNet,CMWNet,RD3D,SPNet} and mainly focus on how to effectively conduct the cross-modal fusion. Different from them, the DANet~\cite{DANet} designs a single stream network for efficiency. Either the two-stream methods or the single-stream one all use the depth map as input and deeply involves it into the feature learning process. As we all know, depth maps acquired by active sensors often carry different types of spatial artifacts such as Gaussian noise and hole pixels and show a checkerboard pattern due to the low distance resolution, as shown in Fig.~\ref{fig:STERE_depth_surfacenorm}. Thus, the depth-based SOD methods all inevitably face this serious interference problem. Even if a high-quality depth sensor may reduce the noise, as a single-modality device, it still needs to cooperate with other devices to handle complex scenes.

\begin{figure*}
	\includegraphics[width=\textwidth]{data/DASNet_CoNet_Ours.pdf}\\

	\centering
	\setlength{\abovecaptionskip}{-8pt}
	\caption{Comparison of three types of depth-free networks.}
	\label{fig:three_types_depth_free}

\end{figure*}

To this end, we rethink the RGB-D SOD problem.
We utilize the multi-task learning (MTL) framework to implement a depth-free model, which can refine the original and coarse depth information to assist salient object detection. In particular, three complementary tasks of depth estimation, saliency and contour prediction are designed and their mutual learning achieves attention focusing and eavesdropping.
The MTL network contains three types of modality features (depth-wise, saliency-wise, and contour-wise). How to effectively integrate them will directly affect the final predictions of these three tasks.
Motivated by the transformer~\cite{transformer}, which can capture global contextual information and establish the long-range dependence,
we put forward a multi-modal filtered transformer (MFT) module to drive each modality stream to draw complementary information from other modalities.
Specifically, we equip with a multi-modal transformer unit at the highest-level layer.
Then, the proposed modality-specific filters (MSFs) are embedded in the transformer unit, which can filter out the task-aware features from the aggregated multi-modal transformer feature.
The MTL network equipped with the MFT can predict a high-quality depth map with fine and smooth distribution, as shown in Fig.~\ref{fig:STERE_depth_surfacenorm}, which is of greater benefit to saliency detection compared to the original depth map.

Our main contributions can be summarized as follows:
\begin{itemize}[noitemsep,nolistsep]
	\item We develop a depth-free network based on multi-task learning framework for the RGB-D SOD task. Through sufficient feature interaction among different tasks, their predictions all are significantly improved.
	\item We design a multi-modal filtered transformer module, which embeds the modality-specific filters to achieve the multi-modal feature fusion from the perspective of attention-in-attention.
	\item Extensive experiments on six benchmark datasets demonstrate that the proposed model performs favorably against the state-of-the-art depth-based and depth-free RGB-D SOD methods under five metrics.
	\item The predicted depth maps have higher quality than the original ones, which can be applied to previous top-performing depth-based RGB-D SOD methods to improve their performance.
\end{itemize}

\section{Related Work} \label{section:related_work}

\subsection{RGB-D Salient Object Detection}
According to whether the depth map is required during the testing phase, we classify existing RGB-D SOD methods into two types: depth-based and depth-free one.
\\
\textbf{Depth-based Methods.} In terms of the number of encoding streams, depth-based methods can be further divided into two-stream~\cite{PCA,CPFP,S2MA,CMWNet,PGAR,BBSNet,UCNet} and single-stream~\cite{DANet} one.  PCANet~\cite{PCA} is the first to  concentrate on the cross-modal fusion between two streams. Some later works~\cite{CPFP,ICNet,DMRA,CMWNet} leverage the depth cues to generate spatial attention or channel attention to enhance the RGB features. Sun \textit{et al.}~\cite{DSA2F} design a new NAS-based model for the heterogeneous feature fusion in RGB-D SOD and Zhou \textit{et al.}~\cite{SPNet} propose  a  novel specificity-preserving  network  to  explore  the  shared  information  between  multi-modal data.
These two-stream designs achieve very good performance.
Recently, Zhao \textit{et al.}~\cite{DANet} combine depth map and RGB image from starting to build a single-stream network.
\\
\textbf{Depth-free Methods.} To the best of my knowledge, there are three depth-free models. Inspired by knowledge distillation, A2DELE~\cite{A2DELE} first transfers the depth knowledge from the depth stream to the RGB stream in the training phase and then only employ the RGB stream for saliency prediction when testing. DASNet~\cite{DASNet} utilizes depth information as an error-weighted map to correct the saliency prediction. As shown in Fig.~\ref{fig:three_types_depth_free} (a), it focuses on the single one task (SOD), the depth branch is just to serve the SOD branch. The most related work is CoNet~\cite{CoNet}. As shown in Fig.~\ref{fig:three_types_depth_free} (b), it first builds three independent branches to generate saliency, depth and edge features under the corresponding supervisions, and then distills all knowledge into the saliency head through multi-modal feature fusion. Different from them, as shown in Fig.~\ref{fig:three_types_depth_free} (c), we apply multi-task learning and deeply achieve cross-modal interaction to jointly train three task branches until the output. In this way, the depth branch can continuously capture useful multi-scale features from the other two branches to refine the depth information. While CoNet~\cite{CoNet} adopts the prediction-to-interaction strategy, which does not implement cross-modal interaction before predicting depth map and salient contour. Thus, its saliency
result is very easily corrupted with the noise of the depth modality.
\begin{figure*}
	\includegraphics[width=\textwidth]{data/Network.pdf}\\

	\centering
	\setlength{\abovecaptionskip}{-8pt}
	\caption{Network pipeline.  It consists of a encoder, five multi-modal decoder  blocks (MDBs). The multi-modal filtered transformer module is only embedded in the MDB-5. We supervise the side-out of each task branch on multiple levels.}
	\label{fig:pipeline}

\end{figure*}

\subsection{Multi-task Learning}
Multi-task learning (MTL) aims to solve multiple tasks by a shared model, which can improve the performance of each individual task via four implicit advantages: the implicit data augmentation, attention focusing, eavesdropping and regularization.
In the context of deep Learning, MTL is typically done with two strategies: hard~\cite{Hard_parameter_sharing} and soft~\cite{soft_parameter_sharing} parameter sharing of hidden layers. Hard parameter sharing is the most commonly used approach in neural networks~\cite{Hard_parameter_sharing_1,Hard_parameter_sharing_2,Hard_parameter_sharing_3}, which usually shares the hidden layers among all tasks, while keeping several task-specific output layers.
In this paper, we also adopt this strategy.
The three tasks share the multi-scale feature extractor, and the task-specific hidden layers in the decoder mutually interweave to achieve attention focusing and regularization under the supervision of ground truth, which guarantees the complementary cues to be fully mined and utilized, thereby making the three tasks complement and reinforce each other.
\subsection{Transformer in Computer Vision}
Transformer was first proposed in ~\cite{transformer}. Due to its outstanding performance, it has revolutionized machine translation and natural language processing.  Recently, DETR~\cite{DETR} applies a transformer network for object detection, which greatly simplifies the traditional detection pipeline and shows excellent performance. And then, more and more transformer models are applied to image classification~\cite{transformer_classification}, object detection~\cite{transformer_detection}, semantic segmentation~\cite{transformer_ss}, object tracking~\cite{transformer_tracking}, video instance segmentation~\cite{transformer_vis}, etc. ViT~\cite{ViT} introduces the transformer for image recognition and models an image as a sequence of patches, which attains excellent results compared to the state-of-the-art CNN. The above works show the effectiveness of the transformer in computer vision.  However, previous works mainly focus on a single modality. How to utilize the transformer technique to achieve multi-modal fusion is what we are interested in.

\section{The Proposed Method}
\subsection{Overall Architecture}
As shown in Fig.~\ref{fig:pipeline}, the network  consists of five encoder blocks, five multi-modal decoder blocks (MDB) and a multi-modal filtered transformer (MFT) module. The encoder-decoder architecture is based on the FPN~\cite{FPN}. The encoder is built on top of a common backbone network, e.g.,  VGG~\cite{VGG}, ResNet~\cite{Resnet} or Res2Net~\cite{Res2Net}, to perform feature extraction of  RGB image. Each MDB is used for feature communication among multiple tasks at the corresponding level. There are two reasons to equip with the MFT only on the highest-level features: (1) These features have the lowest spatial resolution, which is helpful to reduce the amount of calculations. (2) As suggested in~\cite{high_level_attention}, stand-alone attention layers can better integrate global information that usually exists in the high-level layer.

\subsection{Multi-modal Decoder Block}\label{sec:MDB}
To effectively accomplish multi-task learning, it is crucial to cooperatively utilize multi-modal information to enhance each modality branch.  As shown in Fig.~\ref{fig:pipeline}, the basic multi-modal decoder block adopts an information aggregation and distribution structure, dubbed as  \textit{Squeeze-and-Expand}. For each decoder level, we first complete the modality transfer by using the  ground truth of depth, saliency and contour to separately supervise their own feature maps. Next, we conduct  the multi-modal fusion through the \textit{Squeeze} operation. In this paper, except for the MDB-5, the other four decoder blocks directly adopt an element-wise addition layer and a convolution layer to achieve  feature fusion, as follows:
\begin{equation}\label{equ:1}
\begin{split}
F_{DSC}^{i} = Conv(F_{D}^{i} \oplus F_{S}^{i} \oplus  F_{C}^{i}), \quad i \in \left \{1, 2, 3, 4 \right \},
\end{split}
\end{equation}
where $\oplus$ is the element-wise addition and $Conv(\cdot)$ denotes the convolution layer. $F_{DSC}^{i}$ contains the depth-wise, saliency-wise and contour-wise information.  And then, we separately combine $F_{DSC}^{i}$ with each modality feature through different convolution layer, coined as the \textit{Expand} operation.
In fact, the \textit{Squeeze-and-Expand} can be regarded as a multi-modal residual structure.
We apply the proposed  \textit{Squeeze-and-Expand}  in each MDB to achieve the interaction between different tasks, which can effectively integrate multi-task information and then enhance the characteristics of each task. For this reason, we adopt the form of residual fusion, that is, multi-task aggregation features are used as the residual branch of each task feature to provide complementary information. Because the supervision is enforced in each MDB for each task, the supplementary features of the \textit{Squeeze-and-Expand} structure are adaptive for different tasks.  Finally, the enhanced features ($F_{enD}^{i}$, $F_{enS}^{i}$, $F_{enC}^{i}$)  combine the RGB encoder features to participate the following multi-modal decoder block.
\begin{figure*}
	\includegraphics[width=\textwidth]{data/Attention-in-Attention-v3.pdf}\\

	\centering
	\setlength{\abovecaptionskip}{-8pt}
	\caption{Illustration of the multi-modal filtered transformer module.}
	\label{fig:mmft}

\end{figure*}
\subsection{Multi-modal Filtered Transformer}
In the MDB-5, we design a multi-modal filtered transformer (MFT) module to better perceive the global information. As shown in Fig.~\ref{fig:mmft}, it includes the multi-head self-attention, feedforward network and modality-specific filter.
\\
\textbf{Multi-modal Transformer.}
Self-Attention (SA) is a fundamental component in standard transformer.  Given queries $q$, keys $k$ and values $v$, the attention function is written as:
\begin{equation}\label{equ:2}
\begin{split}
SA(q,k,v) = \mathcal{S}((\frac{{q}{k}^\top}{\sqrt{d_k}}){v}),
\end{split}
\end{equation}
where $d_{k}$ is the dimension of $k$ and $\mathcal{S}$ is softmax function.
The multi-head self-attention (MHSA) is formulated as:
\begin{equation}
\label{equ:3}
{MHSA}(q,k,v) = \mathcal{C}(T_{1},...,T_{nh}){\mathbf{W}^O},
\end{equation}
\begin{equation}
\label{equ:4}
T_{i} = SA({q}\mathbf{W}_i^q,{k}\mathbf{W}_i^k,{v}\mathbf{W}_i^v),
\end{equation}
where $\mathcal{C}$ is the concatenation operation along channel axis, $\mathbf{W}_i^Q \in \mathbb{R}^{d_m \times d_k}$, $\mathbf{W}_i^K \in \mathbb{R}^{d_m \times d_k}$, $\mathbf{W}_i^V \in \mathbb{R}^{d_m \times d_v}$, and $\mathbf{W}^O \in \mathbb{R}^{n_h\times d_v \times d_m}$ are parameter matrices.
Please refer to the literature~\cite{transformer} for more details.
In this work, we set $nh$ = 8, $d_{m}$ = 384 and
$d_{k}$ = $d_{v}$ = $d_{m}$/$nh$ = 12.

Another important unit in transformer is the feedforward network (FFN) module, which is used to enhance the fitting ability of the model. It consists of two linear transformation with a ReLU in between. In the multi-modal transformer, we first aggregate and reshape $F_{D}^{5}$, $F_{S}^{5}$, $F_{C}^{5}$ $ \in \mathbb{R}^{C \times H \times W}$ as the transformer input:
\begin{equation}
\label{equ:5}
T_{in} = \mathcal{R}_{1}(\mathcal{C}(F_{D}^{5},F_{S}^{5},F_{C}^{5})),
\end{equation}
where $\mathcal{R}_{1}$ reshapes the input feature map to $\mathbb{R}^{C \times N}$. $N = {H \times W}$ is the number of features.  Next,  $T_{in}$ is fed into MHSA and FFN in order, and then the transformer-based multi-modal fusion feature (TMFF) is generated as follows:
\begin{equation}
\label{equ:6}
MHSA_{out} = T_{in} + MHSA(T_{in} +P_{x},T_{in} +P_{x},T_{in}),
\end{equation}
\begin{equation}
\label{equ:7}
TMFF = \mathcal{R}_{2}(MHSA_{out} + FFN(MHSA_{out})),
\end{equation}
where $\mathcal{R}_{2}$ reshapes the input feature maps to $\mathbb{R}^{C \times H \times W}$ and $P_{x}$ is the spatial positional encoding generated in the same way as DETR~\cite{DETR} to distinguish the position information of the input feature sequence. Since the transformer has many encoder layers, it will dilute the modality-specific information to directly convey TMFF to subsequent layers. To solve this problem, we embed a modality-specific filter to select the useful features.
\\
\textbf{Modality-Specific Filter.} We follow the spatially-unshared and intragroup-shared principles to design the filter.
On the one hand, the differences among the predictions of three tasks are mainly reflected in the location properties of the pixel level.
Therefore, we adopt the dynamic kernel to capture the position-specific information, that is, the filter is spatially unshared.
On the other hand, the high-level features have rich semantic information in different channels. In order to maintain semantic diversity, we adopt the channel-unshared strategy.
However, to save the number of parameters, we refer to the idea of group convolution~\cite{ResNext} and adopt a grouping mechanism to achieve the intragroup-shared but intergroup-unshared filtering. The filtering process can be formulated as:
\begin{equation}
\label{equ:8}
Y_{c_{i},h,w} = \sum_{u,v}F_{\lceil \frac{c_{i}}{C_{g}} \rceil,h,w,u+\lfloor \frac{K_{h}}{2} \rfloor,v+\lfloor \frac{K_{w}}{2} \rfloor}X_{c_{i},h+u,w+v},
\end{equation}
where $X\in \mathbb{R}^{C_{i} \times H \times W}$ and $Y\in \mathbb{R}^{C_{o} \times H \times W}$ are the input and output features, respectively.  $F\in \mathbb{R}^{G \times H \times W \times K_{h} \times K_{w}}$ denotes the dynamic filters, which are yielded by feeding $X$ to two $1$ $\times$ $1$ vanilla convolutional layers. $K_{h} \times K_{w}$ represents the size of the convolution kernel. In this work, we use $3 \times 3$ kernel, and set $G=8$ and $C_{g}=C_{i}//G$.
For convenience, the offset values relative to the center of the convolution kernel are indexed as $(u,v)\in\Delta_{K}=[-\lfloor \frac{K_{h}}{2}\rfloor,...,\lfloor \frac{K_{h}}{2}\rfloor]\times[-\lfloor \frac{K_{w}}{2}\rfloor,...,\lfloor \frac{K_{w}}{2}\rfloor]\in\mathbb{Z}^{2}$, where $\times$ is the Cartesian product and  $\Delta_{K}$ forms a set of two-dimensional coordinate offset.
\\
\textbf{Attention-in-Attention.}
In fact, an \textit{Attention-in-Attention} mechanism is implemented by combining the multi-modal transformer and the modality-specific filter.
The self-attention establishes the intrinsic correlation of queries $Q$, keys $K$ and values $V$, which provides a tight information exchange site for an isolated source of the input feature $F$. The frequently cascaded self-attention may gradually weaken the representation capability of features due to the possible accumulation of modality mismatch.
For this reason, we build an additional attention barrier outside the internal attention, which takes the output $V_{m-int},m\in\{D, S, C\}$ of the internal attention as the input of the external attention and then utilizes  $F_m$ to generate attention weight $Q_{m-ext} \cdot K_{m-ext}$. The external attention establishes the secondary relationship among $Q$, $K$ and  $V$ through the filtering process. Thus, the internal $V_{m-int}$ is filtered and its enhanced part $V_{m-ext}$ is obtained under the bridging guidance of the external attention. The external-internal attention is actually an attention-in-attention structure. The correspondence between the multi-modal filtered transformer and the \textit{Attention-in-Attention} is shown in Fig.~\ref{fig:mmft}.

\subsection{Supervision}
The total training loss can be written as:
\begin{equation}\label{equ:9}
\begin{split}
L= L_{d}+L_{s}+L_{c}.
\end{split}
\end{equation}
For the depth estimation, we follow the related works~\cite{depth3,depth4,depth5} to adopt the combination loss of the L1 and SSIM~\cite{SSIM}.
For the saliency prediction, we use the weighted IoU loss and binary cross entropy (BCE) loss which have been widely adopted in segmentation tasks. We use the same definitions as in~\cite{SPNet,F3Net,PraNet,MSNet}.
For the contour prediction, we only adopt the BCE loss.
\begin{align}
L_{d}= & \sum_{n=1}^{N}(l_{ssim}^{(n)}+l_{1}^{(n)})\\
L_{s}= &\sum_{n=1}^{N}(l_{bce}^{w(n)} + l_{iou}^{w(n)})
\\
L_{c}= &\sum_{n=1}^{N} l_{bce}^{(n)}
\end{align}

As described in Sec.~\ref{sec:MDB}, our model is deeply supervised on five groups of outputs, i.e. $N$ = $5$.
Besides, for the salient contour estimation, its ground truth ($G_{c}$) is calculated based on the ground-truth saliency map ($G_{s}$). Specifically, we employ the morphological dilation and erosion as follows:
\begin{equation}\label{equ:5}
\begin{split}
G_{c}= D_{m}(G_{s})-E_{m}(G_{s}),
\end{split}
\end{equation}
where $D(\cdot)$ and $E(\cdot)$ are the dilation and erosion operations, respectively. $m$ denotes a full one filter of size $m\times m$ for erosion and expansion, which is set to $3$ in this paper.

\section{Experiments}
\subsection{Datasets}\label{sec:datasets_sec}
We evaluate the proposed model on six public RGB-D SOD datasets, which are \emph{DUTLF-D}~\cite{DMRA}, \emph{STERE}~\cite{STERE}, \emph{NJUD}~\cite{NJU2000}, \emph{NLPR}~\cite{early_fusion_1}, \emph{SIP}~\cite{SIP} and \emph{LFSD}~\cite{LFSD}. For comprehensive and fair evaluation, we follow the setting as most previous methods~\cite{SPNet,HDFNet,BBSNet,CMWNet,JLDCF,CoNet,A2DELE}. On the DUTLF-D, we  use  $800$  samples  for  training  and  $400$ samples for  testing. For  the  other  five  datasets,  we use  $1,485$ samples from the NJUD and $700$ samples from the NLPR as the training set and the remaining samples in these datasets are used for testing. Besides, we quantitatively compare the predicted depth map of our method and  CoNet~\cite{CoNet} and other two state-of-the-art depth estimation methods~\cite{TL-Depth,LAP-Depth} on the SIP. This dataset consists of high-resolution RGB color images and depth maps which are designed to contain multiple salient persons per image. The  high-quality camera makes sure that the depth map from the SIP has less noise.

\subsection{Evaluation Metrics}
For the salient object detection task, we adopt several widely used metrics for quantitative evaluation: F-measure ($F_{\beta}^{max}$)~\cite{colorcontrast_Fm}, the weighted F-measure ($F_{\beta}^{w}$)~\cite{Fwb}, mean absolute error ($\mathcal{M}$), the recently released S-measure ($S_{m}$)~\cite{S-m} and E-measure ($E_{m}$)~\cite{Em}. The lower value is better for the $\mathcal{M}$ and the higher is better for others. For the depth estimation task, we use following metrics used  by previous works: Root Mean Square Error (RMSE, RMSE (log)), absolute relative error (AbsRel), squared relative error (SqRel) and depth accuracy at various thresholds $1.25$, $1.25^{2}$ and $1.25^{3}$.
\subsection{Implementation Details}\label{sec:implementationdetails}
Our model is implemented based on the Pytorch and trained on one RTX 3090 GPU for $50$ epochs with mini-batch size $12$.  The input resolutions of RGB images are resized to $352\times352$. We adopt some image augmentation techniques to avoid overfitting, including random flipping,  rotating, and border clipping. For the optimizer, we use the Adam~\cite{Adam}. For the learning rate, initial learning rate  is  set  to $0.0001$. We adopt the ``step'' learning rate decay policy, and set the decay size as $30$ and decay rate as $0.9$. Our code will be made publicly available soon.
\subsection{Comparisons with State-of-the-art}
\subsubsection{Quantitative Evaluation}
~\\
\noindent{\textbf{\textit{Saliency Evaluation.}}}
Since previous RGB-D SOD methods adopt VGG-16, ResNet-50, Res2Net-50 or ResNet-101 as the backbone. For a fair comparison, we separately evaluate the MMFT with different backbones.
Tab.~\ref{tab:RGBDSOD_performance} shows performance comparisons in terms of five metrics. We divide these models into depth-based and depth-free. Our  method  outperforms  all  of  the  competitors and obtains the best performance in terms of all metrics on the DUTLF-D, STERE and SIP datasets. It is worth noting that our model achieves significant improvement against the second best depth-free (CoNet~\cite{CoNet}) and depth-based (SPNet~\cite{SPNet}) models.
Besides, we also conduct an extension for RGB SOD task. We use the MMFT, which is trained on RGB-D SOD datasets, to predict depth maps for the images in RGB SOD datasets, and then follow most RGB SOD methods~\cite{SGL-KRN,Auto-MSFNet,SAMNet,VST,F3Net,GCPA,MINet,ITSD,GateNet} to re-train the MMFT on the DUTS-TR~\cite{DUTS} dataset for RGB SOD task. That is, the predicted depth map is used to supervise the depth branch. Tab.~\ref{tab:rgbsod_performance} shows the results in terms of five metrics on five RGB SOD datasets. Our method performs favorably against these competitors. Especially, it achieves the   best performance on the challenging DUT-OMORN~\cite{DUT-OMRON} dataset.
\\
\textbf{\textit{Saliency Contour Evaluation.}} We evaluate salient contour in terms of two main metrics: $F_{\beta}^{max}$ and $\mathcal{M}$ in Tab.~\ref{tab:Salient_contour_performance}. It can be seen that our method surpasses these competitors on all five datasets. Their contours are extracted from the corresponding saliency maps by morphological processing.
\\
\begin{table*}
	\scriptsize
	\setlength{\abovecaptionskip}{2pt}
	\caption{Quantitative comparison of different RGB-D SOD methods. $\uparrow$ and $ \downarrow$ indicate that the larger scores and the smaller ones  are better, respectively. The best three scores are highlighted in {\color{reda} \textbf{red}}, {\color{mygreen} \textbf{green}} and {\color{myblue} \textbf{blue}}.
		The subscript in each model name is the publication year.  The corresponding used backbone network is listed below each model name.}
	\renewcommand\tabcolsep{1pt}
	\renewcommand\arraystretch{1.3}
	\centering
	\resizebox{\textwidth}{!}

	{
		\begin{tabular}{cc||cccccccccccccccc||ccc>{\columncolor{mygray}}c>{\columncolor{mygray}}c>{\columncolor{mygray}}c>{\columncolor{mygray}}c}
			\toprule[2pt]
			\multicolumn{2}{l||}{} & \multicolumn{16}{c||}{\textbf{{Depth-based Models}}}  & \multicolumn{7}{c}{\textbf{{Depth-free Models}}} \\
			\hline
			\multicolumn{2}{c||}{\multirow{3}{*}{Metric}}   &{S2MA$_{20}$} & {CMWNet$_{20}$} & {BBSNet$_{20}$} & {BBSNet$_{20}$} &{JL-DCF$_{20}$}  & {UCNet$_{20}$} & {PGAR$_{20}$} &{HDFNet$_{20}$}
			&{HDFNet$_{20}$}
			&{CDNet$_{21}$} &{DFMNet$_{21}$} &{DSA2F$_{21}$} &{DCF$_{21}$}&{HAINet$_{21}$} &   {RD3D$_{21}$} &   {SPNet$_{21}$} & {A2DELE$_{20}$}&{DASNet$_{20}$}&{CoNet$_{20}$} & {MMFT} &{MMFT} & {MMFT} & {MMFT}
			\\
			\multicolumn{2}{l||}{}   & ~\cite{S2MA}&~\cite{CMWNet}&~\cite{BBSNet}&~\cite{BBSNet}&~\cite{JLDCF}&~\cite{UCNet}&~\cite{PGAR}&~\cite{HDFNet}&~\cite{HDFNet}&~\cite{CDNet}&~\cite{DFMNet}&~\cite{DSA2F}&~\cite{DCF}&~\cite{HAINet}&~\cite{RD3D}&~\cite{SPNet}&~\cite{A2DELE}&~\cite{DASNet}&~\cite{CoNet}&\cellcolor{mygray}&\cellcolor{mygray}&\cellcolor{mygray}&\cellcolor{mygray}
			\\
			\multicolumn{2}{l||}{}   & VGG-16  & VGG-16& VGG-16& ResNet-50&ResNet-101& VGG-16& VGG-16&  VGG-16& ResNet-50& VGG-16 &ResNet-34& VGG-19& ResNet-50& VGG-16& ResNet-50& Res2Net-50&  VGG-16& ResNet-50&ResNet-101& VGG-16& ResNet-50& Res2Net-50& ResNet-101
			\\

			\hline

			\hline
			\multirow{5}{*}{\emph{\rotatebox{90}{DUTLF-D}}}
			&$F_{\beta}^{max}\uparrow$
			& 0.909
			& 0.905
			& -
			& -
			& 0.924
			& -
			& 0.938
			& 0.926
			& 0.930
			& 0.901
			& -
			& 0.938
			& 0.941
			& 0.932
			& 0.946
			& -
			& 0.907
			& -
			& 0.935
			& 0.950
			& \color{myblue} \textbf{0.955}
			& \color{reda} \textbf{0.958}
			& \color{mygreen} \textbf{0.956}

			\\
			&$F_{\beta}^{w}\uparrow$
			&  0.862
			&0.831
			&-
			&-
			&0.863
			&-
			&0.889
			&0.865
			&0.864
			&0.838
			&-
			&0.908
			&0.909
			&0.883
			&0.909
			&-
			&0.864
			& -
			&0.891
			&0.909
			& \color{myblue} \textbf{0.920}
			& \color{reda} \textbf{0.926}
			& \color{mygreen} \textbf{0.924}
			\\
			& $S_m\uparrow$
			& 0.903
			&0.887
			&-
			&-
			&0.905
			&-
			&0.920
			&0.905
			&0.907
			&0.886
			&-
			&0.921
			&0.924
			&0.909
			&0.931
			&-
			&0.886
			& -
			&0.919
			&0.934
			& \color{myblue} \textbf{0.941}
			& \color{mygreen} \textbf{0.942}
			& \color{reda} \textbf{0.943}

			\\
			& $E_m\uparrow$    &
			0.921
			&0.922
			&-
			&-
			&0.938
			&-
			&0.950
			&0.938
			&0.938
			&0.917
			&-
			&0.956
			&0.957
			&0.939
			&0.957
			&-
			&0.929
			& -
			&0.952
			& \color{myblue} \textbf{0.959}
			& \color{mygreen} \textbf{0.967}
			& \color{mygreen} \textbf{0.967}
			& \color{reda} \textbf{0.968}
			\\
			&$\mathcal{M}\downarrow$  &   0.044
			& 0.056
			& -
			& -
			& 0.043
			& -
			& 0.035
			& 0.040
			& 0.041
			& 0.051
			& -
			& 0.031
			& 0.030
			& 0.038
			& 0.031
			& -
			& 0.043
			& -
			& 0.033
			& 0.029
			& \color{myblue} \textbf{0.027}
			& \color{mygreen} \textbf{0.025}
			& \color{reda} \textbf{0.024}
			\\

			\hline
			\multirow{5}{*}{\emph{\rotatebox{90}{STERE}}}
			&$F_{\beta}^{max}\uparrow$   &
			0.895
			& 0.911
			& 0.901
			& 0.919
			& 0.913
			& 0.908
			& 0.911
			& 0.918
			& 0.910
			& 0.909
			& 0.914
			& 0.910
			& 0.915
			& 0.919
			& 0.917
			& 0.915
			& 0.892
			& 0.915
			& 0.909
			& \color{myblue} \textbf{0.931}
			&  \color{mygreen} \textbf{0.933 }
			&  \color{reda} \textbf{0.935}
			&  \color{blue} \textbf{0.931}

			\\
			&$F_{\beta}^{w}\uparrow$    &
			0.825
			&0.847
			&0.838
			&0.858
			&0.857
			&0.867
			&0.856
			&0.863
			&0.853
			&0.855
			&0.860
			&0.869
			&0.873
			&0.871
			&0.871
			&0.873
			&0.846
			& 0.872
			&0.866
			& \color{myblue} \textbf{0.886}
			& \color{mygreen} \textbf{0.889}
			& \color{reda} \textbf{0.894}
			&  \color{mygreen} \textbf{0.889}
			\\
			& $S_m\uparrow$  &
			0.890
			&0.905
			&0.896
			&0.908
			&0.903
			&0.903
			&0.907
			&0.906
			&0.900
			&0.903
			&0.908
			&0.897
			&0.905
			&0.909
			&0.911
			&0.907
			&0.878
			& 0.910
			&0.905
			& \color{myblue} \textbf{0.923}
			& \color{reda} \textbf{0.926}
			& \color{mygreen} \textbf{0.925}
			&0.921

			\\
			& $E_m\uparrow$    &
			0.926
			& 0.930
			& 0.928
			& 0.941
			& 0.937
			& 0.942
			& 0.937
			& 0.937
			& 0.931
			& 0.938
			& 0.939
			& 0.942
			& 0.943
			& 0.938
			& 0.944
			& 0.942
			& 0.928
			& 0.941
			& 0.941
			& \color{myblue} \textbf{0.949}
			& 0.947
			&  \color{reda} \textbf{0.953}
			&  \color{mygreen} \textbf{0.952}

			\\
			&$\mathcal{M}\downarrow$  &   0.051
			&0.043
			&0.046
			&0.041
			&0.040
			&0.039
			&0.041
			&0.039
			&0.041
			&0.041
			&0.040
			&0.039
			&0.037
			&0.038
			&0.037
			&0.037
			&0.045
			& 0.037
			&0.037
			& \color{myblue} \textbf{0.033}
			& \color{mygreen} \textbf{0.032}
			& \color{reda} \textbf{0.031}
			& \color{myblue} \textbf{0.033}
			\\
			\hline
			\multirow{5}{*}{\emph{\rotatebox{90}{NLPR}}}
			&$F_{\beta}^{max}\uparrow$    &   0.910
			&0.913
			&0.921
			&0.927
			&0.925
			&0.916
			&0.925
			&0.917
			&0.927
			&0.925
			&0.919
			&0.916
			&0.917
			&0.917
			&0.927
			&0.926
			&0.898
			& 0.929
			&0.898
			& \color{myblue} \textbf{0.931}
			&\color{mygreen} \textbf{0.935}
			& \color{myblue} \textbf{0.931}
			&\color{reda} \textbf{0.937}
			\\
			&$F_{\beta}^{w}\uparrow$    &
			0.852
			&0.856
			&0.871
			&0.879
			&0.882
			&0.878
			&0.881
			&0.869
			&0.882
			&0.882
			&0.877
			&0.881
			&0.886
			&0.881
			&0.889
			&\color{mygreen} \textbf{0.896}
			&0.857
			& \color{myblue} \textbf{0.895}
			&0.842
			&0.885
			& {0.894}
			& {0.893}
			& \color{reda} \textbf{0.900}
			\\
			& $S_m\uparrow  $         &
			0.915
			&0.917
			&0.923
			&0.930
			&0.925
			&0.920
			&0.930
			&0.916
			&0.923
			&0.930
			&0.925
			&0.918
			&0.921
			&0.921
			&0.929
			&0.927
			&0.898
			&0.929
			&0.908
			&0.927
			& \color{mygreen} \textbf{0.937}
			& \color{myblue} \textbf{0.933}
			& \color{reda} \textbf{0.938}
			\\
			& $E_m\uparrow$  &
			0.942
			&0.941
			&0.948
			&0.954
			&0.955
			&0.955
			&0.955
			&0.948
			&0.957
			&0.954
			&0.954
			&0.952
			&0.956
			&0.952
			& \color{myblue} \textbf{0.959}
			& \color{myblue} \textbf{0.959}
			&0.945
			&\color{mygreen} \textbf{0.961}
			&0.934
			&0.952
			& {0.958}
			&{0.958}
			& \color{reda} \textbf{0.963}
			\\
			&$\mathcal{M}\downarrow$    &
			0.030
			&0.029
			&0.026
			&0.023
			&0.022
			&0.025
			&0.024
			&0.027
			&0.023
			&0.024
			&0.024
			&0.024
			&0.023
			&0.025
			& \color{myblue} \textbf{0.022}
			& \color{mygreen} \textbf{0.021}
			&0.029
			& \color{mygreen} \textbf{0.021}
			&0.031
			&0.024
			& \color{mygreen} \textbf{0.021}
			&\color{myblue} \textbf{0.022}
			& \color{reda} \textbf{0.020}
			\\
			\hline
			\multirow{5}{*}{\emph{\rotatebox{90}{NJUD}}}
			&$F_{\beta}^{max}\uparrow$   &
			0.899
			&0.913
			&0.926
			&0.931
			&0.912
			&0.908
			&0.918
			&0.924
			&0.922
			&0.919
			&0.921
			&0.917
			&0.917
			&0.920
			&0.923
			&0.935
			&0.890
			&0.911
			&0.902
			&\color{myblue} \textbf{0.939}
			&0.938
			& \color{mygreen} \textbf{0.940}
			& \color{reda} \textbf{0.945}
			\\
			&$F_{\beta}^{w}\uparrow$     &
			0.842
			&0.857
			&0.878
			&0.884
			&0.869
			&0.868
			&0.872
			&0.881
			&0.877
			&0.878
			&0.879
			&0.883
			&0.878
			&0.879
			&0.886
			& \color{mygreen} \textbf{0.906}
			&0.843
			&0.872
			&0.850
			&0.900
			&0.900
			& \color{myblue} \textbf{0.905}
			& \color{reda} \textbf{0.909}
			\\
			& $S_m\uparrow$        &
			0.894
			&0.903
			&0.916
			&0.921
			&0.902
			&0.897
			&0.909
			&0.911
			&0.908
			&0.913
			&0.912
			&0.904
			&0.903
			&0.909
			&0.916
			&0.924
			&0.871
			&0.902
			&0.895
			& \color{mygreen} \textbf{0.928}
			& \color{mygreen} \textbf{0.928}
			& \color{myblue} \textbf{0.927}
			& \color{reda} \textbf{0.930}

			\\
			& $E_m\uparrow$     &
			0.917
			&0.923
			&0.937
			&0.942
			&0.935
			&0.934
			&0.935
			&0.934
			&0.932
			&0.940
			&0.937
			&0.937
			&0.941
			&0.931
			&0.942
			& \color{mygreen} \textbf{0.953}
			&0.916
			&0.936
			&0.924
			&0.944
			&0.944
			& \color{myblue} \textbf{0.950}
			& \color{reda} \textbf{0.954}
			\\
			&$\mathcal{M}\downarrow$  &
			0.053
			&0.046
			&0.039
			&0.035
			&0.041
			&0.043
			&0.042
			&0.037
			&0.038
			&0.038
			&0.039
			&0.039
			&0.038
			&0.038
			&0.037
			& \color{reda} \textbf{0.029}
			&0.047
			&0.042
			&0.046
			&0.033
			&0.033
			& \color{myblue} \textbf{0.031}
			& \color{mygreen} \textbf{0.030}
			\\
			\hline
			\multirow{5}{*}{\emph{\rotatebox{90}{SIP}}}
			&$F_{\beta}^{max}\uparrow$   &   0.891
			&0.890
			&0.892
			&0.902
			&0.904
			&0.896
			&0.893
			&0.904
			&0.910
			&0.888
			&0.904
			&0.891
			&0.900
			&0.916
			&0.906
			&0.916
			&0.855
			&0.900
			&0.883
			& \color{mygreen} \textbf{0.935}
			&0.929
			& \color{myblue} \textbf{0.931}
			& \color{reda} \textbf{0.937}
			\\
			&$F_{\beta}^{w}\uparrow$   &   0.819
			&0.811
			&0.820
			&0.830
			&0.844
			&0.836
			&0.822
			&0.835
			&0.848
			&0.812
			&0.842
			&0.829
			&0.841
			&0.854
			&0.845
			&0.868
			&0.780
			&0.836
			&0.803
			& \color{myblue} \textbf{0.878}
			&0.875
			& \color{mygreen} \textbf{0.881}
			& \color{reda} \textbf{0.893}
			\\
			& $S_m\uparrow$        &   0.872
			&0.867
			&0.874
			&0.879
			&0.880
			&0.875
			&0.876
			&0.878
			&0.886
			&0.862
			&0.885
			&0.862
			&0.873
			&0.886
			&0.885
			&0.894
			&0.828
			&0.877
			&0.858
			& \color{myblue} \textbf{0.909}
			& \color{mygreen} \textbf{0.911}
			& \color{myblue} \textbf{0.909}
			& \color{reda} \textbf{0.917}
			\\
			& $E_m\uparrow$     &
			0.913
			&0.909
			&0.912
			&0.917
			&0.923
			&0.918
			&0.912
			&0.921
			&0.925
			&0.905
			&0.925
			&0.911
			&0.921
			&0.925
			&0.924
			&0.931
			&0.890
			&0.923
			&0.909
			& \color{myblue} \textbf{0.942}
			&0.939
			& \color{mygreen} \textbf{0.943}
			& \color{reda} \textbf{0.950}
			\\
			&$\mathcal{M}\downarrow$  &   0.057
			&0.062
			&0.056
			&0.055
			&0.049
			&0.051
			&0.055
			&0.050
			&0.047
			&0.060
			&0.049
			&0.057
			&0.052
			&0.048
			&0.048
			&0.043
			&0.070
			&0.051
			&0.063
			& \color{myblue} \textbf{0.039}
			& \color{mygreen} \textbf{0.038}
			& \color{mygreen} \textbf{0.038}
			& \color{reda} \textbf{0.034}
			\\
			\hline
			\multirow{5}{*}{\emph{\rotatebox{90}{LFSD}}}
			&$F_{\beta}^{max}\uparrow$
			& 0.862
			& \color{mygreen} \textbf{0.900}
			&0.862
			&0.879
			&0.887
			&0.878
			&0.874
			&0.860
			&0.883
			&0.879
			&0.888
			& \color{reda} \textbf{0.903}
			&0.878
			&0.877
			&0.879
			&0.881
			&0.858
			&0.845
			&0.877
			&0.892
			&0.895
			& \color{myblue} \textbf{0.897}
			&0.884

			\\
			&$F_{\beta}^{w}\uparrow$
			& 0.772
			&0.834
			&0.789
			&0.814
			&0.822
			&0.832
			&0.800
			&0.792
			&0.806
			&0.821
			&0.827
			& \color{reda} \textbf{0.859}
			&0.824
			&0.811
			&0.816
			&0.823
			&0.800
			&0.775
			&0.815
			&0.821
			& \color{myblue} \textbf{0.839 }
			& \color{mygreen} \textbf{0.843}
			&0.827

			\\
			& $S_m\uparrow$
			& 0.837
			&0.876
			&0.845
			&0.864
			&0.862
			&0.864
			&0.853
			&0.847
			&0.854
			&0.859
			&0.870
			& \color{reda} \textbf{0.883}
			&0.856
			&0.854
			&0.858
			&0.854
			&0.833
			&0.827
			&0.862
			&0.868
			& \color{mygreen} \textbf{0.881}
			& \color{myblue} \textbf{0.877}
			&0.871

			\\
			& $E_m\uparrow$    &
			0.876
			& \color{myblue} \textbf{0.908}
			&0.883
			&0.901
			&0.902
			&0.906
			&0.894
			&0.883
			&0.891
			&0.899
			&0.906
			& \color{reda} \textbf{0.923}
			&0.903
			&0.892
			&0.898
			&0.897
			&0.875
			&0.869
			&0.901
			&0.900
			& \color{mygreen} \textbf{0.909}
			& \color{mygreen} \textbf{0.909}
			&0.897

			\\
			&$\mathcal{M}\downarrow$  &   0.094
			&0.066
			&0.080
			&0.072
			&0.070
			&0.066
			&0.074
			&0.085
			&0.077
			&0.073
			&0.068
			& \color{reda} \textbf{0.055}
			&0.071
			&0.079
			&0.073
			&0.071
			&0.077
			&0.099
			&0.071
			&0.072
			& \color{myblue} \textbf{0.064}
			& \color{mygreen} \textbf{0.063}
			&0.066

			\\

			\bottomrule[2pt]
		\end{tabular}
	}
	\setlength{\abovecaptionskip}{2pt}

	\label{tab:RGBDSOD_performance}
\end{table*}
\begin{table*}
	\setlength{\abovecaptionskip}{2pt}
	\caption{Quantitative evaluation on RGB SOD datasets. $\uparrow$ and $ \downarrow$ indicate that the larger scores and the smaller ones  are better, respectively. The best three scores are highlighted in {\color{reda} \textbf{red}}, {\color{mygreen} \textbf{green}} and {\color{myblue} \textbf{blue}}.
	}
	\renewcommand\tabcolsep{5pt}
	\renewcommand\arraystretch{1}
	\label{tab:rgbsod_performance}
	\centering

	\resizebox{\textwidth}{!}
	{
		\begin{tabular}{c|c|c|ccccc|ccccc|ccccc|ccccc|ccccc}
			\toprule[1.5pt]
			\multicolumn{1}{c|}{\multirow{2}*{Method}}&\multicolumn{1}{c|}{\multirow{2}*{Publication}}&\multicolumn{1}{c|}{\multirow{2}*{Backbone}} &\multicolumn{5}{c|}{\emph{DUTS}}&\multicolumn{5}{c|}{\emph{DUT-OMRON}}&\multicolumn{5}{c|}{\emph{ECSSD}}&\multicolumn{5}{c|}{\emph{HKU-IS}}&\multicolumn{5}{c}{\emph{PASCAL-S}}\\

			\cmidrule(r){4-8} \cmidrule(r){9-13} \cmidrule(r){14-18} \cmidrule(r){19-23} \cmidrule(r){24-28}
			&& & $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$ \\
			\midrule[1pt]

			GateNet~\cite{GateNet}&ECCV 2020 & ResNet-50&0.888 	&0.809 	&0.884 &	0.903 &	0.040 &	0.818 &	0.729 	&0.837 &	0.868 &	0.055 &	0.945 &	0.894 &	0.920 	&0.943 	&0.040 &	0.933 &	0.881 &	0.915 &	0.954 	&0.033 &	0.881 &	0.804& 	0.854 &	0.882 &	0.071
			\\

			ITSD~\cite{ITSD} &CVPR 2020& ResNet-50 &0.883 &	0.824 &	0.883 &	0.898 	&0.041 &	0.821 &	0.750 &	0.839 &	0.867 &	0.061 	&0.947 &	0.911 &	0.925 &	0.932 &	0.035 &	0.933 	&0.893 &	0.916 &	0.953 &	0.031 &	0.882 &	0.823 	&0.859 	&0.866 &	0.066

			\\
			MINet~\cite{MINet} &CVPR 2020& ResNet-50&0.884 	&0.825& 	0.883 &	0.917 	&\color{myblue} \textbf{0.037} &	0.810 &	0.738 &	0.832 &	0.873 &	0.056 	&0.948 &	0.911 &	0.925 &	0.953 &	0.034 &	0.935 	&0.899 &	0.919 	&\color{mygreen} \textbf{0.961} &	\color{mygreen} \textbf{0.028} &	0.880 &	0.818 	&0.854 &	0.897& 	0.066

			\\

			GCPA~\cite{GCPA} & AAAI 2020& ResNet-50&0.888 &	0.821 &	0.889 &	0.913 	&0.038 &	0.812 &	0.734 &	0.837 &	0.869 &	0.056 	&0.949 &	0.903 &	0.927 &	0.952 &	0.035& 	0.939 	&0.889 	&\color{myblue} \textbf{0.920} &	0.956 &	0.032 &	0.882 &	0.819 	&\color{mygreen} \textbf{0.864} &	0.899 &	0.064
			\\
			F3Net~\cite{F3Net} &AAAI 2020 & ResNet-50&	0.891 &		0.835 &		0.887 &		0.918 	&	\color{mygreen} \textbf{0.035} &		0.813 &		0.747 &		0.837 &		0.876 &		0.053 	&	0.945 &		0.912 &		0.924 &		0.946 &		\color{myblue} \textbf{0.033} &		0.937 	&	0.900 &		0.916 &		0.959 	&	\color{mygreen} \textbf{0.028} &		0.882 &		0.823 	&	0.857 &		0.892 &		0.064

			\\
			VST~\cite{VST}&ICCV 2021&T2T~\cite{T2T}&0.890 &	0.828 &	\color{reda} \textbf{0.895} &	0.916 	&\color{myblue} \textbf{0.037} &	\color{myblue} \textbf{0.825} &	0.755 &	\color{reda} \textbf{0.849} &	0.872 &	0.058 	&\color{myblue} \textbf{0.951} &	0.910 &	\color{mygreen} \textbf{0.932} &\color{mygreen} \textbf{0.957} &	\color{myblue} \textbf{0.033} &	\color{mygreen} \textbf{0.942} 	&0.898 &	\color{reda} \textbf{0.928} &	\color{mygreen} \textbf{0.961} &	\color{myblue} \textbf{0.029} 	&\color{mygreen} \textbf{0.890} &	0.827 	&\color{reda} \textbf{0.871} &\color{reda} \textbf{0.905} &	\color{myblue} \textbf{0.062}

			\\
			SAMNet~\cite{SAMNet} &TIP 2021&SAM~\cite{SAMNet}&\color{myblue} \textbf{0.894} &	0.840 &	0.890 &	\color{myblue} \textbf{0.923} 	&\color{mygreen} \textbf{0.035} &	0.823 &	\color{myblue} \textbf{0.760} 	&\color{myblue} \textbf{0.844} &	\color{myblue} \textbf{0.885} &	\color{myblue} \textbf{0.051} 	&\color{mygreen} \textbf{0.952}& 	\color{mygreen} \textbf{0.920} &	\color{myblue} \textbf{0.928} &	\color{reda} \textbf{0.958} &	\color{mygreen} \textbf{0.032} &	\color{myblue} \textbf{0.940} 	&\color{reda} \textbf{0.905} &	\color{mygreen} \textbf{0.921} &	\color{reda} \textbf{0.963} &	\color{reda} \textbf{0.027} &	\color{myblue} \textbf{0.887} &	\color{myblue} \textbf{0.832} 	&\color{myblue} \textbf{0.862} &\color{myblue} \textbf{0.903} &	\color{myblue} \textbf{0.062}

			\\
			Auto-MSFNet~\cite{Auto-MSFNet}&ACMMM 2021&ResNet-50 &0.877 &	\color{myblue} \textbf{0.841} &	0.876 	&\color{reda} \textbf{0.931} &	\color{reda} \textbf{0.034}& 	0.798 &	0.757 &	0.831 &	0.876 	&\color{mygreen} \textbf{0.050} &	0.941 &	\color{myblue} \textbf{0.916} &	0.914 &	\color{myblue} \textbf{0.954} &	\color{myblue} \textbf{0.033} 	&0.928 &	\color{mygreen} \textbf{0.904}& 	0.908 &	\color{myblue} \textbf{0.960} &	\color{reda} \textbf{0.027} &	0.877 	&\color{mygreen} \textbf{0.834} &	0.851 &	\color{reda} \textbf{0.905} &	\color{mygreen} \textbf{0.061}

			\\
			SGL-KRN~\cite{SGL-KRN} &AAAI 2021&ResNet-50 &\color{mygreen} \textbf{0.898} 	&\color{reda} \textbf{0.847} 	&\color{myblue} \textbf{0.891} &	\color{mygreen} \textbf{0.926} 	&\color{reda} \textbf{0.034} &	\color{mygreen} \textbf{0.827}& 	\color{mygreen} \textbf{0.765} &	\color{mygreen} \textbf{0.845} &	\color{mygreen} \textbf{0.889} &\color{reda} \textbf{0.049} 	&0.946 &	0.910 &	0.923 &	0.942 &	0.036 &	0.937 	&\color{myblue} \textbf{0.902} &	0.918 &	\color{myblue} \textbf{0.960} &	\color{myblue} \textbf{0.029}& 	\color{myblue} \textbf{0.887} &	0.826 	&0.856 &	0.883 &	0.068
			\\
			\midrule[1pt]
			\rowcolor{mygray}
			\multicolumn{2}{c|}{\multirow{1}*{Ours}}&ResNet-50&\color{reda} \textbf{0.904} &	\color{mygreen} \textbf{0.844} 	&\color{mygreen} \textbf{0.893} 	&\color{reda} \textbf{0.931} &	\color{reda} \textbf{0.034} &	\color{reda} \textbf{0.831} &	\color{reda} \textbf{0.766} &	\color{reda} \textbf{0.849} 	&\color{reda} \textbf{0.893} &	\color{reda} \textbf{0.049} &	\color{reda} \textbf{0.960} 	&\color{reda} \textbf{0.925} 	&\color{reda} \textbf{0.936} &\color{reda} \textbf{0.958} 	&\color{reda} \textbf{0.029} &	\color{reda} \textbf{0.944}& 	0.901& 	\color{mygreen} \textbf{0.921} &	\color{mygreen} \textbf{0.961} 	&\color{myblue} \textbf{0.029} &	\color{reda} \textbf{0.894}& 	\color{reda} \textbf{0.837} &	\color{reda} \textbf{0.871}& 	\color{mygreen} \textbf{0.904} &\color{reda} \textbf{0.059}

			\\
			\bottomrule[1.5pt]
		\end{tabular}
	}

\end{table*}
\begin{table}

	\setlength{\abovecaptionskip}{2pt}
	\caption{Quantitative comparison for salient contour extraction.
	}
	\label{tab:Salient_contour_performance}
	\centering

	\resizebox{\columnwidth}{!}
	{
		\begin{tabular}{c|c|cc|cc|cc|cc|cc}
			\toprule[1.5pt]
			\multicolumn{1}{c|}{\multirow{2}*{Method}}&\multicolumn{1}{c|}{\multirow{2}*{Backbone}} &\multicolumn{2}{c|}{\emph{STERE}}&\multicolumn{2}{c|}{\emph{NJUD}}&\multicolumn{2}{c|}{\emph{NLPR}}&\multicolumn{2}{c|}{\emph{SIP}}&\multicolumn{2}{c}{\emph{LFSD}}\\

			\cmidrule(r){3-4} \cmidrule(r){5-6} \cmidrule(r){7-8} \cmidrule(r){9-10} \cmidrule(r){11-12}
			& &
			$F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$

			\\
			\midrule[1pt]

			RD3D~\cite{RD3D} & ResNet-50 & 0.446&	0.017&  0.480&	0.019&	0.435& {0.014}&  0.390&	0.017&	0.440&	0.019 \\
			SPNet~\cite{SPNet} &	Res2Net-50 & 0.393&	0.018 &  0.490&	0.018&	0.426&	0.014&  0.384&	0.018&	0.377&	0.022    \\

			\rowcolor{mygray}
			MMFT & ResNet-50&	\color{reda} \textbf{0.493}&	 {0.017}&   \color{reda} \textbf{0.501}&	 \color{reda} \textbf{0.017}&	 \color{reda} \textbf{0.452}& {0.013}& \color{reda} \textbf{0.407}&{0.016}&	 \color{reda} \textbf{0.473}&{0.024}\\
			\rowcolor{mygray}
			MMFT & Res2Net-50&{0.480}&	 \color{reda} \textbf{0.016}& {0.494}&	 \color{reda} \textbf{0.017}&{0.443}&	\color{reda} \textbf {0.012}& {0.394}&	 \color{reda} \textbf{0.015}&{0.459}&	 \color{reda} \textbf{0.017}\\

			\bottomrule[1.5pt]
		\end{tabular}
	}

\end{table}
\begin{table}
	\setlength{\abovecaptionskip}{2pt}
	\caption{Quantitative comparison for depth estimation.
	}
	\label{tab:depth_estimation_performance}
	\centering

	\resizebox{\columnwidth}{!}
	{
		\begin{tabular}{c|c|cccc|ccc}
			\toprule[1.5pt]
			\multicolumn{1}{c|}{\multirow{2}*{Method}} &\multicolumn{1}{c|}{\multirow{2}*{Backbone}} &\multicolumn{4}{c|}{{Error}} & \multicolumn{3}{c}{{Accuracy}}\\

			\cmidrule(r){3-6} \cmidrule(r){7-9}
			&& RMSE $\downarrow$   & RMSE (log) $\downarrow$
			& Abs Rel $\downarrow$   & Sq ReL $\downarrow$ & P1 $\uparrow$   & P2 $\uparrow$
			& P3 $\uparrow$
			\\
			\midrule[1pt]
			DASNet~\cite{DASNet} & ResNet-50&0.378&0.053&	0.130&	0.078& 0.739&	0.920&	0.984 \\
			CoNet~\cite{CoNet} & ResNet-101&0.433&0.062&	0.149&	0.094& 0.677&	0.907&	0.984 \\

			TL-Depth~\cite{TL-Depth} & ResNet-101&0.387&0.056&	0.134&	0.081& 0.721&	0.911&	0.983 \\
			LAP-Depth~\cite{LAP-Depth} & ResNet-101&0.404&0.049&	 {0.155}&	{0.111}& \color{reda} \textbf{0.771}&  0.925&	0.982 \\
			\rowcolor{mygray}
			MMFT &ResNet-50& \color{reda} \textbf{0.353} & \color{reda} \textbf{0.049}&	\color{reda} \textbf {0.124}& \color{reda} \textbf{0.070}&   {0.749}&  \color{reda} \textbf{0.931}&	 \color{reda} \textbf{0.986}\\
			\rowcolor{mygray}
			MMFT &ResNet-101& {0.355} & {0.050}&	 \color{reda} \textbf{0.124}&{0.071}&   {0.750}&  {0.927}&	 {0.985}\\
			\bottomrule[1.5pt]
		\end{tabular}
	}
	\vspace{-2.5mm}
\end{table}
\begin{figure*}
	\includegraphics[width=\textwidth]{data/compare_sota_visual_results.pdf}\\

	\centering
	\setlength{\abovecaptionskip}{-8pt}
	\caption{Visual comparison of different RGB-D SOD methods.}
	\label{fig:rgbd_sod_comparison}
	\vspace{-3.5mm}
\end{figure*}
\begin{figure*}
	\includegraphics[width=0.9\textwidth]{data/depth_surface_compare_njud_nlpr_lfsd_dutrgbd_figure6_scale.pdf}\\

	\centering
	\setlength{\abovecaptionskip}{-8pt}
	\caption{Visual comparison of the predicted depth maps (Left) and the original ones (Right) provided by the RGB-D SOD dataset.}
	\label{fig:njud_nlpr_dut_lfsd_depth_surface}
	\vspace{-5.5mm}
\end{figure*}
\begin{figure*}
	\includegraphics[width=0.9\textwidth]{data/DUTS-RGB-Our-Depth.pdf}

	\centering

	\caption{The predicted depth maps on RGB SOD datasets~\cite{DUTS,DUT-OMRON,HKU-IS,ECSSD,PASCAL-S}.}
	\label{fig:rgbsod_depth}

\end{figure*}
\begin{table*}
	\setlength{\abovecaptionskip}{2pt}
	\caption{Qualitative evaluation of applying the predicted depth map to existing depth-based RGB-D SOD methods. ``$\ast$'' and ``$\ddagger$'' mean using the original depth map and ours, respectively.
	}
	\label{tab:applying_depth_to_rgbdsod}
	\centering

	\resizebox{\textwidth}{!}
	{
		\begin{tabular}{c|ccccc|ccccc|ccccc|ccccc|ccccc}
			\toprule[1.5pt]
			\multicolumn{1}{c|}{\multirow{2}*{Method}} &\multicolumn{5}{c|}{\emph{SIP}}&\multicolumn{5}{c|}{\emph{STERE}}&\multicolumn{5}{c|}{\emph{NJUD}}&\multicolumn{5}{c|}{\emph{NLPR}}&\multicolumn{5}{c}{\emph{LFSD}}\\

			\cmidrule(r){2-6} \cmidrule(r){7-11} \cmidrule(r){12-16} \cmidrule(r){17-21} \cmidrule(r){22-26}
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			\\
			\midrule[1pt]

			Two-stream FPN$\ast$ & 0.886&	0.810&	0.847&	0.890&	0.061&	0.882&	0.815&	0.865& {0.899}&	0.051& 0.893&	0.834&	0.868&	0.901&	0.052&	0.896&	0.845&	0.889&	0.920&	0.035 &0.844 &	0.767 &	0.827 &	0.864 &	0.098 \\
			\rowcolor{mygray}
			Two-stream FPN$\ddagger$& \color{reda} \textbf{0.903}&	\color{reda} \textbf{0.840}&	\color{reda} \textbf{0.881}&	\color{reda} \textbf{0.919}&	\color{reda} \textbf{0.045}&	\color{reda} \textbf{0.911}&	\color{reda} \textbf{0.850}&	\color{reda} \textbf{0.890}&	\color{reda} \textbf{0.914}&	\color{reda} \textbf{0.042}& \color{reda} \textbf{0.909}&	\color{reda} \textbf{0.857}&	\color{reda} \textbf{0.894}&	\color{reda} \textbf{0.921}&	\color{reda} \textbf{0.043}&	\color{reda} \textbf{0.914}&	\color{reda} \textbf{0.859}&	\color{reda} \textbf{0.918}&	\color{reda} \textbf{0.945}&	\color{reda} \textbf{0.029}
			&\color{reda} \textbf{0.855} &	\color{reda} \textbf{0.787} &	\color{reda} \textbf{0.839} &	\color{reda} \textbf{0.880} &	\color{reda} \textbf{0.088}
			\\
			RD3D$\ast$~\cite{RD3D} & 0.906 &	0.845 &	0.885 &	0.924 &	0.048&  0.917 &	0.871& 	0.911 &	0.944& 	0.037
			&0.923 	&0.886 	&0.916 	&0.942 &	0.037 &0.927 	&0.889 &	0.929 &	\color{reda} \textbf{0.959} &	0.022 &0.879 &	0.816 &	0.858 &	0.898 &	0.073
			\\
			\rowcolor{mygray}
			RD3D$\ddagger$~\cite{RD3D} & \color{reda} \textbf{0.925} &	\color{reda} \textbf{0.879} &	\color{reda} \textbf{0.905} &	\color{reda} \textbf{0.940} &	\color{reda} \textbf{0.038} &	\color{reda} \textbf{0.932} &	\color{reda} \textbf{0.893} &	\color{reda} \textbf{0.922} &	\color{reda} \textbf{0.948} &	\color{reda} \textbf{0.030} &\color{reda} \textbf{0.937} &	\color{reda} \textbf{0.902} &	\color{reda} \textbf{0.924} &	\color{reda} \textbf{0.947} &	\color{reda} \textbf{0.031} &	\color{reda} \textbf{0.928} &	\color{reda} \textbf{0.893} &	\color{reda} \textbf{0.929} &	{0.957} & \color{reda} \textbf{0.022}&	\color{reda} \textbf{0.888} &	\color{reda} \textbf{0.828} &	\color{reda} \textbf{0.867} &	\color{reda} \textbf{0.898} & \color{reda} \textbf{0.068}\\
			SPNet$\ast$~\cite{SPNet} &0.916 &	0.868 &	0.894 	&0.931 	&0.043 &0.915 &	0.873 &	0.907 &	0.942 	&0.037 &0.935 &	0.906 	&0.924 	&0.953 &	0.029  &0.926 &	0.896 &	0.927 	&0.959 &	0.021 &0.881 &	0.823 &	0.854 &	0.897 	&0.071

			\\
			\rowcolor{mygray}
			SPNet$\ddagger$~\cite{SPNet}& \color{reda} \textbf{0.928}&	\color{reda} \textbf{0.883}&	\color{reda} \textbf{0.907}&	\color{reda} \textbf{0.942}&	\color{reda} \textbf{0.037}&	\color{reda} \textbf{0.931}&	\color{reda} \textbf{0.894}&	\color{reda} \textbf{0.922}&	\color{reda} \textbf{0.952}&	\color{reda} \textbf{0.030}& \color{reda} \textbf{0.941}&	\color{reda} \textbf{0.910}&	\color{reda} \textbf{0.928}&	\color{reda} \textbf{0.954}&	\color{reda} \textbf{0.029}&	\color{reda} \textbf{0.931}&	\color{reda} \textbf{0.898}&	\color{reda} \textbf{0.930}&	\color{reda} \textbf{0.961}&	\color{reda} \textbf{0.021} &	\color{reda} \textbf{0.882}&	\color{reda} \textbf{0.826}&	\color{reda} \textbf{0.866}&	\color{reda} \textbf{0.901}&	\color{reda} \textbf{0.065}\\

			\bottomrule[1.5pt]
		\end{tabular}
	}

\end{table*}
\begin{figure}
	\includegraphics[width=\linewidth]{data/SIP_depth_estimation_compare.pdf}
	\setlength{\abovecaptionskip}{-2pt}

	\centering

	\caption{Visual comparisons with several depth estimation methods~\cite{TL-Depth,LAP-Depth} and CoNet~\cite{CoNet} on the SIP dataset~\cite{SIP}.}
	\label{fig:SIP_depth_comparison}
	\vspace{-2.5mm}
\end{figure}

\noindent\textbf{\textit{Depth Evaluation.}} Tab.~\ref{tab:depth_estimation_performance} shows the results of depth estimation on the SIP dataset in terms of seven common metrics. We compare the proposed method with two depth estimation methods TL-Depth~\cite{TL-Depth} and LAP-Depth~\cite{LAP-Depth} and two depth-free sod methods CoNet~\cite{CoNet} and DASNet~\cite{DASNet}. It can be seen that the proposed method outperforms these models across six metrics. In addition, in order to show the benefits of the predicted depth maps, we apply them to existing depth-based RGB-D SOD methods instead of using the original ones and retrain them, which include the baseline (two-stream FPN) used in~\cite{HDFNet,S2MA,JLDCF,RD3D,SPNet} and existing top
two methods SPNet~\cite{SPNet} and RD3D~\cite{RD3D}. As shown in Tab.~\ref{tab:applying_depth_to_rgbdsod}, all these models obtain a considerable performance gain.

\subsubsection{Qualitative Evaluation}
~\\
\textbf{\textit{Saliency Evaluation.}}
Fig.~\ref{fig:rgbd_sod_comparison} illustrates several visual results of different RGB-D SOD approaches. The proposed method yields the results closer to the ground truth in various challenging scenarios. In the $1^{st}$ and $2^{nd}$ rows, we show two examples which have fuzzy depth maps. From the results, it can be observed that our method can segment the objects well while the other methods more or less lose similar areas inside or around salient objects.  In the $3^{rd}$ and $4^{th}$ rows,
it can be seen that our method has more obvious advantages when handling internal hollow areas, and our saliency maps have sharper contours with the help of the contour decoder.
\\
\textbf{\textit{Depth Evaluation.}}
The proposed method can predict the denoised depth map. As shown in the Fig.~\ref{fig:STERE_depth_surfacenorm} and Fig.~\ref{fig:njud_nlpr_dut_lfsd_depth_surface}, compared with the original depth map provided by  RGB-D SOD dataset, the advantage of our predicted depth map is very significant. To intuitively show the depth map, we utilize it to generate a surface norm map that measures the smoothness of the surface of an object. We can see that the low-quality depth map contains serious Gaussian-like noise and checkerboard pattern. In contrast, the key appearance information such as the contour of the object is better embodied in our depth map. We further qualitative evaluate our depth results on the high-quality SIP~\cite{SIP} dataset, as shown in Fig.~\ref{fig:SIP_depth_comparison}. It can be seen that our predicted depth maps are closer to the ground-truth and retain the human contours very well (See the $1^{st}$ - $4^{th}$ columns). For complex scenes with multiple objects, our method is still able to  perceive the positions of all objects while accurately reflecting the relative depth relationships among them (See the $5^{th}$ - $8^{th}$ columns). Besides, we also show some depth results on several RGB SOD datasets, as shown in Fig.~\ref{fig:rgbsod_depth}. It can be seen that both multiple object and single object cases are accurately measured and even overcome the influence of shadows, which indicates good generalization ability.
\begin{figure}
	\includegraphics[width=\linewidth]{data/compare_ours_conet.pdf}
	\setlength{\abovecaptionskip}{-10pt}

	\centering

	\caption{Visual comparisons between ours and CoNet~\cite{CoNet} for depth estimation,  contour  and saliency prediction.}
	\label{fig:comparison_conet}
	\vspace{-5.5mm}
\end{figure}
\begin{table*}
	\setlength{\abovecaptionskip}{2pt}
	\caption{Ablation  experiments  of  each component for salient object detection. Model 1: the FPN Baseline (Saliency Decoder).  Model 2: + Depth Estimation Decoder. Model 3: + Contour Estimation Decoder. Model 4: Without Squeeze-and-Expand in decoder block. Model 5 - Model 9: + Multi-modal Transformer (1, 2, 4, 6 and 8 iterations). Model 10: + Modality-specific Filter. Model 11: Replace the MFT with general Non-local+Convs. Model 12: Replace the MFT with Convs.
	}

	\label{tab:ablation_study_rgbdsod}
	\centering

	\resizebox{\textwidth}{!}
	{
		\begin{tabular}{c|c|c|ccccc|ccccc|ccccc|ccccc|ccccc}
			\toprule[1.5pt]
			\multicolumn{1}{c|}{\multirow{2}*{Method}}&\multicolumn{2}{c|}{\emph{}} &\multicolumn{5}{c|}{\emph{SIP}}&\multicolumn{5}{c|}{\emph{STERE}}&\multicolumn{5}{c|}{\emph{NJUD}}&\multicolumn{5}{c|}{\emph{NLPR}}&\multicolumn{5}{c}{\emph{LFSD}}\\

			\cmidrule(r){2-3} \cmidrule(r){4-8} \cmidrule(r){9-13} \cmidrule(r){14-18} \cmidrule(r){19-23} \cmidrule(r){24-28}
			& Params (MB)$\downarrow$ & FLOPs (GB)$\downarrow$ & $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$& $F_{\beta}^{max}\uparrow$  & $F_{\beta}^{w}\uparrow$ &  $S_m\uparrow $ &  $E_m\uparrow $ & $\mathcal{M}\downarrow$ \\
			\midrule[1pt]

			Model 1 & 53.54& 	17.98&  0.860& 	0.766& 	0.831 &	0.833 &	0.062 &	0.868 &	0.784 &	0.849 &	0.862 &	0.051 &	0.872&	0.799&	0.858&	0.847&	0.056&	0.873 &	0.810 &	0.874 &	0.882 	&0.038 &	0.800& 	0.713& 	0.777 &	0.801 &	0.117
			\\

			Model 2 & 55.54&	21.35& 0.884  & 	0.801  & 	0.857 &  	0.863  & 	0.054  & 	0.890  & 	0.817 &  	0.873  & 	0.881  & 	0.044  & 	0.888 & 	0.834 & 	0.878 & 	0.872 & 	0.047	 & 0.890  & 	0.827  & 	0.885  & 	0.899  & 	0.034  & 	0.827  & 	0.741  & 	0.804  & 	0.829  & 	0.105
			\\
			\rowcolor{mygray}
			Model 3 & 57.54& 24.71& \color{myblue} \textbf{0.896}&	\color{myblue} \textbf{0.812}&	\color{myblue} \textbf{0.866}&	\color{myblue} \textbf{0.874}&	\color{myblue} \textbf{0.051}&	\color{myblue} \textbf{0.899}&	\color{myblue} \textbf{0.830}&	\color{myblue} \textbf{0.887}&	\color{myblue} \textbf{0.895}&	\color{myblue} \textbf{0.042}& \color{myblue} \textbf{0.893}&	\color{myblue} \textbf{0.846}&	\color{myblue} \textbf{0.888}&	\color{myblue} \textbf{0.892}&	\color{myblue} \textbf{0.044}&	\color{myblue} \textbf{0.896}&	\color{myblue} \textbf{0.840}&	\color{myblue} \textbf{0.894}&	\color{myblue} \textbf{0.910}&	\color{myblue} \textbf{0.032}&\color{myblue} \textbf{0.840} &	\color{myblue} \textbf{0.762} &	\color{myblue} \textbf{0.819} &	\color{myblue} \textbf{0.840} &	\color{myblue} \textbf{0.099}
			\\

			Model 4 & 56.62& 23.28& 0.887 &	0.787 &	0.853 &	0.854 &	0.056&	0.887 &	0.814 &	0.877 &	0.883&	0.044 &	0.884&	0.832&	0.880&	0.878&	0.046&	0.890 &	0.823 	&0.883 &	0.896 	&0.035 &	0.818&	0.726&	0.796&	0.831	&0.108

			\\

			\toprule[1pt]
			\toprule[1pt]
			Model 5 & 60.30&	24.89& 0.910 &	0.841& 	0.882 &	0.901 &	0.046 &	0.911 &	0.855 &	0.900 &	0.916 &	0.038 &	0.910 &	0.869& 	0.902& 	0.911 &	0.038 	&0.909 &	0.860 &	0.911 &	0.932 &	0.028 &	0.851 &	0.792& 	0.834& 	0.855 &	0.083
			\\
			Model 6 & 63.94&	25.12& 0.913 &	0.850 &	0.886 &	0.909 &	0.044 &	0.917 &	0.866 &	0.907 &	0.928 &	0.037 &	0.918 &	0.882 &	0.908& 	0.923 &	0.036 	&0.912 &	0.867 &	0.915 &	0.940 &	0.027 &	0.856 &	0.799 &	0.838& 	0.862 &	0.080
			\\
			Model 7 & 71.22&	25.58&0.918 &	0.857 &	0.893 &	0.917 &	0.043 &	0.921 &	0.872 &	0.908 &	0.936 &	0.036 &	0.924 &	0.882 &	0.915 &	0.930& 	0.036 	&0.918 &	0.873 &	0.920 &	0.944 &	0.026 &	0.864 &	0.808 &	0.846 &	0.874 &	0.075
			\\
			Model 8 & 78.51&	26.05& 0.920 &	0.863 &	0.897 	&0.920 &	0.042 &	0.924 &	0.879& 	0.912 &	0.943 &	0.035 &	0.928 &	0.886 &	0.917 &	0.935 &	0.034 	&0.921 &	0.879 	&0.922 	&0.950 &	0.024& 	0.872 &	0.811 &	0.852& 	0.883 &	0.072
			\\
			Model 9 & 85.79&	26.52& 0.917 &	0.863 &	0.895 &	0.919 &	0.042 &	0.924 &	0.878 &	0.912 &	0.942 &	0.035 &	0.929 &	0.884 &	0.913 &	0.934 &	0.034 	&0.921 &	0.880 	&0.920 &	0.949& 	0.024& 	0.872 &	0.808 &	0.854 &	0.884 &	0.072
			\\
			\rowcolor{mygray}
			Model 10  & 75.96&	25.89& \color{reda} \textbf{0.931}&	\color{reda} \textbf{0.881}&	\color{reda} \textbf{0.909}&	\color{reda} \textbf{0.943}&	\color{reda} \textbf{0.038}&	\color{reda} \textbf{0.935}&	\color{reda} \textbf{0.894}&	\color{reda} \textbf{0.925}&	\color{reda} \textbf{0.953}&	\color{reda} \textbf{0.031} &\color{reda} \textbf{0.940}	&\color{reda} \textbf{0.905}&	\color{reda} \textbf{0.927}&	\color{reda} \textbf{0.950}&	\color{reda} \textbf{0.031}&	\color{reda} \textbf{0.931}&	\color{reda} \textbf{0.893}&	\color{reda} \textbf{0.933}&	\color{reda} \textbf{0.958}&	\color{reda} \textbf{0.022} & \color{reda} \textbf{0.897}& 	\color{reda} \textbf{0.843} &	\color{reda} \textbf{0.877} &	\color{reda} \textbf{0.909} &	\color{reda} \textbf{0.063}
			\\

			Model 11 & 76.20&	26.12&  0.908 &	0.852 &	0.883 &	0.907 &	0.045 &	0.921 &	0.870 &	0.913 &	0.932 &	0.037 	&0.917& 	0.877 &	0.916 &	0.921 &	0.037 	&0.911 &	0.870 &	0.913 &	0.937& 	0.028 &	0.855 &	0.803 &	0.835 	&0.857 	&0.080
			\\
			Model 12 & 76.40&	27.45&  0.894 &	0.808 &	0.868 &	0.879 &	0.052&	0.901 &	0.834 &	0.893 &	0.899 &	0.042 &	0.890 &	0.844 &	0.889 &	0.891 &	0.045 	&0.898 &	0.848& 	0.896 &	0.914 &	0.031 &	0.833 &	0.754 &	0.813& 	0.840 &	0.102
			\\
			\bottomrule[1.5pt]
		\end{tabular}
	}
	   \vspace{-3.5mm}
\end{table*}
\begin{table}
	\setlength{\abovecaptionskip}{2pt}
	\caption{Ablation  experiments  of  each component for depth estimation. Model 1: the FPN Baseline (Depth Estimation Decoder).  Model 2: + Saliency Decoder. Model 3: + Contour Estimation Decoder. The others ``Model *'' correspond to the same model in Tab.~\ref{tab:ablation_study_rgbdsod}.
	}
	\label{tab:ablation_study_depth_estimation}
	\centering

	\resizebox{\columnwidth}{!}
	{
		\begin{tabular}{c|cccc|ccc}
			\toprule[1.5pt]
			\multicolumn{1}{c|}{\multirow{2}*{Method}} &\multicolumn{4}{c|}{{Error}} & \multicolumn{3}{c}{{Accuracy}}\\

			\cmidrule(r){2-5} \cmidrule(r){6-8}
			& RMSE $\downarrow$   & RMSE (log) $\downarrow$
			& Abs Rel $\downarrow$   & Sq ReL $\downarrow$ & P1 $\uparrow$   & P2 $\uparrow$
			& P3 $\uparrow$
			\\
			\midrule[1pt]
			Model 1&0.424&	0.085&	0.180&	0.120&  0.670&	0.851&	0.970  \\
			Model 2&0.394&	0.075&	0.166&	0.096&  0.710&	0.882&	0.974 \\
			\rowcolor{mygray}
			Model 3&\color{myblue} \textbf{0.389}&	\color{myblue} \textbf{0.070}&\color{myblue} \textbf{0.161}&\color{myblue}\textbf{0.093}& \color{myblue} \textbf{0.718}&	\color{myblue} \textbf{0.890}&	\color{myblue} \textbf{0.976} \\

			Model 4 &0.400&	0.078&	0.171&	0.102&  0.700&	0.871&	0.972 \\
			\hline
			\hline
			Model 5 &0.368&0.060&0.147&0.082& 0.739&  0.904 & 0.980\\
			Model 6 &0.362&0.056&0.143&0.078& 0.745&  0.912 & 0.980\\
			Model 7 &0.358&0.054&0.140&0.076& 0.748&  0.917 & 0.982\\
			Model 8 &0.355&0.052&0.137&0.075& 0.752&  0.922 & 0.982\\
			Model 9 &0.356&0.052&0.138&0.074& 0.750&  0.921 & 0.981\\
			\rowcolor{mygray}
			Model 10 & \color{reda} \textbf{0.343}& \color{reda} \textbf{0.048}& \color{reda} \textbf{0.120}& \color{reda} \textbf{0.068}& \color{reda} \textbf{0.763}& \color{reda} \textbf{0.931}  & \color{reda} \textbf{0.984}\\
			Model 11 &0.372&0.064&0.150&0.085& 0.734&  0.900 & 0.978\\
			Model 12 &0.387&0.072&	0.170&	0.105&  0.720&	0.874&	0.973 \\
			\bottomrule[1.5pt]
		\end{tabular}
	}
	\vspace{-2.5mm}
\end{table}

\begin{table}

	\setlength{\abovecaptionskip}{2pt}
	\caption{Ablation  experiments  of  each component for contour estimation. Model 1: the FPN Baseline (Contour Estimation Decoder).  Model 2: + Saliency Decoder. Model 3: + Depth Estimation Decoder. The others ``Model *'' correspond to the same model in Tab.~\ref{tab:ablation_study_rgbdsod}.
	}
	\label{tab:ablation_study_salient_contour}
	\centering

	\resizebox{\columnwidth}{!}
	{
		\begin{tabular}{c|cc|cc|cc|cc|cc}
			\toprule[1.5pt]
			\multicolumn{1}{c|}{\multirow{2}*{Method}} &\multicolumn{2}{c|}{\emph{STERE}}&\multicolumn{2}{c|}{\emph{NJUD}}&\multicolumn{2}{c|}{\emph{NLPR}}&\multicolumn{2}{c|}{\emph{SIP}}&\multicolumn{2}{c}{\emph{LFSD}}\\

			\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} \cmidrule(r){8-9} \cmidrule(r){10-11}
			&
			$F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$
			& $F_{\beta}^{max}\uparrow$   & $\mathcal{M}\downarrow$

			\\
			\midrule[1pt]

			Model 1  &0.414&0.021&0.432&0.019&0.381&0.019&0.332&0.019&0.393&0.026\\
			Model 2 &0.436&0.018&0.450&0.021&0.400&0.017&0.348&0.017&0.410&0.023\\
			\rowcolor{mygray}
			Model 3 & \color{myblue} \textbf{0.442}&\color{myblue} \textbf{0.018}&\color{myblue} \textbf{0.459}&\color{myblue} \textbf{0.020}&\color{myblue} \textbf{0.406}&\color{myblue} \textbf{0.016}&\color{myblue} \textbf{0.356}&\color{myblue} \textbf{0.016}&\color{myblue} \textbf{0.417}&\color{myblue} \textbf{0.022}\\
			Model 4  &0.430&0.019&0.448&0.022&0.394&0.018&0.340&0.018&0.405&0.023
			\\
			\hline
			\hline
			Model 5&0.454&0.016&0.472&0.018&0.420&0.015&0.366&0.017&0.437&0.020\\
			Model 6&0.460&0.017&0.477&0.019&0.422&0.015&0.372&0.016&0.440&0.019\\
			Model 7&0.465&0.018&0.479&0.019&0.427&0.014&0.380&0.017&0.442&0.019\\
			Model 8&0.468&0.018&0.484&0.019&0.430&0.014&0.385&0.017&0.445&0.019
			\\
			Model 9 &0.466&0.018&0.484&0.019&0.429&0.014&0.386&0.017&0.444&0.019
			\\
			\rowcolor{mygray}
			Model 10 &	 \color{reda} \textbf{0.480}&	 \color{reda} \textbf{0.016}& 	 \color{reda} \textbf{0.494}&	 \color{reda} \textbf{0.017}&	 \color{reda} \textbf{0.443}&	\color{reda} \textbf {0.012}&	 \color{reda} \textbf {0.394}&	 \color{reda} \textbf{0.015}&	 \color{reda} \textbf{0.459}&	 \color{reda} \textbf{0.017}\\
			Model 11&0.459&0.017&0.470&0.019&0.418&0.018&0.364&0.017&0.430&0.020\\
			Model 12 &0.440&0.018&0.457&0.020&0.406&0.017&0.350&0.017&0.417&0.022\\

			\bottomrule[1.5pt]
		\end{tabular}
	}
	\vspace{-5.5mm}
\end{table}
\subsection{Comprehensive Comparison with CoNet} \label{sec:Comprehensive_Comparison_with CoNet}
CoNet~\cite{CoNet} unidirectionally utilizes depth attention and contour attention to help RGB-D salient object detection, which is not an implementation of multi-task learning. In order to fully demonstrate our superiority and  difference, we visualize some results in Fig.~\ref{fig:comparison_conet}. Benefiting from the multi-task design and the \textit{Squeeze-and-Expand} strategy, we obtain better predictions on the three tasks. The smooth body, sharp contour and accurate localization are fully reflected in each task.
\subsection{Ablation Studies for Salient Object Detection}\label{sec:AbStd}
In this section, we show the effectiveness of each component for the salient object detection. All the experiments are based on the Res2Net-50 backbone.
\\
\textbf{Multi-task Learning.}
We take the FPN with the RGB input and a saliency decoder branch as the baseline. The depth estimation and contour extraction branches are then added to the baseline, respectively. We apply the \textit{Squeeze-and-Expand} (SE) structure in each decoder branch. As shown in  Tab.~\ref{tab:ablation_study_rgbdsod}, Model $2$ vs. Model $1$ and Model $3$ vs. Model $2$ show the effectiveness of  depth estimation and salient contour extraction for saliency prediction, respectively.
It can be seen that the multi-task network (Model $3$) achieves a significant improvement compared to the baseline with the  gain of $5.87\%$ and $17.65\%$ in terms of $F_{\beta}^{w}$ and MAE on the challenging  STERE dataset. In addition, we remove all SEs to obtain Model $4$, in which three decoder branches are completely independent without any interaction. The performance gap between Model $3$ and Model $4$ indicates the necessity of information interaction among different types of features.
\\
\textbf{Multi-modal Filtered Transformer.} Based on Model $3$, we verify the benefits of multi-modal transformer (MMT) and modality-specific filter (MSF) in multi-modal filtered transformer (MFT), as shown in Tab.~\ref{tab:ablation_study_rgbdsod}.
Model $5$ performs much better than Model $3$, which shows the superiority of multi-modal transformer in the SOD. We conduct a series of experiments about the number of iterations in MMT. The Model $8$ with six MMT iterations achieves the best accuracy among Model $5$ - Model $9$. Then, we equip with the MSF and find that Model $10$  achieves further performance improvement over Model $8$, which verifies the effectiveness of the MSF.
In addition, we replace the MFT module with some convolution layers or general non-local module, and a similar number of parameters and computation are preserved. Compared to Model $11$ and Model $12$, Model $10$ has obvious performance advantage.
\begin{figure}
	\centering
	\includegraphics[width=\linewidth]{data/ablation_study_visual_results.pdf}
	\setlength{\abovecaptionskip}{-10pt}
	\caption{Visual results of each component.}
	\label{fig:visual_results_each_component}
	\vspace{-5.5mm}
\end{figure}
\begin{figure}
	\includegraphics[width=\linewidth]{data/MDB_visual.pdf}

	\centering
	\setlength{\abovecaptionskip}{-10pt}
	\caption{Visualization of each modality feature in MDB-2. }
	\label{fig:MDB_visualization}
	 \vspace{-5.5mm}
\end{figure}
\\
\noindent\textbf{Visual Analyses.}
Fig.~\ref{fig:visual_results_each_component} shows visual results of the ablation study.  Firstly, we can see that ``+ Depth Estimation Decoder'' model localizes salient object more precisely compared to ``Baseline'' model. Next,  ``+ Contour Estimation Decoder'' model further refine the saliency prediction along the contour. Thirdly,  ``+ Multi-modal Transformer'' model integrates the advantages of three kinds of modalities. Finally,  ``+ Modality-specific Filter'' model significantly refines the details of salient objects.
To intuitively show the effectiveness of the proposed \textit{Squeeze-and-Expand} (SE) structure,  we take MDB-2 as an example to visualize each modality feature before and after SE operation in Fig.~\ref{fig:MDB_visualization}. It can be seen that $F_{enD}^{2}$, $F_{enS}^{2}$ and $F_{enC}^{2}$  all have more precise positions and contours  compared with $F_{D}^{2}$,  $F_{S}^{2}$ and  $F_{c}^{2}$. This  verifies that SE can effectively supplement  complementary information   among different modalities.

\subsection{Ablation studies for Depth and Salient Contour Estimation}\label{sec:Ablation studies for Depth and Contour Estimation}
We evaluate each component for depth estimation and contour prediction.  All the experiments are based on the Res2Net-50 backbone. As shown in Tab.~\ref{tab:ablation_study_depth_estimation} and Tab.~\ref{tab:ablation_study_salient_contour}, Model  $2$ vs.  Model $1$ and Model $3$ vs. Model $2$ show the benefits of the multi-task learning.   The performance gap between Model $3$ and Model $4$  indicates the necessity of  embedding Squeeze-and-Expand structure in the decoder to interact information among different types of features. Model $10$ vs. Model $3$ verifies the benefit of the multi-modal filtered transformer.  Model $10$ has obvious performance advantage over Model $11$ and Model $12$ under a similar number of parameters and comparable computational cost.

\section{More Insightful Analyses}
\label{sec:More_Insightful_Analyses}
\subsection{Why can different tasks promote each other?}
\label{sec:Different_Tasks_Promote_Each_Other}
In this work, the three tasks all require to capture vital position and appearance cues about the objects, and they are complementary. Specifically, depth map contains natural contrast information, which provides useful guidance for saliency and contour positioning. Object contour is one of important attributes of saliency and depth modalities. Accurate contour prediction will improve the quality of depth and saliency estimation. Saliency map provides a fine foreground segmentation and its internal consistency is helpful to avoid the checkerboard pattern in depth estimation.

In a multi-task learning network, the backpropagation acts on multiple task branches in parallel. Different  branches share hidden layer features of multiple scales from the encoder network, and the feature representations used for specific task in these hidden layers are also used by the other tasks. These prompt multiple tasks to be learned together. Based on the multi-task framework, the implicit data augmentation, attention focusing, eavesdropping and regularization guarantee the above-mentioned complementary cues to be fully mined and utilized, thereby making the three pixel-level dense prediction tasks promote each other.

\subsection
{Why high-quality depth can be estimated?}
\label{sec:High-quality_Depth_Estimation}
It is an interesting phenomenon that we obtain high-quality depth predictions under the supervision of poor depth maps.
The regularization of multi-task learning can balance the performance of different tasks since each task is actually also regularized by the other tasks. If the predicted depth map approaches the poor ground-truth, it will interfere with the predictions of the other two tasks. To resist the negative effect, the multi-task network interactively adjusts and balances each task prediction. Thus, the depth branch absorbs useful information from saliency and contour branches, e.g., the clues about smooth object region and sharp object contour.

Moreover,  due to the uncertainty of noise, the poor depth ground-truth is more difficulty fitted than good depth map. As reflected in~\cite{DIP}, when using the gradient descent to optimize convolutional neural network, it is not sensitive to noise and will be faster and easier to learn to get a more natural-looking image. In other words, CNN offers high impedance to noise. Therefore, our model can easily learn important and effective signals in the depth map during the training process.

In summary, multi-task learning strategy enables the model to balance the losses and gradients of different tasks. By suppressing noise and mining complementary information from the auxiliary tasks, a good depth map is obtained.

\section{Positive Impacts}
\label{sec:Positive Impacts}
\begin{figure}
	\includegraphics[width=\linewidth]{data/sensor_failure.pdf}
	\setlength{\abovecaptionskip}{-1pt}

	\centering
	\caption{The case of depth sensor malfunction.}
	\label{fig:sensor_failure}

\end{figure}
For the salient object detection task, this work has many positive impacts on the community and practical applications.
\\
\textbf{\textit{First}}, the resulted depth map from our network can help existing depth-based RGB-D SOD methods obtain significant performance gain, as shown in Tab.~\ref{tab:applying_depth_to_rgbdsod}.
\\
\textbf{\textit{Second}}, once the depth-based method cascades our network and adopt the depth map predicted from our network as the input can help the depth-based network get rid of the dependence on the depth-sensor to have a more flexible applicability.
\\
\textbf{\textit{Third}}, as shown in Fig.~\ref{fig:sensor_failure}, these examples are from the NLPR~\cite{early_fusion_1} dataset. We can see that the original depth can not capture any depth information, which illustrates the depth sensor do suffer from stability problems, but our approach can avoid such failure predictions.
\\
\textbf{\textit{Last but not the least}}, we know that one of the important applications of saliency detection is background bokeh. In real smartphones such as the Huawei P40 pro, achieving background bokeh in portrait mode relies on the ToF camera to provide depth information to project bokeh and focus. Different from them, our method not only saves the cost of the ToF camera, but also can predict both depth map as well as saliency map for automatic and pure computer vision based background bokeh.

\section{Conclusion}
In this work,
we propose a multi-task and multi-modal filtered transformer (MMFT) network for depth-free RGB-D salient object detection. First, we design three related dense prediction tasks: depth estimation, salient object detection and contour prediction. The three modal features rely on the \textit{Squeeze-and-Expand} structure to deeply achieve task-aware information exchanges.  Next, to fully integrate the three kinds of features in high level from the global perspective, we use the transformer technique to achieve multi-modal feature fusion and embed the modality-specific filter to better enhance each modality feature. Through multi-task learning, the depth decoder branch can capture appearance and boundary cues from the other two branches to complete and purify the depth information, thereby predicting a high-quality depth map.
Extensive experiments show that our model performs well on the six RGB-D SOD datasets. By using the predicted depth map, the performance of previous state-of-the-art methods can be consistently improved.

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\small{

redpajama_set_name: RedPajamaArXiv


text:
The Magic Christian (1969)
M | 92 min | Comedy
Sir Guy Grand, the richest man in the world, adopts a homeless boy, Youngman. Together, they set out to prove that anyone, and anything, can be bought with money.
Watch on Prime Video rent from $3.99
Sir Guy Grand
Youngman Grand
Isabel Jeans
Dame Agnes Grand
Caroline Blakiston
Hon. Esther Grand
Wilfrid Hyde-White
Captain Reginald K. Klaus
(as Wilfrid Hyde White)
Oxford Coach
Leonard Frey
Laurence Faggot
Laurence Harvey
Ship's Vampire
Traffic Warden 27
Solitary Drinker
Priestess of the Whip
Tom Boyle
My Man Jeff
Victor Maddern
Hot Dog Vendor
Terence Alexander
Mad Major
Joseph McGrath
Terry Southern (screenplay), Joseph McGrath (screenplay), Terry Southern (based on the novel by), Graham Chapman (additional material), John Cleese (additional material), Peter Sellers (additional material)
23 October 2001 | mfisher452
| Still entertaining after all these years
After more than 40 years, The Magic Christian still entertains. Its style is very much of the Sixties, but its profoundly cynical message---that anything can be bought, that everyone has his price---is, if anything, more relevant now than in 1969 when the film was released. The star, of course, is Peter Sellers as the obscenely wealthy Sir Guy Grand, who manages to seem almost childlike as he spreads his bounty of cynicism throughout London.
This is not a great film, or even necessarily a good one, but even second- or third-rate Peter Sellers may be preferable to a lot of first-rate work by others. The childless Sir Guy decides one morning to acquire an heir, so he goes to the park and picks up a homeless man played by Ringo Starr, and adopts him as his son, Youngman Grand. (Ringo actually doesn't have much to do in this film except react to Sellers.) Sir Guy then enlists Youngman in escapades that, in his hands, skewer the stuffed shirts of upper-class London society and turn the most solemn occasions into a carnival of absurdist nihilism. The most extreme comes at the end of the film, where he scatters money into a huge vat of blood, urine and excrement, and then watches as bowler-hatted City of London types wade into it for the money. This scene doesn't quite work. There is an extended sequence aboard a bogus cruise ship called The Magic Christian that tends to try one's patience because it degenerates into a very Sixties psychedelic montage. One moment from this sequence, however, is worth the whole thing: Raquel Welch as the Priestess of the Whip. Dressed as a dominatrix, she never looked more luscious or voluptuous. Film aficionados will appreciate the many old-line British actors who contributed supporting or cameo roles (Spike Milligan, Lawrence Harvey, Richard Attenborough, John Le Mesurier, Wilfrid Hyde-White, Christopher Lee, and others less well known outside the UK) as well as glimpses of younger now-famous faces, especially John Cleese in a hysterically funny scene at Sotheby's. Cleese plays the terminally smarmy, unctuous, patronizing curator Mr. Dougdale, whose supercilious mien is punctured beyond repair by Sir Guy in a scene involving the defacing of a priceless painting. There is a Monty Python skit that looks like it was directly inspired by this scene. This film was shot at about the time of the first season of Monty Python's Flying Circus, and what with the appearance in the film of at least two Pythons that I could identify, there are definitely echoes of Python in it. The other Python was (an uncredited) Graham Chapman as the leader of the Oxford team during the famous Oxford-Cambridge boat race. Watch also for an uncredited Yul Brynner playing a female impersonator who does a sexy torch song. Alert listeners---especially lovers of the classic 1950s BBC radio comedy program the Goon Show---will also notice that Sellers does almost all of the off-screen voices and several voices of characters seen only in long shot, reminiscent of the films of Orson Welles; so if you suddenly think you hear Henry Crun or Major Bloodnok off-screen, it's not your imagination.
All in all, a solid five or six stars out of ten.
Heavens Above!
The Mouse That Roared
Flareup
The Bobo
After the Fox
The Great McGonagall
The Optimists of Nine Elms
What's New Pussycat
Sir Guy Grand (Peter Sellers) refers disparagingly to prospective "Magic Christian" passengers Simon and Deborah Devonshire as being "hardly Burke's", a reference to "Burke's Peerage, Baronetage & Knightage", an authoritative genealogy guide to British blue-blood families.
Youngman Grand, Esq.: You're certainly puttin' everybody on today, Dad.
Sir Guy Grand KG, KC, CBE: Well, you know, Youngman, sometimes it's not enough merely to teach. One has to punish as well. A little bit of the old pause.
Youngman Grand, Esq.: Cause for pause?
Sir Guy Grand KG, KC, CBE: Yes.
When the flock of birds flies into the air at the grouse shoot, the hand of the person releasing them is visible at the bottom of the screen.
Also released in shortened (75 min. and 88 min.) versions.
Mad About The Boy
By Noël Coward
Performed by Peter Sellers (uncredited)
Barnes, London, England, UK
25 December 2018 | Trailers from Hell
Dracula: Prince of Darkness
02 November 2016 | Rolling Stone
'Blindman': The Story Behind a Drunk Beatle's Spaghetti Western
05 July 2016 | The Film Stage
Directors' Favorite Kubrick Films, Jeff Goldblum on Marc Maron, Todd Solondz Talk, and More
Coolest Looking Hooded Character
1969- Which film holds up the best 45 years later?
Favorite Musical Instrument
Stop the Music!

redpajama_set_name: RedPajamaCommonCrawl


text:
Daily Bayonet
Home Health Care & Medical " Sleeveballoon" A New device that can treat Type 2 diabetes and...
" Sleeveballoon" A New device that can treat Type 2 diabetes and obesity
" Sleeveballoon" A New device that can treat Type 2 diabetes and obesity
Good news for Diabetic patients, a device called "Sleeveballoon" has been developed to treat Type 2 diabetes and obesity. The news about the research done by a team of researchers from the King's College London came out in the journal EBioMedicine.
The name very well suggests what the device is, it has got a Sleeve and a Balloon but what it doesn't talk about is the How?! The Sleeveballoon device imitates the process of bariatric surgery. The study was done by running the experiment on 30 rodents which showed amazing results in reducing both the blood sugar or glucose level and obesity.
Currently, the treatment that exists for the same problem is called Gastric Bypass Surgery and despite being extremely effective only 1% of the people are eligible for it of which most of people avoid undergoing complicated treatments.
The study has found out that the device has certain advantages over Gastric Bypass Surgery although the metabolic effects are the same. While in both the treatments the performance of heart and insulin in the body improved the Sleeveballoon did not cause Hypoglycemia after food intake that the Gastric Bypass Surgery caused. The device slows down the digestion process in a person that makes the person feel full for a longer period reducing his/her appetite.
The device is placed on the small intestine and once operated needs to get removed in 6-12 months. The reversal effects on the body after the removal of the device is being tested now.
30% of the World Population is overweight and suffering from Obesity and the number has only been growing. A serious concern of the World Health Organization and that is the very reason why the discussion over harmful effects of obesity and bad food habits that especially include deep-fried items and fast food are encouraged in schools and social media so that people instead of feeling fat-shamed or bullied prioritize their health first and try to go from fat to fit.
In such a scenario the Sleeveballoon device can prove to be a revolutionary device to counterattack this epidemic.
Previous articleNew Pixel 4 will allow you to photograph the night sky
Next articlePrince Harry Got Emotional Talking About wife Meghan Markle "Pregnancy Secret"
Sumit Pathak Digital marketing expert at OhoSeo, having Knowledge of Website Developer through Wordpress, blogging, Seo. apart from this, I am trying to learn daily bases new thing
The new beam of photons is now unmistakable advantage against cancer
Periodontal Ailment: A Patent for New Treatment Technique
New Alzheimer risk gene discovered: Details inside
Lipsense Ingredients
Hoverboard Charger
Old Theories Of 19th Century Scientist Help Discovering New Light Wave
Prince Garg - September 12, 2019 7:30 pm EDT
With the assistance of pioneering work of a 19th-century scientist, researchers have now found an unknown kind of sunshine waves. The phenomena that were...
'The Black Parade': Gerard Way talks "safety" of playing a character
Ruchika Jain - November 1, 2019 12:00 pm EDT
Gerard Way recently sat down to discuss the complexity of creating great albums like The Black Parade, writing The Umbrella Academy and more. From revealing how Dungeons & Dragons taught...
Weidner Apartment Homes with Alaska invests in homeless solutions
vikas shrivastava - September 30, 2019 2:30 pm EDT
One of Alaska's biggest health care provider biggest private funder, biggest private landlord, and biggest private health insured declare they are promising $40 million...
Ocean's New Song "DHL" is The Biggest Takeaways From Frank
Ruchika Jain - October 21, 2019 1:00 pm EDT
It had been so much too lang syne Frank Ocean discharged new music on a random weekend night unexpectedly and stopped the U.S.A. beat...
Selena Gomez Sparks Engagement Rumors! But Who Is The Lucky Guy?
vikas shrivastava - August 24, 2019 2:58 pm EDT
After Bieber, Salina Gomez is very incredible in the Love-tattle world however think about what the Back to your vocalist has made it back....
This terrefic new smartwatch has Alexa and a battery life that lasts for a...
Prime smartwatches from corporations like Apple and Samsung are tremendous, but they have some big weaknesses that not everyone needs to handle. Here is...
"Death on the Nile" full cast Revealed starring Gal Gadot and Letitia Wright
vikas shrivastava - October 3, 2019 11:00 am EDT
Production has begun on Kenneth Branagh's Murder on the Orient specific sequel, Death on the Nile. Based on the 1937 Agatha Christie novel of...
Exclusive: The Walking Dead' Cast Reacts to Lauren Cohan's Return at New York Comic...
kratika parwani - October 8, 2019 11:30 am EDT
It was an eventful day for The Walking Dead fans. During Saturday's the big apple Comic-Con panel, it absolutely was proclaimed that the AMC...
Sundance Film Festival declares 2020 program
Prashant - January 6, 2020 6:00 pm EST
The 2020 Sundance Film Festival has announced its full film series for the January meeting. The 11-day event will feature Sundown's premieres and 118...
Who is Adele's new celeb boyfriend? Revealed
vikas shrivastava - October 2, 2019 2:45 pm EDT
Adele has been on various dates with Skepta - and pals foresee they could become a pair. The song superstars, who both come from Tottenham,...
© Copyright 2019 - Daily Bayonet


text:
Film Review – 'Blackbird' is a Hit and Miss Family Dramedy
by Chris James September 18, 2020, 1:00 pm 3 Comments
Few things give us more thrills than seeing Oscar winning actresses yell family secrets at a dinner table. No boxing match could ever compare to a perfectly placed wine sip, a callous side eye or a cutting toast. Blackbird gives stars Susan Sarandon and Kate Winslet (both Oscar winners from decades past) plenty of scenes to share. However, the dour dramedy only crackles every few moments. There's a lot to latch onto initially in the film, which features a very strong ensemble that includes Rainn Wilson, Lindsay Duncan and Mia Wasikowska, among others. Yet, like an ill-mannered houseguest, it soon wears out its welcome.
Lily (Susan Sarandon) wants to die. ALS is about to get the better of her and soon she won't be able to breathe or move on her own anymore. She instructs her family to come visit her for one final weekend that will end with her taking her own life. Since this form of assisted suicide is illegal, Lily's husband, Paul (Sam Neill) has taken control of the planning so the family can say goodbye and wipe their hands clean of the incident. Her daughter, Jennifer (Kate Winslet), and son-in-law, Michael (Rainn Wilson), are on board with the plan. However, Jennifer's temperamental daughter, Anna (Mia Wasikowska), has her own plans for how she can hold onto her Grandma for just a little bit longer.
The family affair expands further to include Lily's best friend, Liz (Lindsay Duncan), who delivers the best performance of the film. She's always been Lily's confidant throughout their wild youth and waning older years. Duncan sells the history she has with both Liz and Paul, not just in the dialogue, but in the inside jokes and fleeting glances that only come with decades of lived in history. If anything, their relationship almost felt like a septuagenarian version of Miss You Already, an underrated cancer drama with Drew Barrymore and Toni Collette.
In center stage, Susan Sarandon commands the screen. She reminds audiences that no one can compete with her gravely, yet seductive, line readings. More so than anything, she gives texture to Lily's life, making it all the harder to let her go. You believe in the life she has led because Sarandon sells you on the details of that life.
If only the younger members of the cast and the writing were up to par. It's upsetting to not love a Kate Winslet performance. However, as both the glue of the family and an instigator, Winslet only seems to play the text. She never understands her character on a deeper level. It almost feels like her and Lily are meeting for the first time, rather than a somewhat close mother-daughter pair. Her move from "uptight Mom" to a crucial source of conflict later in the movie never feels earned.
On the opposite end, Mia Wasikowska gives the role her all, but is sabotaged by Christian Torpe's script. She enters the film with her partner, Chris (Bex Taylor-Klaus) in tow. At breakneck speed, we learn that the two have been on again off again for quite some time and that Anna suffers from mental health issues. Perhaps that is supposed to explain her often erratic behavior. Yet, the movie doesn't often think through her motivation so clearly. Thus, her role as a wrench in Lily's live never feel fully formed and come off as forced by a heavy handed script.
Director Roger Michell (Notting Hill) demonstrates an acute visual eye and knack for staging his large cast of actors. As the visitors continue to trickle into the house, we hear the breaking of a glass. The camera moves rather gracefully through the family members and focuses, revealing Lily's grasp had let the glass sip. Lily knows it's a sign the disease is making quick work of her. Sarandon decides to smile and toss off a bon mot. "Well, you know it's time when you have to start drinking your Chablis from a plastic cup." We gather so much about the characters and the stakes in these early moments.
What starts out as a sly and effective family drama becomes overplayed by the end. Why bother establishing vivid characters if you're going to trap them in conventional conflicts midway through. Still, for fans of family dramas and the cast, Sarandon and Duncan in particular, Blackbird makes for a compelling afternoon watch.
SCORE: ★★1/2
After watching her get painfully little to work with in The Devil All the Time, I got really excited for a second when I read that Mia Wasikowska gets to play someone who is both temperamental and has secret motivations. But then I got to the 3rd paragraph from the bottom, and now I'm starting to think this hasn't been her best week…
Joey Magidson
Reply to Ryan
For what it's worth…Chris actually liked this one more than I did.
Hey Ryan! I think Mia Wasikowska is a great actress and has many fantastic performances across different genres. I don't know what's happened to such a bright talent, but this is far from her finest hour. Hopefully she finds another vehicle worthy of her considerable talents.
BlackbirdKate WinsletLindsay DuncanMia WasikowskaRainn WilsonReviewRoger MichellSam NeillSusan Sarandon
Written by Chris James
Previous article Montclair Film Festival Announces 2020 Special Event Screenings
Next article Pixar: Making Adults Cry Since 1986
Montclair Film Festival Announces 2020 Special Event Screenings
Pixar: Making Adults Cry Since 1986


The less your body breathes, the more it sweats, and the slower you perform. When you want the power of cotton canvas without a heavy, dragging feeling, these shorts sit right in the middle with a soft, breathable design that doesn't cut short its strength. Further considering your needs on the job, several pockets and belt loops on the outside give you a place to hold onto tools, a phone, or a wallet.


From the pages of sc-fi storylines, artificial intelligence (AI) is now being realized as a valuable tool in the home, commercial verticals, and industrial applications. Analysts from Gartner Research suggest that AI is in the innovation trigger phase of emerging technologies, and is poised to become incredibly influential. "The ability to use AI to enhance decision making, reinvent business models and ecosystems, and remake the customer experience will drive the payoff for digital initiatives through 2025," according to Gartner Research's October 2017 report.
One of the companies helping clients harness the power of AI is AdMobilize. The vision of AdMobilize is to quantify and make sense of the physical world with real-time analytics and data. To put it into a context everyone can understand, "Think about AdMobilize almost like the Google Analytics for the physical world," said Rodolfo Saccoman, the company's cofounder/ CEO. AdMobilize's technology can capture data in real-time via any camera stream. The solution then runs its advanced computer vision algorithms to analyze what the cameras capture. It's all anonymous, private data that collects on smart dashboards within 15 milliseconds.
While AI is still emerging within the ProAV landscape, it's not new. Many automation systems utilize smart sensors and AI of some ilk. Alexa for Business was recently introduced with partners such as Teem. IBM Watson and Ricoh offer co-developed AI-enhanced smardboards, and Harman debuted Watson-enabled cognitive rooms last year. Saccoman believes that the demand is growing for AI, and integration with ProAV is a natural fit. In any physical space—lobbies, elevators, conference rooms, transport hubs, digital signage, out-of-home-advertising (OOH), retail, and college campuses—artificial intelligence can be a valuable tool to better understand the behavior of customers, vehicles, or crowds.
Another example is crowd intelligence. AdMobilize's scalable solutions can provide facility directors with crowd analytics within lobbies to better understand the traffic, the types of demographics, and traffic flow. In a parking lot, as another example, machine learning algorithms can be applied to track how many cars move past as well their makes, styles, and speeds.
AdMobilize has sold its AI technology in more than 60 countries, and integrators are working closely with their computer vision tools to help end users better understand their environments. Information and instant detection data sets can include gender, age brackets, emotions while looking at an image a particular digital screen, or how visitors interact with screens. Saccoman added that this type of dynamic intelligence can help create intuitive dashboards with the ability to extract actionable reports.
AdMobilize provides the AI platform for integration within a ProAV application giving integrators a recurring revenue stream via data sets and data tools. "[We] allow for them to have significant recurring revenue, and without a lot of expenses upfront. Because everything we have in our technology works pretty much with any hardware, any camera, basically any install," Saccoman clarified. The platform works with Android, iOS, Windows, Raspberry, and Linux.
Privacy and Ethics of ProAV+AI: A Slippery Slope?
Many people believe that technology is already too intrusive in our lives, schools, and workplaces, from Google crawling emails to Facebook serving up customized likes to "friends." Many citizens, this author included, feel that we're already being closely monitored. How much data is too much?
Saccoman further explained that his solution is actually more private than Google or Facebook, as they understand exactly who you are and therefore serving ads and products based on your gender, buying patterns, and identification info.
Anytime connected audiovisual feeds run over IP, data security is paramount. What should AV stakeholders keep in mind in terms of security with integrated AI tools?
The global appetite is increasing for AI-fueled systems, but not all misconceptions about have been dispelled.
Saccoman added that AI is another compelling tool in the smart city and smart building proposition. As we see audiovisual systems continue to converge with scalable control systems and IoT sensors, AI can offer specific and actionable insights that help facility managers "get more value" from their space, he said.