text:
\section{Introduction}
Stereo matching is a fundamental task in computer vision which consists in
reconstructing the 3D environment captured from a pair of cameras.
Specifically, stereo matching refers to the process of computing the dense pixel correspondence between a rectified image pair~\cite{geometry}.
This correspondence is expressed as the horizontal pixel distance from one pixel on the left image to its corresponding position in the right image.
This distance is known as \emph{disparity}. This disparity can be estimated for each pixel in the image, leading to a disparity map which can be utilized for various computer vision tasks, such as semantic segmentation~\cite{segstereo,semantic-costV}, stereo-LiDAR fusion~\cite{stereolidar-jaesungchoe}, and 3D object detection tasks~\cite{pseudo_lidar,monodepthdetection,pseudo_lidar++}.

Recently, stereo-vision has benefited from the advances of Convolutional Neural Networks (CNNs)~\cite{alexnet,vgg,resnet}.
Pioneering deep learning-based patch matching approaches, such as~\cite{mccnn,dispnet}, largely outperform traditional hand-crafted techniques~\cite{semi_global_matching}.
Follow-up deep learning-based solutions, inspired by conventional stereo matching techniques~\cite{stereo-taxonomy}, explicitly incorporate geometric information in the matching process~\cite{gcnet}, which produces
significant accuracy improvements.
For example, Kendall~\emph{et al}\onedot~\cite{gcnet} introduce a differentiable cost volume to compute matching costs between the left and right features from the stereo images.
With its larger receptive field and larger capacity, this cost volume approach has demonstrated highly accurate results. This seminal work has inspired various methods that rely on cost volume~\cite{gcnet,psmnet,ganet,cspn}.

\begin{figure}[!t]
\centering
\includegraphics[width=0.65\linewidth]{figure/mainFig/fig_main_20.pdf}
\vspace{-2mm}
\caption[Caption for LOF]{\textbf{Overview.} Given (1)~stereo images,
we extract the (2)~left/right feature maps~($\mathcal{F}_{L}, \mathcal{F}_{R}$)
to construct the (3)~cost volume $\mathcal{V}$.
Our cost volume is aggregated to~(4) estimate the disparity map
and to (5)~detect 3D object.
}
\label{fig:main}
\vspace{-5mm}
\end{figure}

Despite the improvements arising from CNNs, multiple challenges still remain. Particularly, textureless or repetitive structures in the image lead to inaccurate disparity estimation.
Moreover, objects' boundaries tend to be ambiguous.
To overcome these issues, several methods apply multi-task losses using additional information, such as image segmentation~\cite{semantic-costV}.
These approaches rely on 2D contextual information extracted from the image. These 2D cues improve the matching quality but are inherently ambiguous since contiguous pixels in the image space are not necessarily neighbors in 3D.
In contrast, considering an object prior in 3D can reduce the object-level spatial ambiguity effectively. Motivated by this observation, we investigate how such a geometric property can be integrated into a stereo matching network.

In this paper, we propose a stereo matching network that utilizes 2D information as well as 3D contextual information of objects. We named this method \emph{stereo object matching network} (see \Fref{fig:main}).
Our method builds on a cost volume-based approach~\cite{psmnet}, which estimates a depth map using 2D contextual information and includes an implicit 3D space structure via the cost volume.
Concretely, we perform a 3D object detection task, as an auxiliary task, using the cost volume to guide the network to learn meaningful structural information about the objects in the scene.
This strategy reduces the uncertainty of the disparity boundary near the objects (\emph{e.g}\onedot} \def\Eg{{E.g}\onedot,~vehicle).
Therefore, the cost volume in our network considers both conventional disparity loss in a pixel-wise manner and detection loss from the 3D RoIs which are extracted from the cost volume.
Each element in the multi-task losses has geometric-awareness and makes it possible to achieve state-of-the-art performance for depth estimation in the KITTI dataset~\cite{kitti} and Virtual-KITTI dataset~\cite{virtual-kitti}.

\section{Related work}

Stereo matching has been deeply investigated in the past decades.
Recently, deep learning techniques paved the way for more effective and robust matching.
The work by Zbontar and LeCun~\cite{mccnn} is the first technique to employ deep-patch-based stereo matching and
Mayer~\emph{et al}\onedot~\cite{dispnet} introduce an end-to-end architecture to regress the disparity.
More effective and elegant approaches inspired by traditional matching techniques have been developed. It is, for instance, the case of Kendall~\emph{et al}\onedot~\cite{gcnet} who propose to integrate a differentiable cost volume into a deep neural architecture.
Relying on this concept, several works enhance the deep architecture by widening the receptive field using a pyramid network~\cite{psmnet},
exploiting the semi-global cost aggregations~\cite{ganet},
and propagating the spatial pixel-affinity modules~\cite{cspn}.
However, the objects' boundary ambiguity is a challenging issue
and such ambiguity results in a low-quality depth prediction on these areas.
Recent works~\cite{segstereo,semantic-costV,miclea2019real,sun2020real}
integrate semantic segmentation into the disparity estimation to handle this ambiguity issue, but the resulting improvements remain relatively limited due to the inherent ambiguity of this semantic two-dimensional information.

Meanwhile, camera-based 3D object detection increases the performance by exploiting high-quality depth from deep learning-based approaches~\cite{monodepth,dispnet}.
Xu and Chen~\cite{monodepthdetection} propose a multi-level fusion technique and Wang~\emph{et al}\onedot~\cite{pseudo_lidar} convert the depth into pseudo lidar data for vehicle localization.
These approaches demonstrate that the quality of the depth affects the performance of the detection. However, they only utilize depth information as an additional modality in a naive manner (\emph{e.g}\onedot} \def\Eg{{E.g}\onedot,~concatenate with input).

In this study, we directly handle the 3D contextual information by exploiting 3D object detection as an auxiliary task.
Our method utilizes 3D objects' structural information (\emph{e.g}\onedot} \def\Eg{{E.g}\onedot,~size and heading) so that our cost volume is trained to express the detection-based 3D contextual information.
By our cost volume-based detection network,
the proposed approach largely increases the quality of estimated depth.

\begin{figure}[t]
\centering
\includegraphics[width=1.0\linewidth]{figure/archFig/fig_architecture_05.pdf}
\caption{\textbf{Illustration of stereo object matching network.}
Our network consists of three parts: (1) cost volume prediction, (2) 3D object detection, and (3) disparity regression.
The refined cost volume $\mathcal{V}$ is utilized to regress disparity $\widetilde{\mathcal{D}}$ and to detect the 3D objects by multi-task losses that simultaneously affect cost volume.
We visualize the attention of features ($\mathcal{F}_{L}$, $\mathcal{F}_{R}$, $\mathcal{V}$) and cost aggregation $\mathcal{A}$.
We also illustrate the location of the 3D RoI ($\{ \mathbf{p}_{min}, \mathbf{p}_{max} \}$) by the 3D-RPN as the red 3D box in $\mathcal{V}$.
}
\label{fig:architecture}
\end{figure}

\section{Stereo Object Matching}

In this section, we propose a stereo object matching network, which consists of three steps: cost volume prediction, disparity regression, and 3D object detection. Each module of the pipeline is introduced individually, and the overall architecture is depicted in~\Fref{fig:architecture}.

\subsection{Overview}

Given a rectified stereo pair, we propose a disparity estimation method, called \emph{stereo object matching network}, which considers 2D contextual information as well as 3D object-level information. We first extract image features from stereo images and then construct a cost volume~(\Sref{subsection : cost volume}).
The conventional cost volume is used to predict the disparity map out of 2D image information only.
However, since the image features are concatenated and stacked for each disparity hypothesis, this cost volume follows the 3D voxel structure with feature vectors, \emph{i.e}\onedot} \def\Ie{{I.e}\onedot, channel~(C)$\times$disparity~(D)$\times$height~(H)$\times$width~(W).

In this work, we attempt to fully exploit this 3D voxel structure not only to compute \FR{the} disparity but also to embed 3D object-level information via an auxiliary 3D detection task (we discuss the details of how we embed 3D object-level contextual information in \Sref{section : Detection-based 3D contextual embedding}).
It allows us to utilize both 2D contextual and 3D object-level information, which produces accurate disparity estimation~(\Sref{subsection : disparity network}).

\subsection{Cost volume}
\label{subsection : cost volume}

From rectified stereo images, we first extract the stereo image feature maps, \emph{i.e}\onedot} \def\Ie{{I.e}\onedot, left feature map~$\mathcal{F}_L$ and right feature map~$\mathcal{F}_R$ whose size is 16$\times$H$\times$W using the feature extraction layer in PSMNet~\cite{psmnet}.
A strong similarity between two feature vectors from each feature map indicates a high matching probability,
meaning that two pixels corresponding to the two feature vectors are the projection of the same 3D point in the scene.

The initial cost volume~$\mathcal{V}_{init}$ is built as follows~\cite{gcnet,psmnet}:
(1)~shifting right image features horizontally with fixed left image features (each shift corresponding to a candidate disparity value),
(2)~stacking the shifted right and fixed left features across each disparity level,
and (3)~concatenating the left and right features along the channel axis.
This initial cost volume is then refined via stacked hourglass networks~\cite{psmnet,associate_embedding,hourglass}. These stacked hourglass networks are important to regularize the initial cost volume and to increase the receptive field of the network.
The constructed cost volume~$\mathcal{V}$ forms a 4D volume\footnote{Although the physical dimension of $\mathcal{V}$ is ``C$\times$D$\times$H$\times$W'', we can consider $\mathcal{V}$ as a 3D volume with its size ``D$\times$H$\times$W'', where the C-dimensional feature vector is stored at each voxel's location $(u,v,d)$. Thus, we interchangeably describe $\mathcal{V}$ as 3D volume or 4D volume in some cases.}, {which describes the 3D space along the disparity axis, and where high responses are clustered object-wise (see the visualization of $\mathcal{V}$ in \Fref{fig:architecture}).}
This cost volume is the central element of our system since it is explicitly used for both disparity estimation and 3D object detection, especially as an intermediate space to embed 3D object-level information.

\subsection{Disparity network} \label{subsection : disparity network}

Given the learned cost volume $\mathcal{V}$ that embeds both 2D and 3D contextual information, the disparity network aims to regress the disparity values at a subpixel level.
Following~\cite{gcnet}, we perform a cost aggregation process by aggregating the cost volume along the disparity dimension as well as its spatial dimensions.
This cost aggregation reduces the dimensionality of $\mathcal{V}$ into a 3D structure $\mathcal{A}$.
From $\mathcal{A}$, we can estimate the disparity value $\tilde{d}_{u,v}$ at pixel $(u,v)$ as: \vspace{-2mm}
\begin{equation}
    \tilde{d}_{u,v} = \sum_{d=0}^{d_{max}} d \cdot \sigma ({\mathbf{a}}_{u,v}^d),
    \label{equation:intra-pixel}
    \vspace{-2mm}
\end{equation}
where $d_{max}$ indicates the maximum disparity, $\sigma(\cdot)$ denotes softmax operation,
and ${\mathbf{a}}_{u,v}^d$ is the response for disparity value $d$
of the cost aggregation vector ${\mathbf{a}}_{u,v}$ $\in \mathbb{R}^{d_{max}}$ at $(u,v)$.

We compute the disparity loss~$\mathcal{L}_{disp}$ from the estimated disparity map~$\tilde{\mathcal{D}}$ and the ground truth disparity map~$\mathcal{D}$ as: \vspace{-1mm}
\begin{equation}
    \mathcal{L}_{disp} = \frac{1}{N}\sum_{u}\sum_{v} smooth_{L_{1}}({d}_{u,v} - \tilde{d}_{u,v}),
    \vspace{-2mm}
    \label{equation:disp-loss}
\end{equation}
where $N$ is the number of the pixels in $\mathcal{D}$, $\tilde{d}_{u,v}$ is the value of the predicted disparity map $\widetilde{\mathcal{D}}$ at the pixel $(u, v)$,
and $smooth_{L_{1}}(\cdot)$ is the smooth L1 loss function used to compute the loss~\cite{gcnet,psmnet}.

\section{Detection-based 3D contextual embedding}
\label{section : Detection-based 3D contextual embedding}

In this section, we propose a cost volume-based 3D object detection, as an auxiliary task to embed 3D object information.
We exploit an anchor-based detection framework~\cite{faster_rcnn} composed of two stages -- a 3D region proposal network (3D-RPN) and a header network.
While previous techniques have been developed for 2D input~\cite{stereo_rcnn,mutlibox3d}, our 3D object detection uses cost volume as input.
Directly using the cost volume for 3D object detection is meaningful since it encapsulates both 2D and 3D information; however, it remains a complex task.
The challenges of using such a structure to perform 3D object detection, as well as the proposed solutions (selective subpixel sampling and 2D-3D fusion), are discussed below.

\begin{figure}[t]
\centering
\includegraphics[width=1.00\linewidth]{figure/fusionFig/fig_fusion_11.pdf}
\vspace{-3mm}
\caption{\textbf{Illustration of the RoISelect and the fusion-by-occupancy.}
    Given the cost volume $\mathcal{V}$, the estimated disparity $\widetilde{\mathcal{D}}$, and the extracted 3D RoI $\{ \mathbf{p}_{min}, \mathbf{p}_{max} \}$ by 3D-RPN,
    RoISelect estimates the subsampled 2D and 3D RoIs: 2D RoI $\mathcal{R}_{2D}$ (in the estimated disparity map) and 3D RoI $\mathcal{R}_{3D}$ (in the cost volume),
    where we deal with unreliable 2D pixel/3D voxel (\emph{e.g}\onedot} \def\Eg{{E.g}\onedot, background or occlusion region) by filling the corresponding location as zero.
    For the estimated 2D RoI $\mathcal{R}_{2D}$, we apply the fusion-by-occupancy that estimates 3D occupancy RoI by back-projecting $\mathcal{R}_{2D}$ into 3D RoI aligned with $\mathcal{R}_{3D}$.
    After RoISelect and fusion-by-occupancy, we fuse the 3D RoI and the 3D occupancy RoI using the normalization layers~\cite{batch_norm,inst-norm}. The fused 3D RoI is fed into the header network.
}
\label{fig:fusion}
\vspace{-1\baselineskip}
\end{figure}

\subsection{Metric-driven cost volume}
\label{subsection:metric-driven 3d structure embedding}

By virtue of its structure (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot,~3D volumetric space along the disparity axis), the cost volume naturally encodes the 3D space.
However, utilizing this volume as a direct input for 3D object detection is not straightforward for two major reasons.
1)~The disparity is represented in the discrete pixel domain, which is not linearly correlated with metric depth.
For example, a disparity error of one pixel for nearby objects represents a metric error of a few centimeters, while the same discrepancy for faraway objects can result in a significant error (up to a few meters depending on the camera resolution and baseline's length). In other words, an extracted anchor corresponding to a distant object may include only a small region in the cost volume while a near object of physically similar size will appear significantly larger.
2)~A~sampled cost volume usually includes uncertain matching costs due to occlusions or the limited overlapping area between both views.
{Valid matching costs (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot,~high confidence) only exist where the camera can see. That is, we cannot compute valid matching costs beyond the visible surface.}
Therefore, wrongly estimated matching costs on an occluded region might confuse the network to incorrectly localize the objects, which affects the cost volume during training.

To resolve these issues on the cost volume, we propose two strategies (see \Fref{fig:fusion}):
1)~A selective subsampling method, called \emph{RoISelect}, with the aid of subpixel interpolation,
and 2)~2D-3D fusion using occupancy reconstruction, called \emph{fusion-by-occupancy} to inject the continuous {depth} values into the discrete 3D RoI from the cost volume.

\vspace{1mm} \noindent \textbf{Selective subpixel sampling (RoISelect).} \
In the anchor-based detection framework,
since 3D-RPN extracts 3D RoI using pre-defined anchors, the sampling process is essentially required to normalize the RoI to a common size.
In particular, due to the above mentioned potential issues of the cost volume, extracted 3D RoI in the cost volume space has a highly varying size
and uncertain matching costs depending on the location and occlusion.
Thus, we propose a selective subsampling method, called \emph{RoISelect}, which selects high confidence voxels using the estimated disparity map~(\emph{cf}\onedot} \def\Cf{{Cf}\onedot,~\Sref{subsection : disparity network}) and interpolates (samples) them while considering the continuous metric range as depicted in Figs.~\ref{fig:sampling}--(\textcolor{blue}{b},~\textcolor{blue}{c}).
Our subsampling method functions in a way similar to a linear interpolation
(\emph{e.g}\onedot} \def\Eg{{E.g}\onedot,~\emph{RoIAlign}~\cite{mask_rcnn}).
However, we broaden the receptive field of sampling by applying the cubic interpolation only to the disparity axis,
called a deep sample module as in~\Fref{fig:sampling}--(\textcolor{blue}{b}).
In addition, we filter the low confidence voxels that belong to the background disparity information, as shown in~\Fref{fig:sampling}--(\textcolor{blue}{c}). For instance in~\Fref{fig:sampling}--(\textcolor{blue}{c}), the top-right voxel is determined to be omitted because disparity map implies that the high confidence voxels are located at the background area (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot, out of the 3D RoI).
In this case, the omitted voxels do not propagate the gradient to the cost volume (similar to ReLU~\cite{deep-learning-book}) so that we can regularize the cost volume along the disparity-axis\footnote{Details of \emph{RoISelect} is described in the supplementary material.}.
Thanks to our {\emph{RoISelect}}, we can selectively subsample the 3D RoIs that contain the high confidence voxels from the cost volume.

\begin{figure}[!t]
    \vspace{-2mm}
    \centering
    \includegraphics[width=0.99\linewidth]{figure/RoIDenseFig/fig_RoISelect_08.pdf}\\
    \vspace{-3mm}
    \caption{\textbf{Illustration of the RoI sampling strategies.}
    Compared to (a) trilinear interpolation, our \emph{RoISelect} consists of (b) Deep sample module and (c) Selective sample module,
    This is to sample the valid voxels in cost volume by using disparity.}
    \label{fig:sampling}
\vspace{-1\baselineskip}
\end{figure}

{

}

\vspace{1mm} \noindent \textbf{2D-3D fusion~(fusion-by-occupancy).} \
Fusion with metric measurement~(\emph{e.g}\onedot} \def\Eg{{E.g}\onedot, depth) is widely used in camera-based 3D object detection methods~\cite{monodepthdetection,seg2reg,roi-10d}.
These approaches blend 2D contextual features with depth using the multi-level injection~\cite{monodepthdetection} or normalization layers~\cite{seg2reg} to accurately localize the target objects.
For our cost volume, it is necessary to match the dimensions of both 3D RoI~$\mathcal{R}_{3D}$ (from the cost volume) and 2D RoI~$\mathcal{R}_{2D}$ (from the estimated disparity map) for aligned fusion.
There are several ways to encapsulate the estimated depth into the 3D structure.
We propose fusion-by-occupancy to properly fuse the depth into the 3D feature.

As illustrated in~\Fref{fig:fusion},
we back-project the 2D RoI~$\mathcal{R}_{2D}$ using its depth measurements into the 3D space as a binary representation of the object surface, \emph{i.e}\onedot} \def\Ie{{I.e}\onedot, 3D occupancy RoI.
In detail, we assign 1 for the back-projected voxels in the 3D occupancy RoI and fill other voxels with 0.
By doing so, we only propagate the backward gradient through the valid voxels in the 3D RoI. This is to regularize the proper voxels in the cost volume along the disparity axis, as \emph{RoISelect} does.
By reconstructing the 3D surface through our fusion-by-occupancy, we preserve the valid feature voxels within the 3D RoI.
After the reconstruction, we fuse the 3D RoI~$\mathcal{R}_{3D}$ with the 3D occupancy RoI using the normalization layers.
With fusion-by-occupancy, we can align the dimensions of the 2D depth and the 3D feature. Accordingly, we can increase the accuracy of the 3D object detection and the quality of the depth estimation. We validate the fusion-by-occupancy in an ablation study~(see~\Tref{table:ablation-fusion}).

\subsection{Cost volume-based 3D object detection}
\label{subsection : Cost volume-based 3D object detection}
We adopt the anchor-based detection (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot,~two-stage detection) as an auxiliary task to guide the 3D contextual information to the cost volume through the pre-defined anchors.
In addition, the subsequent header network provides more detailed contextual information, such as the heading and the size of vehicles.

\vspace{1mm} \noindent \textbf{3D region proposal network.} \
To seamlessly connect the pre-defined anchors with our cost volume~$\mathcal{V}$, we exploit a 3D region proposal network (3D-RPN) defined in the cost volume coordinate, \emph{i.e}\onedot} \def\Ie{{I.e}\onedot, $(u, v, d)$ coordinate.
Given the cost volume~$\mathcal{V}$, the 3D-RPN is trained to (1)~predict the potential location of the objects with respect to the pre-defined anchors, and (2)~classify the foreground anchors.
The 3D-RPN loss $\mathcal{L}_{rpn}$ consists of the two typical losses from the 3D-RPN~\cite{faster_rcnn} as: \vspace{-1mm}
\begin{equation}
    \mathcal{L}_{rpn} = \mathcal{L}_{class} + \mathcal{L}_{anc},
    \vspace{-1mm}
    \label{equation:rpn-loss}
\end{equation}
where $\mathcal{L}_{class}$ is the binary cross-entropy loss for the foreground classification and $\mathcal{L}_{anc}$ is the anchor localization loss.
The 3D-RPN estimates the location of the 3D RoIs $\{ \mathbf{p}_{min}, \mathbf{p}_{max} \}$ in the cost volume space, as shown in~\Fref{fig:architecture}.

\vspace{1mm} \noindent \textbf{Header network.} \
Given $\mathcal{R}_{3D}$ with a fixed size (by RoISelect and fusion-by-occupancy), we design the header network to predict the center location $(u_{obj}, v_{obj}, d_{obj})$, size $(w_{obj}, h_{obj}, l_{obj})$, and heading $(yaw_{obj})$ of the objects.
The header network is trained to regress the objects' parameters with respect to the location of the 3D RoIs.

We follow up the overall architecture proposed by Mousaivan~\emph{et al}\onedot~\cite{mutlibox3d} and Xu and Chen~\cite{monodepthdetection}, except for the heading prediction.
Recent camera-based 3D object detection methods~\cite{mutlibox3d,monodepthdetection} adopt the alpha observation (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot, local orientation)~\cite{kitti,mutlibox3d} as the heading of the vehicles.
Because their feature representations is limited to the 2D contextual attentions~(C$\times$H$\times$W), the same headed vehicles can be differently visualized depending on their distance to the camera~\cite{mutlibox3d}, which shows a tendency similar to the alpha observation.
However, in our case, the cost volume inherently expresses the 3D structures along the disparity axis.
Thus, we estimate the original heading of the vehicle (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot,~yaw rotation) from the 3D RoIs, instead of the alpha observation. We cover the details of the architecture of the header network in the supplementary material.

Finally, our cost volume is additionally trained by the loss from the detection networks.
The loss function of our stereo object matching network is:\vspace{-1mm}
\begin{equation}
    \mathcal{L}_{total} = \mathcal{L}_{disp} + \mathcal{L}_{rpn} + 2 \cdot \mathcal{L}_{header},
    \vspace{-1mm}
    \label{equation:total-loss}
\end{equation}
where $\mathcal{L}_{header}$ is the loss from the header network, in which we apply (1)~the L1-loss for the regression of the size, heading, and the center location of the objects, and (2)~the binary cross-entropy loss for the confidence prediction.
We further describe our loss design in the supplementary material.

\section{Experimental Result}
We describe the training process using the KITTI dataset~\cite{kitti} and evaluate the accuracy of the depth~(disparity) and detection against a large panel of existing techniques in the KITTI dataset~\cite{kitti} and Virtual-KITTI 2.0 dataset~\cite{virtual-kitti}.
Note that we carefully filtered out the training data to ensure a fair comparison with the depth estimation networks~\cite{eigen_mono_depth,monodepth,psmnet,pseudo_lidar}.
In addition, we conduct an ablation study to validate the different modules of our method.

\subsection{Implementation}
\label{subsection:implementation}
Our training scheme consists of three steps.
First, we pre-train the cost volume prediction part and the disparity regression network with Scene Flow dataset~\cite{dispnet} by setting the learning rate to 0.001 during the 15 epochs.
Second, we continue to train the cost volume prediction network and the disparity regression network
with the KITTI-split by Eigen~\emph{et al}\onedot~\cite{eigen_mono_depth} by setting the learning rate to 0.0005 for 30 epochs. More details are included in the supplementary material.

\begin{comment}
To guarantee a fair comparison, we exclude the part of the training data that are included in the KITTI object validation dataset~\cite{kitti-object} and the KITTI stereo train-val dataset~\cite{kitti-stereo}.
Moreover, we use identical/similar hyper-parameters that recent methods used~\cite{psmnet,pseudo_lidar,DSGN}: the dimensions of cost volume ($C({=}48) \times D({=}48) \times H \times W$). For training, we apply random crop augmentation so the resulting size of input images is $256 \times 512$. For inference, we use the original size of the input images as the recent methods do~\cite{psmnet,pseudo_lidar,DSGN}.
Last, we finetune the whole network~(including the 3D-RPN and header network) with the KITTI object training dataset~(learning rate is set as 0.0001 for 20 epochs).
Since the object's label information, such as the heading and the size, is only provided within the KITTI object benchmark, the last training phase is the key to our training session.

In Virtual-KITTI 2.0 dataset~\cite{virtual-kitti}, we follow the evaluation scheme in Cabon~\emph{et al}\onedot~\cite{virtual-kitti}; we do not finetune the network and exploit the pre-trained weights on the KITTI dataset. Typically, we use part of the data (15-deg-left) for depth evaluation, totally 2,303 images, to test the generalization performance of the network.
\end{comment}

{

}
\begin{table}[t!]
\centering
\caption{\textbf{Quantitative results for depth estimation in the KITTI split by Eigen~\emph{et al}\onedot~\cite{eigen_mono_depth}.}
Stereo matching techniques usually provide disparity estimation (not depth),
while monocular-based studies are mainly evaluated for depth estimation.
Thus, we provide comparisons on both monocular and stereo approaches for a complete evaluation on a unified unit -- depth metric.
* indicates the reproduced results.
M and S mean mono camera and stereo camera, respectively.
}
\vspace{-2mm}
\resizebox{0.935\linewidth}{!}{
\begin{tabular}{|c|c|c|c|c|}
\hline
\multirow{3}{*}{Method} & \multicolumn{1}{c|}{\multirow{3}{*}{Modality}} & \multicolumn{3}{c|}{Depth Evaluation} \\
& \multicolumn{1}{c|}{} & \multicolumn{3}{c|}{(Lower the better)}\\ \cline{3-5}
& \multicolumn{1}{c|}{} & Abs Rel & Sq Rel & RMSE\\ \hline
Eigen~\cite{eigen_mono_depth} & M & 0.203 & 1.548 & 6.307 \\ \hline

DORN~\cite{dorn} & M & 0.072 & 0.307 & 2.727 \\ \hline\hline

PSMnet~\cite{psmnet}* & S & 0.053 & 0.234 & 2.847 \\ \hline

PseudoLiDAR~\cite{pseudo_lidar} & S & 0.052 & 0.281 & 3.027 \\ \hline

DSGN~\cite{DSGN} & S & 0.064 & 0.184 & 2.942 \\ \hline

Ours & S & \textbf{0.027} & \textbf{0.111} & \textbf{1.842} \\ \hline
\end{tabular}
}
\label{table:kitti-eigen-metric}
\end{table}

\begin{comment}
\begin{table}[t!]
\centering
\caption{\textbf{KITTI stereo benchmark.}
Among the recent stereo matching methods that use the similar backbone architecture~\cite{psmnet}, our method shows the lower accuracy in disparity evaluation.
However, the notable meaning is that \FR{remove "the notable meaning is that"} our method typically \FR{remove "typically"} shows the higher accuracy in D1-bg, which means that our method \FR{"strategy" instead of "method"} is specialized in inferring the disparity at the farther areas.
}
\vspace{-2mm}
\resizebox{0.82\linewidth}{!}{
\begin{tabular}{|c|c|c|c|c|}
\hline
\multirow{4}{*}{\text{ } \text{ } Method \text{ } \text{ }} & \multicolumn{3}{c|}{Disparity Evaluation} \\
& \multicolumn{3}{c|}{(Lower the better)} \\ \cline{2-5}
& \multicolumn{2}{c|}{All $(\%)$} & \multicolumn{2}{c|}{Noc $(\%)$} \\ \cline{2-5}
& \text{ } D1-bg \text{ } & \text{ } D1-all \text{ } & \text{ } D1-bg \text{ } & \text{ } D1-all \text{ } \\ \hline
GCnet~\cite{gcnet}         & 2.21 & 2.87 & 2.02 & 2.61 \\ \hline
PSMnet~\cite{psmnet}       & \textbf{1.86} & 2.32 & 1.71 & 2.14 \\ \hline
Ours                       & 1.88 & 3.05 & \textbf{1.63} & 2.73 \\ \hline
\end{tabular}
}
\label{table:disparity-metric}
\end{table}
\end{comment}

{

}
\begin{table}[t!]
\centering
\caption{\textbf{Quantitative results for depth estimation in Virtual-KITTI dataset.}
* means the reproduced results. M and S indicate mono camera and stereo camera, respectively.
}
\vspace{-2mm}
\resizebox{0.935\linewidth}{!}{
\begin{tabular}{|c|c|c|c|c|}
\hline

\multirow{3}{*}{Method} & \multirow{3}{*}{Modality} & \multicolumn{3}{c|}{Depth Evaluation} \\

& \multicolumn{1}{c|}{} & \multicolumn{3}{c|}{(Lower the better)} \\ \cline{3-5}

& \multicolumn{1}{c|}{} & Abs Rel & Sq Rel & RMSE \\ \hline

PSMnet~\cite{psmnet}* & S & 0.050 & 0.517 & 4.909 \\ \hline

PseudoLiDAR~\cite{pseudo_lidar}  & S & 0.080 & 1.044 & 5.137 \\ \hline

DSGN~\cite{DSGN}  & S & 0.059 & \textbf{0.503} & 4.748 \\ \hline\hline

Ours & S & \textbf{0.044} & 0.557 & \textbf{4.619} \\ \hline

\end{tabular}
}
\vspace{-1\baselineskip}
\label{table:virtual-kitti-metric}
\end{table}

{

}
\begin{table}[t!]
\centering
\caption{\textbf{Quantitative results for 2D and 3D detection in the KITTI object validation benchmark.}
Though our method focuses on depth estimation, we provide the accuracy of our detection results (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot,~3D-RPN and header).
Our method shows relatively lower accuracy compared to the recent methods.
Our reasoning is that the cost volume encompasses the 3D space in disparity axis,
which is the weak point in detection 3D object,
while the similar approach~\cite{DSGN} transforms the cost volume into the metric space volume and achieves the highest accuracy in 3D detection task.
M and S mean mono camera and stereo cameras, respectively.}
\resizebox{1.0\linewidth}{!}{
\begin{tabular}{|c|c|c|c|c|}
\hline

\multirow{2}{*}{Method} & \multirow{2}{*}{Modality} & $\text{AP}_{\text{2D}}$ & $\text{AP}_{\text{BEV}}$ & $\text{AP}_{\text{3D}}$ \\

& & (Moderate) & (Moderate) & (Moderate) \\ \hline
Multi-Fusion~\cite{monodepthdetection} & M & -  & 13.63 & 5.69 \\ \hline

M3D-RPN~\cite{mono_rpn}                & M & 83.67 & 21.18 & 17.06 \\ \hline\hline
3DOP~\cite{stereo3d}                   & S & 88.07 & 9.49 & 5.07 \\ \hline
TLnet~\cite{triangulation_detection}   & S & - & 21.88 & 14.26 \\ \hline
StereoRCNN~\cite{stereo_rcnn}          & S & \textbf{88.27} & 48.30 & 36.69 \\ \hline
PseudoLiDAR~\cite{pseudo_lidar}        & S & - & 56.80 & 45.30 \\ \hline
DSGN~\cite{DSGN}                       & S & 83.59 & \textbf{63.91} & \textbf{54.27} \\ \hline\hline
Ours                                   & S & 76.71 & 30.92 & 19.88 \\ \hline
\end{tabular}
}
\label{table:kitti-object-metric}
\end{table}

\begin{figure*}[!t]
\centering
\includegraphics[width=0.85\linewidth]{fig_depth/fig_depth_00.pdf}
\vspace{-3mm}
\caption{\textbf{Qualitative results of ours and the related works~\cite{DSGN,pseudo_lidar++} on the KITTI dataset.}
}
\label{fig:fig-depth}
\vspace{-1\baselineskip}
\end{figure*}
{
}

\subsection{Evaluation} \label{subsection:evaluation}
\noindent \textbf{Metric.} \
We quantitatively evaluate the proposed method for depth estimation (disparity) and 3D object detection.
For the depth estimation, we utilize the depth metric proposed by Eigen~\emph{et al}\onedot~\cite{eigen_mono_depth}.
For the disparity evaluation, we employ the official disparity metric from the KITTI stereo benchmark~\cite{kitti-stereo}.
As for the detection evaluation, we follow the rules provided by the KITTI object benchmark~\cite{kitti-object}, where we only consider the single label (vehicle object).
The detection evaluation is divided into three cases: easy, moderate, and hard.
We measure the Average Precision~(AP) of the 3D detection and the bird's eye view
(BEV) detection, where the threshold of the intersection of union~(IoU) is $0.7$.

We compare our algorithm with the recent deep learning-based depth estimation methods~\cite{monodepth,cspn,pseudo_lidar,psmnet} (both monocular-based and stereo-based methods) as shown in~\Tref{table:kitti-eigen-metric}.
In particular, our evaluation focuses on the quality of the depth rather than the disparity.
Given the camera focal length and baseline, both depth and disparity mathematically indicate the same 3D measurement.
However, while the depth map expresses the 3D measurement in the metric space, the disparity represented in the pixel domain has a non-linear mapping in the metric space. Therefore, the disparity metric may not properly reflect the error for distant objects.
In this regard, recent research works~\cite{pseudo_lidar,pseudo_lidar++} addressed the problem of the quality of disparity in view of the depth.
Thus, to validate the practical usage for advanced tasks working on metric space, such as 3D object detection~\cite{pseudo_lidar,monodepthdetection}, we evaluate our method with respect to the quality of depth.

\vspace{1mm} \noindent \textbf{Comparison.} \
As~in~\Tref{table:kitti-eigen-metric} and~\Tref{table:virtual-kitti-metric}, we compare our method with the recent works for the depth estimation~\cite{eigen_mono_depth,dorn} and the stereo matching~\cite{pseudo_lidar,psmnet,DSGN}.
These methods can be divided into monocular-based, mono-video, and stereo-based methods.
Since stereo matching methods train their network with the different benchmarks and different evaluations, we re-train the publicly accessible methods~\cite{pseudo_lidar,psmnet} to validate the performance with respect to the quality of the depth.
Generally, the stereo-based disparity estimation methods show a higher depth accuracy than those of the monocular-based approaches; in particular, our method further improves the accuracy with a large margin.
In addition, \Fref{fig:fig-depth} shows that our method has a lower depth error around object boundaries, even for distant objects.

This result still supports the quality of our estimated depth.
Thus, our detection-based 3D embedding is helpful to refine the cost volume, and it creates an increase in the depth quality, typically for faraway objects.
We also evaluate the performance of the 3D object detection with camera-based approaches~\cite{mono_rpn,stereo3d,triangulation_detection,pseudo_lidar,DSGN} in~\Tref{table:kitti-object-metric}.
Despite the gap of 3D object detection accuracy between ours and related studies,
our hypothesis (including 3D object detection as an auxiliary task to boost depth estimation performance) is validated.
We provide further analysis regarding the detection performance in the supplementary material.

\subsection{Ablation Study}
\label{subsection:ablation-study}

\begin{table}[!t]
\centering
\caption{\textbf{Ablation study of detection modules and the RoI sampling in the KITTI dataset.}
We intentionally omit the components of detection modules (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot, 3D RPN and Header) and our RoISelect.
We details our RoISelect into the two components, Deep sample and Select~(\emph{cf}\onedot} \def\Cf{{Cf}\onedot,~Figs.~\ref{fig:sampling}(\textcolor{blue}{b, c})).
Method 1 is the baseline~\cite{psmnet} of our network.
It shows that the high-level perception task (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot,~3D object detection) positively affects the low-level vision task (depth estimation).
}
\vspace{-2mm}
\resizebox{1.0\linewidth}{!}{
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline
\multirow{4}{*}{Method} & \multicolumn{4}{c|}{Preserve (\checkmark)} & \multirow{2}{*}{Depth} & \multicolumn{2}{c|}{\multirow{2}{*}{Detection}} \\ \cline{2-5}

& \multicolumn{2}{c|}{Detection} & \multicolumn{2}{c|}{RoI Select} & & \multicolumn{2}{c|}{} \\ \cline{2-8}

& 3D & \multirow{2}{*}{Header} & Deep & \multirow{2}{*}{Select} & \multirow{2}{*}{RMSE} & $\text{AP}_\text{BEV}$ & $\text{AP}_\text{3D}$ \\

& RPN & & sample & & & (Moderate) & (Moderate)  \\ \hline

1 & & & & & 2.847 & - & - \\ \hline

2 & $\checkmark$ & & & & 1.938 & - & -  \\ \hline

3 & $\checkmark$ & $\checkmark$ & & & 1.934  & 21.77 & 13.72 \\ \hline

4  & $\checkmark$ & $\checkmark$ & $\checkmark$ & & 1.924 & 25.80 & 16.90 \\ \hline

5 & $\checkmark$ & $\checkmark$ & $\checkmark$ & $\checkmark$ & \textbf{1.842} & \textbf{30.92} & \textbf{19.88} \\ \hline

\end{tabular}
}
\vspace{-1\baselineskip}
\label{table:ablation-sampling}
\end{table}

\vspace{1mm} \noindent \textbf{RoISelect.} \
We set the baseline (Method~\textcolor{blue}{1}~in~\Tref{table:ablation-sampling}) as the pure disparity estimation network, PSMnet~\cite{psmnet},
which does not contain detection networks (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot, the 3D-RPN and the header network).
In \Tref{table:ablation-sampling}, we evaluate the performance of each method in the KITTI-split~\cite{eigen_mono_depth} validation set, where our network (Method~\textcolor{blue}{5}) shows the best performance in both depth estimation and detection. From this, we can deduce that our sampling strategy (Select in~\Tref{table:ablation-sampling}) can handle occlusion cases by ignoring uncertain matching costs.

\vspace{1mm} \noindent \textbf{Fusion-by-occupancy.} \
We first analyze the influence of the different types of volumes (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot, cost volume $\mathcal{V}$ and cost aggregation $\mathcal{A}$) for the input of the detection modules.
As shown in~\Tref{table:ablation-fusion} (Methods~\textcolor{blue}{6} and \textcolor{blue}{7}), cost volume $\mathcal{V}$ is the appropriate voxel representation to include the detection-based 3D contextual information.
We deduce that channel-wise information in cost volume $\mathcal{V}$ (\emph{i.e}\onedot} \def\Ie{{I.e}\onedot voxel features) is crucial for the detection task, as channel-wise information was crucial in image recognition~\cite{alexnet,vgg,resnet}.

We extend our investigation by fusing $\mathcal{V}$ with the estimated disparity map.
In Method~\textcolor{blue}{8}, we extract the 2D RoIs ($16\times16$) from the disparity map $\widetilde{\mathcal{D}}$ and repeatedly stack the 2D RoIs along the disparity axis ($16\times16\times16$).
Then, we apply the same process, as shown in~\Fref{fig:fusion}.
The last experiment (Method~\textcolor{blue}{9}) is the one that adopts our fusion-by-occupancy (denoted as 3D in~\Tref{table:ablation-fusion}).
The resulting scores show that our method outperforms the 2D fusion method in the evaluation of the two tasks.
These results demonstrate that the way of aligning the dimensions is an important issue for the cost volume-based detection approach.

\begin{table}[!t]
\centering
\caption{\textbf{Ablation study of the 2D-3D fusion in different volumes.}
We validate the influence of type of fusion under two types of volume, cost volume $\mathcal{V}$ and cost aggregation $\mathcal{A}$.
It shows that $\mathcal{V}$ is the appropriate volume for fusion. Moreover, our fusion-by-occupancy (denoted as 3D) increases accuracy in both depth and detection.
}
\vspace{-2mm}
\resizebox{1.0\linewidth}{!}{
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline

\multirow{4}{*}{Method} & \multicolumn{4}{c|}{Preserve (\checkmark)} & \multirow{2}{*}{Depth} & \multicolumn{2}{c|}{\multirow{2}{*}{Detection}} \\ \cline{2-5}

& \multicolumn{2}{c|}{Input volume} & \multicolumn{2}{c|}{Fusion type} & & \multicolumn{2}{c|}{} \\ \cline{2-8}

& \multirow{2}{*}{costV} & \multirow{2}{*}{costA} & \multirow{2}{*}{2D} & 3D & \multirow{2}{*}{RMSE} & $\text{AP}_\text{BEV}$ & $\text{AP}_\text{3D}$ \\

& & & & (ours) & & (Moderate) & (Moderate)  \\ \hline

6 & \checkmark & & &  & 0.139  & 28.94 & 17.96 \\ \hline

7 & & \checkmark & &  & 0.116  & 25.73 & 12.08 \\ \hline

8 & \checkmark & & \checkmark &  & 0.113 & 24.49 & 15.71 \\ \hline

9 & \checkmark & & & \checkmark  & \textbf{0.111} & \textbf{30.92} & \textbf{19.88} \\ \hline

\end{tabular}
}
\vspace{-1\baselineskip}
\label{table:ablation-fusion}
\end{table}

\section{Conclusion}

We have proposed a novel stereo object matching network.
In particular, we investigated how to embed 3D object-level information in the stereo matching framework.
In contrast to existing techniques, we exploit the cost volume structure as a semi-3D space in which the 3D object detection task can be included as an auxiliary task.
However, this integration is not straightforward and requires the use of specific architectures: RoISelect and fusion-by-occupancy to handle the specificity of the cost volume.
This strategy allows us to seamlessly embed 3D object-level contextual information in the cost volume.
With this embedding, we can reduce the disparity ambiguity at the objects' edges and achieve state-of-the-art performance over the recent approaches.

\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


To make the most of the Internet, kids need to be prepared to make smart decisions. Learn the fundamentals of digital citizenship and safety so your child can explore the online world with confidence. This is being offered during the MS conferences, so please come to one of the sessions offered that works best for your schedule.
Time: 9:00 a.m. - 9:45 a.m., 12:00 p.m. - 12:45 a.m. or 2:00 p.m. - 2:45 p.m.


text:
for aviation
for mobility
GoMedia: Promoting infotainment tech to rail transport operators
Etihad: Blindsiding the opposition in a sports sponsorship promotion
Skedgo: Building investor appetite for mobility as a service
Aerogility: Creating understanding of artificial intelligence
Alton Aviation Consultancy: Building a financial media profile
8020 Communications logo
Articles: Blogs
Cessna Delivers Citation X to MJets in Thailand
Aviation - 2nd February 2009
WICHITA, Kan., Feb. 2, 2009 – Cessna Aircraft Company, a Textron Inc. (NYSE: TXT) company, announced today it has delivered a Citation X to Bangkok-based MJets Limited, a leader in business aviation across Southeast Asia. The Citation X joins a Citation CJ3 already in the MJets fleet and it is the second Citation X operating in Thailand.
The MJets Citation X is the first to be specifically available for charter in the country. Passengers will be able to fly non-stop from Bangkok to Beijing, Tokyo, Delhi or Dubai. The Citation X is the world's fastest civilian aircraft and saves passengers significant time compared to scheduled airlines and other business jets.
"MJets is delighted to welcome the Citation X into its charter fleet and confident that passengers will appreciate the aircraft's outstanding combination of cabin comfort, range and unbeatable speed. The Citation CJ3 has been a great success at MJets and there is no doubt that the Citation X will be another winner for the company," said William Heinecke, Principal of MJets and Chairman of The Minor Group.
The Citation X is the world's fastest civilian aircraft with a top cruising speed of 525 knots / 972 km/hr (Mach 0.92). Maximum cruising altitude is 51,000 feet / 15,545 m. Range is more than 3,070 nautical miles / 5,686 km, ideal for super fast non-stop travel around Asia. More than 300 Citation Xs are in service worldwide, with the aircraft speed capability making it a favorite with business and government leaders who place a premium on time efficiency.
Based on unit sales, Cessna Aircraft Company is the world's largest manufacturer of general aviation airplanes. In 2008, Cessna delivered 1,301 aircraft, including 467 Citation business jets, and reported revenues of about $5.662 billion. Cessna has a backlog of $14.5 billion as of December 31, 2008. Since the company was originally established in 1927, some 192,000 Cessna airplanes have been delivered around the world, including more than 5,700 Citations, making it the largest fleet of business jets in the world. More information about Cessna Aircraft Company is available at http://www.cessna.com.
Textron Inc. (NYSE: TXT) is a $14.2 billion multi-industry company operating in 28 countries with approximately 42,000 employees. The company leverages its global network of aircraft, defense and intelligence, industrial and finance businesses to provide customers with innovative solutions and services. Textron is known around the world for its powerful brands such as Bell Helicopter, Cessna Aircraft Company, Jacobsen, Kautex, Lycoming, E-Z-GO, Greenlee, Textron Systems and Textron Financial Corporation. More information is available at www.textron.com.
Forward-looking Information: Certain statements in this release are forward-looking statements and speak only as of the date on which they are made, and we undertake no obligation to update or revise any forward-looking statements. These forward-looking statements are subject to risks and uncertainties that may cause actual results to differ materially from those contained in the statements, including but not limited to the following: changes in worldwide economic and political conditions that impact demand for our products, interest rates and foreign exchange rates; the interruption of production at our facilities or at our suppliers' facilities; the timing of new product launches and certifications of new aircraft products; the occurrence of slowdowns or downturns in customer markets in which our products are sold or supplied; changes in aircraft delivery schedules or cancellation of orders; the launching of significant new products or programs which could result in unanticipated expenses; changes in national or international government policies on the export and import of commercial products; and bankruptcy or other financial problems at major suppliers that could cause disruptions in our supply chain.
_let's talk
Email Address Please enter your email
First Name Please enter your first name
Last Name Please enter your last name
_subscribe
There has been an error submitting your signup request. Please try again.
Digital marketing - 16th January 2023
How PPC fits into a wider communications strategy
Pay-per-click, or PPC as its more commonly known, can be incredibly complementary to a wider communications strategy, depending on your overall objectives. Firstly, what is PPC? PPC is a form of digital ...
Video - 5th January 2023
Aiming for accessibility: Using video captioning to widen your audience
Going into 2023, many companies have developed their video strategies to ensure content is both varied and appealing to target audiences. However, an often-overlooked variable in developing content is whether ...
Social Media - 15th November 2022
Is it… time to BeReal?
Yes, there is another new social media app to add to your phone. BeReal has been created to encourage users to stop using filters and sharing unrealistic images. Its name ...
PR for Aviation
PR for Travel
PR for Mobility
Built by INDIGO. Designed by CRUX.
©2023 8020 Communications. All rights reserved
It's time to outperform
Company Please enter your company name
Phone number Please enter your phone number
How can we help you? Please enter your message
I have read and agree to the 8020 privacy policy Please confirm you have read and agree to our privacy policy
There has been an error submitting your enquiry. Please try again.

redpajama_set_name: RedPajamaCommonCrawl


Sean, they're excellent! Thanks for putting those up on the site.
It's good to see how those crests came about, and what they mean. I always wondered why Wigram Base's crest had a scallop on it!
Excellent to read that someone managed to confuse a Kea for a Kaka on the 14 SQN crest!
Heraldic descriptions always confuse me though - I was just researching the Auckland Provincial coat of Arms to go on the fin of my DC-6 Model, and that took a bit of decoding - "Supported by Apterix rampant" (Held up by Kiwis...) Sinister, Dexter, proper, and they say that aviation has bizarre language!
Excellent collection of crests though, thanks heaps for posting them.
Thanks Sean, those are absolutely fantastic mate. Many of them i had never seen though a good deal of them are familiar, but seeing them day to day around base I never really stopped to study the detail. So, as Craig says, it's great to get the explanations.
I wonder if some of the squadrons didn't get crests, or if the book only covers the postwar squadrons - but then six of the 7 RAF sqns shouldn't be covered.
woah, thanks alot for putting those up i've visited 40 sqn but they didn't share anything about their crests or its meaning. did that take a long time to put together?
I just worked out that all these are post 1952, and thus have the Queen Elizabeth crown atop them. So earlier crests would have looked a little different and carried King George VI's crown on the same design. That explains why the other wartime squadrons are not seen, as these are all the squadrons and units that have served post coronation.
I wonder if there's a pre-1952 version of this book anywhere.
I spoke to the author of the NZAP 3327 Colours. Standards, Flags and Badges in the RNZAF, and he said, that only official crests are listed in the manual. Any other RNZAF crests that exist are not considered "official", as they have not been approved, or verified authentic etc by the RNZAF, or the UK Heraldic type people. The crests listed in the NZAP 3327 have meet the requirements, and are thus listed. I know this might provoke some "feed back" (especially with 30 SQN RNZAF people, but I guess if they have some kind of proof of approval etc, then it is time to get it acknowledged officially. As stated, these two crests are not considered official.
Thanks Sean. That is interesting indeed. I wonder why the squadrons never got their crests certified. I guess many other wartime crests will be in the same boat.
This would seem to indicate that it is not official, as there should only be one version of the 30 Sqn crest.
The author of the NZAP did mention that during WWII, some squadrons just made their own crests and did not get them approved, or did not seek approval because there were more important things to do, other than apply for design approval of crests..
Makes sense as they were not applied to aircraft or anything in an official capacity, and were probably more for hanging in the Mess than anything at that time.
I got the scan out of this, which I picked up the other day. It is dated 1938, so this crest would probably be one of the first ever issued to the RNZAF.
Yes, I got that programme recently too and took note of the crest. Great programme, and great crest. It's not a Kea, it's a bush hawk, or Karewa Rewa-Tara (or Falco novaeseelandiae), according to the write-up under the crest.
The motto 'TE ATETE-KIA TUPATO' means 'Let The Agressor Beware', a great motto.
"This member of the falcon tribe is probably the most fearless of all our birds. Regardless of its own danger, it will pursue its prey quite close to man and his habitations. A spirited and even rapacious hunter, it lives entirely by the chase... and had been known to kill pigeons, ducks and pukekos, and attack heron and shags. It is the vowed enemy of the harrier hawk and loses no opportunity of attacking it... Having sighted the prey, it drops like a miniature thunderbolt, grasps the victim by the alons of or both feet, and destroys life when it alights by crushing the head with its powerful beak."
So there you go. Much more deadly and probably a lot faster than the 2 Squadron Baffins!
I asked Jane Provan at the museum about this crest, and she said she's unaware of any other prewar Territorial crests, and didn't even know about this one!
Just found a suspect looking crest on Trademe, which says it is a 6 Sqn RNZAF crest, but it is very different from the official 6 Sqn RNZAF crest published in the NZAP 3327 Colours. Standards, Flags and Badges in the RNZAF.
Has anyone seen this before, or know anything about it. Tradmen says it came from the Eden Roskill RSA....... It even has the Queens Crown which means it was made after 1953. Appreciate any info tthat can be provided.
As far as I can tell, both 6 Sqn crests are correct. The "Vigilance with Patience" "Tane" Crest was approved postwar, but the "sea Horse" was the original wartime one, whether this was ever officially approved I dont know - perhaps someone with a knowlege of RNZAF Heraldry can clarify. There is a picture of the "sea horse" (what is it?) crest with the older style crown on Page 113 of "Golden age of NZ flying boats" but no clues as to its origin or official status. The current 6 Sqn page on the airforce web page (www.airforce.mil.nz) shows the "Tane" badge. Confused? Iam!
No. 6 Squadron was a Territorial unit postwar, could this be when the crest changed?
I sent off an email to CO 6 Sqn (LTCDR Keith Gilchrist) yesterday as well, and this is his response.
" The approved crest is the one in the NZAP 3327. The one you found on Trademe is an unofficial crest that was proposed by the TAF Sqn in the early 50's, it didn't make it as far as the Heraldry Society for final approval. We have one presented by the Brevet Club last year, obviously there are a few kicking round".
So if the unnapproved crest is from the 1950's the approved one must have come later I'd guess. So I wonder whether No. 6 Squadron had a previous unapproved crest during the war too, when it was either an Army Co-operation Squadron or while it was a Flying Boat Squadron.
Whoa wot a good thread this is thanks. Any chance that Sean can lead us to the source to obtain higher quality images please? Tah.


We examined the effects of individual versus joint action on a Simon task using motion tracking to explore the implicit cognitive dynamics underlying responses. In both individual and joint conditions, participants were slower to respond, and were differentially attracted to the distracter response location, when the spatial component of the stimulus was incompatible with the response location. When two people completed similar two choice tasks together, the results were not statistically different from the individual condition, even though the magnitude of the stimulus-response compatibility effect was slightly larger. Neither was there an increased effect when the partner had no stimulus-response conflict to resolve. We found no evidence for an action conflict when the responses of the two partners were different. These data imply that the literature regarding the Joint Simon task is still in the process of determining the relevant events that interact with and support joint action.


text:
The Red Carpet At The 51st ACM Awards
Filed Under: ACM Awards, Entertainment, Only CBS, Photo Galleries
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Miranda Lambert attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Recording artists Anderson East (L) and Miranda Lambert attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Actress Nicole Kidman (L) and recording artist Keith Urban attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Tim McGraw attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Carrie Underwood attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Musician Dierks Bentley attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Maren Morris, shoe detail, attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Actor/singer Kiefer Sutherland attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Maren Morris, fashion detail, attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Jessie James attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singers T.J. Osborne (L) and John Osborne of the Brothers Osborne attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Kacey Musgraves attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: (L-R) Recording artists Joe Don Rooney, Gary LeVox and Jay DeMarcus of Rascal Flatts attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Musician Billy Gibbons of ZZ Top attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Co-host Dierks Bentley attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Co-host Dierks Bentley (L) and Cassidy Black attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Randy Houser (L) and Tatiana Starzynski attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Jason Aldean (L) and Brittany Kerr attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Karen Fairchild of Little Big Town attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Eric Paslay attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Co-host Luke Bryan (R) and Caroline Boyer attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: (L-R) Singers Phillip Sweet, Karen Fairchild, Jimi Westbrook, and Kimberly Schlapman of Little Big Town attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Kimberly Schlapman of Little Big Town attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Jason Aldean attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Clay Walker (R) and Jessica Craig attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Luke Bryan attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Cyndi Thomson attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Dustin Lynch attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Cole Swindell attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Cassadee Pope attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Actor Tom Hiddleston attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Singer Katy Perry attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Recording artists Bill Satcher, Graham Deloach, Michael Hobby and Zach Brown of A Thousand Horses attend the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
51st Academy Of Country Music Awards - ArrivalsLAS VEGAS, NEVADA - APRIL 03: Tatiana Starzynski, ring detail, attends the 51st Academy of Country Music Awards at MGM Grand Garden Arena on April 3, 2016 in Las Vegas, Nevada. (Photo by David Becker/Getty Images)
Mile High Tree Features New Light Shows In January
Colorado Avalanche Practice January 1, 2020
Denver Broncos 2019 Season Wrap Up
Broncos 16, Raiders 15
Broncos Practice December 27
Lions 17, Broncos 27
Denver Broncos Social Justice Fund Ceremony
Darrent Williams Good Guy Award 2019
Denver Broncos Practice December 20, 2019
Nuggets 113, Magic 104
Broncos 3, Chiefs 23
Together 4 Colorado Toy Drive 2019
Spectacular Sunrise Over Colorado
Breckenridge Race Of The Santas
Broncos 38, Texans 24
Colorado Capitol Christmas Tree Lighting
Denver Broncos Practice December 4, 2019
'My Cause, My Cleats': Denver Broncos Unveil Shoes Designed To Highlight Charities
Next Photos


text:
Did Eric Make The Right Choice?
By	Chris Eades	 December 29, 2015
Thursday, THE BOLD & THE BEAUTIFUL left fans crying out in frustration with a cliffhanger that left the future of Forrester Creations in question. But Friday, Eric revealed that he had decided Ridge — not Rick — should become the company's new CEO. This set off a heated debate in the Twitterverse as to whether or not Eric made the right call. (Folks also wondered what the heck Maya was doing in the meeting, other than stirring up trouble, but that's another matter entirely!) So here's your chance to sound off: Do you agree with Eric's decision? Why or why not? Let's hear your thoughts in the comments!


Since I was a young teen I've had a passion for fighting. I will go as far as saying that I was filled with a violent spirit. I was always attracted by martial Arts, but martial arts cost money that my parents were not wiling to pay, so I engaged in boxing since it was free. I actually got involved in Martial Arts in my mid-twenties.
When I became a Christian, somebody sent me an article that spoke very passionately against Christians getting involved with martial arts. I remember throwing away everything I had related to Martial Arts, trophies, equipment, certificate, etc. At least I knew that the spirit of violence was lifted out of me; it was not important to fight or compete anymore. I did not look back for five years.
One morning, my prayer partner, James Burch, prophesied that God was going to use martial arts in my life for His glory. I told my friend that he was in the flesh. Shortly after that, there was a group of young people in my church who wanted me to teach them martial arts. They convinced me because I also saw the opportunity to minister the gospel to them. I told them to bring friends and they did.
Even as I started teaching, I was not convinced that I was doing the right thing. All that changed when I saw people getting saved. Not only did student respond to the gospel, but also their parents. Unfortunately, I lost my passion for martial arts years ago. I don't talk about martial arts, I don't really read much about martial arts, but I saw it as an instrument to bring forth the gospel. Ministering the gospel is the foundational vision of our program.
Two years ago I went to the Gospel Martial Arts Union Conference and took a small class with Master Michael Proctor. That class changed my life forever as a martial artist. I liked the traditional martial arts, but not presenting a way to defend ourselves from the brutality of the street, I considered us irresponsible. So my vision to my martial arts program is as follows.


text:
Q: Django display list of dictionary in template - dict key is a 'variable' I am using Django 4.0  to display a frontend page which source data is a list of dict.
I want to order the keys of the dict and then display all dict in the list in the same order.
Here is my views.py:
def UserGoalstatus(request, promise_token):
    print("__UserGoalstatus__")
    from cmd_utils import Retrieve_goal
    data = Retrieve_goal(promise_token)
    keys = set()
    for item in data:
        keys.update(set(item))
    key_order = sorted(keys)

context = {
    "data": data,
    "key_order": key_order,
}
return render(request, 'json_table.html', context)

Here is the content of my 'data' variable:
[
{'goal_key': '286815', 'goal_type': 'hotelreservation', 'goal_id': 16149845, 'promise_token': '9ba51cbc-830b-64d603904099', 'campaign_id': 1002204, 'properties': {'price': 100, 'created': '2022-06-13 10:48:34', 'checkout': '2022-06-13', 'currency_code': 'USD', 'completed_booking_status': 1}},
{'goal_key': '1208107', 'goal_type': 'hotelreservation', 'goal_id': 16149846, 'promise_token': '9ba51cbc-830b-64d603904099', 'campaign_id': 1002204, 'properties': {'price': 100, 'created': '2022-06-13 10:48:35', 'checkout': '2022-06-13', 'currency_code': 'USD', 'completed_booking_status': 1}}
]

Here is my html file which I would like to print all content in data in the order of 'key_order'

            {% for key in key_order %}
            {{ key }}
            {% endfor %}

        {% for item in data %}

            {% for key in key_order %}
            {{ item.get(key) }}
            {% endfor %}

        {% endfor %}

This part seems not right :{{ item.get(key) }} , anyone can suggest the right way to access the value mapping to the specific key ?

A: Here is my solution
I need to define a django template filter of my own
The key part is 'get_item' , it can parse dictionary key as 'variable' in Django html now .
For more detailed information refer to links below:
Django guide
stackoverflow answer
views.py
# customized template for html
from django.template.defaulttags import register

@register.filter
def get_item(dictionary, key):
    return dictionary.get(key)

in json_table.html

            {% for key in key_order %}
            {{ key }}
            {% endfor %}

        {% for item in data %}

            {% for key in key_order %}
            {{ item | get_item:key}}
            {% endfor %}

        {% endfor %}

redpajama_set_name: RedPajamaStackExchange