text:
\section{Introduction}\label{sec: intro}

A classical paradigm design for autonomous driving systems often adopts a modular based spirit~\cite{behere2015functional,yurtsever2020survey},
where the input of a planning or controlling unit is based on the outputs from preceding modules in perception.
As we witness the blossom of
end-to-end algorithms and success applications into various domains~\cite{pfeiffer2017perception,hamalainen2019affordance}, there are some attempt implementing such a philosophy in autonomous driving as well~\cite{pomerleau1988alvinn,bojarski2016end,openpilot,bansal2018chauffeurnet,chitta2021neat,prakash2021multi,zhang2021end,chen2021wor,chen2022lav}.
Rather than an isolated staged pipeline, we aim for a framework to directly take raw sensor data as inputs and generate the planning routes or control signals.
A straightforward incentive to do so is that feature representations can thus be optimized simultaneously \textit{within} one network towards the ultimate goal of the system (\textit{e.g.}, acceleration, steering).

One direction of end-to-end pipelines is to focus on the ultimate planning task mainly
without explicit design of the intermediate representation~\cite{codevilla2018end,codevilla2019exploring,prakash2021multi,chen2020learning,chitta2021neat,hu2021safe}.
Reinforcement learning (RL) fits well as a feasible resolution since the planned routes are not unique and each action should be rewarded correspondingly based on the environment.
RL algorithms
are applied to mimic experienced human experts and guide the behavior learning of driving agents~\cite{zhang2021end,Chekroun2021gri}.
Besides RL approaches, some propose to generate cost map with a trajectory sampler with knowledge of the perception environment~\cite{hu2021safe,chitta2021neat} or fusion of sensors in an attention manner~\cite{prakash2021multi,fadadu2022multi}.
These
work aforementioned
achieve impressive performance on challenging scenarios in closed-loop simulation~\cite{dosovitskiy2017carla}. The plausible transfer from synthetic setting to realistic application remains an open question.

Another direction is to explicitly design the intermediate representations in the network, provide convincing interpretability of each module and thus enhance safety towards a stable and robust system. Based on the input
type, explicit approaches are divided into LiDAR-based
\cite{casas2021mp3,zeng2020dsdnet,zeng2019end,cui2021lookout}
and
vision-based~\cite{hu2021fiery,philion2020lift,wang2021learning}
respectively.
LiDAR-based
methods
bundled with high-definition (HD) maps  exhibit good performance on various benchmarks, and they investigate the effect of each module of the system exhaustively. However, the inherent defects of LiDAR, such as recognition of
traffic lights and short-range detection of objects, might confine their applications.

In this paper, concurrent to the interpretable LiDAR-based pipelines~\cite{casas2021mp3,zeng2020dsdnet}, we propose to investigate the potential of \textit{vision-based} end-to-end framework (Fig. \ref{fig:motivation}).
If each module is exquisitely designed, to which extent the performance of each task as in perception, prediction and planning should be improved?

\begin{figure}[tb!]
    \centering
    \includegraphics[width=1.\textwidth]{Figs/motivation.png}

    \caption{Problem setup whereby an interpretable vision-based end-to-end framework in (a) is devised, parallel to the LiDAR-based counterpart by aid of HD maps in (b)
    }
    \label{fig:motivation}

\end{figure}

To answer the above question, the first key challenge for vision-based methods is to appropriately transform feature representations from  perspective views to the bird's eye view (BEV) space.
The pioneering LSS approach~\cite{philion2020lift} extracts perspective features from multi-view cameras, lifts them into 3D with depth estimation and fuses into the BEV space. It is observed that latent depth prediction for the feature transformation between two views is crucial~\cite{hu2021fiery,wang2021learning,reading2021CaDDN}. On a theoretical analysis, this is true since lifting 2D planar information to 3D requires an additional dimension, which is the depth that fits into 3D geometric autonomous driving tasks.
To further improve feature representation,
it is natural to incorporate temporal information into the framework as most scenarios are tasked with video sources.
Descending from~\cite{philion2020lift},
the follow-up literature project past frames' feature onto current coordinate view
either by ego-motion of the self-driving vehicle (SDV) provided by dataset~\cite{hu2021fiery} or learned mapping from optical flow~\cite{wang2021learning}.
These approaches project features in the past frame-by-frame in isolation and feed them into the temporal unit; instead
we accumulate all past aligned features
in 3D space before transforming to BEV, preserving geometry information at best and compensating for more robust feature representations of the current state, which is empirically proven as a better design choice.

Equipped with representative features in BEV space, we formulate prediction task as the future instance segmentation, which is the same as does in FIERY~\cite{hu2021fiery}.
The common practice includes generating uncertainty from data distribution for current state and feeding it in a temporal model to infer predictions under a window of future horizons~\cite{hu2021fiery,wang2021learning}.
A natural incentive to boost
future predictions, which is missing in current literature, is to take into account the motion variations in the \textit{past}. To do so, an additional temporal model with fusion unit could be introduced to reason about the probabilistic nature of both past and future motions.
A stronger version of scene representations could therefore be obtained, which serves as recipe towards the ultimate planning task.

The general idea for motion planners is to output the most likely trajectory, given a sampling of possible candidates and semantic results from preceding modules~\cite{casas2021mp3,prakash2021multi,zeng2020dsdnet,hu2021safe}. In light of an interpretable spirit,
most previous work construct cost volumes, learning-based~\cite{casas2021mp3,chitta2021neat} and/or rule-based~\cite{zeng2019end,Chekroun2021gri,zeng2020dsdnet}, to indicate the confidence of trajectories with a certain form of trajectory modelling in a sampler.
We follow such a philosophy to indicate the most possible candidate with the help of a high-level command, without HD map as guidance.
The outcome trajectory is further refined with the features from the front-view camera (front-view features)
to consider vision-based elements (\textit{e.g.}, traffic lights). This is inspired by MP3~\cite{casas2021mp3}, where they also remove HD maps and feed the network with a high-level command. However, we argue that the vision recognition module in~\cite{casas2021mp3} is off-the-shelve; in this work, we integrate vision information  in form of a lightweight GRU unit within the same network. Such a refinement process serves as a complementary feature boosting towards the final outcome.

\begin{figure}[t]
    \centering
    \includegraphics[width=0.98\textwidth]{Figs/pipeline.png}

    \caption{
    We present \textbf{ST-P3},
    an interpretable end-to-end vision-based framework.

   For \textbf{perception},
   the egocentric aligned accumulation guarantees features (past and present) aligned and aggregated in 3D space
   to preserve geometry information before BEV transformation.
   %
   For \textbf{prediction}, a dual pathway scheme is introduced to bring in past variations in pursuit of future predictions.
   %
   For \textbf{planning}, the prior knowledge
   is fed into a refinement unit to generate the  final trajectory with an integrated cost volume  and sampler from high-level commands
    }
    \label{fig:pipeline}

\end{figure}

To this end, we propose an interpretable vision-based end-to-end system that improves feature learning for perception, prediction and planning altogether, namely \textbf{ST-P3}. Fig. \ref{fig:pipeline} describes the overall framework.
Specifically, given a set of surrounding camera videos, we feed them into the backbone to generate preliminary front-view features. An auxiliary depth estimation is performed to transform 2D features onto 3D space.
An egocentric aligned accumulation scheme first aligned past features to the current view coordinate system.
The current and past features are then aggregated in 3D space to preserve geometric information before the BEV representation.
Apart from a commonly used temporal model for prediction,
this module is further boosted by constructing a second pathway to account for motion variations in the past.
Such a dual pathway modelling ensures stronger feature representations to infer future semantic outcomes.
Towards the ultimate goal of trajectory planning, we integrate prior knowledge
from features in the early stage of the network.
A refinement module is devised to generate the final trajectory with help of high-level commands and no presence of HD maps.
We benchmark our approach against previous state-of-the-arts on both open-loop nuScenes dataset as well as closed-loop simulator CARLA environment.
To sum up, ST-P3 owns the following contributions baked into it:
\begin{enumerate}
    \item For better spatial-temporal feature learning, we propose three novel improvements, \textit{i.e.}, the egocentric aligned accumulation, the dual pathway modelling, and the prior-knowledge refinement for perception, prediction and planning modules respectively. The resulting new end-to-end vision-based network for autonomous driving is called ST-P3.

    %

    \item We investigate each part of an interpretable end-to-end system for autonomous driving tasks
    systematically. As a vision-based counterpart to the study of LiDAR-based approaches~\cite{zeng2020dsdnet}, to our best knowledge, we provide the first detailed analysis and comparison for a vision-based pipeline.

    \item ST-P3 achieves state-of-the-art performance on benchmarks from the popular nuScenes dataset and simulator CARLA. The full suite of codebase, as well as protocols are made publicly available.
\end{enumerate}

\section{Related Work}\label{sec: related work}
We briefly discuss the related works in four aspects.

\noindent\textbf{Interpretable End-to-End Framework. }
We review popular approaches~\cite{cui2021lookout,zeng2019end,zeng2020dsdnet,sadat2020perceive,casas2021mp3} that adopt an explicit design to have clear interpretability of the system and hence prompts safety.
These are LiDAR-based approaches mainly as there are few vision-based solutions in the wild (compared in the previous section already).
The pioneering NMP~\cite{zeng2019end} takes as input LiDAR and HD maps to first
predict the bounding boxes of actors in the future, and then learns a cost volume to choose the best planned trajectory.
The subsequent P3
\cite{sadat2020perceive} work
further achieves consistency between planning and perception
by
a differentiable occupancy representation, which
is explicitly used as cost by planning to produce safe maneuvers.
To avoid heavy reliance on HD maps,
Casas \textit{et al.} (MP3)
\cite{casas2021mp3}
constructs an online map from segmentation as well as the current and future states of other agents;
these
results are then fed into
a sampler-based planner to obtain a safe and comfortable trajectory.
LookOut~\cite{cui2021lookout}
predicts a diverse
set of futures of how the scene might unroll and estimates
the trajectory by optimizing a set of contingency
plans.
DSDNet~\cite{zeng2020dsdnet}
considers the interactions between actors and produces socially consistent multimodal future predictions.
It explicitly exploits the predicted
future distributions of actors to plan a safe maneuver by using a structured planning cost.
In general, LiDAR based approaches demonstrate good performance on challenging urban scenarios.
Unfortunately, the datasets and baselines in these work are not released
to compare.

\noindent\textbf{Bird's eye view (BEV) Representation}
is a natural and perfect fit for planning and control tasks~\cite{ng2020bev,zhang2021end,9123682,chitta2021neat,bansal2018chauffeurnet},
since it avoids problems such as occlusion, scale distortion, \textit{etc.}, and preserves 3D scene layout.
Although information in LiDAR and HD maps can be easily represented in BEV,
how to project vision inputs from camera view to BEV space
is a non-trivial problem.
Some learning based methods~\cite{chitta2021neat,9123682} implicitly project  image input into BEV, but the quality can not be guaranteed since usually we do not have ground truth in BEV to supervise the projection process.
Loukkal \textit{et al.}
\cite{loukkal2021driving} explicitly projects image into BEV using homography between image and BEV plane.
\cite{li2022bevformer,chen2022persformer} aquires BEV features through spatial cross-attention with pre-defined BEV queries.
LSS~\cite{philion2020lift} and FIERY~\cite{hu2021fiery} perform the projection
with estimated depth and image intrinsic, which have shown impressive performance.
We predict depth and project images
in a similar fashion.
Different from FIERY~\cite{hu2021fiery} which transforms past features to current timestamp frame-by-frame correspondingly,
we append all past 3D features to the current ego view (egocentric)
and accumulate the aligned features, providing better representation for subsequent tasks.

\noindent\textbf{Future Prediction.}
Current motion prediction methods~\cite{ngiam2021scene,gu2021densetnt,pmlr-v164-jia22a,LiaoICME22} usually takes ground truth perception information and HD map as input, but they are susceptible to cumulative error when the perception input comes from other modules in real-life application. Taking raw sensor data as input,  end-to-end methods which focus on future trajectory prediction or take it as an intermediate step usually rely on LiDAR and HD map~\cite{luo2018fast,zeng2020dsdnet,zeng2019end,cui2021lookout,wei2021perceive} to detect and predict.
Recently, future prediction in the form of BEV semantic segmentation using only camera input~\cite{hu2021fiery,wang2021learning} has been proposed and shown great performance. However, the evolution process of the past is not well captured and exploited~\cite{hu2021fiery,philion2020lift,wang2021learning}. Inspired by video future prediction~\cite{Hu2020ProbabilisticFP}, we combine the probabilistic  uncertainty with the dynamics of past to predict diverse and plausible future scenes.

\noindent\textbf{Motion Planning.}
We cover previous learning-based motion planning methods and omit traditional approaches in this part.
For implicit methods~\cite{pomerleau1988alvinn,codevilla2019exploring,prakash2021multi}, the network directly generates planned trajectory or control commands. Although such design is direct and simple, it suffers from robustness and lack of interpretability. On the contrary, explicit methods usually
build a
cost map with a trajectory sampler
to generate the desired trajectory by choosing the optimal candidate with the lowest cost.
The cost map can be constructed with hand-crafted rules~\cite{sadat2020perceive,casas2021mp3,cui2021lookout} based on intermediate representations such as
segmentations and HD maps;
or it can be learned directly from the network~\cite{zeng2019end}. DSDNet~\cite{zeng2020dsdnet} combines hand-crafted and learning-based costs to obtain an integrated cost volume. We also adopt this combination
to choose the best trajectory. However, we modify the pipeline by adding an additional GRU refinement unit
with navigation signal to further adjust and optimize the chosen trajectory.

\section{Methodology}\label{sec: method}

An overview of ST-P3 is given in Fig.~\ref{fig:pipeline}.
ST-P3 first extracts features of a sequence of surrounding images and lift them to 3D with depth prediction. They are fused in both spatial and temporal domains with the  egocentric aligned accumulation
(see Sec.~\ref{sec: method-perception}).
We show the dual pathway mechanism in Sec.~\ref{sec: method-prediction}, a novel uncertainty modeling to incorporate history information.
Sec.~\ref{sec: method-plan} elaborates on how we  utilize the intermediate representations to plan a safe and comfortable trajectory.

\subsection{Perception:  Egocentric Aligned Accumulation}
\label{sec: method-perception}

In this stage we need to build a spatiotemporal BEV feature from multi-view camera inputs in past $t$ timestamps.
As discussed in Sec.~\ref{sec: intro}, the direct concatenation~\cite{wang2021learning} has the alignment issue while FIERY~\cite{hu2021fiery} suffers from losing height information.
Towards these problems, here we introduce our accumulative ego-centric alignment method which incorporates two steps, \textit{i.e.}, spatial and temporal fusion.
In the spatial fusion part, multi-view images in all timestamps are processed and transformed to the current ego-centric 3D space.
While in the temporal fusion step, we enhance the feature discrimination of static elements and objects in motion in an accumulative way and adopt a temporal model to achieve the final fusion.
An illustration is depicted in Fig.~\ref{fig:perception}.

\begin{figure}[t]
    \centering
    \begin{minipage}[b]{0.59\textwidth}
      \includegraphics[width=\textwidth]{Figs/perception.png}
    \end{minipage} ~~
    \begin{minipage}[b]{0.37\textwidth}
      \caption{
      Egocentric aligned accumulation for Perception. \textbf{(a)} Feature at current timestamp is lifted to 3D with depth estimation, and pooled to BEV features $x_t$ after alignment;
      \textbf{(b-c)}
      3D features in previous frames are aligned to current
      view and fused
      with all past and current states, so that
      the feature representation

      get enhanced
      }
    \label{fig:perception}
    \end{minipage}

\end{figure}

\noindent\textbf{Spatial Fusion.}
On one hand, features from multi-view images should be transformed to a common frame.
Inspired by~\cite{philion2020lift}, we extract the feature and predict the corresponding depth for each image and then lift it into the global 3D frame.
In particular, each camera image $I_t^n \in \mathbb{R}^{3 \times H \times W}$ is passed through a backbone network to obtain features $f_i^k \in \mathbb{R}^{C \times H_e \times W_e}$ and
depth estimation $d_i^k \in \mathbb{R}^{D \times H_e \times W_e}$ respectively,
where $i \in \{1,\dots,t\}$, $n \in \{1,\dots,6\}$, $C$ is the number of feature channels, $D$ denotes the number of discrete depths and $(H_e, W_e)$ indicates the spatial size.
Since the exact depth information is not available, we spread the feature across the entire ray of space according to the predicted depth distribution by taking the outer product of the matrices:
\begin{equation}
	u_i^k = f_i^k \otimes d_i^k,
\end{equation}
with $u_i^k \in \mathbb{R}^{C \times D \times H_e \times W_e}$.
Then we use the camera extrinsics and intrinsics to transform the camera feature frustums $u_i \in \{u_i^1,\dots,u_i^n\}$ to the global 3D coordinate whose origin is at the inertial center of the ego-vehicle at time $i$.

On the other hand, the spatial fusion needs to align past features to the current frame for the downstream prediction and planning tasks.
With the ego-motion from time $t-1$ to $t$, we can transform the cube obtained at time $t-1$ into the coordinate system centered on the SDV at time $t$.
The same process could be applied to the past frames as well, resulting in ego-centric features $\{u_i^{'}\}$ for all previous timestamps.
Ultimately, the BEV feature maps $b_i \in \mathbb{R}^{C \times H_b \times W_b}$ could be sum pooled from $\{u_i^{'}\}$ along the vertical dimension.

\noindent\textbf{Temporal Fusion.}
Classical temporal fusion methods directly exploit 3D convolutions with stacked BEV features.
However, it is noted that
the corresponding features at the same location of
various cubes should be similar if there are objects that are stationary on the ground, such as lanes and static vehicles.
Due to this property,
we
perform
a self-attention to boost the perceptual ability of static objects by adding the previous BEV feature maps to the current, which can be formulated as follows (where the discount $\alpha = 0.5$ and $\tilde{x}_1 = b_1$):
\begin{equation}
\tilde{x}_t = b_t + \sum_{i=1}^{t-1} \alpha^i \times \tilde{x}_{t-i},
\end{equation}

In order to perceive dynamic objects more accurately, we then feed these features into a temporal fusion network achieved by 3D convolution. To compensate for deviations caused by ego-vehicle motion, we add the motion matrix to the features by concatenating it in the spatial channel:
\begin{equation}
    x_{1\sim t} = \mathcal{C}(\tilde{x}_{1 \sim t}, m_{1 \sim t}),
\end{equation}
where $m_{1 \sim t}$ denotes the ego-motion matrices and $\mathcal{C}$ denotes the 3D conv network.

\subsection{Prediction: Dual Pathway Probabilistic Future Modelling}\label{sec: method-prediction}

In a dynamic driving environment, traditional motion prediction algorithms~\cite{wu2020motionnet,fang2020tpnet,djuric2020uncertainty} often predict future trajectories as a deterministic or multi-modal results.
However, a finite number of probabilities modelling could not cover the complex situation of future due to the interaction among agents, traffic elements and road environments.
In order to deal with the stochasticity of the future, we wish to reason about the conditional uncertainty in the prediction features.
Motivated by Hu \textit{et al}.~\cite{hu2020probabilistic}, we model the future uncertainty as diagonal Gaussians with mean $\mu \in \mathbb{R}^L$ and variance $\sigma^2 \in \mathbb{R}^L$, where $L$ is the latent channel.
Samples from the distribution could serve as a hidden state feature for future use.
It is noted that during training, we use samples $\eta_t \sim \mathcal{N}(\mu_{\mathrm{t}}, \sigma^2_{\mathrm{t}})$ while it will be sampled from $\eta_t \sim \mathcal{N}(\mu_{\mathrm{t}}, 0)$ during inference time.

\begin{figure}[t]
    \centering
    \begin{minipage}[b]{0.59\textwidth}
      \includegraphics[width=\textwidth]{Figs/prediction.png}
    \end{minipage} ~~
    \begin{minipage}[b]{0.37\textwidth}
      \caption{

      Dual pathway modelling for Prediction.
      %
      \textbf{(i)} the latent code is from distribution of feature maps; \textbf{(ii-iii)}

      pathway \underline{\textit{a}} incorporates uncertainty distribution to indicate the multi-modal of future,
      while

      \underline{\textit{b}} learns from  variations in the past, which is beneficial to compensate for information in \underline{\textit{a}}

      }
    \label{fig:prediction}
    \end{minipage}

\end{figure}

The architecture of the prediction module
is depicted in Fig.~\ref{fig:prediction}.
We integrate the BEV features till current timestamp and the future uncertainty distribution into our prediction model, corresponding to two pathways in the dual modelling respectively.
One uses historical features $(x_1, \dots, x_t)$ as GRU inputs, and $x_1$ as the initial hidden state for prediction. The other uses samples from $\eta_t$ as the GRU input and $x_t$ as the initial hidden state.
When predicting the feature at time $t+1$, we combine the predicted features in the form of a mixed Gaussian:
\begin{equation}
    \hat{x}_{t+1} = \mathcal{G}(x_t, \eta_t) \oplus \mathcal{G}(x_{0:t}),
\end{equation}
where $\mathcal{G}$ represents the process of GRU.
And we use the mixture prediction as the base for the following prediction progress.
Through this method, Dual Modelling recursively predicts future states $(\hat{x}_{t+1},\dots, \hat{x}_{t+H})$.

All the features $(x_1, \dots, x_t), (\hat{x}_{t+1},\dots, \hat{x}_{t+H})$ are fed into the decoder $\mathcal{D}$ which has multiple output heads to generate different interpretable intermediate representations.
The outcome is shown in Fig.~\ref{fig:pred outputs}.
For the instance segmentation task, we follow the evaluation metrics in~\cite{hu2021fiery}, where the heads output the instance centerness, offset and future flow.
Meanwhile, the semantic segmentation head mainly focuses on the vehicle and pedestrian which are the main actors in a driving setting.
Furthermore,
as HD map plays a vital role in autonomous driving~\cite{sadat2020perceive,sadat2019jointly,zeng2019end}, we explicitly generate two elements - drivable area and lanes, to provide an interpretable map representation.
A cost volume head is designed for representing the \textit{expense} of each possible location that the SDV can take within the planning horizon. More detailed information on cost volume is illustrated in Sec.~\ref{sec: method-plan}.
Note that we also decode features in the past frames to boost historical features' accuracy, which is required by Dual Modelling. As demonstrated in Sec.~\ref{sec: res-nuscenes}, more accurate feature information could lead to better prediction.

\subsection{Planning: Prior Knowledge Incorporation and  Refinement}
\label{sec: method-plan}

\begin{figure}[t]
    \centering
    \begin{minipage}[b]{0.6\textwidth}
      \includegraphics[width=\textwidth]{Figs/planning.png}
    \end{minipage}
    \begin{minipage}[b]{0.37\textwidth}
      \caption{
      Prior knowledge integration and refinement for Planning.

      The overall cost map includes two subcosts.
      The min-cost trajectory is further refined with front-view features to aggregate vision-based information from camera inputs
      }
    \label{fig:planning}
    \end{minipage}
\end{figure}

As the ultimate goal, a motion planner needs to plan a safe and comfortable trajectory towards the target point.
Given the occupancy predictions $o$ and perceptions of map representations $m$, we design the motion planner which samples a diverse set of trajectories, and picks the one minimizing the learned cost function, inspired by~\cite{zeng2019end,sadat2020perceive,casas2021mp3}.
However, we differentiate from them with an additional optimization step with a temporal model to integrate information of target point and traffic lights.
The overall module is illustrated in Fig.~\ref{fig:planning}.

The cost function makes full use of the learned occupancy probability field (segmentation maps in Prediction) and rich pre-knowledge as well, such as traffic rules, to ensure the safety and smoothness of the final trajectory.
Formally, given the SDV's dynamic state, we adopt the bicycle model~\cite{polack2017kinematic} to sample a set of trajectories $\tau$.
The objective cost function $f$ is composed of subcosts, $f_o$ that evaluates the predicted trajectory at every timestamp $t$ according to the prior knowledge, $f_v$ from prediction decoder (learning-based), and $f_r$ that relates to the overall performances of the trajectory such as the comfort, progress.
Thus the overall cost function can be an equally weighted combination as:
\begin{equation}
    f(\tau, o, m; w) = f_o(\tau, o, m; w_o) + f_v(\tau ; w_v) +f_r(\tau; w_r),
\end{equation}
with $w = (w_o,w_v,w_r)$ being the vector of all learnable parameters.
We briefly describe the subcosts below and refer readers to the Appendix for details.

\noindent\textbf{Safety Cost.} The SDV should not collide with other objects on the road and need to consider their future motion when planning its trajectory.
The planned trajectories cannot overlap the grids occupied by other agents or road elements and need to maintain a certain safe distance at the high-velocity motion.

\noindent\textbf{Cost Volume.} Due to the complexity of road information, we cannot manually enumerate all possible cases or planning cost, thus we introduce a learned Cost Volume generated by the head in Sec.~\ref{sec: method-prediction}.
In order to balance the cost scale, we clip the value so that it does not take a dominant role in evaluating trajectories.

\noindent\textbf{Comfort and Progress.} We penalize trajectories with large lateral acceleration, jerk or curvature.
We wish the trajectory to be efficient towards the destination, hence the trajectory progressing forward would be awarded.

However, the above cost does not contain the target information which is often provided by a routed map; yet, this is not available in our setting.
Thus we adopt the high level command including forward, turn left and turn right, and only evaluate the trajectories according to the corresponding command.
To sum up, the motion planner now outputs one with the minimum cost as:
\begin{equation}
    \tau^* = \arg\min_{\tau_h} f(\tau_h, c) = \arg\min_{\tau_h} f(\tau_h, o, m; w),
    \label{eq:over_obj}
\end{equation}
where $\tau^*$ is the selected trajectory, $\tau_h$ is the trajectory set under the corresponding high level command and $c$ is the overall cost map.
Furthermore, traffic lights are critical for the SDV while it is not explicitly utilized in previous modules. We take it into account through a GRU network to refine the trajectory.
We initialize the hidden state with front camera features from the encoder module, and use each sample point in $\tau^*$ as the input.
It is demonstrated in Tab. \ref{tab:p2_ablation} that the front camera features indeed capture the information of traffic lights, which helps the SDV to launch or stop at intersections.

\subsection{Overall Loss for End-to-End Learning
}\label{sec: method-e2e learning}
We optimize our model with perception, prediction, planning in an end-to-end manner by exploiting the following loss functions:
\begin{equation}
    \mathcal{L} =  \mathcal{L}_{per} + \alpha \mathcal{L}_{pre} + \beta \mathcal{L}_{pla}.
\end{equation}
Note that we follow the protocol in~\cite{kendall2018multi,chen2018gradnorm} where the weights $\alpha$, $\beta$ are learnable rather than fixed, to balance the scale in different tasks according to gradients of the corresponding task loss.

\noindent\textbf{Perception Loss.}
This loss includes the segmentation loss for current and past frames, the mapping loss and an auxiliary depth loss.
For semantic segmentation, we use a top-k cross-entropy loss as in~\cite{hu2021fiery} since the BEV image is largely dominated by the background. For instance segmentation, the $l_2$ distance loss is adopted for the centerness supervision while $l_1$ distance loss for the offset and flow tasks.
We use a cross-entropy loss for the lane and drivable area prediction.
Current methods~\cite{philion2020lift,wang2021learning} utilize downstream tasks to implicitly optimize the depth prediction rather than a direct supervision, yet this approach is remarkably affected by the design of the final loss function  without clear explainability.
Therefore we generate the depth value with other networks beforehand, then use it to direct the prediction. More details are in the Appendix.

\noindent\textbf{Prediction Loss.}
As our prediction module infers the future semantic segmentation and instance segmentation, we keep the same top-k cross-entropy loss as in Perception. Nonetheless, loss in future timestamps would be discounted exponentially due to the uncertainty of the prediction.

\noindent\textbf{Planning Loss.}
Our planning module first select the \textit{best} trajectory $\tau^*$ from the sampled trajectory set $\tau$ as in Eq. (\ref{eq:over_obj}), then a GRU-based network is applied to further refine the trajectory to obtain final output $\tau_o^*$.
Thus the planning loss contains two parts: a max-margin loss, which treats expert behavior $\tau_h$ as a positive example while trajectories sampled as negative ones, and a naive $l_1$ distance loss between the output and the expert trajectory. In particular, we set
\begin{equation}
    \mathcal{L}_{pla} = \max_{\tau} \left[ f(\tau_h, c) - f(\tau, c) + d(\tau_h, \tau) \right]_+ + d(\tau_h, \tau_o^* ),
\end{equation}
where $[ \cdot ]_+$ denotes ReLU and $d$ is the $l_1$ distance between input trajectories.

\section{Experiments}\label{sec: experiments}
We evaluate ST-P3 in both open-loop and closed-loop environments.
We adopt nuScenes dataset~\cite{caesar2020nuscenes} for the open-loop setting,  and CARLA simulator~\cite{dosovitskiy2017carla} for the closed-loop demonstration.
By default we take the $1.0s$ of past context and predict the future $2.0s$ contexts, corresponding to 3 frames in the past and 4 frames in the future. More details on protocols are provided in the Appendix.

\begin{figure}[tb!]
    \centering
    \includegraphics[width=\textwidth]{Figs/output.png}
    \caption{Qualitative results on nuScenes. (b) shows predicted map representation including drivable area and lanes. (c) depicts the segmentation of vehicles and pedestrians. (a-c) are at $t=1s$. (d) represents the overall results from our model - perception, prediction, planning. The intermediate scene representations are robust in the time period, and the SDV successfully generate a safe trajectory to do left-turn without collision with curbsides or the front car
    }
    \label{fig:pred outputs}

\end{figure}

\subsection{Open-loop Experimental Results on nuScenes}\label{sec: res-nuscenes}

\textbf{Perception.} We evaluate the models on map representation and semantic segmentation.
The perceived map includes drivable area and lanes - two most critical elements for the driving behavior, since the SDV is assumed to be driving in the drivable areas and keeping itself in the center of lanes.
The semantic segmentation focuses on vehicles and pedestrians, both of which are the main agents in a driving environment.
We use the Intersection-over-Union (IoU) as the metric, modeling the perception module as a BEV segmentation task.
As shown in Tab. \ref{tab:perception}, ST-P3 gets the highest mean value on the nuScenes validation set, surpassing previous SOTA by \textbf{2.51\%} with our Egocentric Aligned Accumulation algorithm.

\noindent\textbf{Prediction.} Predicting future segmentation in BEV is first proposed in~\cite{hu2021fiery}, thus we select it as our baseline.
We evaluate our model by the metric of IoU, existing panoptic quality (PQ), recognition quality (RQ), and segmentation quality (SQ) following metrics in video prediction area~\cite{kim2020video}.
Note that we predict vehicles which have been shown in the past frames only as we cannot predict those out of nowhere.
Results are shown in Tab. \ref{tab:prediction}. Our model achieves the state-of-the-art in \textit{all} metrics as a consequence of the novel design of the prediction module.
Though the Gaus. version performs a little worse than the Ber. one, we finally choose it for relatively smaller memory usage.

\noindent\textbf{Planning.} For the open-loop planning, we focus on two evaluation metrics: L2 error and collision rate, and adjust the planning horizon to $3.0s$ for a fair comparison.
We compute the L2 error between the planned trajectory and the human driving trajectory,
and evaluate how often the SDV would collide with other agents on the road.
Detailed comparison with previous methods is shown in Tab. \ref{tab:planning}. Note that the Vanilla algorithm is penalized based on how much the trajectory deviates from the ground truth, thus it achieves the lowest L2 error but largest collision rates in all prediction horizons.
ST-P3 obtains the lowest collision rate, implying the superior safety of our planned trajectory.

\begin{table}[tb!]
\caption{\textbf{Perception results.} We report the BEV segmentation IoU (\%) of intermediate representations and their mean value. ST-P3 outperforms in most cases}
\label{tab:perception}
\centering
\scalebox{0.8}{
\begin{tabular}{p{0.2\textwidth}|>{\centering}p{0.12\textwidth}>{\centering}p{0.12\textwidth}>{\centering}p{0.12\textwidth}>{\centering}p{0.12\textwidth}>{\centering\arraybackslash}p{0.12\textwidth}}
\toprule
\multirow{2}{*}{Method}  & \cellcolor{gray!30}Mean Value & Drivable Area     & \multirow{2}{*}{Lane}     & \multirow{2}{*}{Vehicle}     & \multirow{2}{*}{Pedestrian}          \\ \midrule
VED~\cite{lu2019monocular}      & \cellcolor{gray!30}28.19  & 60.82 & 16.74 & 23.28 & 11.93  \\
VPN~\cite{9123682}                       & \cellcolor{gray!30}30.36  & 65.97 & 17.05 & 28.17 & 10.26  \\
PON~\cite{roddick2020predicting}      & \cellcolor{gray!30}30.52  & 63.05 & 17.19 & 27.91 & 13.93  \\
Lift-Splat~\cite{philion2020lift} & \cellcolor{gray!30}34.61 & 72.23 & 19.98 & 31.22 & 15.02  \\
IVMP~\cite{wang2021learning}      & \cellcolor{gray!30}36.76 & 74.70  & 20.94 & 34.03 & \textbf{17.38}  \\
FIERY~\cite{hu2021fiery}    & \cellcolor{gray!30}40.18  & 71.97 & 33.58 & 38.00 & 17.15  \\ \midrule
\textbf{ST-P3} & \cellcolor{gray!30}\textbf{42.69} & \textbf{75.97} & \textbf{40.20} & \textbf{40.10} & 14.48  \\
\bottomrule
\end{tabular}}
\end{table}

\begin{table}[tb!]
\caption{\textbf{Prediction results.} We report semantic segmentation IoU (\%) and instance segmentation metrics from video prediction area. The \textit{static} method assumes all obstacles static in the prediction horizon. ``Ber.'' denotes modeling the future uncertainty as Bernoulli distribution, while ``Gaus.'' means Gaussian distribution. ST-P3 achieves best performance on all temporal segmentation metrics}
\label{tab:prediction}
\centering
\scalebox{0.8}{
\begin{tabular}{p{0.2\textwidth}|>{\centering}p{0.28\textwidth}|>{\centering}p{0.12\textwidth}>{\centering}p{0.12\textwidth}>{\centering\arraybackslash}p{0.12\textwidth}}
\toprule
\multicolumn{1}{l|}{\multirow{2}{*}{Method}} & Future Semantic Seg. & \multicolumn{3}{c}{Future Instance Seg.} \\
\multicolumn{1}{c|}{}                          & IoU $\uparrow$             & PQ $\uparrow$           & SQ $\uparrow$          & RQ $\uparrow$          \\ \midrule
Static                                         & 32.20            &    27.64          &   70.05          &   39.08          \\
FIERY~\cite{hu2021fiery}      & 37.00                                      & 30.20         & 70.20        & 42.90        \\ \midrule
\textbf{ST-P3 Gaus.}                                   & 38.63                                    &  31.72        & 70.15       & 45.22       \\
\textbf{ST-P3 Ber.}                                  & \textbf{38.87}                                     & \textbf{32.09}        & \textbf{70.39}       & \textbf{45.59}       \\
\bottomrule
\end{tabular}}
\end{table}

\begin{table}[tb!]
\centering
\caption{\textbf{Open-loop planning results.} \textit{vanilla} approach is supervised with ground truth trajectories only. ST-P3 achieves the lowest collision rate in all time intervals}
\label{tab:planning}
\scalebox{0.8}{
\begin{tabular}{p{0.2\textwidth}|>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}|>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}>{\centering\arraybackslash}p{0.1\textwidth}}
\toprule
\multirow{2}{*}{Method}                     & \multicolumn{3}{c|}{L2 ($m$) $\downarrow$} & \multicolumn{3}{c}{Collision (\%) $\downarrow$} \\ \cline{2-7}
                                             & 1s      & 2s     & 3s     & 1s        & 2s        & 3s        \\ \midrule
Vanilla                     & \textbf{0.50}    & \textbf{1.25}   & \textbf{2.80}   & 0.68      & 0.98      & 2.76      \\
NMP~\cite{zeng2019end}      & 0.61    & 1.44   & 3.18   & 0.66      & 0.90      & 2.34       \\
Freespace~\cite{hu2021safe} & 0.56    & 1.27   & 3.08   & 0.65      & 0.86      & 1.64      \\  \midrule
\textbf{ST-P3}                & 1.33    & 2.11   & 2.90   & \textbf{0.23}      & \textbf{0.62}      & \textbf{1.27}      \\ \bottomrule
\end{tabular}}
\end{table}

\begin{table}[tb!]
\caption{\textbf{Closed-loop simulation results.} ST-P3 outperforms vision-based baselines in all scenarios, and achieves better route completion performance in long-range tests compared to LiDAR-based method. $^*$: LiDAR-based method}
\label{tab:close-planning}
\centering
\scalebox{0.8}{
\begin{tabular}{p{0.20\textwidth}|>{\centering}p{0.16\textwidth}>{\centering}p{0.16\textwidth}|>{\centering}p{0.16\textwidth}>{\centering\arraybackslash}p{0.16\textwidth}}
\toprule
\multirow{2}{*}{Method} & \multicolumn{2}{c|}{Town05 Short} & \multicolumn{2}{c}{Town05 Long} \\ \cline{2-5}
                        & DS $\uparrow$     & \multicolumn{1}{c|}{RC $\uparrow$}  & DS $\uparrow$             & RC $\uparrow$             \\ \midrule
CILRS~\cite{codevilla2019exploring} & 7.47   & 13.40                    & 3.68           & 7.19           \\
LBC~\cite{chen2020learning}       & 30.97  & 55.01                    & 7.05           & 32.09          \\
Transfuser$^*$~\cite{prakash2021multi}                     & 54.52     & 78.41                    & \textbf{33.15}          & 56.36          \\ \midrule
\textbf{ST-P3}                     & \textbf{55.14}       & \textbf{86.74}                         & 11.45               & \textbf{83.15}               \\
\bottomrule
\end{tabular}}
\end{table}

\subsection{Closed-loop Planning Results on CARLA Simulator}\label{sec: res-carla}

We conduct closed-loop experiments in CARLA simulator to demonstrate the applicability and robustness of ST-P3. It is far more challenging since the driving errors would stack up and lead to dangerous crashes.
Following~\cite{prakash2021multi}, we adopt the Route Completion (RC) - the percentage of route distance completed, and the Driving Score (DS) - RC weighted by an penalty factor that accounts for collisions with pedestrians, vehicles, \textit{etc}.
Tab. \ref{tab:close-planning} shows the comparison with two camera-based algorithms and a LiDAR-based SOTA method.
ST-P3 outperforms the camera-based methods on all metrics and is comparable with the LiDAR-based method.
Tansfuser~\cite{prakash2021multi} has a higher driving score in long routes mainly due to the lower penalty resulting from the shorter traveling distance.
ST-P3 obtains impressive route completion performance which indicates the ability of recovering from collisions, with help of the front-view vision refinement.

\subsection{Ablation Study}\label{sec: exp-ablation}

\begin{table}[tb!]
\caption{\textbf{Ablation on nuScenes validation set.} Exp.1-3 explore the effectiveness of depth supervision (Depth) and egocentric aligned accumulation module (EAA.) for perception. Exp.4-6 is on prediction module, with ``Dual.'' representing Dual Modelling and ``LFA.'' indicating loss for all timestamps. Exp.7-9 show the superiority of combining the sampler (S.) and refinement (R.) units in planning. ``V.IoU'' is the IoU metric of vehicles. ``V.PQ'' is the panoptic quality of vehicles. ``Col.'' denotes the collision rate}
\label{tab:p2_ablation}
\centering
\scalebox{0.8}{
\begin{tabular}{p{0.06\textwidth}|>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}|>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}>{\centering}p{0.1\textwidth}>{\centering\arraybackslash}p{0.1\textwidth}}
\toprule
Exp. & EAA. & Depth & Dual. & LFA. & S. & R. & V. IoU $\uparrow$ & V. PQ $\uparrow$ & L2 $\downarrow$ & Col. $\downarrow$ \\ \midrule
1    &         &         &       &       &       &     &38.00   &   -   &  - &    - \\
2    &\ding{51}&         &       &       &       &     &38.79   &   -   & -  &   -  \\
3    &\ding{51}&\ding{51}&       &       &       &     &40.10   &   -   &  - &   -  \\ \midrule
4    &\ding{51}&\ding{51}&       &       &       &     &37.09   &28.63  &  - &   -  \\
5    &\ding{51}&\ding{51}&\ding{51}&       &       &     &38.16   & 31.35 &  - &   -    \\
6    &\ding{51}&\ding{51}&\ding{51}&\ding{51}&       &     &38.63   & 31.72 &  - &   - \\  \midrule
7    &\ding{51}&\ding{51}&\ding{51}&\ding{51}&\ding{51}&       & - & - &2.128   &  0.850     \\
8    &\ding{51}&\ding{51}&\ding{51}&\ding{51}&       &\ding{51}& - & - &2.321   &  1.089    \\
9    &\ding{51}&\ding{51}&\ding{51}&\ding{51}&\ding{51}&\ding{51}& - & - &1.890   &  0.513    \\
\bottomrule
\end{tabular}}
\end{table}

Tab. \ref{tab:p2_ablation} shows the effectiveness of different modules in ST-P3. We report the vehicle IoU and vehicle PQ for perception and prediction tasks, L2 and collision rate for planning evaluation.
For Exp.1-3 in Tab. \ref{tab:p2_ablation} we present the impact of depth supervision and Egocentric Aligned Accumulation (EAA.) in perception. Note that Exp.1 is identical to FIERY~\cite{hu2021fiery}.
Our module improves \textbf{0.79\%} by adopting EAA. algorithem (Exp.2), and supervising depth explicitly brings an improvement to \textbf{1.31\%} (Exp.3).
Exp.4-6 demonstrate the impact of Dual Modelling and corresponding training method - loss for all states (LFA.) on prediction task.
Since Dual Modelling considers both uncertainty and historical continuity, the correctness of past features plays a vital role in it.
As the results show, these two strategies improves \textbf{1.54\%} and \textbf{3.09\%} to V.IoU and V.PQ respectively.
Exp.7-9 is on the sampler and GRU refinement unit in planning.
A sampler without front-view vision refinement (Exp.7) or an implicit model without prior sampling knowledge (Exp.8) both get a high L2 error and collision rate. Our design remarkably improves the safety and accuracy of the planned trajectory.

\section{Conclusions}

In this paper, we have proposed an interpretable end-to-end vision-based framework for autonomous driving tasks. The motivation behind the improved design is to boost feature representations both in spatial and temporal domains. An egocentric aligned accumulation to aggregate features in 3D space and preserve geometry information is proposed; a dual pathway modelling to reason about the probabilistic character of semantic representations across frames is devised; a prior knowledge refinement unit to take into account road elements is introduced. Together with these improvements within the ST-P3 pipeline, we achieve impressive performance compared to previous state-of-the-arts.

\section*{Acknowledgments}
The project is partially supported by the Shanghai Committee of Science and Technology (Grant No. 21DZ1100100). This work is also supported in part by National Key Research and Development Program of China (2020AAA0107600), Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102) and NSFC (61972250, 72061127003).

\bibliographystyle{splncs04}

\section{Implementation Details of the Network}
\subsection{Architecture for Perception and Prediction}

\subsubsection{Spatial-Temporal Perception.}
For nuScenes dataset, we first crop and resize the original image $\mathbb{R}^{3 \times 900 \times 1600}$ to $\mathbb{R}^{3 \times 224 \times 480}$ and take the past 3 frames to the model, denoted by $I_t^n \in \mathbb{R}^{3 \times 224 \times 480}$, where $t \in \{1,2,3\}, n \in \{1,\dots,6\}$.
We use EfficientNet-b4~\cite{tan2019efficientnet} as the backbone, obtaining features $f_i^k \in \mathbb{R}^{C \times H_e \times W_e}$ and depth estimation $d_i^k \in \mathbb{R}^{D \times H_e \times W_e}$ where $C = 64, D = 48, H_e = 28, W_e = 60$.
Note that the depth ranges from $2m$ to $50m$ with spacing $1m$.
After spreading the feature across the entire ray of space according to the predicted depth distribution, camera images from all angles are denoted by $u_i^k \in \mathbb{R}^{C \times D \times H_e \times W_e}$.
Then with the ego-motion matrix, features for all surround cameras and timestamps could be transformed to the coordinate system centered at the SDV at time $t$, resulting in ego-centric features $\{u_i^{'}\}$.
Finally the BEV features maps $b_i \in \mathbb{R}^{C \times H_b \times W_b}$ could be sum pooled from $\{u_i^{'}\}$ with $H_b =200, W_b = 200$.

In order to boost the features, we can integrate historical information to every frame through an accumulation method in the temporal fusion step.
Then these features are fed into a temporal network realized by 3D convolutions to better align temporal features.
In particular, in order to take different receptive fields into account, we apply different 3D kernel sizes on the time channel with $[(2,3,3),(1,3,3)]$ and a pyramid pooling with kernel size $(2,200,200)$.
We concatenate all the outputs to feed into the final compression convolution layer, getting the final output $x_{1 \sim t} \in \mathbb{R}^{C \times H_b \times W_b}$.
Note that the output features keep the same shape with the input after passing through the temporal network, and features from different timestamps are integrated.

\noindent
\subsubsection{Future Prediction.}
In our experiments, we model the future uncertainty by two different distributions: Gaussian and Bernoulli.
For Gaussian distribution, the present feature $x_t$ is passed through several residual block layers and an average pooling layer to get the hidden state $\mathbb{R}^{32 \times 1 \times 1}$.
Then a 2D convolution with kernel size $(1,1)$ fits the mean and log variance of the Gaussian with $R^{32} \times R^{32}$.
For Bernoulli distribution, since each grid of the spatial is $0-1$, we pass $x_t$ to several residual block layers and through a LogSigmoid to regress the probability in each grid.
For the prediction network, we utilize the convolutional Gated Recurrent Unit as our basic module, and each gate is realized by a 2D convolution with stride $3$.
We combine the predicted features through a trusting gate implemented by a 2D convolution which has 2 output channels, and use it as the weight to sum the two predicted features.
The mixed predicted features are then used as the hidden state for the ``uncertainty'' pathway
and the input state for the ``historical'' pathway.
Through this method, we could recursively predicts future states $(\hat{x}_{t+1},\dots, \hat{x}_{t+H})$.

\noindent
\subsubsection{Decoders for BEV Representations.}
All task heads share the same backbone considering the robustness of the decoder,  which is implemented by the first three layers of ResNet-18~\cite{he2016resnet} and three upsampling layers of factor 2 with skip connections.
Features now has 64 channels, and then are passed to different heads according to the task requirement. Each head is composed of two 2D convolutions with different output channels.
In particular, we set that the number of output channel for all semantic segmentation heads is 2. However, we need to predict offset, center, future flow for instance segmentation task,  thus they are set as 2, 1 and 2 respectively.

\subsection{Planning}
In this section we will give more detailed the scoring functions and the implementation of the refinement GRU network.

\noindent\textbf{Safety Cost.} The SDV should not collide with other objects on the road and need to consider their future motion when planning its trajectory.
For this purpose, we use the predicted occupancy map to penalize the trajectories that intersect with the occupied regions.
Formally, for trajectories $\tau$ at each timestamp $t$, we will penalize $\tau$ if the SDV polygon $g$ intersects with the grids which are occupied by other objects (with a safety margin indicated by parameter $\lambda$), denoted by the $o_g(\tau, t, \lambda)$. The safety cost related to objects collision is given by:
\begin{equation}
    f_o(\tau, o) = \sum_t \sum_g o_g(\tau, t, 0) + o_g(\tau, t, \lambda) v(\tau, t),
\end{equation}
where the first term penalizes the intersection grids whereas the second term penalizes the high-velocity motion with uncertainty occupancy.

Moreover, SDV usually follows a leading vehicle and should keep a certain safe distance from it which mainly depends on the speed of the leading vehicle.
Since the HD map is unavailable and thus the leading vehicle along the center lane is unknown, instead we compute the occupancy grid in front of the SDV with a distance $L$ determined by the SDV velocity.
Hence if we are too close to the leading vehicle, the cost will be large that the trajectory will be evaluated as bad.
However it cannot take effect when it is in lane change manoeuvres since the leading vehicle is not directly in front of SDV.

The above objectives focus on moving objects, meanwhile the vehicle should stay in the middle of the lane line as well.
We can ensure this traffic safety with the perceived lanes in the first stage, with a cost function that is set as the distance to lane lines.

\noindent\textbf{Refinement.} Getting the selected trajectory $\tau^*$ according to the scoring functions, we then refine it through a GRU network. Specially, the hidden state of the GRU is the feature of the front camera from the encoder, and the input state is the concatenation of the current position, the position from the selected trajectory $\tau^*$ and the target point. Note that the initial current position is $(0,0)$. Doing the same process, we could recursively obtain the final refinement trajectory taking into consideration the front camera information and the target point information.

\section{Depth Supervision}

In this section, we introduce how the depth maps of nuScenes dataset are generated for explicit supervision.

We adopt a self-supervised, semantic segmentation required method named FSRE-Depth~\cite{Jung_2021_ICCV}, which has great performance on depth generation. Since FSRE-Depth use semantic segmentation as input, we first train a segmentation model on Mapillary Vistas Dataset~\cite{Mapillary}. Then we apply it on nuScenes dataset to generate semantic results, as shown in Fig. \ref{fig:depth_vis}(b).
In the next step, We train FSRE-Depth with front view images in nuScenes, using ResNet-50 as the backbone. The resolution of input images is $1216 \times 672$. After 45 epochs of training, the model can predict sufficient results on front view images. To leverage the performance of depth model, we further train it on each camera view for 15 epochs.
Thus, our depth maps of all camera views could be produced by this model individually. To evaluate our depth maps, we calculate depth estimation metrics with LiDAR points every 20 images. The result is shown in Tab. \ref{tab:depth_map_eval}, which indicates that our depth maps have decent quality.
This result in Fig. \ref{fig:depth_vis}(c) would be utilized to explicitly supervise the depth estimation in the perception module of ST-P3.

\begin{figure}[tb!]
    \centering
    \subfigure[]{
    \includegraphics[width=0.75\textwidth]{Figs/depth_origin.jpg}
    }
    \quad
    \subfigure[]{
    \includegraphics[width=0.75\textwidth]{Figs/depth_seg.jpg}
    }
    \quad
    \subfigure[]{
    \includegraphics[width=0.75\textwidth]{Figs/depth_depth.jpg}
    }
    \caption{(a) Original image; (b) Segmentation result; (c) Predicted depth map}
    \label{fig:depth_vis}
\end{figure}

\begin{table}[]
\centering
\caption{Depth map evaluation. ARE: absolute relative error; SRE: square relative error; RSME: root mean square error; RSME log: root mean square logarithmic error; $\delta$: accuracy (threshold 1.25)}
\label{tab:depth_map_eval}
\scalebox{0.8}{
\begin{tabular}{p{0.2\textwidth}|>{\centering}p{0.13\textwidth}>{\centering}p{0.13\textwidth}>{\centering}p{0.13\textwidth}>{\centering}p{0.13\textwidth}>{\centering}p{0.13\textwidth}>{\centering}p{0.13\textwidth}>{\centering\arraybackslash}p{0.13\textwidth}}
\toprule
View        & ARE & SQE & RSME   & RSME log & $\delta$  & $\delta^2$  & $\delta^3$  \\
\midrule
FRONT       & 0.1522  & 3.5165 & 6.8756 & 0.2252   & 0.8720 & 0.9474 & 0.9701 \\
FRONT LEFT  & 0.2516  & 2.6591 & 5.7600 & 0.3035   & 0.7290 & 0.8755 & 0.9317 \\
FRONT RIGHT & 0.3030  & 5.9052 & 6.6903 & 0.3278   & 0.7067 & 0.8651 & 0.9231 \\
BACK        & 0.2297  & 5.5757 & 7.8486 & 0.3020   & 0.8011 & 0.9159 & 0.9535 \\
BACK LEFT   & 0.2752  & 3.6375 & 5.7691 & 0.3110   & 0.6982 & 0.8673 & 0.9279 \\
BACK RIGHT  & 0.3021  & 3.9396 & 6.2027 & 0.3413   & 0.6605 & 0.8503 & 0.9279 \\ \bottomrule
\end{tabular}}
\end{table}

\section{Experiments}
\subsection{Protocols}
\indent\textbf{Dataset.}
We evaluate ST-P3 in both open-loop and closed-loop environments.
We adopt nuScenes dataset~\cite{caesar2020nuscenes} for the open-loop setting,  and CARLA simulator~\cite{dosovitskiy2017carla} for the closed-loop demonstration.

For nuScenes, by default we take the $1.0s$ of past context and predict the future $2.0s$ contexts, which corresponds to 3 frames in the past and 4 frames in the future.
Since each batch input to the model contains 7 frames of contexts, we just follow the offical split method to split the dataset into training, validation that consist of 26124 and 5719 samples, respectively.

For CARLA,
we conduct closed-loop evaluation. Note that we still need to train our model in a open-loop manner first.
We follow the dataset collection in~\cite{prakash2021multi} and use the Town05 scenario for evaluation and the rest for training.
It is worth mentioning that  the camera setup in CARLA only consists of 4 cameras; thus it could not cover the $360^{\circ}$ field of view around the SDV.
The context from the past 1.0$s$ and current 4 cameras images are passed to our model, then a trajectory is produced and executed by the simulator until SDV getting to the destination or surpassing the time limitation.

\noindent\textbf{Implementation Details.}
We adopt EfficientNet-B4~\cite{tan2019efficientnet} as the backbone, and detailed model description would be illustrated in the Supplementary. We use the Adam optimizer with a constant learning rate $2 \times 10^{-4}$. We train our model on 4 Tesla V100 GPUs for 20 epochs at mixed precision.
The BEV spatial is $(100m, 100m)$ around the SDV with resolution $(0.50m,0.50m)$ on nuScenes dataset, following the setting in~\cite{hu2021fiery} for a fair comparison. While on CARLA, it is $(40m, 40m)$ around SDV with resolution $(0.20m, 0.20m)$. The time interval between two consecutive frames is 0.5$s$ for both datasets.

\subsection{Open-loop Experimental Results on nuScenes}

In this section we show more qualitative results on nuScenes dataset~\cite{caesar2020nuscenes}. We present the visualization of the learned cost volume and the composite graph of multiple semantic elements in the meantime.
Note that darker color means smaller cost value here, and vice versa. As shown in Fig. \ref{fig:nu_plan}-\ref{fig:nu_turning}, basically all the places occupied by cars have higher cost values, while open drivable areas have lower cost values.

\begin{figure}[tb!]
    \centering
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/nu_0319.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/nu_0169.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/nu_0295.png}
    }
    \caption{Qualitative results of ST-P3 on the straight road. We show our BEV intermediate representations and planned trajectory (\textcolor{blue}{blue}) in the right. We also present the learned subcost map from prediction module. Note that a darker color indicates a smaller cost value}
    \label{fig:nu_plan}
\end{figure}

\begin{figure}[tb!]
    \centering
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/nu_0021.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/nu_0044.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/nu_0376.png}
    }
    \caption{Qualitative results of ST-P3 when turning at intersections}
    \label{fig:nu_turning}
\end{figure}

\subsection{Closed-loop Planning Results on CARLA Simulator}
In this section we show the qualitative results on the CARLA simulator~\cite{dosovitskiy2017carla}, similar to the open-loop results.
As shown in Fig. \ref{fig:error_recover}, when the car deviates from the center of lanes, the detection accuracy of the map drop significantly.
This is probably because the data collection strategy that most training data is on normal circumstance.
When the map accuracy is low, the traditional method which utilizes the sampler and cost map generally deviates from the expected track behavior, but due to our refinement operation, the car can still travel to the expected track correctly.
We provide more visualization in different scenarios in Fig. \ref{fig:normal}-\ref{fig:turning}.

\begin{figure}[tb!]
    \centering
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0008.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0009.png}
    }
    \caption{Qualitative results of ST-P3 in closed-loop planning on the straight road}
    \label{fig:normal}
\end{figure}

\begin{figure}[tb!]
    \centering
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0026.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0028.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0029.png}
    }
    \caption{ST-P3 predicts a slowing down trajectory when detecting a pedestrian}
    \label{fig:slow_down}
\end{figure}

\begin{figure}[tb!]
    \centering
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0227.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0230.png}
    }
    \caption{BEV representations of ST-P3 become normal when driving on a predetermined trajectory (centerlines)}
    \label{fig:error_recover}
\end{figure}

\begin{figure}[tb!]
    \centering
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0384.png}
    }
    \quad
    \subfigure{
    \includegraphics[width=0.98\textwidth]{Figs/0413.png}
    }
    \caption{Qualitative results of ST-P3 in closed-loop when turning at intersections}
    \label{fig:turning}
\end{figure}

redpajama_set_name: RedPajamaArXiv


text:
Yuneec H520 commercial drone is now available for purchase
By Jonathan FeistAugust 29, 2017
Ryan Vergara
Today is the day, folks, it's August 29, 2017 and the folks at Yuneec just informed me that their latest drone is ready to go, we can all now purchase the Yuneec H520.
We first put hands on this drone at CES 2017. This is where I must make an apology, I lost the audio of that experience, else I'd have a video of our excellent interview from the show. Let me just say, if you are looking for a powerful and versatile commercial drone, this is a superb machine to consider.
Top drone manufacturers – Companies that fly
Yuneec drones guide – including the Typhoon H Plus and Mantis Q
Yuneec Drones Guide
At its core, the Yuneec H520 is a beefed up Typhoon hexacopter. Smaller than the H920 series, but just as tough. The drone utilizes the best of accessories, connectivity and, especially, cameras that Yuneec has to offer.
The key to the Yuneec H520 is the interchangeable cameras. Swap between the E50 or E90, with 1/2.3-inch CMOS 12MP or 1-inch 20 MP sensors, respectively. Beyond normal optics, install the CGOET infrared camera for law enforcement or power line operations. It's not stated in the online description at this time, but were were told that the H520 will also accept the existing CGO3 and CGO4 cameras that you'd get on the Typhoon. We'll be sure to update if that is no longer the case.
The base drone purchase will get you two batteries, the airframe and the ST16S remote control. No camera.
While in the air, the landing gear lifts out of the way to allow 360 degree camera rotation. The H520 has a take-off weight of about three and a half pounds with camera installed, scoring a flight time of up to 28 minutes. While up there, scoot along at a stabilized ground speed of about 30 mph, 'sport' mode closer to 40 mph.
The ST16S ground station is a remote control I wish I could use for more drones. It runs Android, has 16 channels of connectivity, a built-in 7-inch display and a range of about 1 mile.
We do not have an Amazon link yet, which we find offers the best shipping options for most of the globe, and usually a discount over going direct to the manufacturer. That said, you can go straight to Yuneec to grab the Yuneec H520 for $1999.99. The E50 camera can be had for $1099.00 and the E90 camera for $1299.99. You can grab additional batteries and propellers as well. We'll save that for another day.
Buy the Yuneec H520
Stay tuned for more coverage of this drone and more from Yuneec. We will be meeting up with them at InterDrone in early September, hopefully we can revisit that Yuneec H520 demo and interview, and maybe there will be something new to look at as well.
Catch Yuneec's demo video after the what's next section below:
Autel Robotics Drones
YuneecYuneec H520
Yuneec, Yuneec H520
Yuneec Mantis Q review – we like where Yuneec is headed
Yuneec drones at InterDrone 2018 – Mantis Q, 3DR and E10T thermal camera
Yuneec Mantis Q announced – specs, price and availability

redpajama_set_name: RedPajamaCommonCrawl


text:
February 07, 2011 Arts & Culture » Film
I Spit on Your Grave (1978/2010)
Being 9 at the time, I was so freaked out, I vowed never to watch the rape-revenge film they so viciously, venomously attacked.
Never did I give "Spit" a try, nor did I want to, until its Blu-ray debut, released simultaneously with its 2010 remake of the same name, when curiosity finally got the best of me. Now that I've seen it, I can throw an educated two cents into the ongoing debate of whether the controversial thriller exploits women or rather is a misunderstood champion of feminism. And that is: Can't it be a little of both?
In the 1978 original , titled "Day of the Woman" before a re-release a couple of years later, Jennifer (Camille Keaton, grandniece of Buster) retreats to a cabin in the woods to write her first novel. Her solitude — and very soul — are shattered when four redneck locals gang-rape her. For one of them, the mentally handicapped Michael, it represents the loss of his innocence, in more ways than one. Instead of calling the police, Jennifer wills herself into a superwoman and takes bloody revenge on each of them.
Last year's remake tells the same sordid story, right down to retaining the characters' names. Per the current remake rules of upping the ante, this time Jennifer (newcomer Sarah Butler) is raped not by four men, but five. From there, director Steven R. Monroe takes his more polished (but still ugly) version down a slightly different path, rearranging the order of Jennifer's murders and making them more gruesome, even culminating in one seemingly pulled from a discarded draft of a "Saw" sequel.
Both films "work," in that viewers are made to feel boiling-point hatred toward the men and empathy for Jennifer, to the point that you want to see the guys punished for their crimes, and believe me, you get to. (Each version has a scene of comeuppance that'll make any guy wince in pain.) Whether the product of low budgets or directorial intent, their grimy, bleak looks contribute to experiences so downbeat, so disturbing, so exhausting, I hardly consider them "entertainment."
At the same time, they are effective in relaying a very simple story; I hesitate to call it a plot. I'll admit I chose to watch the original via subtitles and the commentary of the legendary drive-in movie critic Joe Bob Briggs, who adds a little levity to the grim proceedings. He's also, as always, a wealth of information, providing details both serious and funny ("This was the first in a series of one role for Matthew") as he alternately defends and mocks the movie.
Briggs says that Keaton didn't get the recognition she deserved for her rather brave performance, and I have to agree, extending that to Butler as well (she's the better actress, anyway; the remake boasts more professional acting and production values all around). Jennifer is stripped naked, literally and figuratively, and robbed of everything that you have to wonder, "What madman thought all this up?"
Well, it's Zarchi, and you'd assume him to be a misogynist sleaze. Instead, in the half-hour conversation with him on the original disc's extras, he appears to be a nice, normal guy — just not an extraordinarily talented filmmaker. He was inspired to write the movie after witnessing the aftermath of a rape in real-life, and coming to the victim's aid, even when the police wouldn't.
Thus, despite the nudity and the graphic depiction of the acts (all simulated, mind you), there is nothing erotic or arousing about the film. You've heard experts suggest that rape is not about sex, but power? I get that now. "I Spit on Your Grave" drives that point home before shifting that balance of power to satiate the victim's desire for revenge.
Granted, "I Spit on Your Grave" isn't for everyone. It's not even for a quarter of everyone. Whichever version viewed, if it doesn't bother you, there's something wrong with you. —Rod Lott
Legendary Rain
OCU launches self-searching film series
Film review: Life Itself
Tags: Film, 1978, 2010, Camille Keaton, Day of the Woman, Gene Siskel, I Spit on Your Grave, Meir Zarchi, Roger Ebert, Sarah Butler, saw, Sneak Previews, Steven R. Monroe, Very Special Episode, Video


If you follow any of the local golf community pages on Facebook recently, you'd see many people becoming more and more infuriated with the high fees charged by golf courses.
They're screaming about high priced memberships (which nobody wants to buy at the moment); high annual dues (which really aren't high at all compared to the US, for example); mandatory caddies and buggies; some clubs demanding an F&B minimum per year of members; the price of a set of golf clubs, etc.
If you understand how golf evolves in any given country, you'd realize that Việt Nam is no different than anywhere else and, amazingly enough, they are right on schedule.
First, golf starts out by being available only for the rich. They restrict their clubs through many different ways: Religion, social status, income, gender. The middle class earners are not allowed to join these private clubs and get pissed off enough to build their own clubs.
Yes, of course the lower class income earners get pissed because they cannot join the middle class clubs, so they scream until someone with half a brain hears them and thinks "opportunity".
The local government, or someone with brains, then opens a daily fee, public, open to everyone golf course. There are no caddies. You can rent a pull trolley and lug your clubs around yourself, or use a lightweight bag and carry them. F&B consists of a halfway house with hot dogs, burgers and an occasional chicken leg. Sometimes the F&B is actually contracted out to a third party who takes all the risk.
The golf course itself is nothing to write home about. Often it is designed by someone who barely knows what they are doing and constructed cheaply by someone who knows even less. Maintenance is bare bones… literally.
But, it's golf and people looking to play a game cheaply happily accept it… for now.
Eventually, these people just entering the game at the cheapest public course level then begin to move up the economic ladder and join middle class clubs or they decide to buy a public course themselves and upgrade it. They raise the fees slightly to cover the improvements and most people happily accept it. They can play and have a good time without bankrupting the family.
Here in Việt Nam, we started the same way… almost.
When I opened Kings' Island (Đồng Mô) in August 1993, we were a "semi-private" club. In other words, we were a public course that sold memberships to help quickly recover the investment costs.
Being the only club in the north at the time, selling memberships was easy. It was even easier when you realize the "Founding Memberships" were only US$5,000… Yes, you read that correctly. Of course they slowly increased over time until we are where we are now… overpriced.
We reached the crossroads where players are tired of high prices and new potential players are afraid they cannot afford it, but they can if they are smart about it.
With so much competition nowadays, many clubs are offering discounts for every reason under the sun. Weekdays, weekdays after 2pm, weekdays after 4pm, Ladies day, Twilight golf, Early Bird specials, Golf plus free lunch… Marketing managers and GM's are always looking for ways to fill the course.
There are also various tourist and card companies which offer a very low rate package at many different courses.
The easiest way I know to get discount or free golf is to work for the golf course. Volunteer to be a Marshal on weekends in exchange for free golf rounds. Make a deal to bring a few groups to the course in exchange for free golf for yourself. With a little imagination, you can find/create some great deals for yourself.
Remember, the GM's of these clubs are not just looking for revenue, but also ways to reduce expenses. Offer them a way to get something they need in exchange for free/discount rounds and they might go for it.
Golf equipment doesn't have to be expensive either.
Some players who buy a new set every year try to sell their old ones. Pro Shops are full of excellent secondhand equipment.


Presenting Woochie Pro's very creepy Halloween Scarecrow! This full face mask extends down the neck. It is a soft foam latex appliance mask like they use in TV and Film. Professional quality makeup effects are one of the best ways to bring your character to life. When applied this mask moves with your expressions just like professional Hollywood movie makeup. This appliance can be worn for hours in relative comfort.


Section 5 discusses Landscapes and Destinations, including the transformation of space into place, issues of authenticity in landscape, the transformation of urban and rural landscapes into tourism products and conservation versus development dilemmas. The text is international in focus, encouraging dialogue across disciplinary boundaries and areas of study and will be an invaluable resource for all those with an interest in Cultural Tourism. Brazil offers many cultural and natural attractions but, similar to many other developing countries, it still struggles with issues such as infrastructure, accessibility, product development, service quality, market access and workforce training. These population movements, she argues, likewise are both a cause and effect of economic growth. Bookman is professor of economics at St.
Section 6 refers to Regeneration and Planning, especially the creative turn in cultural tourism, which can be used to avoid problems of serial reproduction, standardisation and homogenisation. Indigenous communities in particular are responding to these opportunities in innovative ways that set them apart from their non-Indigenous predecessors and competitors. Growth and Population Movements in Tourist-Friendly Countries. Brazil offers many cultural and natural attractions but, similar to many other developing countries, it still struggles with issues such as infrastructure, accessibility, product development, service quality, market access and workforce training. Indigenous communities in particular are responding to these opportunities in innovative ways that set them apart from their non-Indigenous predecessors and competitors. The book uncovers some interesting findings that will significantly contribute to the literature on ethnic tourism in developing areas. .
Section 3 focuses on social patterns and trends, such as the mobilities paradigm, performativity, reflexivity and traditional hospitality, as well as considering sensitive social issues such as dark tourism. The social, economic, political and environmental contexts of this emerging global power provide an intriguing backdrop. The social, economic, political and environmental contexts of this emerging global power provide an intriguing backdrop. This section also looks at ways in which cultural tourism is used as a policy instrument for economic development. It explores the impacts of tourism on the environment, changing community attitudes towards tourism, transport infrastructure and sustainability issues in events. This significant volume offers the reader a comprehensive synthesis of this field, conveying the latest thinking and research.
© University of Toronto Press 2018. Section 7 deals with The Tourist and Visitor Experience, emphasising the desire of tourists to be more actively and interactively engaged in cultural tourism. This is essential reading for students, researchers and academics of Tourism as well as those of related studies in particular Cultural Studies, Leisure, Geography, Sociology, Politics and Economics. This significant volume offers the reader a comprehensive synthesis of this field, conveying the latest thinking and research. Section 7 deals with The Tourist and Visitor Experience, emphasising the desire of tourists to be more actively and interactively engaged in cultural tourism. Streams of Tourists: Navigating the Tourist Tides in Late 19th Century Southeast Alaska, Kathryn Bunn-Marcuse 2. List of Figures vii Preface ix Acknowledgments xv Notes on Contributors xvii 1 Current Themes in Indigenous Tourism 1 30 Alexis Celeste Bunten Nelson H.
Indigenous Tourism Movements is a sophisticated, sensitive, and refreshingly frank examination of Indigeneity in the contemporary world. They are not, however, a panacea for developing countries. They juxtapose diverse expressions of identity - from the commodification of Indigenous culture to the performance of heritage for tourists - to illuminate the complex local, national, and transnational connections these expressions produce. The revival of Islam among Muslim populations in the last twenty years is often wrongly perceived as a backlash against westernization rather than as one of its consequences. This section debates tourist autonomy role play, authenticity, imaginaries, cross-cultural issues and inter-disciplinarity Section 2 analyses the role that politics takes in cultural tourism. He and others were inspired by speakers such as entrepreneur Kylik Kisoun-Taylor of who captured the hearts of delegates with his enthusiastic approach to business. Growth and Population Movements in Tourist-Friendly Countries.
Deriding Demand: A Case Study of Indigenous Imaginaries at an Australian Aboriginal Tourism Cultural Park, Alexis Celeste Bunten 2. Or to Europe to experience the Mohawk. Bookman's multicountry analysis demonstrates forcefully that tourism both induces migration and displacement and is enabled by them, in a self-reinforcing circular flow. It is for those who are begin ning to get a grip on the topic, writing research proposals, preparing Field Statements, undertaking research, and writing up honors, masters or doctoral or postdoctoral research Cultural tourism is frequently marketed as an economic panacea for communities whose traditional ways of life have been compromised by the dominant societies by which they have been colonized. Authentication is an interactive process in which a balance of forces defines a state of equilibrium. Kim lives in Calgary, Alberta on Treaty 7 territory.
Indigenous tourism as a transformative process: the case of the Embera in Panama, Dimitrios Theodossopoulos 2. Author by : Alexis C. The text is international in focus, encouraging dialogue across disciplinary boundaries and areas of study and will be an invaluable resource for all those with an interest in Cultural Tourism. Giraudo 7 The Commodification of Authenticity: Performing and Displaying Dogon Material Identity 140 25 Laurence Douny Part Three Knowledge Movements 8 Streams of Tourists: Navigating the Tourist Tides in Late-Nineteenth-Century Southeast Alaska 165 33 Kathryn Bunn-Marcuse 9 Experiments in Inuit Tourism: The Eastern Canadian Arctic 198 24 Nelson H. The book explores the multi-faceted nature of this important phenomenon, drawing on current work in sociology, cultural studies, ethnography, and language studies. He shows how neofundamentalism acknowledges without nostalgia the loss of pristine cultures, constructing instead a universal religious identity that transcends the very notion of culture. At the heart of this development is, on the one hand, the voluntary settlement of Muslims in Western societies and, on the other, the pervasiveness and influence of Western cultural models and social norms.
Tourism is bound closely together with such contexts; feeding and challenging festivals with audiences that are increasingly transient and transnational. The contributors attempt to assess whether the last decade of the twentieth century was a time of constructive transition and whether new patterns of relations are emerging after the recent challenges to the colonial legacy by Aboriginal people. Bunten and Nelson Graburn, along with a diverse group of contributors, frame tourism as a critical lens to explore the shifting identity politics of Indigeneity in relation to heritage, global policy, and development. The book is divided into 7 inter-related sections. Section 1 looks at the historical, philosophical and theoretical framework for cultural tourism. No question, the Indigenous tourism movement has gained major traction in short order and the momentum is impressive. Then, the next big step is authenticity.
Indigenous communities in particular are responding to these opportunities in innovative ways that set them apart from their non-Indigenous predecessors and competitors. The book considers important development issues such as the changing policy context, community benefit tourism and indigenous tourism. It brings together leading specialists from a range of disciplinary backgrounds and geographical regions, to provide state-of-the-art theoretical reflection and empirical research on this significant stream of tourism and its future direction. Investigation into the visual representation and communication of identity through tourism is a means to illustrate three overlapping issues that are prevalent throughout the literature on Indigenous tourism. Indigenous communities in particular are responding to these opportunities in innovative ways that set them apart from their non-Indigenous predecessors and competitors. By using the study of folk villages on Hainan Island, China, the book suggests that authenticity evolves from a static into a more dynamic concept, which can be formulated according to the different stages of development relating to all the stakeholders involved.
Section 7 deals with The Tourist and Visitor Experience, emphasising the desire of tourists to be more actively and interactively engaged in cultural tourism. These population movements, she argues, likewise are both a cause and effect of economic growth. The book draws from theoretical foundations and practical insights, and gives voice to Brazilian researchers who are actively engaged in researching tourism. This book examines some well established controversies in tourism and some newly emerging controversial aspects associated with tourism as an activity and a business. Section 3 focuses on social patterns and trends, such as the mobilities paradigm, performativity, reflexivity and traditional hospitality, as well as considering sensitive social issues such as dark tourism. This site is like a library, you could find million book here by using search box in the widget.


text:
W7 Content: Raiders-Chiefs rivalry
General Published on : 1/1/2005
Raiders, Chiefs go way back
Ex-AFL rivals meet Sunday for 80th time in regular season
by profootballhof.com
(more on the this rivalry at nfl.com)
When the Raiders and Chiefs take the field at Arrowhead Stadium Sunday, it will mark the 80th regular season meeting between the two teams. Add three post-season games to the series history and you have one of the most heated rivalries in the NFL.
The first matchup between the two franchises came in 1960, the AFL's inaugural season. At the time it was the Dallas Texans versus the Oakland Raiders. Following the '62 season, the Texans moved to Kansas City and were renamed the Chiefs. The Raiders played in Oakland through 1981 and then moved to Los Angeles for 13 seasons before relocating back to the Bay area in 1995.
The Raiders defense corrals the Chiefs' Barry Word in the final regular season game of 1991. The two teams battled again six days later in a playoff game. Kansas City won both games.
Regardless of location or win-loss records, the annual battles between these two storied franchises have always brought out the best in each team. The distaste for each other is engrained in players very early in their careers.
"I remember when I was a rookie, the older guys would all talk about how special Raider week was," once commented Chiefs center Tim Grunhard. "They made a point to let us know about all the history that's gone on here, all the bad blood. It's a neat rivalry, it really is."
The regular season series is extremely close. The Chiefs have won 40 games while the Raiders have won 37 times. The teams tied on two occasions. In postseason the Chiefs hold a 2-1 edge.
Kansas City has dominated the series over the past decade. Their 17-3 mark versus the Raiders in the 1990s tied them for the NFL's best record against one club during that ten-year span. That is in stark contrast to the first thirty years of the rivalry. The teams split the 1960s, with each team winning 10 regular season games and one post-season game. In the 1970s, the Raiders claimed the series advantage with 12 wins compared to the Chiefs' six victories. The teams tied twice. In the 1980s, the Raiders earned the upper hand again with 12 wins to the Chiefs' seven wins.
In 1968, the teams faced each other in a post-season contest for the first time. The Raiders came away with a 41-6 drubbing of the Chiefs in an AFL divisional playoff game to earn a berth in the AFL championship game. The following season, the Chiefs and Raiders again met in the playoffs - this time with the AFL title on the line. Kansas City sought and earned revenge with a 17-7 win that placed them in Super Bowl IV.
It was 22 years before the teams would meet again in the post-season. In 1991, the Raiders traveled to Kansas City to face off against the Chiefs in the first round of the AFC playoffs. Sixty minutes later, K.C. walked away with a narrow 10-6 victory in the first-ever playoff game at Arrowhead Stadium.
Week 7 poll: Best AFL Franchise
W8 poll (Best season ever)


text:
Spencer Larsen (born March 4, 1984) is a former American football fullback. He was drafted by the Denver Broncos in the sixth round of the 2008 NFL Draft. He played college football at Arizona, primarily as a linebacker.

Early years
Larsen earned SuperPrep All-Farwest honors and was named First-team All-State by The Arizona Republic as a senior at Highland High School in Gilbert, Arizona after recorded 124 tackles as a senior. He was also the All-Region co-Defensive Most Valuable Player.

College career
In 2007 Larsen was a First-team All-Pacific-10 Conference as a senior at Arizona after leading the conference with 131 tackles (87 solo) in 12 games (12 starts). He added four sacks, one interception, five pass breakups, four fumble recoveries and three forced fumbles. The year before, 2006, Larsen received Second-team All-Pacific-10 Conference honors as a junior after making 89 tackles (63 solo), two sacks (11 yds.) and one interception in 12 games (12 starts). He also had three pass breakups, three fumble recoveries and two forced fumbles. He was also First-team Academic All-Conference. In 2005, he saw action in eight games (6 starts) as a sophomore and totaled 51 tackles (29 solo), one sack, one pass breakup and one fumble recovery. He also received Second-team academic All-Conference recognition. Larsen did not play football from 2003–04 because he was on a mission for the Church of Jesus Christ of Latter-day Saints in Chile. In 2002 Larsen played all 12 games (7 starts) as a true freshman for Arizona and recorded 41 tackles (30 solo), two sacks  and three pass breakups and was named Third-team freshman All-America and First-team freshman
All-Pacific-10 Conference by The Gridiron Report. He was the defensive MVP of the 2008 East–West Shrine Game.

Professional career

Pre-draft

Denver Broncos
On November 16, 2008, in a Broncos 24–20 road win over the Atlanta Falcons, Larsen became the first NFL player since 2003 (and only the fourth since 1990) to start a game on both offense (fullback) and defense (linebacker).  The positions fullback and linebacker both involve considerably more contact than the wide receiver / cornerback positions played by other modern two-way players such as Troy Brown, Deion Sanders, Julian Edelman, and Champ Bailey. He also played on special teams during the game, including the opening kickoff.  Larsen earned Diet Pepsi NFL Rookie of the Week honors for the first time in his career for his performance against the Falcons.

New England Patriots
Larsen signed with the New England Patriots on March 22, 2012. He was placed on season-ending injured reserve on August 27, 2012 and released from the team on March 4, 2013.

Tampa Bay Buccaneers
On July 27, 2013, Larsen signed with the Tampa Bay Buccaneers. On August 26, 2013, he was released by the Buccaneers. Larsen re-signed with the team on September 10, 2013. He was released again on September 14, 2013. He was re-signed on September 25, 2013.

References

External links
 Arizona Wildcats bio

1984 births
Living people
People from Gilbert, Arizona
Sportspeople from Mesa, Arizona
Players of American football from Arizona
American football fullbacks
American football linebackers
Arizona Wildcats football players
Denver Broncos players
New England Patriots players
Tampa Bay Buccaneers players
Latter Day Saints from Arizona
21st-century Mormon missionaries
American Mormon missionaries in Chile

redpajama_set_name: RedPajamaWikipedia


text:
You are here: Home / Press Release Wire / OPD5 Extends Temporary Office Closure Due to COVID-19
January 12, 2021 By Press Release Wire Leave a Comment
OPD5 has extended the temporary closure of both its Overton and Mesquite offices to help mitigate the spread of COVID-19.
OPD5 closed public access to its offices and facilities at the end of November and announced Tuesday that those offices will remain closed to the public until at least Feb. 10, in compliance with Nevada Gov. Steve Sisolak's 30-day extension of a "Statewide Pause."
OPD5 staff will continue to be available during regular business hours of 8 a.m. to 4:30 p.m. via phone and e-mail to answer customer inquiries. OPD5 crews will also continue to operate, maintain and conduct any system repairs if a power interruption or problem were to occur during this time.
Customers who have essential business needs that cannot be conducted electronically, such as new customers signing up for service, can schedule an appointment, if needed, to meet with a customer service representative.
OPD5 also offers several contactless payment options. Payments can be made online or through its mobile app, which is available for download through the Apple Store and Google Play store. Customers can also make payments by phone using the Interactive Voice Response (IVR) system or by speaking with a customer service representative. Additionally, customers can set up autopay from a debit or credit card or checking or savings account.
For more information, visit www.OPD5.com, e-mail inquiries to customerservice@opd5.com or call 702.397.2512 to reach the Overton office or 702.346.5710 for the Mesquite office.
About OPD5
Overton Power District #5 (OPD5) is a not-for-profit power provider with offices in Overton and Mesquite, Nevada. Formed in 1935, OPD5 provides electric power to more than 16,000 customers in a service area spanning more than 1,932 square miles of northeastern Clark County, including the communities of Mesquite, Overton, Logandale, Bunkerville and other parts of the Moapa and Virgin Valleys. For more information, visit www.OPD5.com, and follow us on Facebook, Twitter and Instagram.
Filed Under: Press Release Wire Tagged With: Overton Power District #5 (OPD5)


text:
Pine Lakes Country Club
SC Picture Project / Horry County / Pine Lakes Country Club
The historic Pine Lakes International Country Club in Myrtle Beach was developed as Ocean Forest Country Club in 1926. The golf resort was the enterprise of Greenville textile tycoon and venture capitalist John T. Woodside, founder of Woodside Mill as well as several businesses around the state, including hotels, banks, and other mills. He and two of his brothers purchased 65,000 acres of what was then rural property around Myrtle Beach, including prime ocean front land, and immediately began establishing a golf course. By 1927 a hotel and 27-hole golf course designed by Robert White, the first president of the Professional Golfers Association, had altered the landscape of the burgeoning beach town, paving the way for Myrtle Beach to become one of the nation's most popular vacation destinations.
Bill Fitzpatrick of Taylors, 2013 © Do Not Use Without Written Consent
The clubhouse, seen above, served as the hotel until the Ocean Forest Hotel (below) was completed in 1930. Unfortunately, soon after the hotel was completed, the the Woodsides lost their Myrtle Beach real estate holdings along with most of their other businesses – including the Woodside Mill – as a result of the stock market crash. Frederick Albert Warner Miles later purchased the beach property, and in 1944 he began plans to restructure the resort. According to the National Register, in 1946 he sold 18 of the 27 golf holes to developer John McLeod and hired Robert Smith to redesign the remaining nine holes and add an additional nine. During this time Miles changed the name of the club from Ocean Forest to Pine Lakes.
Contributor Bill Chapman, whose parents purchased a lot from Mr. Robert White in 1948, offers a differing account, he says: "Robert White, a Scotsman and first president of the PGA was the designer of Pine Lakes. He lived on Woodside Avenue across from the golf course. My parents bought a lot from Mr. White in 1948 and built a house on Poinsett Road. Poinsett ran from the Hotel on the oceanfront to the golf course. Robert White had died when another nine holes was added in the early 1960s." He continued with "I know about what happened to 18 of the 27 original holes. Nine were sold off and nine just went back to nature, and in the 1960s were developed. A new nine was added in the early 60s. I was the first starter there in 1963 when I was 12 years old. $22 a week and lunch!"
Boston Public Library, Tichnor Brothers Collection, circa 1930-1945
Under Miles' vision, Pine Lakes grew into a premier golf resort, attracting business luminaries from around the world. In 1954 executives from Time-Life founded Sports Illustrated magazine while on a business trip to Pine Lakes. The resort became private in the 1970s, and the Ocean Forest Hotel was demolished in 1974. While features of the resort remain in historical condition, such as the back nine and the clubhouse facade, Pine Lakes International Country Club – affectionately called "The Grandaddy" – is a modern-yet-traditional club with a renovated front nine, a restored clubhouse and pool, and fine dining.
Pine Lakes Country Club is listed in the National Register as Ocean Forest Country Club:
(Pine Lakes Country Club) The Ocean Forest Country Club is significant for its association with Myrtle Beach's growth and prosperity as a coastal community resort/vacation destination, as an unusual example of Classical Revival architecture, and in the area of landscape architecture for the present 18-hole golf course dating to 1946. The country club is historically associated with an important period in Myrtle Beach history – the tenure of John T. Woodside and the Woodside brothers. In 1926 the Woodsides purchased 65,000 acres of beach front and other prime real estate from Myrtle Beach Farms Company. Although Woodside would lose his holdings in Myrtle Beach following the stock market crash of 1929, his vision for Myrtle Beach and accomplishments during his tenure had a great deal to do with its subsequent growth and prosperity. One of his greatest accomplishments was the development of the Ocean Forest Hotel and Country Club. The club and hotel were designed by an influential New York architect, Raymond Hood. Construction of the club began in 1926 and was completed in 1927. When the club was built, a 27-hole golf course was built in association with the club; it was designed by Robert White, a golf course designer and future president of the Professional Golfers Association of America. Although the original 27-hole course is no longer intact, the present course dates back to 1946, when the new owner, Fred Miles, sold 18 of the 27 holes to John McLeod for real estate development, retaining only nine. Mr. Miles then hired Robert White, a native of St. Andrews, Scotland, to redesign the nine that were left and add nine more.
Plan Your Trip: Pine Lakes Country Club
Where is Pine Lakes Country Club located?
Address: 5603 Granddaddy Drive, Myrtle Beach, SC 29577
Website: http://www.pinelakes.com/
Myrtle Beach SkyWheel 0.8 mile
Broadway at the Beach 4.1 miles
First Presbyterian of Myrtle Beach 5.1 miles
Myrtle Beach Pavilion 5.7 miles
Myrtle Beach Historic Sites
See other South Carolina National Register
Pine Lakes Country Club Map
Chesterfield Inn
Myrtle Beach SkyWheel
Myrtle Beach Pavilion
Bill Chapman says
Robert White, a Scotsman and first president of the PGA was the designer of Pine Lakes. He lived on Woodside Avenue across from the golf course. My parents bought a lot from Mr. White in 1948 and built a house on Poinsett Road. Poinsett ran from the Hotel on the oceanfront to the golf course. Robert White had died when another nine holes was added in the early 1960s.
Thank you so much for your information, we have it updated in the description above!
This history of Pine Lakes is not accurate. I know about what happened to 18 of the 27 original holes. Nine were sold off and nine just went back to nature and in the 1960s were developed. A new nine was added in the early 60s. I was the first starter there in 1963 when I was 12 years old. $22 a week and lunch!
Thank you so much for the information, we are using the National Register of Historic Places as our source for this information. Could you direct us to a formal source perhaps? We would love to look into this further!