text:
\section{Introduction}

\IEEEPARstart{F}{or} a robot to autonomously navigate an unknown and unstructured environment, end-to-end learning~\cite{nguyen2020autonomous, kahn2021badgr} and planning with terrain information~\cite{huang2021efficient, dashora2021hybrid} are two commonly applied approaches. Compared with end-to-end planning methods that directly map sensing to actions such as behavior cloning, planning with terrain information decomposes the complex problem, thus being more data-efficient and interpretable~\cite{guastella2021learning}.
The present work approaches legged robot autonomous exploration from the latter direction (see Fig.~\ref{fig:data-collection}).

A taxonomy of terrain traversability modeling methodologies includes proprioceptive approaches and exteroceptive approaches that further consist of appearance-based and geometry-based approaches~\cite{papadakis2013terrain}. Appearance-based approaches formulate terrain traversability modeling as a terrain type classification problem via image classification or semantic segmentation~\cite{gan2020bayesian}. However, such methods assume that each terrain type corresponds to the same level of traversability. Geometry-based approaches usually generate a geometric representation of the terrain from depth measurements and compute the traversability from local geometric features~\cite{shan2018bayesian}. Recent works~\cite{barnes2017find, gan2021multi} use appearance and geometry data to generate self-supervised traversability labels and train a neural network to infer the terrain traversability.

Traversability labels in exteroceptive-based methods ignore the robot's internal states while in motion across the terrain; hence, the degree of traversability is unknown, and the labels are usually binary (i.e., untraversable and traversable). In contrast, proprioceptive approaches can model terrain traversability as a continuous variable by defining a traversability cost using ground reaction score~\cite{wellhausen2019should}, vibration~\cite{bekhti2020regressed} or stability~\cite{faigl2019unsupervised} from a proprioceptive sensor modality. However, these metrics are chosen based on a user's domain knowledge and may not fully capture the traversability experienced by the robot itself.

\begin{figure}
    \centering
    \includegraphics[width=0.49\columnwidth, trim={0cm 7.5cm 0cm 7.8cm}, clip]{figs/IMG_20210529_155603.jpg}
    \includegraphics[width=0.48\columnwidth]{figs/data-collection-env.png}
    \caption{MIT Mini-Cheetah robot with a customized sensor suite and satellite map of the data collection environment for this work. The Mini-Cheetah robot is equipped with an Intel RealSense depth camera, an Inertial Measurement Unit (IMU), and an Nvidia Jetson Xavier computing device for data recording. Motion and task planning for legged robots in unstructured environments involve explicitly modeling the terrain traversability and solving a sequential decision-making problem to find the ``optimal'' behavior. For unknown environments, terrain traversability modeling using noisy measurements is itself a challenging problem due to the lack of \emph{a priori} knowledge of the surrounding terrain, the variability of terrain conditions, and the \emph{robot-specific} characteristic of traversability.}
    \label{fig:data-collection}
    \vspace{-4mm}
\end{figure}

Recently, Inverse Reinforcement Learning (IRL) for terrain traversability modeling has received increased interests, particularly for autonomous vehicles~\cite{wulfmeier2017large, zhang2018integrating, zhu2020off, jung2021incorporating}. Instead of using a predetermined metric to define traversability cost and associating it with terrain features, IRL-based methods aim to learn this cost directly from a robot's (traversing) behaviors. Doing so bypasses the effort and incompleteness of manually designing the cost function and parts of the learning pipeline. The resulting cost representations of IRL-based methods have also shown better robustness and scalability than the predetermined cost and the cost learned in supervised methods~\cite{wulfmeier2017large, osa2018algorithmic}.

Maximum Entropy Deep IRL (MEDIRL) exploits the representational capacity of neural networks for reward function approximation and has been successfully applied to traversability cost learning and trajectory forecasting for autonomous vehicles~\cite{wulfmeier2017large, zhang2018integrating, zhu2020off, deo2020trajectory, jung2021incorporating, wang2021inverse}. This work adopts the MEDIRL framework to tackle the problem of terrain traversability modeling for legged robots. A big challenge in IRL-based methods is the representation choice of the agent's state. A higher dimensional state generally leads to higher model fidelity but at the expense of an exponential increase in computational complexity. Therefore, most works incorporate higher-order robot states into the reward approximation network as extra features instead of expanding the dimension of the agent's state. However, encoding handcrafted robot kinematic features, such as estimated trajectory curvature~\cite{zhang2018integrating}, overlaid past trajectory~\cite{jung2021incorporating} and average velocities~\cite{deo2020trajectory}, has limited effect on improving the model fidelity due to the inherent approximation and inaccuracy in capturing the dynamics of the interactions with the environment.

Another challenge of applying the MEDIRL framework to legged robot applications is the suboptimality of expert demonstration. While maximum entropy methods are robust to some level of suboptimal demonstrations, they fundamentally seek a reward function that justifies the demonstrated behavior, leading to a policy that explains but rarely outperforms the demonstrator~\cite{brown2019extrapolating,arora2021survey}. Unlike a human driver, the demonstrator for legged robots typically lacks adequate feedback of locomotion performance to make the demonstration optimal or near-optimal. Further, legged robots are more likely to walk in unstructured environments where the optimal policy is difficult to find even for an expert. Thus, we propose to use robot proprioception as a source of feedback to rank each demonstrated trajectory, and introduce a \emph{trajectory ranking loss} in reward learning to learn a reward function that could extrapolate beyond suboptimal demonstrations and a policy outperforming the demonstrator.

    %
    %
    %
    %
    %

This work
offers the following major contributions:
\begin{enumerate}
    \item We propose a deep reward approximation network in the context of IRL that can learn robot inertial features from raw sensor measurements and incorporate the learned features into the reward map.
    \item We extend the MEDIRL framework into a Trajectory-ranked MEDIRL framework, and use locomotion energy as the trajectory preference label to learn an energy-aware reward map for legged robots.
    %
    %
    %
    \item We conduct extensive experiments using real data collected by a quadruped robot in a campus environment, and evaluate our method using a robot simulator.
\end{enumerate}

\section{Related Work}
\label{sec:related_work}

This work is inspired by --- and bridges the gap between --- legged robots terrain traversability modeling and IRL-based traversability cost learning methods for autonomous vehicles.

\subsection{Terrain Traversability Modeling for Legged Robots}

Compared with ground vehicles, legged robots can traverse extreme, off-road, and unstructured terrains. Terrain traversability modeling is crucial for legged robots path planning and foot placement, however, remains challenging due to the wide variation in terrain conditions and robot models.

\citet{wellhausen2019should} propose a self-supervised learning framework to predict terrain properties for a quadruped robot. They compute a \emph{ground reaction score} from recorded force-torque measurements and associate the score with corresponding image data via foothold projection. The automatically annotated images are used to train a score regression network. Similarly, \citet{faigl2019unsupervised} also use robot proprioceptive features to characterize three traversal costs, i.e., mean instantaneous power consumption, mean forward velocity and attitude stability, and attach those costs to the corresponding exteroceptive features for regression.

\citet{bednarek2019touching} propose to use machine learning methods for tactile terrain classification. Later, a neural network is proposed to take variable-length signals such as forces and torques and output terrain classes for robot localization~\cite{buchanan2021navigating}.
\citet{fan2021step} probabilistically model the traversability cost as a combination of multiple risk factors determined by the terrain geometry and robot states.
Our method relates to these works as both exteroception and proprioception are exploited to model the terrain traversability for legged robots. But instead of manually defining the traversability cost function, we learn it from demonstration.

\subsection{IRL for Traversability Cost Learning}

Early works include applying the Maximum Margin Planning (MMP) framework to traversal cost modeling using overhead terrain data~\cite{silver2010learning}. Later, the MEDIRL framework~\cite{wulfmeier2015maximum, wulfmeier2017large} is proposed to exploit the power of deep neural networks in expressing nonlinear reward functions and becomes a common paradigm for traversability cost learning.

In~\cite{wulfmeier2017large}, the reward function is modeled using a fully convolutional network with the inputs of environmental features for each state, i.e., mean height, height variance, and a binary visible indicator. Vehicle kinematics is gradually considered in the MEDIRL framework in addition to environmental context to improve the model fidelity. \citet{zhang2018integrating} integrate the discretized vehicle's past velocity and trajectory curvature into the reward network. Similarly, the reward network in~\cite{deo2020trajectory} takes the encoded robot's motion features as inputs. \citet{jung2021incorporating} propose to incorporate inertial context into the traversability map by overlaying the past trajectory on an occupied grid map. \citet{zhu2020off} encode vehicle kinematic constraints into convolution kernels to accelerate the computation. We argue that handcrafted kinematic features are limited in improving the model fidelity; instead, our reward network learns the inertial features from raw sensor measurements.

\section{Methodology}
\label{sec:methodology}

\subsection{Problem Statement}

We model the process of a legged robot walking over local terrain as an agent following a Markov Decision Process (MDP). An MDP is defined as \mbox{$\mathcal{M} = \{\mathcal{S}, \mathcal{A}, \mathcal{T}, \gamma, r\}$} consisting of a set of states $\mathcal{S}$, actions $\mathcal{A}$, transition probabilities $\mathcal{T}$, a discount factor \mbox{$\gamma \in [0, 1)$}, and a reward function \mbox{$r: \mathcal{S} \rightarrow \mathbb{R}$}. A trajectory is defined as a sequence of state-action pairs \mbox{$\tau: = \{(s_0, a_0), (s_1, a_1), ..., s_{|\tau|}\}$} followed by the agent, and a demonstration is a set of trajectories under a policy \mbox{$\mathcal{D} = \{\tau_0, \tau_1, ..., \tau_{|\mathcal{D}|}\}$}. Given an MDP, a forward RL problem seeks the optimal policy \mbox{$\pi^\ast: \mathcal{S} \to \mathcal{A}$} that maximizes the expected discounted reward (return) \mbox{$J(\pi) = \mathbb{E}[\sum_{t=0}^{\infty} \gamma^t r_t | \pi]$}. We consider the IRL problem: Given an MDP$\backslash r$ and expert demonstration $\mathcal{D}$, to recover the underlying reward function that explains the demonstration, i.e., the reward function under which the demonstrated behavior is optimal. Specifically, we estimate the traversability cost a legged robot will get when traversing a location as the negative reward of that state using Maximum Entropy IRL (MEIRL).

However, the standard MEIRL formulation requires a pre-defined absorbing goal state (by holding the value of goal state $V(s_{goal})$ fixed to 0)~\cite{wulfmeier2015maximum, zhu2020off}, learning a reward function that is goal-conditioned. Other adapted methods relax the requirement~\cite{zhang2018integrating, jung2021incorporating}, but the demonstrated goal state is implicitly learned by the reward function through value iteration and policy propagation. This reward function is naturally suited to trajectory forecasting application, but less generalizable as a traversability cost map. To learn a goal-independent traversability cost map for exploration, we follow a reformulated MEIRL framework in~\cite{deo2020trajectory} which decouples the goal-conditioned reward map into a path reward map and a goal reward map. We define our traversability cost map as the negative path reward map, and it can also be used with a user-defined goal reward map to plan optimal paths to goal states other than the inferred ones.

Our MDP formulation is as follows.
\noindent \textbf{State~space}: We discretize the local terrain around a robot into a 2D grid of a certain resolution. Each cell in the grid could be a location the robot traverses (a path state), or the goal of the robot (a goal state). Thus, our state space is defined as $\mathcal{S}=\{\mathcal{S}_p, \mathcal{S}_g\}$, where $\mathcal{S}_p$ is the set of path states and $\mathcal{S}_g$ is the set of goal states. For instance, the state space of a $m \times m$ grid includes $m^2$ path states and $m^2$ goal states, and $2m^2$ states in total.
\noindent \textbf{Action space}: The actions of the agent are then chosen to be \mbox{$\mathcal{A} = \{up, down, left, right, end\}$} to allow the robot to transit from a path state to an adjacent path state, or to terminate at a goal state in that cell. It can also be easily extended to include 8 directions with evenly spaced 45-degree intervals.
\noindent \textbf{Transition function}: We define a \emph{deterministic} transition function \mbox{$\mathcal{T}:\mathcal{S}_p \times \mathcal{A} \rightarrow \mathcal{S}$} for the agent. The goal states do not have transitions as they are terminal.
\noindent \textbf{Rewards}: Two reward functions are defined, i.e., a path reward function \mbox{$r_p: \mathcal{S}_p \rightarrow \mathbb{R}$} as the reward the agent gets when it transits to a path state, and a goal reward function \mbox{$r_g: \mathcal{S}_g \rightarrow \mathbb{R}$} as the reward received when the agent terminates at a goal state. For instance, the total reward for a trajectory \mbox{$\tau = \{(s_{p0}, a_0),(s_{p1}, end), s_{g1}\}$} is \mbox{$r_{p0} + \gamma r_{p1} + \gamma^2 r_{g1}$}, where $\gamma$ is the defined discount factor.
\noindent \textbf{Traversability costs}: The traversability cost of each cell in the grid is defined as the negative path reward of the corresponding path state \mbox{$c=-r_p: \mathcal{S}_p \rightarrow \mathbb{R}$}.
The learned goal-independent traversability cost map can be used with a user-defined goal reward map (by setting high rewards for some goal states) for planning to a specific goal state or the goal reward map inferred by the reward network for exploration.

\subsection{Maximum Entropy Deep IRL}

MEIRL is formulated to address the challenge of reward ambiguity (i.e., multiple rewards can explain the same behavior) by treating trajectories with higher rewards as exponentially more likely~\cite{ziebart2008maximum}. Under the same assumption, MEDIRL~\cite{wulfmeier2015maximum} further approximates the reward function using a deep neural network parameterized by $\theta$, i.e, \mbox{$\hat{r}_{\theta}(s) = f(\phi(s);\theta)$}, where $\phi(s)$ denote the features of state $s$. It is then mathematically formulated as a maximum-a-posteriori (MAP) estimation problem that maximizes the logarithmic joint posterior distribution of the demonstration and network parameters under the predicted reward:
\begin{align}
\label{eq:medirl}
    \mathcal{L}(\theta) &= \log \mathbf{P}(\mathcal{D}, \theta | r_{\theta})
    = \log \mathbf{P} (\mathcal{D} | r_{\theta}) + \log \mathbf{P} (\theta) \nonumber \\
    &=: \mathcal{L}_\mathcal{D} + \mathcal{L}_{\theta},
\end{align}
which can be divided into a demonstration term $\mathcal{L}_\mathcal{D}$ and a model regularizer $\mathcal{L}_{\theta}$. Applying the chain rule, the reward network can be trained by backpropagating the following MEDIRL gradient with a regularization technique:
\begin{equation}
\label{eq:svf}
\frac{\partial \mathcal{L}_{\mathcal{D}}}{\partial \theta} = \sum_{\tau \in \mathcal{D}} (D_{\tau} - D_{r}) \frac{\partial r_{\theta}}{\partial \theta},
\end{equation}
where $D_{\tau}$ are the demonstrated State Visitation Frequencies (SVF) computed from the training data, and $D_{r}$ are the expected SVF of the policy given the predicted reward~\cite{ziebart2008maximum, wulfmeier2017large}. In this work, we use the Approximate Value Iteration and Policy Propagation algorithm in~\cite{deo2020trajectory} to compute $\mathcal{D}_r$ given the path and goal reward prediction, where $V(s_{goal})$ are set as the current predicted goal rewards instead of 0 at each iteration.

\begin{figure*}
    \centering
    \includegraphics[width=1.6\columnwidth, trim={0cm 2.0cm 0.1cm 0.7cm}, clip]{figs/network_architecture.pdf}
    \caption{The proposed deep reward network architecture and training process for terrain traversability cost learning. The network has two stages: 1) feature learning and 2) feature fusion (yellow). To extract useful features from both proprioceptive and exteroceptive sensory data, the first stage consists of an inertial branch (blue) taking inputs of raw IMU signals and a ResUNet-based environmental branch (pink) taking inputs of the elevation map, elevation variance map and color map. The outputs of both branches are concatenated and fused in the second stage to approximate the reward maps. The network is trained in an end-to-end manner by backpropagating both MEDIRL gradient (green flow) and trajectory ranking loss (red flow). The average inference time for the reward network is 4.6e-04 $\sec$, and 0.018 $\sec$ for solving the RL using a GPU.}
    \label{fig:network-architecture}
\end{figure*}

\subsection{Learning Inertial Features: Network Architecture}

Existing works integrate environmental context and manually encoded robot kinematics into the reward network~\cite{zhang2018integrating, jung2021incorporating, deo2020trajectory}. In these works, the reward function can be written as \mbox{$\hat{r}_{\theta}(s) = f(\phi_e(s), \phi_k(s); \theta)$}, where $\phi_e(s)$ and $\phi_k(s)$ are the handcrafted environmental features and kinematic features of the state $s$. In this work, we propose a network architecture that can learn the robot inertial features directly from proprioceptive sensory data, and integrate them with environmental features in reward approximation: \mbox{$\hat{r}_{\theta}(s) = f(\phi_e(s), \phi_i(s; \theta); \theta)$}. Specifically, we learn a path reward $\hat{r}_{p\theta}(s)$ and a goal reward $\hat{r}_{g\theta}(s)$ using a network with shared parameters.

Inspired by the works~\cite{buchanan2021navigating, lin2021legged} where 1D convolutional modules are employed to efficiently process temporal proprioceptive sensory data of legged robots, we design an inertial branch for the reward network using similar structures. The branch consists of 2 convolutional blocks and 1 fully connected layer. Within the convolutional block, two 1D convolutional layers are followed by a 1D max pooling layer.
Dropout layers are also used as the model regularizer $\mathcal{L}_{\theta}$ in~\eqref{eq:medirl}~\cite{wulfmeier2017large}.
For simplicity, we choose fixed-length Inertial Measurement Unit (IMU) signals as the inputs to this branch.

To model traversability, the geometry and appearance of the local terrain also contain critical information. Thus, another branch of the network aims to incorporate environmental context into the reward learning. Similar to previous works~\cite{wulfmeier2017large, zhang2018integrating, zhu2020off, deo2020trajectory}, we choose the elevation map, elevation variance map and a bird's eye view color map as the environmental features. Due to the spatial nature of environmental inputs, we adopt a ResUNet~\cite{zhang2018road} architecture for image segmentation as the environmental branch based on its quantitative and qualitative performance in experiments.

The proposed reward network therefore has a two-stage architecture, as shown in Fig.~\ref{fig:network-architecture}. The first stage comprises the aforementioned two branches to process inertial (proprioceptive) data and environmental (exteroceptive) data individually. The exteroceptive and proprioceptive data are synchronized using the recorded time stamps. For the environmental maps generated at time $t$ (when the robot is at the center of the maps), the associated IMU signals are collected from $t-\Delta t$ to $t$, where $\Delta t$ is a fixed time window. The inertial features extracted from the first branch are then upsampled and concatenated with the outputs of the environmental branch. Similar to~\cite{zhang2018integrating}, we encode the $n$-dimensional 1D output from the inertial branch as $n$-channel 2D feature maps. Additional 2-channel position encoding feature maps are also concatenated to break translation-invariance of the following convolutional filters. We refer readers to~\cite{zhang2018integrating} for more details. The second stage of the reward network is a block of 2D convolutions that fuses the two types of features into the final reward maps.

\subsection{Trajectory-ranked MEDIRL}

In the MEDIRL framework, the reward network is trained using stochastic gradient descent where the gradient comes from the differences between the demonstrated SVF and expected SVF in~\eqref{eq:svf}. Therefore, the reward function obtained by minimizing the differences will be the one best explaining the demonstration, and the corresponding policy will be suboptimal if the demonstration is suboptimal. To allow reward extrapolation when additional preference information is given, we propose to add a trajectory ranking loss to the MEDIRL framework and extend it to a Trajectory-ranked MEDIRL (T-MEDIRL) framework.

The trajectory ranking loss is inspired by the preference-based IRL methods~\cite{brown2019extrapolating,brown2020better} which seek a reward function that explains the \emph{ranking} over demonstrations (intention), in contrast to the \emph{demonstrations} (behavior), thereby allowing for reward generalization and extrapolation. Specifically, given a sequence of ranked trajectories, the objective is to learn a reward function that assigns higher returns to higher-ranked trajectories. It can thus be formulated as a classification problem that predicts whether a trajectory has a higher rank than another based on the predicted return (originally the predicted reward). It is worth mentioning that the intention is not to compare the optimality of two specific trajectories, but as a type of preference learning to regulate the reward learning process. The classifier can be trained by using the trajectory-rank pairs as data-label pairs and minimizing the pairwise trajectory ranking loss~\cite{brown2020better}:
\begin{equation}
\label{eq:trajectory-ranking-loss}
    \mathcal{L}_{i, j} = -\sum\limits_{\tau_i \prec \tau_j} \log \frac{\exp\sum\limits_{s \in \tau_j}r_{\theta, j}(s)}{\exp\sum\limits_{s \in \tau_i}r_{\theta, i}(s)+\exp\sum\limits_{s \in \tau_j}r_{\theta, j}(s)},
\end{equation}
where $r_{\theta, i}$ and $r_{\theta, j}$ are the predicted reward for trajectory $i$ and $j$, respectively, and \mbox{$\tau_i \prec \tau_j$}
means that trajectory $j$ has a higher rank compared with trajectory $i$. In this work, $r_\theta$ in~\eqref{eq:trajectory-ranking-loss} is the path reward as the goal reward is only received once at the end of each trajectory, but we use $r_\theta$ for generalization purpose.

\begin{algorithm}[t]
\scriptsize
\caption{{\small T-MEDIRL: Trajectory-ranked Maximum Entropy Deep Inverse Reinforcement Learning}}
\label{al:t-meirl}
\begin{algorithmic}[1]
\State \textbf{Input:} $\mathcal{S}, \mathcal{A}, \mathcal{T}, \gamma, \mathcal{D}, \alpha$
\State \textbf{Output:}
network parameters $\theta$

\Function{\textsc{MEDIRL-Grad}}{$r_{\theta}, \tau, \mathcal{S}, \mathcal{A}, \mathcal{T}, \gamma$}
    \State $\pi \gets \textsc{Approx.-Value-Iteration}(r_{\theta}, \mathcal{S}, \mathcal{A}, \mathcal{T}, \gamma)$
    \State $D_{\tau} \gets \textsc{Demo.-State-Visitations}(\tau, \mathcal{S})$
    \State $D_{r} \gets \textsc{Policy-Propagation}(\pi, \mathcal{S}, \mathcal{A}, \mathcal{T})$
    \State $\frac{\partial \mathcal{L}}{\partial r_{\theta}} \gets D_{\tau} - D_{r}$
    \State \Return $\frac{\partial \mathcal{L}}{\partial r_{\theta}}$
\EndFunction

\State
\State Randomly initialize $\theta^1$
\For{$n$ = 1 to $N$}
    \State Randomly choose $\tau_i$ and $\tau_j$ from $\mathcal{D}$
    \State $r_{\theta^n, i}(s) \gets f(\phi_i(s); \theta^n)$ for $\forall s \in \mathcal{S}$
    \State $r_{\theta^n, j}(s) \gets f(\phi_j(s);\theta^n)$ for $\forall s \in \mathcal{S}$
    \State \Comment{Forward reward network}
    \State $\frac{\partial \mathcal{L}_i}{\partial r_{\theta^n, i}} \gets \textsc{MEDIRL-Grad}(r_{\theta^n, i}, \tau_i, \mathcal{S}, \mathcal{A}, \mathcal{T}, \gamma)$
    \State $\frac{\partial \mathcal{L}_j}{\partial r_{\theta^n, j}} \gets \textsc{MEDIRL-Grad}(r_{\theta^n, j}, \tau_j, \mathcal{S}, \mathcal{A}, \mathcal{T}, \gamma)$
    \State \Comment{Compute MEDIRL gradient}
    %
    %
    \State $\mathcal{L}_{i, j} \gets -\sum\limits_{\tau_i \prec \tau_j} \log \frac{\exp\sum\limits_{s \in \tau_j}r_{\theta^n, j}(s)}{\exp\sum\limits_{s \in \tau_i}r_{\theta^n, i}(s)+\exp\sum\limits_{s \in \tau_j}r_{\theta^n, j}(s)}$
    \State \Comment{Compute trajectory ranking loss}
    \State Compute $\nabla_{\theta^n}\mathcal{L}_i, \nabla_{\theta^n}\mathcal{L}_j, \nabla_{\theta^n}\mathcal{L}_{i,j}$ via chain rule
    \State $\theta^{n+1} \gets \theta^n + \alpha \nabla_{\theta^n}\mathcal{L}_i + \alpha \nabla_{\theta^n}\mathcal{L}_j - \alpha \nabla_{\theta^n}\mathcal{L}_{i,j}$
    \State \Comment{Update network parameters}
\EndFor
\end{algorithmic}
\end{algorithm}

\begin{figure*}[t]
    \centering
    \includegraphics[width=0.25\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/sidewalk_elevation.pdf}
    \includegraphics[width=0.25\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/sidewalk_variance.pdf}
    \includegraphics[width=0.25\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/sidewalk_rgb.pdf}
    \includegraphics[width=0.41\columnwidth, trim={2cm 0cm 0cm 0cm}, clip]{figs/input_data/sidewalk_gopro1.png}
    \includegraphics[width=0.36\columnwidth, trim={2cm 8.47cm 2cm 8cm}, clip]{figs/input_data/sidewalk_imu.pdf} \\
    \includegraphics[width=0.25\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/offroad_elevation.pdf}
    \includegraphics[width=0.25\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/offroad_variance.pdf}
    \includegraphics[width=0.25\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/offroad_rgb.pdf}
    \includegraphics[width=0.41\columnwidth, trim={2cm 0cm 0cm 0cm}, clip]{figs/input_data/offroad_gopro2.png}
    \includegraphics[width=0.36\columnwidth, trim={2cm 8.47cm 2cm 8cm}, clip]{figs/input_data/offroad_imu.pdf} \\
    \begin{subfigure}{0.25\columnwidth}
        \includegraphics[width=1.0\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/forest_elevation.pdf}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.25\columnwidth}
        \includegraphics[width=1.0\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/forest_variance.pdf}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.25\columnwidth}
        \includegraphics[width=1.0\columnwidth, trim={6cm 9.6cm 6cm 9.5cm}, clip]{figs/input_data/forest_rgb.pdf}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.41\columnwidth}
        \includegraphics[width=1.0\columnwidth, trim={2cm 0cm 0cm 0cm}, clip]{figs/input_data/forest_gopro2.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.36\columnwidth}
        \includegraphics[width=1.0\columnwidth, trim={2cm 8.47cm 2cm 8cm}, clip]{figs/input_data/forest_imu.pdf}
        \caption{}
    \end{subfigure}
    \caption{Visualization of the generated data (network inputs) in different environments, for instance, sidewalk, trail and forest. (a) Elevation map. (b) Elevation variance map. (c) Color map. The demonstrated past (red) and future (cyan) trajectories are overlaid for visualization purpose and are not part of the feature map. (d) Third-person view of the current state (not an input). (e) The IMU signals collected within the past 0.5 $\sec$.}
    \label{fig:network-input}
\end{figure*}

We use the trajectory ranking loss to regulate the reward learning in the proposed T-MEDIRL algorithm (Algorithm~\ref{al:t-meirl}). Given the MDP$\backslash r$ and demonstration with ranked trajectories, the T-MEDIRL framework learns to approximate the reward function $r_{\theta}$ by backpropagating the MEDIRL gradient and the trajectory ranking loss together, as shown in Fig.~\ref{fig:network-architecture}. At each iteration, we randomly choose two (batches of) trajectories and obtain the corresponding predicted returns (reward functions) by forwarding the reward network. The per-trajectory MEDIRL gradient can be computed using the MEDIRL-Grad function, while the trajectory ranking loss is computed using~\eqref{eq:trajectory-ranking-loss} on a cross-trajectory basis. Finally, the network parameters are updated according to the learning rate $\alpha$ and gradients computed using the chain rule.

\subsection{Locomotion Energy Ranked Reward Extrapolation}

Energy cost is one of the major factors that can quantify the locomotion performance of legged robots, and an important consideration in path/motion planning. As a source of robot-terrain interaction, it also contains critical information about the traversed terrain. \citet{faigl2019unsupervised} use mean power consumption as an indicator of the terrain traversability, and \citet{walas2016terrain} optimize the gait parameters for a humanoid robot based on the energy expenditure and locomotion stability. However, the energy cost is usually unknown or unpredictable to the human demonstrator when collecting demonstrations of legged robot walking, leading to a severe suboptimality issue, i.e., the demonstrated trajectory might be feasible but energy inefficient.

To extrapolate beyond the suboptimal demonstrations for legged robots and learn a reward function that can lead to more energy-efficient path planning, we propose to use the Average Energy Consumption (AEC) of each trajectory for trajectory ranking in the T-MEDIRL framework. The AEC is computed by dividing the overall energy consumption of each trajectory by the trajectory length, where the overall energy consumption can be obtained using the recorded robot joint states as in~\cite{walas2016terrain}:
\begin{equation}
    e_{\tau} = \sum_{i=1}^n \langle \lvert \boldsymbol{u}_i \rvert, \lvert \Delta \boldsymbol{q}_i\rvert \rangle,
\label{eq:overall-energy}
\end{equation}
where $\lvert \cdot \rvert$ takes the element-wise absolute value, $\langle \cdot, \cdot \rangle$ denotes the dot product, $\boldsymbol{u} \in \mathbb{R}^{m}$ and $\Delta \boldsymbol{q} \in \mathbb{R}^{m}$ are the joint torque and joint displacement between two time stamps for $m$ joints, respectively, and $n$ is the total length of the signals collected along the entire trajectory $\tau$.
In this work, if $\tau_j$ has a lower AEC value than $\tau_i$, then $\tau_i \prec \tau_j$.

\section{Experiments}
\label{sec:experiments}

\begin{figure*}[t]
    \centering
    %
    %
    %
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-rgb.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-rw-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-rw-goal.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-svf-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-rw-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-rw-goal-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-svf-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-rw-path-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-rw-goal-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-4-8-svf-path-baseline.png}
    \includegraphics[width=0.01\columnwidth, height=0.14\columnwidth]{figs/revision/colorbar.png}
    %
    %
    %
    \\

    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-rgb.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-rw-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-rw-goal.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-svf-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-rw-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-rw-goal-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-svf-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-rw-path-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-rw-goal-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-5-7-svf-baseline.png}
    \includegraphics[width=0.01\columnwidth, height=0.14\columnwidth]{figs/revision/colorbar.png}
\\
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-rgb.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-rw-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-rw-goal.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-svf-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-rw-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-rw-goal-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-svf-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-rw-path-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-rw-goal-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/1-2-6-svf-path-baseline.png}
    \includegraphics[width=0.01\columnwidth, height=0.14\columnwidth]{figs/revision/colorbar.png}
    \\
    %
    %
    %
    %
    %
    %
    %

    %
    %
    %
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-rgb.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-rw-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-rw-goal.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-svf-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-rw-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-rw-goal-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-svf-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-rw-path-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-rw-goal-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-1-8-svf-path-baseline.png}
    \includegraphics[width=0.01\columnwidth, height=0.14\columnwidth]{figs/revision/colorbar.png}
    %
    \\
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-rgb.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-rw-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-rw-goal.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-svf-path.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-rw-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-rw-goal-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-svf-path-wo.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-rw-path-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-rw-goal-baseline.png}
    \includegraphics[width=0.15\columnwidth]{figs/revision/2-6-6-svf-path-baseline.png}
    \includegraphics[width=0.01\columnwidth, height=0.14\columnwidth]{figs/revision/colorbar.png}
\\
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-rgb.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-rw-path.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-rw-goal.png}
        \caption{}
    \end{subfigure}
       \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-svf-path.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-rw-path-wo.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-rw-goal-wo.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-svf-path-wo.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-rw-path-baseline.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-rw-goal-baseline.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.15\columnwidth}
        \includegraphics[width=1.0\columnwidth]{figs/revision/3-1-4-svf-path-baseline.png}
        \caption{}
    \end{subfigure}
    \begin{subfigure}{0.01\columnwidth}
    \vspace{-5mm}
    \includegraphics[width=1.0\columnwidth, height=14\columnwidth]{figs/revision/colorbar.png}
    \end{subfigure}
    %
    %
    %
    %
    %
    %
    %
    %
\\
    \caption{Qualitative results on traversability cost learning and future trajectory prediction in Experiment 1. (a) are the input color maps overlaid with the demonstrated past (red) and future (cyan) trajectories. (b)-(d) are the qualitative results of our method with inertial feature learning: from (b) to (d) are respectively the inferred path reward maps, goal reward maps, and the SVF visualization of the path states representing the predicted future trajectory distributions. (e)-(g) are the corresponding results of our method without inertial feature learning, and (h)-(j) are of the baseline method with handcrafted kinematic features~\cite{zhang2018integrating}. The last row shows a failure case and the color map is shown in the right.
    }
    \label{fig:exp1-qualitative-results}
\end{figure*}

\subsection{Legged Robot Walking Dataset}

We collect data using a quadruped robot with a customized sensor suite operating in a campus environment including diverse scenes. The robot platform, an MIT Mini-Cheetah, and the data collecting environment are shown in Fig.~\ref{fig:data-collection}.
We manually teleoperate the robot to walk around by sending the commands of longitudinal and lateral velocity and yaw rate.

We generate a dataset of over 1000 trajectories and their corresponding environmental features, IMU signals and AEC values using both exteroceptive and proprioceptive sensor measurements from different raw data sequences. For the environmental features, we generate the elevation map, elevation variance map and color map of a region of \mbox{8 m $\times$ 8 m} with 0.1 m resolution. Specifically, we use ORB-SLAM2~\cite{mur2017orb} to estimate the robot poses, and build a local robot-centric probabilistic elevation map using point clouds captured by the depth camera. Elevation mapping~\cite{fankhauser2018probabilistic} models the height measurements by a Gaussian distribution and fuses multi-frame measurements using a kalman filter. We generate the elevation and elevation variance map using the mean and variance of each grid's Gaussian distribution. The color map is generated by averaging the RGB values of all points falling into each grid.

The demonstrated trajectory is generated by projecting the robot poses onto the local grid map, including a past trajectory and a future trajectory. The past trajectory is a sequence of grid indices traversed by the robot during a period of time before map building, whereas the future trajectory is during the same period of time after map building. The IMU signals used for the inertial feature learning are collected within the past 0.5 $\sec$, and the AEC value is computed using the joint state measurements of 12 joints (3 joints per leg) along the past trajectory.
Examples of the generated data are visualized in Fig.~\ref{fig:network-input}. We use a 70/30 split to divide our data into training and test sets for the following experiments.

\subsection{Inertial Feature Learning Evaluation}

\begin{figure*}
    \centering
    \begin{minipage}{0.5\textwidth}
    \centering
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-4-2-rw-path.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-4-2-rw-goal.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-4-2-svf.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-4-2-rw-path-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-4-2-rw-goal-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-4-2-svf-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-4-2-rgb.png} \\
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-12-3-rw-path.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-12-3-rw-goal.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-12-3-svf.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-12-3-rw-path-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-12-3-rw-goal-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-12-3-svf-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-12-3-rgb.png}\\
    \includegraphics[width=0.14\columnwidth]{figs/revision/3-5-5-rw-path.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/3-5-5-rw-goal.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/3-5-5-svf.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/3-5-5-rw-path-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/3-5-5-rw-goal-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/3-5-5-svf-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/3-5-5-rgb.png}
    \\
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-6-8-rw-path.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-6-8-rw-goal.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-6-8-svf.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-6-8-rw-path-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-6-8-rw-goal-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-6-8-svf-t.png} \hspace{-0.20cm}
    \includegraphics[width=0.14\columnwidth]{figs/revision/2-6-8-rgb.png}
    \end{minipage}
    \begin{minipage}{0.48\textwidth}
    \centering
    \includegraphics[width=0.75\columnwidth]{figs/exp2/simulator-robot.png} \\
    \vspace{0.07cm}
    \includegraphics[width=0.24\columnwidth, trim={0cm 0cm 0cm 0.3cm}, clip]{figs/exp2/simulator1.png}
    \includegraphics[width=0.24\columnwidth]{figs/exp2/simulator2.png}
    %
    \includegraphics[width=0.24\columnwidth, trim={0cm 0cm 1cm 0cm}, clip]{figs/exp2/simulator3.png}
    \end{minipage}
    \caption{Left block: Qualitative results on traversability cost learning and future trajectory prediction in Experiment 2. From left to right, the first three columns are the inferred path reward maps, goal reward maps and the predicted future trajectory distributions obtained using the MEDIRL algorithm. The next three columns are those obtained using our proposed T-MEDIRL algorithm with energy-based trajectory ranking. The last column is the corresponding input color maps overlaid with demonstrated past (red) and future (cyan) trajectories. The magenta region in the color maps is due to a hardware issue of the camera. Right block: Mini-Cheetah Simulator. The bottom row shows the robot is following the optimal future trajectory.}
    \label{fig:exp2-qualitative-results}
\end{figure*}

This experiment aims to evaluate the performance of the proposed reward network in reward learning, i.e., how well the learned reward function explains the expert demonstration. Therefore, we employ two commonly used metrics in IRL problems for quantitative evaluation: the Negative Log-Likelihood (NLL) of the expert demonstration under the learned policy, and the Hausdorff Distance (HD)~\cite{kitani2012activity} between demonstrations and trajectories sampled with the learned policy. Here, the learned policy is obtained via value iteration given the learned reward function. For a fair comparison, all networks in this experiment are trained using the MEDIRL framework under the same MDP formulation.

We first conduct an ablation study on the environmental branch to ensure the performance of terrain feature extraction. We compare three different architectures used in literatures: ResNet~\cite{deo2020trajectory}, UNet~\cite{ronneberger2015u}, and ResUNet~\cite{zhang2018road}. The quantitative results of reward learning using only environmental features are given in the upper part of Table~\ref{table1}. In the rest of the experiment, we use the best performing ResUNet as our environmental branch. Next, we compare the performance of our method with the inertial feature learning and of a baseline approach using handcrafted kinematic features~\cite{zhang2018integrating}. For additional comparison with the networks without inertial feature learning, we list the quantitative results in the lower part of Table~\ref{table1}. The superiority of our method for legged robots can be seen in Table~\ref{table1}, where our network with the inertial branch has the best prediction results using both metrics.

The qualitative results are given in Fig.~\ref{fig:exp1-qualitative-results}. The first column are the input color maps for each scene, overlaid with past trajectories (red) and ground truth future trajectories (cyan). The results of each method are then shown side by side. The color of each pixel in the path reward map represents the reward of traversing that state (the negative path reward is the traversability cost of that state), whereas the goal reward map indicates the reward of terminating at that state. The corresponding path SVF visualization indicates the distributions of the predicted future trajectories under the inferred rewards.
The qualitative results also show our method with inertial feature learning generally has the best performance in reward learning and trajectory prediction compared with the baseline and our method without inertial features, although fails in certain scenarios (last row of Fig.~\ref{fig:exp1-qualitative-results}).

\begin{table}[t]
\caption{Traversability Cost Learning/Future Trajectory Prediction Performance Quantitative Comparison in Experiment 1. For environmental branch ablation study, only environmental features are used as the network input. By learning and incorporating robot inertial features in the proposed reward network, the inferred reward function (traversability cost) becomes robot-state-dependent, thus achieving better performance than purely environment-dependent reward.}
\label{table1}
\centering
{\scriptsize
\begin{tabular}{l|l|c|c}
\toprule
& Method & NLL $\downarrow$ & HD $\downarrow$ \\
\hline
\multirow{3}{*}{Env. Branch Ablation} & \rd{ResNet~\cite{deo2020trajectory}} & \rd{0.9490} & \rd{13.4957} \\
& \rd{UNet~\cite{ronneberger2015u}} & \rd{0.9016} & \rd{12.0014} \\
& \rd{ResUNet~\cite{zhang2018road}} & \rd{\bf 0.8419} & \rd{\bf 9.8219} \\
\hline
\multirow{2}{*}{\rd{Inertial Feat. Evaluation}} & \rd{Dilated CNN w/ kinematics~\cite{zhang2018integrating}} & \rd{0.8821} & \rd{10.1953} \\
& \rd{ResUNet w/ inertial features} & \rd{\bf 0.7915} & \rd{\bf 8.4012} \\
\bottomrule
\end{tabular}
}
\end{table}

\subsection{Energy-based Reward Extrapolating Evaluation}

We evaluate the effect of locomotion energy ranked reward extrapolation in this experiment. We use the AEC value to rank the demonstrated trajectories in the training set, and train the proposed reward network using the proposed T-MEDIRL framework. For comparison, we choose the same reward network trained using the MEDIRL framework as the baseline. The qualitative results are given in the left block of Fig.~\ref{fig:exp2-qualitative-results}. We notice that the path reward maps learned by T-MEDIRL are less smooth and more likely to induce a multimodal distribution over path and goal states. Given the difference in performance, the less smooth reward map can be associated with their only difference, namely the trajectory ranking loss. To further evaluate the performance, we quantitatively evaluate the predicted trajectories by both methods.

As the purpose of introducing energy-based trajectory ranking loss is to learn a reward/policy better than demonstration, quantitative metrics other than NLL and HD are needed in this experiment. In classical IRL problems, the quality of the predicted reward can be evaluated by comparing with the ground truth reward. Due to the lack of ground truth reward in real-world problems, we introduce the following two metrics for evaluation.

The first one is classification accuracy.
The reward learning in T-MEDIRL framework can be formulated as a classification problem, where the trajectory with higher predicted return is classified as higher ranked given the predicted reward. Thus, we compare the classification accuracy on the test set for both methods. Specifically, for any pair of test trajectories, we compute their returns from the predicted reward maps, if the one having higher return corresponds to a smaller AEC value, it is considered to be correct. The accuracy is computed by dividing the total number of correct classifications by the total number of comparisons.

Secondly, to evaluate whether the reward map learned by our T-MEDIRL algorithm can lead to a more energy-efficient policy, we generate optimal future trajectories for both methods, and compare the AEC values of the planned trajectories. To this end, we use a simulation environment for Mini-Cheetah that provides the robot model, a state estimator and a controller. We implement a simple PID-based waypoint follower to send motion commands to the controller. For each planned trajectory of the test data, we load the corresponding elevation map to the simulator to simulate the terrain conditions, and make the robot follow the planned trajectory under the trot mode (right block of Fig.~\ref{fig:exp2-qualitative-results}). The yaw commands are automatically generated to follow the robot's moving direction.

\begin{table}[t]
\caption{Traversability Cost Learning/Future Trajectory Prediction Performance Quantitative Comparison in Experiment 2. We log the simulated robot joint states during the trajectory following for all planned trajectories in the test set, using a sampling time of $2~\mathrm{ms}$. For each method, the AEC computed for all planned trajectories is reported. Using a $24 \mathrm{V}$, $6 \mathrm{Ah}$ battery, the difference shown below in AEC corresponds to about seven minutes extra operation. }
\label{table2}
\centering
{\scriptsize
\begin{tabular}{l|c|c}
\toprule
Method & ours w/ MEDIRL~\cite{wulfmeier2017large} & ours w/ T-MEDIRL \\
\hline
NLL $\downarrow$ & \rd{\bf 0.7734} & \rd{0.8132} \\
\hline
HD $\downarrow$ & \rd{\bf 8.1460} & \rd{9.9126} \\
\hline
Accuracy $\uparrow$ & \rd{40.01\%} & \bf \rd{64.12\%} \\
\hline
AEC $\downarrow$ & \rd{$4.634\times 10^{-2} \mathrm{J}$} & \rd{\bf $\bf 4.179\times 10^{-2} \mathrm{J}$} \\
\bottomrule
\end{tabular}
}
\end{table}

The quantitative results are given in Table~\ref{table2}, where we report the classification accuracy and AEC besides NLL and HD. As expected, MEDIRL has better prediction performance (shown by its lower NLL and HD) compared with T-MEDIRL, indicating that MEDIRL learns a reward function that better explains the demonstration, and a policy closer to the demonstrated behavior. In contrast, the reward maps learned using T-MEDIRL lead to a much higher classification accuracy, which shows the effect of the energy-based trajectory ranking loss in reward regulation. Last but not least, the lower simulated AEC of the optimal trajectories planned by T-MEDIRL directly shows that the reward maps learned by T-MEDIRL are extrapolated beyond the suboptimal demonstrations and lead to a more energy-efficient policy than demonstration.

\subsection{Discussion}
The limitation of this work is that the simplified discrete states and actions are unable to fully capture the agile motion capability of a legged robot. However, adding more dimensions to the state, such as orientation, velocities, and higher-order derivatives, comes with an exponential increase in computation complexity. To model complex legged robot locomotion as a tractable IRL problem, designing a hierarchical action and state space, e.g.~\cite{kolter2007hierarchical}, is an interesting future research direction.

The following framework is a feasible research direction to deploy this work for online exploration missions. The IIG algorithm in~\cite{ghaffari2019sampling} does not require any goals to compute exploration policies. Combined with a Model Predictive Control (MPC)~\cite{teng2021toward,teng2022error}, it can provide an integrated kinodynamic planner that takes the robot stability, control constraints, and the value of information from sensory data into account. In particular, the developed work here can serve as the data-driven information function in the IIG algorithm to compute a partial ordering of future states.

\section{Conclusion}
\label{sec:conclusion}

This work tackles the problem of terrain traversability modeling for legged robots using deep IRL. Two issues have been considered in this work. First, legged robots with high mobility tend to have more sudden motions. Existing DIRL works using purely environmental features or handcrafted kinematic features might fail to capture these movements. The proposed deep reward network can effectively learn robot inertial features from proprioceptive sensory data and integrate the learned features into the reward map.
Secondly, we have proposed a T-MEDIRL algorithm and trajectory ranking strategy using energy to deal with the suboptimality of legged robot demonstrations. By introducing a preference label for each demonstrated trajectory, the inferred reward map can lead to a more energy-efficient policy.

{\scriptsize
\balance
\bibliographystyle{bib/IEEEtranN}

redpajama_set_name: RedPajamaArXiv


text:
TEES & PINS
HEXAGON EDITIONS
SEPARATION IS A MYTH
Dispelling with the myth of separation; the idea that we're each a separate self, separate from each other, the natural world and the universe; the idea that we're isolated selves; is all merely a myth. We are not separate; and from the centre of our own self we are all connected to the all and everything.
"My print has arrived, and it's soooooooo beautiful, thank you!"
Becky (Fremantle, Australia)
The hexagon is widely considered to be a sacred shape often associated with the geometry of the natural world and is considered to be a symbol of harmony and balance. It is seen in the construction of the bee's honeycomb hive, and in the make-up of their eyes. The shape can be seen in crystals, snowflakes and in natural rock formations such as the Giants Causeway, in Ireland. It is seen on Saturn's north pole as a cloud formation and, back-down-to-Earth, the Benzene Ring (six carbon atoms joined to form a hexagon) is the basic component of many organic compounds. The number six is also considered to be a sacred number (and being the number of sides to the hexagon).
Printed on Somerset Velvet cotton paper (330gsm)
Numbered and signed with a blind-embossed monogram stamp for authentication
540 x 540mm / Limited edition 96
750 x 750mm / Limited edition 9
All prices are for unframed prints
+ postage and packaging
UK £10
Europe £15
Worldwide £20
If you order more than one poster in a single order, you'll only pay the one price as listed for P&P
unless the order exceeds capacity for one piece of packaging
Please note the design seen here on-screen is the digital version of the final artwork and due to the differences with on-screen digital colour (along with screen contrast, brightness and so on) and printed colour, the colours seen on this page are a representation only of what will be the final printed colours (which are gorgeous!) and should not be considered to be accurate to the final artworks. All frames shown are for visual purposes only – posters will be supplied unframed.

redpajama_set_name: RedPajamaCommonCrawl


Schistosoma mansoni Diagnosis Pre- and Post-Repeated-Praziquantel Treatment.
Kato-Katz thick smears and urine point-of care circulating cathodic antigen tests (POC-CCA) for the diagnosis of Schistosoma mansoni infections in primary school children in Uganda. Whilst Kato-Katz thick smears enable the concurrent diagnosis and quantification of soil transmitted helminthiases, POC-CCA can only detect (intestinal) schistosomiasis infections. However, POC-CCA are rapid, more culturally acceptable and have a higher sensitivity as infection intensities decrease with on-going mass drug administration. See Lamberton et al.
Visceral Leishmaniasis Relapse in HIV Patients—A Role for Myeloid-Derived Suppressor Cells?


Manufacturer Info: 47RE Stage 1 Package, 1996 - Early 98 Dodge 2wd Part # 3099122188 1996 - Early 98 Dodge Ram 2WD w/ 12V 5.9L Cummins, Stage 1 Auto Trans Package. Includes: - ATS Built 47-RE Auto Trans - ATS 5 Star Viskus Clutch Drive Torque Converter - ATS Extra Deep Transmission Pan In 1996, Dodge went from the hydraulic 47RH to the partially electronic 47RE. The 47RE features steel planets as well as an electronic governor pressure solenoid and transducer. In re-designing the 47RE, ATS addressed inherent weaknesses to produce a strong, reliable transmission for towing, daily driving or racing. The result is a transmission featuring: Five Star billet torque converter Performance transmission pump Modified high performance clutch packs/bands and steels Valve body built around reliability and increased transmission life The modification process began with the torque converter. Instead of a stamped converter with a single lock-up clutch, the ATS Stage 1 47RE comes with our patented Five-Star torque converter. Utilizing a billet 12 bolt cover, five square tab lock-up clutches, a billet stator and other modifications, this converter is designed for heavy loads, years of reliability and brute power. The goal is to efficiently transfer your 12 or 24V Cummins' power through the converter into the transmission. In order to handle the increased efficiency of the converter, we addressed the 47RE from bellhousing to tailshaft. The transmission's pump is modified to progressive raise line pressure, above the factory settings, to provide clamping force to the clutch packs. High-performance clutch packs are then added. Pressure plates are precisely machined to add 1 more forward and 1 more direct clutch. High performance bands, billet band strut, billet accumulator piston, and heavy duty overdrive bearings strengthen the clutch pack/band assembly. Next is the valve body. Dodge 47RE transmissions can be plagued by sticking throttle valve issues. The valve controls shift timing and fluid flow to the clutch packs. When the valve sticks, there can be a no "up-shift" condition. In time, this can wear on the clutch packs and internals to hasten the transmission's life. ATS machines the bore to accept a Teflon sleeve, this ensures precise movement and control of the valve. In addition, the valve body is modified to circulate fluid in park, reverse and neutral, capable of locking the torque converter in any forward gear, and raising line pressure vs. stock. The Stage 1 (as well as all ATS Transmission Packages) comes standard with the ATS Deep Pan. Featuring a finned design to dissipate heat, and adding five quarts of fluid vs. stock, the pan helps keep the transmission running cool. The ATS 47RE Stage 1 is designed for trucks with stock to mild power increases, light towing and daily driving. For moderate to extreme power applications, heavy towing or competition use, call ATS to discuss which transmission package would fit your application. All ATS transmission packages come backed with a 5 year / 500,000 mile warranty.


Titanium and Brass Bridge Pins for acoustic guitars! It looks like titanium and brass upgrades are not just for locking tremolos (or breakfast) any more. Take your acoustic guitar to the next level with the ultimate in acoustic upgrades.
Adam Reiver and Phil Collen (Def Leppard) undergoing the titanium bridge pin beta test. Needless to say, Phil is a convert!


Kate is delicate bridal headpiece handcrafted with shimmering crystals in a beautiful twist. This adjustable headband is the perfect mix of simple and elegant. The most comfortable headband for your special day. No slip, no headache headband.


I was wondering if anyone ever takes their Brompton on the trails, or gravel path? There are some flatter, fairly smooth dirt and gravel trails around me that would be nice to zip around on.
I've done it a couple times, but not sure if I'd recommend it. The combination of thin tires with the smaller wheel diameter make it more technically challenging. Be prepared to have to put your foot down quickly to prevent toppling over. I've ridden clipped in on gravel on my full size road bike, and that can be challenging at times, but I felt the brompton to be a little tougher. I think the brompton could do well on dirt roads as long as it's not muddy, but deep gravel, you may sink into the road in certain spots.
Certainly doable, except sand. Thin tires will just get stuck in sand. I don't ride in snow but I guess it is a similar story. Go slow down the hills to avoid a whole lot of problems. Make sure to keep your break pads clean as they'll wear down the small rims. Small wheels contact big obstacles at a steep angle and have higher risk of pinch flats or bottoming out. Rim brakes don't perform as well as disc brakes and are less effective with a smaller diameter rim. Pedals don't have much gound clearance so you cannot lean to turn tight corners, I couldn't learn to do it properly on flat ground even..
Other than having to slow down a bit, it is just like any other bike with similar tire width.
Oh, memories. Google sent me up a fire road somewhere south of Berlin, multiple times. Nothing like hitting sand a mile or two in and pushing the bike for the next mile because it's still faster than backtracking and going around.
I ride my Brompton on flat gravel tow paths that run alongside the canals here in the UK. That's definitely doable and fun when it's dry. Not so much when it's muddy though!
I rode mine over gravel I couldn't avoid from street construction and it somehow popped my chain off in the middle of an intersection. It was terrible pulling over with a front basket full of groceries and youtubing on the sidewalk how to put the chain back on in the summer sun.
This happened to me a few times. I try to avoid pedaling hard on gravel.
As for the chain: dry lube + carrying a small rag means that you don't have that dirty hands when you need to put your chain back on.


Anfernee Chansamooth can help you build your authentic brand, grow your audience, make more money, and amplify your influence online.
Should you attend a Tony Robbins event?
I had just graduated NLP practitioner training.
I had no job, no clients, and bills to pay.
My trainer at the time suggested that charging $1,000 per month was reasonable for a quality coach.
As a newbie getting started I didn't feel that I was worth that amount, not even close.
Then i discovered that that one particular coach had blogged about how he built a thriving business with putting out a "pay what you want" offer when he started.
So I gave it a go.
One month. 4 sessions. Pay what you can. 10 spots only.
I learnt something powerful about value and pricing during that experience.
Firstly, the market will dictate what your product or service is worth.
I got 8 paying clients, and on average they were paying $200 for 4 x 1hr career coaching sessions.
Some paid more, some paid less.
– Help the client map out a clear roadmap of how we'd work together beyond the first month, and tie it to a specific outcome they're wanting to achieve.
– Mindset and removing limiting beliefs is helpful, but often the best results will come from helping your clients implement faster.
I'm now getting paid more than $1k per month to help clients implement marketing systems that frees up their time and allows them to focus on their core strengths.
It's done for you, not DIY.
This is the first step. The next step will be to build scalability into the business and I've got a plan for that.
Have I got the right clients and how can I better help them to execute faster?
You can't fake adding value.
Remember that one kid in the playground who acted like they were having heaps of fun, making a lot of noise, and constantly screaming "look at me, look at me"?
But inside what they really wanted was someone to give them a hug and tell them that they were valued and valuable just for being who they are.
I used to be that kid.
So it's interesting when I observe adults playing out that role online, like in FB groups.
None of these behaviours are adding value.
I'm not sure if you've experienced it or not, but people can FEEL when someone is looking after themselves first, even if their post is disguised as serving others.
Who are you really serving?


Whether decide to work as an individual or form a team, collaboration is always encouraged throughout the hackathon!
Also, join in on one of our monthly Office Hour Calls the 2nd Wednesday of each month for the duration of the Hackathon. Call links will be published a few days before the call. The December call is 8-9 am Pacific Time (PST) December 12th 2018. You can join via this link.
Finally, the hackathon Discussion Page is another forum to ask technical and general questions about the hackathon.


text:
Tel. (852) 2523 8979
Fax (852) 2845 7352
resource@counselling.org.hk
[Enquiry] Mon - Fri: 9:15 - 19:00
ReSourceTCC.LTD
About ReSource
News/Resourceful Information
Our Counsellors
Home » History and Mission
In 1971, Reverend John Foster, then the Dean of St. John's Cathedral, was vastly concerned about the growing number of troubled marriages he saw at the time. At the same time, he met Mrs. Patricia Nicholl, who has worked with the National Marriage Guidance Council for ten years in England prior her arrival in Hong Kong. After much discussion and planning, they formed the Marriage Guidance Council (M.G..C.), in 1973. Thus, what would eventually become ReSource The Counselling Centre (ReSource) was born.
In the beginning, ReSource did not even have an office of its own, except for a room used for counselling at the Matilda & War Memorial Hospital. Fortunately, M.G.C. was able to gain a room as a use for an office in the Council of Social Services in the Duke of Windsor House in Wanchai, and later the first counselling room was in the old Lane Crawford Building. After three moves, the Service finally set up rooms in 1983 at 1303, Ruttonjee House, 11 Duddell Street, Central, Hong Kong. By then, the counselling services being offered had developed beyond marital relations, and so the name was changed to the "Marriage & Personal Counselling Service".
In 1995, MPCS further moved within the same building to Suite 1602, then to a larger premises at Suite 501 in 1999. Finally, in October 1997, MPCS changed its name to "ReSource The Counselling Centre". The new name reflects how MPCS has evolved in a changing society. In view of the ever-changing needs, ReSource The Counselling Centre Ltd was incorporated in June 2002
In June 2008, ReSource moved into a larger place in 23/F, Li Dong Building to serve the increasing numbers of clients whom come from a diversity of backgrounds.
In contrast, the staff only consisted of a part-time volunteer secretary and a few counsellors in 1972, whereas today, ReSource has a full-time Office Manager, Administrative Secretary, a Part-time Receptionist and another twelve counsellors. Since August 1985, a full-time Cantonese-speaking counsellor has been employed to work with the Chinese Community and is also able to offer a walk-in crisis counselling service. Now, we have counsellors who come from all over the world, speaking numerous different languages, and are able to serve clients from any cultural backgrounds.
The first MGC counsellors, selected to work with Mrs. Patricia Nicholl, had to be trained in Hong Kong as there was a shortage of qualified and experienced individuals. From the autumn of 1987, MPCS required that those applicants who wished to counsel for the organisation must have obtained related academic qualifications or have previous counselling experience, and they were screened in a three-step process before being able to work at ReSource. Presently, the Professional Services Committee (formerly the Executive Committee) will only consider the acceptance of counsellors with experience and qualifications after their attendance over a detailed interview on a case by case basis.
ReSource is a registered charity, and a member agency of the Community Chest. ReSource has been partly funded by the Community Chest for many years, the Fu Tak Iam Foundation Ltd since October 2015 and the Hong Kong Jockey Club Charities Trust up to March 2008. Other revenue sources are counselling fees paid by clients, private donations, and contributions from charitable organisations.
ReSource offers not only counselling for a wide range of issues, such as marriage, separation, divorce, depression, anxiety, stress, family problems, mental health support, alcohol and drug abuse, personal growth and relationships, but we also offer mental health services even for those who suffer from mental illnesses. We also had counsellors working with schools and the Cancer Support Group, CanSurvive. We have participated in RTHK's Radio Programmes on a regular basis, such as 'Teen Time' and the lunch time show. ReSource is also frequently asked to comment on pertinent issues by the press.
HELPING OTHERS HELP THEMSELVES
ReSource is non-sectarian and is not affiliated to any cultural organisation. Our counsellors are not required to assent to any dogma or beliefs; they are drawn from many different backgrounds. ReSource strongly believes in the autonomy and rights of the individual whilst recognizing the need to be a contributing and fulfilled member of society. Counsellors are expected to acknowledge the value and dignity of every human being regardless of race, sex, age, sexual orientation, beliefs, or socioeconomic background. Moreover, there is an additional intrinsic principle: the acceptance of and empathy with the client.
In the past years, we are specialised in offering services for marriages and partnerships which are in the process of breaking up. It is a confidential process designed to help separating couples reach their own mutually acceptable agreements regarding the ongoing arrangements for their children and/or the resolution of financial matters. It is a form of assisted negotiation in which a trained, impartial mediator facilitates both parties to communicate and negotiate in a dispute. Now, we offer an array of mental health services which target at providing a more holistic support for anyone who is experiencing mental health issues.
Copyright © Counselling Service Since 1973 All rights reserved.
Powered by ❤ Prohub Ltd
Copyright © Counselling Service Since 1973 All rights reserved.


Join us as we share our favorite tips to eat healthy even as busy moms!
In this episode, Stephanie and Aglaée share their secrets to eat healthy foods while running around a toddler and working. Learn about our must-have ingredients and convenience foods in the kitchen, how we plan our meals, how we batch cook, how we make our time in the kitchen more enjoyable and our top 3 favorite quick and simple meals that can be put together in fewer than 5 minutes!
On this episode, Danielle discusses picky eaters that are on the bottom of growth charts. You'll also learn how to work with your pediatrician; problems with the traditional recommendations for gaining weight; reasons why a child may be underweight; protein requirements for kids and ways to ensure intake; why kids throw food on the floor and play with their food; how sign language can help picky eaters and so much more! You'll walk alway from this episode equipped with tons of strategies for making healthy eating more fun and less stressful!
[32:45] Action steps you can implement right now!


Rust-Oleum chalked ultra matte paint creates an ultra-matte, velvety smooth finish with excellent adhesion and coverage. Elegant timeless colors bring new life to tired and worn pieces. Can be used on many interior surfaces such as bare and finished wood, glass, ceramic, canvas, metal and laminate. Covers up to 150 square ft.
Really beautiful chalk paint. I painted a dark dresser and used two coats. Highly recommend this brand and the price is reasonable. Will definitely use rust-oleum again.
Worked very well, no priming or sanding and looks great!
I want to paint EVERYTHING with this stuff. Painted an old cabinet that was a vinyl wood color- (eyesore) and now it matches great in my sons new room! Glad i got this size because i want to paint about 30 other pieces now :) Sorry i dont have a before photo of how terrible the shelf was.
15 of 17 people found the following review helpful.
Awesome value for a great chalk paint!
This chalk paint is a wonderful value as it dries quickly and has a nice matte finish. I applied two coats to an espresso headboard and a cherry wood set without sanding and it honestly came out much better than expected. The paint takes some getting used to as it is less forgiving in terms of going over multiple times before it dries as you will just pick the paint right back up onto the brush, however once you get the hang of it it's much easier. I really recommend this paint to those who may not want to fork out the extra cash for Annie Sloan's line of chalk paint but still want that nice matte finish or maybe want to experiment with chalk paint before investing in the nicer brands.


Simon Dyson's DQ at the BMW Masters has renewed the debate about whether viewers should be able to "help" the officials catch rules violations. It's the same old story -- Dyson tapped down a spike mark on his putting line, which is a clear violation, but nobody caught it until a viewer called in several hours after Dyson signed his card. From what I've heard, it appears Dyson just had a "brain fart" and did it absentmindedly, but it's still a violation of the rules.
And, as we have all become painfully aware, the penalty for signing an incorrect scorecard is disqualification.
I think it's time the Rules of Golf entered the 21st Century and the DQ penalty is eliminated -- well, mostly eliminated. There are some rule breaches for which a DQ is the ONLY prescribed penalty. In those cases, the DQ penalty should be enforced as it always has.
What I'm talking about here are those pesky breaches where you receive a stroke penalty if you catch it before you sign your scorecard... or get DQ'ed if you don't. The penalty for the rule breach is adding strokes to your score. The penalty is for signing an incorrect scorecard is a DQ.
It's the incorrect scorecard penalty that I want to see eliminated. If you're going to allow viewers to call in rule breaches -- viewers who may not even see the event until several hours after the round is over -- then you've got to remove the penalty.
Why was that penalty created in the first place? Clearly it was to prevent cheating. But in the modern game, we have electronic scorekeeping to keep everybody honest. My gosh, at many events everybody's scores are constantly updated on giant scoreboards around the course!
But, more importantly, having your peers and fans consider you a cheater is the kiss of death to your career. Just think about the fallout from Brandel Chamblee's gaff in the last couple of weeks when he said Tiger had been "...a little cavalier with the rules." The fact that Tiger's agent reacted so quickly -- and that Chamblee backpedaled just as quickly -- should be enough to make that clear. And with the ever-present video cameras on the course to make sure everything gets seen -- that no cheater has a realistic chance of getting away with his crime -- the need for the DQ penalty simply isn't there anymore.
The field is "protected" because violations are enforced. Nobody wants to see the violations ignored.
Because the infractions are on tape, it's a simple matter for both players and officials to go back and verify exactly what happened. Consequently there are fewer questions about the rightness of a ruling.
And players don't get the sudden shock of arriving at the course for their next round only to find there will be no next round. Players don't mind taking the penalty for their errors... but they'd like the chance to correct their scorecards and avoid a DQ.
It's time we bring golf out of the Dark Ages and into the Digital Age. Is that too much to ask? It's time to do away with the DQ penalty!!!
But that's a good example, Phil. If they could figure out Annie's score well enough to determine that her scorecard was wrong, they could determine the correct score. (If you don't know the true score, how can she be DQ'ed for an incorrect score? What are they measuring it against?) They should have just corrected the score and applied any stroke penalty that she had incurred. All these DQs are going to do is slow play even more because eventually players will call officials for everything.


text:
Cristiano Ronaldo Slammed By Fans After Joining the Saudi Arabian Club Al-Nassr
Cristiano Ronaldo signed the biggest deal in the history of football for a player. The modern-day legend has joined the Saudi Arabian Club Al-Nassr for a whopping US £177...
IPL Auction 2023 Where to Watch and When to Watch Live
IPL Auction 2023 all is set to begin at 2:30 in Kochi today, 23rd December 2022. The rights...
Upcoming EV Cars in India in 2023
Upcoming EV Cars in India in 2023: Tata Nixon EV, Tata Tigor EV, Tata Tiago EV, and Mercedes-Benz...
Shocking! Check India's Latest FIFA Ranking
FIFA World Cup 2022 is in full swing in Qatar and the Indians are cheering for their favourites...
Viacom18 Kicks Off 'Isse Bada Kuch Nahi Campaign' to Fuel the Fervor for Fifa World Cup 2022
The FIFA World Cup Qatar 2022 campaign, Isse Bada Kuch Nahi, was launched by Viacom18 Sports, India's newest...
Tata Motors Introduces Nexon EV Prime with Enhanced Features @ Rs 14.99 lacs
News Flickonclick Team - July 13, 2022
Tata Motors has announced the launch of the Nexon EV Prime with additional features, with prices starting at Rs 14.99 lakhs, India, for the basic...
Supermoon 2022: One of the century's biggest moons will be witnessed, Check the details
The biggest supermoon of 2022 is approaching. This upcoming space spectacular is a must-see since it will be quite unusual. For those concerned with...
YouTuber Gaurav Taneja arrested on his Birthday by Noida Police, Check details
On Saturday 9th July 2022, Noida Police imprisoned Vlogger and YouTuber Gaurav Taneja after a big crowd assembled outside the Sector 51 metro station...
Akasa Air Careers, Owner, Share Price, Uniform, Launch Date, and Everything Else You Should Know
News Virat Verma - July 7, 2022
Akasa Air, the ultra-low-cost airline, is set to begin its operation in July 2022 after acquiring 8 Boeing 737 MAX planes. Akasa Air is...
Which Keyword has the most search volume on Shaadi.com? no, not IPS or IAS
News Flickonclick Team - July 6, 2022
The minister Union Minister Rajeev Chandrasekhar recently joked on the Internet. He said that the most searched keyword on Shaadi.com is neither IAS nor...
New Rules for Pan Card, Income Tax, Online Payments, Credit Card and More You Should Know in 2022
The government of India and other significant organisations such as the Reserve Bank of India has announced a set of new rules applicable in...
1...456...162Page 5 of 162
Prime Minister Narendra Modi unveiled the logo for India's G20 presidency


text:
Child and Family Development
View allNewsPartnersSpotlight
Meet Eli
Spotlight February 27, 2019
Eli's friendly smile and cheerful disposition bring joy to everyone around him. "He pretty clearly has a special sparkle," says his dad, Eric. "When he laughs it just bubbles up from deep down and you can't help but laugh too," mom, Megan, adds. At two-and-a-half years of age, Eli loves swimming, dancing with dad, and…
Meet Tori
Tori enjoys music, exploring the garden and reading books. "She loves going to the park and she loves to swing and climb," said her mom, Nicki, adding that Tori is social and eager to "get involved in everything." The 2-year-old enjoys singing "Set it Free" with her older sister, Charli, who is six, and is…
Meet Theo
Theo is a pint-sized Renaissance man with a flair for watercolor painting, music and literature. The outgoing 2-year-old, who has radiant red hair and a brilliant smile, is a natural to visit local schools as the Poster Boy for the 2018-2019 Kids for Kids Penny Drive! Theo, who has hearing impairments, can now say more…
Meet Samuel and Jacob
Keeping up with Jacob and Samuel, twin toddlers with a zest for exploring, is a major workout for their parents. "They run, jump and climb. They are both like monkeys," says Jeanette, mother of the boys. It is hard to imagine that these active two-and-a-half-year-olds weighed just over a pound each at birth and spent…
Meet Rylan
Rylan is a busy toddler who likes summersaults and soccer. Rylan was just 9 months old when a pediatric nurse at Joya Child & Family Development gave him a quick Spot Vison screening that changed his life. The nurse reviewed the screening results and referred Rylan to a pediatric ophthalmologist for an in-depth exam. The…
Meet Nixson
Nixson's love of lawn care extends to mowers, leaf blowers, weed whackers and any other piece of noise-making gadgetry used to tidy up a yard. "He is up to date on landscaping trends," said his mom, Angela. When Nixson hears a lawnmower, he drops everything to watch the action. The 3-year-old is becoming confident…
Meet Linden
A bright-eyed little friend named Linden is walking! Linden was born with spina bifida, a spinal defect that threatens mobility. A few weeks shy of his third birthday, Linden took his first steps! It was a big moment for his family and for our staff at Joya Child & Family Development. The family's journey began…
Meet Lemmy
Lemmy enjoys reading books, watching Sesame Street and speeding around on a Ybike with her 7-year-old brother, Radlee, making siren noises in pursuit. With a loveable personality, constant smile and hearty laugh, Lemmy is a joy to parent. "She's a very, very happy girl. We got lucky with her," said her mom, Cora. Two-year-old Lemmy…
Meet Harper
Everyone loves Harper. With a heart-melting smile and a friendly wave, Harper quickly makes friends "She loves people and likes to smile at everyone we see," said her mom, Hagen. Harper has a syndrome that can cause a variety of medical and developmental challenges, including heart defects, immune system challenges, poor eyesight and delays in…
2118 W. Garland Ave.
Joya. All Rights Reserved.


Our only goal is to supply fellow vapers with an affordable yet delicious alternative to the current e-juice market.
Nothing like a delicious vape of fresh black cherry lemonade on a hot summer day.
A sweet red raspberry candy, similar in taste to a red fun dip.
The same sweet blueberry cotton candy we offer, but with a kick of menthol.
Like chewing on a fresh piece of sweet fruit stripes gum... except the flavor keeps on going after 30 seconds!
Guaranteed to be the best orange vape you have ever had! Like popping a brand new orange tic-tac into your mouth, or licking a cold orange creamsicle at the end of a long summer day.


text:
John Conyers leaves House committee post amid sexual misconduct claims
Democratic congressman denies allegations he used tax dollars to settle cases with women, as Nancy Pelosi faces criticism for praising his work
Amanda Holpuch in New York
@holpuch
Mon 27 Nov 2017 15.56 EST First published on Sun 26 Nov 2017 13.24 EST
John Conyers said he looked forward 'to vindicating myself' before the House ethics committee. Photograph: Alex Brandon/AP
The Democratic congressman John Conyers stepped down as ranking Democrat on the House judiciary committee on Sunday, amid allegations he used taxpayers' money to settle cases with women who accused him of sexual misconduct.
The news came shortly after the House minority leader, Nancy Pelosi, was widely criticised for calling the 88-year-old an "icon" because of his work in the civil rights movement.
"John Conyers is an icon in our country," she said. "He has done a great deal to protect women."
Trump reaffirms support for 'accused child molester' Roy Moore
Appearing on NBC's Meet the Press, Pelosi stopped short of calling for Conyers' resignation from Congress. The Michigan representative, she said, would "do the right thing".
Conyers, who denies the allegations, said in a statement he wanted to stay on the committee but did not want to undermine his colleagues as he faces an ethics investigation.
"I very much look forward to vindicating myself and my family before the House committee on ethics," Conyers said.
In a statement announcing Conyers' decision, Pelosi said: "We must ensure the Congress has a climate of dignity and respect with zero tolerance for sexual harassment."
Jerrold Nadler of New York, the second-ranking Democrat on the judiciary committee, said in a statement: "Mr Conyers has a 50-year legacy of advancing the cause of justice, and my job moving forward is to continue that critical work."
Lisa Bloom, an attorney for a woman who filed a sexual harassment complaint against Conyers in 2014, said in a statement her client was forced to sign a confidentiality agreement, which she asked Conyers and Congress to release.
"Basic fairness and decency dictate that if Mr Conyers can speak publicly about the matter, the woman should be free to do so as well," Bloom said.
In their own statement, 12 women who worked for Conyers over 30 years said: "We do not condone or justify sexual harassment and believe the ongoing national debate is important. While we do not pass judgment on the specific allegations reported in the press or the women who brought them, our experiences with Mr Conyers were quite different than the image of him being portrayed in the media."
Conyers was "a gentleman and never behaved in a sexually inappropriate manner in our presence", the women said. "He was respectful, valued our opinions, challenged our thinking, and treated us as professionals."
The statement ended: "We believe in due process and support allowing the ethics investigation to run its course."
Al Franken 'feels terribly' about sexual misconduct allegations
The allegations against Conyers come amid a cascade of accusations of sexual misconduct in politics, Hollywood and other industries.
In Democratic ranks, the Minnesota senator Al Franken has been accused of inappropriate behavior by four women. He has issued apologies and submitted his case to the ethics committee.
"I'm embarrassed and ashamed," Franken told the Minnesota Star Tribune on Sunday.
Franken contacted local Minnesota media a day before returning to Congress. The first allegation against him was made just before the Thanksgiving recess and he has kept a low profile in the nearly two weeks since.
In response to a question about whether he expects more women to come forward, Franken said: "If you had asked me two weeks ago, 'Would any woman say I had treated her with disrespect?' I would have said no.
"So this has just caught me by surprise ... I certainly hope not."


The Sisterhood is a group of women who are Influencing the world for Christ as "Sisters Serving The Savior". They have around 40 active members, meet once a month and are very active in missions and serving others. Offerings are taken up at the meetings and spent on immediate needs, such as school uniforms or helping a family who lost a job, etc.
Some of the ongoing projects they do include; Backpack items for school children, blessing bags and visiting angels for shut-ins or anyone who needs lifting up. They also sponsor the annual Lottie Moon Christmas Auction.
Ministries supported include: Noel Gandy a church planter in Iowa, the Wilcox Gallery to prevent human trafficking, the Christian Women's Job Corps, and Jennifer Walley with the True Heart Trio. Granite pieces are sold at local festivals to raise money to offset the cost of church members serving on mission trips. They also sponsor a church wide Valentine's Banquet each year where donations go to this same cause.