text:
\section{Experiments and Evaluation}

This section gives description of the characteristic experiments for the proposed decomposition algorithm including mapping function calibration and baseline comparison to evaluate the advantages of the decomposition method, compared with the method taking raw tactile displacement field as input.

\subsection{Mapping Function Calibration}

Calibration is performed to find the mapping functions $g_n$, $g_t$ and $g_\tau$. Here we choose regression using a small amount of data, considering the dimensional reduction that our algorithm realizes. We collect force and torque data using highly accurate Force/Torque sensor depicted in Fig. \ref{calibration_setup}a. ATI nano-17 is installed on and driven by the UR10 robot arm. A simple 3D-printed gripper is mounted on tool side of the Force/Torque sensor and used as a fixture of objects. To examine the consistency between and generalization capability to a wide range of objects with different shapes, sizes, textures, hardness and elasticity, 6 objects with these variances are selected, as given in Fig. \ref{calibration_setup}b. when collecting dataset, objects are tightly grasped by the gripper and pressed onto the sensing surface of the tactile sensor. Typical tactile deformation images are shown in Fig. \ref{calibration_setup}c, with the corresponding object labels.

\begin{figure}
	\centering
	\includegraphics[width=0.44\textwidth]{experimental_setup.png}
	\caption{Experimental setup of data collection for calibration and baseline comparison. (a). Tactile image and contact force/torque collection with robot-arm-driven fixture fixing object to make contact on the tactile sensor. (b). 6 objects for contact making. (c). Examples of contact images for 6 objects.}
	\label{calibration_setup}
\end{figure}

\begin{figure*}
	\centering
	\includegraphics[width=0.95\textwidth]{data_fitting.png}
	\caption{Calibration data and fitting results. Data collected using different objects are scattered with different colors. Data regression methods include RANSAC linear model and MLP regression. From left to right, charts are $F_n$ vs. $S_n$, $F_t$ vs. $S_t$ and $F_\tau$ vs. $S_\tau$.}
	\label{fig: char_plots}
\end{figure*}

As for the size of the calibration dataset, a total amount of 300 data points are collected, with 50 for each object. For every object, motions along  surface normal and tangent include pressing, surface dragging and twisting with randomized distance and angles in every data collection trial. The ranges of these randomized distance and angles are carefully adjusted to fit the working range of the sensor without dealing damage or too much wearing to the elastomer.

The calibration data is presented in Fig. \ref{fig: char_plots},  calculations of $S_n$, $S_t$ and $S_\tau$ use the calculation pipeline in Fig. \ref{calculation_flow}. Qualitatively, the linearity of the data is strong in the selected working range, which achieves dimensional reduction and guarantees low complexity for models to approximate the distribution of data. Besides, we notice that for normal force, the data distribution is less concentrated compared with those of the other two sets of data. It is ascribed to the lack of capability of the monocular camera inside the sensor to capture the markers' motion along the sensor surface normal, in which direction the deformation of elastomer balances a  large portion of external normal force. As a result, only divergence motion in 2D plane is used for calculating normal force, leading to a larger variance in the distribution for the normal force data subset.

Two models with low complexity is fitted to the three sets of data. First, linear model with RANSAC outlier rejection algorithm is chosen, considering there exist some outliers in the data. For example, some of the measured normal forces are of negative values, which is impossible in common cases. RANSAC iteratively chooses group of inliers that lead to the lowest regression error. Second model is a three-layer multi-layer perceptron (MLP) that was used in the previous works \cite{de2012force}\cite{fang2018dual}. Since the underlying distribution is of relative low dimension, the regression of MLP to the data can also generate good performance when small model is applied given small amount of data. As shown in Fig. \ref{fig: char_plots}, RANSAC linear model and MLP model have close prediction values, except that RANSAC performs better in capturing underlying distribution in normal force case by rejecting outliers from object 1 and object 6.

\subsection{Baseline Comparison}

Baseline comparison is given in this section. Regarding the prediction performance of contact force and torque based on $S_n$, $S_t$ and $S_\tau$ calculated from the decomposed components and raw displacement vector fields, three regression models are compared with mean-square-error (RMSE) metric. For decomposed 1D data fitting, RANSAC linear model, and MLP regression with three-layer structure with 10 hidden units are adopted. As for input vectors without decomposition, a significantly larger MLP regressor with five-layer structure and $512 \times 128 \times 10$ hidden units is used.

Two MLP models are all fully trained with L-BFGS optimizer. 6-fold cross validation with splits of the data from different objects are used for evaluation of the overall performances of models and also biases toward certain objects. The results are shown in  Table \ref{tab:RMSE}. RANSAC linear model excels in terms of the mean  RSME of the prediction for normal force and tangential force cases, whereas MLP regressor on the 1D data performs better in the aspect of the prediction variances and slightly better in the mean RSME for torsional case. This could be attributed to the outlier rejection mechanism of RANSAC linear model to sustain the disturbance of noises, which lead to lower average prediction errors. As expected, all three models give a larger RSME in the normal force case when being evaluated on data collected with object 6 after being trained on the other 5 objects during cross validation. However, the combination of raw displacement vector with complex MLP gives a lower variance in this case, showing more consistent performance across different objects and with noises. In summary, linear models with decomposition capture the underlying distribution better given small amount of available data, while one can expect MLP without decomposition can improve highly if large dataset is collected.

\begin{table}[]
\centering
\caption{RMSE of different methods on estimating the contact force and torque based on the decomposed deformation vector fields or raw deformation one.}
\label{tab:RMSE}
\resizebox{0.45\textwidth}{!}
{%
\begin{tabular}{c|c|c|c|c}
\hline
\multicolumn{2}{c|}{RMSE} & \multicolumn{2}{c|}{Decomposition} & No decomposition \\ \hline
\multicolumn{2}{c|}{Method} & \begin{tabular}[c]{@{}c@{}}RANSAC \\ Linear\end{tabular} & \begin{tabular}[c]{@{}c@{}}MLP \\ Regression\end{tabular} & \begin{tabular}[c]{@{}c@{}}MLP \\ Regression\end{tabular} \\ \hline
\multicolumn{2}{c|}{\begin{tabular}[c]{@{}c@{}}Model \\ complexity\end{tabular}} & 2 & 10 & $512\times128\times10$ \\ \hline
\multirow{2}{*}{Normal (N)} & Mean & \textbf{2.952} & 3.286 & 4.482 \\
 & Stdv & 2.584 & 2.497 & \textbf{1.295} \\ \hline
\multirow{2}{*}{Tangential (N)} & Mean & \textbf{0.241} & 0.242 & 1.544 \\
 & Stdv & 0.033 & \textbf{0.032} & 0.813 \\ \hline
\multirow{2}{*}{Torsional (Nmm)} & Mean & 5.862 & \textbf{5.621} & 6.769 \\
 & Stdv & 1.547 & \textbf{1.353} & 3.621 \\ \hline
\end{tabular}%
}
\end{table}

\section{Grasping Tasks}

\begin{figure}
	\centering
	\includegraphics[width=0.485\textwidth]{friction_data.png}
	\caption{Contact force signals under multiple sliding motion trials. Upper right chart is generated by measuring ratios $F_t/F_n$ at peaks of $F_t$ (as blue circles marked). Lower right chart is the ratio $F_t/F_n$ inside the window delineated in purple dash rectangle.}
	\label{friction_data}
\end{figure}

\begin{figure}
	\centering
	\includegraphics[width=0.35\textwidth]{grasping_setup.png}
	\caption{Adaptive grasping force control experiment. (a). Robotiq 2-finger 140 gripper with FingerVision as fingertips holding an object and then the object is pressed by hand till slip occurs. (b). Gripper with FingerVision holds an object and then the tangential load is increased/decreased by loading and unloading weights on top of the object. }
	\label{grasping_setup}
\end{figure}

In this section, effectiveness of the proposed contact force and torque estimation method for vision-based sensors is tested in grasping tasks. Sensing and visualization of contact information as well as adaptive control under external disturbances have been challenging tasks in robotic manipulation. Besides, situations are even more complex when introducing soft contact that brings in nonlinearity in deformation. Fig. \ref{friction_data} shows tangential force and normal force signals during multiple surface sliding trials (data collected by ATI nano-17 Force/Torque sensor). It is noticed from the chart on the upper right in Fig. \ref{friction_data} that the friction coefficient (equals to the ratio of tangential and normal forces $F_t/F_n$ when tangential force reaches each peak, as marked by blue circles) does not remain constant under different normal forces, which is one of the significant properties differences between hyperelastic contact and rigid contact. It also shows that within each trial of surface sliding, the ratio $F_t/F_n$ follows similar evolution: the ratio first rises; once reaches the maximum static friction coefficient, the ratio vibrates in a narrow band; the ratio drops afterwards, suggesting the occurrence of shear slip, as exhibited in the lower right chart of Fig. \ref{friction_data}.

\subsection{Grasping Stability Visualization}

Taking behavior of $F_t/F_n$ during slip phases into consideration, we implement a visualization system for monitoring of grasping force and slip, as shown in Fig. \ref{contact_phase_transitions}. Since the friction coefficient is not constant, we take the average of friction coefficients across working range of normal force as the nominal value for simplification and visually illustrate friction cones \cite{mason2001mechanics} with this coefficient. The FingerVision sensors are installed on Robotiq 2-finger 140 gripper, serving as finger tips and sensing units, mimicking human fingertips. With the force and torque estimation module, we illustrate transitions of contact phases by classifying the spaces where contact force vectors reside w.r.t. the friction cones. As given in Fig. \ref{contact_phase_transitions}(a), contact statuses are classified into 4 phases: 1) Stable contact; 2) Incipient slip; 3) Slipping; 4) Recovery phase when force vector is regulated back into the yellow or green regions. In Fig. \ref{contact_phase_transitions}(b) contact forces are shown as arrows in green (when the vectors are within the friction cones) and red (outside of the  friction cones). The capability of indicating contact phases is beneficial to grasp reconfiguration for stable grasp.

\subsection{Feedback Control of Grasping Force}

In-hand manipulations of objects usually require minimal grasping forces, because the contact condition keeps switching between unstable and stable statuses, e.g. pen rolling in human hand. And when picking up fragile objects, power grasps also need to be avoided. Thus, adaptive control for grasping force is critical in many scenarios. Here we implement a simple feedback controller that takes in contact force estimation and maintains the ratios $F_t/F_n$ in a band on the peripheries of the friction cones (visualized with nominal friction coefficients as described previously). Details of the controller are given in Algorithm \ref{ag:controller}. In the algorithm, variables with subscripts \textit{l} and \textit{r} belong to the left and right contacts.  A conservative control strategy is implemented in our work. To maintain the contact forces in the vicinity of friction cone margins, the gripper decreases the opening if both left and right forces exceed the upper limits of band of the cones and increases the opening while both left and right forces are lower than the lower limits.

\begin{algorithm}
	\caption{Grasping force controller}
	\hspace*{\algorithmicindent} \textbf{Input:} Contact forces  $F_l$, $F_r$; Gripper opening $D_g$;  \\
	\hspace*{\algorithmicindent} \quad \quad  \quad Band width $d$; Friction coefficient $\mu$.\\
	\hspace*{\algorithmicindent} \textbf{Output:} Gripper requested opening $D_r$.
	\label{pseudoPSO}
	\begin{algorithmic}[1]
		\State Initialize $r_l$, $r_r$ with $\mu$
		\While {True}
		\State $r_l \gets F_{lt}/F_{ln}$, $r_r \gets F_{rt}/F_{rn}$

		\If{$r_l > \mu+d/2$ \textbf{and} $r_r > \mu+d/2$ }
		\State $D_r = D_g - 1$
		\EndIf

		\If{$r_l < \mu+d/2$ \textbf{and} $r_r < \mu+d/2$ }
		\State $D_r = D_g + 1$
		\EndIf

		\EndWhile
	\end{algorithmic}
	\label{ag:controller}
\end{algorithm}

\begin{figure}
	\centering
	\includegraphics[width=0.475\textwidth]{contact_phases.png}
	\caption{Schematic diagram of contact phases and visualization for experiment in Fig. \ref{grasping_setup}(a). Force signals are plotted in Fig. \ref{grasping_plots}(a-b).}
	\label{contact_phase_transitions}
\end{figure}

The controller performs well in maintaining stable contact using minimal grasping forces in the object holding experiment during loading and unloading of weights that result in increase and decrease of tangential forces. As demonstrated in Fig. \ref{grasping_plots}(c-d), with controller being active, there are no or much shorter periods of crossovers(indicated as periods when $F_t/F_n$ dramatically rises that leads to contact slip). The ratio $F_t/F_n$ in right-side fingertip recovers quickly from the crossover region (rendered in yellow in Fig. \ref{grasping_plots}) due to the regulation of the force controller. The regulation process can also be seen from the visualization system, which keeps the contact force vectors around the margins of friction cones. Without force control, there is an extended longer period of crossover during loading process. The gripper fails to maintain contact forces inside the friction cones. Grasp fails if at least one contact breaks. It is worth noting that after slip happens on the surface, another crossover occurs due to the fact that dynamic friction coefficient is lower than static friction coefficient. The signals of left and right sensors are not of exactly the same forms, which could stem from the variances in sensor fabrication and calibration, object alignment difference for two contact surfaces and gripper pose not being exactly upright that leads to imbalanced loads on two fingertips.

\begin{figure}
	\centering
	\includegraphics[width=0.485\textwidth]{implement_plots.png}
	\caption{Grasping contact force signals under loads. Force x, y and z are the projections of the contact force onto the sensor surface coordinate system in Fig. \ref{calibration_setup}. (a-b). Changes of contact forces for manual press on grasped object, with constant opening distance between two fingertips. Four contact phases are illustrated in different colors. (c-d). Contact force signals during loading and unloading, with active grasp force controller. (e-f). Contact force signals during loading, without  grasping force controller. The unloading process is not given since contacts are broken, which leads to grasping failure.}
	\label{grasping_plots}
\end{figure}

\section{Method Description}

\begin{figure}
	\centering
	\includegraphics[width=0.48\textwidth]{response_simulation.png}
	\caption{Displacement fields of elastomer body under three load configurations shown in Fig. \ref{force_configuration}(a-c).}
	\label{response_simulation}
\end{figure}

Vision-based tactile sensors, such as Gelforce, FingerVision, Gelsight, make use of the deformation captured by the camera to infer contact forces by following hyperelastic continuum mechanics \cite{sato2010finger}, data fitting with calibration \cite{de2012force}\cite{fang2018dual} or both combined \cite{sato2010finger}. For analysis of hyperelastic deformation, finite element method (FEM) is commonly used. FEM approximates stress and strain response under external force that governed by continuum mechanics with finite number of nodes. To obtain an accurate result of surface motion, it is a common practice to increase the number of nodes with a proper meshing method, which results in increased dimension of the stiffness matrix that might be over demanding for computation in real-time applications. In this work, we take advantage of the insight that the displacement fields of the elastomer show unique and consistent graphical patterns under different single-axial loads (normal, tangential, and torsional loads) in simulation. These patterns possess quantitative properties that can be leveraged to formulate mapping from vector field with patterns to contact forces.

\subsection{Behavior under Different Loads}

For contact in reality, any surface traction comes in the form of contact friction, and thus tangential force would not exist without normal pressure being applied simultaneously. To explore the behavior of the displacement field under loads along different axes separately, we simulate with hyperelastic material in Abaqus. As shown in Fig. \ref{force_configuration}, within circle region on the top (in red), uniformly distributed normal force, tangential force and torsion along the surface normal (directions are shown with arrows) are applied with fixed bottom faces as boundary conditions. With these three configurations, typical simulation results are shown in Fig. \ref{response_simulation}. The displacement vector fields are obtained by further interpolating on a fixed-spacing grid and rendered with colors coding vectors' magnitude.

\begin{figure}
	\centering
	\includegraphics[width=0.44\textwidth]{decomposition_simulation.png}
	\caption{Decomposition result of simulated displacement field with the multi-axial loads. (a). Displacement vector field. (b). Curl-free component. (c). Divergence-free component. (d). Harmonic component.}
	\label{decomposition_simulation}
\end{figure}

Let $\vec{v}_i = \left( \delta x_i, \delta y_i \right) $ denotes $i^{th}$ displacement vector and $ V_i=\left\lbrace p_i, \vec{v}_i \right\rbrace $ denote $i^{th}$ displacement vector associated with position $ p_i = \left( x_i, y_i\right)$ being the start of the vector. Assume that the rotational centers $c$ of configuration are known, let $\vec{r}_{ij}$ be the arm of moment of $i^{th}$ vector w.r.t $j^{th}$ rotation center. Let $\vec{r}^a_i$ be the arm of moment w.r.t divergence center (the cross in Fig. \ref{response_simulation}(a)) and $\vec{r}^b_i$ be the arm of moment w.r.t the contact center (location of vector with maximum magnitude, the cross in Fig. \ref{response_simulation}(b)). From the displacement vector fields in simulation, it is observed that three graphical patterns of divergence, unidirection and rotation can be generated under normal, tangential and torsional forces correspondingly. With these patterns in Fig. \ref{response_simulation}, we notice the following quantitative properties:

\begin{itemize}
    \item For pattern (a), norm of vector summation $\smash{|\sum^{N-1}_{i=0} \vec{v}_{i}|}$ and magnitude of summation of moments w.r.t. the divergence center $\smash{\sum ^{N-1}_{i=0} \vec{r}^a_{i} \times \vec{v}_{i}}$ both yield small values, while summation of vector norms $\smash{\sum^{N-1}_{i=0} |\vec{v}_{i}|}$ gives a significantly larger value.

    \item For pattern (b), summation of moments w.r.t. the contact center $\smash{\sum ^{N-1}_{i=0} \vec{r}^b_{i} \times \vec{v}_{i}}$ yields a small magnitude, while norm of vector summation  $\smash{|\sum^{N-1}_{i=0} \vec{v}_{i}|}$ gives a larger value by comparison.

    \item For pattern (c), norm of vector summation $\smash{|\sum^{N-1}_{i=0} \vec{v}_{i}|}$ yields a small value, while summation of moments w.r.t the rotational center $\smash{\sum^{M}_{j} \sum ^{N-1}_{i=0} \vec{r}_{ij} \times \vec{v}_{i}}$ gives a much larger magnitude.
\end{itemize}
where N is the number of vectors, and M is the number of rotational centers of the vector field.

Assuming that an arbitrary vector field $\vec{V}$ is composed of vector fields with these diverging $\vec{V}_n$, unidirectional $\vec{V}_t$, and rotational $\vec{V}_{\tau}$ patterns, and following the quantitative properties above, we have formulations below

\begin{equation}
\begin{aligned}
    S_n &= \sum^{N-1}_{i=0, \vec{v}_i \in \vec{V}_n} |\vec{v}_{i}| = \sum^{N-1}_{i=0, \vec{v}_i \in \vec{V}_n + \vec{V}_\tau} |\vec{v}_{i}|\\
    S_t &= |\sum^{N-1}_{i=0, v_i \in \vec{V}_t} \vec{v}_{i}| = |\sum^{N-1}_{i=0, \vec{v}_i \in \vec{V}} \vec{v}_{i}|\\
    S_\tau &= \sum^{M}_{j} \sum ^{N-1}_{i=0, \vec{v}_i \in \vec{V}_\tau} \vec{r}_{ij} \times \vec{v}_{i}= \sum^{M}_{j} \sum ^{N-1}_{i=0, \vec{v}_i \in \vec{V}_n + \vec{V}_\tau} \vec{r}_{ij} \times \vec{v}_{i}\\

\end{aligned}
\label{eq: three_values}
\end{equation}
where $\vec{V} = \vec{V}_n + \vec{V}_t + \vec{V}_\tau$, and $S_n$, $S_t$ and $S_\tau$ are summation of vector norms on $\vec{V}_n$, norm of vector summation on $\vec{V}$, and total moments of vectors w.r.t. rotational centers on $\vec{V}_\tau$.

In reverse, estimation of contact force and torque can follow the scheme of computing $S_t$ given a displacement field $\vec{V}$ first, then decomposing $\vec{V}$ into $\vec{V}_n$ and $\vec{V}_\tau$ for computation of $S_n$ and $S_\tau$ following Eq. (\ref{eq: three_values}). The problem boils down to finding a suitable decomposition method.

\subsection{Decomposition Algorithm}

HHD method is a tool widely used in flow physics analysis to gain insights into such features as critical points, divergence source, sink, rotational vertex and curl distribution, etc.\cite{guo2005efficient}. H. Bhatia et al. \cite{bhatia2014natural} presented a natural HHD (nHHD) to tackle data-dependent boundary condition selection problem. In our work, we adopt nHHD to compute separated vector fields for the reason that in a 2D space, the displacement of elastomer under torsional and normal loads from simulation results have similar pattern representations to that in divergence-free and curl-free fields decomposed by nHHD.

According to \cite{bhatia2014natural}, considering the above smooth displacement vector field $\vec{V}: \mathbb{\si{\ohm}} \rightarrow \mathbb{R}^n$, where $\mathbb{\si{\ohm}} \subseteq \mathbb{R}^n$ (e.g. n = 2 in 2D case), we have

\begin{equation}
	\vec{V} =\vec{d} \ +\ \vec{r} \ +\vec{h}
	\label{hdd_decomposition}
\end{equation}
where $\vec{d}$ denotes curl-free component ($\nabla \times \vec{d} =\vec{0}$), $\vec{r}$ is divergence-free component ($\nabla \cdot \vec{r} =0$) and $\vec{h}$ is harmonic ($\nabla \times \vec{h} =\vec{0}, \nabla \cdot \vec{h} =0$). Eq. (\ref{hdd_decomposition}) is further transformed into Eq. (\ref{hdd_potential}) in the form of gradients of two scalar potential functions $R$ and $D$

\begin{equation}
	 \begin{split}
	\vec{V} & =\nabla D+\nabla \times R +\vec{h}\\
	& =\nabla D+J\nabla R+\vec{h}
	\end{split}
	\label{hdd_potential}
\end{equation}
where $\vec{d} \ =\ \nabla D$ and $\vec{r} =\nabla \times R =J\nabla R$ with $J$ being the $\pi/2$-rotation matrix. By applying divergence and curl operations, we obtain following Poisson equations

\begin{equation}
 \begin{array}{l}
\Delta D=\nabla \cdot \vec{V}\\
\Delta R=-\nabla \cdotp J\vec{V}
\end{array}
\label{poisson_function}
\end{equation}

 Therefore, Eq. (\ref{poisson_function}) can be solved using Green's function in the domain to obtain $\vec{d}$ and $\vec{r}$, and data-dependent boundary conditions are imposed to derived harmonic component $\vec{h}$ uniquely. For more implementation details of the solving process, it is recommended to refer to \cite{bhatia2014natural}. The rotational centers $c_+$ and(or) $c_-$ in $\vec{r}$ are localized where the maxima and(or) the minima of $R$ are achieved over the discrete domain of $\vec{V}$ if the extrema exist, and the arms of moments $\vec{r}_{i+}$ and(or) $\vec{r}_{i-}$ of $i^{th}$ vector can be obtained as Eq. (\ref{eq:arm_of_moment}) presents.

\begin{equation}
\label{eq:arm_of_moment}
\begin{aligned}
    c_+ &= p_{i_{max}} | i_{max} = \argmax_i R_i \\
    c_- &= p_{i_{min}} | i_{min} = \argmin_i R_i\\
    \vec{r}_{i+} &= p_i - c_+\\
    \vec{r}_{i-} &= p_i - c_-
\end{aligned}
\end{equation}
where $R_i$ is the value of the potential function $R$ at $p_i$.

\begin{figure}
	\centering
	\includegraphics[width=0.3\textwidth, height=3.5cm]{calculation_flow.png}
	\caption{Contact force and torque computation pipeline.}
	\label{calculation_flow}
\end{figure}

The result of HHD for the simulated displacement field under multi-axial loads is given in Fig. \ref{decomposition_simulation}. Although it is noticeable that the patterns given in the separated components are not identical to those shown in Fig. \ref{response_simulation} in terms of distribution of vector magnitude, calculation of $S_n$, $S_t$, and $S_\tau$ remains valid according to Eq. (\ref{eq: three_values}).
By combining the procedure proposed in section \RNum{3}-A and nHHD algorithm, we present the procedure to compute $S_n$, $S_t$ and $S_\tau$. $S_t$ is obtained from the raw displacement field following Eq. (\ref{eq: three_values}), and in parallel, the raw displacement field is fed into HHD module to generate two fields of interests: curl-free and divergence-free fields. $S_n$ and $S_\tau$ are calculated from these two vector fields with Eq. (\ref{eq: three_values}) and Eq. (\ref{eq:arm_of_moment}). The calculation scheme is illustrated in Fig. \ref{calculation_flow}.

Let the mapping from $S_n$, $S_t$ and $S_\tau$ to contact normal force $F_n$, tangential force $F_t$ and torque along surface normal $F_\tau$ be functions $g_n$, $g_t$ and $g_\tau$, respectively which connect to the estimations of contact force and torque in Eq. (\ref{eq: mapping_functions}). With the significantly dimensional reduction from tactile displacement vector field to $S_n$, $S_t$ and $S_\tau$, it can be expected that the complexity of the model used to predict contact force and torque using decomposed results will be much lower, compared to that  using raw displacement field.

\begin{equation}
    \begin{aligned}
        F_n &= g_n(S_n)\\
        F_t &= g_t(S_t)\\
        F_\tau &= g_\tau(S_\tau)
    \end{aligned}
    \label{eq: mapping_functions}
\end{equation}

\section*{Acknowledgment}

\section{Conclusion}

In this work, we develop a contact force and torque estimation method for vision-based tactile sensor using Helmholtz-Hodge Decomposition (HHD). Starting from observations of the relations between contact force and torque and marker displacement patterns, we establish the mapping from decomposed components of HHD to contact force and torque estimation. In characteristic experiment, the force and torque estimation results show high linearity and guarantee lower demands for data size and better accuracy on predictions using models with low complexity. The proposed method is further tested in both contact stability visualization and grasping with adaptive force control for verification of effectiveness and presents potential in facilitating studies of grasping stability metric. Future works fall mainly on integrating sensor and algorithm into grasping system to predict high-level physical information including object center of mass, estimation of object dynamics, and prediction of grasping stability.

\section{INTRODUCTION}

Tactile sensing has been investigated and proven to play critical roles in human interaction with the environment. For a robotic system, tactile sensor is also a key component for its perception system, especially in contact-rich manipulation tasks. However, tactile sensing technologies are relatively unexplored, comparing with great attention drawn to studies of visual perception principles and developments of algorithms in these decades, despite its complementary role to visual sensing in robotic scene perception.

The past few decades have seen vast emergence of various types of tactile sensors with different transducing principles, including capacitive, piezoelectric, piezoresistive, magneto-electric, etc. \cite{Dahiya:2010:TSH:1771964.1771965}. Recently, vision-based tactile sensors have been thriving and appearing in various robotic systems with advantages of easy fabrication, high resolution, and multi-axial deformation sensing capability, e.g. Gelforce \cite{sato2010finger}, FingerVision \cite{yamaguchi2016combining}, Gelsight \cite{yuan2015measurement} and a more compact Gelslim in \cite{donlon2018gelslim}.
In our previous work, we have developed a vision-based tactile sensor also called FingerVision \cite{zhang2018fingervision} (the name FingerVision was first introduced in \cite{yamaguchi2016combining}) and it was proven to be effective in the slip detection task. In this work, we aim at further exploiting the capability of recovering contact force and torque from the displacement field of the vision-based tactile sensor.

\begin{figure}
	\centering
	\includegraphics[width=0.4\textwidth]{fingervision_model_image.png}
	\caption{FingerVision tactile sensor. (a). Rendered 3D model  (in cutaway view). (b). Sensor prototype (details are referred to \cite{zhang2018fingervision}). (c-e). Raw image obtained from sensor, image with tracked displacement vectors and image with grid interpolated displacement vectors overlaying on top, respectively.}
	\label{fingervision_model}
\end{figure}

There are various ways to encode tactile signals, among which contact force and torque estimation from raw tactile information is of special interest, for it directly relates to statics or dynamics of the object during the interaction. For instance, human's intuitive feeling of finger skin traction and pressure and estimation of the center of mass of objects greatly enhance the success rate of dexterous, dynamic manipulation. In a robotic system, similarly, accurate force feedback helps robot capture the motion of the object and state transitions including contact making, slipping and contact breaking. Therefore, it endows robots with the capability of assessing grasp stability, which is essential for the successful execution of complex manipulation tasks.

For FingerVision sensor we developed in \cite{zhang2018fingervision} as reprinted in Fig. \ref{fingervision_model}, the sensing body is a clear elastomer with embedded black markers used as vision tracking features. The marker displacement vectors are seen as the grid sampling of the deformation in the elastomer layer.
When applied with external force and torque, deformation occurs in the hyperelastic body of FingerVision following continuum mechanics and the deformation fields show corresponding patterns under specific single-axial surface force and torque.
Bearing this hint, decomposition of raw displacement vector field into multiple separated vector fields with specific patterns would be potentially helpful to decouple the deformation under multi-axial loads. However, the displacement field patterns are also correlated with shapes of contact area and affected by nonlinear deformation induced by large contact forces and torques, and interference of force and torque between axes. Thus, evaluation of method's generalization capability and proper selection of working range are necessary.

In this paper, our goal is to effectively recover the contact surface force and torque from vision-based tactile sensors. Toward this target, these contributions are generated in our work:

\begin{itemize}
	\item Introduction of the displacement field patterns of elastomer on the vision-based tactile sensor when applied with single-axial forces and their quantitative properties are presented.
	\item Proposal of a method to decompose displacement field of vision-based sensors into components that can be further used in estimating contact force and torque based on Helmholtz-Hodge Decomposition algorithm. The proposed method is both data efficient and with low model complexity for regression.
\end{itemize}

The rest of this paper is arranged in the following structure: Section \RNum{2} introduces previous works related to methods of force estimation for tactile sensors. In section \RNum{3}, we explain the patterns observed and formulate mapping functions from vector fields of specific patterns to corresponding contact forces in simulation. Afterwards, we propose that HHD algorithm can be used to decompose displacement vector field into components with similar patterns that leads to estimation of contact force and torque. In section \RNum{4}, extensive characteristic experiments and comparison to state-of-the-art methods are given to show the effectiveness of the proposed method. In section \RNum{5}, we integrate the proposed method into a contact stability visualization and grasping force feedback control framework. Finally, discussion and conclusion are drawn in section \RNum{6}.

\section{Related Works}

\subsection{Tactile Sensors and Force Measurement}

Vision-based tactile sensors attract increasing attention for its sensing capability with multi-modal contact information in addition to advantages of superior sensing resolution, including deformation \cite{sato2010finger}\cite{yamaguchi2016combining}, object texture \cite{johnson2009retrographic,dong2017improved,yuan2017connecting}, contact area estimation \cite{donlon2018gelslim}, geometry reconstruction \cite{johnson2009retrographic}\cite{yuan2017gelsight} and force estimation \cite{sato2010finger}\cite{yamaguchi2016combining}\cite{yuan2017gelsight}\cite{mcinroe2018towards}. Besides, vision-based tactile sensors have been shown to perform well in high-level tasks like object recognition \cite{yuan2017connecting}, localization of dynamic object \cite{li2014localization}, and slip detection \cite{yuan2015measurement}\cite{zhang2018fingervision}\cite{dong2017improved}\cite{van2018slip}. Surface deformation serves as a basic signal modality for above higher-level information in these sensing systems.

\begin{figure}
	\centering
    \includegraphics[width=0.35\textwidth]{force_setup_simulation.png}
	\caption{Applied contact force configurations in simulation. (a). Normal force distributed uniformly. (b). Unidirectional tangential force distributed uniformly. (c). Torsional force along normal axis. (d). Combination of tangential, normal, and torsional forces.}
\label{force_configuration}
\end{figure}

Since the contact deformation is only one of the intermediate states for robotic manipulation feedback loop, researchers have been putting efforts into developing methods for recovering contact forces for tactile sensors. Generally, contact pressure distribution is relatively easier to be extracted for traditional capacitive, piezoelectric tactile array \cite{Dahiya:2010:TSH:1771964.1771965} or sensors utilizing total internal reflective (TIR) principle as presented in \cite{begej1988planar}. However, multi-axial-force estimation is much more challenging by comparison.
 Ohka et al. \cite{ohka1996data} presented a tactile sensor made of a rubber layer and a pyramid-shaped indenter on an acrylic plate that was able to capture changes of indentations of the pyramid array into the rubber skin with camera. According to the changes of the indentation areas, they successfully predicted three-axial contact forces.
 Sato et al. \cite{sato2010finger} fabricated a vision-based sensor called Gelforce with double-layer markers in different colors as tracking targets, which enables the measurement of motion along the surface normal via tracking the movement differences between markers in two layers. Based on an observational method and calibration, multi-axis force could be extracted from this complex fingertip-shaped sensor. Calibration procedures were specifically designed for the sensors making contact with probe-shape objects and generalization testing to different contact objects were not performed.
  Vogt et al. \cite{vogt2013design} built a microfluid-based flexible skin that can detect and differentiate normal and shear force, whereas the system suffered from a lower response time that was not suitable for robotic scenarios. In addition, the microfluid-based sensor could only estimate force and was inferior in multi-modality sensing by comparison with vision-based tactile sensors.

  Neural network has shown its usefulness in recovering contact force for tactile sensors.
  Maria et al. \cite{de2012force} designed a tactile sensor using an array of paired light emitters and receivers that was able to capture deformation in local region and infer contact forces with trained neural network.
  In \cite{fang2018dual}, multi-layer neural network was utilized in mapping from markers displacement field to three-axial contact force with a relatively low error on a Gelsight-like sensor. However, neural networks are usually not data-efficient and suffers from overfitting when only a small amount of data is available. Additionally, above works also didn't discuss the generalization performance on different contact objects.
  In our work, we start by observing the response patterns of displacement field to different force and torque configurations, and based on the observation, we decompose vector field into components containing individual patterns to infer decoupled contact forces. This method significantly reduces the dimension of deformation vector field and is shown to retain good invariance to different contact objects.

\subsection{Helmholtz-Hodge Decomposition}

Helmholtz-Hodge Decomposition is commonly used in motion analysis, e.g. target tracking in computer vision, computational fluid motions analysis \cite{guo2005efficient}, acting as feature extraction to capture divergence source, sink and vertex of rotational motion for vector fields. HHD describes a vector field in the form of the summation of a divergence-free, a curl-free, and a harmonic flow, with manually set boundary condition imposed to get a unique solution. In \cite{bhatia2014natural}, Bhatia et al. proposed a natural Helmholtz-Hodge Decomposition (nHHD) method enabling defect-free analysis for various boundaries conditions with a data-driven method. In this work, we adopt nHHD to decompose our displacement field into separated components corresponding to the responses of specific external contact forces. We show that this tool is effective in recovering contact forces for the deformable medium used in most vision-based tactile sensors by quantitative analysis, although theoretical relations between the decomposition component patterns and patterns observed in the simulation have not been established yet.

\section{FingerVision Design and Fabrication}

This section introduces the conceptual design of FingerVision, followed by fabrication process in detail.

\subsection{Conceptual Design}

The basic scheme to get contact information from optical-based tactile sensor is by inducing large
deformation of material in contact region for camera to capture the deformation. Similar to \cite{yamaguchi2016combining}, our sensor consists of elastomer with embedded markers, camera and supporting frame.  Although the tactile sensor developed in \cite{yamaguchi2016combining} is versatile on sensing multiple modalities, the relatively low stability during real robotic action makes
their sensor not an ideal tactile component for robots. In the work of \cite{dong2017improved}, Gelsight sensor is compact,
easy to use, able to reconstruct super fine contact surface, however, extra calibration of correspondence between color and surface normal is required.
Our sensor design emphasizes on stability, durability, readiness to use and adopts optical-based sensor scheme for its superior properties.

\begin{figure}
	\centering
	\includegraphics[width=0.46\textwidth]{diagram_render_and_finish.png}
	\caption{Design and fabrication of FingerVision tactile sensor. (a). Conceptual diagram of sensor. (b). Finish of tactile sensor. (c). Fabrication process. (c). Rendered model of sensor supporting frame. (d). Casting in action with frame in mold. (e). Intermediate step after markers attached.}
\end{figure}

For the design of optical-based tactile sensor, the following issues need to be taken into account.
\begin{itemize}
	\item \textit{Functionality}. To enhance the stability of markers tracking, consistent background and even illumination are critical. Wide view angle fisheye is necessary because large view field is needed given short focusing range.
	\item \textit{Number
of components}. Large number of components increases degree of difficulty during fabrication process and augments the complexity of system, which requires more characteristics to be set properly.
	\item \textit{Compactness}. Compactness of sensor determines whether it can be installed on robot fingers and limits the range of tasks it can be applied to.
\end{itemize}

Conceptual design of FingerVision is depicted in Fig. 1(a). Fisheye camera, a transparent elastic
layer, and the supporting frame comprise the hardware of the sensor. Inside the elastomer,
markers are used as tracking feature points reflecting the displacement of material in
neighborhood, and with a compact 2D marker array, dense displacement field of contact
surface can be acquired. We choose fisheye as transducer due to the relative larger field of view,
which achieves a better trade-off between the compactness of sensor and large sensing area.
High tracking stability is guaranteed with even illumination, rather than the way of light guiding through acrylic plate as reported in \cite{yuan2015measurement}, in our design, the LED light source
is put outside of white supporting frame (via 3D printing), and the light transmits through the resin
with large diffusion , which consequently illuminates the sensing area evener. Unlike the design in \cite{yamaguchi2016combining}, we coat the gel layer with another reflective layer to block external light sources that can deteriorate the tracking
performance of sensor. Similarly, the supporting frame is designed to be fully enclosed to increase tracking stability.

Fig. 1(c) and (d) show the rendered 3D model of sensor frame and mold for the casting
of Gel layer respectively. After iterations of design, fabrication and redesign, it is found that
distributed locking holes in the outer ridge of frame are also important to maintain firm attachment between silicone rubber layer and the frame, learned from \cite{yamaguchi2017implementing}.

\subsection{Fabrication}

Our fabrication process includes three steps as shown below, in addition, material and components are also described.

1) \textit{3D printing of sensor frame and casting mold}. Benefiting from the easily available 3D printing, we are able to accelerate the prototyping process. The sensor frame and
mold are made of Epoxy Resin with SLA 3D printing technology,
possessing properties of smooth surface finish, good accuracy, and mechanical strength.

2) \textit{Elastomer casting and marker embedding}. As for material of the elastomer, we
choose a transparent silicone rubber \textit{Solaris} from Smooth-On, Inc. This material, come
with two parts, A and B, and cures with weight ratio of \textit{1A:1B}. The casting lasts for 24 hours in room temperature and can be accelerated in the oven, the shore hardness of cured
silicone rubber is 15A. To put markers on the surface, the markers are patterned on a water transfer paper
first, and follow the product instruction to transfer ink from paper to gel surface. In this manner,
the markers density is much higher than that of the method used in \cite{yamaguchi2016combining} by
embedding plastic beads into the elastic layer. Fig. 1(d) shows the casting of sensor prototype in action.

3) \textit{Assembly of sensor components}. The camera we use is a commercially available CCD web
camera with fisheye lens. Before the camera is installed on the frame,
fisheye camera calibration is required. Here we adopt chessboard camera calibration method for
fisheye lens to estimate intrinsic matrix. Since the working focus range of camera is short compared to other application scenarios, calibration chessboard needs to be printed with small size as well.
After these three steps, we obtain the optical-based tactile sensor as shown in Fig. 1(b).
\section{Sensor Signal Processing}

\begin{figure}
	\centering
	\includegraphics[width=0.48\textwidth]{data_processing_flow.png}
	\caption{Sensor data processing pipeline and corresponding results (with sensor pressed by thumb). (a). Captured input image. (b). Key points detection result. (c). Marker tracking vectors. (d). Interpolated marker displacement field.}
\end{figure}

After acquiring images of contact region, additional processing is conducted to obtain contact deformation field. Fig. 2 shows the signal acquisition and processing pipeline with results step-by-step. Markers detection,
tracking algorithm and deformation field interpolation are three essential components of
the system. For the markers are 2D array of black dots on a lighter background, we adopt
blob detection to calculate centers of every black dots. With fully enclosed chamber and even illumination, the blob detection works stably with fine-tuned parameters. A sample image and its key points are shown in
Fig. 2(a) and (b). Compared with transparent sensing elastomer design in \cite{yamaguchi2016combining}, our sensor outperforms in terms of robustness in various working scene.

\begin{algorithm}

	\caption{Markers Tracking Algorithm}

	\begin{algorithmic}

		\WHILE {TRUE}
		\STATE Capture image
		\STATE $centroids \gets blobDetection(image)$
		\IF {First loop}
		\STATE $T, initPos \gets centroids$
		\STATE Continue

		\ELSE
		\STATE $C \gets KNN(initPos, centroids)$

		\FOR {$k \gets 1$ \TO  $N$}
		\STATE $lastPoint \gets T_k$
		\STATE $currentPoint \gets C(lastPoint)$

		\IF {$dist(lastPoint, currentPoint) \leq d$}
		\STATE Update lastPoint in T with currentPoint
		\ENDIF
		\ENDFOR
		\ENDIF

		\ENDWHILE
	\end{algorithmic}
\end{algorithm}

Recovering of deformation field requires tracking of markers sequentially on time domain. After the centroids of markers are obtained in marker detection module, we proceed to
track the movement of every marker individually. Commonly there are two approaches for tracking problems, one is by updating correspondences between consecutive frames and then tracing back to the initial frame to obtain displacements, another is by registering correspondences of markers between first frame and current frame. For registration methodology, the state-of-the-art of non-rigid registration methods are
coherent points drift method ( CPD) \cite{myronenko2007non} and elastic Iterative Closet Points (ICP) \cite{stewart2003dual} method. Although registration methods is able to compute correspondences between arbitrary two frames, which makes it more robust and less interdependent between frame to frame, the iterative optimization scheme severely deteriorates the real-time sensing capability.

Therefore, we adopt the scheme of updating correspondences frame by frame, and the tracking algorithm with
tolerance to markers detection failure is developed. Firstly K-nearest neighborhood
method is used to obtain nearest points correspondences $C$ between previous frame and
current frame, and then corresponded pairs with euclidean distance larger than manually set threshold $d$ are rejected, which is reasonable by assuming that the displacements of points are small
with the short time elapsed between consecutive frames. Thereafter, valid point
correspondences are updated in tracking matrix $T$ as latest correspondence matrix
associating initial marker positions and that of last updated markers.

Deformation of elastic material is naturally smooth with its low pass filter effect for external excitation.
Therefore, to further increase the density of deformation field and obtain
vector field with fixed size in fixed locations to facilitate further usages, we perform displacement
field interpolation based on the tracked displacement vectors. Here radius basis function (RBF) interpolation
method is selected to obtain smooth interpolation with relatively lower computation cost. The interpolation result is shown in Fig. 2. The frame rate of our set up (Intel Core i7-7700K@4.2GHz, 8 cores) with the processing algorithm is 15 FPS, which is adequate for various robotic tasks.

\begin{figure}
	\centering
	\includegraphics[width=0.45\textwidth]{response_three_excitation.png}
	\caption{Deformation field and orthogonal projection when excited with different directional forces (Colormap to value: Green: neutral, Red: positive, Blue: negative). (a). Applied with Normal force. (b). Applied with normal and tangential force. (c). Applied with normal and rotational force along contact surface normal.}
\end{figure}

Contact motion can be decomposed into normal pressing, tangential translation,
torsional motion. However, in real life, contact breaking happens when pure tangential force is applied without normal pressing. Fig. 3 shows the deformation field obtained with FingerVision sensor while being excited by pure normal, tangential coupled with normal, and torsional coupled with normal forces with a doom shape indent head. It is obvious that our sensor signal can distinguish different contact forces successfully, which is useful for various robot manipulation tasks. Application to slip detection is implemented in next section and it is shown to be effective.

\section{Slip Detection}
In this section, we propose a learning based slip detection framework making use of contact deformation field generated by FingerVision sensor. The framework is demonstrated to accurately classify contact into slip and non-slip states using tactile information only.
\subsection{Model Description}

\begin{figure*}
	\centering
	\includegraphics[width=0.7\textwidth, height=3.5cm]{diagram_NN.png}
	\caption{The architecture of slip detection framework using convolutional LSTM network and FingerVision Data.}
\end{figure*}

The architecture of slip detection framework is shown in Fig. 4. Knowing that slip occurrence is a spatiotemporal event on the contact surface, we adopt convolutional LSTM for its capability of capturing spatiotemporal features transition simultaneously \cite{shi1506convolutional}. Compared to the structure concatenating time distributed CNN with LSTM as \cite{li2018slip}, convolutional LSTM network is preferred because spatial feature extraction parameters (convolution kernel weights) are also the hidden node parameters and back propagation update is performed on both domains simultaneously. Besides, the well-performing CNN used in \cite{li2018slip} Inception V3 model contains over 7 million parameters \cite{canziani2016analysis}, which makes the forward pass computation too demanding for real-time implementation. In this perspective, It is alleviating that our model has drastically lower 269,826 parameters. We feed 10 frames of multi-channel images containing deformation field with x axis projection, y axis projection, and magnitude value jointly into convolutional LSTM network sequentially. Since the multiplication operation in common LSTM is replaced by convolutional operation \cite{shi1506convolutional}, the output of LSTM is also 3D tensors. The output of convolutional LSTM is flatten and fed to fully connected layer (FC layer) afterwards. LSTM layer with 64 hidden nodes with $5\times5$ kernal size is set for our model. Multiple layer LSTM structure is beneficial for long term dependency learning, however, considering our data is collected in a relatively short period, single layer structure is constructed. Finally, the FC layer outputs two variables representing the probability of stable contact or slip.

As for the training setup, Pytorch package is used to build the network and our model is trained on Nvidia GeForce 1060 (6GB graphic memory). In the training phase, The network weights are initialized with random small values and we choose two classes cross entropy as loss function. Adam optimizer with learning rate of $1\times10^{-5}$ is used for network parameters update.

\subsection{Experimental Analysis}

\begin{figure}
	\centering
	\includegraphics[width=0.45\textwidth]{dataset_objects.png}
	\caption{12 daily objects used in dataset collection.}
\end{figure}

\textbf{Data collection}. To distinguish between slip happening and stable contact, we take advantage of the intuitive human sense of slip since there are no other more reliable artificial slip sensing solutions available except for success/failure labelling of grasp as been implemented in \cite{li2018slip}, \cite{calandra2017feeling}, which is derivative results of slip or stable contact. For data collection, 12 daily objects are used as shown in Fig. 5. Objects comes in different size, shape and material. During every collection, an object is pressed against sensor surface by human hand, then force are applied to objects to generate target motion in a certain period. The labelling of a specific data is determined by human feeling of slip, translational and rotational slip are included and incipient slip is also labelled into slip category. Every raw data contains 15 frames of $30\times30$ pixel interpolated deformation field images (3-channels: dx, dy and magnitude) with an acquisition rate of 15 HZ. The sensor is fixed on a table during data collection to avoid false labelling. Fig. 6 shows an example of data collection with USB flash disk being the contact object.

In our work, a relative small size of dataset is first manually collected and further slicing to increase dataset volume is performed. Total amount of 1600 data with 800 slip class and 800 non-slip data are obtained. By slicing out subsets from raw data as in \cite{li2018slip}, we increase the dataset size by 5 times to 8000. Five subsets with each containing 10 successive frames are selected out of raw data using a sliding window with stride of 1 frame. As for dataset splitting, 90\% of data is used for training and 10\% for testing.

\begin{figure}
	\centering
	\includegraphics[width=0.45\textwidth, height=3cm]{data_collection_sample.png}
	\caption{An example of data collection process with USB flash disk. The first row are images recorded by another external camera monitoring the whole process (notes that images in first row are not accurately synchronized to images in later rows). The last 3 rows are magnitude, x axis projection, y axis projection of deformation field respectively.}
\end{figure}

\begin{figure}
	\centering
	\includegraphics[width=0.45\textwidth, height=4cm]{training_process.png}
	\caption{The network model training process.}
\end{figure}

\textbf{Results analysis}. From training history plotted in Fig. 7, we can see that our prediction accuracy on testing dataset rises over 95\% in first 1000 steps with batch size of 32, equivalent to a training time of 6m23s under our hardware setting. \cite{li2018slip} and \cite{van2018slip} are recent works similar to our paper, Table 1 presents a comparison between the performance of our method and these works in terms of accuracy. According to the testing results, our model using sequential frames of interpolated contact deformation field shows higher accuracy than other works. Furthermore, with a much lower input resolution and relatively shallower network structure, our forward pass time is as low as 0.014 seconds, making it a suitable framework to satisfy real-time requirements of robotic manipulation tasks with mobile computing devices. On another perspective, the performance also shows the effectiveness of our tactile sensor on tracking of contact events. For robot grasping problem, our sensor with proper algorithms can surely improve robot performance by providing dynamic spatiotemperal awareness of contact events.

\begin{table}
	\centering
	\caption{Slip detection accuracy of different works. Highest accuracy in each experiments are used.}
	\begin{tabular}{|c|c|c|}
		\hline
		\textbf{Input} & \textbf{Framework} & \textbf{Accuracy} \\
		\hline
		Interpolated deformation field (image) & ConvLSTM & \textbf{97.62\%} \\
		\hline
		Tactile and external image \cite{li2018slip} & CNN+LSTM & 88.03\% \\
		\hline
		Spectral univariate signals \cite{van2018slip} & LSTM & 94.5\% \\
		\hline

	\end{tabular}

\end{table}

redpajama_set_name: RedPajamaArXiv


text:
Trending publication
Nutter Bank Report, June 2008
06.30.2008 | Legal Update
1. Fees May Be Imposed and Costs Passed On During Default Cure Period
2. Altering Equity Credit Lines Raises Compliance Issues, FDIC Warns
3. Auditor Attestation Compliance Deadline Extended for One Year
4. Guidance on Managing Third-Party Risks Issued by FDIC
5. Other Developments: FDIC Appeal Process and FHA Mortgage Insurance
Full Reports
During the 90-day mortgage default cure period required by recent Massachusetts legislation, a bank may charge a borrower a fee up to a specified cap for revising loan terms and may pass on other appropriate costs and expenses as long as the borrower is not assessed any fee that is attributable to the exercise of his or her right to cure the default, according to a recent Division of Banks opinion letter. In Opinion 08-016 issued on June 25, the Commissioner of Banks stated that payments made by a lender to protect its interests in the property, including but not limited to payments of real estate taxes, hazard insurance premiums, condominium fees, condominium special assessments, and other municipal charges such as water, sewer and electric, are appropriate and may be passed on to the borrower. The Commissioner of Banks "strongly encourages" lenders to advise a borrower in default, preferably when the required notice of the borrower's right to cure is provided, that the borrower remains responsible for complying with other provisions of the mortgage such as provisions requiring the borrower to make the payments specified above and that, if necessary, the lender will take steps to protect its secured status which would lead to additional charges to the borrower. The law also allows late fees and per diem interest to be charged to the borrower during the cure period.
Nutter Notes: Chapter 244, Section 35A was added to the General Laws by Chapter 206 of the Acts of 2007. Under the new law, Massachusetts homeowners facing foreclosure are entitled to a notice of statutory delinquency and a 90-day right to cure a payment default once during any 5-year period. The 90-day cure period applies to foreclosures initiated on or after May 1, 2008. Subsection (d) of Section 35A states in part that "[t]o cure a default prior to acceleration under this section, a mortgagor shall not be required to pay any charge, fee, or penalty attributable to the exercise of the right to cure a default," with the exception of late fees (as permitted under Chapter 183, Section 59 of the General Laws) and per-diem interest to cure such default. Among the fees and charges specifically prohibited by the law are attorneys' fees relating to the borrower's default that are incurred by the mortgagee during the 90-day cure period. In reviewing Chapter 206, the Commissioner of Banks concluded that the prohibition only applies to fees and charges related to curing a default, and noted that Chapter 206 amends the revision of terms statute, Chapter 183, Section 63A of the General Laws, to increase the fee that may be charged for a revision of terms of a mortgage loan from ½ of 1 percent to 1 percent of the outstanding principal loan balance or the revised balance, whichever is greater.
Banks that suspend or reduce home equity lines of credit as a result of increased risk exposure must comply with applicable consumer protection requirements, the FDIC said in Financial Institution Letter 58-2008 issued on June 26. Because home values have declined in many areas across the country and many borrowers are experiencing financial difficulties, home equity lines of credit may present increased risk, the FDIC said. In response to the growing risk of loss, banks "may appropriately reduce or suspend the . . . credit limits of some borrowers [on home equity lines of credit]" under the circumstances allowed by the Federal Reserve Board's Regulation Z which implements the Truth in Lending Act and as set forth more fully in the May 2005 Interagency Credit Risk Management Guidance for Home Equity Lending, and the October 2006 Addendum. If, for risk management purposes, a bank decides to reduce or suspend a home equity line of credit, legal requirements that protect consumers must be followed. A bank must comply with the requirements of Regulation Z, and should be aware of additional requirements under the Federal Trade Commission Act, the Equal Credit Opportunity Act and the Fair Housing Act. Banks should work with borrowers, wherever possible, "to minimize hardships that may result from such suspensions or reductions."
Nutter Notes: Regulation Z generally prohibits lenders from changing the terms of home equity lines of credit, with certain exceptions. For example, Regulation Z expressly permits lenders to prohibit additional extensions of credit or reduce the applicable credit limit during any period in which the value of the dwelling that secures the plan "declines significantly below the dwelling's appraised value" for purposes of the plan. To use this exception, a lender must determine that a "significant decline" in value has occurred. The term "significant decline" is not defined in the regulation. However, the Federal Reserve Board's Official Staff Interpretations of Regulation Z include an example indicating that, while a "significant decline" in value will vary according to the circumstances, such a decline has occurred if the unencumbered equity is reduced by 50 percent. According to the Official Staff Interpretations, a lender is not required to obtain an appraisal before suspending credit privileges, but there must be a significant decline in value. Although full individual appraisals need not be obtained, banks should have a sound factual basis for determining that a property has experienced a significant decline in value. For example, automated valuation models or local tax assessments may be used under certain circumstances.
The Securities and Exchange Commission has approved a one-year extension of the compliance deadline for smaller public companies to meet the Section 404(b) auditor attestation requirements of the Sarbanes-Oxley Act. In the June 20 announcement, the SEC said that it has received Office of Management and Budget approval to proceed with data collection for a study of the costs and benefits of Section 404 implementation, focusing on the consequences for smaller companies. The results of the study are expected to become available before the end of the extension period. With the extension, smaller companies will now be required to provide the attestation reports in their annual reports for fiscal years ending on or after Dec. 15, 2009. SEC Chairman Cox first proposed the one-year delay for small businesses in December 2007, and the SEC formally proposed the extension in February of this year. Section 404 has two provisions. Section 404(a) requires company management to assess the effectiveness of the company's internal controls over financial reporting, while Section 404(b) requires an auditor attestation on management's assessment. Larger companies, comprising more than 95 percent of the market capitalization of the equity securities markets in the United States, have been subject to both provisions since 2004.
Nutter Notes: The SEC staff will begin the collection of data through interviews and other outreach. With the key financial data for annual reports becoming available to companies this spring, the SEC staff will be moving forward with interviews and a web-based survey as part of its effort to collect "real-world data" from a broad array of companies, "analyzing what drives costs, particularly for smaller companies," and where companies and investors derive the benefits from Section 404. The SEC staff's cost-benefit study will help determine whether the new management guidance on evaluating the internal controls over financial reporting issued by the SEC in June 2007 and the Public Company Accounting Oversight Board's Auditing Standard No. 5 approved by the SEC in July 2007 are having the intended effect of facilitating more cost-effective internal control evaluations and audits of smaller reporting companies. The study includes gathering new data from a broad array of companies about the costs and benefits of compliance with the Section 404 requirements. The extension of the Section 404(b) compliance date for smaller companies is the latest in a series of SEC actions to help reduce unnecessary compliance costs for smaller companies. In 2007, the SEC issued new guidance for management's Section 404 assessment to help companies focus their reviews on key internal control issues.
The FDIC has published guidance describing risks to financial institutions that may arise from third-party relationships and recommending risk management principles to provide an outline for assessing, measuring, monitoring and controlling the risks associated with those relationships. The guidance is attached to Financial Institution Letter FIL-44-2008 released on June 6. The potential risks addressed in the guidance include strategic risk, reputation risk, operational risk, transaction risk, credit risk, and compliance risk. According to the guidance, there are four elements of an effective risk management process: risk assessment, due diligence in selecting a third party, contract structuring, and oversight. The FDIC stated that it will hold an institution's board of directors and senior management responsible for identifying and controlling risks arising from third-party relationships to the same extent as if the activity were conducted by the institution directly. The guidance indicates that the principal focus of a supervisory review of an institution's third-party risk management will be management's record of and process for assessing, measuring, monitoring and controlling third-party risks.
Nutter Notes: The FDIC's third-party risk management guidance is based on and supplements the principles contained in other guidance that has previously addressed third-party risk in the context of specific functions, such as information technology. The FDIC stated that the guidance is intended to assist in the effective management of third-party relationships, and should not be considered to be a set of required procedures. Regardless of the nature of the potential relationship, the guidance recommends that all risk assessments begin with a consideration of whether the relationship is consistent with the institution's business strategy and should include an analysis of costs, benefits, legal issues and the risks particular to the third party under consideration. The guidance also suggests that an expanded analysis may be called for if the relationship relates to a new activity or a new product or service for the institution. The guidance recommends that initial risk assessment include a review of the institution's ability to provide adequate oversight of the proposed relationship and estimates of the long-term financial effect of the relationship.
FDIC Proposes Amendments to Its Supervisory Appeals Process
The FDIC is requesting comments on a proposal to amend its Guidelines for Appeals of Material Supervisory Determinations. The proposed amendments, published on May 27, are intended to better align the FDIC's Supervisory Appeals Review Committee process with the supervisory determinations and appeals procedures of the other federal banking agencies. The FDIC's Guidelines for Appeals of Material Supervisory Determinations describe the types of determinations that are eligible for review and the process by which appeals will be considered and decided.
Nutter Notes: The FDIC proposes to modify the supervisory determinations eligible for appeal to eliminate the ability of an institution to appeal to the Supervisory Appeals Review Committee any "[f]ormal enforcement-related decisions, including determinations and the underlying facts and circumstances that form the basis of a recommended or pending formal enforcement action, and FDIC determinations regarding compliance with an existing formal enforcement action." Comments on the proposed amendments are due by July 28.
FHA Changes Policy to Facilitate Foreclosure Sales
The Federal Housing Administration announced a temporary policy change on June 13 that allows for the immediate sale of vacant foreclosed properties with FHA–backed mortgage insurance. For one year, foreclosed properties marketed and sold by property disposition firms on behalf of lenders will no longer be subject to the customary 90-day waiting period to qualify for FHA mortgage insurance.
Nutter Notes: FHA mortgage insurance is not available for a property owned by the seller for 90 or fewer days, which is meant to prevent so-called "flipping" of properties. The policy causes foreclosed properties to remain vacant for 90 days in order to qualify for FHA mortgage insurance. FHA's new policy will permit the immediate sale of foreclosed properties to legitimate borrowers wishing to use FHA-insured financing, the FHA said.
Nutter Bank Report
Nutter Bank Report is a monthly electronic publication of the Banking and Financial Services Group of the law firm of Nutter McClennen & Fish LLP. Chambers and Partners, the international law firm rating service, has ranked Nutter's Banking and Financial Services practice among the top banking practices in the nation in the 2007 Chambers and Partners U.S. rankings. The "well known and well-versed" Nutter team "excels" at corporate and regulatory banking advice, according to the 2007 Chambers Guide. Visit the 2007 U.S. rankings at ChambersandPartners.com. The Nutter Bank Report is edited by Matthew D. Hanaghan. Assistance in the preparation of this issue was provided by Lisa M. Jentzen. The information in this publication is not legal advice. For further information, contact:
Gene A. Blumenreich
gblumenreich@nutter.com
Kenneth F. Ehrlich
kehrlich@nutter.com
Michael K. Krebs
mkrebs@nutter.com
More Publications >

redpajama_set_name: RedPajamaCommonCrawl


text:
Q: pyspark check if value in the column adheres to a fixed format I am trying to check if column "phone_number" in my pyspark data frame is in a fixed format.

*

*format :
804-8048888
Request :

*

*How can I check if all values in the column are in the above same
format.

*As there is a "-" in the phone number i cant cast the column as int
and check is_valid()

A: Using a test dataframe like this one :
df.show()

+------------+
|phone_number|
+------------+
| 804-8048888|
|     8048888|
+------------+

with a regex, you can validate your format easily.
from pyspark.sql import functions as F

df.withColumn(
    "is_valid",
    F.when(
        F.regexp_extract(F.col("phone_number"), r"\d{3}-\d{7}", 0)
        == F.col("phone_number"),
        True,
    ).otherwise(False),
).show()

+------------+--------+
|phone_number|is_valid|
+------------+--------+
| 804-8048888|    true|
|     8048888|   false|
+------------+--------+

A: Here is another way.
df = spark.createDataFrame([('1', '804-8048888'), ('2', '1234567890')]).toDF('id', 'phone')

from pyspark.sql.functions import *

df.withColumn('validate', expr("phone rlike '^[0-9]{3}-[0-9]{7}$'")).show()

+---+-----------+--------+
| id|      phone|validate|
+---+-----------+--------+
|  1|804-8048888|    true|
|  2| 1234567890|   false|
+---+-----------+--------+

redpajama_set_name: RedPajamaStackExchange


Donald Trump went after everyone at the Republican presidential debate Saturday night, including the crowd.
Trump attacked Jeb Bush saying he was "so wrong" on foreign policy when the crowd erupted into loud boos.
"Just so you understand, you know who that is? That's Jeb's special interests and lobbyists talking," Trump said.
Even though moments later the crowd cheered him for saying America hasn't won in the Middle East he attacked them again when they booed him for taking a shot at former candidate Lindsey Graham.
"I only tell the truth lobbyists," he shot back at the crowd.
And he might have a point.
A January 19 article by the Washington Post explained that many lobbyists go from debate to debate like teenagers following a rock 'n' roll band.
Trump might be on to something.


ImpJob.Com is one of the leading websites to help individuals in finding the Jobs in Govt public sector, banks, Indian defense like Army, Navy, Air force, Railway, Police, teaching and many more.
We are putting best efforts to cover every recent opening Exam detail, Admit cards, Results in Sarkari Departments. We also add information in a user-friendly manner with the links of the resources if available for more details. To get details of directly in your inbox, subscribe below.
Our aim is to provide the latest news regarding the latest job openings in the government sector. Our mission to provide one place collection of the latest Sarkari Naukri / government jobs on time as soon as it published by employer and government sectors. All jobs opening collected from respective official recruitment website or portal.
Feel free to Contact Us for any suggestions, feedback etc.
Disclaimer : -The ImpJob.Com Website is not associated with any kind of Government Organization. Here we are only providing Govt Jobs information. We have collected a recruitment notification from various Online/Offline sources. We are advised to All the Readers that before apply once you visit & confirm notification on the official website of the relevant department. Here we are not responsible for any Inadvertent Error that may have crept in the information being published in this Website and for any loss to anybody or anything caused by any Shortcoming, Defect or Inaccuracy of the Information on this Website.


First appointment, had to keep calling to find out when technicians would arrive. Then when they did arrive 1 hour before the time frame ended they said they didn't have time to do the job. So... First appointment, had to keep calling to find out when technicians would arrive. Then when they did arrive 1 hour before the time frame ended they said they didn't have time to do the job. So took off work for nothing. Called office and spoke to Service manager and rescheduled appointment for 2 weeks later and was assured our address would be first on the list for 8-12 and techs would be there about 8-8:30. 9 am came and still no techs or call. I called, they said tech didn't answer phone and they would send email and call back. Of course no one called back. So again I called at 10 and was told the techs wer picking up equipment and would be there at 11. Tech called at 11:15 and said they would be there in 30 minutes. Techs came at noon and left at 3 pm. The install was for 2 outside, 2 inside cameras and 11 contacts. No instructions on how to use cameras nor instructions on phone. Techs only answered questions if we asked. All cameras and contacts were working when they left. However, next day 1 outside camera offline and receive auto call that system has a communication error. Called Ackerman and had to do a test with them. They informed us that the office received the alarm and all was fine. Camera still offline and same panel error. NOT A HAPPY CUSTOMER!


Key Features: Each sneaker's deployment of bold, typographic elements, and the harnessing of a hiking-inspired, high-traction tooling system.
Release Date: Reported as being March 2, 2018.
Editor's Notes: After previewing a pair of tie-dye NMD at the start of this month, and taking a look at the sneaker's high-price Chanel collaboration back in late October, it would appear that adidas's collaborative Pharrell Williams sneaker is about to expand even more.
Famed Instagram account @yeezymafia recently took to the platform to show off three new colorways (excluding the aforementioned tie-dye version). Featuring two, ultra-clean duo-tone iterations and an additional tie-dye option, the sneaker pack is expected to release on March 2, 2018. Be sure to stay tuned for further updates on where to buy them. In the meantime, take a look at the new silhouettes below.
In other news, Pharrell was also spotted wearing a brand new Hu trail sneaker whilst out in Shanghai. What are your thoughts on it?
On a similar note, our favorites from the Hu hiking collection can now be shopped here.


Compare Wood vs Composite Deck Cost. A cedar wood deck will cost $30 or more per square foot. A composite deck will cost $30 to $45 per square foot installed.
Estimate the cost of building a high-performance composite deck with help it Cost to Build a Deck? costs for a deck run about $12.12 per square foot.
composite deck cost per square foot weight of wpc per square feet ,Composite Decking Boards Price. weight of wpc per square feet. Low-Maintenance Composite Fence.


text:
Q: Set default value to zero when count is null  select rd.description||'('||rs.department_code||')' DepartmentName,
              rd.code DepartmentCode,
              rs.description||'('||emv.section_code||')' SectionName,
              emv.section_code SectionCode,
              rsg.description||'('||emv.staff_group_code||')' StaffGroupName,
              emv.staff_group_code StaffGroupCode,
              e.staff_id STAFFID,
              e.surname || ', ' || e.given_name FullName,
              hp.description Position,
              het.description EmploymentType,
              to_char(e.Join_Date, 'dd/MM/yyyy') JoinDate,
              e.employee_no EmployeeNo,
              edt.shift Type,
              to_char(edt.timesheet_date,'Mon') Mth,
              to_char(to_Date(edt.timesheet_date),'mm') MthNum,
              nvl(count(*), 0) Days
        FROM employee e,
            employee_daily_timesheet edt,
            employee_assignment_vw emv,
            ref_section rs,
            ref_department rd,
            hris_position hp,
            hris_employment_type het,
            ref_staffgroup rsg
        WHERE e.employee_no = emv.employee_no
        AND edt.assignment_no = emv.assignment_no
        AND to_char(timesheet_date,'yyyy')=TO_CHAR(TO_DATE('01/31/2015','MM/dd/yyyy'),'yyyy')
        AND  emv.section_code = rs.code
        AND rs.department_code = rd.code
        AND e.position_code = hp.code(+)
        AND e.employment_type_code = het.code(+)
        AND emv.staff_group_code = rsg.code
        AND edt.shift='OFF'
        GROUP BY  rd.description||'('||rs.department_code||')',
                  rd.code,
                  rs.description||'('||emv.section_code||')',
                  emv.section_code,
                  rsg.description||'('||emv.staff_group_code||')',
                  emv.staff_group_code,
                  e.staff_id,
                  e.surname || ', ' || e.given_name,
                  hp.description ,
                  het.description ,
                  to_char(e.Join_Date, 'dd/MM/yyyy') ,
                  e.employee_no,
                  edt.shift,
                  to_char(edt.timesheet_date,'Mon'),
                   to_char(to_Date(edt.timesheet_date),'mm')

Im trying to count the days where the Shift is equal to 'OFF', but it doesnt display records when count is zero or when a month doesnt have an 'OFF' shift. How can I set the value to zero when count is null or when the month doesnt have an 'OFF' shift.

A: I can't understand your query without knowing the data model (and why in 2016 are you using the ancient (+) syntax for outer joins?!) but the general principle is simple.  A group by query only returns rows for groups that have some data.  For example:
select deptno, count(*)
  from emp
 group by deptno;

This will only returns rows for departments that have at least one employee.  But you say "I want to see all departments, even if they have no employees".  Well, the EMP table doesn't contain all the departments, but table DEPT does.  So we can use an outer join (I'll use ANSI syntax) like this:
select d.deptno, count(e.empno)
  from dept d
       left join emp e on e.deptno = d.deptno
 group by d.deptno;

Note that the driving table has changed from EMP, which doesn't have all the deptno values, to DEPT which does.  If there are no employees for deptno=60 then the query will still return a row for that DEPT, thanks to the outer join.
Note also that I have used count(e.empno) not count(*), because I am trying to count employees in the department, not rows returned by the query prior to grouping.  If I used count(*) then deptno=60 would return a count of 1 (because there is 1 DEPT row with deptno=60, outer joined to 0 EMP rows, resulting in one result).
If you understand this principle and your data model then you should be able to write an analogous query for your case.


New Delhi, India, July 18, 2014: Emerson Network Power, a business of Emerson, and a global leader in maximizing availability, capacity and efficiency of critical infrastructure, today announced that it has appointed Computer Source Ltd. as a distributor for its range of uninterruptible power supply (UPS) and data center infrastructure management (DCIM) products in Bangladesh. The association is part of the company's plans to partner with the growing reseller community in Bangladesh and to expand its presence in the burgeoning IT landscape of the country.
The business association aims to cater to the growing need of UPS and DCIM solutions among enterprises and small and medium businesses in Bangladesh.


A small, firm swelling under the skin may develop at the site of a recent vaccination. It should start to disappear within a couple weeks. If it persists more than three weeks, or seems to be getting larger, you should contact your veterinarian.


text:
package com.github.programmerr47.photostealer.util;

import android.animation.Animator;
import android.animation.AnimatorSet;
import android.animation.ValueAnimator;
import android.graphics.drawable.ColorDrawable;
import android.graphics.drawable.TransitionDrawable;
import android.support.v4.view.ViewCompat;
import android.view.View;
import android.view.ViewGroup;
import android.view.animation.AccelerateDecelerateInterpolator;
import android.view.animation.AccelerateInterpolator;
import android.view.animation.DecelerateInterpolator;

import com.github.programmerr47.photostealer.R;
import com.github.programmerr47.photostealer.representation.PhotoStealerApplication;

/**
 * @author Michael Spitsin
 * @since 2015-08-06
 */
public class AnimationUtil {

    public static Animator showAndHideSuccessNetworkState(final View container, float offsetHeight) {
        int toolbarHeight = (int) offsetHeight;
        int containerHeight = container.getMeasuredHeight();
        if (containerHeight == 0) {
            containerHeight = (int) PhotoStealerApplication.getAppContext().getResources().getDimension(R.dimen.connection_state_height);
        }

        container.setY(toolbarHeight - containerHeight);

        final ValueAnimator yAnim = ValueAnimator.ofInt(toolbarHeight - containerHeight, toolbarHeight);
        yAnim.addUpdateListener(new ValueAnimator.AnimatorUpdateListener() {
            @Override
            public void onAnimationUpdate(ValueAnimator animation) {
                int val = (Integer) animation.getAnimatedValue();
                container.setY(val);
            }
        });

        final ValueAnimator yAnimReverse = ValueAnimator.ofInt(toolbarHeight, toolbarHeight - containerHeight);
        yAnimReverse.addUpdateListener(new ValueAnimator.AnimatorUpdateListener() {
            @Override
            public void onAnimationUpdate(ValueAnimator animation) {
                int val = (Integer) animation.getAnimatedValue();
                container.setY(val);
            }
        });

        yAnim.setDuration(Constants.DEFAULT_ANIMATION_DURATION);
        yAnimReverse.setDuration(Constants.DEFAULT_ANIMATION_DURATION);

        yAnim.setInterpolator(new DecelerateInterpolator());
        yAnimReverse.setInterpolator(new AccelerateInterpolator());

        AnimatorSet animation = new AnimatorSet();
        animation
                .play(yAnimReverse)
                .after(3000)
                .after(yAnim);
        animation.start();
        return animation;
    }

    public static Animator showNetworkState(final View container, float offsetHeight) {
        int toolbarHeight = (int) offsetHeight;
        int containerHeight = container.getMeasuredHeight();
        container.setY(toolbarHeight - containerHeight);

        final ValueAnimator yAnim = ValueAnimator.ofInt(toolbarHeight - containerHeight, toolbarHeight);
        yAnim.addUpdateListener(new ValueAnimator.AnimatorUpdateListener() {
            @Override
            public void onAnimationUpdate(ValueAnimator animation) {
                int val = (Integer) animation.getAnimatedValue();
                container.setY(val);
            }
        });

        yAnim.setDuration(Constants.DEFAULT_ANIMATION_DURATION);

        AnimatorSet animation = new AnimatorSet();
        animation.setInterpolator(new DecelerateInterpolator());

        animation.play(yAnim);
        animation.start();
        return animation;
    }

    public static Animator showProgress(final View progressView, final View hidingView, final View container) {
        final ValueAnimator alphaAnim = ValueAnimator.ofFloat(0, 1);
        final int progressWidth = progressView.getMeasuredWidth();
        final int containerWidth = container.getMeasuredWidth();
        alphaAnim.addUpdateListener(new ValueAnimator.AnimatorUpdateListener() {
            @Override
            public void onAnimationUpdate(ValueAnimator animation) {
                float val = (Float) animation.getAnimatedValue();
                progressView.setAlpha(val);
                hidingView.setAlpha(1 - val);

                if (AndroidUtils.hasLollipop()) {
                    container.setElevation(AndroidUtils.dpToPx((int)(val * 2)));
                } else {
                    ViewCompat.setElevation(container, AndroidUtils.dpToPx((int)(val * 2)));
                }

                ViewGroup.LayoutParams params = container.getLayoutParams();
                params.width = (int)(containerWidth * (1 - val) + progressWidth * val);
                container.setLayoutParams(params);
            }
        });
        alphaAnim.setDuration(Constants.DEFAULT_ANIMATION_DURATION);

        AnimatorSet animation = new AnimatorSet();
        animation.setInterpolator(new AccelerateDecelerateInterpolator());
        animation.addListener(new Animator.AnimatorListener() {
            @Override
            public void onAnimationStart(Animator animation) {
                progressView.setVisibility(View.VISIBLE);
                hidingView.setVisibility(View.VISIBLE);
            }

            @Override
            public void onAnimationEnd(Animator animation) {
                hidingView.setVisibility(View.INVISIBLE);
            }

            @Override
            public void onAnimationCancel(Animator animation) {
                hidingView.setVisibility(View.VISIBLE);
                progressView.setVisibility(View.INVISIBLE);
            }

            @Override
            public void onAnimationRepeat(Animator animation) {
            }
        });

        animation.play(alphaAnim);
        animation.start();
        return animation;
    }

    public static Animator hideProgress(final View progressView, final View revealView, final View container, final int originalContainerWidth) {
        ValueAnimator alphaAnim = ValueAnimator.ofFloat(1, 0);
        final int progressWidth = progressView.getMeasuredWidth();
        alphaAnim.addUpdateListener(new ValueAnimator.AnimatorUpdateListener() {
            @Override
            public void onAnimationUpdate(ValueAnimator animation) {
                float val = (Float) animation.getAnimatedValue();
                progressView.setAlpha(val);
                revealView.setAlpha(1 - val);

                if (AndroidUtils.hasLollipop()) {
                    container.setElevation(AndroidUtils.dpToPx((int)(val * 2)));
                } else {
                    ViewCompat.setElevation(container, AndroidUtils.dpToPx((int)(val * 2)));
                }

                ViewGroup.LayoutParams params = container.getLayoutParams();
                params.width = (int)(originalContainerWidth * (1 - val) + progressWidth * val);
                container.setLayoutParams(params);
            }
        });
        alphaAnim.setDuration(Constants.DEFAULT_ANIMATION_DURATION);

        AnimatorSet animation = new AnimatorSet();
        animation.setInterpolator(new AccelerateDecelerateInterpolator());
        animation.addListener(new Animator.AnimatorListener() {
            @Override
            public void onAnimationStart(Animator animation) {
                progressView.setVisibility(View.VISIBLE);
                revealView.setVisibility(View.VISIBLE);
            }

            @Override
            public void onAnimationEnd(Animator animation) {
                progressView.setVisibility(View.INVISIBLE);
            }

            @Override
            public void onAnimationCancel(Animator animation) {
                revealView.setVisibility(View.INVISIBLE);
                progressView.setVisibility(View.VISIBLE);
            }

            @Override
            public void onAnimationRepeat(Animator animation) {
            }
        });

        animation.play(alphaAnim);
        animation.start();
        return animation;
    }

    public static void changeColorOfView(View view, int startColor, int endColor) {
        ColorDrawable[] color = {new ColorDrawable(startColor), new ColorDrawable(endColor)};
        TransitionDrawable trans = new TransitionDrawable(color);
        view.setBackgroundDrawable(trans);
        trans.startTransition(Constants.DEFAULT_ANIMATION_DURATION);
    }
}

redpajama_set_name: RedPajamaGithub