text:
\section{Introduction}
\subfile{sections/intro}

\section{Background}
\subfile{sections/background}

\section{Methods}
\subfile{sections/rev_res_unit}

\section{Related Work}
\subfile{sections/related_work}

\section{Experiments}
\subfile{sections/evaluation}

\section{Conclusion and Future Work}
\subfile{sections/future_work}

\bibliographystyle{abbrvnat}

\subsection{Experiment details}
For our CIFAR-10/100 experiments, we fixed the mini-batch size to be 100. The learning rate was initialized to 0.1 and decayed by a
factor of 10 at 40K and 60K training steps, training for a total of 80K steps. The
weight decay constant was set to $2 \times 10^{-4}$ and the momentum was set to 0.9. We subtracted the mean image, and augmented the dataset with random cropping and random horizontal
flipping.

For our ImageNet experiments, we fixed the mini-batch size to be 256, split across 4
Titan X GPUs with data parallelism \citep{simonyan2014very}. We employed synchronous SGD \citep{chen2016revisiting} with momentum of 0.9. The model was trained for 600K
steps, with factor-of-10 learning rate decays scheduled at 160K, 320K, and 480K
steps. Weight decay was set to $1 \times 10^{-4}$.  We applied standard input
preprocessing and data augmentation used in training Inception networks
\cite{szegedy2015inception}: pixel intensity rescaled to within [0, 1], random
cropping of size $224 \times 224$ around object bounding boxes, random scaling, random horizontal
flipping, and color distortion, all of which are available in TensorFlow. For the original ResNet-101, We were
unable to fit a mini-batch size of 256 on 4 GPUs,
so we instead averaged the gradients from two serial runs with mini-batch size
128 (32 per GPU).  For the RevNet, we were able to fit a mini-batch size of 256 on 4 GPUs (i.e.~64 per GPU).

\subsection{Memory savings}
Fully realizing the theoretical gains of RevNets can be a non-trivial task and
require precise low-level GPU memory management. We experimented with two different implementations within TensorFlow:

With the first, we were able to reach
reasonable spatial gains using ``Tensor Handles'' provided by TensorFlow, which
preserve the activations of graph nodes between calls to \texttt{session.run}.
Multiple \texttt{session.run} calls ensures that TensorFlow frees up
activations that will not be referenced later. We segment our computation graph
into separate sections and save the bordering activations and gradients into
the persistent Tensor Handles. During the forward pass of the backpropagation
algorithm, each section of the graph is executed sequentially with the input
tensors being reloaded from the previous section and the output tensors being
saved for use in the subsequent section.  We empirically verified the memory
gain by fitting at least twice the number of examples while training ImageNet.
Each GPU can now fit a mini-batch size of 128 images, compared the original
ResNet, which can only fit a mini-batch size of 32. The graph splitting trick
brings only a small computational overhead (around 10\%).

The second and most significant spatial gains were made by implementing each residual stack as a \texttt{tf.while\_loop} with the \texttt{back\_prop} parameter set to \texttt{False}. This setting ensures that activations of each layer in the residual stack (aside from the last) are discarded from memory immediately after their utility expires. We use the \texttt{tf.while\_loop}s for both the forward and backward passes of the layers, ensuring both efficiently discard activations. Using this implementation we were able to train a 600-layer RevNet on the ImageNet image classification challenge on a single GPU; despite being prohibitively slow to train this demonstrates the potential for massive savings in spatial costs of training extremely deep networks.

\newpage
\subsection{Sample implementation}
\begin{python}
import tensorflow as tf

def forward(inputs, weights, weights_per_layer):
    """Perform forward execution of a reversible stack.

    Args:
      inputs: A pair of Tensors to be propagated through
        the stack.
      weights: A TensorArray containing
        `weights_per_layer*layer_count` elements.
      weights_per_layer: An integer.
    Returns:
      outputs: A pair of Tensors with the same shape as
        `inputs`.
    """
    def loop_body(layer_index, inputs, weights):
        layer_weights = weights.gather(
            tf.range(i*weights_per_layer,
                     (i+1)*weights_per_layer))
        outputs = execute_layer(inputs, layer_weights)
        return (layer_index+1, outputs, weights)

    _, outputs, _ = tf.while_loop(
        lambda i, *_: i < layer_count,
        loop_body,
        [tf.constant(0), inputs, weights],
        parallel_iterations=1,
        back_prop=False)
    return outputs
\end{python}
\newpage
\begin{python}
def backward(
  outputs, output_grads, weights, weights_per_layer):
    """Perform backpropagation of a reversible stack.

    Args:
      outputs: A pair of Tensors, the outputs from a
        reversible stack.
      output_grads: A pair of Tensors, the gradients
        w.r.t. the `outputs`.
      weights: A TensorArray containing
        `weights_per_layer*layer_count`
      weights_grads: A TensorArray containing
        `weights_per_layer*layer_count` elements.
      weights_per_layer: An integer.
    Returns:
      outputs: A pair of Tensors with the same shape as
        `inputs`.
    """
    def loop_body(layer_index,
                  outputs,
                  output_grads,
                  weights,
                  weights_grads):
        layer_weights = weights.gather(tf.range(
          i*weights_per_layer, (i+1)*weights_per_layer))
        (inputs, input_grads,
         layer_weights_grads) = backprop_layer(
            outputs, output_grads, layer_weights)
        weights_grads = weights_grads.scatter(
          tf.range(i*weights_per_layer,
                   (i+1)*weights_per_layer),
          layer_weights_grads)
        return (layer_index-1,
                inputs,
                input_grads,
                weights,
                weights_grads)

    (_, inputs, input_grads, _,
     weights_grads) = tf.while_loop(
        lambda i, *_: i >= 0,
        loop_body,
        [tf.constant(layer_count - 1), outputs,
         output_grads, weights, weights_grads],
        parallel_iterations=1,
        back_prop=False)
    return inputs, input_grads, weights_grads
\end{python}
\newpage
\begin{python}
def backprop_layer(
  outputs, output_grads, layer_weights):
    """Perform backpropagation of a reversible layer.

    Args:
      outputs: A pair of Tensors, the outputs from
        this reversible layer.
      output_grads: A pair of Tensors, the gradients
        w.r.t. the `outputs`.
      weights: A pair of Tensors, the first holding
        weights for F and the second holding weights
        for G.
    Returns:
      outputs: A  Tensor with the same shape as
      `inputs`.
    """
    # First, reverse the layer to retrieve inputs
    y1, y2 = output[0], output[1]
    F_weights, G_weights = weights[0], weights[1]

    # lines 2-4 of Algorithm 1
    z1_stop = tf.stop_gradient(y1)

    G_z1 = G(z1_stop, G_weights)
    x2 = y2 - G_z1
    x2_stop = tf.stop_gradient(x2)

    F_x2 = F(x2_stop, F_weights)
    x1 = y1 - F_x2
    x1_stop = tf.stop_gradient(x1)

    # Second, compute gradients
    y1_grad = output_grads[0]
    y2_grad = output_grads[1]

    z1 = x1_stop + F_x2
    y2 = x2_stop + G_z1
    y1 = z1

    # lines 5-9 of Algorithm 1
    z1_grad = tf.gradients(
      y2, z1_stop, y2_grad) + y1_grad
    x2_grad = tf.gradients(
      y1, x2_stop, z1_grad) + y2_grad
    x1_grad = z1_grad

    F_grads = tf.gradients(y2, G_weights, y2_grad)
    G_grads = tf.gradients(y1, F_weights, z1_grad)

    # Finally, construct return values
    inputs = (x1_stop, x2_stop)
    input_grads = (x1_grad, x2_grad)
    weight_grads = (F_grads, G_grads)

    return inputs, input_grads, weight_grads
\end{python}

\subsection{Backpropagation}
\label{sec:backprop}

Backpropagation \cite{rumelhart1986learning} is a classic algorithm for
computing the gradient of a cost function with respect to the parameters of a
neural network. It is used in nearly all neural network algorithms, and is
now taken for granted in light of neural network frameworks which implement
automatic differentiation \cite{abadi2016tensorflow,al2016theano}. Because
achieving the memory savings of our method requires manual implementation of
part of the backprop computations, we briefly review the algorithm.

We treat backprop as an instance of reverse mode automatic differentiation
\citep{autodiff1981}. Let $v_1, \ldots, v_K$ denote a topological
ordering of the nodes in the network's computation graph $\mathcal{G}$,
where $v_K$ denotes the cost function $\mathcal{C}$. Each node is defined as a function
$\nodeFunctionK{i}$ of its parents in $\mathcal{G}$. Backprop
computes the total derivative $\mathrm{d} \mathcal{C} / \mathrm{d} v_i$ for each node
in the computation graph. This total derivative defines the the effect on
$\mathcal{C}$ of an infinitesimal change to $v_i$, taking into account the indirect
effects through the descendants of $v_k$ in the computation graph. Note that
the total derivative is distinct from the partial derivative $\partial f /
\partial x_i$ of a function $f$ with respect to one of its arguments $x_i$,
which does not take into account the effect of changes to $x_i$ on the other
arguments. To avoid using a small typographical difference to represent a
significant conceptual difference, we will denote total derivatives using
$\costDeriv{v_i} = \mathrm{d} \mathcal{C} / \mathrm{d} v_i$.

Backprop iterates over the nodes in the computation graph in reverse
topological order. For each node $v_i$, it computes the total derivative
$\costDeriv{v_i}$ using the following rule:
\begin{equation}
\costDeriv{v_i} = \sum_{j \in \mathrm{Child}(i)} \left(
  \frac{\partial f_{j}}{\partial v_i} \right)^\top
  \costDeriv{v_j}, \label{eqn:backprop}
\end{equation}
where $\mathrm{Child}(i)$ denotes the children of node $v_i$ in
$\mathcal{G}$ and $\partial f_{j} / \partial v_{i}$
denotes the Jacobian matrix.

\subsection{Deep Residual Networks}

\begin{figure}[t]
  \centering
  \hspace{2cm}
  \includegraphics[scale=0.50]{figs/graphics-02}
  \hspace{1cm}
  \includegraphics[scale=0.50]{figs/graphics-10}
  \caption{(left) A traditional residual \rname{} as in Equation
  \ref{residualeqn}. (right-top) A basic residual function. (right-bottom) A
  bottleneck residual function.}
  \label{resdiag}
\end{figure}

One of the main difficulties in training very deep networks is the problem of
exploding and vanishing gradients, first observed in the context of recurrent
neural networks \cite{bengio1994learning}. In particular, because a deep
network is a composition of many nonlinear functions, the dependencies across
distant layers can be highly complex, making the gradient computations
unstable. Highway networks \cite{srivastava2015highway} circumvented this
problem by introducing skip connections.  Similarly, deep residual networks
(ResNets) \cite{he2016deep} use a functional form which allows information to
pass directly through the network, thereby keeping the computations stable.
ResNets currently represent the state-of-the-art in object recognition
\citep{he2016deep}, semantic segmentation \citep{wu2016wider} and image generation \citep{pixelcnn2016}. Outside of vision, residuals have displayed impressive performance in audio generation \citep{van2016wavenet} and neural machine translation \citep{bytenet2016},

ResNets are built out of modules called residual blocks, which have the following form:

\begin{equation} \label{residualeqn}
    y = x + \mathcal{F}(x),
\end{equation}

where \(\mathcal{F}\), a function called the residual function, is typically a shallow neural net.
ResNets are robust to exploding and vanishing gradients because each residual block is able to pass signals directly through, allowing the signals to be propagated faithfully across many layers.
As displayed in Figure \ref{resdiag},
residual functions for image recognition generally consist of stacked batch
normalization ("BN") \citep{ioffe2015batch}, rectified linear activation
("ReLU") \citep{nair2010rectified} and convolution layers (with filters of
shape three "C3" and one "C1").

As in \citet{he2016deep}, we use two residual \rname{} architectures: the basic residual function (Figure \ref{resdiag} right-top) and the bottleneck residual function (Figure \ref{resdiag} right-bottom). The bottleneck residual consists of three convolutions, the first is a point-wise convolution which reduces the dimensionality of the feature dimension, the second is a standard convolution with filter size 3, and the final point-wise convolution projects into the desired output feature depth.

\begin{align}\label{resunits}
\begin{split}
a(x) &= ReLU(BN(x)))\\
c_k(x) &= Conv_{k \times k}(a(x))\\
\\
Basic(x) &= c_3(c_3(x))\\
Bottleneck(x) &= c_1(c_3(c_1(x)))
\end{split}
\end{align}

\subsection{Reversible Architectures}

Various reversible neural net architectures have been proposed, though for motivations distinct from our own.
\citet{maclaurin2015gradient} made use of the reversible nature of
stochastic gradient descent to tune hyperparameters via gradient descent. Our proposed method is inspired by nonlinear independent components estimation (NICE) \citep{dinh2014nice, dinh2016density}, an approach to unsupervised
generative modeling. NICE is based on learning a non-linear bijective transformation between the data space and
a latent space.  The architecture is composed of a series of blocks defined as follows, where $x_1$ and $x_2$ are a partition of the units in each layer:
\begin{equation} \label{nice}
\begin{split}
    y_1 &= x_1\\
    y_2 &= x_2 + \mathcal{F}(x_1)
\end{split}
\end{equation}
Because the model is invertible and its Jacobian has unit determinant, the log-likelihood and its gradients can be tractably computed. This architecture imposes some constraints on the functions the network can represent; for instance, it can only represent volume-preserving mappings. Follow-up work by
\citet{dinh2016density} addressed this limitation by introducing a
new reversible transformation:
\begin{equation} \label{rnvp}
\begin{split}
    y_1 &= x_1\\
    y_2 &= x_2 \odot \exp(\mathcal{F}(x_1)) + \mathcal{G}(x_1).
\end{split}
\end{equation}
Here, \(\odot\) represents the Hadamard or element-wise product. This
transformation has a non-unit Jacobian determinant due to multiplication by
\(\exp\left(\mathcal{F}(x_1)\right)\).

\subsection{Implementation}

\begin{table}[t]
  \centering
  \caption{Architectural details. 'Bottleneck' indicates whether the residual unit type used was the \(Bottleneck\) or \(Basic\) variant (see Equation \ref{resunits}). 'Units' indicates the number of residual units in each group. 'Channels' indicates the number of filters used in each unit in each group. 'Params' indicates the number of parameters, in millions, each network uses.}
  \label{tab:arch_detail}
  \begin{tabular}{c|ccccc}
  \toprule
  Dataset        & Version       & Bottleneck & Units    & Channels        & Params (M)  \\
  \midrule
  CIFAR-10 (100) & ResNet-32     & No         & 5-5-5    & 16-16-32-64     & 0.46 (0.47) \\
  CIFAR-10 (100) & RevNet-38  & No         & 3-3-3    & 32-32-64-112    & 0.46 (0.48) \\
  \midrule
  CIFAR-10 (100) & ResNet-110    & No         & 18-18-18 & 16-16-32-64     & 1.73 (1.73) \\
  CIFAR-10 (100) & RevNet-110 & No         & 9-9-9    & 32-32-64-128    & 1.73 (1.74) \\
  \midrule
  CIFAR-10 (100) & ResNet-164    & Yes        & 18-18-18 & 16-16-32-64     & 1.70 (1.73) \\
  CIFAR-10 (100) & RevNet-164 & Yes        & 9-9-9    & 32-32-64-128    & 1.75 (1.79) \\
  \midrule
  ImageNet       & ResNet-101    & Yes        & 3-4-23-3 & 64-128-256-512  & 44.5        \\
  ImageNet       & RevNet-104 & Yes        & 2-2-11-2 & 128-256-512-832 & 45.2        \\
  \bottomrule
  \end{tabular}
\end{table}

We implemented the RevNets using the TensorFlow library
\citep{abadi2016tensorflow}.  We manually make calls to TensorFlow's automatic
differentiation method (i.e.  \texttt{tf.gradients}) to construct the
backward-pass computation graph without referencing activations computed in the
forward pass. While building the backward graph, we reconstruct the input
activations ($\hat{x}_1, \hat{x}_2$) for each \rname{} (Equation
\ref{reversibleblock}); Second, we apply \texttt{tf.stop\_gradient} on the
reconstructed inputs to prevent auto-diff from traversing into the reconstructions'
computation graph, then call the forward functions again to compute
($\hat{y}_1, \hat{y}_2$) (Equation \ref{reversibleblock}).  Lastly, we use
auto-diff to traverse from ($\hat{y}_1, \hat{y}_2$) to ($\hat{x}_1, \hat{x}_2$)
and the parameters ($w_{\cal F}, w_{\cal G}$). This implementation leverages
the convenience of the auto-diff functionality to avoid manually deriving
gradients; however the computational cost becomes $5N$, compared with $4N$ for Algorithm \ref{revgradd}, and  $3N$ for ordinary backpropagation (see Section \ref{gradientcomp}). The full theoretical efficiency can be realized by reusing
the \(\cal F\) and \(\cal G\) graphs' activations that were computed in the
reconstruction steps (lines 3 and 4 of Algorithm \ref{revgradd}).

\begin{table}[t]
  \centering
  \caption{Classification error on CIFAR}\label{tab:cifar}
  \begin{tabular}{cccccccc}
    \toprule
    \multirow{2}{*}{\vspace{-2mm}Architecture} &
    \multicolumn{2}{c}{CIFAR-10 \citep{cifar10-100}} &&
    \multicolumn{2}{c}{CIFAR-100 \citep{cifar10-100}} \\
    \cmidrule{2-3}\cmidrule{5-6}
     & ResNet & RevNet && ResNet & RevNet \\
    \midrule
    32 (38) & \bf{7.14\%} & 7.24\%      && 29.95\%      & \bf{28.96\%} \\
    110     & \bf{5.74\%} & 5.76\%      && 26.44\%      & \bf{25.40\%}  \\
    164     & 5.24\%      & \bf{5.17\%} && \bf{23.37\%} & 23.69\%       \\
  \bottomrule
  \end{tabular}\\[4mm]
\end{table}

\begin{table}[t]
  \centering
  \caption{Top-1 classification error on ImageNet (single crop)}\label{tab:imagenet}
  \begin{tabular}{cc}
    \toprule
    ResNet-101 & RevNet-104 \\
    \midrule
    \bf{23.01\%} & 23.10\% \\
  \bottomrule
  \end{tabular}\\[4mm]
\end{table}

\begin{figure}[t]
\centering
\small
\includegraphics[width=0.45\textwidth]{figs/imagenet_ce.pdf}
\includegraphics[width=0.45\textwidth]{figs/imagenet_acc.pdf}
\caption{Training curves for ResNet-101 vs.~RevNet-104 on ImageNet, with both
  networks having approximately the same depth and number of free parameters.
  {\bf Left:} training cross entropy; {\bf Right:} classification error, where
  dotted lines indicate training, and solid lines validation.}
\label{fig:imgnet}
\end{figure}

\begin{figure}[t]
\centering
\small
\includegraphics[width=0.32\textwidth]{figs/cifar_graderr.pdf}
\includegraphics[width=0.32\textwidth]{figs/cifar_ce.pdf}
\includegraphics[width=0.32\textwidth]{figs/cifar_acc.pdf}
\caption{{\bf Left:} angle (degrees) between
  the gradient computed using stored and reconstructed activations throughout
  training. While the angle grows during training, it remains small in
  magnitude. We measured 4 more epochs after regular training length and did not
  observe any instability.
  {\bf Middle:} training cross entropy;
  {\bf Right:} classification
  error, where dotted lines indicate training, and solid lines validation; No
  meaningful difference in training efficiency or final performance was
  observed between stored and reconstructed activations.}
\label{fig:auto_vs_manual}
\end{figure}

\subsection{RevNet performance}
\label{rev_net_perf}
Our ResNet implementation roughly matches the previously reported
classification error rates \cite{he2016deep}. As shown in Table
\ref{tab:cifar}, our RevNets roughly matched the error rates of traditional
ResNets (of roughly equal computational depth and number of parameters) on
CIFAR-10 \& 100 as well as ImageNet (Table \ref{tab:imagenet}). In no condition
did the RevNet underperform the ResNet by more than 0.5\%, and in some cases,
RevNets achieved slightly better performance.  Furthermore, Figure
\ref{fig:imgnet} compares ImageNet training curves of the ResNet and RevNet
architectures; reversibility did not lead to any noticeable per-iteration
slowdown in training. (As discussed above, each RevNet update is about
1.5-2$\times$ more expensive, depending on the implementation.) We found it
surprising that the performance matched so closely, because reversibility would
appear to be a significant constraint on the architecture, and one might expect
large memory savings to come at the expense of classification error.

{\bf Impact of numerical error.}
As described in Section \ref{gradientcomp}, reconstructing the activations over
many layers causes numerical errors to accumulate. In order to measure the
magnitude of this effect, we computed the angle between the gradients computed
using stored and reconstructed activations over the course of training.
Figure~\ref{fig:auto_vs_manual} shows how this angle evolved over the course of
training for a CIFAR-10 RevNet; while the angle increased during training, it
remained small in magnitude. Figure~\ref{fig:auto_vs_manual} also shows
training curves for CIFAR-10 networks trained using both methods of computing
gradients. Despite the numerical error from reconstructing activations, both
methods performed almost indistinguishably in terms of the training efficiency
and the final performance.

\subsection{Reversible Residual Networks}

RevNets are composed of a series of reversible blocks, which we now define. We must partition the units in each layer into two groups, denoted $x_1$ and $x_2$; for the remainder of the paper, we assume this is done by partitioning the channels, since we found this to work the best in our experiments.\footnote{The possibilities we explored included columns, checkerboard, rows and
channels, as done by \cite{dinh2016density}. We found that performance was consistently superior using the
channel-wise partitioning scheme and comparable across the remaining options. We note that channel-wise partitioning has also been explored in the context of multi-GPU
training via 'grouped' convolutions \citep{krizhevsky2012imagenet}, and more
recently, convolutional neural networks have seen significant success by way of
'separable' convolutions \citep{sifre2014rigid, chollet2016xception}. }
Each reversible block takes inputs $(x_1, x_2)$ and produces outputs $(y_1, y_2)$ according to the following additive coupling rules -- inspired by NICE's \citep{dinh2014nice}
transformation in Equation \ref{nice} -- and residual functions \(\mathcal{F}\) and
\(\mathcal{G}\) analogous to those in standard ResNets:
\begin{align}
y_1 &= x_1 + \mathcal{F}(x_2) \nonumber \\
y_2 &= x_2 + \mathcal{G}(y_1)
\end{align}
Each layer's activations can be reconstructed from the next layer's activations as follows:
\begin{align}
x_2 &= y_2 - \mathcal{G}(y_1) \nonumber \\
x_1 &= y_1 - \mathcal{F}(x_2)
\end{align}

Note that unlike residual blocks, reversible blocks must have a stride of 1 because otherwise the layer discards information, and therefore cannot be reversible. Standard ResNet architectures typically have a handful of layers with a larger stride. If we define a RevNet architecture analogously, the activations must be stored explicitly for all non-reversible layers.

\subsection{Backpropagation Without Storing Activations}\label{gradientcomp}

To derive the backprop procedure, it is helpful to
rewrite the forward (left) and reverse (right) computations in the following way:
\begin{alignat}{2} \label{reversibleblock}
    z_1 &= x_1 + \mathcal{F}(x_2) \qquad&\qquad z_1 &= y_1 \nonumber\\
    y_2 &= x_2 + \mathcal{G}(z_1) \qquad&\qquad x_2 &= y_2 - \mathcal{G}(z_1)\\
    y_1 &= z_1 \qquad&\qquad x_1 &= z_1 - \mathcal{F}(x_2) \nonumber
\end{alignat}
Even though $z_1 = y_1$, the two variables represent distinct nodes of the computation
graph, so the total derivatives $\gradd{z_1}$ and $\gradd{y_1}$ are
different. In particular, $\gradd{z_1}$ includes the indirect effect
through $y_2$, while $\gradd{y_1}$ does not. This splitting lets us
implement the forward and backward passes for reversible blocks in a modular
fashion.
In the backwards pass, we are given the activations $(y_1, y_2)$ and their total
derivatives $(\gradd{y_1}, \gradd{y_2})$ and wish to compute the
inputs $(x_1, x_2)$, their total derivatives $(\gradd{x_1},
\costDeriv{x_2})$, and the total derivatives for any parameters associated with ${\cal F}$ and ${\cal G}$. (See Section~\ref{sec:backprop} for our backprop notation.)
We do this by combining the reconstruction formulas (Eqn.~\ref{reversibleblock})
with the backprop rule (Eqn.~\ref{eqn:backprop}). The resulting algorithm is
given as Algorithm \ref{revgradd}.\footnote{We assume for notational clarity
that the residual functions do not share parameters, but Algorithm
\ref{revgradd} can be trivially extended to a network with weight sharing, such
as a recurrent neural net.}

By applying Algorithm \ref{revgradd} repeatedly, one can perform backprop on a
sequence of reversible blocks if one is given simply the activations and their
derivatives for the top layer in the sequence. In general, a practical
architecture would likely also include non-reversible layers, such as
subsampling layers; the inputs to these layers would need to be stored explicitly
during backprop. However, a typical ResNet architecture involves long sequences
of residual blocks and only a handful of subsampling layers; if we mirror the
architecture of a ResNet, there would be only a handful of non-reversible
layers, and the number would not grow with the depth of the network. In this
case, the storage cost of the activations would be small, and independent of
the depth of the network.

{\bf Computational overhead.} In general, for a network with $N$
connections, the forward and backward passes of backprop require approximately
$N$ and $2N$ add-multiply operations,
respectively. For a RevNet, the residual functions each must be recomputed
during the backward pass. Therefore, the number of operations required for
reversible backprop is approximately $4 N$, or roughly $33\%$
more than ordinary backprop. (This is the same as the overhead
introduced by checkpointing \cite{martens2012training}.) In practice, we have found the forward and backward passes to be about equally expensive on GPU architectures; if this is the case, then the computational overhead of RevNets is closer to $50\%$.

\begin{algorithm}
\caption{Reversible Residual Block Backprop}\label{revgradd}
\begin{algorithmic}[1]
\Function{BlockReverse}{$(y_1, y_2), (\gradd{y}_1,\gradd{y}_2)$}
   \State $z_1\gets y_1$
   \State $x_2\gets y_2 - {\cal G}(z_1)$
   \State $x_1\gets z_1 - {\cal F}(x_2)$
   \State $\gradd{z}_1\gets \gradd{y}_1 +
      \left( \frac{\partial {\cal G}}{\partial z_1} \right)^\top
      \gradd{y}_2$  \Comment{ordinary backprop}
   \State $\gradd{x}_2\gets \gradd{y}_2 +
      \left( \frac{\partial {\cal F}}{\partial x_2} \right)^\top
      \gradd{z}_1$ \Comment{ordinary backprop}
   \State $\gradd{x}_1\gets \gradd{z}_1$
   \State \(\gradd{w}_{\cal F}\gets
      \left( \frac{\partial {\cal F}}{\partial w_{\cal F}} \right)^\top
      \gradd{z}_1\) \Comment{ordinary backprop}
   \State \(\gradd{w}_{\cal G}\gets
      \left( \frac{\partial {\cal G}}{\partial w_{\cal G}} \right)^\top
      \gradd{y}_2\) \Comment{ordinary backprop}
   \State \textbf{return} $(x_1, x_2)$ and
      $(\gradd{x}_1, \gradd{x}_2)$ and
      \((\gradd{w}_{\cal F}, \gradd{w}_{\cal G})\)
\EndFunction
\end{algorithmic}
\end{algorithm}

{\bf Modularity.} Note that Algorithm \ref{revgradd} is agnostic to the form of
the residual functions ${\cal F}$ and ${\cal G}$. The steps which use the
Jacobians of these functions are implemented in terms of ordinary backprop,
which can be achieved by calling automatic differentiation routines
(e.g.~\verb+tf.gradients+ or \verb+Theano.grad+). Therefore, even though
implementing our algorithm requires some amount of manual implementation of
backprop, one does not need to modify the implementation in order to change the
residual functions.

{\bf Numerical error.} While Eqn.~\ref{reversibleblock} reconstructs the
activations exactly when done in exact arithmetic, practical \verb+float32+
implementations may accumulate numerical error during backprop. We study the
effect of numerical error in Section~\ref{rev_net_perf}; while the error is
noticeable in our experiments, it does not significantly affect final
performance. We note that if numerical error becomes a significant issue, one
could use fixed-point arithmetic on the $x$'s and $y$'s (but ordinary floating
point to compute ${\cal F}$ and ${\cal G}$), analogously to
\cite{maclaurin2015gradient}. In principle, this would enable exact
reconstruction while introducing little overhead, since the computation of the
residual functions and their derivatives (which dominate the computational
cost) would be unchanged.

redpajama_set_name: RedPajamaArXiv


text:
Tuesday, 19th May 2009
Benchmark For Live Entertainment
The below Press Release has been released:
500 Dancers Fly to Los Angeles for Extraordinary Auditions.
Amazing Innovations in Staging and Set – New Benchmark for Live Entertainment
Rehearsals for the Michael Jackson shows are well underway and already the King of Pop is hard at work preparing his concerts with show director Kenny Ortega and contributing choreographer and dancer from the Dangerous and HIStory tours, Travis Payne.
Shrouded in secrecy and the subject of much speculation as to their content, the shows have created a level of excitement never witnessed before. Fittingly, for such an out of this world show, rehearsals are taking place south of Silicone Valley, west of the NASA Space Center!
One of the highlights of rehearsals has been the dance auditions. With over 5,000 applications received, the number was narrowed down to 500 potential candidates, flying into California from all over America, and as far away as Australia, Canada, Japan, Europe, UK, and South America. They are some of the greatest dancers the world has to offer, and they were there for one reason – Michael! His influence inspired many of them to become dancers in the first place.
The competition was intense during the 3 days of auditioning at the Nokia Theatre L.A. LIVE in Los Angeles. Michael Jackson was present along with Kenny and Travis to make the final selections. The challenge to secure a place in the show became a reality for fewer and fewer as the level of competition increased. Dancers were tested on their strength, skill, style, creativity, fitness, commitment and execution
Ortega says, "The level of dancers that we saw at the auditions was incredible. Michael was extremely hands on throughout the process in working with the dancers and selecting the final candidates. It was an exciting, intense and highly creative process and not an easy decision in the end."
For your front row seat at the auditions log onto www.MichaelJacksonLive.com for behind-the-scene footage.
The final line-up of just 12 dancers includes an exceptional mix of international performers. Selected to perform alongside Michael Jackson at London's O2 arena are US dancers Charles Kapow, Ricardo Reid, Tyne Stecklein, Misha Hamilton, Mekia Cox, Nicholas Bass, Danielle Rueda-Watts and Christopher Grant. As well as, Timor Steffens from Germany, Australian dancer Shannon Holtzapffel, Daniel Celbre and Devin Jamieson from Canada.
Randy Phillips, President & CEO of AEG Live, commented, "Watching Mike and Kenny create this spectacle using the greatest repertoire in contemporary music is one of the most exciting experiences in my career in live entertainment. The fans who are flocking to see Michael Jackson's return to the stage will not be disappointed. From the special effects to the quality of the dancers and musicians, we are setting a new standard for major arena shows".
Be clear that this is not just a concert, "This Is It" will be a fully integrated musical experience combining theatre, film, music and the latest advances in technology. Movies, music and movement all rolled into one! . Michael is present at rehearsals overseeing every detail day in day out and constantly pushing the bar slightly higher.
Ortega adds, "It is an intense and exciting creative period. Onstage with the greatest entertainer of our time, we have a strong, talented, dynamic group of dancers and musicians. These are artists whose talent, dedication, skill and passion for performing with their idol is felt every hour of every day spent rehearsing. What is going to take place on that stage at The O2 arena in July will truly be inspiring."
By the end of the 50 concerts, 750,000 fans will have witnessed one of the greatest musical events in history. This will be the last chance to see the King of Pop in London – a historic stage show, and once in a lifetime concert experience – this really is it!
Note To Editors – One of the world's most popular entertainers, Michael Jackson has sold over 750 million units worldwide and is one of the very few artists to be inducted twice into the US Rock and Roll Hall of Fame. The Guinness Book of World Records has recognised Michael Jackson as the Most Successful Entertainer of All Time and Thriller as The Biggest Selling Album of All Time. Jackson won 13 Grammy Awards and received the American Music Award's Artist of the Century Award.
Michael Jackson's "This Is It" 50 concerts sold out in a matter of hours on March 13th smashing box office records.
Production Notes:
Kenny Ortega, Producer / Director
Travis Payne, Associate Director /Choreographer
Michael Cotten, Production Designer
Patrick Woodroffe, Lighting Designer
Michael Bearden , Musical Director
Bugzee Hougdahl, Production
Robb Wagner, Screen Producer
Kenny Ortega, the renowned Director, Producer, Choreographer and creative talent is hard at work collaborating with Michael Jackson in the overall design and direction for the upcoming 'This Is It' concert shows.
Ortega previously worked on Michael Jackson's Dangerous and HIStory tours, some of the most spectacular shows ever staged. Ortega, having once been in partnership with Gene Kelly, is most recently known for his work as the Director and Choreographer of Disney's, High School Musical 1, 2 & 3. Ortega also Produced and Directed the Barbra Streisand's Millennium concert tour and is a two time Emmy award winner for his work as Director and Choreographer of the 2002 Winter Olympics opening and closing ceremonies in Salt Lake City, Utah.

redpajama_set_name: RedPajamaCommonCrawl


These forms are generally completed and signed electronically using the secure & confidential client portal. A link to the portal will be emailed to you after you schedule your first session. However, if you prefer to fill them out by hand or if you would like to review them prior to scheduling, you may download them here.


Have We Hit Peak PUBG?
PlayerUnknown's Battlegrounds is slowly dying! Okay, that's more than a bit hyperbolic, but the game has experienced its first noteworthy drop in player count, which has folks questioning if the game is about as popular as it is going to get.
To be clear, PUBG seeing a dip in player count means it's only hitting concurrent numbers on par with what most other developers would consider fantastic figures for lifetime sales of any other game. But, based on a recent report from Gameinformer, the New Year has brought with it a noteworthy dip in activity, which is only really important because of the game's success.
Battlegrounds pulls in an insane number of daily players but, over the past month, the average has dropped off by several million. The fact that "several million" is a term that can be applied to a "noticeable drop" in game activity goes to prove just how successful PUBG has been and, if we're not goofing off like in this story's lede, very much so continues to be.
PUBG grew in popularity throughout 2017, hitting a fever pitch through the fall when it seemed like everyone on the planet was either playing, streaming or watching streams of the game. In December, update 1.0 dropped on the PC, marking the game's official exit from early access. That's also when PUBG launched on the Xbox One in early access, well on its way to a "full launch" this year.
All told, PUBG has shipped more than 30 million copies on Steam with another 4 million sold on Xbox One. The game's peak concurrent player count on the PC is more than 3 million. In the past month, though, the number of concurrent players has been on the decline. As of this writing, the current player count is 529,997, with the peak in the past 24 hours being nearly 1.7 million.
In other words, it looks like the honeymoon phase is officially over. We expect to see spikes pop up if a new map, mode or major update launches but, based on what we're seeing, folks are speculating that PUBG may have hit its ceiling.
This can be attributed to a number of factors. For starters, the game has been rolling in one form or another for over a year at this point. Second, games like Fortnite: Battle Royale have started winning over a portion of the audience. Until recently, PUBG hasn't really had any competition in the 100-player arena genre, so other quality contenders are going to snag some of the attention. Also, there have been a lot of big games launched in the past several months, including a new Call of Duty and, more recently, Monster Hunter: World. In other words, folks may have moved on to other pastures for the time being.
We'll just have to wait and see if this trend continues. In the meantime, we figure PUBG is doing absolutely fine, even if a portion of its players have started to stray from the pack.


So when someone says that Eve is dying I always wonder which twin are we looking at? The player population is one measure and the companies health seems to be the other. But if one twin starts to fall the other will try to support it as best it can (or fall right along with him/her). I am sure that and unchecked fall of one will kill both and that will happen someday. But this is not (yet) that day.
But summer for Eve is an odd time. I am sure any of us living in a place like Iceland would take advantage of fair weather whenever possible so it is not a huge surprise when they all seem to take their off time in the same months. This causes a slowdown in production and change to the game so the doom-cryers shout that Eve is dying. By the same regard players from around the globe seem to also have lives and some minor awareness of the outdoors. We go out and try RL for a bit and don't log onto the game as much. DEATH DOOM DESPAIR!!
I have often admitted to being an optimist but even I acknowledge that all things end, someday, myself included. That does not mean I go through each day browsing casket catalogues, just that I am aware of mortality in all things. Will Eve die? yup. It will die when one twin or the other gives up. I do not surrender easily to panic, despair, doom. Knowledge of mortality does not mean we have to race towards it rushing past all the life between now and then.
I'll be in there, waiting for you and if I am the last player? I will turn out the lights when I am done.
Oh, while writing this I had lots of images and research that molded my thoughts without making it into the post. From the Matrix 'not like this' to history of conjoined twins who died. But in the end? I gots stuff to do, in game and out.
This entry was posted in Uncategorized and tagged Blog Banter, eve online, OOC. Bookmark the permalink.
I guess it's the wake of Fanfest 2016 and the lack of big news about the future.


text:
PM Imran, Angela Merkel discuss Kashmir
FeaturedNational
By Web Desk Staff On Aug 23, 2019
Islamabad: Prime Minister Imran Khan held a telephonic conversation with German Chancellor Angela Merkel on Friday to discuss India's illegal and unilateral actions aimed at altering the disputed status of Indian occupied Jammu and Kashmir and changing its demographic structure.
The prime minister informed the Chancellor that India's actions were in direct contravention of the UNSC resolutions, international law and its own solemn commitments.
The prime minister highlighted the dire human rights situation in the Indian occupied Jammu and Kashmir, including complete lockdown, blackout on all forms of communication and severe shortage of food and medicines.
He stressed that intensified Indian repression could result in massive loss of Kashmiri lives, which must be prevented at all costs.
The prime minister also underlined concerns about some false flag operation staged by India or some other ill-conceived step on the LoC to divert the world's attention.
He emphasized that India's actions had serious implications for peace and security in the region and the international community had the responsibility to act urgently.
Chancellor Merkel stated that Germany was closely observing the situation. She underlined the importance of de-escalation of tensions and resolution of issues peacefully.
The two leaders agreed to continue to work together for peace and stability in the region.


Unable to capture from YouTube but other sites work.
The most common cause of difficulties in downloading YouTube videos comes from accessing the secure https version of YouTube as opposed to the regular http version of the site. RMC5 will not download secure streams. Try visiting http://www.youtube.com. If the http switches back to https, please use the troubleshooter linked-to below.
A secondary cause of trouble comes with newer high speed internet connections and Internet Protocol Version 6 being used. Internet Protocol Version 6 (TCP/IPv6) needs to be turned off - it will not affect your connection speed.


2 events found on Gioia's timeline.
Gioia is a 25.68 m / 84′4″ luxury motor yacht. She was built by Numarine in 2018. With a beam of 6.6 m and a draft of 2.2 m She is powered by MAN engines of 560 hp each giving her a maximum speed of 11 knots and a cruising speed of 9 knots. Gioia's maximum range is estimated at 3500 nautical miles. The motor yacht can accommodate 8 guests in 4 cabins. The yacht was designed by Can Yalman.


text:
I-Corps™
03 Nov Midwest universities form regional innovation alliance with $3.5M award
Posted at 11:25h in News	by	Amanda Prestia
Posted November 3rd, 2016 by the University of Michigan. Contact Jane Sugiyama, 734-763-2908 janesugi@umich.edu
ANN ARBOR⎯A new four-university alliance will help researchers across the Midwest turn their inventions into marketable products to benefit society.
Supported by a $3.5 million grant from the National Science Foundation's Innovation Corps program, the new alliance establishes the Midwest I-Corps Node. It is led by the University of Michigan Engineering's Center for Entrepreneurship and includes the University of Illinois, Purdue University and the University of Toledo.
Together, these institutions have an annual research budget of $2.6 billion. The universities have sent more than 100 teams to the NSF National I-Corps program and trained more than 400 teams and 1,200 participants throughout the region. The node is designed to form the backbone of a network that educates, supports and connects academic researchers to the entrepreneurial ecosystem across the region.
"The geographic spread, diverse industrial base and scale of the node will help the entire region connect technology, market needs, people and money together in a way that was not previously possible," said Jonathan Fay, Midwest I-Corps Node executive director and U-M Center for Entrepreneurship managing director.
NSF I-Corps program director Lydia McClure is equally enthusiastic about the latest partnership.
"The Midwest Node will undoubtedly provide high quality programming, research and tools for the Midwest region and the nation," McClure said. "This node is an expansion on the successful I-Corps programs at U-M."
To achieve the goal of creating a collaborative network utilizing resources and expertise external to each university, the node will host world-class training programs for scientists and engineers throughout the region. These intensive courses are designed to get scientists and engineers to extend their focus beyond the laboratory. It requires researchers to examine the commercial potential of their technology to avoid building a product that no one wants—which is the No. 1 reason that startups fail.
"Our programs will enable us to build upon our strengths in technology research, as well as bolster our industry partnerships," said Jed Taylor, director of operations at the University of Illinois Technology Entrepreneurship Center. "The projects accelerated by the node will be pipelined into support programs to help mature them to the point of license, partnership or funding to launch a standalone startup."
The Midwest I-Corps Node activates in January 2017 and will be funded for five years. In addition to hosting the National I-Corps program, each university involved will host regional programs at their universities and across their respective states.
"We appreciate the spirit of collaboration demonstrated by U-M CFE's leadership, and it's exciting to be a part of a network that is working to shift the culture of traditional academic research toward impact inspired research in the Midwest," said Matthew Lynall, management professor and deliberate innovation for faculty director at Purdue University's Discovery Park.
Ian Steff, executive vice president and chief innovation officer of the Indiana Economic Development Corporation, also looks forward to the collaborative impact of the Midwest I-Corps Node.
"Leveraging the resources of these outstanding research universities is of great benefit," Steff said. "This exciting program has a great deal of potential to support the Midwest's ecosystem for entrepreneurship in further developing industries of the 21st century."
Midwest Node, National Science Foundation's Innovation Corp, News, NSF I-Corps, Purdue University, University of Illinois, University of Michigan, University of Michigan Engineering's Center for Entrepreneurship, University of Toledo
Governor Abbott appoints Lubbock woman as one of four to Product Development and Small Business Incubator Board
Top UT Inventions and Innovations of 2019
Kimberly Gramm Named Associate Vice President of Innovation & Entrepreneurship
Teaming Up for the Innovation Roller Coaster
Get Out of the Lab!
Brad Burke Caltech Deshpande Foundation engineering entrepreneurship Errol Arkilic Ewing Marion Kauffman Foundation Heath Naquin I-Corps I-Corps Methodology I-Corps Node I-Corps Nodes IC² Institute innovation Innovation Corps Juan Sanchez Michigan National Science Foundation New York City NSF NSF I-Corps NSF Innovation Corps OwlSpark Press Release Regional research Rice Alliance Rice Alliance for Technology and Entrepreneurship Rice University Richard Lester Robert Peterson southern California Southwest I-Corps southwest region STEM SW I-Corps teams Texas Texas A&M Texas A&M University Texas Tech Texas Tech University University of Texas University of Texas at Austin UT Austin


text:
Fangirl Star Wars Superheroes
Fangirls Around the Web: January 30, 2013
January 30, 2013 January 29, 2013 Fangirl	"Star Wars" (Dark Horse comic), Ashley Eckstein, Brian Wood, Cat Taber, Fangirls Around the Web, Star Wars, The Clone Wars, Wonder Woman
The Clone Wars VIPs tour Osprey during USO visit to MCAS Miramar
We'll kick off this edition of Fangirls Around the Web with several items about Star Wars.
Over at the Star Wars Blog, Ashley Eckstein shares her account of the recent special showing of The Clone Wars sponsored by the USO at the MCAS Miramar in California, and Bonnie Burton recaps J.J. Abrams' lifelong appreciation of Star Wars. On her blog Fangirls in the Force, Lillian Skye has a great interview with Cat Taber about The Clone Wars and her Games for Soldiers charity.
Brian Woods' series Star Wars from Dark Horse is already a smashing success with issue #1, which sold out its first printing in 24 hours. The second printing also sold out before issue #2 hit shelves, and now a third printing is scheduled for late February. I never doubted Leia's starring role would kick butt!
In honor of the launch of Star Wars, Maria Selke wrote an X-Wing Pilots Edition of Star Wars Thursday for The Fandom Post.
Wood also launches a new X-Men title from Marvel in April with an all-female cast. He discussed his reasons for the cast extensively in an interview with Wired. The announcement was also featured in USA Today, including this quote:
He wants to challenge the double standards that have been in superhero books for years, were Wolverine can sleep with anybody but if a female character does it twice, she's promiscuous, which Wood sees constantly online.
"To everybody's credit, these people are often shot down immediately for being sexist and unfair, but that is a very common thing," Wood says.
"We're just going to do it. We're not going to worry about that. If Kitty or Rogue has basic human bodily urges, tough luck (to those opposed). To me, that's as much of the X-Men as anything else.
Speaking of female characters in comics, Jill Pantozzi has a great essay, Hey, That's My Cape!: Women and Comic Books in 2013, at IGN.
Wonder Woman is also in the news. Owen Quinn at Following the Nerd takes a Retro Look Back at the original Wonder Woman television series, from 1975-1979, while Comic Vine considers the ingredients for a successful updated version of the show. TVLine reports that the series in development at The CW, Amazon, is still a possibility for the 2013-14 season, but might be farther out. TVLine previously reported this comment:
"We're waiting to see the script and are busy casting Diana," said Pedowitz of Amazon, adding that this project has little in common with NBC's ill-fated David E. Kelley-produced reboot. "I have not seen David's project [but] this is an origination story [like] Smallville."
Jaimie Alexander, who played Sif in Thor, told MTV that Wonder Woman deserves the star treatment:
"There needs to be a positive role model. If you're going to make 'Wonder Woman,' make it like 'The Bourne Supremacy,'" she said. "Let's do something awesome like that. Like 'Alias.' That was a fantastic show. Why can't we do that?"
Rachel Weisz of The Mummy fame was quoted at The Frisky talking about playing strong female characters:
I never really understand when people say that I like playing strong characters. I always find that a little weird because, I could be wrong, but I don't think people ever say that to men. It's like, well what should I play, a weak character? Does that mean that I can lift weights? I guess it means strong-minded, right? But it's weird because people never say that about a male actor — Michael Fassbender or George Clooney, for example — that they love playing strong characters. It's a weird gender thing.
I'll end on a personal fangirl note: As a college intern, the Marine Osprey shown in the featured image was one of the programs under development in the office. It's pretty awesome to see it go from a picture on a wall to the real thing!
Tricia Barr took her understanding of brand management and marketing, mixed it with a love of genre storytelling, and added a dash of social media flare to create FANgirl Blog, where she discusses Star Wars, fandom, and strong female characters. She also writes about Star Wars for Random House's science fiction and fantasy blog Suvudu.com and for Star Wars Insider magazine.
In her spare time, Tricia puts the finishing touches on her first novel, Wynde. For excerpts and tales of her adventures in creating a fictional universe, hop over to TriciaBarr.com.
For updates on all things FANgirl follow @FANgirlcantina on Twitter or like FANgirl Zone on Facebook.
← Jaina Solo ArtFX Hits Shelves
Female Characters from the Expanded Universe for Episode VII? →
Tesla and Star Wars: The Power of the Female Dollar
October 15, 2015 Linda Comments Off on Tesla and Star Wars: The Power of the Female Dollar
Linda's Star Wars: The Old Republic Diary – Part Four
March 3, 2012 Linda Comments Off on Linda's Star Wars: The Old Republic Diary – Part Four
GeekGirlCon 2012: The Star Wars Saturday That Almost Didn't Happen
September 13, 2012 Fangirl 4


text:
Playing the US Lottery Online
Whether you're interested in winning the jackpot or just want to play for fun, the United States lottery system has a lot to offer. There are currently 45 different states operating a state-wide lottery, as well as the Virgin Islands and Washington DC. Depending on your state, you may be able to purchase tickets online. Some lottery websites also provide Instant Games, which are casino-style games that can be played from the comfort of your home. If you win, you'll need to visit a local lottery claim center to receive your prize.
The first US state to introduce a lottery was New Hampshire, which launched its lottery system in 1964. Today, the lottery is the largest of the lotteries in the country, with prizes ranging from $10,000 to $300,000. Other games include Mega Millions, Powerball, and Keno. In addition to these national games, players can also play several in-house draw games and multi-state draw games.
Currently, there are six states that have legalized online lotteries: Illinois, Massachusetts, Nevada, North Carolina, Pennsylvania, and Rhode Island. While these states are the only ones to legally offer lottery games on the Internet, more states are working towards launching their own online lotteries.
In Illinois, online ticket sales started in March 2012. After the pilot program proved successful, the website began offering Daily Lottery Games and Instant Games. Later, the website expanded to include Powerball lottery tickets. In December 2012, Illinois began selling Powerball and Mega Millions online. Currently, Illinois offers four draw games and four exclusive draw games.
In Massachusetts, there are eight draw games to choose from. Players can also opt for three multi-state draw games. The lottery has also made a commitment to donate 25 cents of every dollar collected on a ticket to the state's Common School Fund. There are two other multi-state draw games, including Lucky for Life.
In Rhode Island, players can participate in four in-house draw games, and one multi-state draw game, including MegaHits. The lottery also features 25 Instant Games, including Bingo and Wild Money. In addition, Rhode Island is a member of the Multi-State Lottery Association, which allows them to participate in the Powerball, Mega Millions, and Lucky for Life. If you win a larger prize, you may need to provide your identification documents before you can receive your payout.
The lottery in Indiana is called the Hoosier Lottery. Players can participate in the Powerball, Mega Millions, Cash4Life, and other in-house games. Ticket prices vary, ranging from $1 to $20. Some lottery tickets can be cashed in at retail locations, and smaller winnings can be withdrawn to your bank account. If you win, you'll need your ID, a certified mailing service, or a ticket with an IRS form attached.
While the online lottery is growing, it's important to keep up with the changing laws. Some states have made their games available online, while others have made it difficult to buy online. While you're able to purchase tickets online, you'll need to be at least 18 years old to do so.


Charcoal works like a magnet, instantly locking onto and drawing out deep down dirt and oil that can cause blackheads. Infused with natural charcoal, these innovative pore strips draw out excess oil for the deepest clean. In just 10 minutes, you'll remove weeks worth of build-up, see 3x less oil and achieve your deepest clean.
Polyquaternium-37, Silica, Water (Aqua), Glycerin, Polysilicone-13, Iron Oxides/CI 77499, PEG-12 Dimethicone, Methylparaben, Menthol, Charcoal Powder, Styrene/Stearyl Methacrylate Crosspolymer.


text:
Lotus Prime – No Muck, No Lotus
By Ellie Mullins
Back at the end of last year, Lotus Prime made his musical debut with a beautiful single 'Spirit Tracks'. Previously being involved in the acting world and wanting to pursue music also, it's worked out in his favour as he has bags of talent to kickstart an extremely successful career in electronic music. He has dabbled in making music since he was a teenager but it wasn't until recently where he decided to make a career out of it and explore his other creative side. Listening to the amount of talent he has (which shines through in the original productions), it's a wonder as to why he didn't start making music properly a long time ago but he has now fully realised his talent and passion.
Further adding to the Lotus Prime discography, the artist who's real name is Braeden Baade has released his follow-up to 'Spirit Tracks' called 'No Muck, No Lotus'. With a deep and meaningful story behind it, it has all the strength and depth to become a very successful track. Going through a massively tough time in his life, he says that a lotus can only grow in mud (hence the title 'No Muck, No Lotus') and the hardships he has been faced with have allowed him to become more vulnerable and as a result, stronger.
The track itself takes a darker, more bass heavy tone and is a perfect audio accompaniment to the story behind it. Showcasing another side to his musical talents, he is able to produce multiple different styles of electronic sounds to create tracks that are unique to each other but also shining examples of the signature sound he has carved for himself.
'No Muck, No Lotus' is available on Spotify below and Apple Music and SoundCloud now.
Image Credit: LeMar Hayes
Tags: Lotus Prime, No Lotus, No Muck
About the Author / Ellie Mullins
A 24 year old dance music enthusiast from Manchester, UK. Lover of all genres, especially dubstep and house. Find me at gigs and festivals across the world.


Director Keith Dearling says brokers are wasting a lot of time using platforms to search for sub-prime homeloans because the in-built underwriting of these systems cannot cope with the more complex nature of these applications.
He says the main problem is that clients can be quoted unrealistic initial rates from a prime lender and then get turned down, leaving clients with a "footprint" on their credit record saying that they have been turned down for a loan.
Dearling says clients can be confused by being quo-ted a lower rate at the agreement in principle stage but getting a higher rate once their credit status has been checked.
He says there are clear advantages to using these systems – they provide audit trails and centralised information – but advisers should not rely on the accuracy of their information when talking to clients who are likely to end up with sub-prime loans.
Rebstone Mortgage & Finance adviser Marilyn Holmes says she often goes direct to the underwriters of main lenders as sub-prime quotes cannot be sourced.
Multi-ties will not fundamentally change the market immediately but will provide opportunities in the longer term, claims Aegon group chief executive David Henderson.
Standard Life has raised concerns that some IFAs are not fully aware of the regulatory changes for pure protection products from January 2005.


I can see right now that I'm going to have to be more explicit when I talk to you.
What is implicit in Corinthians is explicit in Colossians.
Convicts not pardoned with an explicit restoration of suffrage privileges are disfranchised - a rare clause in the United States.
Of these early writers, Hatchett is the most explicit, the various bituminous substances being by him classified and defined.
Please leave explicit directions for the housekeeper, so she knows what needs to be done.
Be sure to make yourself aware of the explicit details of the contract before signing the paperwork.
The explicit statement that he was buried completes the contrast between him and his greater predecessor.
That was an explicit statement of Venetian aims and contentions: the place and people had made each other and now belonged exclusively to each other.
I wasn't sure what to do as the instructions for our final assignment were not made explicit.
Upon the nature of the reasoning by which in mathematical science we pass from data to conclusions, Hume gives no explicit statement.
We progress by making explicit the oppositions contained in the fundamental synthesis, by uniting these opposites, analysing the new synthesis, and so on, until we reach an ultimate pair.


This year has seen the College establish a new Foundation. One of the first initiatives of this was to set up a Year 7 Social Group who then took reponsibility for organising various functions over the year for their respective House Groups.
The aim of this was to welcome our new year 7 parents, encourage them to feel part of the Marcellin community, meet other year 7 parents, and to become involved in different activities.
Many of this year's Year 7 parents have embraced this and it has been really pleasing to meet many of you, and to see your faces popping up in various volunteer roles within the school, and at events such as Mothers and Fathers Day Breakfasts, the Gala Ball etc.
Thank you to you all for being a part of this new initiative, and below is a photographic taste of the year.


What are my goals? You should know what your end goal is for investing before purchasing anything. For example, are you looking to save for retirement or do you want to make money fast? How much money are you willing to invest and what type of return are you hoping for? Your goals will influence the type of investment you enter in.
What is my risk tolerance level? Every investment carries some level of risk. Investments with a low risk will make small gains and losses while investments with a high risk stand to gain or lose higher amounts of money. Investors should consider how comfortable they are with smaller versus bigger losses and gains before entering an investment opportunity.
Is the investment registered and the person selling it licensed? In the state of Indiana, all securities and the people who sell them must be licensed and registered with the Indiana Securities Division. Use the searchable databases or contact the Securities Division to make sure everything checks out before you enter an investment.
It's a time-tested truth that education is the key to success in nearly all areas of life—including investing. Consumers that make informed investment choices are significantly less likely to fall victim to fraud, and in turn, less likely to lose their money in unsound investments.
We have included some helpful information and valuable resources for current investors and Hoosiers who may be considering investing in the future.
Get written information. Every investment opportunity should have a prospectus that outlines the details of the investment. Once you enter an investment, be sure to closely examine your financial statements for suspicious activity.
Ask questions. If the person is legitimate, they should not hesitate to answer any questions you have. If they hold back details or avoid giving written information, tell them you are not interested and walk away.
Don't rely solely on the testimony of others. Just because someone else made money doesn't mean you will or that the investment is legitimate.
Be skeptical of promises of high returns. If someone offers returns that seem too good to be true, they probably are.
For more information on specific types of investment fraud, visit Securities 101 and the Library of Resources.
The person is selling investments or acting as a loan broker without being licensed to do so. Contact the Securities Division at 1-800-223-8791 or click here to use the searchable databases to check licensing and registration.
The person doesn't provide written information. Every investment opportunity should have a circular or prospectus that outlines the details of the investment.
The person pressures you to act now, not giving you time to fully research the investment opportunity.
The person offering the investment is hesitant to answer questions or avoids giving you full responses.
The person is always coming to you rather than inviting you to their office. It may be that the office doesn't exist.
If a senior citizen is targeted with fear-based appeals, especially if they are living on a fixed income and worried about having enough money to make ends meet.
A person trying to take advantage of a common bond, trust or set of beliefs. More often this is occurring in religious communities.
A mortgage broker asking for upfront fees outside of bonafide fees for services provided by someone other than the loan broker. Examples of legitimate upfront fees may include credit reports, appraisals, and title examinations.
Observing a financial professional promise and "deliver" returns that are unsustainable in the current market.
The person tries to sell offshore investments, which can be very high risk.
Annuity: A financial product sold by financial institutions that is designed to accept and grow funds from an individual and then, upon annuitization, pay out a stream of payments to the individual at a later point in time. Investors are typically locked in for a specific amount of time and can incur high fees if they try to withdraw money from the annuity early.
Bond: A debt investment in which an investor loans money to an entity (corporate or governmental) that borrows the funds for a defined period of time at a fixed interest rate. Bonds are used by companies, municipalities, states and U.S. and foreign governments to finance a variety of projects and activities.
Broker: An individual or firm that charges a fee or commission for executing buy and sell orders submitted by an investor.
Certificate of Deposit: A savings certificate entitling the bearer to receive interest. A CD bears a maturity date, a specified fixed interest rate and can be issued in any denomination. CDs are generally issued by commercial banks and are insured by the FDIC. The term of a CD generally ranges from one month to five years.
Futures: A futures contract is a legally binding agreement to buy or sell a commodity or financial instrument at a later date. Futures contracts are standardized according to the quality, quantity and delivery time and location for each commodity. The only variable is price.
Hedge Fund: An aggressively managed portfolio of investments that uses advanced investment strategies such as leveraged, long, short and derivative positions in both domestic and international markets with the goal of generating high returns. Legally, hedge funds are most often set up as private investment partnerships that are open to a limited number of investors and require a very large initial minimum investment. Investments in hedge funds are illiquid as they often require investors keep their money in the fund for at least one year.
Investment Adviser: any person or group that makes investment recommendations or conducts securities analysis in return for a fee, whether through direct management of client assets or via written publications.
Mutual Fund: A fund operated by an investment company that pools funds from multiple investors and invests it in stocks, bonds, short-term money market instruments, and/or other securities.
Option: a contract between a buyer and a seller that gives the buyer the right—but not the obligation—to buy or to sell a particular asset at a later day at an agreed price. In return for granting the option, the seller collects a payment (the premium) from the buyer.
Promissory Note: A written, dated and signed two-party instrument containing an unconditional promise by the maker to pay a definite sum of money to a payee on demand or at a specified future date.
Prospectus: A formal legal document, which is required by and filed with the Securities and Exchange Commission, that provides details about an investment offering for sale to the public. A prospectus, also known as a circular or offering, should contain the facts that an investor needs to make an informed investment decision.
Return on Investment: A performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments. To calculate ROI, the benefit (return) of an investment is divided by the cost of the investment; the result is expressed as a percentage or a ratio.
Risk: the chance that an investment's actual return will be different than expected. This includes the possibility of losing some or all of the original investment. Risk is usually measured by calculating the standard deviation of the historical returns or average returns of a specific investment. Every investment carries some level of risk. There is never a guarantee that you will make any money.
Stock: A type of security that signifies ownership in a corporation and represents a claim on part of the corporation's assets and earnings.
Suitability: A situation (and sometimes a legal requirement) that an investment strategy meets the objectives and means of an investor. In most parts of the world financial professionals have a duty to take steps that ensure that an investment is suitable for a client.
Viatical Settlements: An arrangement in which someone with a terminal disease sells his or her life insurance policy at a discount from its face value for ready cash. The buyer cashes in the full amount of the policy when the original owner dies. Also referred to as a Life Settlement, this type of investment is incredibly risky.


text:
You are here: Home / News / It's Official: Elsevier is Acquiring Mendeley
It's Official: Elsevier is Acquiring Mendeley
Filed by Gary Price on April 9, 2013
Rumors about a possible acquisition have been circulating for months.
Today, a blog posts from Mendeley and Elsevier formally announce the Elsevier acquisition.
From the Mendeley Blog Post We Learn:
The most important things first: very little will change for you as a Mendeley user. In fact, Mendeley is only going to get better for you. For starters, we are doubling everyone's storage space at no cost. Your free Mendeley account now comes with 2GB, Mendeley Plus and MIE accounts get upgraded to 5GB, and Mendeley Pro accounts to 10GB. There will always be a free version of Mendeley, and our functionality will continue to improve, now even faster than before.
Your data will still be owned by you, we will continue to support standard and open data formats for import and export to ensure that data portability, and – as explained recently – we will invest heavily in our Open API, which will further evolve as a treasure trove of openly licensed research data.
On a more personal note, let me also explain why we chose to team up with Elsevier at this point. Mendeley had just raised a significant round of funding from existing and new investors, with more investors wanting to join. Also, Mendeley's revenues from our individual and team premium accounts, as well as our new Mendeley Institutional Edition, had tripled over the past year. We could have continued on our path independently, yet we felt that the opportunity to give our users access to better content, more data, and faster development was just too exciting to pass up.
Read the Complete Mendeley Blog Post
From the Elsevier Blog Post:
For Elsevier, we can build upon strong foundations in search and discovery by adding capabilities in document and citation management and sharing. By offering integration between Mendeley, Scopus and ScienceDirect, we can make this combined platform the central workflow and collaboration site for authors. In addition, we will be able to provide greater access to a growing repository of user-generated content while building tools that will enable researchers to search this growing body of research more precisely.
By joining forces, we will be in an even better position to support the needs of researchers. Our resources will enable Mendeley to continue building on its platform, keeping it free for individuals while introducing new content and interoperability that will make it even more useful.
Mendeley will remain a separate platform with a distinctive brand — the favorite daily destination site of researchers to check updates on their network, collaborate with other researchers, access their stored content, get alerts to relevant research domains, and make progress on their workflow efficiency and research in general.
For institutions, the Mendeley Institutional Edition (MIE) will continue to be available. MIE is a tool that helps universities analyze research activity in real-time, providing a complement to the traditional Impact Factor system of academic citations. It also enables librarians to extract more value out of resources by optimizing their subscriptions and providing a better service to their researchers by aggregating anonymized statistics about faculty and student paper reading habits.
Read the Complete Blog Post and News Release
Although financial terms were not disclosed, TechCrunch estimates Elsevier paid between between $69 million and $100 million.


In this live webinar with Dr. Daud Scott, we discuss the topic of awakening vital life force and vibrant health and wellness as it relates to the key and critical dimensions of our lives, such as with regards to spiritual development and the clarification of consciousness, effectiveness in business and career, and joy and love in one's personal relationships.
Additional topics covered include the classical and wholistic model of health and wellness in contrast to the segmented and symptom oriented current medical model, the importance of exemplifying the Islamic ideal in our lifestyle choices and states of consciousness, the preeminent importance of embracing personal responsibility in one's life, a balanced understanding of divine destiny, and much more.
This series of videos + webinar were fantastic and very useful. Thank you very much.
You are very much welcome. I am glad that you have enjoyed them.


Today is the International Day for the Elimination of Violence Against Women. I've been following @countdeadwomen on twitter as they list all those who have been victims of domestic or misogynistic violence so far this year and, yeah, it is very depressing reading.
Here is something positive though. The charity Refuge help around 2800 women and children every day. On Christmas day they will have women in their shelters who arrived with only the clothes on their backs and their children in their arms. They won't be able to buy their kids any presents this year, but you can, in a few minutes, for a few quid.
I'm sitting here in my lovely warm house, contemplating Christmas food and still wondering what to buy for my kids this year. For Refuge I've bought a toy that was one of baby M's favourite birthday presents. She has boxes full of toys and whatever I get her for Christmas probably won't be as exciting as the paper it comes in. I'm ridiculously lucky. I can't pretend to be able to imagine what it's like to be a mother in a refuge at Christmas. But I do know the joy that that toy brings to a little child and that, even if nothing else I buy causes much excitement, this gift will be appreciated on Christmas morning.
What an absolutely great idea – I'm on it. Thank you.