text:
What Is Intuitive Eating and How Can It Help You Ditch Fad Diets for Good?
By Matt Crossman | December 28, 2021 | 0
Here they come. Holiday feasts followed incongruously by healthy eating tips, after which we will be inundated with advice as to which diet will help you shed the weight you put on over the holidays.
What if—hear me out here—you just said, "Screw it, I'm going to eat what I want this holiday season?" What if you walked past the plate of fudge, stopped, cut yourself a big ol' piece, stuffed it in your face, and didn't feel guilty about it, didn't run extra laps to make up for it, didn't do anything but thank Whoever It Is You Thank for creating fudge in the first place?
What if—more crazy talk coming—you adopted a similar attitude all year long? What if you let your weight be your weight and didn't obsess with myriad complicated, expensive and ultimately ineffective ways to control it?
Christy Harrison, a solopreneur, author and intuitive-eating coach, wrote Anti-Diet: Reclaim Your Time, Money, Well-Being, and Happiness Through Intuitive Eating after struggling with disordered eating.
Popularized by Elyse Resch and Evelyn Tribole, intuitive eating has 10 principles, which (at the risk of oversimplifying) can be summarized as eating the way you were born to. You eat following physical, not emotional, cues. You eat when you are hungry and stop when you are full. Although this does not give you a license to eat fudge for all three meals, you ignore the food police, your own shame and other cultural pressures.
Intuitive eating is not a diet but rather a way of thinking about food. "It's stripping away all of the diet mentality stuff, all the diet-culture baggage, and rules and regulations," Harrison says.
Harrison says re-ordering her thinking about food has made her more grounded and more productive as a solopreneur. Energy she used to spend on obsessing about food she now is free to use elsewhere.
"I don't think I would have been able to focus on so many different projects—writing a book, having a podcast, doing a private practice, doing all these interviews, all the stuff that I juggle as a solopreneur on a day-to-day basis."
In her book and an interview, she offered insight about how to build a healthier view of food.
Health and wellness are not the be-all, end-all.
This might be the most challenging of Harrison's views. It is possible to overdo an obsession with health and wellness, she says. "[Allow] your body to end up at the weight it wants to be, which I think is a scary thing for a lot of people in this culture, because this culture does really stigmatize higher weight bodies."
It's OK to eat what you want. "When you destigmatize foods and look at them all as equally worthy options, it takes away the irresistible pull toward 'forbidden' foods (don't those always taste the sweetest?)," Harrison writes in Anti-Diet. "Instead, you're able to choose what you truly want and need in any given moment."
Diet culture teaches us not just that this or that eating plan will help us lose weight but that it is morally superior. When you eat the wrong food, or too much of it, or both, you fail. If you eat to deal with that failure, the shame gets worse. Harrison and others assail that mode of thinking.
"The battle of the bulge is a battle against biology, so obesity is not some moral failing. I can't stress enough that becoming overweight is a normal, natural response to the abnormal, unnatural ubiquity of calorie-dense, sugary and fatty foods," writes Michael Greger in How Not to Diet.
Avoid highs and lows.
Extremes in any direction—overly restrictive diets, too much exercise for the sole purpose of burning off calories, obsession with which foods are "right and wrong," an unyielding focus on what and how much you eat—are harmful, physically, mentally and emotionally. "That's kind of part of this larger cultural context in which polarization is being rewarded," Harrison says.
Listen to your body, not the news.
"The first step that I would recommend for people is tuning out all of that breaking news about the food that you should avoid," and here she jokingly mimicked news reports: "'Chocolate is good for you, it'll make you live longer. Just kidding, chocolate will take 10 years off your life.'"
Gluten, dairy, meat, whatever—someone says eat it, and someone else says don't. None of them are completely right. "Drawn to the trendiest approaches, many people who strive to lose weight constantly bounce between fads: low-fat, low-carb, South Beach, Atkins, DASH, Zone, Ornish, keto, intermittent fasting… the list goes on and on," writes Brad Stulberg in The Practice of Groundedness. "The best diet is one to which you can stick."
That's easier said than done. Ninety-five percent of dieters put the weight they lost back on. As many as two-thirds gain more weight than they lost.
None of this is new. Harrison writes that since the 1960s, the problems with diet culture have been covered and discussed. And yet here we are, in 2021, still chasing novel ways to be thin, without thinking deeply about whether that's a goal worthy of the time, energy and money we spend on it, or if the means we are using to pursue it are likely to work.
You are not what you eat.
"A healthy attitude towards food is not having your identity so wrapped up in what you eat, or how you move your body, or your body size or things like that," Harrison says.
Defy the "Life Thief."
Harrison calls the omnipresence of diet culture the Life Thief, and defines it like this: "This system of beliefs that lionizes smaller bodies, demonizes larger ones, demonizes certain foods and elevates others. This system of beliefs really steals our time, our energy, our money, our happiness and our well-being."
Matt Crossman
Matt Crossman is a writer based in St. Louis. He writes about sports, travel, adventure and professional development. Email him at [email protected]
https://www.success.com/author/matt-crossman/
The Baddest Bull Rider Has a Secret to Success: Always Pick the Baddest Bull
What Happened When I Woke Up at 4:30 for 30 Days
5 Lessons Death-Defying Adventure Taught Me About Facing My Fears as a Solopreneur
How to Be a Successful Solopreneur: 4 Lessons From Renaissance Festivals
← Be Loud and Be Proud: Luvvie Ajayi Jones on Forging Your Own Path and Owning Your Successes5 Tips to Set Up Your Social Media for Success in 2022 →


You often arrive home tired from a long and hectic day at work. Your neck and back ache from all the prolonged sitting and standing at work. What is the first thing you do? Flop into your sofa? Perhaps not. You light a candle first, right?
From ancient gatherings to fancy dinner parties, high-end candles have been used across the world. But candles are not just beautiful sources of light. They can also significantly enhance your health and well-being.
Here are six ways candles add that much-needed spark into your life.
Aromatherapy is the practice of using scents to affect your mind and body. This practice comes with the knowledge that the human body is sensitive to the effects of different scents. Your brain's chemistry changes when you are exposed to particular smells.
For instance, if you are suffering from sleep issues, you can light up scented candles to have a good night's sleep. Candles with the scent of chamomile are effective at reducing restlessness and anxiety.
By focusing your attention on the candlelight an hour before going to bed, your eyes and the entire nervous system begin to relax. In turn, this allows for a restful and good night's sleep.
If you often work overtime, scented candles can help clear your mind and maintain focus.
There is definitely no way for you to work at your desk for hours when you are feeling foggy. So it is important for you to clear your mind of any distractions. There is no better way of doing this than by enjoying the smell of a scented candle. Scents like cypress, thyme, and basil work well in work settings because they encourage deep thought and reflection.
The ability to reduce stress is probably the most famous benefit of candles. People who enjoy meditating easily find inner peace when they choose to focus on candlelight. The light emitted by candles has a calming effect, which helps increase self-awareness as well as minimises stress. For an extra dose of relaxation, opt for lavender-scented candles.
As you focus on the candle flame without blinking, tears will rush from your eyes. This helps cleanse your eyes, removing any blockages and dirt on each of your eyes' surfaces.
Moreover, focusing on the flame will help strengthen the muscles and nerves of your eyes. Errors of refraction can get rectified and your optic nerves become stronger. By performing this activity regularly, the eye strain caused by constant exposure to computer screens is eliminated.
The regular practice of candlelight meditation helps keep your brain balanced and alert. Through this, your mind learns how to become unperturbed and calm. In turn, this helps greatly in improving memory. Moreover, by practicing this type of meditation each day, your willpower will get stronger.
Did you also know that scented candles can help bring back one's memory? Human beings usually associate different memories with scents that may have been used during a forgotten time. Scents are effective tools for any time you wish to take yourself back to a certain event or memory.
Depression negatively affects the way you think, how you act, and how you feel. It causes feelings of sadness and a loss of interest in things once enjoyed. The good news is that it is treatable.
One excellent way of erasing the feeling of depression is through candlelight meditation. The flame of the candle can soothe the mind as well as improve the supply of oxygen to the brain.
Whether you want to be relieved from the effects of a hectic, stressful day, or you wish to promote inner peace, the ways in which scented candles can boost your life are endless. You can make lighting candles a part of your daily ritual for you to bring balance, warmth, and joy into your world.
When it comes to lighting candles, safety should also be prioritised. Never leave them unattended, and be aware of essential candle burning tips. When your work schedule doesn't permit you to light a candle and enjoy it every time, do it whenever you are not busy. Mornings, evenings, vacations or weekends are all great times to work luxury candles into your life.


We're all aware of the bubbles we live in to some extent, and how technologies like Facebook encourage us to stay within them, but unless you live off the grid—and clearly you don't—you're probably affected by your comfortable bubble more than you think. Everything you do, say, and buy creates a feedback loop you may think you've broken, but with the wrong goals in mind it's the bubble you're going to live in for the rest of your life and possibly never know it.
"In just a few short years, Steve Jenkins and Derek Walter, dads of Esther the Wonder Pig, have cemented a place for themselves among the world's most well-known and successful animal activists. With a focus on humour, and creating lasting connections between people and animals, their "Kindness Is Magic" approach to advocacy has caught the attention of well over one-million followers from all over the world. In 2014 Steve and Derek founded the Happily Ever Esther Farm Sanctuary in Campbellville, Ontario, where they continue to rescue abandoned and abused farmed animals."


Please post only flashrom/coreboot/seabios or other firmware related contents.
You can either post text output of lspci, flashrom etc., or a binary file like a bios blob.
Text files and pastes will be available for public download, binaries can NOT be downloaded to prevent abuse.


text:
\section{Introduction and Related Work}

We consider one kind of unsupervised
reinforcement learning problem: Given a Markov decision process (MDP) but no reward
information, is it possible to learn and store a compact object that, for
any reward function specified later, provides the optimal policy for that
reward, with a minimal amount of additional computation?  In a sense,
such an object would encode in a compact form the solutions of all possible planning problems in
the environment. This is a step towards building agents
that are fully controllable after first exploring their environment in an
unsupervised way.

Goal-oriented RL methods~\cite{andrychowicz2017hindsight, plappert2018multi} compute policies for a series of
rewards specified in advance (such as reaching a set of target states), but
cannot adapt in real time to new rewards, such as weighted combinations
of target states or dense rewards.

Learning a model of the world is another possibility, but it still
requires explicit planning for each new reward; moreover, synthesizing
accurate trajectories of states over long time ranges has proven
difficult~\cite{talvitie2017self, ke2018modeling}.

Instead, we exhibit an object that is both simpler to learn than a model of the world, and contains the information to recover near-optimal policies for any reward provided a posteriori, without a planning phase.

\cite{borsa2018universal} learn optimal policies for all rewards that are
linear combinations of a finite number of feature functions provided in
advance by the user.
This limits applications: e.g., goal-oriented tasks
would require one feature per goal state, thus using infinitely many
features in continuous spaces.  We reuse a policy parameterization from
\cite{borsa2018universal}, but introduce a novel representation with better
properties, based on state occupancy prediction instead of expected
featurizations.
We use theoretical advances on successor state learning from
\cite{successorstates}. We obtain the following.

\begin{compactenum}[\hspace{0pt}\textbullet]
    \setlength{\itemsep}{0pt}
\item We prove the existence of a learnable ``summary'' of a reward-free
discrete or continuous MDP, that provides an explicit formula for optimal
policies for any reward specified later. This takes the form of a pair of
representations $F\from S\times A\times Z\to Z$ and $B\from S\times A\to Z$
from state-actions into a representation space $Z\simeq {\mathbb{R}}^d$, with
policies $\pi_z(s)\deq \argmax_a \transp{F(s,a,z)}z$ .
Once a reward is
specified, a value of $z$ is computed from reward values and $B$;
then $\pi_z$ is used.
Rewards may be specified either explicitly as a function, or as target
states, or by samples as in usual RL setups.

\item We provide a well-principled unsupervised loss for $F$ and
$B$: if training is perfect, then the policies are provably
optimal for all rewards (Theorem~\ref{thm:main}). \option{In finite spaces, perfect training is
possible with large enough dimension $d$
(Proposition~\ref{prop:existence}).} With imperfect
training, sub-optimality is proportional to the training error
(Theorem~\ref{thm:approx}).

\item We provide a TD-like algorithm to train $F$ and $B$ to minimize
this
loss, with function approximation, adapted from recent methods for
successor states \cite{successorstates}. As usual with TD, learning seeks
a fixed point but the loss itself is not observable.

\item We prove viability of the method on several environments from mazes
to pixel-based MsPacman and a virtual robotic arm.  For single-state
rewards (learning to reach arbitrary states), we compare this method to
goal-oriented methods such as HER.  (Our method is not a substitute
for HER: in principle they could be combined, with HER improving replay
buffer management for our method.) For more general rewards, which cannot
be tackled a posteriori by trained goal-oriented models, we provide
qualitative examples.

\item We also illustrate qualitatively the sub-optimalities
(long-range behavior is preserved but local blurring of rewards occurs)
and the representations learned.
\end{compactenum}

\paragraph{Related work.} \NDY{Move whole section to end? difficult to
understand
without knowing our method}
\cite{borsa2018universal} learn optimal policies for rewards that
are linear combinations of a finite number of feature
functions provided in advance by the user.
\NDY{a bit repetitive of intro}
The approach of \cite{borsa2018universal} cannot tackle generic
rewards or
goal-oriented RL: this would require introducing one feature per possible
goal state, requiring infinitely many features in continuous
spaces.

Our approach does not require user-provided features describing the
future tasks, thanks to using successor \emph{states}
\cite{successorstates} where
\cite{borsa2018universal} use successor \emph{features}. Schematically,
and omitting actions, successor features start with user-provided
features $\phi$, then learn $\psi$ such that $\psi(s_0)=\sum_{t\geq 0}
\gamma^t \E [\phi(s_t)\mid s_0]$.
This limits applicability to rewards that are
linear combinations of $\phi$.  Here we use successor \emph{states},
namely, we learn two representations $F$ and $B$ such that
$\transp{F(s_0)}B(s')=\sum_{t\geq 0} \gamma^t \Pr(s_t=s'\mid s_0)$.  This does
not require any user-provided input.

\option{Thus we learn two representations instead of one. The learned
backward representation $B$ is absent from \cite{borsa2018universal}. $B$
plays a different role than the user-provided features $\phi$ of
\cite{borsa2018universal}: learning of $F$ given $B$ is not learning of
$\psi$ given $\phi$. The features $\phi$ of
\cite{borsa2018universal} are split between our learned $B$ and the
functions $\phi$ we can use if the reward is known to depend on only part of
the state.}

We use a similar parameterization of policies by $\transp{F(s,a,z)}z$ as
in \cite{borsa2018universal}, for similar reasons, although $z$
encodes a different object.

Successor representations where first defined in
\cite{dayan1993improving} for finite spaces\option{, corresponding to an older
object from Markov chains, the
fundamental matrix \cite{kemenysnell1960,
bremaud1999markov,grinstead1997introduction}}.
\option{\cite{stachenfeld2017hippocampus} argue for their relevance for cognitive
science.}
For successor
representations in continuous spaces, a
finite number of features $\phi$ are specified first; this can be used
for generalization within a family of tasks, e.g.,
\cite{barreto2017successor,zhang2017deep,grimm2019disentangled,hansen2019fast}.
\cite{successorstates} moves from successor features to successor states
by providing pointwise occupancy map estimates even in continuous spaces, without using
the sparse reward $\1_{s_t=s'}$.
We adapt successor state learning algorithms from \cite{successorstates},
which also introduced simpler versions of $F$ and $B$ for a single, fixed
policy.

There is a long literature on goal-oriented RL. For instance,
\cite{pmlr-v37-schaul15} learn goal-dependent value functions\option{,
regularized via an
explicit matrix factorization}. Goal-dependent value functions have been investigated in earlier works such as~\cite{foster2002structure} and~\cite{sutton2011horde}. Hindsight experience replay (HER)~\cite{andrychowicz2017hindsight} improve the sample efficiency of multiple goal learning with sparse rewards.
A family
of rewards has to be specified beforehand, such as reaching arbitrary
target states.  Specifying rewards a posteriori is not possible:
for instance, learning to reach target states does not extend to reaching
the nearest among several goals, reaching a goal while avoiding forbidden
states, or maximizing any dense reward.

For finite state spaces, \cite{jin2020reward}
provide an algorithm to constitute a
training set by reward-free interactions with a finite environment, such
that any optimal policies later computed on this training set instead of
the true environment are provably $\varepsilon$-optimal, for any reward.  They
prove tight bounds on the necessary set size. Planning still
has to be done for each reward.

\NDY{Mahadevan 2007 proto value functions + subsequent work? maybe not.
builds a basis for value functions by a small rank approx of the
Laplacian of a discretized MDP, similar to FB, but
still needs to learn value function afterwards via least squares policy
iteration, so it's different}

\paragraph{Acknowledgments.} The authors would like to thank
Léonard Blier, Diana Borsa, Rémi Munos, Tom Schaul, Corentin Tallec and
Nicolas Usunier for numerous comments, technical questions, references, and
suggestions for the presentation that helped improve the text.

\section{Problem and Notation}
\label{sec:notation}

We consider a reward-free Markov decision process (MDP)
$\mathcal{M}=(S,A,P,\gamma)$ with state space $S$ (discrete or continuous),
action space $A$ (discrete for simplicity, but this is not essential),
transition probabilities $P(s'|s,a)$ from state $s$ to $s'$ given action
$a$,
and
discount factor $0 < \gamma < 1$ \cite{sutton2018reinforcement}.  If
$S$ is finite,
$P(s'|s,a)$ can be viewed as a matrix; in general, for each $(s,a)\in
S\times A$, $P(\d s'|s,a)$ is a probability measure on $s'\in S$.  The
notation $P(\d s'|s,a)$ covers all cases.
\option{All functions are assumed to be measurable.}

Given $(s_0,a_0)\in S\times A$ and a policy
$\pi\from S\to \mathrm{Prob}(A)$, we denote
$\Pr(\cdot|s_0,a_0,\pi)$ and $\E[\cdot|s_0,a_0,\pi]$ the probabilities and expectations under state-action sequences
$(s_t,a_t)_{t \geq 0}$ starting with $(s_0,a_0)$ and following policy $\pi$
in the environment, defined by sampling $s_t\sim P(\d
s_t|s_{t-1},a_{t-1})$ and $a_t\sim \pi(s_t)$.

Given a reward function $r\from S\times A\to {\mathbb{R}}$,
the $Q$-function of $\pi$ for $r$ is $Q_r^\pi(s_0,a_0)\deq \sum_{t\geq 0}
\gamma^t \E [r(s_t,a_t)|s_0,a_0,\pi]$.  We assume that rewards are bounded, so
that all $Q$-functions are well-defined. We state the
results for deterministic reward functions, but this is not
essential.
\option{We abuse notation and write greedy policies as $\pi(s)=\argmax_a Q(s,a)$ instead of
$\pi(s)\in \argmax_a Q(s,a)$. Ties may be broken any way.}

We consider the following informal problem: Given a reward-free
MDP $(S,A,P,\gamma)$, can we compute a convenient object $E$ such that,
once a reward function $r\from S\times A\to {\mathbb{R}}$ is specified, we can
easily (with no planning) compute from $E$ and $r$ a policy $\pi$ such that $Q_r^\pi$ is
close to maximal?

\section{Outline of the Method}
\label{sec:outline}

The method has three phases: an \emph{unsupervised learning phase} where we
learn a pair of representations $F$ and $B$ in a reward-free way, by
observing state transitions in the environment; a
\emph{reward
estimation phase} where we estimate a policy parameter $z_R$ from reward
observations, or directly set $z_R$ if the reward is known (as in
goal-oriented RL); and an \emph{exploitation phase}, where we directly set a policy
based on $F$, $B$, and $z_R$. The last two phases may be
merged in some cases.

\paragraph{The unsupervised learning phase.}
We set a representation space $Z={\mathbb{R}}^d$, used to represent both states and
reward functions.
We learn a pair of ``forward'' and ``backward'' representations
\begin{equation}
F\from S\times A\times Z\to Z,
\qquad
B\from
S\times A \to Z
\end{equation}
For each $z\in Z$, we define the policy $\pi_z(s)\deq
\argmax_a \transp{F(s,a,z)}z$.

No rewards are used in this phase, and no
family of tasks has to be specified manually.
$F$ and $B$ are trained such that, for any
state-actions $(s_0,a_0)$, $(s',a')$, and any $z\in Z$,
\begin{equation}
\label{eq:FBprobs}
\transp{F(s_0,a_0,z)}B(s',a')\approx \sum_{t\geq 0} \gamma^t \Pr
\left( s_t=s',\,a_t=a' \mid
s_0,\,a_0,\,\pi_z\right)
\end{equation}
(We actually learn probability densities,
which make more sense in continuous spaces, see
\eqref{eq:defM}--\eqref{eq:defm}.)
Since $\pi_z$ depends on $F$, \eqref{eq:FBprobs} is a fixed point
equation\option{, similarly to the dependency between the optimal $Q$-function
and optimal policy in ordinary $Q$-learning}.

Intuitively, $F$ is a representation of the future of a state
under a certain policy. $B$ represents \option{the past of a state, or} the ways to
reach that state (see also Appendix~\ref{sec:succpred}). If $\transp{F}B$ is large, then it is possible to
reach the second state from the first.
 This is akin to a
model of the environment, without synthesizing state trajectories.

This unsupervised learning phase is fully principled: If
learning of $F$ and $B$ is successful, then this is guaranteed to provide
all optimal policies (Theorem~\ref{thm:main}). Namely, for any reward
function $r(s,a)$, the optimal policy for $r$ is the policy $\pi_z$
with
$z\deq
\E_{(s,a)} [r(s,a)B(s,a)]$. For instance, $\pi_{B(s,a)}$ is the
optimal policy to reach $(s,a)$.

Approximate solutions still provide approximately optimal policies, with
optimality gap directly proportional to the error on $\transp{F}B$
(Appendix, Theorem~\ref{thm:approx})\option{, so the guarantees are
non-empty}.

$F$ and $B$ can be learned off-policy from observed transitions in the
environment, via the Bellman equation for \eqref{eq:FBprobs},
\begin{equation}
\transp{F(s_0,a_0,z)}B(s',a')=\1_{s_0=s',\,a_0=a'}\,+\gamma \E_{s_1\sim P(\d
s_1|s_0,a_0)} [\transp{F(s_1,\pi_z(s_1),z)}B(s',a')]
\end{equation}
in discrete spaces.
We leverage
efficient algorithms for \emph{successor states} \cite{successorstates},
also valid in continuous spaces,  and which learn without using the
sparse reward $\1_{s_0=s',\,a_0=a'}$ (see Section~\ref{sec:thm}).

In finite spaces, exact solutions  $F$ and $B$ exist (Appendix,
Prop.~\ref{prop:existence}), provided the dimension $d$ is large enough.
In infinite spaces, arbitrarily good approximations can be obtained by
increasing $d$, corresponding to a rank-$d$ approximation of the pairwise
cumulated transition probabilities in \eqref{eq:FBprobs}.
The dimension $d$ controls how many types of rewards can be optimized
well;
even a relatively small $d$ can provide useful behaviors, see the
experiments and Appendix~\ref{sec:dimension}.
The algorithm is linear
in $d$, so $d$ can be taken as large as the neural network models can
handle.

For exploration in this phase, we use the policies being learned: the
exploration policy chooses a random value of $z$
from some distribution (e.g., Gaussian), and
follows $\pi_z$ for some time (Appendix,
Algorithm~\ref{FB algo}). However, the algorithm can also work from
an existing dataset of off-policy transitions.

\option{The optimality guarantee and training algorithm are detailed in
Section~\ref{sec:thm}.}

\paragraph{The reward estimation phase.} Once rewards are available, we estimate a reward
representation (policy parameter) $z_R$ by weighing the representation $B$
by the reward:
\begin{equation}
\label{eq:zR_outline}
z_R\deq\E [r(s,a) B(s,a)]
\end{equation}
where the expectation must be computed over the same distribution of state-actions
$(s,a)$
used to learn $F$ and $B$. There are several scenarios.

If the reward is known explicitly, this phase is unnecessary. For
instance, if the reward is to reach a target state-action $(s_0,a_0)$ while avoiding some
forbidden state-actions $(s_1,a_1),...,(s_k,a_k)$, one may directly set
\begin{equation}
z_R=B(s_0,a_0)-\lambda \sum B(s_i,a_i)
\end{equation}
where the constant $\lambda$ adjusts the negative reward for visiting a forbidden state.
This can be used for goal-oriented RL.

If the reward is known algebraically as a function $r(s,a)$, then $z_R$ may be computed by averaging the function $r(s,a)B(s,a)$ over a
replay buffer from the unsupervised training phase.

If the reward is black-box as in standard RL
algorithms, then the exploration policy has to be run again for some
time, and $z_R$ is obtained by averaging
$r(s,a)B(s,a)$ over the states visited.

If it is not possible to run the
exploration policy again,
one may learn a model $\hat r(s,a)$ of $r(s,a)$ on some reward
observations from any source,
then estimate $z_R=\E
[\hat r(s,a) B(s,a)]$ by averaging the model over a replay buffer from the
unsupervised training phase.
In this case, reward
identification may be done in parallel to exploitation.

An approximate value for $z_R$ still provides an approximately
optimal policy (Appendix, Prop.~\ref{prop:approxr} and
Thm.~\ref{thm:approxzR}).

\option{Since reward functions are represented by $d$-dimensional
vectors $z_R$, loss of precision necessarily occurs. In practice, for
small $d$ we notice
some
blurring of rewards between nearby states (Fig.~\ref{figure: grid heatmap
}), for reasons discussed in
Section~\ref{sec:thm}.}

\paragraph{The exploitation phase.} Once the reward representation $z_R$
has been
estimated, the $Q$-function is estimated as
\begin{equation}
Q(s,a)=
\transp{F(s,a,z_R)}\,z_R.
\end{equation}
The corresponding policy $\pi_{z_R}(s)=\argmax_a
Q(s,a)$ is used for exploitation.

Fine-tuning was not needed in our experiments, but it is possible to fine-tune the
$Q$-function using actual rewards,
by setting $Q(s,a)=
\transp{F(s,a,z_R)}z_R + q_\theta(s,a)$ where the fine-tuning model
$q_\theta$ is initialized to $0$
and learned via any standard $Q$-learning method.

\section{The Forward-Backward Representation of
an MDP Encodes the Optimal Policies}
\label{sec:thm}

We now present in more detail the algorithm used to learn $F$ and $B$, and the
associated theoretical guarantee.

\paragraph{Background on successor states.}  We start with some definitions and
an algorithm from \cite{successorstates} for successor states of a single policy.
The successor state operator $M^\pi$ of a policy $\pi$ is defined as
follows. For each state-action $(s_0,a_0)$, $M^\pi(s_0,a_0,\cdot)$ is a measure
over the state-action space $S\times A$, representing the expected
discounted time spent in each set $X\subset S\times A$, namely,
\begin{equation}
\label{eq:defM}
M^\pi(s_0,a_0,X)\deq \sum_{t\geq 0} \gamma^t \Pr\left((s_t,a_t)\in X \mid
s_0,\,a_0,\,\pi
\right)
\end{equation}
for each $X\subset S\times A$.  Viewing
$M$ as a measure deals with both discrete and continuous spaces.

In practice, $M^\pi$ is represented by a function $m^\pi(s_0,a_0,s',a')$ taking a pair of
state-actions and returning a number.
Namely,
assume access to a training set of transitions in the MDP (off-policy,
reward-free). Let $
\rho(\d s,\d a)$ be the probability distribution of state-actions in this
training set.
The measure $M^\pi(s_0,a_0,\cdot)$ may be represented as a
density with respect to $\rho$:
\begin{equation}
\label{eq:defm}
M^\pi(s_0,a_0,\d s',\d
a')\eqd m^\pi(s_0,a_0,s',a')\,\rho(\d s',\d a')
\end{equation}
where $m^\pi$ is an ordinary
function (or a distribution, see Appendix~\ref{sec:diracs}).

Algorithms exist to train
a parametric model $m^\pi_\theta(s_0,a_0,s',a')$
of $m^\pi$
from a training set of off-policy observed
transitions $(s_0,a,s_1)$ in the MDP, based on a Bellman-like equation for
$M$ \cite{successorstates}. Namely, sample a transition
$(s_0,a,s_1)$ from the training set, generate an action $a_1\sim
\pi(a_1|s_1)$,
sample a
state-action $(s',a')$ from the training set, independently from
$(s_0,a,s_1)$. Then update the parameter $\theta$ by
$\theta\gets \theta+\eta \del \theta$ with learning rate $\eta$ and
\begin{equation}
\label{eq:mtd}
\del \theta\deq \partial_\theta m^\pi_\theta(s_0,a,s_0,a) +
\partial_\theta m^\pi_\theta(s_0,a,s',a')
\,\times \left(
\gamma \,m^\pi_\theta(s_1,a_1,s',a')
-m^\pi_\theta(s_0,a,s',a')\right)
\end{equation}
The true successor state density $m^\pi$ is a fixed point of this update
in expectation \cite{successorstates} \option{(it is the only fixed point in the tabular or
overparameterized case)}.
Variants exist, such as using a target network for
$m^\pi_\theta(s_1,a_1,s',a')$ on the right-hand side, as in DQN.

The naive way to estimate successor states via a Bellman equation from
\eqref{eq:FBprobs} would
involve a sparse reward term $\1_{s_0=s',\,a=a'}$, resulting in
infinitely sparse rewards in continuous spaces. However, in expectation,
this term's contribution is known algebraically:
\eqref{eq:mtd} avoids sparse rewards in a principled way thanks to
the
$\partial_\theta m^\pi_\theta(s_0,a,s_0,a)$ term
\cite{successorstates}.

It is not necessary to know the training distribution $\rho$: all
quantities are expressed as expectations under $\rho$, namely,
expectations over the training set. When training under data
distribution $\rho$, one learns the density $m^\pi$ of $M^\pi$ with respect
to $\rho$; this will be useful below.

\paragraph{The forward-backward representation provides the optimal
policies.}
The next theorem contains the core idea of our algorithms. In short, if we can learn
two representations $F$ and $B$ of state-actions such that $\transp{F}B$
approximates the successor states of certain policies,
then we can compute all optimal policies from $F$ and $B$.

\begin{thm}[Forward-backward representation of an MDP]
\label{thm:main}
Consider an MDP with state space $S$ and action space
$A$.
Let $Z={\mathbb{R}}^d$ be some representation space. Let
\begin{equation}
F\from S\times A\times Z\to Z,\qquad B\from S\times A\to Z
\end{equation}
be two functions. For each $z\in Z$, define the policy
\begin{equation}
\pi_z (s) \deq \argmax_a \transp{F(s,a,z)}z.
\end{equation}
Let $\rho$ be any probability distribution on $S\times
A$ (e.g., the distribution of state-actions under some exploration
scheme).

Assume that $F$ and $B$ \option{have been chosen (trained) to} satisfy the
following: for any $z\in Z$, and any state-actions $(s,a)$
and $(s',a')$, the quantity $\transp{F(s,a,z)}B(s',a')$ is equal
to the successor state density $m^{\pi_z}(s,a,s',a')$ of policy
$\pi_z$ with respect
to $\rho$.

Then, for any bounded reward function $r\from S\times A\to {\mathbb{R}}$, the following
holds. Set
\begin{equation}
\label{eq:zR}
z_R\deq \E_{(s,a)\sim \rho} \left[r(s,a)B(s,a)\right].
\end{equation}
Then $\pi_{z_R}$ is an optimal policy for reward $r$ in the MDP.
Moreover, the optimal $Q$-function $Q^\ast$ for reward $r$ is
\begin{align}
Q^\ast (s,a)&=\transp{F(s,a,z_R)}z_R.
\end{align}
\end{thm}

The theoretical guarantee extends to approximate training of $F$
and $B$\option{, with optimality gap
proportional to the $\transp{F}B$ training error}. This is important, as
exact finite-$d$ representations are often impossible in continuous
spaces. Namely: if, for some reward $r$, the error
$\abs{\transp{F(s,a,z_R)}B(s',a')-m^{\pi_{z_R}}(s,a,s',a')}$ is at
most $\varepsilon$ on average over $(s',a')\sim \rho$ for every $(s,a)$, then
$\pi_{z_R}$ is $3\varepsilon\norm{r}_\infty/(1-\gamma)$-optimal for $r$ (Appendix,
Theorem~\ref{thm:approx}\option{, with additional results on weaker norms}).

\option{A similar statement holds if we use $\bar
m(s,z,s',a')+\transp{F(s,a,z)}B(s',a')$ instead of just
$\transp{F(s,a,z)}B(s',a')$ to learn successor states (Appendix,
Theorem~\ref{thm:main2}). Here $\bar
m$ is any function that does not depend on $a$. Since $\bar m$ has no
rank restriction, the finite rank approximation only applies to the
advantage function.} \NDY{move to appendix?}

\paragraph{Learning $F$ and $B$.}
Theorem~\ref{thm:main} suggests to choose a parametric model for the
representations $F$ and $B$,
and then train $F$ and
$B$
such that $\transp{F}B$ approximates the successor state density $m$,
via any successor state learning algorithm.

This unfolds as follows (Appendix, Algorithm~\ref{FB algo}). Assume access to a training dataset of
transitions $(s_0,a,s_1)$ in the MDP, e.g., from some exploration
policies.
The samples $(s_0,a)$ in this dataset follow some
unknown distribution $\rho$; we make no assumptions on $\rho$. Then:
\begin{compactenum}[\hspace{0pt}\textbullet]
    \setlength{\itemsep}{0pt}
\item In
 the unsupervised learning phase, train $F$ and $B$ so that $\transp{F}B$ approximates the
successor state density $m^{\pi_z}$ for every $z$. This can use
any successor state learning algorithm, adapted to incorporate
$z$. Here we use \eqref{eq:mtd}.

For this,
choose a parametric model $F_\theta,
B_\theta$ for the representations $F$ and $B$.
At each step, pick a $z$ at random, pick a
batch of transitions $(s_0,a,s_1)$ from the training set, set the next
actions
$a_1\deq \pi_z(s_1)$, and pick a batch of state-actions
$(s',a')$ from the training set, independently from $z$ and
$(s_0,a,s_1)$. Then apply the successor
state update \eqref{eq:mtd} to
the parametric model
$m_\theta^{\pi_z}(s_0,a,s',a')=\transp{F_\theta(s_0,a,z)}B_\theta(s',a')$
(Algorithm~\ref{FB algo}).
\NDY{I'm not in favor of giving more
formulas here}

For sampling $z$, we use a fixed distribution (rescaled Gaussians, see
Appendix~\ref{sec:setup}). Any number of values of $z$ may be sampled: this does not
use up training samples. We use a target network with soft updates
(Polyak averaging) as
in DDPG. For training we also replace the greedy policy
$\pi_z=\argmax_a \transp{F(s,a,z)}z$ with a regularized
version $\pi_z= \mathrm{softmax}(\transp{F(s,a,z)}z/\tau)$ with fixed
temperature $\tau$ (Appendix~\ref{sec:setup}). Since there is
unidentifiability between $F$ and $B$ (Appendix, Remark~\ref{rem:normB}),
we normalize $B$ via an auxiliary loss in Algorithm~\ref{FB algo}.

\NDY{? discuss variance from picking $(s',a')$ independently from the rest; no
easy solution apart fom heuristics that will produce HER-style bias.
Maybe a bit specialized}

\item In the reward estimation phase, we estimate the reward representation $z_R$ via
\eqref{eq:zR}, following one of the use cases from
Section~\ref{sec:outline}. The distribution $\rho$ in \eqref{eq:zR} is
the same as the one used to define the density $m^\pi$; thus, $z_R$
must be estimated over the same distribution of state-actions used to
train $F$ and $B$.

If the reward is known directly, $z_R$ can be
set by hand following \eqref{eq:zR}. For instance, for the optimal policy to reach state-action $(s,a)$, we directly
set $z_R=B(s,a)$. \option{(A unit reward at $(s,a)$ corresponds to
$z_R=B(s,a)/\rho(s,a)$ in \eqref{eq:zR}, but $\rho$ is unknown and scaling the reward yields
the same optimal policy.)} \NDY{could do an appendix to explain that
pointwise rewards in continuous spaces should be treated as Dirac
measures, so actually there's no need for $\rho$ in that case}
This extends to finite combinations of states.
If $r$ is provided as a function, then $\E_{(s,a)\sim \rho} [r(s,a)B(s,a)]$
may be estimated directly on $(s,a)$ in a replay buffer. The case of $r$
accessible only by samples has been discussed in
Section~\ref{sec:outline}. \NDY{I did what I could to clarify after Nico's
remarks}

Stochastic rewards may be used to estimate $z_R$. Indeed, if $\E
\tilde r(s,a)=r(s,a)$ then $\E[\tilde
r(s,a)B(s,a)]=r(s,a)B(s,a)$.

\item In the exploitation phase, the exploitation
policy can be directly set to $\pi_{z_R}$, which selects the action
$\argmax_a \transp{F(s,a,z_R)}z_R$.
\end{compactenum}

\paragraph{Incorporating prior information.} Trying to plan in
advance for all possible rewards in an arbitrary environment may be too
generic and problem-agnostic, and become difficult in large environments,
requiring long exploration and a large $d$ to accomodate all rewards.
Prior information can be incorporated in several ways into the $FB$
method,
by focussing on certain types of rewards or parts of the space.

First, $F$ and $B$ may be trained only on values of $z_R$
corresponding to
a category of rewards specified in advance.
For instance, training only on $z_R=B(s)$ with $s$ in some set $S'\subset
S$ learns the optimal
policies for reaching all states $s\in S'$. Any
prior (e.g., Bayesian) on rewards may be incorporated this way. In our
experiments, $z$ was just sampled from a rescaled Gaussian.

Second, we are often interested in rewards depending, not on the full
state, but only on a part or some features of the state (e.g., a few
components of the state, such as the position of an agent, or its
neighbordhood, rather than the full environment). If this is known in
advance, the representation $B$ can be trained on that part of the state
only, with the same theoretical guarantees (Appendix,
Theorem~\ref{thm:main2}). $F$ still needs to use the full state
as input.  This way, the FB model \eqref{eq:FBprobs} does not have to learn
how often every $(s',a')$ is reached, only the part of interest in
$(s',a')$.
More generally, if $\phi\from S\times A
\to G$ is a feature map to some features $g=\phi(s,a)$, and if we know
that the reward will be a function $R(g)$, then the same theorem holds
with $B(g)$ everywhere instead of $B(s,a)$, and with the successor
density $m^\pi(s,a,g)$ instead of $m^\pi(s,a,s',a')$ (Appendix,
Theorem~\ref{thm:main2}). \option{Learning the
latter is done
by replacing $\partial_\theta m_\theta^\pi(s_0,a,s_0,a)$ with
$\partial_\theta m_\theta^\pi(s_0,a,\phi(s_0,a))$ in the first term in
\eqref{eq:mtd} \cite{successorstates}.}
Rewards can be arbitrary functions of $g$, so this is more general than
\cite{borsa2018universal} which only considers rewards linear in $g$.
For instance, in MsPacman below, we let $g$ be the position of the agent,
so we can optimize any reward function that depends on this position.

Third, if we are interested only in part of the space, focusing the
exploration policy on that part will provide representations $F$ and $B$ that
minimize the loss over that part of the space. This could lead to optimal
policies for rewards ``around a typical set of behaviors'', presumably a
more realistic goal than fully agnostic unsupervised training in complex
environments.
\paragraph{Discussion: What kind of representation is learned? Which
information on the reward gets lost?} As reward functions are represented
by a finite-dimensional object  $z_R$, some approximation is incurred on
rewards
(unless fine-tuning is used at exploitation time). By
\eqref{eq:zR}, a reward is represented by $0$ if and only if it is
$L^2(\rho)$-orthogonal to all components of $B$.
So the representation $B$ is not only
part of a transition probability model, but also a reward regularizer.

The loss associated to \eqref{eq:FBprobs}, and minimized by a
Bellman-like training, is
\begin{equation} \label{eq:loss}
\E_{(s,a)\sim
\rho,\,(s',a')\sim \rho,\,z\sim P_z}
\abs{\transp{F(s,a,z)}B(s',a')\,-
\textstyle \sum_{t\geq 0}
\gamma^t P_t(\d
s',\d a'\mid s,a,\pi_z)/\rho(\d s',\d a')}^2
\end{equation}
where $P_t$
is the distribution of states reached at step $t$ by policy $\pi_z$
starting at $(s,a)$,
and $P_z$ is some user-chosen distribution,
such as a Gaussian or a prior over tasks (see above).

Thus, $F$ and $B$ provide a rank-$d$ approximation to the transition
probabilities in the environment.  For a single $z$, this loss is
minimized when $F$ and $B$ are the SVD of the cumulated  transition
matrix $\sum_t \gamma^t P_t$, truncated to the largest $d$ singular
values.
The SVD is computed in the $L^2(\rho)$ norm: the
data distribution influences the representations learned by focusing on
the part of the space covered by $\rho$.

For a fixed policy, on a finite space, $\sum_t \gamma^t P_t=(\Id-\gamma
P)^{-1}$ as matrices. Thus the largest singular values of
$\sum_t \gamma^t P_t$ correspond to the smallest singular values of the
Markov chain Laplacian $\Id-\gamma P$. The associated singular vectors
loosely correspond to
long-range (low-frequency) behavior \cite{mahadevan2007proto}:
presumably, $F$ and $B$ will represent spatially smooth rewards first,
while local detail may be lost (Fig.~\ref{figure: grid heatmap } and Section~\ref{sec:exp}).
\cite{stachenfeld2017hippocampus} argue for the cognitive relevance of
low-dimensional approximations of successor representations.

The features learned in $F$ and
$B$ may have broader interest. \NDY{expand?}

\section{Experiments}
\label{sec:exp}

We first consider the task of reaching arbitrary goal states. For this,
we can make quantitative comparisons to existing goal-oriented baselines.
Next, we illustrate qualitatively some tasks that cannot be tackled a
posteriori by goal-oriented methods, such as introducing forbidden
states. Finally, we illustrate some of the representations learned.

\subsection{Environments and Experimental Setup}
We run our experiments on a selection of environments that are diverse in term of state space dimensionality, stochasticity and dynamics.
\begin{compactenum}[\hspace{0pt}\textbullet]
    \setlength{\itemsep}{0pt}
    \item Discrete Maze is the classical gridworld with four rooms. States are represented by one-hot unit vectors.
    \item Continuous Maze is a two dimensional environment with impassable walls. States are represented by their Cartesian coordinates $(x,y) \in [0,1]^2$. The execution of one of the actions moves
the agent in the desired direction, but with normal random noise added to the position of the agent.
\item FetchReach is a variant of the simulated robotic arm environment
from~\cite{plappert2018multi} using discrete actions instead of continuous actions.  States are 10-dimensional vectors consisting of positions and velocities of robot joints.
\item Ms.\ Pacman is a variant of the Atari 2600 game Ms.\ Pacman, where an episode ends when the agent is captured by a monster~\cite{rauber2018hindsight}.
States are obtained by processing the raw visual input directly from the screen. Frames
are preprocessed by cropping, conversion to grayscale and downsampling to $84\times84$ pixels. A state $s_t$ is the concatenation of $(x_{t-12}, x_{t-8}, x_{t-4}, x_t)$ frames, i.e. an $84\times84\times4$ tensor.
An action
repeat of 12 is used. As Ms.\ Pacman is not originally a
multi-goal domain, we define the goals as the 148 reachable coordinates
$(x,y)$ on the screen; these can be reached only by learning to avoid monsters.
\end{compactenum}
For all environments, we run algorithms for 800 epochs. Each epoch
consists of 25 cycles where we interleave between gathering some amount
of transitions, to add to the replay buffer, and performing 40 steps of
stochastic gradient descent on the model parameters. To collect
transitions, we generate episodes using some behavior policy. For both
mazes, we use a uniform policy while for FetchReach and Ms.\ Pacman, we
use an $\epsilon$-greedy policy with respect to the current approximation $F(s, a, z)^\top z$ for a sampled $z$. At evaluation time, $\varepsilon$-greedy policies are also used, with a smaller
$\varepsilon$. More details are given in Appendix~\ref{sec:setup}.

\subsection{Goal-Oriented Setting: Quantitative Comparisons}
\begin{figure}[t]
    \centering
    \includegraphics[width=0.43\textwidth]{grid/learning_curve_small.pdf}
     \includegraphics[width=0.43\textwidth]{grid/final_score.pdf}

    \caption{\label{figure: perf in discrete maze} \small Comparative performance of FB for different dimensions and DQN in the discrete maze. \textbf{Left}: the policy quality averaged over 20 randomly selected goals as function of the the first 200 training epochs. \textbf{Right}: the policy quality averaged over the goal space after 800 training epochs.}
\end{figure}

\begin{figure}[t]
    \centering
    \includegraphics[width=0.43\textwidth]{pacman/learning_curve_small.pdf}
     \includegraphics[width=0.43\textwidth]{pacman/final_score.pdf}

    \caption{ \label{figure: perf in pacman} \small Comparative performance of FB for different dimensions and DQN in Ms.\ Pacman. \textbf{Left}: the success rate averaged over 20 randomly selected goals as function of the first 200 training epochs. \textbf{Right}: the success rate averaged over the goal space after 800 training epochs.}
\end{figure}

\begin{figure}[t]
    \centering
    \includegraphics[width=0.35\textwidth]{grid/only_heatmap_FB-25.pdf}
     \includegraphics[width=0.35\textwidth]{grid/only_heatmap_FB-75.pdf}

    \caption{ \label{figure: grid heatmap } \small Heatmap of $\max_{a}
    F(s, a, z_R)^\top z_R$ for $z_R = B(\text{\score{1}{1}})$
    \textbf{Left}: $d = 25$. \textbf{Right}: $d = 75$.}
\end{figure}

We investigate the
FB representation
over goal-reaching tasks and compare it to goal-oriented baselines:
DQN\footnote{Here DQN is short for goal-oriented DQN, $Q(s, a,
g)$.}, and DQN with HER when needed. We define sparse reward
functions. For Discrete Maze, the reward function is equal to one when
the agent's state is equal exactly to the goal state. For Discrete Maze,
we measured the quality of the obtained policy to be the ratio between
the true expected discounted reward of the policy for its goal  and the
true
optimal value function, on average over all states.
For the other environments, the reward function is equal to one when the
distance of the agent's position and the goal position is below some
threshold, and zero otherwise. We assess policies by computing
the average success rate, i.e the average number of times the agent
successfully reaches its goal.

Figs.~\ref{figure: perf in discrete maze} and~\ref{figure: perf in
pacman} show the comparative performance of FB for different dimensions
$d$, and DQN and DQN+HER respectively in Discrete Maze and Ms.\
Pacman (similar results in FetchReach and Continuous Maze are provided in
the Appendix). The performance of FB consistently increases with the
dimension $d$ and the best dimension matches the performance
of the goal-oriented baseline. In Ms.\ Pacman, DQN totally fails to learn and
we had to add HER to make it work.

In Discrete Maze, we observe a drop of performance for $d=25$:
this is due to the spatial smoothing induced by the small rank
approximation and the reward being nonzero only if the agent
is exactly at the goal. This spatial blurring is clear on
heatmaps for $d=25$ vs $d=75$
(Fig.~\ref{figure: grid heatmap }). With $d=25$ the agent often stops
right next to its goal.

To evaluate the sample efficiency of FB, after each epoch, we
evaluate the agent on 20 randomly selected goals. Learning curves are
reported in Figs.~\ref{figure: perf in discrete maze} and~\ref{figure:
perf in pacman} (left). In all environments expect FetchReach, we observe
no loss in sample efficiency compared to the goal-oriented baseline. In
Ms.\ Pacman, FB even learns faster than DQN+HER.

\subsection{More Complex Rewards: Qualitative Results}

\begin{figure}[t]
    \centering
    \includegraphics[width=0.25\textwidth,  trim = {1cm 1cm 1cm 1cm}]{grid/traj_locked_door_FB-100.pdf}
     \includegraphics[width=0.25\textwidth, trim = {1cm 1cm 1cm 1cm}]{continuous/traj_neg_FB-300.pdf}
     \includegraphics[width=0.25\textwidth,  trim = {1cm 1cm 1cm 1cm}]{pacman/neg_FB-300.pdf}
    \caption{\label{figure: neg reward} \small Trajectories generated by
    the
    $\transp{F}B$ policies for the task of reaching a target position
    (star shape  \score{1}{1}) while avoiding forbidden positions
    (red shape \protect\tikz\protect\draw[red,fill=red] (0,0) circle (.7ex);).s
    }
\end{figure}

\begin{figure}[t]
    \centering
    \includegraphics[width=0.25\textwidth,  trim = {1cm 1cm 1cm 1cm}]{grid/traj_nearest_FB-100.pdf}
     \includegraphics[width=0.25\textwidth, trim = {1cm 1cm 1cm 1cm}]{continuous/traj_nearest_FB-300.pdf}
     \includegraphics[width=0.25\textwidth,  trim = {1cm 1cm 1cm 1cm}]{pacman/nearest_FB-300.pdf}
    \caption{\label{figure: equal reward} \small Trajectories generated
    by the $\transp{F}B$ policies for the task of reaching the closest among
    two equally rewarding positions (star shapes  \score{1}{1}).
    (Optimal
    $Q$-values are not linear over such mixtures.)}
\end{figure}

\begin{figure}[t]
    \centering

     \includegraphics[width=0.30\textwidth, trim = {1cm 1cm 1cm 1cm}]{continuous/contourf_neg_FB-300.pdf}
    \includegraphics[width=0.30\textwidth, trim = {1cm 1cm 1cm 1cm}]{continuous/contourf_nearest_FB-300.pdf}
    \caption{\label{figure: mix reward in continuous maze} \small Contour
    plot of  $\max_{a \in A}F(s, a, z_R)^\top z_R$ in Continuous Maze.
    \textbf{Left}: for the task of reaching a target while avoiding a
    forbidden region, \textbf{Right}: for two equally rewarding targets.}
\end{figure}

We now investigate FB's ability to generalize to new tasks that cannot be solved
by an already trained goal-oriented model: reaching a goal with forbidden
states imposed a posteriori,
reaching the nearest of two goals, and choosing between a small, close
reward and a large, distant one.

First, for the task of reaching a target position $g_0$
\score{1}{1} while avoiding some forbidden positions $g_1, \ldots g_k$
\tikz\draw[red,fill=red] (0,0) circle (.7ex);, we  set $z_R =
B(g_1) - \lambda \sum_{i=1}^k B(g_i)$ and run the corresponding
$\varepsilon$-greedy policy defined by $F(s, a, z_R)^\top z_R$.
Fig.~\ref{figure: neg reward} shows the resulting trajectories, which
succeed at solving the task for the different domains. In Ms.\ Pacman,
the path is suboptimal (though successful) due to the sudden
appearance of a monster along the optimal path. (We  only plot the
initial frame; the full series of frames along the trajectory is in
the Appendix.) Fig.~\ref{figure: mix reward in continuous maze} (left)
provides a contour plot of $\max_{a \in A}F(s, a, z_R)^\top z_R$ for the
continuous maze and shows the landscape shape around the forbidden regions.

Next, we consider the task of reaching the closest target among two
equally rewarding positions $g_0$ and $g_1$, by setting $z_R = B(g_0) +
B(g_1)$. The optimal $Q$-function is
\emph{not} a linear combination of the $Q$-functions for $g_0$ and $g_1$.
Fig.~\ref{figure: equal
reward} shows successful trajectories generated by the policy
$\pi_{z_R}$.
On the contour
plot of $\max_{a \in A}F(s, a, z_R)^\top z_R$ in Fig.~\ref{figure: mix
reward in continuous maze} (right), the two rewarding positions appear as
basins of attraction.  Similar results for a third task are shown in the
Appendix: introducing a ``distracting'' small reward next to the initial
position of the agent, with a larger reward further away. The Appendix
includes
embedding visualizations for different $z$ and for Discrete Maze
and  Ms.\ Pacman.

\subsection{Embedding Visualizations}

\begin{figure}[t]
  \centering
  \includegraphics[width=0.26\textwidth]{continuous/env.pdf}
    \includegraphics[width=0.25\textwidth,  trim = {1.5cm 1.5cm 1.5cm 1.5cm}]{continuous/forward_embedding_FB-50.pdf}
     \includegraphics[width=0.25\textwidth,  trim = {1.5cm 1.5cm 1.5cm 1.5cm}]{continuous/backward_embedding_FB-50.pdf}
    \caption{\label{figure: t-sne} Visualization of FB embedding vectors
    on Continuous Maze after projecting them in two-dimensional space
    with t-SNE.
    \textbf{Left}: the states to be mapped. \textbf{Middle}: the $F$
    embedding. \textbf{Right}: the $B$ embedding.}
\end{figure}

We visualize the learned FB state embeddings for Continuous Maze by
projecting them into 2-dimensional space using
t-SNE~\cite{van2008visualizing} in Fig.~\ref{figure: t-sne}. For the
forward embeddings, we set $z=0$ corresponding to the uniform policy. We
can see that FB partitions states according to the topology induced by
the dynamics: states on opposite sides of walls are separated in the
representation space and states on the same side lie together.

\section{Conclusion}

The FB representation is a learnable mathematical object that
``summarizes'' a reward-free MDP. It
provides near-optimal policies for any reward specified a
posteriori, without planning. It is learned from black-box reward-free
interactions with the environment. In practice, this unsupervised method performs comparably to
goal-oriented methods for reaching arbitrary goals, but is also able to
tackle more complex rewards in real time.
\option{The representations learned encode the MDP dynamics and may have
broader interest.}
\NDY{improve?}

\bibliographystyle{alpha}

redpajama_set_name: RedPajamaArXiv


text:
Церква Всіх Святих — парафія і храм греко-католицької громади Гусятинського деканату Бучацької єпархії Української греко-католицької церкви в селі Городниця Чортківського району Тернопільської області.

Історія церкви
У селі у 1772 році було збудовано муровану греко-католицьку церкву Святої Трійці, яка сьогодні належить громаді ПЦУ. Греко-католицька парафія і храм діяли до 1946 року. У 1946—1990 роках парафія і храм належали до РПЦ.

Парафія церкви Всіх Святих села Городниця мас сучасну історію. Вона була утворена в 1993 році парафіянами (понад 170 осіб), які вийшли зі складу православної громади УАПЦ. Церкву об лаштували у тому ж році у напівзруйнованому приміщенні римо-католицького костьолу, який був діючим у 1895—1946 роках. Коли комуністична влада закрила костьол, його використовували, як колгоспний склад. Після проведених реставраційно-ремонтних робіт і освячення храму 20 листопада 1993 року у ньому розпочали проводити богослужіння.

При парафії діють: братство «Матері Божої Неустанної Помочі» і Параманне братство.

Парохи
 о. Василь Вайла (1993—2003),
 о. Олександр Бондаренко (2003—2005),
 о. Ігор Фуга (2005—2007),
 о. Богдан Байда (з 2007).

Джерела

Церкви Чортківського району
Городниця (Чортківський район)
Церкви Всіх Святих

redpajama_set_name: RedPajamaWikipedia


text:
Google starts worldwide campaign to legalize gay marriage, Target donates to gay activists
07/09/2012 Wintery Knight	9 Comments
From The Weekly Standard.
Google yesterday reportedly launched a worldwide gay marriage campaign, officially beginning in Poland and Singapore.
"Google is launching a new campaign called 'Legalize Love' with the intention of inspiring countries to legalize marriage for lesbian, gay, and bisexual people around the world," dot429.com reports. "The 'Legalize Love' campaign officially launches in Poland and Singapore on Saturday, July 7th. Google intends to eventually expand the initiative to every country where the company has an office, and will focus on places with homophobic cultures, where anti-gay laws exist."
A Google spokesman, Mark Palmer-Edgecumbe, tells the website, "We want our employees who are gay or lesbian or transgender to have the same experience outside the office as they do in the office. It is obviously a very ambitious piece of work."
[…]As the Huffington Post notes, "Google has indicated its support of the issue before. In 2008, the company published a blog post opposing a California amendment that would ban same-sex marriage (the amendment passed)."
And here's Target:
For June "Pride Month," dedicated to celebrate lesbian, gay, bisexual, and transgender persons, the Target Corp. announced it would sell "gay pride" t-shirts and donate 100 percent of the proceeds to the Family Equality Council (FEC), a group that, among other issues, supports same-sex marriage and works to remove restrictions against LGBT persons adopting children.
In June, Target Corp. agreed to sell t-shirts in its stores nationwide for $12.99 to help the Family Equality Council raise $120,000 towards the defeat of the Minnesota Marriage Amendment in the Great Lakes State this November, according to Minnesota Public Radio.
The t-shirts, 10 varieties, promoted pro-gay themes such as the rainbow coupled with the word "Pride" or "Harmony" and "Love is Love." A typical ad for one of the t-shirts read: "PRIDE Mens Heather Grey Rainbow Colorblock Crew Graphic T-Shirt. Target will Donate 100% of PRIDE Merchandise Sales to Family Equality Council. Free shipping when you spend $50."
People say that big business is conservative, but it really isn't. Small business is conservative. Big Business is leftist. The more that we conservatives can block political contributions by big unions and big business, the better things will be for the institution of marriage and the rights of children to a stable relationship with a mother and a father.
Big BusinessGay ActivismGay MarriageGay RightsGoogleSame-Sex MarriageTarget
Previous PostWilliam Lane Craig comments on the Higgs Boson "God" particleNext PostAre radical feminists able to court and marry successfully?
9 thoughts on "Google starts worldwide campaign to legalize gay marriage, Target donates to gay activists"
Hmm… I wonder if Google also wants to "legalize" the "love" between siblings. And parents and (grown) children. And grandparents and grandchildren. I mean, we have to expect that eventually 'adult incest' — which is NOT presently treated by the left with acceptance and celebration — will eventually be so, or they risk the charge of hypocrisy with regard to their favorite motto: "whatever two consenting adults do sexually is their own business."
"Slouching toward Gomorrah," indeed ! >:-(
On a side note, I suppose I should consider abandoning my few remaining Google services (Google Reader, Gmail) by way of a personal protest, for whatever good it would do. :-\
Google's view is that any number of people who want to have any kind of relationship should be able to call it marriage. If a willingness to live together is what qualifies two or more people for marriage, then anything goes. That's Google's view, and that's what they promote.
Joshua Postema (@JoshPostema) says:
What is the difference between marriage and any other relationship then, in their view? If you broaden "marriage" enough to include any type of relationship, why even call it marriage anymore?
It won't do you any good, other than symbolism, to abandon your google services until the majority of websites are willing to stop using google anaylitcs as that one piece of javascript allows google to really track you across the web (via IP) and target advertising – they're only real source of income…everything else they do is just icing on the cake.
Now, I would propose that we ban all political donations from any legally created entity – donations should only come from living, breathing, human adults and nothing else.
Chillingworth says:
I disagree, I think we need less legal restrictions on campaign donations, not more—you know, freedom of speech and all that (I'd be happy to make the practical structural case, too)—but I am in favor of people being creeped out by Google and refusing to use its services. I've been holding out and not switching to G-mail for years, partly because I heard they were cooperating with the Communist Chinese government to keep the citizens from freely accessing the Internet, but partly because of a vaguer fear that Google would take over the world. Maybe that's irrational, but, well, here we are.
About all the free services people get from Google, Facebook, etc., someone somewhere said something very interesting: If it's free, you are not the customer; you are the product.
I think that's a good argument. Sometimes I am tempted to support what will work without really thinking about the principle of the matter.
micahklaz says:
A while back I found out (accidentally, mind you) that looking up things like "gay" or "homosexual" on Google makes a rainbow flag thing appear under your search bar. I just tested that now while I wrote this post, and it seems to be gone now. But it happened.
I really like using Google products. I'm on Google Chrome right now. :(
There's always Firefox!
Technically speaking, yes, Firefox is certainly a browser that exists and even functions. I do not see how this is relevant. >_>

redpajama_set_name: RedPajamaCommonCrawl


text:
Clarence M. Kelley
Clarence M. Kelley was born in Kansas City, Missouri in 1911. He received his Bachelor of Arts degree from the University of Kansas in 1936. He continued his education at the University of Missouri, where he earned a law degree in 1940. That same year he joined the Federal Bureau of Investigation as a Special Agent in October 7, 1940. Kelley served in the United States Navy from July 22, 1944, to April 9, 1946, having been granted military leave from the FBI. Returning from military service Kelley was assigned to the Kansas City office, where his performance earned him a promotion to field supervisor.
From July 1953 to July 1957 he served as Assistant Special Agent in Charge at the Houston, Seattle, and San Francisco offices. He was then transferred to the Training and Inspection Division at FBI Headquarters, becoming an Inspector. In December, 1957 he was promoted to Special Agent in Charge of the Birmingham office and was reassigned to the Memphis office in November 1960, where he served as Special Agent in Charge until his retirement from the FBI on October 24, 1961.
After retiring from the FBI, Kelley became the Chief of Police in Kansas City, Missouri. During that time he was a member and officer of several civic associations. He is acknowledged and respected as an innovator who created the first helicopter division as well as installing the first computer systems in the Kansas City Police Department. In 1970 Kelley received the J. Edgar Hoover Gold Medal for Outstanding Job Service, presented by the Veterans of Foreign Wars. The following year he was named to the Presidential Advisory Committee, and served on both the National Advisory Commission on Criminal Justice Standards and Goals and on the FBI National Academy Review Committee from 1972-1973.
On June 7, 1973, President Richard M. Nixon nominated Kelley to be Director of the FBI. The United States Senate confirmed the nomination June 27 and he was sworn in on July 9. Kelley was the first director of the FBI to be appointed through the nomination and confirmation process. Kelley announced his intention to retire in 1977, prompting an exhaustive year-long search for a successor.
In 1982, Clarence Kelley returned home to Kansas City and founded Clarence M. Kelley and Associates (CMKA). CMKA has grown to be the Midwest's leader in security solutions and risk management providing highly professional armed and unarmed guard services, executive protection and investigations of all types.
Meritorious Service Award
Clarence M. Kelley and Associates sponsors this award. It is presented twice per year to recognize a member of the chapter who shows "dedication to an exemplary career in law enforcement". The CMK award will recognize a Kansas member during the chapter's Spring Re-Trainer and a Missouri member during the chapter's Fall Re-Trainer. The first award was presented during the Fall Session, 2001.
Meritorious Service Award Recipients
2019 Kansas Recipient
Edwin Pavey
Alvin Thimmesch
Kenneth Francis
Sidney Mitchell
William "Bill" Cornwell
Bob Odell
John Meier
Sam Breshears
Randy Adair
Bruce Mellor
Ellen Hanson
Mark Goodloe
William Delaney
Gary Steed Missouri Recipient
Clarence Hoffman
L. Earle Conner
Howard Brown
Robert Ahring
Kristopher Turnbow
Sue Ash
Randy Boehm
Dave Burlew
David Zimmerman
Scott Rhoad
Steve Beamer
Mike Hasty
James Simpson
Spring 2019 Clarence M. Kelley Award Recipient Fall 2018 Clarence M. Kelley Award Recipient
Gary Steed James Simpson


text:
using System;
using FFImageLoading.Work;

namespace FFImageLoading.Transformations
{
    [Preserve(AllMembers = true)]
	public class TintTransformation : ITransformation
	{
		public TintTransformation()
		{
			Helpers.ThrowOrDefault();
		}

		public TintTransformation(int r, int g, int b, int a)
		{
			Helpers.ThrowOrDefault();
		}

		public TintTransformation(string hexColor)
		{
			Helpers.ThrowOrDefault();
		}

		public bool EnableSolidColor { get; set; }

		public string HexColor { get; set; }

		public int R { get; set; }

		public int G { get; set; }

		public int B { get; set; }

		public int A { get; set; }

        public IBitmap Transform(IBitmap sourceBitmap, string path, ImageSource source, bool isPlaceholder, string key)
		{
			return Helpers.ThrowOrDefault<IBitmap>();
		}

		public string Key => Helpers.ThrowOrDefault<string>();
	}
}

redpajama_set_name: RedPajamaGithub


text:
Barbara's Club
Remember your dreams
About Barbara Sher
Contact Barbara
Member Profile / Login
Hanging Out Registration Re-Opens in April 2021
Hanging Out will be the jewel in the crown of Barbara's Club. To be honest, I think it's going to be the jewel in the crown of my lifetime's work. That's what it feels like. —Barbara Sher, 2012
Barbara called this members-only program Hanging Out, because she wanted it to be like those wonderful moments just hanging out together at her retreats and weekend workshops. Patty Newbold will be your guide now that Barbara is no longer with us.
Registration Re-Opens April 8, 2021
$197 for a full year
Barbara put together her most beloved ideas and stories and wrote her best stuff ever for Hanging Out. This was bigger than writing another book. It was a huge job, but every minute was a labor of love for her. And with this program she gives you videos from her retreats and workshops, loads of audio, new interactive stuff and other amazing things we can't describe here or we'll go on forever. All this in addition to the exercises that made a big difference to you in the past.
After almost a year HO is still my favourite thing on the internet and in my inbox. So many things I could just read over and over and find something each time. The world didn't change but I'm different in it. -JD
I'm excited about this exercise. I've printed it out to work on over the weekend. Barbara, I look forward to working it through and circling back. You're a gemstone! -A Hanging Out member in Week 38
Once you're a member, the rest will be easy. Every week, three times a week, via email, I'll send you some very special things to read, watch and do. They're not what you expect, I'm pretty sure of that. As much as you may like my books, what you're going to find in Hanging Out is very new and very different. You'll want to make comments, compare notes and talk to the other members of Hanging Out, and that's exactly what happens here. It changes the whole experience into something entirely new, something most of us are missing and need as much as vitamins and sunshine. The effect on most members has been profound.
You can't buy Hanging Out all at once like a book that ends up on that pile of books you plan to read, and you can't stick it back on your bookshelf and forget what your life is supposed to be. Barbara learned too much about what's important for us humans to take that kind of chance.
You'll receive emails from her every Monday, Wednesday and Friday for a year, announcing the day's video, audio message, Identity Exercise, Reality Series success story or Brain Candy she prepared for you. Barbara's Director of Sher Success Teams and WriteSpeak Coach, Patty Newbold, will be reading and responding to your online comments. You'll also be able to read Barbara's replies to comments from our first eight years of Hanging Out.
Hanging out has really developed clarity in my life. One thing that I love about you Barbara is that you never tell anyone to just try harder and you don't shame in any way. You support and encourage which is so very rare. You make sense of the non-sense. This group has provided something wonderful for me. Even the subtle messages are always very powerful. —Michelle
It's easier to say what it's not. Hanging Out is not a how-to course or a motivation program. I sometimes think it's designed for people who have worked too hard and forgot what it's like to wake up in a goal-free world full of fresh, stimulating ideas, fun and camaraderie.
I'll always have courses for you on Barbara's Club so you can find your dreams and achieve your goals, but Hanging Out is not about goal achievement. Instead, it's more like what one member called "a profound and delicious form of Identity Rehab."
I absolutely love the idea of Barbara's Club being integrated into the everyday world—nothing to leave behind—and a consistent source of inspiration. Thank you! —Victoria, by email
If you join us here, Barbara's emails and online messages will be popping into your life briefly, happily and relentlessly, over and over, waking up your senses, making you laugh and go exploring and, most important of all, helping you remember who you are and how good life is supposed to be.
You'll receive emails from Barbara every Monday, Wednesday and Friday for a year, announcing the day's video, audio message, Identity Exercise, Reality Series success story or Brain Candy she prepared for you. Barbara's Director of Sher Success Teams and WriteSpeak Coach, Patty Newbold, will be reading and responding to your online comments. You'll also be able to read Barbara's replies to comments from our first eight years of Hanging Out.
Thank you from the bottom of my heart for creating Hanging Out with Barbara. There have been so many "favorite" posts. One that stands out is your description of how you decorated your apartment with all the glorious references to color. I wanted to sink into one of your armchairs and absorb it all. Finding our originality. Yes. Discovering ways to express it. Yes. Realizing that time is short and excuses long. Yes. Boxing up all our should, would and could and relaxing into today, into now, into this moment. Yes. Finding that joy is the reward for allowing oneself to just be who you are. Yes. So many wonderful lessons I have learned, relearned, and unlearned this year…What I have learned this year has changed my life. For that I am eternally grateful. – A Hanging Out Member at the end of the year
No risk – full refund if you cancel your subscription within the first 30 days
OR pay by the month – $24 per month
Registration Re-Opens on April 8, 2021
Buying a membership for someone else? Purchase a gift certificate. Good for a full year of Hanging Out at the then-current price whenever your recipient registers, which must be during an open registration period.
$197 for a full year membership gift certificate
Gift certificate mailed within 48 hours. Contact webmaster@barbarasclub.com if you do not receive yours.
769 thoughts on "Hanging Out"
Silvia Martinez on December 29, 2020 at 2:07 pm said:
Hi There, so sorry to hear that registration was until 12.27.2020, is there a way to make an exception…Today is 12.29.2020 and is the day that I can pay for the fee.
Thank you for your consideration!
Hi There, so sorry to hear that registration was until 12.27.2020. Is there a way to make an exception, please. Today is 12.29.2020 and is when I can afford to pay the fee.
Patty Newbold on December 29, 2020 at 4:18 pm said:
Silvia, we will open registration again in April. I hope you will join us then.
Kate on December 27, 2020 at 9:18 pm said:
Is there video chatting zooming skypeing etc in Hanging Out or is it all written communication between members?
It's all written.
Michale Estar on December 16, 2020 at 11:47 pm said:
Dear Barbara Sher Club Friends,
I am very happy to receive these prompts throughout the weeks.
I wish to share a gift at this time of a link to a YouTube video that I helped post entitled Plucking – Dr. Margaret Berry, D.C.. In the video Margaret demonstraits a rejuvenating massage technique called Plucking. plucking the ligaments and tendons. She offers a full demonstration of this technique and then a 5 minute shortened version that one can do after showering or even before getting out of bed in the morning. It helped her get back to work as a Chiropractor after a serious car accident. At the time she was told that she would no longer be able to continue to offer chiropractic work due to her injuries. She was able to fully recover. Happy Holidays of Light Greetings, Michale
Anastasia Yuzva on December 16, 2020 at 6:55 am said:
Dear teams,
I didn't have an opportunity to follow the decision of the court in this regard – does anybody know if by now new members are accepted for the Hanging Out that starts on December 27th?
Patty Newbold on December 16, 2020 at 9:47 am said:
New members are being accepted through December 27th. Two buttons to join are on this page. What is not available this time is the monthly payments option. The full year option is accepting members every day, and I expect we'll have a great time together.
Sally T. on December 10, 2020 at 8:53 am said:
I think you should provide a monthly subscription fee option, for those with financial issues.
Ty on December 10, 2020 at 10:51 am said:
Hello Sally,
As noted above and in comments below, the monthly option will be added as soon as the court allows it. The pandemic has slowed the estate settlement process substantially. The new team carrying the ball after Barbara's death hope it occurs by April 2021. Meantime, enjoy Barbara's other materials.
Best regards, Ty
Patty Newbold on December 10, 2020 at 11:27 am said:
Thanks for getting Sally a quick answer to her question, Ty!
Kathleen on October 21, 2020 at 8:15 pm said:
I just saw Barbara's wonderful Ted Talk and I thought " I would like to talk to this sage and down to earth woman". Maybe she can help me find my dream!
Then I saw that she had died this year. I immediately started to cry because death took another person, and death is final.
For a brief moment I had hope that maybe she could help me get through the lonliness and non connection I feel in my life.
Thank you for listening. I really needed to tell Barbara how much I liked her ease and message.
Maybe she can help me from heaven.
Cathi A Thomas on December 16, 2020 at 4:05 pm said:
Kathleen, order Barbara's wonderful books! I have been a follower for years, even went to one of her workshops in Memphis, TN. She was so genuine and I had the pleasure of speaking to her. BUY HER BOOKS! Watch her videos! Keep in touch with all the people who follow her and I guarantee you will fill the void in your life by understanding that we ALL feel the way you do. Just DO IT ! Best of everything to you, dear.
Nemira on October 6, 2020 at 12:31 pm said:
Hi, I wonder why does data for registering always change? Previously, it was October 2, later on, October 8. Now is December 10. It looks the carrot ahead horse nose.
Patty Newbold on October 7, 2020 at 1:54 pm said:
I am so sorry that we have needed to keep pushing it back. We cannot reopen registration until the court has its say about Barbara's will and choice of executor, and we have no control over their backed-up schedule.
Nemira on October 7, 2020 at 6:14 pm said:
Thank you for your answer. Regards, a future member.
I'm delighted that you will be joining us, Nemira!
traveler on September 16, 2020 at 1:04 am said:
I'm interested, too. I'm not sure how diligent I can be about it, all things considered. But it will give me something to look forward to, and a place where I can hang out. 😉
Patty Newbold on September 17, 2020 at 2:50 pm said:
Traveler, we will admit new members just as soon as the judge due to review Barbara's will does so. Barbara and I made plans for all of her online programs to continue with or without her, but we did not figure on a pandemic making the process of appointing her executor such a slow one. Please add your name to her mailing list to be among the first to hear that we're admitting new members again.
LISA on September 14, 2020 at 4:29 pm said:
Is this still going?
No and yes, Lisa. Anyone who started Hanging Out in January, before Barbara's death, is still involved. We will admit new members just as soon as the judge due to review Barbara's will does so. Barbara and I made plans for all of her online programs to continue with or without her, but we did not figure on a pandemic making the process of appointing her executor such a slow one. Please add your name to her mailing list to be among the first to hear that we're admitting new members again.
Your free Kick Starter Mini Class!
Join The online Idea Party!


Here at Inliven Medical, we cater to medical issues arising from joint pain and osteoarthritis. Our expertise is in non-surgical management and treatment of joint injuries and problems and musculoskeletal conditions such as arthritis of the various joints especially in the knee, ankle and hip joints. Our strengths lie in providing innovative solutions to osteoarthritis and other degenerative conditions of the joints especially in early disease. We strongly believe in the interdisciplinary model of management for such lifestyle conditions and we work closely with surgeons, medical specialists, rehabilitation professionals and other allied healthcare specialists to co-manage and deliver the best care for the patient.


I understand that it is now past mid-September, and trust me, I'm totally gung-ho on apple this and pumpkin that – lots of autumn nummy nums to come! It's just that I seem to transition my baking over to fall more quickly and eagerly than my cooking.
I made this incredible Grilled Salmon recipe with a very summery Avocado-Melon Salsa, because the goings are still good at the farmers' market and I needed to bid adieu to barbecuing season.
The key to cooking fish is paying attention (whether on the grill or otherwise). It's so disappointing to make accidental fish jerky from a luscious (and often costly) piece of wild salmon. Distraction is the enemy, so prepare everything ahead of time in order to give your seafood the attention it deserves.
I mean, the salsa should be finished, the table should be set, and take your pee break now – because you're not walking away from the barbecue!
I'm sorry, that was really bossy.
The fish is ready to be removed from the grill when you see just a hint of darker flesh in the centre of the thickest part of the filet. It'll finish cooking through off of the heat.
Do you continue to grill through the fall?
Toss all ingredients together in a medium bowl, starting with the lesser amount of jalapeno and adding more to taste (some are very hot, others are more mild). Season to taste with pinches salt and a bit of pepper. Set aside.
Preheat grill on high heat and oil grates. Season salmon generously with salt and pepper. Grill, skin-side up, for 5 minutes with lid closed. Carefully flip the fish (I use two metal spatulas), close lid, and cook an additional 2-4 minutes, until a paring knife inserted in the thickest part reveals the fish is mostly opaque, with just a trace of translucent, darker colour flesh in the centre (it'll continue cooking a bit off of the grill). Transfer to a serving platter and top with salsa.
Poke bowls are a fresh, summery meal that's both filling and healthy. They're having a moment right now, with poke restaurants popping up on every corner, but they're genuinely a cinch to whip up in your own kitchen.


Math League is a mathematics contest which is being held in many places around the world such as the U.S and Canada. The "Math League-High School Contest 5" aims at measuring students' English and Maths skills as well as enabling them to compare themselves with their peers internationally.
We are proud to announce the result of our student Berkan Doruk Eke from grade 11A on the being Student With A Perfect Score list internationally. We congratulate Doruk and wish him continued success.
This page was created on 16.4.2019 07:47:19 and has been visited 33 times.


Sign up here to receive daily weather forecasts and/or severe weather emails from the WGEM StormTrak Weather Team. The severe weather email you receive will contain the alert headlines (i.e. Tornado Watch for Adams County, IL. in effect from July 19 at 7:12 PM CDT until July 19 at 9:00 PM CDT). Below the headline is a link that says, "Click here for more information." Clicking that link will direct the user to the web page where you will find more specific information.


Edward J. Hall loves maps. How they look. What they tell us about our immediate surroundings as well as our place on the planet, past and present.
Ed is a retired map librarian, who spent over 30 years amassing the second largest collection of maps in the country, housed here at Kent State University.
Ed is also a philanthropist who wants to ensure that future generations of Kent State students know how to read and understand maps (rather than just following the voice on their GPS). "There's a real need for acquainting people with their cities," he says. Inspired by late faculty colleagues Jim Rinier (geology) and Art Herrick (biology) who designated Kent State in their estate plans, Ed and his wife, Peg, have funded a gift annuity, which at maturity will benefit the Map Library in the Kent State University Libraries. "We're grateful that Kent State provided me with a livelihood for 30 years, and also paid for our daughter's graduate school education," says Ed. "A gift annuity also lets us do something for the university while getting something back."
Native New Englanders, Ed and Peg met while attending the University of Vermont, from which Ed earned undergraduate and graduate degrees in history and geography and Peg earned an education degree. Ed also later attained a Master's in Medical Librarianship from the State University of New York in Albany.
Ed owes his love of maps to two former deans of University Libraries, Jordan Hodgkins and Hy Kritzer, as well as President Lyndon Johnson. He was teaching at a community college in Massachusetts when President Johnson announced a new program for teachers in urban studies and geography. Ed was selected as one of only 30 teachers in the country to bring these subjects into America's classrooms. The curriculum also gave him a sense of how cities develop. "I can go anywhere in the U.S., check out a map and pinpoint city centers, retail sectors, etc., even before driving around."
In 1973, Ed came to Kent State to interview for a faculty position in geography, but was wooed away to become a map librarian where he could serve many departments as well as the surrounding community. His love of maps and a keen skill for trading placed him in one of the most unusual positions at the university: caring for its huge map collection, and trading duplicate maps with other cities and libraries for ones that complimented the collection. During his tenure at Kent State, Ed split his time between University Libraries and the Map Library; he now serves on the University Libraries Advisory Council.
As to the lasting legacy he is leaving for future generations to explore and enjoy, Ed recently noted, "I'm so pleased to support Kent State and the Map Library — it's one of Kent State's true gems. And I'm proud to have been a part of it for so many years."
To learn more about how a gift annuity can pay you now and benefit Kent State in the future, call us at 330-672-1000 or email the Center for Gift and Estate Planning at giftplan@kent.edu.


Introducing Club med Bintan Island and a Special Offer!
This is a sponsored conversation written by me on behalf of Clubmed. The opinions and text are all mine.
It's no secret that Vaughan and I are avid island lovers. We lived on Koh Samui island for a year, honeymooned on Koh Lanta and Koh Tao, lived two hours from Tioman Island in Malaysia and took every opportunity we had to hop onto as many islands as possible. Perhaps it's the warm ocean of South East Asia, or the exceptional marine life which awaits beneath the water's surface but mostly it's the fact that we get to enjoy so many different water sport adventures on most islands.
When it comes to Bintan Island, this is most definitely the case. Only a 1 hour ferry ride from Singapore (I wish we had known about this whilst living in Malaysia), Club Med Bintan Island offers almost every kind of adventure activity available including various watersports.
Come April to October, the calm waters provide the best playground for snorkelling, diving ( and learning), jet skiing and water skiiing, whilst November to March brings in the swell perfect for long boarding, body boarding and wind surfing.
For those couples and families who really enjoy Club Med's all-inclusive getaways, the new flagship Club Med on Bintan is definitely worth visiting! Being such a short trip from Singapore, you can include this vibrant city in your holiday itinerary.
The true beauty of a Club Med resort lies in its all-inclusive nature so that once you've paid, you won't have to fork out any further cash for drinks, meals or activities. Enjoy the abundance on offer at the resort, simple zone out poolside or include in a few luxurious spa treatments. Best of all for couples with kids, there's a kids club where they can be entertained whilst you truly get a timeout.
Being new parents ourselves, we do love lots of down time but also love to explore and keep active whilst away. Club Med offers an incredible amount of activities such as yoga, beach volleyball and soccer, aqua fitness, waterpolo, badminton as well as cycling and mountain biking to name a few.
Excursions: A romantic sunset cruise!
Santa Maria Cave: Just opposite Trikora beach, this place of worship and statue was built by a Dutch priest in the 18th century and still gathers worshipers on Sundays and during the week for prayers.
Senggarang Chinese Village: A fishing village with houses built on stilts believed to be one of the original beginnings for Chinese immigrants who first arrived in the 18th century. Whilst there visit the 200 year old Banyan tree temple which has been said to bring good fortune to those who visit it.
Pasar Oleh Oleh: Shop for local delicacies and souvenirs at this market housed in local-style huts. Open daily from 10:00-23:00.
Sebung Village: Seafood lovers unite and always find their way to Sebung village for the best seafood on the island. Food is served in Kelong-style restaurants overlooking the Sebung River.
By train or by plane, we organise the appropriate* transportation, as well as the transfers to and from your destination Village. You receive all the necessary information before departure (flight times, airline tickets, check-in times, etc.) Your only task: packing a suitcase. *You can also organise on your own by choosing the package "without transportation".
Designed by the world's most inspired architects and designers, the rooms and suites in each Village are unique. They share one common feature: the comfort and well-being they offer. Lazy sleep in mornings, restorative naps, beauty moments, … you'll love every moment you spend in the room.
From breakfast to the evening meal without forgetting breaks for snacks, everything is included in your All Inclusive package. Each Village offers a generous selection of varied dishes and snacks, at practically all hours and with a very demanding French touch. It may take you a while to decide between the deliciously spicy stuffed crab or the pizza bianca with ricotta cheese. Here's a tip: take both!
During the day, but also at night, alcoholic or non-alcoholic drinks* are included in your package and unlimited (but always in moderation). No need to worry about the bill as you sip a fresh mango juice by the pool or enjoy a glass of pinot noir by the chimney. *Served at the bar or by the glass. We also offer a selection of drinks à la carte such as brandy or whiskies (but not included in the package).


Our production is based on two modern manufacturing departments. Production organisation has been arranged in cooperation with the best equipment suppliers in Poland and the world leaders in the business.
Our production facility covers 5,300 m2 and meets the GMP, HACCP and other standards. Moreover, we have our own warehouse which enables us to secure continuity of supplies of or products to customers. We have furnished our departments with the most advanced equipment which enables us to meet the strictest of standards for the pharmaceutical and food industries, thus allowing our customer to receive the highest quality merchandise.
We own a quality control laboratory which conducts rigorous evaluation of every stage of our production, according to meticulously developed procedures.
The principal area of activity of the Quality Control Department is testing of materials, therapeutic products, diet supplements and service products.
Routine checks are conducted in two directions depending on the requirements of the pharmaceutical law and food law.
Quality Control Department Laboratory is equipped with basic control and measurement devices which, among others, utilises spectrophotometry, pH-metering, conductometry, refractometry, polarimetry. In the future we plan to develop instrumental methods of analysis which enable prompt and precise quality control of the tested materials and products, as well as adjustment of analytical methods to the changing requirements. Another specialisation of the Quality Control Department is testing of durability of therapeutic products and shelf life testing. The Department personnel receives regular training which translates into a high level of services which in turn guarantees a high quality of our products.
Our Company takes environment protection seriously and puts much effort in minimising our impact on the environment. Our environmental protection services continually monitor impact which our facility has on the surrounding making sure that the next generation inhabitants of our town and vicinity could still enjoy the natural and unpolluted environment.
We manufacture lozenges for the pharmaceutical company GlaxoSmithKline Pharmaceuticals, Medagro International and Omega Pharma NV. The automated production process is covered by the GMP (Good Manufacturing Practice) quality assurance system. The standards we apply place us among the leading companies in the business.
Here we manufacture a wide range of long-life confectionery products presented in detail in our offer. In order to secure health safety of our products we have implemented the Hazard Analysis and Critical Control Point (HACCP) system.


text:
Body found in missing Berthoud teen's truck
Mesa County police arrest man sought in disappearance of ex-girlfriend, find body in truck.
Body found in missing Berthoud teen's truck Mesa County police arrest man sought in disappearance of ex-girlfriend, find body in truck. Check out this story on coloradoan.com: http://noconow.co/28pQNnP
Jacy Marmaduke, jmarmaduke@coloradoan.com Published 7:22 a.m. MT June 10, 2016 | Updated 4:56 a.m. MT June 11, 2016
Ashley Marie Doolittle, and Tanner George Flores.(Photo: Courtesy of Larimer County Sheriff's Office)
The lifeless body of a young adult female was found Friday when Mesa County law enforcement arrested a Berthoud teen connected to the disappearance of his ex-girlfriend.
Tanner George Flores, 18, was arrested in Mesa County in western Colorado, hours after the Larimer County Sheriff's Office sought the public's help in finding 18-year-old Ashley Marie Doolittle of Berthoud, who police say recently ended a one-year relationship with Flores. Investigators said Flores had been missing since Thursday afternoon and might have been armed with a handgun.
The sheriff's office on Friday would not say whether the body Mesa County deputies found Friday is that of Doolittle. The Mesa County Coroner's Office will release the identity and cause and manner of death of the body found in Flores' truck at a later date, according to a release.
ALTERCATION: Loveland man allegedly assaults police officer
Doolittle was expected home at 7:30 p.m. Thursday, police say. Her family reported her missing at 7:55 p.m. after searching the local area and discovering her unoccupied car near Lon Hagler Reservoir. Police were told that Flores, missing since 3:30 p.m. Thursday, was upset and distraught over the couple's recent breakup.
After investigating the scene at the reservoir southwest of Loveland, LCSO sought the public's help in locating the couple and Flores' vehicle, a white 1999 Dodge diesel pickup.
A tip received at 8 a.m. Friday prompted a search near the town of Collbran, where Mesa County sheriff's officers arrested Flores. The release did not specify where officers found Flores or the body, which hasn't been identified. LCSO officials declined to provide further information Friday.
CRIME: Bond set for Larimer County 'Most Wanted' fugitive
Flores is being held in the Mesa County Jail, pending criminal charges.
Information about where Flores will be tried if he faces criminal charges, or if he will be transferred to the Larimer County Jail, wasn't available Friday. Larimer County Sheriff's office spokesman David Moore told the Coloradoan those decisions will be made as the investigation progresses.
Moore does not expect those details to come before Monday. The Larimer County Sheriff's Office will lead the investigation with help from Mesa County law enforcement.
Investigators ask anyone with information regarding the disappearance of Doolittle to call the Larimer County Sheriff's Office at 970-498-5163.
The truck of 18-year-old Tanner George Flores. (Photo: Courtesy of Larimer County Sheriff's Office)
Read or Share this story: http://noconow.co/28pQNnP


text:
Breaking News Industry News Touring News Venue News
Janet Jackson And New Edition To Play The Inaugural Concert At Lynn Family Stadium
Lynn Family Stadium
CelebrityAccess Staff Writers Contact MePosted on 21 March, 2022, 03:51 PM EDT March 21, 2022
LOUISEVILLE, KY (CelebrityAccess) — Janet Jackson and the legendary R&B group New Edition are slated to perform at the first ever concert at the ASM Global-managed Lynn Family Stadium.
"This is a historic moment for Lynn Family Stadium and the Louisville community," said ASM Global General Manager Eric Granger. "We are pleased to be able to work with Louisville-based concert promoter 85° West to bring our first concert to the stadium. This concert will not only provide an exciting new entertainment opportunity for the Louisville region. It also helps showcase the venue's ability to offer a variety of programming and will open doors to welcome more world-class artists to perform to fans at Lynn Family Stadium in the future."
The concert, which will take place on May 7th, as part of the festivities surrounding this year's Kentucky Derby, will see all six original members of New Edition — Bobby Brown, Johnny Gill, Ralph Tresvant, Michael Bivins, Ronnie Devoe and Ricky Bell – performing at the stadium.
Ms. Jackson will also be on hand, performing some of her fan favorite hit from her extensive catalog that has sold more than 100 million records over the course of her storied career.
The 15,000-capacity Lynn Family Stadium first opened its doors in 2020 and has the Louisville City FC since its debut and Racing Louisville FC of the National Women's Soccer League (NWSL) since 2021.
#ASMGlobal
#janetjackson
#LynnFamilyStadium
#NewEdition
Posted by Staff Writer - January 27, 2023
Pukkelpop Cancels Hear Hear! Festival's 2023 Return
Join CelebrityAccess Now