Egypt's Suez Canal revenues rose to $ 447.1 million in July from $427.2 million in June, the government said on Tuesday.
The canal is the fastest shipping route between Europe and Asia and one of the government's main sources of foreign currency. Egypt has been struggling to revive its economy since a 2011 uprising hit tourism and foreign investment.
A recent press release by the canal's authority said that almost 100% of the vessels engaged in Asia-Europe trade cargo now transit the Suez Canal.
The head of the Suez Canal Authority, Mohab Mamish, pointed out that the waterway's share of the cargo trade between the East Coast of the United States and Southeast Asia had increased to 52% in 2016, compared to 25% in 2006.
Moreover, the canal's revenues from March to July 2017 reached $ 2.167 billion compared to $ 2.105 billion in the same period of 2016, an increase of 3%.
Transit activity has also been on the rise since the canal expansion was completed, a pattern that has continued this year.
A total of 7294 ships passed through the waterway between March and July, almost 4% more than in the same months of 2016.
A new record of 97 vessels in a single day has also been set, Mamish confirmed, while the new Suez Canal has contributed to shortening the waiting and transit time for southbound vessels by 7 hours.
Did CBE miscalculate including part of hot money within foreign exchange reserves?


text:
James McGowan Strode (1804–1857/1860) was a militia officer and politician from the U.S. state of Illinois. He served in the Illinois militia during the Winnebago War and the Black Hawk War. Strode, originally from Tennessee, lived much of his life in Galena, Illinois. In Galena, during the Black Hawk War he was given command of the 27th Regiment of the Illinois militia and oversaw the construction of a fort in that city. Strode was involved in combat during the war at the infamous Battle of Stillman's Run. In 1835 Strode was elected to represent much of the region of Illinois north of Peoria in the Illinois State Senate.

Early life
James M. Strode was from the U.S. state of Kentucky but lived much of life in Illinois. During the late 1820s Strode lived in Springfield, Illinois but from 1831 he lived in the city of Galena, Illinois.

Military service
James M. Strode began his military service on July 20, 1827 as the Winnebago War was being waged on the frontier in the mining region of southern Wisconsin. Strode, living in Springfield, Illinois at the time, was one of hundreds of Sangamon County volunteers called up by Illinois Governor Ninian Edwards. Edwards sent the forces into the lead mining region around Galena, Illinois to protect settlers there from raids by the Ho-Chunk. Strode enlisted as a private and his company commander was Captain Bowling Green (or Bowlin Green). Once in Galena, a company of volunteers was formed under the command of Henry Dodge and Strode served as company commander at the rank of captain; Strode's company was mustered into service on August 26, 1827 and discharged on September 16, 1827.

Strode traveled to Chicago for the first time in 1831 and returned occasionally throughout his life. On one trip to the settlement at Chicago in 1832, with Benjamin Mills and Judge Richard M. Young, the group became aware of the early unrest in the Black Hawk War. Once in Chicago they warned the settlers there of the war and its danger to their settlement.  Strode was at the disastrous defeat of Major Isaiah Stillman's militia force at Stillman's Run. An engagement he attended just for "the fun." Back in Galena, Strode rose to the rank of colonel and was given command of the frontier fort in Galena and the 27th Regiment of the Illinois Militia.

The fort was constructed in Galena beginning on May 19, 1832. The same day as fort construction began Strode detailed a small unit to carry dispatches from to General Henry Atkinson at Dixon's Ferry (present-day Dixon, Illinois). The group was ambushed by Kickapoo warriors later that day and one of the messengers was killed. On May 21, 1832 Strode declared martial law in Galena; his proclamation demanded that all able bodied men work on completing the fort from 9 a.m. to 6 p.m. until it was finished. His proclamation also prohibited the sale of "spirits" at grocers and taverns from 8 a.m. until 7 p.m.

The fort in Galena, known as the Stockade Refuge, was located at the center of town on Perry Street. It featured two blockhouses, one centered and the other in the corner of the stockade. The stockade's other three corners contained fortified houses. Strode's fortified home was about 200 feet to the east of the fort, along Bench Street. He also spent his command during the war passing down various orders to respond to incidents of violence across the frontier, such as the Plum River raid.

Political career
In August 1832 Strode was elected as a Democratic state senator and moved to Chicago a year later.  As a state senator Strode represented much of the area north of Peoria, and in 1835 he introduced legislation authorizing a government loan to commence work on the Illinois and Michigan Canal. The US$500,000 loan was negotiated by Illinois Governor Joseph Duncan, and in June 1836 construction on the canal began. In July 1836 he was commissioned as the register for the U.S. Land Office in Chicago, whose offices were located in the Saloon Building on Clark Street.

Later life
Strode worked as an attorney for much of his life and after 1837 he was a "practicing member" of the Chicago Bar Association. Strode worked as a prosecutor until 1848. On May 16, 1848, in McHenry County, he married Cynthia Ann Beardsley (1832–1897), with whom he had a son and a daughter (Luella). He moved to Woodstock around the time of his marriage and served as McHenry County judge, 1854-1857. Strode died in Kentucky, where he had gone to settle a family estate. This was before 1861, when Cynthia remarried as a widow with Ephraim Pease.

Notes

References
Kett, H.F. and Co. The History of Jo Daviess County, Illinois, (Google Books), H.F. Kett & Co., Chicago: 1878. Retrieved 18 August 2007

1804 births
19th-century deaths
American people of the Black Hawk War
People from Galena, Illinois
Politicians from Springfield, Illinois
Illinois state senators
19th-century American politicians

redpajama_set_name: RedPajamaWikipedia


text:
UK Game Charts: Guitar Hero vs Beatles
And Colin McRae motors in...
Posted 14 Sept 2009 by Staff
Okay, Guitar Hero 5 launched on September 11th. The Beatles: Rock Band launched on September 9th (you may have missed the news on that one if you were living in a disused cave in Lativia). The Beatles edition of Rock Band managed to make its ELSPA chart demo debut in a higher position than any other version of the game (Number 4). However, Guitar Hero 5 outsold it and came out at Number 1.
One theory here is that, despite an advertising barrage across all media, Beatles fans are either patient (waiting for a price drop) or simply want to listen to the music rather than pretending to play along with it. Or, Guitar Hero is just more popular.
Also new to the chart is Colin McRae Dirt 2 (Number 2), a fine figure of a racing game. Other notables see Arkham Asylum holding a top three spot (Number 3). Ashes Cricket maintains the last vestiges of summer at Number 10, while Mario Kart Wii returns to the Top 10.
(NE) - 1 - Guitar Hero 5 - Red Octane
(NE) - 2 - Colin Mcrae: Dirt 2 - Codemasters
(1) - 3 - Batman: Arkham Asylum - Eidos
(NE) - 4 - The Beatles: Rock Band - EA Games
(4) - 5 - Wii Fit - Nintendo
(2) - 6 - Wii Sports Resort - Nintendo
(6) - 7 - Il2 Sturmovik: Birds Of Prey - 1C Company
(3) - 8 - Call Of Duty 4: Modern Warfare - Activision
(12) - 9 - Mario Kart Wii - Nintendo
(5) - 10 - Ashes Cricket 2009 - Codemasters
Companies: 1C Activision Blizzard Codemasters Eidos Electronic Arts Nintendo Red Octane
Games: Ashes Cricket 2009 Batman: Arkham Asylum Call of Duty 4: Modern Warfare DiRT 2 Guitar Hero 5 IL-2 Sturmovik: Birds of Prey Mario Kart Wii The Beatles: RockBand Wii Fit Wii Sports Resort
Jun|'18
Dec|'18
Persona 3: Dancing in Moonlight / Persona 5: Dancing in Starlight
Psalms 14 Sep 2009 09:05
Didn't see that one coming...
PaulRayment 14 Sep 2009 09:28
Would say the high price point made the difference but the closely priced GH5 seemed to do OK with relatively meager promotion compared to The Beatles RB.
Could the typical GH/RB player just not care about Beatles music? And the age range that does care are a bit more frugal and would rather spend that money on the music or paying their bills?
I think it goes to prove the limited appeal of one band music titles. Especially when the band concerned, isn't that challenging instrumentally.
Thinking about it, could this all be down to the work of Courtney Love? She has been giving GH5 a lot of free press recently...
Posting of new comments is now locked for this page.
A Plague Tale: Innocence Review
Persona 3: Dancing in Moonlight / Persona 5: Dancing in Starlight Review
Civilization VI (Switch) Review
SEGA Mega Drive Classics - Switch Review
Spyro Reignited Trilogy Review
Hitman 2 Review
The Council: Episode 4: Burning Bridge Review
Valkyria Chronicles 4 Review
Yakuza Kiwami 2 Review
Marvel's Spider-Man Review

redpajama_set_name: RedPajamaCommonCrawl


Learning to care: medical students' reported value and evaluation of palliative care teaching involving meeting patients and reflective writing.
Over recent years there has been an increase in teaching of both palliative care and reflective practice in UK medical schools. The palliative care teaching at the University of Cambridge School of Clinical Medicine is multi-faceted and involves students writing reflective essays after individually meeting patients approaching the end of life during their final year general practice and hospital medicine placements. This paper draws on two studies examining this teaching element to analyse what the students found valuable about it and to comment on the practice of meeting patients and subsequent reflective writing. Two studies have explored students' perceptions of these course components. The first was a thematic analysis of 234 reflective essays from 123 students written in 2007-2008, including examining what students wrote about the exercise itself. The second project involved a semi-structured questionnaire that students completed anonymously; this paper reports on the free text elements of that study [sample size =107]. Since similar themes were found in both studies, the coding structures from each project were compared and combined, enabling triangulation of the findings around what the students found valuable from the palliative care teaching involving meeting patients and reflective writing. Overall, students reported that these components of the palliative care teaching are valuable. Four main themes were identified as aspects that students valued: (1) dedicated time with patients, (2) learning about wider elements of treatment and holistic care, (3) practicing communication skills, and (4) learning about themselves through reflective writing. Some students expressed a dislike for having to formally write a reflective essay. It is possible to arrange for all of the medical students to individually meet at least two patients receiving palliative or end of life care. Students found these encounters valuable and many wrote about the benefit of formally writing about these experiences. Students reported finding this model useful in widening their skill-set and understanding of palliative care.


text:
\section{Introduction}

Reinforcement learning has gained tremendous popularity in recent years \cite{sutton2018reinforcement}. Simulation-based methods for reinforcement learning or stochastic control have achieved notable success \cite{silver2009reinforcement}. One particularly simple simulation-based method, called Monte Carlo Exploring Starts (MCES), was introduced in detail in the classic book by Sutton and Barto \cite[Chapter 5]{sutton2018reinforcement}. In this method, the value function is estimated by the average simulated returns and the policy is updated using a greedy policy based on the current estimate of the value function. Because of its fundamental simplicity and importance, Sutton and Barto stated that the convergence of the MCES algorithm to the actual optimal value is ``one of the most fundamental open theoretical questions in reinforcement learning"  \cite[p.~99]{sutton2018reinforcement}.

Partial results on convergence analysis of MCES exist in the literature. Most notably, Tsitsiklis \cite{tsitsiklis2002convergence} proved that MCES, which he termed optimistic policy iteration, converges under two assumptions. First, each state is selected for updating with the same frequency. Second, the problem is strictly discounted with a discount factor less than one. This result was extended to the undiscounted case by Chen \cite{chen2018convergence} under the assumption that all policies are proper (i.e., reaching a terminal state is inevitable under all policies). A more recent result by Wang and Ross \cite{wang2020convergence} proved convergence of MCES under the assumption of optimal policy feed-forward environments, where states cannot be revisited under an optimal policy. We note that the approach taken in \cite{wang2020convergence} mostly uses finite graph and probabilistic argument, whereas the approach in \cite{tsitsiklis2002convergence} (and also \cite{chen2018convergence}) is along the lines of stochastic approximation \cite{bertsekas1996neuro,robbins1951stochastic}.

In this paper, we investigate the convergence of MCES/optimistic policy iteration in the undiscounted case without the assumption of optimal policy feed-forward environments and without the assumption that all polices are proper. Compared with the results in \cite{tsitsiklis2002convergence,chen2018convergence,wang2020convergence}, we consider both uniform and nonuniform exploring starts. In the uniform case, we extend the results of Tsitsiklis \cite{tsitsiklis2002convergence} to the undiscounted, i.e., stochastic shortest path problem.
For the case that all policies are proper, our proof differs from that in \cite{chen2018convergence} and is closer in spirit to that of \cite{tsitsiklis2002convergence} (see how Lemma \ref{lem:main} generalizes Lemma 2 in \cite{tsitsiklis2002convergence}). We also discuss how to work around the proper policy assumption. In the nonuniform case, we argue that the choice of stepsize should be component-dependent to agree with the classical version of MCES discussed in \cite{sutton2018reinforcement}.
We believe the convergence results established here could help further settle the long-standing open problem. As a side result, we also prove a version of the  supermartingale convergence theorem that is commonly used in the literature of stochastic approximation, whose proof, however, is not available in classic books such as \cite{bertsekas1996neuro}. Furthermore, we
provide an alternative and hopefully more direct treatment of stochastic approximations directly based on the supermartingale martingale convergence theorem (cf. Chapter 4 of the classic book \cite{bertsekas1996neuro}), which may be of independent interest.

The paper is organized as follows. In Section \ref{sec:pre}, we present the problem formulation and the preliminaries for proving the convergence of MCES/optimistic policy iteration. In Section \ref{sec:main}, we present the convergence proof for the case that all policies are proper. We discuss the case without the proper policy assumption in Section \ref{sec:improper} and the case with nonuniform initial exploring in Section \ref{sec:nonuniform}. A simple illustrative example is presented in Section \ref{sec:ex}. Some concluding remarks are presented in Section \ref{sec:con}. The Appendix includes a self-contained treatment of supermartingale convergence and stochastic approximation results.

\section{Problem formulation and preliminaries}\label{sec:pre}

\subsection{Markov decision problem}

Let $M=(S,A,P)$ be a \textit{Markov decision process}, where $S=\set{1,\cdots,n}$ is a finite set of states, $A$ is a finite set of
actions, and $P:\,S\times A\times S\rightarrow[0,1]$ is a transition probability function. For each action $a\in A$, we can represent $P(\cdot,a,\cdot)$ as a matrix $P(a)$ whose entries $P_{ij}(a)$ satisfy
$$
P_{ij}(a)=P(i,a,j)=\mathbb{P}(s_{t+1}=j|s_t=i,a_t=a),
$$
where $\set{(s_t,a_t)_{t=0}^{t=\infty}}\subseteq S\times A$ is an evolution of the MDP $M$. In words, $P_{ij}(a)$ denotes the probability of having a transition from the state $i$ to the state $j$ under the action $a$.

A \textit{policy} is a function $\mu:\,S\rightarrow A$. Clearly, the set of all policies is finite. We denote this set by $\Pi$. Given a policy $\mu\in \Pi$, we define the cost-to-go value\footnote{We use a cost function formulation as commonly seen in stochastic control, which is equivalent to a reward function formulation in reinforcement learning, albeit the difference of using minimization in place of maximization for values.} of the policy starting from a state $i$ as
$$
J^\mu (i) = \mathbb{E}\left[ \sum_{t=0}^\infty \alpha^t g(s_t,\mu(s_t)) | s_0=i \right],
$$
where $\set{s_t}_{t=0}^\infty$ is a state evolution under the policy $\mu$, $g:\,S\times A\rightarrow\mathbb{R}$ is the stage cost, and $\alpha\in[0,1]$ is a discount factor. The optimal cost-to-go value $J^*$ is defined as
$$
J^*(i) = \min_{\mu\in\Pi} J^{\mu}(i).
$$
Since the set of policies is finite, the optimal value is always attainable by an optimal policy. That is, there exists $\mu^*\in\Pi$ such that $J^{\mu^*} = J^*$. A Makov decision problem often is concerned with finding the optimal value $J^*$ and an optimal policy $\mu^*$ (which may not be unique).

We will primarily be focusing on the so-called \textit{stochastic shortest path problem} in this paper, i.e. the Markov decision problem above with $\alpha=1$. To make the cost-to-go value well-defined, we assume that there exists a terminal state, denoted by 0, and modify the transition probability function to satisfy $\sum_{j=1}^nP_{ij}(a)\le 1$ and $P_{i0}(a)=1-\sum_{j=1}^nP_{ij}(a)$ for all $i\in S$ and $a\in A$. In addition, the terminal state is assumed to be a trap state in the sense that $P_{00}(a)= 1$ and $P_{0j}(a)= 0$ for all $a\in A$ and $j\in S$. We also assume $g(0,a)=0$ for all $a\in A$ such that $J^\mu(0)=0$ for all $\mu\in\Pi$. Hence we do not need to discuss the value at state $0$.

\subsection{Dynamic programming operators}

We define two dynamic programming operators $T_{\mu}:\,\mathbb{R}^n\rightarrow\mathbb{R}^n$ and $T:\,\mathbb{R}^n\rightarrow\mathbb{R}^n$ as follows. Given $J\in\mathbb{R}^n$ and $\mu\in\Pi$, let
\begin{equation}\label{eq:tmu}
T^{\mu}J(i) = g(i,\mu(i)) + \alpha\sum_{j=1}^n P_{ij}(\mu(i))J(j),
\end{equation}
and
\begin{equation}\label{eq:t}
TJ(i) = \min_{a\in A} \left[g(i,a) + \alpha\sum_{j=1}^n P_{ij}(a)J(j)\right].
\end{equation}
For convenience, we can write (\ref{eq:tmu}) in a vector format as
$$
T_\mu J = g_\mu  + \alpha P_{\mu}J,
$$
where $g_\mu=[g(1,\mu(1))\;g(2,\mu(2))\;\cdots\;g(n,\mu(n))]^T\in\mathbb{R}^n$ and $P_{\mu}=(P_{ij}(\mu(i)))\in\mathbb{R}^{n\times n}$. %
It follows that, for each $J\in\mathbb{R}^n$, there exists $\mu\in\Pi$ such that
$$
T J =  T_\mu J.
$$
Such a policy is called a \textit{greedy policy} corresponding to $J$.

\subsection{Optimistic policy iteration with Monte Carlo policy evaluation}

Following \cite{tsitsiklis2002convergence}, we can write the main procedure of optimistic policy iteration using Monte Carlo simulations for policy evaluation as
\begin{equation}\label{eq:J}
J_{t+1} = (1-\gamma_t) J_t + \gamma_t (J^{\mu_t}+w_t),
\end{equation}
where $J_t$ is the current value vector, $\gamma_t$ is a scalar stepsize parameter (time-varying but deterministic), and $J^{\mu_t}$ is the expected cost value of the current policy $\mu_t$. Given the current value $J_t$, a greedy policy $\mu_t$ is chosen according to
\begin{equation}\label{eq:mu}
T_{\mu_t} J_t = T J_t.
\end{equation}
The noise $w_t$ captures the discrepancy between the expected cost $J^{\mu_t}$ and observed cumulative cost $J^{\mu_t}+w_t$. Let $\mathcal{F}_t$ be the natural filtration generated by the process (\ref{eq:J}). Since the observed cumulative cost gives an unbiased estimate $J^{\mu_t}$, we have
$
\mathbb{E}[w_t|\mathcal{F}_t] = 0.
$
Furthermore, the variance of $w_t$ (conditioned on $\mathcal{F}_t$) is only a function of the initial state and the current policy $\mu_t$. Because the numbers of states and polices are finite, we also have
$
\mathbb{E}[\norm{w_t}^2|\mathcal{F}_t] \le C,
$
for some constant $C$.

\subsection{Preliminaries}

We present some technical preliminaries for convergence analysis. We focus on the shortest path problem (i.e. $\alpha=1$).  A policy $\mu\in\Pi$ is said to be proper if the terminal state $0$ is reached with probability 1 from any initial state.
\begin{assum}\label{as:proper}
All policies in $\Pi$ are proper.
\end{assum}

\begin{assum}\label{as:step}
The stepsize parameter satisfies
$
\sum_{t=0}^{\infty} \gamma_t=\infty
$
and
$
\sum_{t=0}^{\infty} \gamma_t^2<\infty.
$
\end{assum}

Based on Assumption \ref{as:proper}, a well-known result is that the dynamic programming operators $T$ and $T_\mu$ are contractive with respect to a
weighted maximum norm.

\begin{lem}\cite[Proposition 2.2, p.~23]{bertsekas1996neuro}\label{lem:contraction}
If Assumption \ref{as:proper} holds, then there exists some $\beta\in [0,1)$ and a vector $\theta\in\mathbb{R}^n$ of positive components such that
$$
\sum_{j=1}^nP_{ij}(a)\theta(j)\le \beta\theta(i),\quad \forall i\in S,\quad \forall a\in A.
$$
In particular, this statement implies that
$$
\norm{T_{\mu}J_1-T_{\mu}J_2}_{\theta} \le \beta \norm{J_1-J_2}_{\theta},\quad \forall \mu\in\Pi, \forall J_1,J_2\in\mathbb{R}^n,
$$
and
$$
\norm{TJ_1-TJ_2}_{\theta} \le \beta \norm{J_1-J_2}_{\theta},\quad \forall J_1,J_2\in\mathbb{R}^n,
$$
where the weighted maximum norm $\norm{\cdot}_{\theta}$ is defined by $\norm{J}_\theta=\max_{1\le i\le n}\frac{\abs{J(i)}}{\theta(i)}.$
\end{lem}

Let $\theta\in\mathbb{R}^n$ be a vector of positive components. Define $\Theta=\diag{\theta(1),\theta(2),\cdots,\theta(n)}$. Let $\mathbf{1}\in\mathbb{R}^n$ be the column vector with all components equal to 1. The above lemma shows that, in matrix form,
\begin{equation}\label{eq:one}
P_\mu\Theta \mathbf{1} \le \beta\Theta \mathbf{1}, \quad \forall \mu\in \Pi,
\end{equation}
where the inequality is interpreted component-wise\footnote{In the sequel, all vector inequalities are interpreted component-wise.}. We refer to this as a weighted contractive property for $P_\mu$.

We also recall the following property on the dynamical programming parameters $T$ and $T_{\mu}$ for a stochastic shortest path problem.

\begin{lem}\cite[Lemma 2.2, p.~21]{bertsekas1996neuro}
For every scalar $c\ge 0$, $J\in\mathbb{R}^n$, and $\mu\in\Pi$, we have
\begin{equation}\label{eq:error}
T (J+c\mathbf{1}) \le T J + c \mathbf{1},\quad T_{\mu} (J+c\mathbf{1}) \le T_{\mu} J + c \mathbf{1},
\end{equation}
where $c$ is any nonnegative scalar. If $c$ is negative, then the inequalities are reversed.
\end{lem}

We can also prove a slight modification of the above lemma using (\ref{eq:one}).

\begin{lem}\label{lem:error}
Suppose that Assumption \ref{as:proper} holds. For every scalar $c\ge 0$, $J\in\mathbb{R}^n$, and $\mu\in\Pi$, we have
\begin{equation*}
T (J+c\Theta \mathbf{1}) \le T J + \beta c \Theta  \mathbf{1},\quad T_{\mu} (J+c\Theta \mathbf{1}) \le T_{\mu} J + \beta c \Theta \mathbf{1},
\end{equation*}
where $c$ is any nonnegative scalar. If $c$ is negative, then the inequalities are reversed.
\end{lem}

\begin{proof}
Let $\mu$ be a greedy policy corresponding to $J$, i.e., $T_{\mu} J = TJ$.  By (\ref{eq:one}), we have
\begin{align*}
T (J+c\Theta \mathbf{1})& \le T_\mu(J+c\Theta \mathbf{1})= g_\mu + P_\mu(J+c\Theta \mathbf{1})\\
&= g_\mu + P_\mu J + P_\mu c\Theta \mathbf{1}  = T_\mu J + P_\mu c\Theta \mathbf{1} \\
& \le  T_\mu J + \beta c \Theta \mathbf{1} =  T J + \beta c \Theta \mathbf{1}.
\end{align*}
The above also shows $T_{\mu} (J+c\Theta \mathbf{1}) \le T_{\mu} J + \beta c \Theta \mathbf{1}$ for any $\mu\in\Pi$.
\end{proof}

By the contraction mapping theorem, Lemma \ref{lem:contraction} implies the following convergence result.

\begin{lem}\cite{denardo1967contraction,bertsekas1996neuro}\label{lem:exist}
If Assumption \ref{as:proper} holds, we have, for every  $J\in\mathbb{R}^n$ and $\mu\in\Pi$,
$$
\lim_{t\rightarrow\infty} T^t J = J^*,\quad \lim_{t\rightarrow\infty} T^t_{\mu} J = J^{\mu},
$$
where $J^*$ and $J^\mu$ are the unique fixed points of $T$ and $T_{\mu}$, respectively.
\end{lem}

The next lemma is a modified version of Lemma 2 in \cite{tsitsiklis2002convergence}. Let $\theta\in\mathbb{R}^n$ and $\Theta=\text{diag}\{\theta(1),\theta(2),\cdots,\theta(n)\}$ be defined above. For the sequence $\set{J_t}_{t=0}^{\infty} \subset \mathbb{R}^n$, define
$$
c_t = T J_t - J_t,\quad \lambda_t=\max(c_t,0),\quad t\ge 0,
$$
where $\max$ is taken component-wise. Then clearly $\lambda_t$  is a nonnegative vector and $c_t\le \lambda_t$.

\begin{lem}\label{lem:main}
Suppose that Assumption \ref{as:proper} holds.
For every $t\ge 0$, we have
\begin{enumerate}
\item $T_{\mu_t}^k J_t\le J_t + \frac{\norm{\Theta^{-1}\lambda_t}_{\infty} \Theta\mathbf{1}}{1-\beta}$, for all $k\ge 1$,
\item $J^{\mu_t}\le J_t + \frac{\norm{\Theta^{-1}\lambda_t}_{\infty} \Theta\mathbf{1}}{1-\beta}$,
\item $J^{\mu_t}\le T J_t + \frac{\beta\norm{\Theta^{-1}\lambda_t}_{\infty} \Theta\mathbf{1}}{1-\beta}$.
\end{enumerate}
\end{lem}

\begin{proof}
Note that $\norm{\Theta^{-1}\lambda_t}_{\infty}$ is the weighted maximum norm of $\lambda_t$ with respect to the vector $\theta$. From (\ref{eq:mu}), we have $T_{\mu_t} J_t = T J_t$. It follows that $T_{\mu_t} J_t = J_t + c_t$. Applying $T_{\mu_t}$ to both sides of this equation gives
\begin{align*}
T_{\mu_t}^2 J_t & = T_{\mu_t}(J_t + c_t)  = g_{\mu_t} + P_{\mu_t}(J_t + c_t) \\
& =  T_{\mu_t} J_t + P_{\mu_t}c_t = J_t + c_t + P_{\mu_t}c_t.
\end{align*}
By induction, we obtain
\begin{equation}\label{eq:tmuk}
T_{\mu_t}^k J_t= J_t + (I+P_{\mu_t}+P_{\mu_t}^2+\cdots+P_{\mu_t}^{k-1})c_t.
\end{equation}
We have, for $m\ge 1$,
\begin{align}
P_{\mu_t}^{m}c_t &= P_{\mu_t}^{m}\Theta \Theta^{-1} c_t \le P_{\mu_t}^{m}\Theta \Theta^{-1} \lambda_t \le P_{\mu_t}^{m} \Theta \norm{\Theta^{-1} \lambda_t}_{\infty} \mathbf{1}\notag\\
& =   \norm{\Theta^{-1} \lambda_t}_{\infty} P_{\mu_t}^{m-1} P_{\mu_t} \Theta \mathbf{1} \le  \norm{\Theta^{-1}\lambda_t }_{\infty}  P_{\mu_t}^{m-1}  \beta \Theta \mathbf{1}\notag \\
& = \beta   \norm{\Theta^{-1}\lambda_t }_{\infty} P_{\mu_t}^{m-1} \Theta \mathbf{1} \notag\\
&\le \beta^m \norm{\Theta^{-1}\lambda_t }_{\infty}  \Theta\mathbf{1},\label{eq:pmut}
\end{align}
where the first two inequalities follow from the fact that elements of $P_{\mu_t}^{m-1} P_{\mu_t} \Theta$ and  $P_{\mu_t}^{m-1} P_{\mu_t} \Theta\Theta^{-1}$  are nonnegative and
we can bound components of $c_t$ with $\lambda_t$ and $\Theta^{-1} \lambda_t$ with $\norm{\Theta^{-1}\lambda_t}_{\infty}\mathbf{1}$, the third inequality follows from (\ref{eq:one}), the last inequality follows from an inductive argument, and the equations follow from straightforward rearrangements. Part of the above inequality also shows that, for $m=0$, $c_t\le  \norm{\Theta^{-1} \lambda_t}_{\infty} \Theta \mathbf{1}$. Hence by (\ref{eq:tmuk}) we obtain
\begin{align}
T_{\mu_t}^k J_t&\le J_t + (1+\beta+\beta^2+\cdots+\beta^{k-1})\norm{\Theta^{-1}\lambda_t}_{\infty} \Theta\mathbf{1}\notag\\
&\le J_t + \frac{\norm{\Theta^{-1}\lambda_t}_{\infty} \Theta\mathbf{1}}{1-\beta}, \label{eq:tmuk2}
\end{align}
where in the first inequality we used (\ref{eq:pmut}) and the fact that $c_t\le\lambda_t \norm{\Theta^{-1}\lambda_t}_{\infty} \Theta\mathbf{1}$. We proved item (1). Since $\lim_{k\rightarrow\infty} T_{\mu_t}^k J_t = J^{\mu_t}$, we proved item (2) by letting $k\rightarrow\infty$. Finally, applying $T_{\mu_t}$ to both sides of the inequality in item (1) and using the fact $J^{\mu_t}=T_{\mu_t} J^{\mu_t}$, we obtain
\begin{equation*}
J^{\mu_t} = T_{\mu_t} J^{\mu_t} \le T_{\mu_t}(J_t + \frac{\norm{\Theta^{-1}\lambda_t}_{\infty} \Theta\mathbf{1}}{1-\beta}).
\end{equation*}
By Lemma \ref{lem:error} and the fact that $T_{\mu_t} J_t= T J_t$ , we obtained item (3).
\end{proof}

\section{Convergence analysis for the stochastic shortest path problem with proper policies}
\label{sec:main}

The convergence analysis starts with an asymptotic estimate for $c_t = T J_t - J_t$ and $\lambda_t=\max(c_t,0)$. All convergence and asymptotic estimates for random variables in this section are understood in the sense of probability 1.

\begin{lem}\cite{tsitsiklis2002convergence}\label{lem:ct}
Under Assumption \ref{as:step}, we have
$$\limsup_{t\rightarrow\infty} c_t \le 0\text{ and }\lim_{t\rightarrow\infty} \lambda_t =0.$$
\end{lem}
\begin{proof}
The proof for $\displaystyle\limsup_{t\rightarrow\infty} c_t \le 0$ was established in \cite{tsitsiklis2002convergence} for the case $\alpha<1$. The same argument holds for $\alpha=1$. Here is an outline of the proof. Since $T_{\mu_t}J=g_{\mu_t} + P_{\mu_t} J$ for any $J\in\mathbb{R}^n$, we can verify that
\begin{align*}
    T J_{t+1} &\le T_{\mu_t} J_{t+1} = T_{\mu_t}((1-\gamma_t)J_t +\gamma_t J^{\mu_t} + \gamma_t w_t) \\
    & = J_{t+1}+(1-\gamma_t)(TJ_t-J_t) + \gamma_t v_t,
\end{align*}
where we need to use the fact that $T J_t = T_{\mu_t}J_t$ and $v_t = P_{\mu_t} w_t-w_t$. By the property on $w_t$, we have $\mathbb{E}[v_t\,|\,\mathcal{F}_t]=0$ and $\mathbb{E}[\norm{v_t}^2\,|\,\mathcal{F}_t]\le C'$ for some constant $C'$. Hence, $c_t$ satisfies
$$
c_{t+1} \le (1-\gamma_t)c_t + \gamma_t v_t.
$$
Consider another iteration
$$
V_{t+1} = (1-\gamma_t)V_t + \gamma_t v_t.
$$
If $V_0=c_0$, then a comparison argument shows that $c_t\le V_t$ for all $t\ge 0$. By a standard supermartingale convergence argument on stochastic iterations \cite[Chapter 4, p. 143]{bertsekas1996neuro} (see also Proposition \ref{prop:sa} and Lemma \ref{lem:v} in the Appendix), one can show that $V_t$ converges to 0 in probability 1. Hence, $\limsup_{t\rightarrow\infty} c_t \le 0.$ Since $\lambda_t=\max(c_t,0)$, it follows that $\lim_{t\rightarrow\infty} \lambda_t =0$.
\end{proof}
In particular, the above lemma shows that, for any $\varepsilon>0$, there exists $t(\varepsilon)$ such that
$$
\frac{\beta\norm{\Theta^{-1}\lambda_t}_{\infty}\Theta \mathbf{1}}{1-\beta} \le \varepsilon\Theta \mathbf{1},\quad \forall t\ge t(\varepsilon).
$$
Putting this into Lemma \ref{lem:main}(3) shows that
$$
J_{\mu_t} \le T J_{t} + \varepsilon \Theta\mathbf{1},\quad \forall t\ge t(\varepsilon).
$$
By (\ref{eq:J}), we obtain
\begin{align*}
J_{t+1}&= (1-\gamma_t)J_t + \gamma_t (J^{\mu_t}+w_t)\\
& \le (1-\gamma_t)J_t + \gamma_t T J_{t} + \gamma_t \varepsilon \Theta\mathbf{1} + \gamma_t w_t,\quad \forall t\ge t(\varepsilon).
\end{align*}
Define a mapping $H_\varepsilon:\,\mathbb{R}^n\rightarrow\mathbb{R}^n$ as
$
H_\varepsilon J =  T J + \varepsilon \Theta\mathbf{1}
$
and consider the sequence $\set{Z_t}$ generated by
\begin{align*}
Z_{t+1} &= (1-\gamma_t)Z_t + \gamma_t T Z_{t} + \gamma_t \varepsilon\Theta \mathbf{1} + \gamma_t w_t\\
&= (1-\gamma_t)Z_t + \gamma_t (H_\varepsilon Z_t  + w_t),\quad t\ge t(\varepsilon),
\end{align*}
and $Z_{t(\varepsilon)}=J_{t(\varepsilon)}$. Then, by comparison,
\begin{equation}
    \label{eq:JZ}
    J_t\le Z_t,\quad\forall t\ge t(\varepsilon).
\end{equation}

Since $T$ is a contraction under the weighted maximum norm $\norm{\cdot}_\theta$, so is $H_\varepsilon$. By Proposition 4.4 in \cite{bertsekas1996neuro} (see also Proposition \ref{prop:sa} in the Appendix), we know that $Z_t$ converges to the unique fixed point of $H_\varepsilon$, denoted by $Z_\varepsilon^*$.

The following lemma estimates the fixed point of $H_\varepsilon$ relative to $J^*$.

\begin{lem}\label{lem:Z}
Under Assumption \ref{as:proper}, we have
$$
J^* -\frac{\varepsilon}{1-\beta}\Theta\mathbf{1}\le Z^*_{\varepsilon}\le J^* +\frac{\varepsilon}{1-\beta}\Theta\mathbf{1}.
$$
\end{lem}

\begin{proof} By Lemma \ref{lem:error} and $T J^* = J^*$, we have
\begin{align*}
H_\varepsilon(J^*+\frac{\varepsilon}{1-\beta}\Theta\mathbf{1}) &= T(J^*+\frac{\varepsilon}{1-\beta} \Theta\mathbf{1}) + \varepsilon\Theta\mathbf{1} \\
& \le T J^* + \frac{\varepsilon\beta}{1-\beta} \Theta\mathbf{1} + \varepsilon\Theta\mathbf{1}\\
& = J^* +\frac{\varepsilon}{1-\beta}\Theta\mathbf{1}.
\end{align*}
It follows that
$$
Z^*_{\varepsilon} = \lim_{k\rightarrow\infty}H^k_{\varepsilon}(J^*+\frac{\varepsilon}{1-\beta}\Theta\mathbf{1})\le  J^* +\frac{\varepsilon}{1-\beta}\Theta\mathbf{1},
$$
where we used monotonicity of $H_\varepsilon$ (implied by that of $T$). Similary, by Lemma \ref{lem:error}, we can show that
\begin{align*}
H_\varepsilon(J^*-\frac{\varepsilon}{1-\beta}\Theta\mathbf{1}) \ge J^* - \frac{\varepsilon}{1-\beta}\Theta\mathbf{1}
\end{align*}
and
$
Z_\varepsilon^*\ge J^*-\frac{\varepsilon}{1-\beta}\Theta\mathbf{1}.
$
\end{proof}
We now state and prove the main result of the paper.

\begin{thm}\label{prop:main}
Under Assumptions \ref{as:proper} an \ref{as:step}, the sequence $J_t$ generated by the optimistic policy iteration (\ref{eq:J}) and (\ref{eq:mu}), applied to a stochastic shortest path problem, converges to $J^*$, with probability 1.
\end{thm}

\begin{proof}
Given any $\varepsilon>0$, by the argument preceding (\ref{eq:JZ}), there exists $t(\varepsilon)$ such that $J_t\le Z_t$ for all $t\ge t(\varepsilon)$. Since $\displaystyle\lim_{t\rightarrow\infty}Z_t=Z_{\varepsilon}^*$, it follows that $\displaystyle\limsup_{t\rightarrow\infty} J_t\le Z_{\varepsilon}^*$. By Lemma \ref{lem:Z}, we have $\displaystyle\limsup_{t\rightarrow\infty} J_t\le J^* +\frac{\varepsilon}{1-\beta}\Theta\mathbf{1}$. Since the choice of $\varepsilon>0$ is arbitrary, we obtain $\displaystyle\limsup_{t\rightarrow\infty} J_t\le J^*$. By the definition of $J^{\mu_t}$ and $J^*$, we have $J^{\mu_t}\ge J^*$. Hence, (\ref{eq:J}) implies
$$
J_{t+1} \ge (1-\gamma_t)J_t + \gamma_t J^* + \gamma_t w_t.
$$
Consider the iteration
$$
Y_{t+1} = (1-\gamma_t)Y_t + \gamma_t J^* + \gamma_t w_t
$$
with $Y_0=J_0$. Then the sequence $\set{Y_{t}}$ converges to $J^*$ (see Proposition 4.4 in \cite{bertsekas1996neuro} or Proposition \ref{prop:sa} in the Appendix). By comparison, $\displaystyle\liminf_{t\rightarrow\infty} J_t\ge J^*$. Hence, $\displaystyle\lim_{t\rightarrow\infty} J_t = J^*$.
\end{proof}

\section{Relaxing the proper policy assumption}\label{sec:improper}

Assumption \ref{as:proper} requires that all policies are proper. In this section, we discuss how to relax this assumption. For the stochastic shortest path problem, the following relaxed assumption was proposed in \cite{bertsekas1991analysis} (see also \cite[Chapter 2]{bertsekas1996neuro}).

\begin{assum}\label{as:relax}
There exists at least one proper policy, and every improper policy yields an infinite cost for at least one initial state, i.e., for every improper $\mu\in\Pi$, $\displaystyle J^\mu(i)=\lim_{k\rightarrow\infty}[\sum_{t=0}^{k-1}P_\mu g_\mu]_i=\infty$ for some $i\in S$.
\end{assum}

\begin{lem}\cite{bertsekas1991analysis}\label{lem:mod}
If Assumption \ref{as:relax} holds, we have, for every  $J\in\mathbb{R}^n$ and proper $\mu\in\Pi$,
$$
\lim_{t\rightarrow\infty} T^t J = J^*,\quad \lim_{t\rightarrow\infty} T^t_{\mu} J = J^{\mu},
$$
where $J^*$ and $J^\mu$ are the unique fixed points of $T$ and $T_{\mu}$, respectively.
\end{lem}

There is a problem, however, to analyze the convergence of the optimistic policy iteration (\ref{eq:J}) and (\ref{eq:mu}) under Assumption \ref{as:relax}. Unlike in the standard policy iteration, we cannot guarantee the greedy policy generated by the optimistic iteration is always proper. Hence the value iteration (\ref{eq:J}) will possibly attain infinity and become invalid. To overcome this issue, a natural way would be to let the process terminates with a small probability at each stage. This is equivalent to modifying the Markov decision process $M$, by adding a transition with a small probability to the terminal state under each action, so that it satisfies Assumption \ref{as:proper}.

A natural question to ask is whether the optimal value of the modified problem stays close to that of the original problem and whether an optimal policy obtained for the modified problem remains an optimal policy for the original problem.

Formally, we define a modified MDP $\hat{M}=(S,A,\hat{P})$ from the original MDP $M=(S,A,P)$ as follows. Let $\hat{P}_{i0}(a) = P_{i0}(a)+p_\varepsilon\sum_{j=1}^{n}P_{ij}(a)$ and $\hat{P}_{ij}(a)=(1-p_\varepsilon){P}_{ij}(a)$ for all $i,j\in S$ and $a\in A$, where $p_\varepsilon\in (0,1)$ is a small probability to be chosen. Then
$\hat{P}_{\mu}=(1-p_\varepsilon)P_\mu$ for all $\mu\in\Pi$ and $\hat{M}$ satisfies Assumption \ref{as:proper}. Let $\hat{J}^*$ denote the optimal value for $\hat{M}$ and $\hat{\mu}^*$ a corresponding optimal policy.

\begin{prop}\label{prop:mod}
Suppose that $M$ satisfies Assumption \ref{as:relax}. For every $\varepsilon>0$, there exists some $\delta>0$ such that, if $p_\varepsilon\in(0,\delta)$, then $\norm{\hat{J}^*-J^*}\le\varepsilon$. Furthermore, if $\delta>0$ is sufficiently small, then $p_\varepsilon\in(0,\delta)$ implies that $\hat{\mu}^*$ is also an optimal policy for $M$.
\end{prop}

\begin{proof}
The proof consists of two main parts. First we show that, by Assumption \ref{as:relax}, any improper policy for $M$ necessarily has large cost-to-go value for at least one component and hence cannot be optimal for $\hat{M}$ (even if it becomes proper with the modification), provided that $p_\varepsilon$ is chosen sufficiently small. We then show that the value of a proper policy in $\hat{M}$ remains close to its value in $M$, provided that $p_\varepsilon$ is sufficiently small. As a result, the optimal value remains close and optimal policy remains the same for $p_\varepsilon$ chosen sufficiently small.

Let $\mu$ be an improper policy for $M$. Consider the Jordan normal form of $P_\mu$:
$$
Q^{-1}P_{\mu}Q = \begin{bmatrix}I_p & 0\\0 & C \end{bmatrix},
$$
where $Q$ is a nonsingular matrix and $C$ has spectral radius $\rho(C)<1$. We obtain $I_p$ because the eigenvalue 1 of $P_\mu$ is semisimple \cite[p.~696]{meyer2000matrix}. The dimension of $I_p$ cannot be zero because otherwise $\mu$ would be a proper policy. It follows that
\begin{equation}\label{eq:sumP}
Q^{-1}\sum_{t=0}^{k-1}P_{\mu}^tQ = \begin{bmatrix} kI_p & 0\\ 0 & \sum_{t=0}^{k-1}C^t
\end{bmatrix},
\end{equation}
where $\sum_{t=0}^\infty C^t = (I-C)^{-1}$. Since $\hat{P}_\mu=(1-p_\varepsilon)P_\mu$, we have
\begin{equation*}
Q^{-1}\sum_{t=0}^{k-1}\hat{P}_{\mu}^t Q = \begin{bmatrix} \frac{1-(1-p_{\varepsilon})^{k}}{p_{\varepsilon}}I_p & 0\\ 0 & \sum_{t=0}^{k-1}(1-p_\varepsilon)^tC^t
\end{bmatrix},
\end{equation*}
where $\sum_{t=0}^\infty (1-p_\varepsilon)^tC^t = (I-(1-p_\varepsilon)C)^{-1}$.

By Assumption \ref{as:relax}, $\displaystyle\lim_{k\rightarrow\infty}\big[\sum_{t=0}^{k-1}P_\mu^t g_\mu\big]_i=\infty$ for some $i\in S$. This is equivalent to
$$
\lim_{k\rightarrow\infty}\left[Q \begin{bmatrix} kI_p & 0\\ 0 & \sum_{t=0}^{k-1}C^t
\end{bmatrix} Q^{-1} g_\mu\right]_i=\infty,
$$
which is again equivalent to
$$
\lim_{k\rightarrow\infty}\left[Q \begin{bmatrix} kI_p & 0\\ 0 & 0
\end{bmatrix} Q^{-1} g_\mu\right]_i=\infty
$$
in view of $\sum_{t=0}^\infty C^t<\infty$. Since $\frac{1-(1-p_{\varepsilon})^{k}}{p_{\varepsilon}}\rightarrow k$, as $p_{\varepsilon}\ra0$, and $\sum_{t=0}^\infty (1-p_\varepsilon)^tC^t = (I-(1-p_\varepsilon)C)^{-1}$ is continuous w.r.t. $p_\varepsilon$ and hence bounded for $p_\varepsilon\in[0,1]$, it is straightforward to verify that, for any $c>0$, there exists $\delta>0$, such that
\begin{align}
&\lim_{k\rightarrow\infty}\big[\sum_{t=0}^{k-1}\hat{P}_\mu^t g_\mu\big]_i\notag \\
& =\lim_{k\rightarrow\infty}\left[Q \begin{bmatrix} \frac{1-(1-p_{\varepsilon})^{k}}{p_{\varepsilon}}I_p & 0 \\ 0 & \sum_{t=0}^{k-1} (1-p_\varepsilon)^tC^t
\end{bmatrix} Q^{-1} g_\mu\right]_i\notag\\
& >c,\quad \forall p_\varepsilon \in (0,\delta]. \label{eq:c}
\end{align}

Now consider a proper policy $\mu$ for $M$ and let $\Pi_0$ denote the set of all proper policies for $M$. Clearly $\mu$ remains a proper policy for $\hat{M}$. The cost vector for $\mu$ in $\hat{M}$ is the unique solution to
$$
\hat{J}^{\mu} = \hat{T}_{\mu} \hat{J}^{\mu}.
$$
Note that $\hat{T}_{\mu}$ changes continuous with respect to $p_\varepsilon$ and $\hat{T}_{\mu}=T_{\mu}$ when $p_\varepsilon=0$. Since $T_{\mu}$ has a unique solution $J^\mu$, it follows that $\hat{J}^{\mu}$ also changes continuously with respect to $p_\varepsilon$. Hence for any $\rho>0$, there exists $\delta>0$ such that
\begin{equation}\label{eq:rho}
    \norm{\hat{J}^{\mu}-J^{\mu}} < \rho,\quad \forall p_\varepsilon \in (0,\delta],\,\forall \mu\in\Pi_0,
\end{equation}
because the number of policies is finite. Let $J^*$ be the optimal value of $M$ and define
$$
d = \min_{\mu\in\Pi_0\atop J^{\mu}\neq J^*}\norm{J^{\mu}-J^*}.
$$
Choose any $\rho<\frac{d}{2}$ and $\delta$ accordingly such that (\ref{eq:rho}) holds. Choose
$
c = \max_{\mu\in \Pi_0} J^{\mu} + \rho
$
and reduce $\delta$ accordingly such that (\ref{eq:c}) holds. In view of (\ref{eq:rho}) and the definition of $c$, any improper policy (w.r.t. $M$) cannot be optimal for $\hat{M}$ for all $p_\varepsilon \in (0,\delta]$. Furthermore, suppose that $\hat{J^*}$ is the optimal value and $\hat{\mu}^*$ is an optimal policy for $\hat{M}$. Then $\hat{\mu}^*$ is proper w.r.t. $M$. We claim that $J^{\hat{\mu}^*}=J^*$ and hence $\hat{\mu}^*$ is an optimal policy for $M$. Suppose this is not the case. Then $J^{\hat{\mu}^*}-J^*\ge d$. Since $\norm{\hat{J}^{\hat{\mu}^*}-J^{\hat{\mu}^*}}<\frac{d}{2}$, it follows that $\hat{J}^{\hat{\mu}^*}>J^*+\frac{d}{2}$. Let ${\mu}^*$ be a proper optimal policy for $M$. Then  (\ref{eq:rho}) implies that $\hat{J}^{\mu^*} <  J^{\mu^*} + \frac{d}{2} =J^* + \frac{d}{2}$. Hence $\hat{J}^{\mu^*}<\hat{J}^{\hat{\mu}^*}$ and $\hat{\mu}^*$ cannot be an optimal policy for $\hat{M}$, which is a contradiction. Thus  $\hat{\mu}^*$ is also an optimal policy for $M$. The proof is complete.
\end{proof}

\section{The case with nonuniform initial exploration}
\label{sec:nonuniform}

The version of optimistic policy iteration described by (\ref{eq:J}) and (\ref{eq:mu}) is synchronous in the sense that $n$ trajectories are simultaneously observed at each iteration, one for each initial state. It is pointed out in \cite{tsitsiklis2002convergence} that the scenario of picking one single state (randomly, uniformly, and independently) at each iteration to generate a trajectory from and update the cost-to-go value at this state can be captured by the following iteration:
\begin{equation}\label{eq:uniform}
    J_{t+1}(i) = \left\{\begin{aligned}
    &(1-\gamma_t)J_t(i) + \gamma_t (J^{\mu_t}(i) + w_t(i)),  \\
    &\qquad\qquad\text{ with probability } \frac{1}{n}, \\
    & J(i),   \qquad\qquad\qquad\text{otherwise}.
    \end{aligned}\right.
\end{equation}
Furthermore, this algorithm can be equivalently described in the form
\begin{equation}
    \label{eq:J2}
    J_{t+1} = (1-\frac{\gamma_t}{n})J_t + \frac{\gamma_t}{n}(J^{\mu_t}+v_t),
\end{equation}
where
$$
v_t(i) = w_t(i) + (n\chi_t(i)-1)(-J_t(i) + J^{\mu_t}(i) + w_t(i)),
$$
where each $\chi_t(i)$ is a random variable satisfying $\chi_t(i)=1$ if state $i$ is selected and $\chi_t(i)=0$ otherwise. Then, it can be shown that $v_t$ satisfies $\mathbb{E}[v_t|\mathcal{F}_t]  = 0$ and
$
\mathbb{E}[\norm{v_t}^2|\mathcal{F}_t] \le A+B\norm{J_t}^2,
$
for some constants $A$ and $B$, where we used the fact that $J^{\mu_t}$ is bounded, because there are only a finite number of policies. To use an argument similar to that in the proof of Theorem \ref{prop:main}, one needs to show that $\mathbb{E}[\norm{v_t}^2|\mathcal{F}_t]$ is bounded.
\begin{prop}\label{prop:boundJ}
Under Assumptions \ref{as:proper} and \ref{as:step}, the sequence $J_t$ in (\ref{eq:J2}) is bounded.
\end{prop}

\begin{proof}
There exists some $D>0$ such that $\norm{J^{\mu_t}}\le D$ for all $t\ge 0$. Boundedness of $J_t$ follows from Proposition 4.7 in \cite{bertsekas1996neuro} (see also Proposition \ref{prop:sa} in the Appendix).
\end{proof}
Hence, $\mathbb{E}[\norm{v_t}^2|\mathcal{F}_t]$ is bounded, this time by a sequence of random variables $A_t$ that are $\mathcal{F}_t$-adapted and bounded. By a similar argument to the proof of Theorem \ref{prop:main}, one can show that the values $J_t$ generated by (\ref{eq:J2}) converge to $J^*$ under the same assumptions.

A natural question is whether we can extend (\ref{eq:J2}) to the case where the states are chosen according to a nonuniform distribution such that each state has a non-zero probability of being selected. This would lead to the following update rule for $J_t$:
\begin{equation}
    \label{eq:J3}
    J_{t+1} = (1-\Gamma_t)J_t + \Gamma_t(J^{\mu_t}+v_t),
\end{equation}
where $\Gamma_t=\gamma_t\diag{p(1),p(2),\cdots,p(n)}$ and each $p(i)$ is the probability of state $i$ being selected. It is conjectured in \cite{tsitsiklis2002convergence}  that this may not converge (at least with the proof method therein). We also believe this is the case, although a concrete counterexample is yet to be constructed (see Section \ref{sec:ex} for a numerical example).

Here we provide a slightly different perspective. We argue that, in the case where the states are selected non-uniformly for updating, the stepsize should be different for each state. In fact, for the classical version of MCES \cite[p.~99, Chapter 5]{sutton2018reinforcement}, we have the stepsize given by
\begin{equation}\label{eq:step1}
\gamma_i(t)=\frac{1}{n_i(t)},
\end{equation}
where $n_i(t)$ is the number of times that state $i$ is selected up to iteration $t$, so that $J_t(i)$ is equal to the average of the simulated cumulative costs for state $i$ up to iteration $t$. This choice of stepsize is non-deterministic, but easy to implement. Alternatively, if we know \textit{a priori}  the probability of selecting each state for updating, we can design a time-varying but deterministic stepsize as
\begin{equation}\label{eq:step2}
\gamma_t(i)=\frac{\hat{\gamma}_t}{p(i)},
\end{equation}
where $\hat{\gamma}_t$ is any stepsize satisfying Assumption \ref{as:step}. We prove that both (\ref{eq:step1}) and (\ref{eq:step2}) lead to convergence of the following iteration
\begin{equation}\label{eq:nonuni}
    J_{t+1}(i) = \left\{\begin{aligned}
    &(1-\gamma_t(i))J_t(i) + \gamma_t(i) (J^{\mu_t}(i) + w_t(i)),  \\
    &\qquad\qquad\text{ with probability } p(i), \\
    & J(i),   \qquad\qquad\qquad\text{otherwise},
    \end{aligned}\right.
\end{equation}
where $p(i)>0$ is the probability that state $i$ is being selected at each iteration.

\begin{prop}
Under Assumptions \ref{as:proper} and \ref{as:step}, both choices of stepsizes (\ref{eq:step1}) and (\ref{eq:step2}) lead to convergence of $J_t$  in (\ref{eq:nonuni}) to $J^*$ in probability 1.
\end{prop}

\begin{proof}
We first consider the case (\ref{eq:step2}).
We can equivalently write the update rule as
\begin{equation}
    \label{eq:newJ}
    J_{t+1} = (1-\hat{\gamma}_t)J_t + \hat{\gamma}_t(J^{\mu_t}+\hat{v}_t),
\end{equation}
where $\hat{\gamma}_t$ is any stepsize satisfying Assumption \ref{as:step} and
$$
\hat{v}_t(i) = w_t(i) + (\frac{\chi_t(i)}{p(i)}-1)(-J_t(i) + J^{\mu_t}(i) + w_t(i)),
$$
where each $\chi_t(i)$ is a random variable satisfying $\chi_t(i)=1$ if state $i$ is selected and $\chi_t(i)=0$ otherwise. One can easily verify that
$\mathbb{E}[\hat{v}_t|\mathcal{F}_t]  = 0$ and
$$
\mathbb{E}[\norm{\hat{v}_t}^2|\mathcal{F}_t] \le \hat{A}+\hat{B}\norm{J_t}^2,
$$
where $\hat{A}$ and $\hat{B}$ are constants. Similar to Proposition \ref{prop:boundJ}, $J_t$ generated by (\ref{eq:newJ}) is bounded. The same argument as in the proof of Theorem \ref{prop:main} can be used to show that $J_t$ converges to $J^*$.

Now consider (\ref{eq:step1}). We can write the update rule as
\begin{equation}
    \label{eq:newJ2}
    J_{t+1} = (1-\hat{\gamma}_t)J_t + \hat{\gamma}_t(J^{\mu_t}+\hat{v}_t+\hat{u}_t),
\end{equation}
where $\hat{\gamma}_t=\frac{1}{t+1}$,
\begin{align*}
\hat{v}_t(i) &= w_t(i) + \bigg(\frac{\chi_t(i)}{p(i)}-1\bigg)(-J_t(i) + J^{\mu_t}(i)) \\
& \qquad + \bigg(\frac{(t+1)\chi_t(i)}{n_i(t)}-1\bigg)w_t(i),
\end{align*}
and
$$
\hat{u}_t(i) = \bigg(\frac{t+1}{n_t(i)}-\frac{1}{p(i)}\bigg)\chi_t(i)(-J_t(i)+J^{\mu_t}(i)).
$$
By the strong law of large numbers,  $\frac{n_i(t)}{t+1}\rightarrow p(i)$ in probability 1 as $t\rightarrow\infty$. It is easy to see that there exists  bounded and $\mathcal{F}_t$-adapted sequences $A_t$ and $B_t$ such that $\mathbb{E}[\hat{v}_t|\mathcal{F}_t]  = 0$ and
$
\mathbb{E}[\norm{\hat{v}_t}^2|\mathcal{F}_t] \le {A}_t+{B}_t\norm{J_t}^2.
$
Moreover, there exists an $\mathcal{F}_t$-adapted random
sequence $\theta_t$ such that $\theta_t\rightarrow 0$ in probability 1, as $t\rightarrow \infty$ and
$
\norm{\hat{u}_t}\le \theta_t(\norm{J_t}+1).
$
We can then use the same argument as in the proof of Theorem \ref{prop:main} to show $J_t$ converges to $J^*$ in probability 1, in which we need to use Proposition 4.5 in \cite{bertsekas1996neuro} (see also Proposition \ref{prop:sa} in the Appendix).
\end{proof}

\begin{rem}
All the convergence results obtained for the undiscounted case in this paper can be extended to the case of temporal difference $TD(\lambda)$ and model-free case ($Q$-learning) without much difficulty. Such results are left out due to the space limit. Interested readers should be able to refer to Sections 4 and 5 in \cite{tsitsiklis2002convergence} and combine the argument there with those in Sections \ref{sec:main}--\ref{sec:nonuniform} of this paper.
\end{rem}

\section{An illustrative example} \label{sec:ex}

We use a simple example (adapted from \cite[Example 5.11]{bertsekas1996neuro}) to illustrate the convergence behaviours of different variants of MCES.

\begin{exmp}\label{ex1}
We consider a discounted problem
(i.e. $\alpha<1$) with two states and deterministic transitions show in Figure \ref{fig:ex}. Note that a discounted problem can be turned into an equivalent shortest path problem \cite{bertsekas1995dynamic} by adding a terminal state and modifying the transition probability such that each transition has $1-\alpha$ probability reaching the terminal state.

The states consist of $S=\set{1,2}$ and the actions $A=\set{l,r}$. The transitions from each state under each action can be seen from Figure \ref{fig:ex}. We define the stage cost $g$ as $g(1,r)=g(2,l)=0$ and $g(1,l)=g(2,r)=1$. Intuitively, for each of the two states, the cost to move is 0 and the cost to stay is 1. This example was used in \cite{bertsekas1996neuro} to show possible divergence of iteration (\ref{eq:J3}) when we do not restrict the frequency of selecting each of the two states for value updates.

There are in total four different policies $\mu_l$, $\mu_r$, $\mu_g$, and $\mu_w$, defined as follows: $\mu_l(1)=\mu_l(2)=l$, $\mu_r(1)=\mu_r(2)=r$, $\mu_g(1)=r,\mu_g(2)=l$, and $\mu_w(1)=l,\mu_w(2)=r$. The intuitive meaning of $\mu_l$ is to always move to the node on the left, while $\mu_r$ is to always move to the right. The optimal policy $\mu_g$ moves from each node to the opposite node, and the ``worst" policy $\mu_w$ always stays at the current node. It is straightforward to compute the cost-to-go value for each policy as follows:
\begin{align*}
J^{\mu_w} = \begin{bmatrix} \frac{1}{1-\alpha} \\ \frac{1}{1-\alpha}\end{bmatrix}, J^{\mu_g} = \begin{bmatrix} 0\\0\end{bmatrix},
J^{\mu_l} = \begin{bmatrix} \frac{1}{1-\alpha} \\ \frac{\alpha}{1-\alpha}\end{bmatrix},
J^{\mu_r} = \begin{bmatrix} \frac{\alpha}{1-\alpha} \\ \frac{1}{1-\alpha}\end{bmatrix}.
\end{align*}
Furthermore, it can be verified that $\mu_l$ is a greedy policy for a value vector $J$, if $J(1)\le J(2)-\frac{1}{\alpha}$, and  $\mu_r$ is a greedy policy for a value vector $J$, if $J(2)\le J(1)-\frac{1}{\alpha}$. The optimal policy $\mu_g$ is a greedy policy for a value vector $J$, if $\abs{J(2)-J(1)}\le \frac{1}{\alpha}$. This is also depicted in Figure \ref{fig:sim}(d), where the two black dashed lines ($J(2)=J(1)\pm\frac{1}{\alpha}$) separate the domains of the different greedy policies.
Note that the optimal policy $\mu_g$ in this example does not satisfy the optimal policy feed-forward environment assumption in \cite{wang2020convergence}, because both states are revisited under the optimal policy.
\end{exmp}

\begin{figure}[ht]
    \centering
\begin{tikzpicture}[node distance=2cm and 1cm,>=stealth',auto, every place/.style={draw}]
    \node [place] (S1) {1};

    \coordinate[node distance=1.1cm,left of=S1] (left-S1);
    \coordinate[node distance=1.1cm,right of=S1] (right-S1);

    \node [place] (S2) [right=of S1] {2};

    \path[->] (S1) edge [loop above] node {$l$} ();
    \path[->] (S1) edge [bend left] node {$r$} (S2);
    \path[->] (S2) edge [bend left] node {$l$} (S1);
    \path[->] (S2) edge [loop above] node {$r$} ();

\end{tikzpicture}
    \caption{A two-state deterministic system. }%
  \label{fig:ex}
\end{figure}

\begin{figure}[ht]
    \includegraphics[width=0.5\textwidth]{J_plots_100K.pdf}
    \caption{Optimistic policy iteration on Example \ref{ex1} under different choices of stepsize: (a) Iteration (\ref{eq:nonuni} with stepsize choice (\ref{eq:step2}) with $\hat{\gamma}_t=\frac{1}{t+1}$. (b) Iteration (\ref{eq:nonuni} with stepsize choice (\ref{eq:step1}). (c)-(d) Iteration (\ref{eq:J3} with stepsize choice (\ref{eq:step2}) with $\gamma_t=\frac{1}{t+1}$. The black dashed lines in (d) indicate separated domains of the three greedy policies $\mu_l$, $\mu_g$, and $\mu_r$ from top left to bottom right.}
        \label{fig:sim}
\end{figure}

We simulate the optimistic policy iteration (\ref{eq:nonuni}) with different probabilities $p(1)=p$ and $p(2)=1-p$. Figures \ref{fig:sim}(a) and \ref{fig:sim}(b) show convergence of (\ref{eq:nonuni}) using stepsize choices (\ref{eq:step2}) and (\ref{eq:step1}), respectively. Results for iteration (\ref{eq:J3}) are shown in Figure \ref{fig:sim}(c) and \ref{fig:sim}(d). Convergence is observed for (\ref{eq:J3}) only when $p=0.5$ (i.e., in the case of uniform selection).

\section{Conclusions}\label{sec:con}

We investigated the convergence of optimistic policy iteration, also known as Monte Carlo Exploring Starts (MCES), for the stochastic shortest path problem. These results complement known partial results on this topic and thereby help settle this long-standing open question.

There are at least two possible extensions of this work. First, the results in this paper assume that only the initial state of a simulated trajectory is picked for value updates. It would be interesting to prove convergence for the first-visit and every-visit versions of MCES \cite{sutton2018reinforcement}, in which the first and every state visited on the trajectory, respectively, will be selected for value updating. Second, as pointed out in \cite{tsitsiklis2002convergence}, it would be interesting, and perhaps very challenging, to generalize the results to situations where function approximations are used to  represent values.

\begin{ack}                               %
The author gratefully acknowledges the NSERC DG, CRC, and Ontario ERA programs for funding support.
\end{ack}

\bibliographystyle{plain}        %

redpajama_set_name: RedPajamaArXiv


text:
home > features > One Night Only - May 11, 2016: 6th Annual Grateful Dead Meet-Up At The Movies: Sullivan Stadium 7/2/1989
https://www.dead.net/features/meet-movies/one-night-only-may-11-2016-6th-annual-grateful-dead-meet-movies-sullivan-stadiu
One Night Only - May 11, 2016: 6th Annual Grateful Dead Meet-Up At The Movies: Sullivan Stadium 7/2/1989
Come one, come all! Join us as we boogie down the aisles of your local moving picture house for the 6th Annual Meet-Up At The Movies. This year's special cinema event is a bona fide classic - the previously unreleased Grateful Dead concert from Sullivan Stadium in Foxboro, Massachusetts on July 2, 1989. Catch the Dead in fine form, kicking off their summer tour to a sold-out crowd of more than 60,000 fans, and pulling out surprises at every turn - from the rare show-opener "Playing In The Band" to the only time they ever started a second set with "Friend Of The Devil." The concert also includes a spectacular performance of "Wang Dang Doodle," one of the band's finest sing-along versions of "Tennessee Jed," two Bob Dylan covers ("Queen Jane Approximately" and "Quinn The Eskimo") along with a medley of Traffic's "Dear Mr. Fantasy" segued into the Beatles' "Hey Jude." Not to mention a sequence that will bring you back to the Dead circa 1973: "Truckin'"> "He's Gone"> "Eyes Of The World."
And if that's not enough, we've got an exclusive interview with Grateful Dead archivist David Lemieux on the making of the upcoming 12-disc July 1978: The Complete Recordings boxed set as well as previously unseen performance footage from Dead & Company.
"Grateful Dead Meet-Up at the Movies: Sullivan Stadium 7/2/89" will only be playing on the silver screen for one night! Come out and connect with Dead Heads in your neighborhood on Wednesday, May 11th at 7:00pm local time.
Tickets are now available at participating theater box offices and online at www.FathomEvents.com.
WATCH "HE'S GONE"
WATCH "TENNESSEE JED"
Grateful Dead Hour no. 1544
Week of April 23, 2018
Last of four featuring the complete unreleased soundboard recording of 6/12/80 in Portland. According to Deadhead legend, My St Helens erupted during "Fire on the…
The Season of "What Now?" (Again!)
Now... Where Were We?
Well, we've been pretty much right here — that is, wherever "here" might happen to be at any given moment in this strange, still-pretty-new world…
Submitted by lilgoldie on Fri, 04/22/2016 - 15:36
Homepage Feature blurb
Come one, come all! Join us as we boogie down the aisles of your local moving picture house for the 6th Annual Meet-Up At The Movies. This year's special cinema event is a bona fide classic - the previously unreleased Grateful Dead concert from Sullivan Stadium in Foxboro, Massachusetts on July 2, 1989.
Custom Teaser
Meet-Up At The Movies
Sullivan Stadium 7/2/1989
Submitted by _ on Fri, 04/22/2016 - 16:10
@the movies...
memorable even today.... Playin' Crazy Fingers and Wang Dang Doodle all in the first set....followed by da Buff and Philly, a Summer 89 trifecta.... I miss Jerry....
Submitted by marye on Fri, 04/22/2016 - 17:51
if you're going to Emeryville
find me and say hi.
Submitted by mbarilla on Sat, 04/23/2016 - 06:00
Can't wait for this showing !! Summer 89 Tour
I won free tickets last year for Alpine Valley 7.19.89 , going again to the Lennox AMC in Columbus and it was sure a hoot, but I'm surprised the cinema didn't go up in smoke. I will be ready for 7.2.89 showing and wearing the Summer Tour 89 shirt with the skeleton playing keys, wearing bandana, and smoking. I have been listening to 7.15.89 from Deer Creek and I am uncertain if there is video footage of that night, but would love to see if they have,, any way it's a sweet 1st set opening with "Bertha" and closing with "Bird Song" and some gems buried in the space between
11 years 1 month
Submitted by wissinomingdeadhead on Mon, 04/25/2016 - 04:44
Could be.....
An interesting night out as I'm taking the wife with me to this years MUATM. In typical fashion the wife asked me "what should I wear", I said anything you want she said "how about if I go naked" I said if this years MUATM were featuring a show from 1969 that would work but from 1989 not so much, she said "oh okay what are you wearing"? I said I'll go naked her reply to that was "don't do that you'll empty the theater" I love that woman. Like I said could be an interesting night out let's hope the night ends in a state of nakedness. I was hoping that this years MUATM would've featured either 8/4/76 or 11/24/78 alas it's not to be. Maybe next year.
Submitted by alvarhanso on Fri, 04/29/2016 - 15:51
He's Gone
That is an incredible version of He's Gone! Loved the Smokestack Lightnin' jamming at the end. Jerry singing Gospel style in the end was fantastic. I hope I can make it to a theater, none less than an hour away so it seems...
Submitted by jawn on Fri, 04/29/2016 - 17:56
Foxboro '89 was epic
One of my favorite shows. This He's Gone is one of my favorites I got to experience right up there with Louisville 90 and Cincy 85. Was an absolutely gonzo 20 year-old on fungi and Woodstocks at the time, but recorded some stuff in a notebook in between dancing and freaking at this show that still exists (however less than interesting it may be in retrospect). The look of Brent smiling and Bob and Jerry smiling back a couple times here is the closest I've come to actually being happy about anything in almost 3 months. Ungrateful Living. Had a taper friend's version of this for years. This pro version is the shit.
Submitted by parkerpines on Fri, 04/29/2016 - 19:25
Ready for E-ville!
That 'He's Gone' is one of the best ever. This is my church and Jerry is my pastor. The wife and I will see y'all in Emeryville!
Submitted by redemption searcher on Fri, 04/29/2016 - 19:44
MUATM
Cool story wissinomingdh. Any year works for nakedness. Have made this each year since the 2nd in 2012, missed the first year as I never heard about it, and then my father died in early 2012 and these have been a good tonic for the hardness of that. Maybe because they always remind me of how in his later years he asked me to find the GD channel on his satellite radio and put it in a preset after macular degeneration had taken its toll, and that makes me smile since he never really understood my obsession with them back in the day in the '80s/90s or how upset I was in '95 when Jer went to the great beyond. Looks like they may work their way through the entire '89 summer since that was all filmed. O.K. with me, 7/9 and 10 being my first shows, but hopefully there will be some older ones too. Always love seeing millennials at these things getting into Dead for first time.
Submitted by rzamotoo on Sat, 04/30/2016 - 11:30
Foxboro!
Last time I saw Brent performing the Rolling Stones song "Last Time". Epic Show!
Submitted by GratefulJedd on Mon, 05/02/2016 - 06:48
Please release films on BLURAY!
Please release these films on bluray! This is one of my favorite all-time Dead experiences but I am unable to attend the showing & my family would dearly love to see it any/ every day possible. Thanx for reading my comment. Be well, one & all...
Submitted by kevflaherty on Sat, 05/07/2016 - 05:25
Will be my first ever MUATM. Had tickets last two years and had to skip due to work. This year I'll be there!! Can't wait to see how it goes. Theater I am seeing it at is sold out so should be a fun time
Submitted by Andpeter on Tue, 05/10/2016 - 09:38
No Coeur dAlene This Year!
Have not missed one since they started several years ago but not playing in Coeur dAlene this year, very disappointed!
Submitted by doodahman58 on Tue, 05/10/2016 - 14:53
Shut out from MUATM
Very disappointed, went to the last two Meet Ups in Evansville, IN and had a great time. Evansville was on the list again this year until recently then it got yanked. Nearest theater is 90 miles away, no can do. Bummer!!
Submitted by MJE on Wed, 05/11/2016 - 20:37
Pretty Joyous!
I'm not sure how I could be the first one here to comment, but here I am....all alone. Well, it was a good show with some really great moments. I had never heard them do that "Dear Mr. Fantasy/ Hey Jude" routine before. Maybe I'm the only one who hasn't. I loved it. "Don't Ease Me In" almost caused me to levitate, I think. It just jumped at us with joyous power. Even the "Quinn" encore had that joy factor. I've been disappointed with the small size of the audience at every one of these events. I'm not sure how it's worthwhile to even run it at the two theaters I've been to over the four years. It's nice to have space and relax, but I'd like to at least make believe it's like the old days. Can't go home, I guess.
Submitted by wissinomingdeadhead on Wed, 05/11/2016 - 21:20
Just got home.....
From the Philly theater, Riverview Plaza, a rather large crowd was in attendance, dancing in the aisles, beach ball being tossed around & bathroom break at DRUMZ all in all a typical Dead experience. Here's hoping that next years MUATM is either 8/4/76 OR 11/24/78. I received many compliments on my SSFD Station 12 SYF t-shirt. The wife is out like a light, however I will have her give her thoughts on her very first MUATM in the AM, for a non Deadhead she had tons of fun!!!!!!! Her exact words were I've never seen dancing in the aisles in a movie theater before and who had the beach ball?!!!!!!! Also DL2 explaining how 1978 TCR came to fruition was informative, didn't it seemed like he was in own house? Very cool indeed. Now awaiting the FedEx guy with my very own copy of 1978 TCR. HAPPY MUATM DAY, DEADLAND!!!!!!!
Submitted by gcrueger on Wed, 05/11/2016 - 22:16
Miracle Ticket
A few dozen faithful gathered here in Atlanta and it's always good to find the tribe again. Great show (the vocal trade-offs in He's Gone and the Dear Mr. Fantasy / Hey Jude medley stand out as particularly unique), personal memories of Dylan and the Dead two years earlier at the same venue, enthusiastic crowd, and best of all a kind stranger gave me a ticket. No, I didn't have my finger in the air, and I was more than happy to wait in line behind two other people and pay my fair share, but fate brought me my first Miracle ticket :) Goes to show you don't ever know!
Submitted by muhlberg on Wed, 05/11/2016 - 23:10
Good Pick
What a rowdy show this must have been. I was surrounded by folks who were at the show, all the way on the other end of the country in Emeryville, California, Twenty Seven years later. Wavering audio quality, which might explain the lack of a soundboard seed for this show on archive org Btree, etc. Still, a standout performance with "x-factor" energy, obviously. I think the Playin' only began a show twice elsewise in the 1980s. Folsom Field 80' and MSG 82'. This has that nasty uninhibited and good-humored 89' confidence and interplay amidst. The brief renditions of "Playin -> Crazy Fingers" clearly stir their enthusiasm and Garcia is totally playful throughout. It's wild to see him dominate the second set, in leu of Weir's hesitation to take the lead after "He's Gone" Jerry steadily commands the jam towards "Eyes of The World." Strangely dark moments of this show. "Wang Dang Doodle" is straight up nasty delta blues, unlike the diluted arrangements in years to come. "To Lay Me Down" has depth to its rustiness. I think it's the song's sole performance of 89'? Even the second set opener "Friend of The Devil" is eerily off-kilter. From the audience recording I had heard in the past, the "Dear Mr. Fantasy" segment caught my attention. The MOATM revealed much more. Next show, Buffalo (legendary), onward to Philly, New York, and of course Alpine Valley. RELEASE A DVD OF THIS! PEOPLE WILL BUY. :)
Submitted by SPACEBROTHER on Wed, 05/11/2016 - 23:26
Caught it
This was a tremendously enjoyable show. I wouldn't hesitate to purchase this on CD/DVD. During the July '78 box promo, Dave mentioned a criteria for release being that a show must be as enjoyable to listen to, even after the fortieth time as it is the first time. Anything from Summer '89 would be as enjoyable even after hundreds (even thousands) of listens. Shame we're limited to only a single one time ever viewing of 7/2/89, 7/18/89, 7/19/89 and 4/12/72. The notion that there are no plans to release these is just....silly.
Submitted by redemption searcher on Thu, 05/12/2016 - 00:14
Lemieux?
Great time had by all. Except...when was the Lemieux interview about the '78 box? Was it at the very beginning before Foxboro '89? We walked in right before Playin' and then never saw it. After Foxboro was the Dead & Co. which my John Mayer-hating friend bitched about, then that was it. Fantastic Crazy Fingers and Eyes, and Fantasy>Jude>Sugar Mags finale. Had never heard this show. Jerry was so animated, at one point towards the end, he was smiling so much, whole theater started cheering!
Submitted by kevflaherty on Thu, 05/12/2016 - 02:11
Great Time
My first, but certainly not last. Had a great time with a good crowd at the theatre and yes Redemption Searcher, the DL2 interview for the 78 box was prior to Playin. they also played the Dead & Co stuff prior to show where I went
Submitted by SPACEBROTHER on Thu, 05/12/2016 - 02:16
I walked in while they were playing some Dead & Company, followed by a Fathom Events promo followed by the Lemieux clip. The Foxboro concert started promptly after that beginning with an awesome Playin'>Crazy Fingers>Wang Dang Doodle opener. If I had it on DVD, I would watch it again right now. The 6/21/89 Shoreline performance that was a pay-per-view back then would make another nice release. Way overdue for a release from this tour.
Submitted by Oroboros on Thu, 05/12/2016 - 03:45
I agree with Spacebrother, please make these MUATM
selections available for purchase. Yes, at times the video and audio may fluctuate in quality, but the PLAYIN' (pun intended) is so edifying. These shows have been a delight! To see the interplay and outerplay (is that a word, well it is now) was transportive. My wife and I, along with about 30 other people were delighted and cheered the boys repeatedly throughout the show. I implore Rhino to sell us some video. Teachers open the door, but you must enter by yourself.
Submitted by ffskitty on Thu, 05/12/2016 - 06:31
Dead & Co
Great time last nite. So good to see the band smiling/laughing. Anyone know when the Dead & Co segment was recorded? Noticeably more people in attendance this year. Yay
Submitted by Strider 808808 on Thu, 05/12/2016 - 06:19
Workingmans Band
Downtown Albuquerque theater was half full. Larger crowd than any prior years for GD movies. Very animated crowd. Many dancing. When I saw the Dead in the later 80s and 90s I already felt like an old timer. Now I damn near feel like old man time. Some of the aspects of the film that impressed me, it showed a band working hard. Jerry was so on and into the music it reminded me why WE love this band so much. I was also impressed with the "you are there" feeling of being on the stage with the band. The sound seemed to be edited to emphasize each band member as camera shots changed. I felt like I was hearing new sounds coming from the band. The crowd in the theater was quite a mix of young and old. It's sobering to see all those who may have been original long hairs fifty years ago. And here in 2016 to see where the legacy has evolved to is mind bending. And here's to the music of one of the hardest working bands in history.
stream order
That's weird that the Dead&Co. was before Foxboro for some people because it was at the end in WarringtonPA. Guess i don't really understand the stream to-the-theaters technology, because I thought it's at the same time and in same order for everyone..
Submitted by EBDead66 on Thu, 05/12/2016 - 07:05
Movie night!
Haven't missed one yet! Took a friend who's a VW bus-100 show veteran, but he's younger, all post Brett era. Small crowd here in our burg, but dancing & cheering. Huge crowd at that show! Jerry & the Rythym Devils on fire! Sound a bit off, but great just to be immersed in the sound & energy of Jerry & the whole band on a great night. On stage view shows how fragile they all were individually, but working hard together to get "there". When they get "there" & you are there, that's the best place to be!
Submitted by Lesh Life on Thu, 05/12/2016 - 07:06
MUATM!
Really enjoyed the movie/show here in Maryland. Theatre was 2/3 full, compared to barely 1/3 last year. All age ranges in attendance, which is great to see. Dead & Co. before and after the show was nice. I loved the energy and interplay of the band, and especially watching Jerry beaming ear to ear. Audio quality: Mostly great except Jerry was buried in the mix. There were times he seemed to drop out of the mix altogether, especially during solos. Did Bob do the final mix down? :) Oh well, I'm just happy we get to see these jewels from the vault--and hopefully we can purchase them someday. Can't wait till next year!
I think this was my favorite one yet.
Everybody happy. Everybody healthy. Playing their hearts out. Those cool banners. A moment when it seemed like everything was going great and might go on that way forever... Also, the camera crew did stunning work, and the boys were clearly comfortable enough with them that the chemistry was great. In Emeryville, the sound started out a little low but the theater folks fixed it fast.
Dead & Co.
In Emeryville they played it before and after the main event.
Submitted by TracyS on Thu, 05/12/2016 - 08:54
Missed it....
Hi,This year I missed the meet up... This was the first year I was in Myrtle Beach for the meet up, but it was also the 1st time that they didn't have the meet up playing @ the local theater. They even carried Fare Thee Well last summer. I was @ that Foxboro show & I am also hoping they release the DVD!!!!! I had told my friends to get the info at the end of the show for the "pre-order", but alas.... there was nothing.... If they don't release it, maybe it can be streamed for the poor souls who couldn't attend this year!.. Just a thought!
Submitted by _ on Thu, 05/12/2016 - 09:01
Re: Sullivan Stadium
Excellent show, first set was spectacular, second even better. Looking for commercial release to complete the trifecta along with truckin up to buffalo and Crimson white and indigo. Best 3 consecutive shows since 5/7 5/8 and 5/9 1977.... Goddamit I miss Jerry.
Submitted by Sun King on Thu, 05/12/2016 - 09:28
this was the first muatm i've been able to attend. our theater was not very crowded, less than 20 people. i agree that i thought jerry was low in the mix, too. i was curious if the fact that i saw it in a large movie complex if they felt they couldn't crank it. the movie started with david talking. then the show. then dead & co. they didn't even turn the lights down till after playin'! i found it interesting that this years release was the opening show of the summer tour, and last years was the last show of the tour! i was at the alpine show. my guess is the alpine show was better. the sugaree alone was worth the price of admission! after the movie we came home and listened to the 3rd cd of dave's pick 18, the 7/16 part. it was just awesome! i need to listen to more of the 7/17 again, but man o man 7/16 sounded good.
Submitted by Cryptical70 on Thu, 05/12/2016 - 10:22
OCTAPAD!!!!!!!
All of the musicians were on fire at this show, but the real star of the show was the OCTAPAD, the Rhythm Devils' eight-armed electronic percussion monster!!!!! I truly love the evolution of technology within the Dead's sound over the years. However, when the second set settled into the Drums/Space, nothing compared to what Mickey and Billy created. Such impressive and outlandish sound effects courtesy of Octapad. Man, the drums sounded huge and loud! Anybody know if Octapad is still manufactured and used by modern percussionists? Since this show was 27 years ago, there's a possibility that new gadgets have replaced the old ones.
Submitted by wolff on Thu, 05/12/2016 - 10:41
please put the dvd out it was so nice to see Happy Jer..Gosh i miss em
Submitted by sisterearth on Thu, 05/12/2016 - 12:00
and the band played on....
Was a groovy time last night. Only 8 people at the show! This theatre was deluxe, large reclining seats, a bar in the theatre, great sound quality! I enjoyed it a lot!Wore some groovy glow stuff and had fun dancing in my seat. A real good time!
Absolutely these are still used. Of course they've been updated, but Roland definitely still sells it.
Submitted by BobLoblaw on Fri, 05/13/2016 - 11:39
Great Meet-Up
Really had a splendid time at the screening last night. A good turnout of several dozen folks at the Riverside in Reno with a bit o' dancing and plenty of clapping. Biggest ovation was for the first iso of Brent. The vocal interplay at the end of He's Gone was tremendous and the unique set openers were treats of the highest order. Naturally the hometown crowd approved of the first line in Friend of the Devil!
Submitted by Lilgirlprd on Fri, 05/13/2016 - 07:16
Groovy!
Had a groovy time in Oklahoma City last night for the movie meet up. The theater was a little over half full and had one guy dancing freely on the floor. I wanted to dance with him so bad, but kept getting distracted by this echo that kept permeating Bobby and Jerry's vocals. It was pretty intense - the echo factor. It started 3-4 songs in and finally went away after they came back for The Mighty Quinn. The beginning didn't have any audio and the theater still had the lights up. Someone had to go tell the staff it had started. We all missed the beginning interview up until about a minute left to go before the show started. They guy who told the staff came back in (to a silent, grumpy theater) and said a bunch of teenagers were running it. I can't help but wonder if they were the reason the echo was so bad...maybe they were messing around with the audio? Anyway, aside from all that, a groovy time was had by all!
Re sound quality
Interesting as the SBD I have does not have the vocal issues mentioned. Wonder what the audio feed was?
Submitted by DonkeyOdie on Thu, 05/12/2016 - 14:01
Total Bust in Ocala, FL
TWELVE PEOPLE?!!!!TWELVE?!!!!! It was a travesty. A couple of them didn't even know the songs and were just looking for drugs. There were smatterings of applause after each song, one big WHOOP after the opening PITB>Fingers>Wang-Dang, but nothing more. No room to really dance, so I wound up drumming on the chair in front of me (may have caused irreparable damage to the poor seat back, but...) It was just sad. BUT, I had a blast in spite of them. Good show. Thought the "To Lay Me Down" was a little weak. Mickey seemed a bit aggressive on the drums for such a slow ballad. Even Jerry seemed a bit perturbed by the inability to keep the beat on it. Loved how Bobby and Brent broke up when they tried a little dual vocal during the "He's Gone". Got a taste of Dead and Co. both before and after the show (which was cool). In all, a good show with a lousy audience around me to enjoy it. Let's hope next year is better. Maybe a bit more advertising beforehand. We'll see.
Submitted by liketohike on Fri, 05/13/2016 - 11:55
A First
I am lucky to have attended my first ever MUATM with my husband at my local theater which is only a few miles from home. In true Saint Louis fashion there were lots of Heads in attendance.(Saint Louis loves the Dead!) The Saint Louis metro area had 3 showings at different theaters. Pretty sure there was something in the air that night... It was good to see young and old and even infants at the show. I was never able to see Jerry live and I feel that was as close as I can get. The crowd was excited and cheered after every song. There was some chatter during drums but it happens. The Dead and Co clip was played afterward and kept everybody from leaving, they just stopped in place and listened...some who hadn't heard them yet were amazed at John's guitar skills. Always happy to have a local Dead event and I will attend this one for as many years as they have it. MUATM started the month and year my husband and I were married so it is that much more special to us. Keep on Truckin'
Submitted by greenbean on Fri, 05/13/2016 - 12:04
Release it :)
Hope this one gets an official video release, really would like to see it again. Had fun at the theater!
Submitted by terrapin201 on Fri, 05/13/2016 - 13:47
This was the first meet up I went to. Truly was a meet up as I met someone I had not seen in at least 10 years. I had a great time. Show was very good, with Dear Mr. Fantasy>Hey Jude Reprise the highlight of the concert. Paramus NJ had a good crowd, about 80% full, about 200 people. Light applause after each song and no dancing in the aisles. No issues with sound or film quality. Excellent night and will definitely go again next year.
@likeytohikey...
So glad you made it! Congrats on the anniversary and welcome to the club! Repeat after me... "I'm a deadhead and I miss Jerry" The congregation responds in unison in a David Gans like disaffected tone... "We know" ;-)
Submitted by Cfweiss on Sat, 05/14/2016 - 09:41
Turn it up!
This was a great show, enjoyed the unusual setlist a lot. Jerry really looked like he was having a good time. Seeing his ear to ear grin felt great. As usual Brent nailed Mr. Fantasy. Heavy Traffic. The crowd was small and unfortunately so was the amount of volume we were provided. Clearly the theater employees didn't understand that we weren't there for book club. Bobby is the only guy I know of that can rock the shorty shorts and still be cool. Looki g forward to 2 nights of Dead and Co at the good ol' Apline Valley. Bobby really seems re-energized with this line up. It isn't the same band, but they have got it going on.
Submitted by GregL on Mon, 05/16/2016 - 10:54
They did this Saturday Night
They did this Saturday Night at the drive in, Cover band before the movie two screens to a packed house, crowd was great what a party Nothin left ta dew but :) :) :)
Submitted by bethy0543 on Tue, 05/17/2016 - 21:45
Please release this wonderful meet-up show!!!
Had a blast at the movie, and want so much to be able to get a blu-ray (with fabulous sound, of course :)) and possibly a cd as well???PlEEEEEEZE
Submitted by javaliga on Mon, 06/20/2016 - 12:50
Still Thinking About It
I agree that, for me, the Dear Mr. Fantasy > Hey Jude reprise was the highlight of the show. I had never heard that one before. Is there any recording of that (or similar) somewhere? [Update] According to David Lemieux, you can find that sequence on a 1988 Road Trip release or in the Spring 1990 box set. This was my fourth one. Getting to know the regulars. Theater in Germantown MD was about 1/3 - 1/2 full. They turned up the sound this year. Great time!
Submitted by _ on Wed, 05/25/2016 - 12:28
Re Sullivan Stadium
There is a SBD audio of the show floating around somewhere on the net...and agreed it was the first time I heard that medley as well...
Submitted by fourwindsblow on Sat, 06/04/2016 - 10:49
Here is the end of set 2 if anyone wants to hear it again. http://www.dead.net/features/tapers-section/october-19-october-25-2015
/sites/g/files/g2000007851/files/2018-10/grateful-dead-6th-annual-meet-up-at-the-movies-2016-foxborough-ma-1989-final.jpg
greenworld
I definitely love every part of it and I've also saved it to see new information on your site. Thank you hopefully useful. Obat Herbal Infeksi Lambung Obat Herbal Penetral Asam Lambung Tinggi Gastric Health Tablet
fourwindsblow


Reducing risk factors for Alzheimer's and promoting successful aging.
Stress can eat away, little by little or in large chunks, at both your physical and mental health. It is most threatening when it is long-term and chronic.
Stress triggers a reaction in your body: the body releases the hormone cortisol, the heart beats faster, blood pressure rises. Cortisol is better known as the "stress hormone" because stress causes its release from the adrenal cortex into the bloodstream. Too much and prolonged levels of cortisol can have a negative effect.
Over time, stress may cause or worsen depression, heart disease, lower immunity, increased abdominal fat and other conditions that are risk factors for dementia. Further, research shows that high blood pressure and/or high levels of cortisol may contribute to or speed the development of Alzheimer's disease or other memory loss. Compounding this, the body is less able to lower cortisol levels with age or illness.
In general, damage control to achieve healthy levels of cortisol includes fixing any underlying problems that are causing stress and relaxing your body and mind. Healthcare providers may suggest counseling and/or medication, and lifestyle modifications. Other lifestyle choices for successful aging like healthy diet, being active and socializing can also help manage stress.
While acute stress is good for us, waking up the immune system and stimulating us to perform better, it is chronic stress that is the demon.
Chronic stress is inflammatory, and inflammation stands at the root cause of disease, especially for the cardiovascular system and the brain. Stress shrinks the hippocampus; memory is compromised. Among multiple research studies on this point, the Framingham Heart Study found that people during midlife whose blood shows signs of inflammation are twice as likely to later develop Alzheimer's disease as people with no signs of inflammation.
When it comes to mitigating stress or preventing disease process, think Mediterranean. What can we learn from this lifestyle to turn stress into strength?
Put a positive spin on your life story. A sunny disposition sheds stress. It's not about feigned cheerfulness, but rather consciously reframing negatives into positives to release you from a worry loop.
Be kind. When you are kind and forgiving, you can let go of any perceived injustice. And don't forget to be as kind to yourself as you would be to a friend.
Embrace humor. Humor also works well to reduce worries to absurdities and release them.
Emote. Communicate what's on your mind, be spontaneous and at ease. On the other hand, people who experience chronic stress are self-silencing, self-suppressors, often sacrificial givers. It's okay to be kind and to give, but not if you are going to tip over with resentment--an example of the inflammatory process.
Be social. Smile when you greet strangers, schedule get-togethers with friends and family, and party. Happiness is contagious—the more, the merrier.
Be active. Not only will you shed stress hormones when you work out, but you will be more likely to reinterpret a problem into a solution when endorphins—our own feel-good brain chemicals—are released as a relaxation response.
Follow a Mediterranean diet. The regimen makes for a healthy diet for body and mind.
Drink coffee daily. It's a potent anti-oxidant and mood elevator.
Get sleep. Sleep deprivation can perpetuate stress. You will wake up irritable, and anything could set you off.
Ultimately, stress management means preparing for life's instabilities and setting aside time to revitalize. While you might not be able to deal with the big stressors in your life, just yet, you can begin to handle the little ones. Small steps, giant gains.
For caregivers, stress often comes with the territory, given the enormous emotional and practical toll of their responsibilities.
These tips may help caregivers reduce stress, mitigate burnout and age successfully.
Get educated. Knowledge is empowering so learn more about your loved one's illness, by reading books, attending workshops and consulting with healthcare professionals.
Learn caregiving techniques. Key areas are communication skills, safety concerns and managing behavioural challenges and activities of daily living.
Think positive. Focus on your loved one's remaining strengths.
Foster communication with healthcare providers. Be involved in your loved one's medical care. Ask questions about the progression of the disease, express concerns and discuss treatment options.
Take care of financial, legal and long-term care planning issues. Try to involve your loved one, if he or she is still capable, in decision-making related to future care and end-of-life issues.
Make time for yourself. Join caregiver support groups. Pursue interests beyond your caregiving role, such as music, hobbies, journaling and art—and don't feel guilty about carving out your own space.
Maintain your own physical and mental health. Do physical and mental exercises, socialize, pursue other leisure activities, get enough sleep and have regular checkups. Seek medical help if signs of depression emerge.
Don't go it alone. Support systems are critical so discuss your situation with family and friends, and set up a care team. Join a support group to connect with peers in your community.
Reach out for help. Take advantage of national and local organizations that provide counseling, information, support groups, respite care services, etc. To contact the Alzheimer's Foundation for Caregiving in Canada (AFCC), click here.
Information contained on this site should not be considered a replacement for consultation with a qualified healthcare professional.
Please note that research studies are open to interpretation, and new research is continually emerging.
©2013 Alzheimer's Foundation for Caregiving in Canada. All rights reserved. Website Design Phoenix - Gettingraphic Design.


text:
King's Ely uses cookies to improve your experience. By using the site you agree to our privacy policy. Learn more
School Portals
Old Eleans
The Spirit of King's
Pupil achievements
Acremont admissions
Junior admissions
Senior admissions
King's Ely Acremont & Nursery
King's Ely Junior
Activities and Co-curriculum
Ely Cathedral Boys' Choir
Ely Cathedral Girls' Choir
King's Ely Senior
School Journalism
Taster Days
Welcome to King's Ely; it is a remarkable school. People have been educated here, in the shadow of Ely Cathedral, for well over 1,000 years, which means we have had plenty of practice at doing our job. However, we are not about historic achievements or illustrious alumni but about children and young people because they are our future.
We look after the academic and pastoral needs of around 1,000 boys and girls aged 1 to 18 with boarders from 7 years old. We came into being to educate choristers and now we have two cathedral choirs, junior boys and senior girls, with choral scholars of both genders in the Sixth Form. That is a small part of the story and King's Ely is a school where we believe education is an adventure. Students of all ages are encouraged to take risks in their learning, pushing themselves beyond the boundaries of their expectations, discovering more about the world around them, and, in so doing, more about themselves. What makes us different is our determination to instil in the young people in our care a real enthusiasm for learning and our belief that all students can achieve when their learning style is fully engaged. This can be challenging, even uncomfortable, but King's Ely students know that they are well supported and that their teachers believe in them, so they take the risks that carry their learning to new heights.
Sue Freestone
The Spirit of King's
Follow us @Kings_Ely
Our Year 8 King's Ely Junior students were set the ultimate challenge just before the end of term in their annual F… https://t.co/xk9xmN3eiH
Alan Parkinson, Head of #Geography at King's Ely Junior, is one of the first teachers in the UK to become a United… https://t.co/A2UWSVekvs
We have a vacancy for a Dining Hall Assistant! 😀 The role is 25 hours per week, Monday to Friday from 9am-2.30pm du… https://t.co/DtV4NZiDH4
At King's Ely Acremont, every single day is an adventure. Discover more by coming along to our next Open Event on F… https://t.co/u8CC00Rohs
Our Year 10 #Geography students enjoyed a fieldtrip to Preston Montford Field Studies Centre in Shrewsbury just bef… https://t.co/t7OTk2Jy9k
We empower our pupils to challenge themselves and push beyond the boundaries of their own expectations to achieve more than they ever believed possible. Along the way they learn and live the values of Energy, Courage and Integrity.
Girls and Boys | Ages 1- 18 | Day and Boarding
Ely, Cambridgeshire
admissions@kingsely.org
School web design by TWK


U.S. President Trump listens at a meeting on cyber security in the Roosevelt Room of the White House January 2017.
Planning and information-gathering matters, but experts caution that for the EO to succeed long-term, it needs to exit this phase as quickly as possible. Beginning the urgent, proactive work of implementing a robust national cybersecurity posture and defending critical infrastructure can't wait, as destabilizing cyberattacks like the WannaCry and NotPetya ransomware outbreaks showed this summer.
The goal of a speedy review process is a good one, but has not materialized. "Unfortunately, leadership from the executive branch on cybersecurity has been weak," senator John McCain said at the end of August. "The last administration offered no serious cyber deterrence policy and strategy. And while the current administration promised a cyber policy within 90 days of inauguration, we still have not seen a plan."
White House and agency officials wouldn't detail a tally of which EO deadlines have been met, but said that the reports are coming along. "Departments and agencies continue implementing Cybersecurity Executive Order 13800 and have made significant progress," a National Security Council spokesperson tells WIRED. "While they continue to work towards the deadlines outlined in the Executive Order, the release of products may vary over time. However, many of the deliverables will be used to inform work going forward."
Some of the progress has been tangible, and so far, at least, it seems that agencies are trying to stay on top of the imposed deadlines. On Wednesday, the White House's American Technology Council and Office of American Innovation even submitted its draft federal IT modernization report (stipulated by the EO) to the President.
"I was part of that group that said 'the Executive Order looks really good. It could be better, but we all know that it could be a lot worse, and the true test will be in the execution,'" says Kiersten Todt, a researcher at the University of Pittsburgh Institute for Cyber Law, Policy, and Security who was the executive director of the Commission on Enhancing National Cybersecurity under Barack Obama. "In talking with those in the White house and at the Office of Management and Budget, my understanding is that for the first wave of reports, which were due on August 9, most of those came in."
There is other friction as well. Numerous members of the National Infrastructure Advisory Council (which advises the Department of Homeland Security on infrastructure issues and cybersecurity) resigned last week, citing diverse concerns about the Trump Administration's agenda, including about cybersecurity. "You have given insufficient attention to the growing threats to the cybersecurity of the critical systems upon which all Americans depend," they said in a resignation letter obtained by NextGov.
Experts fear that some combination of infighting and staffing shortages at federal agencies could slow the executive order's momentum. "To the extent that you've got staffing gaps, that's going to inhibit your ability to do implementation. It's going to slow you down," says Michael Daniel, president of the Cyber Threat Alliance and a former cybersecurity coordinator for the White House. "The timelines in the executive order are achievable, but some of them are pretty aggressive."
Though the EO is halfway through its first year, there is still time to complete its initial phase and move forward on schedule. "I don't think we're going to get a report card on which reports were and weren't completed, but I work with a lot of those agencies and I know they started acting on it," says Josh Corman, a cybersecurity policy expert at the Atlantic Council. "But whatever those reports were prepared for should be tempered by the more recent game changers such as NotPetya that did some pretty nasty damage to critical infrastructure."
Some say that the year-long analysis and planning phase laid out in the executive order is worthwhile, while others argue that the need for action is more pressing. Given the very real threats to national digital security and infrastructure, at this point the EO must at the very least stay as close to schedule as possible at this point the EO must simply stay as close to the schedule as possible so the federal government is ready to address very real potential threats to national digital security and infrastructure.


Prices for apartments in Ho Chi Minh City's (HCMC) primary market rose 1.7% q-o-q and 3.6% y-o-y in 4Q2017, according to the Vietnam Property Market Brief by JLL.
A total of 15,193 units were launched in the Vietnamese city in 4Q2017, which is a 29% increase q-o-q and 71.2% increase y-o-y. Most of the units launched were in the mid-end segment, which accounted for 45% of total launches. Sales, totalling 16,181 units in 4Q2017, outpaced the number of units launched. Units priced from US$1,000 to US$2,000 psm ($124 to $248 psf), the mid-end segment, accounted for 58% of the sales volume, notes JLL. And a good proportion of the sales, 35.2%, were of units that will be completed and handed over by end-2018.
In HCMC's secondary market for apartments, prices also rose in 4Q2017, but the pace of 0.1% q-o-q and 0.5% y-o-y was more subdued compared with those in the primary market. The performance was better than in 3Q2017, however, which registered a price decline of 0.7% q-o-q and a slight 0.1% y-o-y increase.
In the landed residential segment, which comprises villas and townhouses, primary market prices fell 3.8% q-o-q in 4Q2017, but registered a 13.6% y-o-y increase. The number of units launched totalled 1,802, which is a 30% q-o-q increase. Owing to strong market momentum, 2,139 units were sold, says JLL, who adds that this is nearly triple the 4Q2016 figure, and the highest on record since 2000.


You are invited to attend our Critical Mass gallery exhibition opening reception. Meet the artists and catch a glimpse of the unique and varied art being created every day at the Millworks.
Self-parking options will be limited for this event. We encourage everyone to use rideshare, carpool, public transit or ride a bike.
We strongly encourage you to consider using Uber or Lyft. That way you can enjoy a nice glass of wine and get whisked from your door to our door with no hassle.
Artworks in the exhibition will be available for sale during the opening reception and throughout the exhibition.
"This is going to be fun and awesome!"
Why are we hosting a gallery exhibition at the Millworks? To show off the awesome talent and awesome art being made right here in the community. That's why. We have over 80 incredibly people creating art and other works in our studios everyday and this their chance to show the world what they do.


Unfortunately, professional review of the Ara Fell game is not yet ready. This game is on the list and will be reviewed in the nearest feature. Meanwhile, you can find more from the official description below.
We publish only honest game reviews based on real game experience. If you think that our Ara Fell game review is not full enough and you have something to add - write to us. We looking for talented authors to make our reviews better!
There's not a lot of questions about Ara Fell. Ask insistently in the comments so you can make it frequent!
©2016. Stegosoft Games. Developed by Stegosoft Games. Licensed to and published by Stegosoft Games. Stegosoft Games, Ara Fell and the Ara Fell logo are trademarks of Stegosoft Games. All rights reserved.