%!TEX root = uber-driver-strategy.tex
\section{Uber Driver strategies}
In this study, we compare four kinds of Uber drivers,
\begin{enumerate}
    \item Naive Uber driver with no strategy.
    \item Uber driver with re-location strategy.
    \item Uber driver with flexible work schedule strategy.
    \item Uber driver with both flexible work schedule and re-location strategy.
\end{enumerate}
In the following sections, we formulate each of the above four problems.

\section{Problem Formulation}
Consider a set of $n$ discrete zones of a city denoted by $\mathcal{X}$. During our data collection we sample rides for each pair of zones to collect information for the route suggested by Uber every 15 minutes. Since the optimal route suggested by Uber to its drivers can change based on the traffic conditions prevalent at the given time of the day, it is essential to model each of the variables for every time step. Hence, there is an implied time superscript for each of the variables described in the table below.
\begin {table}[h]
\caption {Description of Notation}
\label{tab:notation1}
\centering
\begin{tabular}{l c p{.5\textwidth}}
\toprule
       &                  & Definition \\
\midrule
$\mathcal{X}_0$&:& Home zone of the Uber driver \\
$N$&:& Maximum work time units for Uber driver \\
$B$&:& Overall budget time units within which to consume $N$ work time units. \\
$d_{ij}$&:& Distance between zones $i$ and $j$  \\
$c_{ij}$&:& Driver cost for trip from zone $i$ to zone $j$ \\
$e_{ij}$&:& Driver earnings from a passenger trip from zone $i$ to zone $j$ \\
$\tau_{ij}$&:& Travel time from zone $i$ to zone $j$  \\
$s_{i}$&:& Surge multiplier active at zone $i$ \\
$\lambda_{i}$&:& Poisson passenger arrival rate at zone $i$ \\
$\mu_{i}$&:& Poisson driver arrival rate at zone $i$ \\
$f_{ij}$&:& Fraction of passengers at zone $i$ destined to zone $j$ \\
\bottomrule
\end{tabular}
\end {table}

To simplify the notation, let us denote the net reward of a trip from zone $i$ to zone $j$ with $r_{ij} = s_i e_{ij} - c_{ij}$. However, the driver earnings in case of an empty ride (without a passenger) are zero resulting into a negative net reward. Our algorithms described in subsequent sections assume non-negativity of rewards. A slight modification allows us bypass this problem. Let $C = \max_{i,j}\{c_{ij}\}$\footnote{We take maximum of $c_{ij}$ over all time intervals.}. Our modified net reward $r'_{ij} = r_{ij} + C$ ensures the non-negativity of the net reward for every ride. Since every reward is offset by equal amount, our modification does not affect the optimal decision at any step in the Markov Decision Process. However, we take this modification into account while calculating overall driver earnings in each of the strategies. Here onwards, we drop the prime notation while refering to this modified net reward.
\subsection{Interpreting empirically observed transition frequencies matrix $F$}
The empirically observed transition matrix is denoted by $F$, each of its entries being $f_{ij}$ as described in the table above. By adjusting the size of city zones such that the diameter of the city zone is maximum walking distance, we can ensure that there are no within-zone passenger rides i.e., $f_{ii} = 0$. We modify the matrix $F$, denoting it by $F'$, such that the entry $f'_{ii}$ indicates the probability that an Uber driver unsuccessfully `busy-waits' for a passenger within zone $i$, i.e., driver does not find a passenger in a unit time interval. Furthermore, for every row of the matrix we re-normalize the entries $f'_{ij}, i \neq j$ such that they denote conditional probabilities of a driver from node $i$ taking a ride to node $j$, given that he finds a passenger.

\subsection{Modeling probability of successful busy-wait}
For brevity, again, we drop the time superscript in this section. Let $\lambda$ and $\mu$ denote the poisson arrival rates of passengers and drivers within a given zone. Let the number of passenger and driver arrivals in a unit time interval be denoted by $N(\lambda)$ and $N(\mu)$ respectively. Assuming that passenger and driver arrivals follow independent Poisson processes, the random variable $K = N(\lambda) - N(\mu)$ follows a Skellam distribution.
\begin{eqnarray}
\mathrm{Pr}[K=k] &=& e^{-(\lambda +\mu)} \bigg(\frac{\lambda}{\mu}\bigg)I_{k}\Big(2\sqrt{\lambda \mu}\Big)
\end{eqnarray}
where $I_k(z)$ is the modified Bessel function of the first kind. We can depict the values taken by the random variable $K$ as states of a Markov Chain. Using this, we can model the entry $f'_{ii}$ of the modified transition frequency matrix.
\hc{Although for simplicity, we assume the independence of the passenger and driver arrival processes, we can accomodate correlated processes as well with slight modification.}
\begin{center}
\begin{tikzpicture}[->, >=stealth', auto, semithick, node distance=2.5cm]
\tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=1]
\node[state]    (-inf)               {$...$};
\node[state]    (-2)[right of=-inf]   {$-2$};
\node[state]    (-1)[right of=-2]   {$-1$};
\node[state]    (0)[right of=-1]   {$0$};
\node[state]    (+1)[right of=0]   {$1$};
\node[state]    (+2)[right of=+1]   {$2$};
\node[state]    (+inf) [right of=+2] {$...$};
\path
(-inf)  edge[bend left]         node{$\lambda$}   (-2)
(-2)    edge[bend left]         node{$\lambda$}   (-1)
        edge[bend left,below]   node{$\mu$}       (-inf)
(-1)    edge[bend left]         node{$\lambda$}   (0)
        edge[bend left,below]   node{$\mu$}       (-2)
(0)     edge[bend left]         node{$\lambda$}   (+1)
        edge[bend left,below]   node{$\mu$}       (-1)
(+1)    edge[bend left]         node{$\lambda$}   (+2)
        edge[bend left,below]   node{$\mu$}       (0)
(+2)     edge[bend left]        node{$\lambda$}   (+inf)
        edge[bend left,below]   node{$\mu$}       (+1)
(+inf)  edge[bend left,below]   node{$\mu$}       (+2);
\end{tikzpicture}
\end{center}

Whenever the above Markov Chain is in non-positive state, there are more drivers than passengers in a zone. We assume the worst case scenario in which our strategic driver is at the last position in a FIFO queue. Hence, for a state $k \leq 0$ of the Markov Chain, the driver has to wait for $(|k| + 1)$ passenger arrivals in a unit time interval for a successful busy-wait resulting in a passenger ride in the same time interval. Since the passenger arrival process is a Poisson process with rate $\lambda$, the probability of $k$ passengers arriving in a unit time interval is given by,
\begin{eqnarray*}
\mathrm{Pr}[N(\lambda) = k] &=& \frac{\lambda^{k}e^{-\lambda}}{k!}
\end{eqnarray*}
Hence, probability of a driver finding a passenger in a unit time interval within zone $i$ is given by,
\begin{eqnarray}
1 - f'_{ii} &=& \sum_{k \leq 0} \mathrm{Pr}[K = k] \times \mathrm{Pr}\big[N(\lambda) = |k| + 1 \big] \nonumber \\
f'_{ii} &=& 1 - \sum_{k \leq 0} \mathrm{Pr}[K = k] \times \mathrm{Pr}\big[N(\lambda) = |k| + 1 \big] \\
f'_{ij} &=& (1 - f'_{ii}) \times f_{ij}
\end{eqnarray}
The above algebraic transformation maintains the row stochasticity of the matrix $F'$. Furthermore, $\forall i,j: f'_{ij} > 0$ (assuming that there is at least one passenger ride between each pair of zones in any time interval). Here onwards, for simplicity of notations, we always refer to this modified empirically observed transition matrix as $F$.

\section{Naive Strategy}

In the naive strategy, an Uber driver essentially performs a random walk over the city zones. At the end of every ride, the driver waits in the same zone for his next passenger pickup. We refer to the action of waiting for a passenger pickup in current zone as `busy-waiting' and denote it by $a_{bw}$. Hence, in the naive strategy, the set of available actions for a driver at any stage is a singleton set, $\mathcal{A} = \{a_{bw}\}$. Let $R(i, a)$ denote the expected immediate reward of taking action $a$ while within zone $i$. In the naive strategy, for the only action $a_{bw}$, we have $R(i,a_{bw}) = \vec{f}_i.\vec{r}_{i}$ where $\vec{f}_i = \{f_{i1}, ..., f_{in}\}$ is the row vector of the transition matrix $F$, and $\vec{r}_{i} = \{r_{i1}, ..., r_{in}\}$ is the net reward vector associated with each transition, such that $r_{ii} = 0$.\\

In the dynamic program modeling of the naive strategy, the expected overall earnings i.e., value function at time $t$, for a zone $i$, $v_t(i)$ is given by,
\begin{eqnarray*}
v_t(i) &=& R(i, a_{bw}) + \vec{f}_i.\vec{v}_{t'}
\end{eqnarray*}
where $\vec{v}_{t'} = \{v_{t + \tau_{i1}}(1), ..., v_{t + \tau_{in}}(n)\}$ is the value function value at time $t + \tau_{ij}$ corresponding to each transition from zone $i$ to zone $j$. Note that $v_{t + \tau_{ii}}(i) = v_{t+1}(i)$ as it simply implies unsuccessful busy-wait.

\section{Relocation Strategy}
In the relation strategy, an idle Uber driver in zone $i$ can perform one of the two actions:
\begin{enumerate}
    \item Busy wait for a passenger ride request in the current zone, denoted by $a_{bw}$.
    \item Take an empty ride to a different zone $j$ in the city, denoted by $a_j$, where $j \neq i$.
\end{enumerate}
Hence, in the relocation strategy, the set of available actions for a driver at any stage, $\mathcal{A}$ is a set of $n$ elements. Like the previous strategy, let $R(i, a)$ denote the expected immediate reward of taking action $a$ while within the zone $i$. Hence, we have,
\begin{enumerate}
\item $R(i, a_{bw}) = \vec{f}_i.\vec{r}_{i}$ where $\vec{f}_i$ and $\vec{r}_i$ are as defined for the naive strategy.
\item $R(i, a_j)_{j \neq i} = r_{ij}$
\end{enumerate}

In the dynamic program modeling of the relocation strategy, the value function at time $t$, for a zone $i$, is given by,
\begin{eqnarray*}
v_t(i) &=& \max
    \begin{cases}
    R(i, a_{bw}) + \vec{f}_i.\vec{v}_{t'}, \\ \\
    \max_{a_j} \bigg\{R(i,a_j) + v_{t + \tau_{ij}}(j)\bigg\}, j \neq i, t + \tau_{ij} \leq N
    \end{cases}
\end{eqnarray*}
where $\vec{v}_{t'}$ is same as defined for the naive strategy.

\section{Flexible Schedule Strategy}
In the flexible schedule strategy, every Uber driver has an allocated overall budget of $B$ time units within which he works for a maximum of $N$ time units. The motivation of this strategy is as follows. A typical Uber driver desires to work for a maximum of 40 hours per week. However, it is a non-trivial problem to determine which are the optimal 40 hours due to varying demand and surge rates all through the week. The flexible schedule strategy aims to figure out the optimal in expectation work schedule for 40 hours in a week of a total 168 hours. In this strategy, we assume that a driver can log out of the system any time to return to his home zone as a part of his flexible schedule. Hence depending on the current zone of the driver, he has following set of actions available to him.
\begin{itemize}
    \item \textbf{If driver is in home zone}
    \begin{enumerate}
        \item Idle wait in the home zone, $a_{iw}$. Idle waiting involves logging out of the system till a more optimal in expectation time to re-enter the system occurs.
        \item Log into the system and busy wait for a passenger ride request in the home zone, denoted by $a_{bw}$.
    \end{enumerate}
    \item \textbf{If driver is not in home zone}
    \begin{enumerate}
        \item Busy wait for a passenger ride request in the current zone, $a_{bw}$.
        \item Log out of the system, drive back to home zone to idle wait over there, $a_{iw}$.
    \end{enumerate}
\end{itemize}
However, we can merge the above to situations into a single set of actions by accounting for the extra cost of an empty ride to home zone that is imposed on a driver not in the home zone, while calculating the net rewards vector. Hence, in the flexible schedule strategy, the set of available actions for a driver at any stage is $\mathcal{A} = \{a_{iw}, a_{bw}\}$. The expected immediate rewards of taking a particular action while within zone $i$ are as follows:
\begin{enumerate}
    \item $R(i, a_{iw}) = r_{ij}$, where $r_{ii} = 0$.
    \item $R(i, a_{bw}) = \vec{f}_i.\vec{r}_i$ where $\vec{f}_i$ and $\vec{r}_i$ are as defined for the naive strategy.
\end{enumerate}

Now, in the dynamic program modeling of the flexible schedule strategy, the value function is defined at time $t$ and budget $b$, for a zone $i$. It is given by,
\begin{eqnarray*}
v_{t,b}(i) &=& \max
    \begin{cases}
    R(i, a_{bw}) + \vec{f}_i.\vec{v}_{t',b'} \\ \\
    R(i, a_{iw}) + v_{t,b'}(\mathcal{X}_0)
    \end{cases}
\end{eqnarray*}
where in case of busy-waiting action, $\vec{v}_{t',b'} = \{v_{t + \tau_{i1}, b + \tau_{i1}}(1), ..., v_{t + \tau_{in}, b + \tau_{in}}(n)\}$ is the value function value at time $t + \tau_{ij}$ and budget $b + \tau_{ij}$, corresponding to each transition from zone $i$ to zone $j$. Note that, $v_{t + \tau_{ii}, b + \tau_{ii}}(i) = v_{t+1, b+1}(i)$ as it simply implies unsuccessful busy-wait. In case of idle waiting action, $v_{t, b + \tau_{i\mathcal{X}_0}}(\mathcal{X}_0) = v_{t, b+1}(\mathcal{X}_0)$ if $i = \mathcal{X}_0$. This takes into account that while idle-waiting, only the budget time units are consumed.

\section{Relocation + Flexible Schedule Strategy}
This is the most general strategy formed by combining the relocation as well as flexible schedule strategies described in the previous sections. In this strategy, a driver has complete freedom for decisions regarding work schedule as well one regarding taking empty rides to different zones of the city in order to maximise total expected reward. Just like the flexible schedule strategy, a driver is allocated overall budget of $B$ time units within which he works for a maximum of $N$ time units. An idle driver in zone $i$ has following set of actions available to him at any stage in the strategy.
\begin{enumerate}
    \item Busy wait for a passenger ride request in the current zone, $a_{bw}$.
    \item Idle wait in the home zone, $a_{iw}$.
    \item Empty ride to a different zone $j$ in the city, denoted by $a_j$, where $j \neq i$.
\end{enumerate}
Hence, in this strategy, the set of available actions for an idle driver at any stage $\mathcal{A}$ is a set of $n+1$ elements. The expected immediate rewards of taking particular action while within zone $i$ are as follows:
\begin{enumerate}
    \item $R(i, a_{bw}) = \vec{f}_i.\vec{r}_i$.
    \item $R(i, a_{iw}) = r_{ij}$ where $r_{ii} = 0$.
    \item $R(i, a_j)_{j \neq i} = r_{ij}$.
\end{enumerate}

In the dynamic program modeling of this strategy, the value function is again defined at time $t$ and budget $b$, for a zone $i$. It is given by,
\begin{eqnarray*}
v_{t,b}(i) &=& \max
    \begin{cases}
    R(i, a_{bw}) + \vec{f}_i.\vec{v}_{t',b'} \\ \\
    R(i, a_{iw}) + v_{t,b + \tau_{i\mathcal{X}_0}}(\mathcal{X}_0) \\ \\
    \max_{a_j} \bigg\{R(i,a_j) + v_{t + \tau_{ij}, b + \tau_{ij}}(j)\bigg\}, j \neq i, t + \tau_{ij} \leq N, b + \tau_{ij} \leq B
    \end{cases}
\end{eqnarray*}
where all notations are as defined in the previous sections.
% \section{Uber Driver with re-location strategy}

% \note{Need to re-formulate to include the last home trip of Uber driver at the end of the day.}\\

% Consider a set of \emph{n} nodes as vertices of a graph $G=(V,E)$ defined over a geographical area. We are interested in creating an optimal contingency plan of trips, for a particular driver who starts his day at node $v_0 \in V$. We assume that the driver decides the maximum units of service time $T$ that he would spend ferrying passengers at the beginning of the day. Let time $t$ denote the number of time units left for a driver before the end of his pre-decided service time.
% \begin{itemize}
% \item For $i,j \in V, d_{ij}$ is the Euclidean distance between the two nodes. The costs to the driver in terms of gas and vehicle depreciation value in traversing this edge is $c_{ij}$ and his earnings in ferrying a passenger from $i$ to $j$ is represented by $p_{ij}$. Time required to traverse the edge is denoted by $\tau_{ij}$. The costs, the earnings and the time required to make this ride are all proportional to the distance travelled.

% \item Let $S^{(t)}_{n \times 1}$ be the surge multiplier vector over the nodes of the graph. Each entry $s_i^{(t)}$ denotes the surge multiplier in effect at node $i$ at time $t$. When $s_i^{(t)} > 1$, the driver earning from ferrying a passenger from node $i$ to $j$ of the graph is $s_i^{(t)} \times p_{ij}$.

% \item The arrival process of passengers at node $i$ at time $t$ is a Poisson process with rate $\lambda_i^{(t)}$. Let vector $\Lambda^{(t)}_{n \times 1}$ denote the passenger arrival rates across all the vertices of $G$ at time $t$. While some passengers may arrive simultaneously (compound Poisson process), we assume that this effect is negligible. Furthermore, we assume that if the driver is present at node $i$ when a passenger arrives, he picks up the passenger with no further delay. As the inter-arrival times of passengers are exponentially distributed, the expected value of idle time for a driver waiting at node $i$ is $1/\lambda_i^{(t)}$. We implicitly assume that the passenger arrival process is stationary for a finite time slice around $t$.

% \item The matrix $M^{(t)}_{n \times n}$ represents the transition probabilites between the nodes of the graph such that the fraction of passengers at node $i$ whose destination is node $j$ at time $t$ is represented by entry $m_{ij}^{(t)}$, where $\forall t, m_{ij}^{(t)} \in \mathbb{R}_{\geq 0}, m_{ii}^{(t)}=0$ and $\sum\limits_{j}m_{ij}^{(t)}=1$.

% \item H(.) denotes the Heaviside step function. \\
% \begin{eqnarray*}
% H(x) = \max
%     \begin{cases}
%     0, & \text{if } x < 0, \\
%     1, & \text{if } x \geq 0
%     \end{cases}
% \end{eqnarray*}
% \end{itemize}

% \begin {table}[h]
% \caption {Description of Notation}
% \label{tab:notation2}
% \centering
% \begin{tabular}{l c p{.5\textwidth}}
% \toprule
%        &                  & Definition \\
% \midrule
% $d_{ij}$&:& Distance between nodes $i$ and $j$  \\
% $c_{ij}$&:& Driver cost for trip from node $i$ to node $j$ \\
% $p_{ij}$&:& Driver earnings from passenger trip from node $i$ to node $j$ \\
% $\tau_{ij}$&:& Travel time from node $i$ to node $j$  \\
% $s_{i}^{(t)}$&:& Surge multiplier active at node $i$ at time $t$ \\
% $\lambda_{i}^{(t)}$&:& Poisson passenger arrival rate at node $i$ at time $t$ \\
% $m_{ij}^{(t)}$&:& Fraction of passengers at node $i$ destined to node $j$ at time $t$ \\
% $H(.)$&:& Heaviside function \\
% \bottomrule
% \end{tabular}
% \end {table}

% We formulate a dynamic program to find the contingency plan to maximise the expected earnings of a driver in his service time $T$. Each entry $OPT(t,i)$ of the matrix $OPT$ represents the maximum expected earnings in the remaining $t$ units of service time when the driver is stationed at node $i$. The maximum expected revenue of the driver at the beginning of the day is thus $OPT(T, v_0)$.

% \begin{itemize}
% \item Initialization
%     $\forall i, OPT(0,i) = 0$.

% \item While calculating each $OPT(t,i)$, our algorithm is faced with two possible choices.
%     \begin{enumerate}
%     \item Keep waiting at node $i$ for a passenger.
%     \item Traverse to some other node $j$ in search of a passenger.
%     \end{enumerate}
% \item It chooses whichever choice maximises the expected earnings in the remaining time units $t$.
% \begin{eqnarray*}
% OPT(t,i) = \max
%     \begin{cases}
%     \sum\limits_{j\neq i}m_{ij}^{(t')}\bigg[\Big(s_{i}^{(t')} p_{ij} - c_{ij}\Big) + OPT(t'-\tau_{ij},j)\bigg], & \text{if } \mathbf{E}[\tau_i|M] \leq t',\\ \\
%     \max\limits_{j}\Big[\Big(OPT(t-\tau_{ij},j) - c_{ij}\Big)\times H(t - \tau_{ij})\Big], & \text{otherwise}
%     \end{cases}
% \end{eqnarray*}
% where $t' = t - \frac{1}{\lambda_i^{(t)}}$, denoting the expected time units left after waiting for a passenger at node $i$.

% \item While solving the above dynamic program, we maintain an output vector corresponding to a series of driver choices between waiting for a passenger at his current node $i$ or taking an empty ride to some other node $j$.
% \end{itemize}

% % \textbf{Some issues:}
% % \begin{enumerate}
% % \item What happens when the condition $\frac{1}{\lambda_i^{(t)}} + \mathbf{E}[\tau_i|M] \leq t$ is not satisfied, while at the same time $\forall j, H(t-\tau_{ij})=0$ i.e., there is no node $j$ such that $t-\tau_{ij}>0$? Should the driver stay at node $i$ (first condition is probabilistic in nature) or should he simply call it a day? \\
% % \item If the condition $\frac{1}{\lambda_i^{(t)}} + \mathbf{E}[\tau_i|M] \leq t$ is satisfied, to calculate the quantity $OPT(t-\tau_{ij},j)$, we will need future values of transition matrix $M^{(t -\tau_{ij})}$ and surge multiplier vector $S^{(t - \tau_{ij})}$. Either, we will have to assume that a driver has complete knowledge of how the system will behave during the day, right at the beginning of the day or make some assumption e.g., transition matrix and surge multiplier vector remain fixed at the initial values throughout the day, $\forall t, M^{(t)} = M^{(T)}, S^{(t)} = S^{(T)}$. \\
% % \end{enumerate}

% \section{Uber Driver with flexible work schedule strategy}

% In this section, we formulate a dynamic program to find optimal work schedule for an Uber driver with a fixed budget of $B$ units of service time. We asssume that the driver uses up his budget before the end of maximum $T$ units of time. For example, a driver who works for a maximum of 40 hours per week shall have $B=40$ and $T=168$ hours. Each entry $OPT(t, i, b)$ of the matrix $OPT$ represents the maximum expected earnings with $b$ time units of budget and $t$ time units of week left. The maximum expected revenue of the driver at the beginning of the week is thus $OPT(T, v_0, B)$.
% \begin{itemize}
% \item Initialization
%         $\forall it, OPT(t, i, 0)=0$.

% \item While calculating each $OPT(t, i, b)$, our algorithm is faced with distinct possibilities depending on $i$. \\ \\
% If $i=v_0$ (driver is at home node),
%     \begin{enumerate}
%     \item Stay out of system, waiting for a better time to enter the system.
%     \item Enter the system, wait for a passenger at node $v_0$.
%     \end{enumerate}
% If $i \neq v_0$ (driver is not at home node),
%     \begin{enumerate}
%     \item Stay in the system, wait for a passenger at node $i$.
%     \item Exit the system, drive back to home node $v_0$.
%     \end{enumerate}

% \item The driver chooses whichever choice maximises the expected earnings in remaining $b$ units of his budget. \\
%         \begin{eqnarray*}
%         \text{if }i=v_0, \\
%         OPT(t, i, b) &=& \max
%         \begin{cases}
%         OPT(t-1, i, b)\\ \\
%         \sum\limits_{j\neq i}m_{ij}^{(t')}\bigg[\Big(s_{i}^{(t')} p_{ij} - c_{ij}\Big) + OPT(t'-\tau_{ij},j, b'-\tau_{ij})\bigg] \\
%         \end{cases} \\ \\ \\
%         \text{if }i\neq v_0, \\
%         OPT(t, i, b) &=& \max
%         \begin{cases}
%         \sum\limits_{j\neq i}m_{ij}^{(t')}\bigg[\Big(s_{i}^{(t')} p_{ij} - c_{ij}\Big) + OPT(t'-\tau_{ij},j, b'-\tau_{ij})\bigg] \\ \\
%         \Big(OPT(t-\tau_{iv_0}, v_0, b - \tau_{iv_0}) - c_{iv_0}\Big) \times H(b - \tau_{iv_0}) \\ \\
%         OPT(t-1, i, b)
%         \end{cases} \\ \\ \\
%         \end{eqnarray*}
% where $b' = b - \frac{1}{\lambda_i^{(t)}}$, denoting the expected budget time units left after waiting for a passenger at node $i$.

% \item While solving the above dynamic program, we maintain an output vector corresponding to a series of driver choices between waiting at home node, waiting for a passenger at his current node or taking an empty ride back home.
% \end{itemize}

% \section{Uber Driver with both re-location and flexible work schedule strategy}

% Here, we formulate a dynamic program to find optimal work schedule and re-location strategy for an Uber driver with fixed budget of $B$ units of service time. Like the previous section, we assume that the driver uses up his budget before the end of maximum $T$ units of time. We maintain a matrix $OPT$ where each entry, $OPT(t, i, b)$ represents the maximum expected earnings with $b$ time units of budget and $t$ time units of the week left. Similar to previous section, the maximum expected revenue of the driver at the beginning of the week is thus $OPT(T, v_0, B)$.
% \begin{itemize}
% \item Initialization
%         $\forall it, OPT(t, i, 0)=0$.
% \item While calculating each $OPT(t, i, b)$, our algorithm is faced with distinct possibilities depending on $i$. \\ \\
% If $i=v_0$ (driver is at home node),
%     \begin{enumerate}
%     \item Stay out of system, waiting for a better time to enter the system.
%     \item Enter the system, wait for a passenger at node $v_0$.
%     \item Enter the system, traverse to some other node $j$ in search of a passenger.
%     \end{enumerate}
% If $i \neq v_0$ (driver is not at home node),
%     \begin{enumerate}
%     \item Stay in the system, wait for a passenger at node $i$.
%     \item Stay in the system, traverse to some other node $j$ in search of a passenger.
%     \item Exit the system, drive back to home node $v_0$.
%     \end{enumerate}

% \item The driver chooses whichever choice maximises the expected earnings in remaining $b$ units of his budget. \\
% \newpage
%         \begin{eqnarray*}
%         \text{if }i=v_0, \\
%         OPT(t, i, b) &=& \max
%         \begin{cases}
%         OPT(t-1, i, b)\\ \\
%         \sum\limits_{j\neq i}m_{ij}^{(t')}\bigg[\Big(s_{i}^{(t')} p_{ij} - c_{ij}\Big) + OPT(t'-\tau_{ij},j, b'-\tau_{ij})\bigg] \\ \\
%         \max\limits_{j\neq i}\Big[\Big(OPT(t-\tau_{ij},j, b - \tau_{ij}) - c_{ij}\Big)\times H(b - \tau_{ij})\Big]
%         \end{cases} \\ \\ \\
%         \text{if }i\neq v_0, \\
%         OPT(t, i, b) &=& \max
%         \begin{cases}
%         \sum\limits_{j\neq i}m_{ij}^{(t')}\bigg[\Big(s_{i}^{(t')} p_{ij} - c_{ij}\Big) + OPT(t-\tau_{ij},j, b'-\tau_{ij})\bigg] \\ \\
%         \max\limits_{j}\Big[\Big(OPT(t-\tau_{ij},j, b - \tau_{ij}) - c_{ij}\Big)\times H(b - \tau_{ij})\Big] \\ \\
%         OPT(t-1, i, b)
%         \end{cases} \\ \\ \\
%         \end{eqnarray*}

% \item While solving the above dynamic program, we maintain an output vector corresponding to a series of driver choices between waiting at home node, waiting for a passenger at his current node , taking an empty ride to some other node in search of a passenger or taking an empty ride back home.
% \end{itemize}

% \section {Driver waiting time}

% Here, we formulate the driver waiting time period. Let $\lambda_{i}^{(t)}$ be the Poisson passenger arrival rate and $\mu_{i}^{(t)}$ be the Poisson driver arrival rate at node $i$ at time $t$. Further, let $N_{i,p}^{(t)}$ and $N_{i,d}^{(t)}$ denote the number of passenger arrivals and driver arrivals at node $i$ in a time slice centered around time $t$. Assuming that passenger arrival and driver arrival are independent Poisson processes {(?)}, the probability mass function for the random variable $K = N_{i,p}^{(t)} - N_{i,d}^{(t)}$ follows a Skellam distribution given by,

% \begin{eqnarray*}
% p(k, \lambda_{i}^{(t)}, \mu_{i}^{(t)}) = \mathrm{Pr}[K=k] = e^{-(\lambda_{i}^{(t)}+\mu_{i}^{(t)})} \bigg(\frac{\lambda_{i}^{(t)}}{\mu_{i}^{(t)}}\bigg)I_{k}\Big(2\sqrt{\lambda_{i}^{(t)}\mu_{i}^{(t)}}\Big)
% \end{eqnarray*}
% where $I_k(z)$ is the modified Bessel function of the first kind. We can depict the values taken by the random variable $K$ as states of a Markov Chain.

% \begin{center}
% \begin{tikzpicture}[->, >=stealth', auto, semithick, node distance=2.5cm]
% \tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=1]
% \node[state]    (-inf)               {$...$};
% \node[state]    (-2)[right of=-inf]   {$-2$};
% \node[state]    (-1)[right of=-2]   {$-1$};
% \node[state]    (0)[right of=-1]   {$0$};
% \node[state]    (+1)[right of=0]   {$1$};
% \node[state]    (+2)[right of=+1]   {$2$};
% \node[state]    (+inf) [right of=+2] {$...$};
% \path
% (-inf)  edge[bend left]         node{$\lambda_{i}^{(t)}$}   (-2)
% (-2)    edge[bend left]         node{$\lambda_{i}^{(t)}$}   (-1)
%         edge[bend left,below]   node{$\mu_{i}^{(t)}$}       (-inf)
% (-1)    edge[bend left]         node{$\lambda_{i}^{(t)}$}   (0)
%         edge[bend left,below]   node{$\mu_{i}^{(t)}$}       (-2)
% (0)     edge[bend left]         node{$\lambda_{i}^{(t)}$}   (+1)
%         edge[bend left,below]   node{$\mu_{i}^{(t)}$}       (-1)
% (+1)    edge[bend left]         node{$\lambda_{i}^{(t)}$}   (+2)
%         edge[bend left,below]   node{$\mu_{i}^{(t)}$}       (0)
% (+2)     edge[bend left]        node{$\lambda_{i}^{(t)}$}   (+inf)
%         edge[bend left,below]   node{$\mu_{i}^{(t)}$}       (+1)
% (+inf)  edge[bend left,below]   node{$\mu_{i}^{(t)}$}       (+2);
% \end{tikzpicture}
% \end{center}

% We can use this to further model the probability of a driver waiting for $m$ time units before getting a ride. The probability of $k$ passengers arriving in an time interval is given by $\frac{\big(\lambda_{i}^{(t)}\big)^k e^{-\lambda_{i}^{(t)}}}{k!}$. Let us assume that when the above Markov Chain is in a positive state, a driver immediately gets a passenger ride, without any delay since there are excess number of passengers waiting to hail a ride compared to the drivers. However, when the Markov Chain is in a non-positive state, the driver has to wait for the passengers to arrive. In the worst case, if the Markov Chain is in state $k$, the driver has to wait for $(|k|+1)$ number of passenger arrival events. Let $\text{Pr}[\text{waiting time}=m]$ denote the probability that the driver gets the passenger in time interval $m$ and $\text{Pr}[\text{pax ride}=m_{k}]$ denote the probability of getting a passenger in $m$-th time unit if the Markov Chain is in state $k$.

% \begin{eqnarray*}
% \text{Pr}[\text{waiting time}=m] &=& \sum\limits_{-\infty}^{0} \text{Pr}[K=k] \times \text{Pr}[\text{pax ride}=m_{k}] \\
% \text{Pr}[\text{pax ride}=m_{k}] &=& \sum\limits_{n=0}^{|k|} \text{Pr}[\text{n passengers in m-1 time units}] \times \text{Pr}[(|k|+1-n)\text{ passengers in m}^{th}\text{ time unit}] \\
% &=&\sum\limits_{n=0}^{|k|}\frac{\big((m-1)\lambda_{i}^{(t)}\big)^{n}e^{-(m-1)\lambda_{i}^{(t)}}}{n!} \times \frac{\big(\lambda_{i}^{(t)}\big)^{(|k|+1-n)}e^{-\lambda_{i}^{(t)}}}{(|k|+1-n)!}
% \end{eqnarray*}


\documentclass[12pt]{article}
\author{David Alves}

\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{dirtytalk}
\usepackage[a4paper, total={6.5in, 8.75in}]{geometry}
\usepackage{forest}
\usepackage{listings}
\usepackage{relsize}
\usepackage{skak}
\usepackage{tikz}
\usepackage{titling}
\usepackage{wrapfig}
\usepackage{xcolor}

\usetikzlibrary{decorations.pathreplacing}
\usetikzlibrary{patterns}

\def\multichoose#1#2{\ensuremath{\left(\kern-.3em\left(\genfrac{}{}{0pt}{}{#1}{#2}\right)\kern-.3em\right)}}

\newcommand{\ProblemStatement}[1]{
\subsection*{Problem Statement}
#1
\subsection*{Solution}
}

% If uncommented, next line hides problem statements
%\renewcommand{\ProblemStatement}[1]{}

\title{Math 142 Problem Set 9}
\author{David Alves}
\date{2016-10-27}

\begin{document}
\pagenumbering{gobble}

\begin{center}
\large \thetitle \\
\theauthor \\
\thedate
\end{center}

\subsection*{Sources}

    \begin{itemize}
    \item http://tex.stackexchange.com and https://www.sharelatex.com for help with \LaTeX
    \end{itemize}

\section{Five Triangles (Optional)}
\ProblemStatement{
Let G be a simple graph with 10 vertices and 26 edges. Show that G has at least 5 triangles.
}

I didn't have time to figure this one out before the due date. Will add a solution later.

\section{Generating with Powers of Two}
\ProblemStatement{
Show, \emph{with generating functions}, that there is exactly one way to represent each non-negative integer as a sum of \emph{distinct} powers of 2. (i.e. there is a unique binary expansion of each integer)
}

\begin{align*}
    F(x)&=\prod_{i=0}^{\infty}(1+x^{2^i})\\
    &=(1 + x^{2^0})(1 + x^{2^1})(1 + x^{2^2})(1+x^{2^3})(1+x^{2^4})\ldots \\
    &=(1 + x^1)(1 + x^2)(1 + x^4)(1+x^8)(1+x^{16})\ldots \\
\end{align*}

We define $F(x)$ to be a generating function which represents the choice of using or not using each power of two, for all powers of two. For each power of two, we use a parenthesized term $(1 + x^{2^i})$ which represents the choice between using that power of two  or not using it. The generating function is a product of these parenthesized terms for all powers of two, since we must make such a decision for all powers of two.

In order to show that there is exactly one way to represent each non-negative integer as a sum of distinct powers of 2, we must show that the product above is equal to $1 + x^1 + x^2 + x^3 +\ldots$

\begin{proof}
    We prove this by induction. Let $p(n)$ be the following statement:
\[
    \prod_{i=0}^{n}(1+x^{2^i}) = 1 + x^1 + x^2 + x^3 +\ldots + x^{2^{n+1}-1}
\]
For $n=0$, we have $1+x = 1+x$, so $p(0)$ is true.
Next we must show that for $n \ge 0$, if $p(n)$ is true, then $p(n+1)$ must be true. If we multiply both sides of $p(n)$ by $(1 + x^{2^{n+1}})$ we get:

\begin{align*}
    \left(1 + x^{2^{n+1}}\right)&\left(\prod_{i=0}^{n}(1+x^{2^i})\right) = \left(1 + x^1 + x^2 +\ldots + x^{2^{n+1}-1}\right)\left(1 + x^{2^{n+1}}\right)\\
    \prod_{i=0}^{n+1}(1+x^{2^i}) &= \left(1 + x^1 + x^2 + x^3 +\ldots + x^{2^{n+1}-1}\right)\left(1 + x^{2^{n+1}}\right)\\
\prod_{i=0}^{n+1}(1+x^{2^i}) &= \left(1 + x^1 + x^2 + x^3 + \ldots + x^{2^{n+1}-1}\right)+\left(x^{2^{n+1}} + x^{2^{n+1}+1} + \ldots + x^{2^{n+1}+2^{n+1}-1}\right)\\
\prod_{i=0}^{n+1}(1+x^{2^i}) &= \left(1 + x^1 + x^2 + x^3 + \ldots + x^{2^{n+2}-1}\right)\\
\end{align*}

The final equation is $p(n+1)$. Thus if $p(n)$ is true then $p(n+1)$ must also be true, which completes the inductive proof. Thus we have shown that there is exactly one way to represent each non-negative integer as a sum of distinct powers of two.

\end{proof}

\section{Counting Subset Lists}
\ProblemStatement{
Fix positive integers $n$ and $k$, and let $S = [n]$. Find the number of $k$-lists $(T_1, T_2, \ldots, T_k)$ of subsets $T_i$ of $S$ such that for every $i < j, T_i \subset T_j$.
}

There are $(k+1)^n$ such $k$-lists.

\begin{proof}
Consider a list $T$. Each element $e \in S$ is either not present in $T$ or there is some element of $T$, $T_x$, which is the first member of $T$ to contain $e$. Since we know that for every $i < j, T_i \subset T_j$, every element in $T$ after $T_x$ must contain $e$, and by definition every element before $T_x$ does not contain $e$. Therefore whether $e$ is in or not in each element of $T$ is completely determined by the first element which contains $e$. Thus there are $k+1$ possibilities for each element of $S$: It first appears in element $T_x$ (with $k$ possible values of $x$), or it does not appear anywhere in $T$. This gives $(k+1)^n$ possible lists by the product principle.
\end{proof}

\section{Counting Item Lists}
\ProblemStatement{
We have $n$ kinds of objects, and we want to count the number of ways to select a $k$-list/tuple of the objects. Find the number of ways to do this if we have:
\begin{enumerate}
\item Just one of each kind
\item Infinite objects of each kind
\item Just $l$ objects of each kind
\end{enumerate}
You're allowed to just give a generating function, too.
}

\subsubsection*{One of Each Type}
\begin{align*}
    n < k &: 0\\
    n \ge k &: \frac{n!}{(n-k)!}
\end{align*}

If $n < k$, there are 0 ways to select a $k$-list since there aren't enough objects. If $n \ge k$, there are $n$ choices for the first item, $n-1$ choices for the second (since we can't use the same type again), $n-2$ choices for the third, etc. This gives $\frac{n!}{(n-k)!}$ possible $k$-lists.

\subsubsection*{Infinite of Each Type}
There are $n$ choices for each item, giving $n^k$ $k$-lists by the product principle.

\subsubsection*{Limited Number of Each Type}

\[
  [x^k]\left(\left(1 + \frac{x^1}{1!} + \frac{x^2}{2!} + \frac{x^3}{3!} + \ldots + \frac{x^l}{l!}\right)^nk!\right)
\]

We can express the solution in the form of a generating function. For each kind of item, there can be between 0 and $l$ of that kind present in the list. The choice of how many of a single kind to use can be represented as the expression $(1 + x + x^2 + \ldots + x^l)$. Multiplying together $n$ such terms (one per kind) gives a polynomial where the coefficient of $x^y$ is the number of ways you can make a set of size $y$ using $n$ kinds of items with up to $l$ of each item. If every item in this set were distinguishable, there would be $k!$ orderings of the $k$ items into a list. However, we also need to use the quotient principle to account for multiple items of a single kind, which are indistinguishable from each other. We do this by dividing the $x^z$ term of each kind by $z!$, since that term represents using $z$ of the same kind.
\section{Binary Codes}
\ProblemStatement{
Let a \emph{codeword} of length $n \ge 1$ be a list of $n$ binary digits (0's and 1's). Call a codeword \emph{even} if it has an even number of 1's. For example, 0110110 is even but 0111 is not. For each $n$, find the number of codewords of length $n$.
}

There are $2^{n-1}$ even codewords for $n \ge 1$.
\begin{proof}
We prove this by induction. Let $E_n$ be the set of even codewords with length $n$ and $O_n$ be the set of non-even codewords of length $n$. Let $p(n)$ be the statement that $|E_n| = |O_n| = 2^{n-1}$.  For $n = 1$, there are two codewords, \say{0} and \say{1}. Of these two, only \say{0} is even. Thus $|E_1| = |O_1| = 2^0$, so $p(1)$ is true.

Next we show that if $p(n)$ is true, $p(n+1)$ must also be true for $n \ge 2$. Each member of $E_{n+1}$ either ends in a 0 or a 1. If it ends in a 0, then that codeword with the final 0 removed is a member of $E_n$. If it ends in a 1, then the codeword with the final 1 removed is a member of $O_n$. All members of $E_n$ will be in $E_{n+1}$ when you append a 0, and all members of $O_n$ will be in $E_{n+1}$ when you append a 1. Thus $|E_{n+1}| = |E_n| + |O_n|$. Similarly, each member of $O_{n+1}$ either ends in a 0 or a 1. If it ends in a 0, then that codeword with the final 0 removed is a member of $O_n$. If it ends in a 1, then the codeword with the final 1 removed is a member of $E_n$. All members of $O_n$ will be in $O_{n+1}$ when you append a 0, and all members of $E_n$ will be in $O_{n+1}$ when you append a 1. Thus $|O_{n+1}| = |O_n| + |E_n|$. Since we know  $p(n): |E_n| = |O_n| = 2^{n-1}$, we have shown that $|E_{n+1}| = |O_{n+1}| = 2^{n}$, which completes the inductive proof.
\end{proof}

\section{Time Spent \& Thoughts}

Much harder than our typical problem set. I was stuck on the five triangles problem for quite a while, and problem 4 part C was pretty tricky. I enjoyed the problems quite a bit but it was very time-consuming to try and finish all of them, so by the end I was pretty tired of it. I spent about 12 hours on the problem set.

\end{document}


% Part: first-order-logic
% Chapter: tableaux
% Section: propositional-rules

\documentclass[../../../include/open-logic-section]{subfiles}

\begin{document}

\iftag{FOL}
      {\olfileid{fol}{tab}{prl}}
      {\olfileid{pl}{tab}{prl}}

\olsection{Propositional Rules}

\subsection{Rules for $\lnot$}

\begin{defish}
\AxiomC{\sFmla{\True}{\lnot !A}}
\RightLabel{\TRule{\True}{\lnot}}
\UnaryInfC{\sFmla{\False}{!A}}
\DisplayProof
\hfill
\AxiomC{\sFmla{\False}{\lnot !A}}
\RightLabel{\TRule{\False}{\lnot}}
\UnaryInfC{\sFmla{\True}{!A}}
\DisplayProof
\end{defish}

\subsection{Rules for $\land$}

\begin{defish}\noindent
\AxiomC{\sFmla{\True}{!A \land !B}}
\RightLabel{\TRule{\True}{\land}}
\UnaryInfC{\sFmla{\True}{!A}}
\noLine
\UnaryInfC{\sFmla{\True}{!B}}
\DisplayProof
\hfill
\AxiomC{\sFmla{\False}{!A \land !B}}
\RightLabel{\TRule{\False}{\land}}
\UnaryInfC{$\sFmla{\False}{!A} \quad \mid \quad \sFmla{\False}{!B}$}
\DisplayProof
\end{defish}

\subsection{Rules for $\lor$}

\begin{defish}
\AxiomC{\sFmla{\True}{!A \lor !B}}
\RightLabel{\TRule{\True}{\lor}}
\UnaryInfC{$\sFmla{\True}{!A} \quad \mid \quad \sFmla{\True}{!B}$}
\DisplayProof
\hfill
\AxiomC{\sFmla{\False}{!A \lor !B}}
\RightLabel{\TRule{\False}{\lor}}
\UnaryInfC{\sFmla{\False}{!A}}
\noLine
\UnaryInfC{\sFmla{\False}{!B}}
\DisplayProof
\end{defish}

\subsection{Rules for $\lif$}

\begin{defish}
\AxiomC{\sFmla{\True}{!A \lif !B}}
\RightLabel{\TRule{\True}{\lor}}
\UnaryInfC{$\sFmla{\False}{!A} \quad \mid \quad \sFmla{\True}{!B}$}
\DisplayProof
\hfill
\AxiomC{\sFmla{\False}{!A \lif !B}}
\RightLabel{\TRule{\False}{\lor}}
\UnaryInfC{\sFmla{\True}{!A}}
\noLine
\UnaryInfC{\sFmla{\False}{!B}}
\DisplayProof
\end{defish}

\subsection{The Cut Rule}

\begin{defish}
\AxiomC{}
\RightLabel{\Cut}
\UnaryInfC{$\sFmla{\True}{!A} \quad \mid \quad \sFmla{\False}{!A}$}
\DisplayProof
\end{defish}

The \Cut{} rule is not applied ``to'' a previous !!{signed formula}; rather,
it allows every branch in !!a{tableau} to be split in two, one branch
containing $\sFmla{\True}{!A}$, the other~$\sFmla{\False}{!A}$. It is
not necessary---any set of !!{signed formula}s with a closed
!!{tableau} has one not using \Cut---but it allows us to combine
!!{tableau}s in a convenient way.

\end{document}


\documentclass[../main.tex]{subfiles}
\begin{document}

\subsection{One-to-All}
\label{q:onetoall}
\begin{question}
Given are $P$ processors. Processor $p(0)$ sends an element to all processors (one-to-all).
\begin{enumerate}
	\item Given the BSP program.
	\item What is the $h$-relation.
	\item How can this be improved, given that $l << g$.
\end{enumerate}
\end{question}
\begin{solution} The solution to the subquestions are given below.
\begin{enumerate}
	\item The BSP program for one-to-all communication is given below.
\begin{lstlisting}
void oneToAll(){
	bsp_begin(P);
	bsp_push_reg(element);
	for(pid=1; pid<P; pid++){
		bsp_put(pid, data, element, 0, sizeof(data));
	}
	bsp_sync();
	bsp_end();
}
\end{lstlisting}
	\item The $h$-relation is given by $h = max\{h_s,h_r\}$. In the previous case, $p(0)$ sends a single element to
	$(p-1)$ processors.
	\begin{equation}
		h = (p - 1)
	\end{equation}
	The BPS cost is given by $T_\text{comm}(h) = hg + l$, making the cost of this broadcast $(p-1)g + l$.
	\item If the cost of $l$ is significantly less than $g$, splitting the broadcast over multiple communication
	supersteps can reduce cost. Am example is a multi-level broadcast where every processor sends the element to two
	others. In $\log_2(P)$ supersteps, every processor has received the element.
	\begin{equation}
		T_\text{comm} = \lceil \log_2(P) \rceil (2g + l)
	\end{equation}
\end{enumerate}

\end{solution}

\subsection{LU Factorization}
\label{q:lu}
\begin{question}
LU factorization with 2-phase broadcast.
\begin{enumerate}
	\item Give the total sequential cost and its order.
	\item Give the total BSP cost and its order when the data is distributed over $\sqrt{P} \times \sqrt{P}$
	processors.
	\item Given the order of the total cost when distributed over $1 \times P$ processors.
	\item How is the parallel overhead influenced by the different distribution when the problem size grows.
\end{enumerate}
\end{question}
\begin{solution} The solution to the subquestions are given below.
\begin{enumerate}
	\item The number of flops in the LU decomposition with row permutations is given in \autoref{eq:tseqlu}.
	\begin{equation}\label{eq:tseqlu}
		\begin{array}{ll}
		T_{seq} & = \displaystyle\sum_{k=0}^{n-1} (2(n - k - 1)^2 + n - k -1) \\
				& = \displaystyle\sum_{k=0}^{n-1} (2k^2 + k) \\
				& = \dfrac{(n-1)n(2n-1)}{3} + \dfrac{(n-1)n)}{2} \\
				& = \dfrac{2n^3}{3} - \dfrac{n^2}{2} - \dfrac{n}{6} \\
		\end{array}
	\end{equation}
	The sequential LU-decomposition algorithm is $\mathcal{O}(n^3)$.
	\item The number of flops for the parallel LU-decomposition with two-phase broadcast and an \emph{optimal} 2D distribution ($N = \sqrt{P}, M = \sqrt{P}$) is given in \autoref{eq:tparlu}.
	\begin{equation}\label{eq:tparlu}
			T_{LU}  = \dfrac{2n^3}{3p} + \dfrac{3n^2}{2\sqrt{p}} + \dfrac{3n^2g}{\sqrt{p}} + 8nl
	\end{equation}
\item The distribution of processors impacts superstep (1), (3), (6) and (7) as we can find in \cite[p.~70]{bisseling04}. Their new cost is given in \autoref{tbl:lucost}.
	\begin{table}[H]
	\centering
	\begin{tabular}{ll}
		\toprule
		Superstep & Cost  \\
		\midrule
		(1) & $l$ \\
		(3) & $(P-1)g+l$ \\
		(6) & $(C_{k+1} - \lceil C_{k_1}/P  \rceil )g + l$ \\
		(7) & $((P-1) \lceil C_{k+1}/P \rceil )g + l$\\
		\bottomrule
	\end{tabular}
	\caption{New superstep costs.}
	\label{tbl:lucost}
	\end{table}
	Superstep (1) can be removes since no data is sent, making its cost zero. The dominant cost is caused by the matrix update.
	\begin{equation}
		R_{k+1} + C_{k+1} = (n-k-1) \frac{M+N}{p} + 2
	\end{equation}
	For $M = N = \sqrt{P}$, the value is minimal. With the current distribution, this cost is $p (n-k-1) + 2$ or a factor $p$ larger than with the previous one.
\item
\end{enumerate}
\end{solution}

\subsection{Parallel Efficiency}
\label{q:efficiency}
\begin{question}
Given a multicore program for calculating the average value of a matrix $A$. What is the efficiency and how can we do
better?
\end{question}
\begin{solution} The \emph{Parallel Efficiency} is defined in \autoref{eq:pareff}.
	The efficiency correlates the increase in speedup with the number of processors or can also be seen as the fraction of the total computing power that is usefully employed \cite[p.~141]{bisseling04}.
	The efficiency of a given parallel algorithm can be increased by either decreasing the number of processors (Amdahl's Law) or increasing the problem size $n$.
\begin{equation}\label{eq:pareff}
	E_p(n) = \frac{T_\text{sec}(n)}{p \cdot T_p(n)} = \frac{S_p(n)}{p}
\end{equation}
\end{solution}

\subsection{BSP Quicksort}
\label{q:qsort1}
\begin{question}
Give a high-level description of a BSP quicksort algorithm. For each step, calculate the BSPC cost ($a + bg + cl$).
Calculate the total cost and give the overhead and efficiency. Discuss your results.
\end{question}
\begin{solution} The algorithm is described below.

\begin{enumerate}
	\item \textbf{Superstep 1:} Determine and broadcast the pivot. This translates to a regular broadcast, with BSP-cost $(p-1)g + l$. Optionally, this can be achieved with a recursion tree, resulting in a cost of $(g+l) \log (p)$.
	\item \textbf{Superstep 2:} Locally rearrange the array assigned to each process. It takes $(n/p)$ FLOPS to compare each value with the pivot. This costs $(n/p) + l$
	\item \textbf{Superstep 3:} Determine the locations in the globally rearranged array that the local elements will go to. This can be achieved by either using \emph{prefix sums} or \emph{partial sums}. Regardless, it translates to all-to-all communication which costs $(p-1)g + l$.
	\item \textbf{Superstep 4:} In this step the the global rearrangement is performed. This is again a case of all-to-all communication, costing $(p-1)g + l$.
\end{enumerate}

The total cost is then the sum of the cost of each superstep. These are repeated $\log(n)$ times.

\begin{equation}
\frac{n \cdot \log(n)}{p} + 3\log(n)(p-1)g + 4\log(n)l
\end{equation}

\end{solution}

\end{document}


\section*{Price Data Analysis}
%\subsection*{Interpretation}
%automate interpretation to a certain extent by learning about circumstances through online data.

\subsection*{Time Series Analysis \footnotesize\textit{Ching-Chia Wang}}
Food price data has a natural temporal relation between different data points. It is important in the analysis to extract significant temporal statistics out of data. We will focus on analyze stationarity, autocorrelation, trend, and seasonality of our price datasets in R.

\subsubsection*{Test of Stationarity of Price Series}
Stationarity of a series guarantees that the mean and variance of the data do not change over time. This is crucial for a meaningful analysis, since if the data is not stationary, we can not be sure that anything we derive from the present will be consistent in the future.

We have conducted the KPSS test for stationarity on some representative series from the price dataset. A test result of a sample series is presented in the table below. In general, the price series of foods in India have p-values less than 0.05, which indicates that we have to reject the null hypothesis that the series is stationary. On the other hand, the first differences of the price series have p-values larger than 0.05, which means that they are potentially stationary time series. Based on this test result, we will use the first difference of the series in later analysis.

\begin{table}[h]
\begin{tabular}{|l|l|l|}
\hline
\hspace{5pt} & Price Series & First Difference of Price Series \\ \hline
p-value of null hypothesis "Trend" & 0.01 & 0.1\\ \hline
\end{tabular}
\label{table:kpss}
\caption{Table of results of KPSS-test of stationarity on the sample series: Price of "Jyoti" Potato in West Bengal. A p-value less than 0.05 indicates a rejection of the null hypothesis that the series is stationary under the 95\% confidence level. \cite{kwiat}}
\end{table}

\subsubsection*{Autocorrelation of Price Series}
Autocorrelation(ACF) is another important trait in time series data. It suggests the degree of correlation between current value and the value with a period before. By plotting correlograms (autocorrelation plots) of our data, we will be able to identify if the fluctuation of prices may be due to white noise or other hidden structures.

The figure below is the correlogram of a sample series. The dotted line indicates the region of white noises. Any signal located inside that region is considered statistically insignificant. The ACF value at lag 0 is fixed at 1 because current value is 100\% correlated with itself. Observe that we have some short positive signals near lag 0, and there appears to be some annual patterns in price changes. The current change in price of the series seems to be positively correlated with those a few months before, and a few years before, up to 4 years.

\begin{figure}
    \centering
    \includegraphics[width=.7\textwidth]{./img/correlogram.png}
    \caption{Correlogram of the first difference of the sample series: Price of "Jyoti" Potato in West Bengal.}
\end{figure}

\subsubsection*{Seasonality}
Seasonality is reasonably expected in our agricultural related time series. We have conducted a standard seasonality decomposition on our price dataset. One sample result is shown in Figure 2. The price series in the first subplot is decomposed into 3 series: seasonal, trend, and remainder. Slight annual patterns are presented, but note that they are in small magnitude comparing to the price series. One important observation is that the remainder series is highly volatile and inconsistent over time.

\begin{figure}
    \centering
    \includegraphics[width=.7\textwidth]{./img/seasonal.png}
    \caption{Correlogram of the first difference of the sample series: Price of "Jyoti" Potato in West Bengal.}
\end{figure}

\section*{Prediction Models}
We decided to evaluate different prediction models on the processed price time series. Since the processing of the price data made available by the Indian government yielded very few usable time series we constrained ourselves to try the different prediction model on a set of \emph{daily} wholesale price series that had over 90\% support and could be sufficiently cleaned and interpolated.

\subsection*{Time Series Forecasting \footnotesize\textit{Ching-Chia Wang}}

\subsubsection*{Regression with ARMA error}
In our regression model, the target vector is the sum of linear combinations of multiple regressors and an ARMA (Auto-Regressive Moving Average) error shock. Our regression model is formulated as such:

$Y = \beta * X + \epsilon,   \epsilon: ARMA(p,q)$,

where Y is the target fitted variables (the price series), X contains the regressors, $\beta$ is a linear coefficient matrix, and $\epsilon$ is the error shock. The ARMA modeling of error shock consists of the auto-regressive part (lag variables) and the moving average part (effects from recent random shocks). The fitting of the ARMA model and the historical error is accomplished by maximum likelihood estimation. On the other hand, through OLS (Ordinary Least Squares) or GLS (General Least Squares) processes, we can obtain an optimal $\beta$ which minimized the mean square error between the predicted Y and the actual Y.

The prediction dataset consists of a food price series, an oil price series, an inflation index series, 2 weather series (temperature and rainfall), and some extra series generated from the previous 3 series with pre-defined time lag in quarters or years. By introducing some extended series, we try to capture the annual cyclic pattern observed in the time series analysis. The food price series is used as the target prediction variable, while the others are regressors.

We use the regression model to predict the price series in a rolling fashion using R. First, a window of some fixed size, such as 3 years, defines the range of the training data. After fitting the model, the algorithm will then forecast a fixed length of future days after the training window. Then, the end of the window moves to the end of the previous forecasted dates so that the process can predict for the next days. This process runs recursively, and our price prediction is in continuous days up to the end of the dataset.

Our experimental result is shown in the figure and table below. The 95\% or 99\% quantiles of prediction error in the table means that 95\% or 99\% of the prediction errors fall inside the interval of the quantiles. As forecasting days increase, the root mean square error and the quantiles rise too. The best result is using a moving window size of 3 years and running 1 week price forecasting as 95\% of the prediction errors are less than 1 rupee.

\begin{table}[h]
\begin{tabular}{|c|c|c|c|c|c|c|}
\hline
\hspace{5pt} Moving Training Window Size & 3 years & 3 years & 5 years & 5 years  \\ \hline
Rolling Forecast Step & 1 week & 1 month & 1 week & 1 month  \\ \hline
RMSE & 1.0023 & 1.3397 & 1.8486 & 3.0762 \\ \hline
95\% Quantiles of Prediction Error & (-0.61, 0.84) & (-1.63, 2.18) & (-0.67, 1.12) & (-2.24, 2.07) \\ \hline
99\% Quantiles of Prediction Error & (-1.39, 1.91) & (-2.68, 3.7) & (-2.61, 1.94) & (-4.06, 2.68)  \\ \hline
\end{tabular}
\label{table:time series prediction results}
\caption{Prediction result of regression with ARMA error shock with different training moving window size and forecasting steps.}
\end{table}

\begin{figure}[!ht]
    \centering
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/regression_prediction.png}
        \caption{Prediction result of the sample series: Price of "Jyoti" Potato in West Bengal.}
        \label{subfig:tmp1}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/histogram_of_errors.png}
        \caption{Histogram of prediction errors of the sample series: Price of "Jyoti" Potato in West Bengal.}
        \label{subfig:tmp2}
        \end{subfigure}

    \caption{Example of time series cleaning process}
    \label{fig:tm_pred}
\end{figure}

\subsection*{Neural networks - introduction \footnotesize\emph{Stefan Mihaila}}
Artificial neural networks are computational models with very varied
applications in pattern recognition. Their capability of learning impressively
complex patterns and finding deep meaning in data have caused such models
to gain widespread attention and become a very popular choice for a large
number of prediction tasks. For the purpose of price predictions, we have
mainly focused on two different types of neural networks: \emph{feed-forward
neural networks} and \emph{echo-state neural networks}.

\subsection*{Feed Forward Neural Networks - Multilayer Perceptrons \footnotesize\emph{Stefan Mihaila}}
A feed-forward neural network is generally described by the number of neurons
and the connections between them. Each connection is assigned a weight (real
number), and the value (the activation in neuro-slang) of a neuron is computed
as the weighted sum of its input neurons. This value is then passed through a
non-linear function that ``squeezes'' the real number into $[-1, 1]$. It is
this non-linear mapping that allows neural networks to learn non-linear
patterns from the data.

In the case of feed-forward neural networks, the neurons and the connections
form a directed acyclic graph. In other words, a neuron cannot have a
connection to itself (neither directly nor through any path). In this sense,
the information is only sent ``forward'' (in one direction). Such a network can
be arranged in layers, based on the existing connections. Therefore, a more
practical way of describing a FFNN model is by the number of layers and by
the connections between them (most often full connection between neurons is
used).

Two layers, namely the first and last, have special meaning: they are called
the \emph{input layer} and \emph{output layer}, respectively.
The other layers (the ones in the middle) are called \emph{hidden layers}.

The \emph{input layer} is used as the ``source'' (the place from where the data
is fed into the network), and the \emph{output layer} is viewed as the ``sink''
(the place where a prediction is made on the input data). If our data-set has 5
input columns and one target column (to be predicted) then we will place 5
neurons in the input layer, each being responsible for taking values in one of
the columns and one output neuron in the output layer.

The process of training a neural network is the process of assigning weights
(real values) to all the connections in such a way that when we are feeding
input from our training set into the network, we get an output very close to
the correct output. This closeness is measured in terms of an error function
(such as the mean squared error). In order to optimize the weights of such a
network, we have to find a minimum of this error function, process which is
achieved most commonly through \emph{gradient descent}. The process of adapting
the weights after every gradient descent step is called \emph{backpropagation}
(because we are going backwards from output to input).

Note that unlike echo-state neural networks, FFNN do not have an internal state
(i.e. do not have a memory). As such, they are most commonly used to train
non-sequential data (data for which the order of the data-points does not matter).
This is obviously not the case for a time series.  However, a FFNN can be
abused into learning sequential data by feeding a sliding window of input over
a given interval.

\begin{figure}
    \centering
    \includegraphics[width=.7\textwidth]{img/ffnn_prezi.png}
    \caption{A feed-forward neural network with two hidden layers}
\end{figure}

\subsection*{Feed Forward Neural Networks - models used \footnotesize\emph{Stefan Mihaila}}
During our project, I have thought of numerous FFNN models that would make
sense to be trained, but most of my ideas in the beginning were overly
optimistic: I assumed we can train a single large neural network to
simultaneously predict all prices for all products in all regions; feeding all
of those inputs at once into the network would have been very useful as it
would have allowed the training process to find correlations across different
regions and different products.

However, it turned out that this was impossible to do, as the available time
series are way too sparse and gapped to allow such models to be trained.
As such, I resorted to individually training a network for each of a few
wholesale daily time series which had good data (downloaded by Anton, selected,
preprocessed and interpolated by Ching-Chia).

As some time series are more complex than others, I had to try different
model hyperparameters for each time series (to have good generalization and
prevent over-fitting). The hyperparameters are the number
of hidden layers and the number of neurons in each hidden layer. I have mainly
worked with the following models of 2 and 3 hidden layers: $(10, 10)$,
$(10, 10, 5)$, $(15, 15)$, $(15, 20)$, $(20, 20)$. For all the models, I have
used fully connected layers (i.e. each pair of consecutive layers were fully
connected).

To maximize the prediction power of the trained network, I have used $40$
inputs to the network:
\begin{enumerate}
\item A sliding window of the normalized per-region monthly \emph{precipitation} amounts over the last 9 months (9 input neurons)
\item A sliding window of the normalized per-region monthly mean \emph{temperatures} over the last 9 months (9 input neurons)
\item Sliding window for last 9 months of normalized \emph{international oil prices} (9 input neurons)
\item Sliding window for last 4 months of the \emph{consumer price index} (4 input neurons)
\item Sliding window for the last 36 days of prices, in 4 day jumps (9 input neurons)
\end{enumerate}

\begin{figure}[H]
    \centering
        \begin{subfigure}[b]{.45\linewidth}
        \centering
        \includegraphics[width=\textwidth]{img/ffnn/1.pdf}
        \caption{Wholesale daily prediction for potato in Vadhvan (Gujarat)}
        \label{subfig:ffnn_pred_1}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{.45\linewidth}
        \centering
        \includegraphics[width=\textwidth]{img/ffnn/2.pdf}
        \label{subfig:ffnn_pred_2}
        \caption{Wholesale daily prediction for apple in Bareilly (Uttar Pradesh)}
        \end{subfigure}
    \caption{Four-day-ahead predictions with the FFNN on various time series}
\end{figure}

Because the prices are sometimes stable for a few days in a row, I am trying to
predict prices in 4 day jumps (i.e. given input at $t-36, t-32, t-28, \dots,
t$, the target is the price at $t+4$). I am then filling in the predictions for
$t+1$, $t+2$ and $t+3$ using linear interpolation between $t$ and $t+4$.

\begin{figure}[H]
        \begin{subfigure}[b]{.45\linewidth}
        \centering
        \includegraphics[width=\textwidth]{img/ffnn/3.pdf}
        \caption{Wholesale daily prediction for onion in Bareilly (Uttar Pradesh)}
        \label{subfig:ffnn_pred_3}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{.45\linewidth}
        \centering
        \includegraphics[width=\textwidth]{img/ffnn/4.pdf}
        \label{subfig:ffnn_pred_4}
        \caption{Wholesale daily prediction for rice in Bareilly (Uttar Pradesh)}
        \end{subfigure}
    \caption{Four-day-ahead predictions with the FFNN on various time series}
\end{figure}

Such a network should be able to predict 4 days in advance without receiving
any feedback. However, after 4 days, the network should receive feedback. I
believe this can be stretched for longer periods, but the limited time and the
long durations of training the models has not allowed me to test this
hypothesis.

\begin{figure}[H]
    \centering
        \begin{subfigure}[b]{.45\linewidth}
        \centering
        \includegraphics[width=\textwidth]{img/ffnn/5.pdf}
        \label{subfig:ffnn_pred_5}
        \caption{Wholesale daily prediction for potato in Burdwan (West Bengal)}
        \end{subfigure}
        \begin{subfigure}[b]{.45\linewidth}
        \centering
        \includegraphics[width=\textwidth]{img/ffnn/6.pdf}
        \label{subfig:ffnn_pred_6}
        \caption{Wholesale daily prediction for potato in Champadanga (West Bengal)}
        \end{subfigure}
    \caption{Four-day-ahead predictions with the FFNN on various time series}
\end{figure}

On the time series we have selected, the FFNN seems to be performing quite
well. The testing of the neural network was performed as follows: the first
$85\%$ of the time series was used for training the network, the last $15\%$ of
the time series was used for testing. The blue plot highlights the true series,
the green plot shows the network prediction for the train data (naturally very
good, as it has already seen the data) and the red plot shows the prediction
over test data (data it has never seen). The performance can be estimated by
comparing the red plot to the blue plot. Remember that the network still gets
feedback on how it's doing, but only every 4 days.

\begin{figure}[H]
    \centering
    \includegraphics[width=.5\textwidth]{img/ffnn/7.pdf}
    \caption{Wholesale daily prediction for rice in Champadanga (West Bengal)}
    \label{subfig:ffnn_pred_7}
\end{figure}

\subsection*{Recurrent Neural Networks (RNN) \footnotesize\emph{Fabian Brix}}
A recurrent neural network (RNN) is a neural network with feedback connections, enabling signals to be fed back from a layer $l$ in the network to a previous layer. As opposed to feedforward neural networks where a layer $l$ can only receive inputs from a previous layer, in an RNN the weight matrix for a layer $l$ can contain input weights from \emph{all} other neurons in the network. The simplest form of an RNN consists of an input, an output and one hidden layer as depicted in \ref{fig:simple_rnn}.
\begin{figure}
    \centering
    \includegraphics[width=.7\textwidth]{./img/simple_rnn.png}
    \caption{source: wikipedia}
    \label{fig:simple_rnn}
\end{figure}

\subsubsection*{General description of a discrete time RNN}
A discrete time RNN is a graph with $K$ input units $\vec{u}$, $N$ internal network units $\vec{x}$ and $L$ output units $\vec{y}$. The activation (per layer) vectors at point n in time are denoted by $\vec{u}(n) = (u_1(n),...,u_n(n))$, $\vec{x}(n) = (x_1(n),...,x_n(n))$, $\vec{y}(n) = (y_1(n),...,y_n(n))$. Edges between the units in these sets are represented by weights $\omega_{ij}\neq0$ which are gathered in adjacency matrices. There are four types of matrices:\par
\begin{itemize}
	\item $\vec{W}^{in}_{N\times K}$ contains inputs weights for an internal unit in each row respectively
	\item $\vec{W}_{N\times N}$ contains the internal weights. This matrix is usually sparse with densities $5\%-20\%$
	\item $\vec{W}^{out}_{L\times (K+N+L)}$ contains the weights for edges, which can stem from the input, the internal units and the outputs themselves, leading to the output units.
	\item $\vec{W}^{back}_{N\times L}$ contain weights for the edges that project back from the output units to the $N$ internal units
\end{itemize}

In a \emph{fully recurrent network} every unit receives input from all other units neurons and therefore input units can have direct impact on output units. Output units can further be interconnected.\par

\paragraph{Evaluation}
The calculation of the new state of the internal neurons in time-step $n+1$ is called evaluation.
\[
\vec{x}(n+1)=\vec{f}(\vec{W}^{in}\vec{u}(n+1)+\vec{W}\vec{x}(n)+\vec{W}^{back}\vec{y}(n))
\]
where $f=(f_1,...,f_N)$
\paragraph{Exploitation}
The output activations are then computed from the internal state of the network in the exploitation step.
\[
\vec{y}(n+1)=f^{out}(\vec{W}^{out}(\vec{u}(n+1),\vec{x}(n+1),\vec{y}(n)))
\]
where $f^{out}=(f^{out}_1,...,f^{out}_L)$ are the output activation functions and the matrix of output weights is multiplied by the concatenation of input, internal and previous output activation vectors.\par
RNNs can in theory approximate any dynamical system with chosen precision, however training them is very difficult in practice. In the following section we are going to describe our use of an RNN that exhibits exactly these properties yet is easy to train.

\subsection*{Echo State Networks}
Echo State Networks (ESN) are a type of discrete time RNNs for which training is straightforward with linear regression methods.  The temporal inputs to the network are transformed to a high-dimensional \emph{echo state}, described by the neurons of a sparsely connected \emph{random} hidden layer which is also called a reservoir. The output weights are the only weights in the network that can change and are trained in a way to match the desired output. ESNs and the related liquid state machines (LSMs) form the field of \emph{reservoir computing}. This part of the report is based on \cite{esn_rep}.

\begin{figure}
    \centering
    \includegraphics[width=.6\textwidth]{img/ESN_prezi.png}
    \caption{Network structure of an ESN with 1-dim. input and output}
\end{figure}

\subsubsection{Echo State Property}
The intuitive meaning of the \emph{echo state property} (ESP) is that the internal state is \textbf{uniquely} determined by the history of the input signal and the teacher forced output, given that the network has been running long enough. Teacher forcing essentially means that the output $\vec{y}(n-1)$ is forced to be equal to the next time series value $\vec{u}(n)$ and thus to the next input.
\begin{frm-def}
For every left infinite sequence $(\vec{u}(n),\vec{y}(n-1)),n=\dots,-2,-1,0$ and all state sequences $\vec{x}(n),\vec{x'}(n)$ which are generated according to
\begin{align*}
	\vec{x}(n+1)=\vec{f}(\vec{W}^{in}\vec{u}(n+1)+\vec{W}\vec{x}(n)+\vec{W}^{back}\vec{y}(n))\\
	\vec{x'}(n+1)=\vec{f}(\vec{W}^{in}\vec{u}(n+1)+\vec{W}\vec{x'}(n)+\vec{W}^{back}\vec{y}(n))
\end{align*}
it holds true that $\vec{x}(n)=\vec{x'}(n)$ for all $n \leq 0$.
\end{frm-def}
The echo state property is ensured through the matrix of internal weights $W$

\begin{frm-thm}
Define $\sigma_{max}$ as largest singular value of $\vec{W}$, $\lambda_{max}$ as largest absolute eigenvalue of $\vec{W}$.
\begin{enumerate}
\item If $\sigma_{max} < 1$ then the ESP holds for the network
\item If $\|\lambda_{max}\| > 1$ then the network has no echo states for any input/output interval which contains the zero input/output tuple (0,0)
\end{enumerate}
\end{frm-thm}
In practice it suffices to make sure the negation of the second point holds.

\subsubsection*{Training the ESN}
The state of the ESN is a function of the inputs it has been presented with. Each state describes an oscillator at a given time n and during training the relation between these oscillators and the output has to be learnt. Before training can be started the reservoir weights have to be normalized by the spectral radius of the weight matrix $W$. The hyperparameters are the reservoir size (dimensionality of W) and a rescale parameter of the weight matrix for slightly increasing/decreasing the spectral radius. This part of the report is baed on a comprehensive overvie of practical issues of ESNs \cite{esn_practical}.
%http://stackoverflow.com/questions/21940860/echo-state-network-learning-mackey-glass-function-but-how

\paragraph*{Initial state determination}
Prior to training the Echo State Network has to be run for an initial set of inputs discarding the results. A spectral radius close to unity implies slow forgetting of the starting state and therefore a substantial part of the training set has to be invested. We used an initiation sequence of 740 points which equates to two years of data.
%EchoStatesTechRep.pdf, p.32
%The network is run for a first set of inputs and the results are then discarded. If the spectral radius is close to unity, implying slow forgetting of the starting state, the initial set has to be a substantial part of the training dataset.
%"Likewise, when the trained network is used to predict a sequence, a long initial run of the sequence must be fed into the network before the actual prediction can start. This is a nuisance because such long initial transients are in principle unnecessary"
%"By contrast, a recurrent neural network such as our echo state network, but also such as the networks used in [5] need long initial runs to “tune in” to the to-be-predicted sequence. [5] tackle this problem by training a second, auxiliary “initiator” network."

\paragraph*{Batch learning with Ridge Regression}
The easiest way to train an Echo State Network is through minimizing the residual of an overdetermined linear set of equations. After the reservoir initialization phase, at init\_index of the time series data, the internal states are saved together with the network inputs at every time step as rows to a matrix $X$. This results in a linear set of equations $XW^{out}=Y$, where $Y \in \mathbb{R}^N$ is the vector of outputs starting at init\_index+1. Since the system is overdetermined with no unique solution for the weights $W^{out}$ that perfectly fits all equations one applies Tikhonov regularization instead of ordinary least squares. Consequently we minimize the residual of the SLE with an added regularization term:\\
\begin{equation}
    \|XW^{out}-Y\|^2 + \|\nu W^{out}\|^2
\end{equation}
The Tikhonov regularizer allows finding a stable solution that will not adversely affect the network output by improving the conditioning of the problem. The corresponding normal equations become:
\begin{equation}
    W^{out}=(X^TX+\nu I)^{-1} X^TY
\end{equation}

\paragraph*{Teacher forcing}
Teacher forcing is the simplest training method in which the respective value of the target time series $y_{target}(n)=u(n+1)$ is fed in at every consecutive time step with input weights $W^{in}$.

\paragraph*{Feedbacks}
During training with feedbacks the output $y(n)$ produced by the network is additionally fed back into the reservoir with feedback weights $W^{back}$. There further exists the possibility to decouple the network from the actual input during \emph{online} training which we will discuss later.

\paragraph*{Leaky integrator neurons}
\emph{Taken from Echo State Tech Rep. page 26/27}\\
% exploitation: \vec{f}(\vec{W}^{in}\vec{u}(n+1)+\vec{W}\vec{x}(n)+\vec{W}^{back}\vec{y}(n))
In order for the Echo State Network to be able to learn slowly and continuously changing dynamics and thereby to capture long-term phenomena in the price series we feed in, we need a way to introduce continuous dynamics. This is done via approximation of the differential equation of a continuous-time leaky integrator network
\begin{equation}\label{eq:leaky}
    \frac{d\vec{x}}{d\vec{t}} = C (-\alpha\vec{x} + \vec{f}(\vec{W}^{in}\vec{u}(n+1)+\vec{W}\vec{x}(n)+\vec{W}^{back}\vec{y}(n)))
\end{equation}
where C is a time constant and $\alpha$ the leaking decay rate. For the approximation we introduce a stepsize $\delta$:
\begin{equation}
    \vec{x}(n+1)=(1-\delta C \alpha) + \delta C (\vec{f}(\vec{W}^{in}\vec{u}(n+1)+\vec{W}\vec{x}(n)+\vec{W}^{back}\vec{y}(n)))
\end{equation}
In the implementation we used a simple weighted average of consecutive reservoir states $(1-\alpha)x(n-1)+\alpha x(n)$.

\begin{comment}
\begin{frm-thm}
Let a network be updated according to \eqref{eq:leaky}, with teacher-forced output.
\end{frm-thm}
\end{comment}

\paragraph*{Online learning}
In contrast to learning the weights once in batch-mode after the training run is completed one can also train the output weights in an online fashion after each consecutive presentation of an input. This method allows for a better adaption of the weights to local structures present in the data. There exist several algorithms for online training of the Echo State Networks and related reservoir networks. In the following we will shortly discuss two of the basic algorithms which are most often mentioned in academic literature.
\begin{itemize}
\item \textbf{Least Mean Squares (LMS):} LMS algorithms try to find the optimal weights (filtered coefficients) that minimize the mean squared error for an adaptive filter which is represented by the output weights $W^{out}$ in the case of the ESN. Essentially, LMS algorithms are stochastic gradient descent methods that perform updates according to the error present at time n that update the weights according to the a priori error and a predefined constant in every iteration.\\
\bigskip
\indent
\begin{algorithm}[H]
\textbf{Initialization:}\\
$W^{out}(0) = \vec{0}$\\
\textbf{Update steps:}\\
\begin{enumerate}
\item $e(n) = y_{teach}-W^{out}(n-1)x(n)$
\item $W^{out}(n)=W^{out}+\mu e(n)x(n)$
\end{enumerate}
% \caption{Algo}
\end{algorithm}
\item \textbf{Recursive Least Squares (RLS):} RLS algorithms have a faster convergence speed compared to LMS algorithms. RLS optimizes the discounted square a priori error $\sum_{k=1}^{n} \lambda^{n-k}((y_{teach}(k) - y(n))^2$ with forgetting factor $\lambda$ at every timestep n. All signals generated in the sequence $1 \leq k \leq n$ are taken into account using the inverse correlation matrix of inputs P. The inputs for the adaptive filter in the case of the ESN can be the concatenation of input, generated echo states and last output.\\
\bigskip
\indent
\begin{algorithm}[H]
\textbf{Initialization:}\\
$\lambda < 1$\\
$P_0 = \delta I$ where $\delta \propto \sigma(input)$\\
$W^{out}(0) = \vec{0}$\\
\textbf{Update steps:}\\
\begin{enumerate}
\item $\pi(n) = P(n-1)^T \cdot x(n)$
\item $\gamma(n) = \lambda + \pi(n)x(n)$
\item $k(n) = \pi(n)/\gamma(n)$
\item $e(n) = y_{teach}-W^{out}(n-1)x(n)$
\item $W^{out}(n)=W^{out}+k(n)e(n)$
\item $P(n) = [P(n-1)-k(n)\pi(n)s]/\lambda$
\end{enumerate}
% \caption{Algo}
\end{algorithm}
The standard RLS algorithm is unstable during training, diverges when the $P(n)$ loses positive definiteness. Choosing $\lambda$ very close to 1 was temporarily enough to alleviate the problem. The QR decomposition-based RLS (QR-RLS) algorithm can, however, resolve completely this instability. Instead of working with the inverse correlation matrix of the input signal, the QR-RLS algorithm performs QR decomposition directly on the correlation matrix of the input signal. Therefore, this algorithm guarantees the property of positive definiteness and is more numerically stable than the standard RLS algorithm.
%http://zone.ni.com/reference/en-XX/help/372357A-01/lvaftconcepts/aft\_rls\_algorithms/
\end{itemize}

\paragraph*{Parameter selection with Maximum Entropy Bootstrap (Meboot)}
In order to find the best parameters for generalization during training of the neural network models with we tried to create replicate time series of a selected price sequence dataset. The method employed to this end is called 'Maximum Entropy Bootstrap' (meboot) and was introduced by H.D. Vinod in 2006. [reference]. The reason for the use of this specific method is that, due to temporal dependence, time series cannot simply be randomly sampled into a new dataset. The meboot algorithm allows for construction of random replicates of the given time series showing the same statistical properties. We did, however, not manage to fully implement the algorithm from how it was described in the paper.

\subsubsection*{Demonstration}
In this section we demonstrate the predictive power of the simplest ESN configuration with a reservoir size of 1000 neurons. Ridge Regression is used to train on and predict a time series generated using the \href{http://www.scholarpedia.org/article/Mackey-Glass_equation}{Mackey-Glass differential equation} and noise.

%\begin{subfigure}[b]{.33\textwidth}
%\centering
%\includegraphics[width=\textwidth]{./img/plots/esn/mg/data1000.png}
%\caption{Sample of 1000 training points of Mackey Glass series}
%\label{subfig:mg_train}
%\end{subfigure}

\begin{figure}[!ht]
    \centering
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/mg/resact.png}
        \caption{Sample of ESN activations resulting from the training series}
        \label{subfig:mg_act}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/mg/pred1000_br.png}
        \caption{Near perfect prediction 1000 points Mackey Glass testset by ESN}
        \label{subfig:mg_pred}
        \end{subfigure}
    \caption{Results of simple ESN trained with Ridge Regression on a Mackey Glass time series consisting of 2000 points; parameters: spectral radius $1.25$, leaking rate $0.3$.}
    \label{fig:mackeyglass}
\end{figure}

\subsubsection*{Obtained results}
The mackey glass time series is generated by a function with added noise. Since the development of commodity prices doesn't follow a similarly easily describable pattern we did not expect to obtain comparable results. Once trained on several years of price series data the ESN is capable of predicting prices very accurately on a day-to-day basis without any need for retraining. However, due to the limited usefulness of day-to-day predictions, we would like to predict prices farther into the future with relative accuracy. So far we have only achieved predicting the general trend for 6 to 7 days in isolated cases where the dynamics of the training period favoured a prediction and we are not sure if greater accuracy is achievable with these models. %Our experiments with inputting additional data like temperature, precipitation, consumer price index (inflation) and twitter data show NO? significant improvements.

% NO BLANK SPACES
\begin{figure}[!ht]
    \centering
        \begin{subfigure}[b]{.45\linewidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/daily/gujarat_potato_online_7d_1d.png}
        \caption{Wholesale price of potato, state Gujarat; test RMSE in INR: 0.199}
        \label{subfig:res_7d_1}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{.45\linewidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/daily/uttar_pradesh_redonion_7d_1d.png}
        \caption{Wholesale price of red onion, state Uttar Pradesh; test RMSE in INR: 0.145}
        \label{subfig:res_7d_2}
        \end{subfigure}
    \caption{7 day day-to-day prediction for two different commodities in different states}
    \label{fig:res_7d}
\end{figure}

\begin{figure}[!ht]
    \centering
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/daily/gujarat_potato_online_30d_1d.png}
        \caption{Wholesale price of potato, state Gujarat; test RMSE in INR: 0.293}
        \label{subfig:res_30d_1}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/daily/uttar_pradesh_redonion_30d_1d.png}
        \caption{Wholesale price of red onion, state Uttar Pradesh; test RMSE in INR: 0.145}
        \label{subfig:res_30d_2}
        \end{subfigure}
    \caption{30 day day-to-day prediction for two different commodities in different states}
    \label{fig:res_30d}
\end{figure}

\begin{figure}[!ht]
    \centering
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/daily/gujarat_potato_online_60d_1d.png}
        \caption{Wholesale price of potato, state Gujarat; test RMSE in INR: 0.310}
        \label{subfig:res_60d_1}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/daily/uttar_pradesh_redonion_60d_1d.png}
        \caption{Wholesale price of red onion, state Uttar Pradesh; test RMSE in INR: 0.127}
        \label{subfig:res_60d_2}
        \end{subfigure}

    \caption{60 day day-to-day prediction}
    \label{fig:res_60d}
\end{figure}

Adjusted for inflation the prices mostly lose their upwards trend and should become easier for the network to predict. This is the case, but good predictions are again obtained more by chance than anything else by selecting subsets of a series to train on. Modeling the dynamics of the whole series seems to remain too complex for the network.

\begin{figure}[!ht]
    \centering
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/red_onion_pred_6d_batch.png}
        \caption{Wholesale price of red onion, accurate prediction produced for 5 days by training with a limited amount of data somewhere in the middle of the series}
        \label{subfig:res_ch6d}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{./img/plots/esn/red_onion_initN3000_30d.png}
        \caption{Wholesale price of red onion, trying to reproduce accurate prediction by training on the last two years, best result of various parameter choices}
        \label{subfig:res_ch30d}
        \end{subfigure}

    \caption{Results from training with ridge regression on a very limited amount of data maximum three years. The prediction can be gotten about right even for one month horizon. However, so far it adjusting the training phase and the spectral radius is more of a game of chance. The influence of the reservoir size seems to be neglectable having tried sizes of 500, 1000, 2000.}
    \label{fig:res_chance}
\end{figure}

In their current state, due to the very limited amounts of available data, the twitter indicators cannot prove their use. Significantly bolstered, however, they would offer a wide range of possibilities. Matching these indicators to available price data series could show up relationships between conversations and the actual price. These relationships could be used to track price changes in real-time on a regional level through the conversations on twitter and thereby prove an interesting alternative to the inadequate data monitoring of the Indian government. Improving predictions, if possible, would certainly require a lot more focused research in the area of recurrent neural networks for time series prediction. The ESN was implemented from scratch in scientific python.

\subsubsection*{Further ESN research}
Further research concerning the stability of online training of the ESN and its predictability power would encompass the implementation of a stable RLS algorithm such as the Inverse QR-RLS as well as the implementation of the Backpropagation and Decorrelation (BPDC). The latter does not require a specific set up of the network rendering concerns about the initialization run and the spectral radius of the weight matrix void. Using these algorithms a performance analysis of the network has to be done before we can test the influence of additional inputs. Maybe research into Linear System Theory can provide answers as to how to make the network responsive to its own inputs in the generative run. It would lastly be interesting  to compare the ESN to its "rival" the Long Short Term Memory Network trained with the Evolino method.

\subsection*{Data Mining}
\emph{Alexander Buesser} \\

In this section we give a brief introduction to association rule mining and detail the implementation and evaluation of the system. In order to run the code you will need to install the libraries Scipy and Orange.

\subsubsection{A brief introduction to Association rule mining}

The objective of association  mining is the elicitation of useful rules from which new knowledge can be derived. Association mining applications have been applied to many different domains including market basket analysis, risk analysis in commercial environments, clinical medicine and crime prevention. They are all areas in which the relationship between objects can provide useful knowledge.

Itemsets are identified by the use of two metrics support and confidence.

Support is a measure of the statistical significance of the rule. Rules with a very low support are more likely to occur by chance. With respect to the market basket analysis items that are seldom bought together by customers are not profitable to promote together. For this reason support is often used as a filter to eliminate uninteresting rules.

Confidence on the other hand is is a measure on how reliable the inference made by a rule is. For a given rule $A \implies B$, the higher the confidence, the more likely it is for the item set B to be present in the transactions that contain A. IN a sense confidence provides an estimate of the conditional probability for B given A.

It is worth noting that the inference made by an association rule does not necessarily imply causality. Instead the implication indicates a strong concurrence relationship between items in the antecedent and consequent of the rule.

So how do we go about implementing this in an algorithm? A rather naive approach would be to check if each itemset satisfies minimum support. However this is rather inefficient and unnecessary. We can make use of of the observation that every subset of a frequent item set also has to be a frequent item set. This works the other way as well. If a set is not frequent then its superset can not be frequent either. This observation allows us to prevent unnecessary computation. The Apriori Algorithm uses this downward closure to identify frequent item sets. Candidates that do not satisfy minimal support are pruned, which automatically reduces the algorithm's search space. Once frequent item sets have been generated the algorithm enters the second face, namely generating derivations for which the metric minimal confidence is used.

\subsubsection{Implementation}

The implementation is guided through three stage namely data crunching, classification and  mining where the later is a straight forward implementation of the mining library orange. \\

In order to run the Apriori Algorithm we needed to do some preprocessing since the data available was in an incompatible format. The data readily available to us was in the form
of Date Country City Category Commodity 1, Date Country City Category Commodity 2. In the statistics community this kind of format is referred to as the "long format". In order to run Apriori we need to convert the table to a "wide format" meaning that all commodities need to be related to one index. You can imagine a matrix where the y axis is described by city + date and the x axis is labeled with all the available commodities. To do the conversion we used Stata, a statistical software which is mostly used in the field of social science. Before reading the data into state we filtered the document for special characters ("", /) and replaced all unavailable price informations with "NA". For this purpose we used the filter.py script. Once we read in the data into state we removed duplicates and started the conversion. To reproduce the correct table format you can follow our implementation in the stata script.  The data is now in the correct table format what remains to be done is slicing the table according to a city and sorting the data in order of decreasing time stamps. These steps are performed in the dataFrame.py. We now continue to process the files by classifying continues variables into categories. We therefor filtered the maximum and minimum increase/decrease in price to establish a range of values. Depending on whether a price increase fall into the first, second or third of the price range we classified it as small, medium or big increase/decrease respectively. The categorical data can now be processed by the assoc.py file. The library orange provides an implementation of the a priori gen algorithm. We simply set the min support value and write the 10 rules with the highest support count to a file.

\subsubsection{Results}

What we were hoping to find were seasonal related price changes such as those we experience when shopping for fruits and vegetables at our local grocery stores. Rules such as $Tomatoes = big increase \implies potatoes = small decrease$ could serve

Our initial granularity was set to weekly data. From our meta analysis we observed that prices quite frequently stay unchanged over the period of several weeks. This resulted in an overwhelming amount of rules in the form of commodity $A = unchanged \implies B= unchanged$. This made it really hard to filter the set for insightful rules. We therefor decided to compare price changes over the period of 12 weeks. Although the majority of the rules still remained in the above form we managed to extract some relations related to price changes.

We conclude that the strongest correlations exist between commodities with unchanged prices. This observation underlines the nature of food commodities. Agricultural commodities are known to be less volatile then for example energy commodities. We further noticed that depending on the region we would have a totally different rule count. Capital cities tend to have a bigger rule base meaning that there exist a higher correlation between products. Two extreme example are Dassau, which resulted in no patterns at all and Shillong which produced over 260000 rules. In addition our experiments showed the finer the granularity of the data the higher the support and correlation between products. This is only natural given that prices can stay stable over the period of weeks or even months.

\subsubsection{Trading advice }

As concluded above our algorithm found many associations between products. In order to interpret most of the results specialized domain knowledge in trading with commodities is  necessary. Some more obvious rules we managed to interpret were the following:

$BreadLocal=big increase MilkCowBuffalo=unchanged increase \implies \\MaidaNA=unchanged increase$. If prices of bread inflate it is save to trade Maida, as its price will most certainly stay stable. \\$BiscuitGlucose=unchanged increase \implies BesanNA=small increase$, similarly means that stable pries in biscuits will imply a small increase in Besan.

\begin{comment}
\subsubsection*{Finding the right topology for the specific prediction problem}

\subsubsection*{Converge}
conceive network that converges fast to speed up training\par
"Known supervised training techniques for RNNS comprise Back Propagation Through Time (BPTT), Real Time Recurrent Learn- ing (RTRL) or Extended Kalman Filtering (EKF) all of which have some major drawbacks."
{\em Application of BPTT to RNNs requires stacking identical copies of the network thus unfolding the cyclic paths in the synaptic connections. Unlike back-propagation used in feed-forward nets, BPTT is not guaranteed to con- verge to a local error minimum, computational cost is O(TN2) per time step where N is the number of nodes, T the number of epochs. In contrast RTRL needs O((N + L)4) (L denotes number of output units), which makes this algorithm only applicable for small nets. The algorithm complexity of EKF is O(LN2). EKF is mathematically very elaborate and only a few experts have trained predefined dynamical system behaviors successfully}\par

\cite{jaeger_echo_state_RNN}
The third section explains how echo state networks can be trained in a
supervised way. The natural approach here is to adapt only the weights
of network-to-output connections. Essentially, this trains readout functions
which transform the echo state into the desired output signal. Technically,
this amounts to a linear regression task.
\end{comment}