text:
\section{Introduction}

The Koopman operator is an infinite-dimensional operator that governs the evolution of scalar observations in the state space for a nonlinear dynamical system, as shown in Equation \ref{eqn.koopman}. It has been proven \cite{koopman1931hamiltonian} that there exists a Koopman Operator $\mathcal{K}$ that can advance measurement functions in a linear operator fashion in an infinite-dimensional space.
\begin{equation}\label{eqn.koopman}
    \begin{gathered}
        \mathcal{K} g = g \circ \boldsymbol{F}
    \end{gathered}
 \end{equation}
where, the $\boldsymbol{F}$ represents the dynamics that map the state of the system forward in time; the $g$ is the representation of the system in the infinite-dimensional space. For practical use, existing approaches attempt to compute a finite-dimensional approximation of this operator. Dynamic Mode Decomposition (DMD) has been proven to be an effective way to find reduced-order models to represent the higher-dimensional complex systems \cite{williams2015data, williams2016extending, williams2014kernel, koopman1931hamiltonian}. For more extension work of Koopman operator methods to controlled dynamical refer to \cite{korda2018linear, proctor2016dynamic, kaiser2017data, kaiser2020data, broad2018learning, you2018deep,ma2019optimal}. However, the use of the Koopman operator as a linear predictor has been hindered by the computational complexity. A major concern is that, as the dimension of state space increases, the numerical method to approximate Koopman operator is running at a rapidly increasing polynomial time complexity to compute a rich set of basis functions. Another route to find Koopman operator approximation is through using Deep Neural Networks (DNN). The DNN provides a tremendous capacity to store and map lower-dimensional measurements to higher-dimensional lifted representations. Fruitful research results have been found in applying DNN in the linear embedding of nonlinear dynamics field \cite{lusch2018deep}.

Deep Learning of Koopman Representation for Control (DKRC) exemplifies a recent research trend of using DNN as a linear embedding tool to learn nonlinear dynamics and control the system based on the learned dynamics. In this framework, the optimal control is being achieved by combining a global model of the system dynamics and a local model for local replanning. The key feature of DKRC is that it extends the use of DNN for the finite-dimensional representation of the Koopman operator in a controlled dynamical system setting. It benefits from a data-driven learning algorithm that can automatically search for a rich set of suitable basis functions ($\psi(x)$) to construct the approximated linear model in the lifted space \cite{Han:2020f}. We can then rewrite the Equation \ref{eqn.koopman} into Equation \ref{eqn.koopman_psi}:
\begin{equation}\label{eqn.koopman_psi}
    \begin{gathered}
        \psi(x_{t+1}) = \mathcal{K} \psi(x_t)
    \end{gathered}
 \end{equation}

In this paper, we implement the Model Predictive Control (MPC) after learning the dynamics of the lifted high-dimensional linear system with DKRC. Other methods, such as regression or DAGGER (i-LQR or MCTS for planning) methods have been introduced after the learned model is available. However, those methods suffer from distribution mismatch problems or issues with the performance of open-loop control in stochastic domains. We propose using MPC iteratively, which can provide robustness to small model errors that benefit from the close loop control. As authors mentioned in Ref \cite{psrl}, if the system dynamics were to be linear and the cost function convex, the global optimization problem solved by MPC methods would be convex, which means in this situation the convex programming optimization techniques could be promising to achieve some theoretical guarantees of convergence to an optimal solution. Therefore MPC would certainly perform better in a linear world. MPC represents state of the art for the practice of real-time optimal control \cite{ddmpc}. MPC is an online optimization solver method that seeks optimal state solutions at each time step under certain defined constraints. We only take the first value of the planned control sequence of each time step. MPC has been proven to be tolerant to the modeling errors within Adaptive Cruise Control(ACC) simulation \cite{acc} \cite{cprm}. To solve the convex problem, we used CVXPY \cite{cvxpy} in this study.

Deep Deterministic Policy Gradient (DDPG) is a model-free, off-policy algorithm using DNN function approximators that can learn policies in high-dimensional, continuous action spaces \cite{ddpgo}. DDPG is a combination of the deterministic policy gradient approach and insights from the success of Deep Q Network (DQN) \cite{dqno}. DQN is a policy optimization method that achieves optimal policy by inputting observation and updates policy by back-propagate policy gradients. However, DQN can only handle discrete and low-dimensional action spaces, limiting its application considering many control problems have high-dimensional observation space and continuous control requirements. The innovation of DDPG is that it extends the DQN to continuous control domain and higher dimensional system and has been claimed that it robustly solves more than 20 simulated physics tasks, including classic problems such as cartpole swing-up, dexterous manipulation, legged locomotion and car driving \cite{ddpgo}. In addition, DDPG has been demonstrated to be sample-efficient compared to the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), which is also a black-box optimization method widely used for robot learning \cite{ddpgc}.

Since DKRC and DDPG are both capable of solving dynamic systems control problems with high-dimensional and continuous control space, direct comparison is desirable by constructing the two algorithms from the ground up and applying them to solve the same tasks. We also compare the DKRC to the analytical model obtained by the classic Euler-Lagrange Linearization method. The comparison are examined in the Inverted pendulum environment and Lunar Lander Continuous Control environment in OpenAI Gym \cite{brockman2016openai}.

This paper is organized into four sections. In Section \Romannum{2}, we briefly introduce the algorithms we use to achieve the comparison results. In Section \Romannum{3}, we set up the optimization problems in our simulation environment. In Section \Romannum{4}, we present the comparison results between the model-based and model-free approaches. Section \Romannum{5} concludes.

\section{Algorithm}
In this section we briefly introduce the algorithm of DDPG and DKRC.

\subsection{Deep Koopman Representation for Control}
Deep Koopman Representation for Control (DKRC) is a model-based control method which transfers a nonlinear system to a high-dimensional linear system with a neural network and deploys model-based control approaches like model predictive control (MPC). DKRC benefits from massive parameters of the neural network. By utilizing neural networks as a lift function, we can get a robust transformation from a nonlinear system to a lifted linear system as requirements. This paper also presents an auto decoder neural network \cite{dec} to map the planned states of MPC in lifted state-space back to the non-lifted state space to verify the DKRC control beyond the above comparison. A schematic diagram of the DKRC framework is shown in Figure \ref{fig:DKRC}:

\begin{figure}[ht]
    \centering
    \includegraphics[width=0.65\textwidth]{Attachments/Alg/DKRC_schematics.pdf}
    \caption{Schematics of DKRC framework}\label{fig:DKRC}
\end{figure}

The implementation of DKRC can be divided into the following steps:
\begin{enumerate}
    \item
    Build a neural network $\psi_{N}(x|\theta)$, where $N$ is the dimension we intend to lift, $x_t$ is the state space observations of the dynamical system at time $t$, $\theta^\psi$ represent parameters of the neural network. We decouple the optimal control problem into two function modules. The DKRC module handles the mapping process from the state space to the system in higher-dimensional space, which exhibits linear system behavior. We then feed the mapped states $z_t$ to the controller, which uses state-of-the-art planning algorithms to construct cost-to-go function and plan optimal control for the system. Those state-of-the-art control algorithms, such as iLQG and MPC, typically cannot work to the best efficiency when the system is highly nonlinear in nature. The introduction of the DKRC algorithm provides promising results in expanding the success of those conventional control algorithms into the field of challenging control problems in nonlinear dynamical systems.
    \item
    The DKRC algorithm achieves the lifting from low-dimension space to high-dimension space using the recent success of the deep neural networks in the control field. We use an autoencoder neural network structure as the backbone of DKRC. The encoder part of the autoencoder network will serve as the lifting function that maps the low-dimensional state observations to high-dimensional Koopman representations. Considering DKRC executes optimal MPC control in a lifted high-dimensional state space, it is also interesting to know how the DKRC makes control planning in high dimensional state space. To verify DKRC's planning behavior in a high-dimensional state space, we build a decoder neural network that maps the lifted space back to the original space. The schematic of the relationship between DKRC encoder and decoder is shown in Figure \ref{fig:decoder}.

    In this example, we have three state-space observations, for which we lift to Koopman representation with a dimension of eight (intermediate result in the center hidden layer). The decoder network will also map the lifted states back to the original state space with a dimension of three as the resulting output. Through this way, we can verify that our approach is behaving consistently in mapping between the low and high dimensional spaces, which is vital in assessing its reliability and avoiding a complete black-box behavior from a simple feed-forward neural network without validation.

    \begin{figure}[ht]
    \centering
    \includegraphics[width=0.5\textwidth]{Attachments/decoder.png}
    \caption{Schematics of autoencoder neural network sturcture of DKRC}\label{fig:decoder}
    \end{figure}

    \item
    To tune the parameters of the autoencoder network ($\theta^\psi$), we carefully design and closely monitor loss functions during the training process. Two loss functions are used in this task. We define the first loss function, $L1$, in Equation \ref{eqn.loss1},
    \begin{equation}\label{eqn.loss1}
        \begin{gathered}
         L1(\theta)=\frac{1}{L-1}\sum_{t=0}^{L-1}\parallel \psi(x_{t+1};\theta)-K*\psi(x_t;\theta)\parallel\\ K=\psi(x_{t+1};\theta)*\psi(x_t;\theta)^\dagger
        \end{gathered}
     \end{equation}
    \item
    The $\dagger$ sign represents the pseudo-inverse operation. By minimizing this loss function, we ensure that the Koopman operator theorem in Equation \ref{eqn.koopman} and \ref{eqn.koopman_psi} is being strictly enforced.
    Since we are interested in the dynamical system with control, we seek to identify the linearized system with $A$ and $B$ coefficient matrices. During the model training iterations, we can get the matrices by minimizing Equation \ref{eqn.ab}
     \begin{equation}\label{eqn.ab}
        \begin{gathered}
            M = [A,B] = argmin_{A,B}{|| \psi_{N}(X_{t+1}|\theta) - {A}\psi_{N}{({X_t}}|\theta) - {B}{U_t} ||}\\
        \end{gathered}
     \end{equation}
    For larger data sets with $K\gg N$, instead of solving the least square problem associated with Equation \ref{eqn.ab} directly, it is beneficial to solve a slightly modified normal equation as defined in Equation \ref{eqn.absol}
     \begin{equation}\label{eqn.absol}
        \begin{gathered}
            \boldsymbol{V} = M \boldsymbol{G}
        \end{gathered}
     \end{equation}
     where,\
         $$\boldsymbol{G} = {\begin{bmatrix}
            \psi_{N}(X_t|\theta)\\
            U_t\\\end{bmatrix}}{\begin{bmatrix}
            \psi_{N}(X_t|\theta)\\
            U_t\\\end{bmatrix}}^{T}, \
     \boldsymbol{V} = \psi_{N}(X_{t+1}|\theta){\begin{bmatrix}
            \psi_{N}(X_t|\theta)\\
            U_t\\\end{bmatrix}}^{T}$$\\
     Any solution to Equation \ref{eqn.absol} is a solution to Equation \ref{eqn.ab}. The size of the matrices $\boldsymbol{G}$ and $\boldsymbol{V}$ is the $(N+m)\times (N+m)$ and $N\times (N+m)$ respectively, hence independent of the number of samples ($K$) in the data set \cite{lpfnd}.\\
     We can also define a second loss function $L2(\theta)$ in Equation (\ref{eqn.loss2}), which ensures a controllable lifted linear states space.
    \begin{equation}\label{eqn.loss2}
        \begin{gathered}
        L2(\theta) = (N-rank(controllability(A,B)))
        \end{gathered}
    \end{equation}
     \item
     After training converges, we achieve a neural network model $\psi_{N}(x|\theta)$ that is capable of lifting state space to high-dimensional space with learned dynamic models coefficients $A$, $B$, and $C$. The $C$ matrix is necessary for later controller cost function design and is being defined as $C = X_t*\psi_{N}(X_t|\theta)^\dagger$
     \item
     Finally, we implement Model Predictive Control (MPC) or Linear-Quadratic Regulator (LQR), with a cost function in the format of $J(V) =\sum_t \psi_{N}(X_t|\theta)^\top C^\top Q C \psi_{N}(X_t|\theta)+u_t^\top R u_t$, \\ where $Q_{lifted} = C^{T} Q C$

\end{enumerate}

To summarize the above discussion and also the information depicted in Figure \ref{fig:DKRC}, the algorithm is listed below:
\DontPrintSemicolon
\begin{algorithm}[ht]
\SetAlgoLined
\KwIn{observations: x, control: u}
\KwOut{Planned trajectory and optimal control inputs: ($z_{plan}$, $v_{plan}$)}
\begin{itemize}
    \item Initialization
        \begin{enumerate}
            \item Set goal position $x^*$
            \item Build Neural Network:
            $\psi_N(x_t;\theta)$
            \item Set $z(x_t;\theta) = \psi_N(x_t;\theta) - \psi_N(x^*;\theta)$
        \end{enumerate}
    \item Steps
    \begin{enumerate}
        \item Set $K=z(x_{t+1};\theta)*z(x_t;\theta)^\dagger$
        \item Set the first loss function $L1$\\
        $L1(\theta)=\frac{1}{L-1}\sum_{t=0}^{L-1}\parallel z(x_{t+1};\theta)-K*z(x_t;\theta)\parallel$
        \item Set the second loss function $L2$\\
        $[A,B]={\bf z}_{t+1}
        \begin{bmatrix}
            {\bf z}_{t} \\
               U
         \end{bmatrix}
         \begin{pmatrix}
            \begin{bmatrix}
               {\bf z}_{t}\
               U
            \end{bmatrix}
            \begin{bmatrix}
               {\bf z}_{t} \\
               U
            \end{bmatrix}
         \end{pmatrix}^\dagger$\\

         $L2(\theta) = (N-rank(controllability(A,B)))+||A||_1+||B||_1$

         \item Train the neural network, updating the complete loss function\\ $L(\theta)=L1(\theta)+L2(\theta)$
         \item After converging, We can get system identity matrices A, B, C\\

         $C=\boldsymbol X_t*\psi_N(\boldsymbol X_t)^\dagger$
         \item Apply LQR or MPC control with constraints
    \end{enumerate}

\end{itemize}
\caption{Deep Koopman Representation for Control (DKRC)}
\end{algorithm}

\subsection{Deep Deterministic Policy Gradient}
Deep Deterministic Policy Gradient (DDPG) is a reinforcement learning algorithm based on an actor-critic framework. The DDPG and its variants have been demonstrated to be very successful in designing optimal continuous control for many dynamical systems. However, compared to the above model-based optimal control, DDPG employs a more black-box style neural network structure that outputs control directly based on learned DNN parameters. The DDPG and its variants typically require a significant amount of training data and a good design of the reward function to achieve good performance without learning an explicit model of the system dynamics. A schematic diagram of the DDPG framework is shown in Figure \ref{fig:DDPG}.
\begin{figure}[ht]
    \centering
    \includegraphics[width=0.65\textwidth]{Attachments/Alg/DDPG_schematics.pdf}
    \caption{Schematics of DDPG framework}\label{fig:DDPG}
\end{figure}

 The control goal of the DDPG algorithm is to find control action $u$ to maximize the rewards (Q-values) evaluated at the current time step. The training goal of the algorithm is to learn neural network parameters for the above networks so that rewards can be maximized at each sampled batch, while still being deterministic. The training of the DDPG network will then follow a gradient ascend update from each iteration, as defined in Equation \ref{eqn.DDPG_grad}

\begin{equation}\label{eqn.DDPG_grad}
    \begin{gathered}
        {\nabla_{\theta^{\mu}}}J \approx \mathbb{E}[\nabla_{\theta^{\mu}}Q(x,u|\theta^Q)|x=x_t, u=\mu(x_t|\theta^\mu)]
    \end{gathered}
\end{equation}

In Equation \ref{eqn.DDPG_grad}, we have two separate networks involved: Value function network $\theta^Q$ and policy network $\theta^\mu$. To approximate the gradient from those two networks, we design the training process relying on one replay buffer and four neural networks: Actor, Critic, Target Actor, Target Critic.

At the gathering training samples stage, we use a replay buffer to store samples. A replay buffer of DDPG essentially is a buffered list storing a stack of training samples $(x_t, u_t, r_t, x_{t+1})$. During training, samples can be drawn from the replay buffer in batches, which enables training using a batch normalization method \cite{replayb}. For each stored sample vector, $t$ is current time step, $x$ is the collection of state observations at current time step $t$, $r$ is numerical value of reward at current state, $u$ is the action of time step $t$, and finally, the $x_{t+1}$ is the state in the next time step $t+1$ as a result of taking action $u_t$ at state $x_t$. Replay buffer makes sure an independent and efficient sampling \cite{replaye}.

As discussed above, the DDPG framework is a model-free reinforcement learning architecture, which means it does not seek an explicit model for the dynamical system. Instead, we use a value function network, also called Critic network $Q(s,a|{\theta^{Q}})$, to approximate the result from a state-action pair. The result of a trained Critic network estimates the expected future reward by taking action $u_t$ at state $x_t$. If we take the gradient of the change in the updated reward, we will be able to use that gradient to update our Policy network, also called Actor network. At the end of the training, we obtain an Actor network capable of designing optimal control based on past experience. Since there are two components in the formulation of the gradient of reward in Equation \ref{eqn.DDPG_grad}, we can apply the chain rule to break the process into evaluating gradients from Actor and Critic networks separately. The total gradient is then defined as the mean of the gradient over the sampled mini-batch, where $N$ is the size of the training mini-batch taken from replay buffer $R$, as shown in Equation \ref{eqn.policy}. Silver et al. showed proof in Ref. \cite{silver2014deterministic} that Equation \ref{eqn.policy} is the policy gradient that can guide the DDPG model to search a policy network to yield the maximum expected reward.
\begin{equation}\label{eqn.policy}
    \begin{gathered}
        {\nabla_{\theta^{\mu}}}J \approx \frac{1}{N} \sum_t [\nabla_u Q(x,u|\theta^Q)|_{x=x_t, u=\mu(x_t)} \nabla_{\theta^{\mu}} \mu(x|\theta^\mu)|_{x_t}]
    \end{gathered}
\end{equation}

In this case, dynamical responses from the system are built into the modeling of the Critic network, thus indirectly modeled. The accuracy in predicted reward values is used as the only training criterion in this process, signifying the major difference between the model-free reinforcement learning and the model-based optimal control method such as DKRC. Some of the later observations in different behaviors from model-based vs. model-free comparison have roots stemming from this fundamental difference.

We separate the training of the DDPG into the training of the Actor and Critic networks.

Actor, or policy network ($\theta^{\mu}$), is a simple neural network with weights $\theta^\mu$, which takes states as input and outputs control based on a trained policy $\mu(x,u|{\theta^{\mu}})$. The Actor is updated using the sampled policy gradient generated from the Critic network, as proposed in Ref. \cite{pga}.

Critic, or value function network, is a neural network with weights $\theta^Q$, which takes a state-action pair $(x_t,u_t)$ as input. We define a temporal difference error term, $e_t$ (TD error), to track the error between the current output compared to a target Critic network with future reward value using future state-action pair as input in the next time step. The total loss from the Critic network during a mini-batch is defined as $L$ in Equation \ref{eqn.error_loss}, which is based on Q-learning, a widely used off-policy algorithm \cite{qlr}.
\begin{equation}\label{eqn.error_loss}
    \begin{gathered}
        e_t = (r_t + \gamma Q^{'}(x_{t+1}, \mu^{'}(x_{t+1}|{\theta^{\mu^{'}}})| \theta^{Q^{'}})) - Q(x_t,u_t|\theta^{Q})\\
        L = \frac{1}{N} \sum_t{e_t}^2
    \end{gathered}
\end{equation}
In this Critic loss function in Equation \ref{eqn.error_loss}, the terms with prime symbol $(')$ is the Actor/Critic model from the target networks. Target Actor $\mu^{'}$ and target Critic $Q^{'}$ of DDPG are used to sustain a stable computation during the training process. These two target networks are time-delayed copies of the actual Actor/Critic networks in training, which only take a small portion of the new information from the current iteration to update target networks. The update scheme of these two target networks are defined in Equation \ref{eqn.target_nn}, where $\tau$ is Target Network Hyper Parameters Update rate super parameter with a small value ($\tau \ll 1$) to improve the stability of learning.
\begin{equation}\label{eqn.target_nn}
    \begin{gathered}
        {\theta^{Q^{'}}} \leftarrow \tau \theta^Q + (1-\tau)\theta^{Q^{'}}\\
        {\theta^{\mu^{'}}} \leftarrow \tau \theta^{\mu} + (1-\tau)\theta^{\mu^{'}}\\
    \end{gathered}
\end{equation}

To train DDPG, we initialize four networks ($Q(s,a|\theta^Q), \mu(s|\theta^\mu), Q^{'}(s,a|\theta^{Q^{'}}), \mu^{'}(s|\theta^{\mu^{'}})$) and a list buffer ($R$) first, then select an initial action from actor, executing action $a_t$ and observe reward $r_t$ and new state $s_{t+1}$, store $(s_t, a_t, r_t, s_{t+1})$ in the buffer $R$, sampling mini training batch input randomly from R, updating critic by Equation \ref{eqn.error_loss}, updating actor by Equation \ref{eqn.policy}, updating target actor and critic by Equation \ref{eqn.target_nn} until loss value converges.

\bigskip

\section{Experiment Setup}
To obtain benchmark comparison results, we use the classic 'Pendulum-v0' OpenAI Gym environment \cite{brockman2016openai} to examine the behaviors of controllers built based on DDPG and DKRC for this inverted pendulum problem. The OpenAI Gym is a toolkit designed for developing and comparing reinforcement learning \cite{rl}. The inverted pendulum swing-up problem is a classic problem in the control literature, and the goal of the system is to swing the pendulum up and make it stay upright, as depicted in Figure \ref{fig:env_intro}. Although both the two approaches are data-driven in nature, thus versatile in deployment, We would like to still stick with the classic control problem, which has rich documentation of analytical solution for later comparisons. As will be shown in the next section, the DKRC approach not only successfully finishes the control task using a learned dynamics directly resembling the analytical solution but can also explain the system in lifted dimensions using Hamiltonian Energy Level theorem.
\begin{figure}[ht]
    \centering
    \includegraphics[width=0.45\textwidth]{Attachments/env_intro/pendulum.png}
    \caption{Environment visualization}\label{fig:env_intro}
\end{figure}
\subsection{Problem set-up}
 A visualization of the simulation environment is shown in Figure \ref{fig:env_intro}. As shown in the picture, the simple system has two states $x = [\theta,\dot{\theta}]$ and one continuous control moment at the joint, which numerically must satisfy $-2\leq u\leq2$. This limit in control magnitude is applied with the intention to add difficulty in designing the control strategy. For the default physical parameter setup, the maximum moment that is allowed for the environment will not be sufficient to raise the inverted pendulum to the upright position in one single move. The control strategy needs to learn how to build momentum by switching moment direction at the right time. The OpenAI Gym defines the observations and control in Equation \ref{eqn.state_space_pen}.\\
\begin{equation}\label{eqn.state_space_pen}
    \begin{gathered}
        \boldsymbol{\chi}=[\cos{\theta}, \sin{\theta}, \dot{\theta}], \quad where\ \theta\in[-\pi,\pi], \dot{\theta}\in[-8,8] \\
        \boldsymbol{U}=[u], \quad where\ u\in[-2,2]
    \end{gathered}
\end{equation}\\
The cost function is designed in Equation \ref{eqn.reward}. The cost function tracks the current states ($\theta$) and control input ($u$). The environment also limits the rotational speed of the up-swing motion, by including a $\dot{\theta}^2$ term in the cost function. By minimizing this cost function, we ask for the minimum control input to ensure the pendulum ends at the upright position with minimum kinetic energy levels. During the comparison, we are using the Equation \ref{eqn.reward} directly in the controller design for the DKRC, whereas we negate cost function and transforms it into a negative reward function, where 0 is the highest reward for the DDPG training. By using this simple simulation environment, we ensure the two approaches are being compared on the same basis with the same reward/cost function definition.
\begin{equation}\label{eqn.reward}
    \begin{gathered}
        reward = \theta^2 + 0.1\dot{\theta}^2 + 0.001u^2
    \end{gathered}
\end{equation}
The dynamical system of the swing-up pendulum can be analytically solved by Euler–Lagrange method \cite{ele}, with its governing equation shown in Equation \ref{eqn.lagrange}, where $\theta = 0$ is the upright position, $g$ is the gravitational acceleration, $m$ is the mass of the pendulum, $l$ is the length of the pendulum.
\begin{equation}\label{eqn.lagrange}
    \begin{gathered}
        \ddot\theta = -\frac{3g}{2l}\sin(\theta + \pi) + \frac{3}{ml^2}u
    \end{gathered}
\end{equation}
By converting the second order ordinary differential equation(ODE) in Equation \ref{eqn.lagrange} to the first order ODE, we can get Equation \ref{eqn.ODE}.
\begin{equation}\label{eqn.ODE}
    \begin{gathered}
        \frac{d}{dt}\begin{bmatrix}
        \cos\theta\\
        \sin\theta\\
        \dot\theta \end{bmatrix} = \begin{bmatrix}
        0 & -1 & 0\\
        1 & 0 & 0\\
        0 & \frac{3g}{2l} & 0
        \end{bmatrix}\begin{bmatrix}
        \cos\theta\\
        \sin\theta\\
        \dot\theta \end{bmatrix}+
        \begin{bmatrix}
        0\\
        0\\
        \frac{3}{ml^2} \end{bmatrix}u\\
    \end{gathered}
\end{equation}
We use the default physical parameters defined in the OpenAI Gym, i.e., $m=1$, $l=1$, $g=10.0$, $dt=0.05$. We discretize the model using zero order hold (ZOH) method \cite{zoh} with a sampling period $dt$. As a result, we can achieve the linearized governing Equation \ref{eqn.zoh} with observed states $y_t$. The numerical values for the hyperparameters used in the default problem setting are also listed in Equation \ref{eqn.zoh_mat}.
\begin{equation}\label{eqn.zoh}
    \begin{gathered}
        x_{t+1} = {A_d}{x_t} + {B_d}{u_t}\\
        y_t = C{x_t}
    \end{gathered}
\end{equation}
with\\
\begin{equation}\label{eqn.zoh_mat}
    \begin{gathered}
        A_d = \begin{bmatrix}
        0.9988 & -0.04998 & 0\\
        0.04998 & 0.9988 & 0.05\\
        0.01875 & 0.7497 & 1
        \end{bmatrix}\
        B_d = \begin{bmatrix}
        0\\
        0\\
        0.15 \end{bmatrix}\\
        C = \begin{bmatrix}
        1 & 0 & 0\\
        0 & 1 & 0\\
        0 & 0 & 1 \end{bmatrix}
    \end{gathered}
\end{equation}
We use $A_d$ and $B_d$ as the benchmark of learned dynamic models comparison between DKRC and Euler-Lagrange Linearization in later sections.

\subsection{Parameters of DDPG and DKRC}
The following Table (\ref{my-label}) is the parameters we use to obtain DDPG and DKRC solutions. Both two methods are trained on NVIDIA Tesla V-100 GPUs on an NVIDIA-DGX2 supercomputer cluster.
\begin{table}[ht]
\centering
    \caption{Parameters of different methods}
\label{my-label}
\begin{tabular}{ |c|c|c|c| }
\hline
Method & Parameter & Value \\
\hline
 {}
& {Buffer size} & 1e6 \\
{}   & {Batch size} & 64 \\
{}   & {$\gamma$ (Discount factor)} & 0.9 \\
DDPG   & {$\tau$ (Target Network Update rate)} & 0.001 \\
{}   & {Target \& Actor learning rate} & 1e-3 \\
{}   & {Target \& Critic learning rate} & 1e-2 \\
{}   & {Training epochs} & 5e4 \\
\hline
DKRC
& {Lift dimension} & 8 \\
{}   & {Training epochs} & 70 \\
\hline
\end{tabular}
\end{table}

The result of the DDPG solution is an Actor neural network with optimal parameters for the nonlinear pendulum system - $\mu(x;\theta^{*})$.

The result of the DKRC solution is identity matrices $A_{lift}$, $B_{lift}$, $C$ of the lifted space, and a lift neural network $\psi_{N}(x;\theta)$ for observations of the unknown dynamical system.

\bigskip
\section{Results and Comparison}

\subsection{Control Strategies of DDPG and DKRC}
We present results from the DKRC vs DDPG by specifying five initialization configuration for the problem. We have choices of defining the starting position and also the initial disturbance in the form of starting angular velocity of the pendulum. In this study, we initialize the pendulum at five different positions:\ $\theta_0 = \pi$ (lowest position), \ $\theta_0 = \frac{\pi}{2}$ (left horizontal position), \ $\theta_0 = -\frac{\pi}{2}$ (right horizontal position), $\theta_0 =  \pm \frac{\pi}{18}$ (close to upright position). The initial angular velocity of pendulum at different initial positions is $\dot{\theta_0} = 1rad/s$ (clockwise). For better visualization, we map the angle from $[-\pi, \pi]$ to $[0, 2\pi]$, where the upright position (goal position) is always achieved at $\theta=0$ and $\dot{\theta} = 0$. The result is shown in Figure \ref{fig:traj_lowes}$-$ \ref{fig:traj_lu}.

\begin{figure}[!htbp]
\centering
\includegraphics[width=0.4\textwidth]{Attachments/beha/low0.png}
\includegraphics[width=0.4\textwidth]{Attachments/beha/low1.png}
\caption{Trajectories of Swinging Pendulum implementing DDPG model (left) vs. DKRC model (right) initialized at the lowest position}\label{fig:traj_lowes}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[width=0.4\textwidth]{Attachments/beha/lh0.png}
\includegraphics[width=0.4\textwidth]{Attachments/beha/lh1.png}
\caption{Trajectories of Swinging Pendulum implementing DDPG model (left) vs. DKRC model (right) initialized at the left horizontal position}\label{fig:traj_lh}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[width=0.4\textwidth]{Attachments/beha/rh0.png}
\includegraphics[width=0.4\textwidth]{Attachments/beha/rh1.png}
\caption{Trajectories of Swinging Pendulum implementing DDPG model (left) vs. DKRC model (right) initialized at the right horizontal position}\label{fig:traj_rh}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[width=0.4\textwidth]{Attachments/beha/lu0.png}
\includegraphics[width=0.4\textwidth]{Attachments/beha/lu1.png}
\caption{Trajectories of Swinging Pendulum implementing DDPG model (left) vs. DKRC model (right) initialized at the close left upright position}\label{fig:traj_lu}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[width=0.4\textwidth]{Attachments/beha/ru0.png}
\includegraphics[width=0.4\textwidth]{Attachments/beha/ru1.png}
\caption{Trajectories of Swinging Pendulum implementing DDPG model (left) vs. DKRC model (right) initialized at the close right upright position}\label{fig:traj_ru}
\end{figure}

Figure \ref{fig:traj_lowes}$-$ \ref{fig:traj_ru} show that DDPG and DKRC have similar control strategies in most initial positions. DDPG needs less time to arrive at the goal position than DKRC when the pendulum is initialized on the right side. It tends to use smaller control torque as a result of that DDPG has constraints term for control torque. Still, DDPG never succeeds in getting an absolute upright position, i.e., at the final position, a non-zero control input is always required to sustain a small displacement away from the goal position. On the contrary, DKRC can achieve a precise goal position with much less training time than DDPG. Once a proper dynamical system model can be learned directly from data, it makes more sense to execute control using model-based controller design such as MPC.

Another way to show the differences in control strategies is by plotting the measured trajectories during repeated tests. In Figure \ref{fig:deployment}, we test the pendulum game for 50 games with a total of 10000 time-steps utilizing both methods (DDPG \& DKRC) solutions.
\begin{figure}[ht]
\centering
\includegraphics[width=0.42\textwidth]{Attachments/50games/DDPGtheta.png}
\includegraphics[width=0.42\textwidth]{Attachments/50games/dkrctheta.png}
\caption{50 pendulum games data recorded using DDPG model (left) vs. DKRC model (right), color mapped by energy}\label{fig:deployment}
\end{figure}
In this comparison, we plot the measurements of $\dot{\theta}$ vs. $\theta$ on a 2D basis, colored by the magnitude of the cost function defined in Equation \ref{eqn.reward}. The goal is to arrive at the goal position ($\dot{\theta}=0$ and $\theta=0$, the center of each plot) as quickly as possible. Figure \ref{fig:deployment} shows that DDPG tends to drive the states into "pre-designed" patterns, and execute similar control strategies for the 50 games. Therefore the data points on the left subplot appear to be less than the one on the right. DKRC, on the other hand, tends to exhibit different control behavior due to the local replanning using MPC. The result of the local replanning is that it generates multiple trajectories solving the 50 games with different random initializations. During this comparison, we illustrate that DDPG is indeed deterministic, which is a good indicator of the system's reliability. However, we want to point out that the robustness of the system will also benefit from a local replanner available under different initializations or disturbances since the "pre-designed" patterns are learned from past experience, therefore, cannot guarantee a viable solution for unforeseeable situations when we move onto more complex systems.

By transforming the observed states, we can also show the relationship between the designed control and the energy levels in the system. Consequently, our control goal is to achieve the lowest energy level of the system in terms of lowest magnitudes in both kinetic energy and potential energy. In Figure \ref{fig:pendulum_energy}, we present the Hamiltonian energy level plots with respect to the measured states ($\theta$, $cos(\theta)$, $sin(\theta)$, and $\dot{\theta}$) for the forced frictionless pendulum system, each colored by the same energy level scale. The energy term used in forced pendulum is defined as $\frac{1}{2}\dot{\theta}^2+cos(\theta)+u$.
\begin{figure}[ht]
\centering
\includegraphics[width=0.49\textwidth]{Attachments/50games/ddpgenergy.png}
\includegraphics[width=0.49\textwidth]{Attachments/50games/DKRC_energy.png}
\caption{DDPG(left) vs. DKRC(right): Recorded Observations \& Energy}\label{fig:pendulum_energy}
\end{figure}
Figure \ref{fig:pendulum_energy} shows that DKRC's trajectories are concentrated in lower energy areas compared to DDPG, which means it intends to minimize the energy directly. This behavior also resembles many design strategies used in classic energy-based controller design approaches.

\subsection{Control Visualization using Decoder Neural Network}
Another benefit of having a model-based controller design is the interpretability of the system. We can preview the design trajectory since our MPC controller implements the receding horizon control. By deploying one pendulum game, we obtain one planned trajectory for the linear system in the lifted dimension space. To get the comparison between the planned trajectory and the measured trajectory in the state space, we utilize the decoder neural network obtained during the training to map the trajectory in higher-dimensional space (8-dimension for inverted pendulum) to the lower-dimensional space (2-dimension). The comparison between such recovered trajectory planning and the actual trajectory is presented in Figure \ref{fig:mpctraj}. In this figure, we again mapped the $\theta$ in the range of $0$ to $2\pi$ for visualization purposes, whereas the goal position is still at the $\dot{\theta}=0$ and $\theta=0$ position.

In this example case, we initialize the pendulum close to the goal position but give it a moderate initial angular velocity pointing in the opposite direction to the goal position. The DKRC can plan a simple trajectory with continuous control. During the process, we executed MPC multiple times and used feedback measurements to improve the design trajectory. The planned trajectory is being closely followed except certain locations close to $0$ and $\pi$ position. The outlier behavior comes from the neural network treatment for the discontinuity and does not pollute the efficient control for the entire problem. It is worth noticing that, to arrive at this result, we learn the unknown nonlinear dynamics using a purely data-driven approach, and we have to go through an encoding-decoding process to recover the planned trajectory. It is promising to state that the use of Koopman representation for nonlinear control can help with system interpretability, which is currently an active research area.
\begin{figure}[ht]
\centering
\includegraphics[width=0.5\textwidth]{Attachments/mpctraj.png}
\caption{DKRC's planned trajectory and actually executed trajectory in Inverted pendulum}\label{fig:mpctraj}
\end{figure}

To verify whether DKRC is valid in a more complex environment than Inverted pendulum, we also deploy it in the Lunar Lander - continuous control environment of OpenAI Gym. A simple explanation of 'Lunar Lander' is exhibited in Figure \ref{fig:envll}. The control goal is to guide the lunar lander to arrive at the landing zone as smoothly as possible \cite{Han:2020f}. The system is also unknown and must be learned from data. We implement the DKRC framework and use MPC for trajectory planning, and the result is shown in Figure \ref{fig:llmpctraj}.

\begin{figure}[ht]
\centering
\includegraphics[width=0.46\textwidth]{Attachments/env_intro/lunar.png}
\caption{Lunar Lander Environment}\label{fig:envll}
\end{figure}
The actual trajectory measured in state-space is shown in hollow red circles. The planned trajectory is colored by the distance away from the originally planned location. It is evident that in the region where the originally planned location is in the immediate vicinity (dark blue color), the actual trajectory is following the plan very precisely. We implement a finite-horizon control during each MPC planning phase. We plan and execute control with several steps beyond the current state as displayed in lighter green color. The actual trajectory slowly deviates from the planned trajectory when it starts to drift away from the originally planned location. This behavior is expected since we are relying on open-loop control during those finite horizon plannings. The actual trajectory and the projected trajectory merge again once the next round of the MPC control is executed.

In this figure, we demonstrate that the deviation from the planned trajectory and the measured trajectory is from the open-loop planning, rather than from the error introduced while passing the state inputs through the encoder-decoding neural network. The proposed structure is capable of recovering the designed strategy in higher-dimensional space and improving the system's interpretability.
\begin{figure}[ht]
\centering
\includegraphics[width=0.6\textwidth]{Attachments/decoderll.png}
\caption{DKRC's planned trajectory and actual executed trajectory in Lunar Lander, color mapped by the distance between the planned point and the executed point}\label{fig:llmpctraj}
\end{figure}

\subsection{Robustness Comparison}
To compare the robustness of these two methods, we introduce noises to the state measurements and observe the control outcome from DDPG and DKRC. The noise is designed as multiplying the states with a noise ratio, which is randomly selected from range $[0.6, 1]$ during deployment. In this test, we assume the learned dynamics from DDPG and DKRC are not affected by the noise; only the observations during deployment are affected, representing a cyber-security attack during the operation phase. The new state inputs would be $x = x*noise$ in this scenario. The result for five repeat games is shown in Figure \ref{fig:stable}.
\begin{figure}[ht]
\centering
\includegraphics[width=0.48\textwidth]{Attachments/robust/ddpg.png}
\includegraphics[width=0.48\textwidth]{Attachments/robust/dkrc.png}
\caption{Five pendulum games with input states noise using DDPG (left) vs. DKRC (right) solutions}\label{fig:stable}
\end{figure}
 Each line with the same color in the subplot represents measurements from a single game. Even with a high noise ratio, DKRC can succeed in the control task for three out of five games, whereas DDPG fails every game in Figure \ref{fig:stable}. In this test, we demonstrate that the DKRC is capable of designing a control strategy based on noisy data and continuously adjust the control based on the feedback control loop so that it is still robust in a noisy environment. The robustness of DKRC is another advantage compared to a neural-network-based control system, which relies heavily on state measurement accuracy.

\subsection{Learned dynamics of DKRC compared to Euler-Lagrange analytical solution}
 To illustrate the validity of the learned dynamics using DKRC, we present benchmark comparisons between the proposed DKRC framework and the Euler-Lagrange method's analytical solutions. As previously shown in Equation \ref{eqn.zoh} and \ref{eqn.zoh_mat}, we have obtained the identity matrices for the linearized system using ZOH method. We are making the same assumption for the linearized system in the lifted-dimensional space by the DKRC method. For comparison with different dimension embeddings, we pick the matrices corresponding to the top $n$ eigenvalues from the $A$ and $B$ obtained through DKRC. For the inverted pendulum problem, we collect $K$ time-step data (only need $K=2000$ data points) to obtain Koopman representation of the system. The resultant dimension is $N=8$ ($K \gg {N}$). The learned dynamics $A, B$ of the lifted linear system $\psi_{N}(x|\theta)$ are shown in the following matrices\\
\\
$A_{DRKC} =
    \scriptsize{\begin{bmatrix}
                \begin{array}{ccc|ccccc}
    1.01 & -0.06 & -0.01 & -0.08 & -0.08 & 0.06 & -0.01 & -0.01 \\
    0.00 & 0.98 & -0.04 & 0.02 & 0.06 & -0.07 & 0.01 & 0.02 \\
    0.06 & 0.05 & 0.93 & 0.06 & 0.00 & 0.00 & -0.03 & 0.02\\
    \hline
    0.03 & -0.00 & -0.01 & 0.97 & -0.04 & 0.05 & -0.01 & -0.01\\
    0.11 & 0.02 & -0.03 & 0.01 & 0.98 & 0.05 & -0.07 & 0.01\\
    0.04 & 0.05 & 0.01 & 0.02 & 0.01 & 1.01 & -0.05 & 0.04\\
    0.01 & 0.014 & 0.02 & 0.01 & 0.08 & -0.04 & 0.96 & 0.04\\
    -0.00 & -0.01 & -0.02 & -0.00 & 0.08 & -0.08 & -0.00 & 1.00
    \end{array}
    \end{bmatrix}} \
B_{DRKC} = \scriptsize{\begin{bmatrix}
    0.00014\\
    0.00018\\
    -0.00024\\
    0.00017\\
    0.00021\\
    0.00038\\
    0.00008\\
    -0.0001 \end{bmatrix}}$\\ \\
 To measure the similarity of $A_d$ (in Equation \ref{eqn.zoh_mat}), $A_{DDPG}$, $A_{DKRC}$ we achieved, we adopt the Pearson correlation coefficient(PCC) method \cite{pcc} in Equation \ref{eqn.pcc}, where matrices with bar operator, e.g. $\bar{A}$, is the sample mean of that matrix. The result $r(A,B)\in[0,1]$ represents the correlation between the two matrices. Two matrices ($A$ and $B$) are more similar when $r$ is closer to 1.\\
     \begin{equation}\label{eqn.pcc}
        \begin{gathered}
            r(A,B) = \frac{\sum_m \sum_n(A_{mn}-\bar{A})(B_{mn}-\bar{B})}{\sqrt{(\sum_m \sum_n(A_{mn}-\bar{A})^2)(\sum_m \sum_n(B_{mn}-\bar{B})^2)}}
        \end{gathered}
    \end{equation}\\
To compare the similarity of these two matrices with different dimensions, we extract the left top $3\times3$ part of the $A_{DKRC}$. We achieve a correlation score of $r(A_{d}, A_{DKRC})=0.8926$, which means the lifted linear system of DKRC is very similar to the analytical system model solved by Euler-Lagrange method. This high correlation score indicates that the data-driven Koopman representation of the system can reveal the intrinsic dynamics purely based on data samples.

On a separate note, we do not expect the two results are exactly the same since by lifting the system to a higher-dimensional space, we have more neurons in DKRC neural networks to store the system information that was not included in the previous comparison. In addition, we also deploy the linearization model obtained by the Euler-Lagrange method with MPC to the same OpenAI Gym environment to examine the effectiveness by direct linearization without lifting. Unlike DKRC or DDPG, which can work for any arbitrary initial configuration, control designed by the Euler-Lagrange method only works when the pendulum's initial position is between $[-23.4^{\circ}, 23.4^{\circ}]$ with small initial angular velocity disturbance. A sample comparison between Euler-Lagrange MPC and DKRC is exhibited in Figure \ref{fig:lec}.

As shown in Figure \ref{fig:lec}, DKRC only spends around $20\%$ time of Euler-Lagrange linearization method to make pendulum converge to the upright position.

\begin{figure}[ht]
\centering
\includegraphics[width=0.48\textwidth]{Attachments/LE/dkrcandll.png}
\includegraphics[width=0.48\textwidth]{Attachments/LE/ll.png}
\caption{One pendulum game using DKRC model (left) vs. Euler-Lagrange method (right); Both tests initialized at $\theta_0 = \frac{\pi}{18}$, $\dot{\theta_0} = 0.5$}\label{fig:lec}
\end{figure}

\section{Conclusions}
We provide a systematic discussion of two different data-driven control frameworks: Deep Deterministic Policy Gradient (DDPG) and Deep Learning of Koopman Representation for Control (DKRC). Both the two methods are data-driven and adopt neural networks as central architecture, but the controller design is model-free for DDPG, whereas DKRC utilizes a model-based approach. Our experiments in a simple swing-up pendulum environment demonstrate the different solutions achieved by DKRC and DDPG. The DKRC method can achieve the same control goal as effective as the DDPG method but requires much less training epochs and training samples (70 epochs vs. $5\times10^4$ epochs). Due to the physics model-based nature, DKRC provides better model interpretability and robustness, which are both critical for real-world engineering applications.

\section*{Acknowledgment}
Y.Han would like to acknowledge research support from ONR award N00014-19-1-2295.

\bigskip
\bibliographystyle{unsrt}

redpajama_set_name: RedPajamaArXiv


Who or what inspired your latest novel There's A Possibility?
Actually, it was a Patrick Dempsey movie called Made of Honor and I loved the plot so much, but of course I wanted to see how you could twist it to be gay. This has been in the back of my mind ever since it came out in 2008. I'm glad it's finally out of my head and on the pages!
Oh wow. I guess I'd have to go with Nymphadora Tonks from Harry Potter. She was always a badass in my opinion. Youngest Auror of her time AND a Hufflepuff. Fun colored hair. She was my idol when she appeared in the books.
Character sketches! My absolute favorite part of the story is understanding my characters and where they come from, what their purposes are and where I think they're headed.
Mostly everyday people I live around and myself. I think I put a little piece of me into all of my characters. I try to write what I know.
If it weren't for my cat begging for my attention all of the time, I'd saythat it was my living room couch with lots of pillows and blankets and a hot cup of coffee. But I generally find myself at Starbucks to avoid distraction (go figure).
See the thing. Plot the thing. Write the thing. Get blocked by the thing. Cry over the thing. Write more of the thing. Send the thing to Editor and then cry when it returns looking like a crime scene. Finish polishing the thing. Cry again over the thing because it's finished. Cry more over the thing when people enjoy the finished product.
I think it made me more aware of how I write, definitely with my use of adverbs and how I use the words "that" and "like" a LOT and how I need to correct it both in my speech and in my writing.
My editor. Claire works miracles with the shite I send her! LOL.
Everything from my readers is memorable. Even if it's not a stellar review, I am happy that someone took the time to read what I wrote and at least gave it a shot. Of course, I love the great reviews, but I take all of them in stride.
Am I lame if I say Harry Potter? It was the first real set of books I ever fell in love with and have changed the way I look at books for the rest of my life. I may never be as great as Ms. Rowling, but I certainly thank her for giving me the gift of loving the written word and the magical places it can take you.
Do you have any new books coming out? If so what are they about?
My next book is called The Sound of Silenceand it's about a young woman who has been mute for over half of her life, falling in love with one of her caretakers and her caretaker trying to defend her own emotions against the world that thinks they are wrong.
Cream and sugar with a splash of black coffee. Or just a soy caramel macchiato from Starbucks.
Lou's upcoming book The Sound of Silence will be available on Amazon Kindle August 31st. It is available for preorder now by clicking here.
I just finished reading Dangerous Waters by Radclyffe, and I have had the opportunity to interview the author about it!
Who or what inspired your latest novel Dangerous Waters?
I was impressed by the incredible response from people—from community members to first responders—to the two nearly back-to-back hurricanes in Texas and Florida in 2017. The selfless outpouring of aid and support made me want to capture some of that as a backdrop to a First Responders novel, so I quickly re-arranged my writing schedule to fit Dangerous Waters in as the next book.
Every book I've ever read has helped create my life-long love of reading. I started writing lesbian fiction to fill a gap in what I was able to find to read at the time. Now even though there is a wealth of new queer fiction available, I enjoy the process of writing and sharing my work with readers.
If I had to pick one: Safe Harbor, since it is one of the earliest ones and begins the Provincetown Tales. I have always loved the characters and the setting—one of my favorite places to visit.
Again – not a fair question since I have quite a few characters I like to revisit and write about. At the moment it's Blake Remy, a trans teen first introduced in the second book in the Rivers series (Prescription for Love). His story has turned out to be a major secondary plotline in three books, and I get lots of emails asking me to write more about him. Writing a young queer character has been challenging and rewarding.
For me, any story, but particularly a romance, is about the characters. I start with two characters who find themselves in a situation they never expected, often at odds at first, who learn more about themselves as they come to know each other.
Most of the situations I write about come from real life events I have either experienced (as in the medical romances) or events I've read about that seem to me to be important and interesting for readers. Life is filled with "every day" heroes and those are the stories I like to tell.
I can write anywhere, but I prefer writing while sitting on a sofa rather than at a desk. I don't listen to music although I can edit a draft while watching baseball on TV 😊.
I dictate my first draft with voice-to-text software and then edit the transcription on computer. I write from page one to the end, and I never write scenes out of sequence. The book grows as the characters interact.
I learned to be a much better self-editor as I became more experienced. Being published has helped me be a better writer.
Becoming a member of the Romance Writers of America. Attending the annual meetings and learning from the best romance writers in the world has really helped me improve my craft.
I've been fortunate to receive countless messages of support over the years, and they are all invaluable. I am always especially happy when someone tells me my books helped them come out or to realize they are not alone.
The first book I ever read with a lesbian character—that was when I understood who I was.Beebo Brinkerby Ann Bannon.
Americano with ½ and ½.


text:
Mr Bet Casino Review (NZ)
Welcome to the world of Mr Bet, where everything is just a little bit better.
That's what they say. But are you convinced? No worries! That's why we're here for you - ready to tell you all about it. So hang tight and let us do our thing! We'll have your back in no time at all. And who knows? You might even enjoy yourself as well.
400% Up to NZ$2250 Bonus
Vast collection of online casino games
Accepts cryptocurrencies
Live dealer available
Not available in some countries
Visit Mr Bet Casino
Overview Of Mr Bet
Bettors can expect a warm welcome from this New Zealand casino, with anything from free spins and bonuses galore waiting for them on sign-up. Plus there's plenty more on offer throughout the site; whether it be games providers or security measures, there really is something for everyone at Mr Bet Casino NZ (in fact, everyone will find something they like if we're being honest!).
What Bonuses Does Mr Bet Offer?
The welcome bonus is designed to assist in attracting players and developing any online casino by enticing individuals to sign up and recommend it to their friends and relatives. This isn't anything new; casinos have tried a variety of methods to attract clients, including offering free money and even free spins. Mr Bet Casino has an incredible welcome bonus which is a fantastic incentive for new New Zealand players. So without further adieu, Mr Bet's gorgeous welcome bonus package offers a 400% bonus up to NZD 2,250 on your first four deposits. Here's the breakdown:
1st deposit = 150% bonus up to $225
2nd deposit = 100% bonus up to $450
3rd deposit = 50% bonus up to $750
4th deposit = 100% bonus up to $825
Don't worry, we already did the math for you. Just a reminder, the welcome bonus is available within 5 days after you register. The more you play, the greater your bonus will be. It's easy as pie.
More Mr Bet Bonuses and Promotions
Mr Bet offers you additional bonuses and promotions to make your gaming experience more pleasurable. Aside from the eye-catching welcome bonus package, there are a variety of bonuses and specials accessible. The 5% cashback is one of them. Mr Bet reimburses 5% of your losses from the prior week to your account. However, it's worth noting that the overall amount spent should be more than NZD 750. The best thing is that the cashback is calculated automatically. Next up is the Alien Threat Tournament which is part of the Black Friday celebration. A call to arms for all New Zealand players to fight the aliens in this Alien Threat tournament. The prize pool for this tourney is around NZD 4,500 and the minimum bet is NZD 1.50.
What Online Pokies Does Mr Bet Have?s
Another fantastic element of Mr Bet Casino is the variety of online casino games and game providers available, which distinguishes it from the competition. The greatest news is that they have a wide range of games for everyone. Video pokies, a variety of table games (blackjack, roulette, baccarat, poker, and video poker), as well as scratch cards are all available. Players are guaranteed to have fun and excitement playing their favourite games when they play at Mr Bet casino due to the fact that they are partnered with the top game providers in the business. You may recognize some of these names: Play'n Go, Pragmatic Play, Playtech, Microgaming, Quickspin, Evolution and Yggdrasil to name a few. Whatever you have in mind, Mr Bet has it.
How Good is Mr Bet Security?
Online casinos are increasingly concerned with privacy. Whether or not a casino is reputable, privacy breaches are inevitable. Mr Bet Casino, on the other hand, offers one of the most well-designed gaming sites. However, they encrypt all of the data you provide them to ensure that your information is safe. They never keep sensitive stuff like your credit card number or email address. They understand the need for privacy and security, which is why they've made it their top priority to provide a secure environment for all players. All deposited funds are kept in a trust account until they're withdrawn or exchanged into a currency that's acceptable on the casino site. All transactions are encrypted with SSL (secure socket layer) and funnelled through the community. All of this protects you from people stealing your information.
What Payment Methods Can I Use With Mr Bet?
At Mr Bet, you may wager on a variety of games using a variety of deposit options. Wire transfers, debit cards, credit cards, and e-wallets are accepted. Here are the accepted payment methods for New Zealand: VISA, Paysafecard, Skrill, ecoPayz, Neteller and even cryptocurrencies like Bitcoin, Ethereum and Litecoin. Aside from Paysafecard, you may withdraw your winnings from any of the aforementioned methods above. They have a user-friendly website, making it simple to make withdrawals.
What Is The Customer Support Like At Mr Bet?
Customer support at Mr Bet Casino is available via a variety of methods. You may contact them by email, call their toll-free number, or send them a message through live chat. Their live chat support can be reached by either clicking the online help link or by going to the Contact Us section at the bottom of the page. If you prefer via email, just send them your concerns at [email protected] For our New Zealand folks with urgent concerns and, they can speak to a customer care team directly, here's their toll-free number +64-80-000-4650. The best part of all this is that their support is available 24 hours a day, 7 days a week.
Mr Bet is a New Zealand online casino with an enticing welcome bonus, promotions and bonuses for various occasions. They are partnered with the top game providers in the business to offer players video pokies, table games (blackjack, roulette, baccarat), scratch cards as well as other casino games like poker and even cryptocurrencies. Mr Bet offers one of the most well-designed gaming sites that's secure by encrypting all data you provide them. You may deposit using wire transfers, debit or credit cards or e-wallets which can be used to withdraw winnings from any of these methods too! Customer support is available 24/7 a week via email, live chat and phone support. All of this is provided at Mr Bet casino which makes it easy to have fun, excitement and winnings! Have a go with them to get the time of your life!
Launched Date
Currencies Allowed
NZD, CAD, BRL, CLP, MXN, NOK, PEN
Withdrawal Method
MasterCard, Neteller, Visa, EcoPayz, Skrill, Paysafe Card, Transferencia Bancaria Local, iDebit, AstroPay Card, Diners Club International, Discover, Interac, Sepa, Pay4Fun, Litecoin, Rapid Transfer
Withdrawal Limit
NZ$30,000 per month
Curacao - 1668/JAZ
Languages Available
English, Finnish, Norwegian, French, Portuguese, Spanish, Russian, Japanese, Suomi
New Zealand, Argentina, Austria, Brazil, Canada, Chile, Colombia, Croatia, Finland, Germany, Iceland, India, Latvia, Luxembourg, Macedonia, Malta, Mexico, Monaco, Norway, Paraguay, Peru, Romania, San Marino, Uruguay
Casumo Casino Review (NZ)
Casumo Casino Review Introduction In this Casumo Casino review, we see that their design bears more than a passing resemblance... Read More
Betway Casino Review (NZ)
Betway Casino Review Introduction In this Betway Casino review, we can see that they have an extensive range of games... Read More
32Red Casino Review (NZ)
32Red Casino Review Introduction In this 32Red Casino review, we see that 32Red has a great first deposit bonus of... Read More

redpajama_set_name: RedPajamaCommonCrawl


Increased body mass index (BMI) is associated with an increased risk of multiple cancers based on observational epidemiological data. Higher BMIs are also closely associated with increased levels of endogenous insulin, insulin-like growth factors, inflammatory cytokines, and other factors that can have downstream pro-cancer growth effects.
Handelsman Y, Leroith D, Bloomgarden ZT, et al. Diabetes and cancer—an AACE/ACE consensus statement. Endocr Pract. 2013;19:675-693.
Giovannucci E, Harlan DM, Archer MC, et al. Diabetes and cancer: a consensus report. Diabetes Care. 2010;33:1674-1685.


text: Q: How to design the leaf node if I want to fix the node size with variable length kv pair in B tree How to design the leaf node if I want to fix the node size with variable length kv pair in B+ tree? Say if I want to fix the leaf node size with 4KB and insert as much kv pairs as it can hold. I found that many of implementation of the B+ tree are using array and fix the size of array, but It can not be work for the variable length key or value. I wonder if it could be possible by any chance?

redpajama_set_name: RedPajamaStackExchange


While cover art only comes once an album, a gig poster is a way that a band can visually explore their music every time they play a show. And there really is a lot to be said about a great gig poster. If they connect with music lovers, they can fill seats and even become cult artwork for college dorms everywhere.
Still it can be hard to translate music into images, so we're going to break down this art form into six keys that might help you get started. To make things simpler, we're going to do this on mobile devices and skip the crayons and paint tubes, because we know that we can get equally awesome results. The PicsArt photo editor fits the bill for the task here, because this app has a deep and flexible editor as well as a Drawing Tool, which is a great combo for any artistic undertaking. Here are six key tips to cracking the art of the gig poster.
Every band has a personality, so build a unifying theme in every layer. Double Exposures are a great way to combine images and backgrounds, and effects can help define the energy of each element, whether it's a vintagestyle filter or a more transformative effect that makes an image look handpainted with watercolors. PicsArt has an extensive effects library, and lets you control the blending mode of each photo or piece of clipart that you add in.
More than in any other project, the font that you use here is super, incredibly, outrageously important! Your band name is your flag, and your font is almost more important than the name. A lot of names could work for different styles of music, but fonts speak to specific feelings. Hippy bubble letters, angry slashing letters, cool minimalism… do not underthink your font.
One of the best ways to link your artwork to the vibe of the band is to simply make sure that you are listening to the music while you create. Creating something independently of the music is fine, but it's much easier to feel your way into your project if you can hear the thudding drums and jangling guitars. That energy will make its way into the final results.
A gig poster is an advert, and that means that you want to grab attention from the distractions of the street. Nothing jumps out at us faster than the color itself, so start with strong colors; neon is near impossible to ignore for example. Beyond that, a strong image is key. You don't need complexity or detail, a hard hitting bonkers idea will do. Who can walk by a bulldog's jowls dangling over a collared shirt? Cutting and pasting from photos in the app makes this as easy as cake.
Another basic element to any poster's feeling is something that underlies the entire poster itself — the texture! You can actually play with various types of textures without buying sheets of expensive paper. PicsArt has texture masks that will give your poster the tactile look of everything from newspaper to a brick wall.
The final test of any poster is the simplest, but it's critical that you keep this in mind. Ask yourself, "would I hang this poster on my wall?" Well… wouldyou? If it doesn't belong on your wall, then you're not connecting to it and that means other people probably won't either. Your poster should look so good that even people who have never heard the band will want it hanging over their beds.
Are feeling the groove yet? Sink yourself into a gig poster project and see if you can strike a winning chord. Download the PicsArt app for free on Android, iOS or Windows Phone to give it a go!


The 82-year-old exposed himself in a nutty, blame-everybody speech Sunday in the West Bank city of Ramallah. His intent was to bury President Trump for declaring that America recognizes Jerusalem as Israel's capital and plans to move our embassy there, but instead, Abbas made the president look wiser than Solomon.
Trump's action, which broke with past presidents who gave Palestinians a heckler's veto over our policies, ripped the mask off Abbas' seeming moderation to reveal the same old Jew hatred that has been the cause of war after war between Israel and its neighbors.
Equally important, the speech, which rambled on for two-and-a-half hours, confirmed what Israel has been saying for years: Abbas never would have made a lasting peace with the Jewish state.
And without that peace, a Palestinian state will remain a pipe dream.
Abbas' outlandish claims included a demand that Britain apologize for the 1917 Balfour Declaration, which endorsed the idea of a Jewish homeland, and his charge that Israel drugs Palestinian children.
Incredibly, he even insisted that European Jews chose to remain there during the Holocaust.
He neglected to mention the reason why: During the war years, British authorities blocked most Jews from going to Palestine following demands from Arab leaders.
After the war, Abbas continued, Europeans wanted to get rid of their surviving Jews, so deported them to Palestine.
Those conspiracy theories echo Abbas' long-ago Ph.D. thesis from a Russian university that the Holocaust was overstated and that Jews had no historic claims to Palestine. Apparently, facts and time haven't educated him.
The vitriol was extraordinary, even by Mideast standards. Abbas leveled a personal curse on Trump's house, meaning his descendants, and spewed invective at fellow Arabs, especially Saudi Arabia, telling them to "go to hell" for criticizing him.
One possible explanation is that, after Trump's declaration about Jerusalem, Abbas felt trapped. He is not popular at home, and even his efforts to spark demonstrations over Jerusalem fell flat.
It's also likely that Abbas was rebuffed by other countries that were critical of Trump's move, including some in Europe and the Mideast. They probably told Abbas that he had no alternative except to deal with the US if he had any hopes of brokering a deal with Israel so that part of Jerusalem could also be a Palestinian capital.
Instead of following that conciliatory path that might have yielded real progress, Abbas lashed out, declaring the end of negotiations, promised to keep paying terrorists' families and talked of canceling existing agreements with Israel.
His condemnations, which included attacks on America's UN Ambassador Nikki Haley and David Friedman, the American ambassador to Israel, almost guarantee that Trump will follow through with plans to slash American aid to the UN agency that ministers to Palestinians, perhaps by half.
And Abbas likely sidelined himself from any dealings with the Trump administration, which has promised to propose a settlement plan. But with no obvious replacement and no timetable for elections, the upshot is that Abbas' blaze-of-glory act will do nothing for Palestinians except further delay their hopes of getting a state.
This sorry moment is no victory for anyone — except the truth. Rarely has there been such clarity that Israel has no partner for peace.
As such, the possibility of more terrorist violence cannot be minimized. Just yesterday, Israeli authorities removed a cellphone bomb from Joseph's Tomb in Nablus in the West Bank.
It was detonated by security forces not long before 1,000 Jewish worshippers were scheduled to come to the spot considered to be the burial site of the biblical patriarch.
Oddly, Abbas condemned the attempted slaughter. But after his fiery remarks, what did he expect?
Now that Dr. Ronny Jackson has declared President Trump to be in "excellent" health and said the president scored 30 out of 30 on a cognitive test, look for the left to demand an investigation — of the doctor.
After all, he might have Russian connections.
In other words, the next generation of politicians is being made in factories.
The MTA just shot a giant hole in its credibility as well as its ban on political advertising. And it's impossible not to conclude that it acted to curry favor with Gov. Andrew Cuomo.
Cuomo holds the agency's purse strings, controls the board of directors and appointed its chairman, Joe Lhota.
All of which is relevant to The Post's report that ads showing Cuomo with smiling children saved by organ donations and another touting the state's family leave policy have popped up across the subway system just as the governor enters his re-election campaign.
Asked to comment, the MTA referred a reporter to Cuomo's office, which said an ethics ruling found the governor could appear in public service ads until 90 days before an election.
The organ donor ads were sponsored by health care unions and hospital lobbyists that endorsed Cuomo in the past, while the MTA donated space for the family leave ads.
Both are shameful end runs around the agency's policy and the governor ought to pull the plug. With no real opponent, healthy approval numbers and a whopping $31 million war chest, he looks politically strong.
Asking that the ads be removed would be a sign of true strength.


Become a Member of the Congo Square Preservation Society Today!
* Make a positive contribution to Congo Square's legacy and future.
* Discounts on Congo Square Merchandise & Events.
* Invitation to special members' only events.
* Ability to serve on committees.
* Artist performance opportunities & assistance.
You may pay for your Membership below. Once you have made your payment, you will be re-directed to this site to complete your short application process!
If you are unable to pay online or complete the application, please email us at congosquarepreservationsociety@gmail.com to arrange application in person!


Espresso always tastes better in the right cup. If you're a casual espresso drinker or have a passion for espresso why not have something unique, sophisticated and handmade in the U.S. that you can show with pride.
Each cup is handmade using a hand thrown process and individually hand painted so no one cup will be identical to the next.
A good espresso cup is made with some thickness to hold the warmth of the drink. Most machine made espresso cups hold only 2 ounces. These cups designed with a 2 inch wide mouth, a thick handle positioned in the middle of the side of the cup for easy grasping, and hold 4 ounces of your favorite espresso.
Why not start your morning with a sip of your very own espresso drink in a specially designed cup to call your own. Try 2 for a morning drink to share with someone special or impress your guests with a set of 4.
Vineyard Green hand painted color will accent any kitchen. They are microwaveable and dishwasher safe.


My professor made some amendments and I had to redo the whole project. But I have a part time job as well, so I had no time to finish my paper at all. So I found this website and asked for editing. Not only did they make it perfect, but the prices are very reasonable. It was such a relief for my wallet!essay writer. To quizlets with the paper instructions . The writer will follow the guidelines you input in the box below.


text:
Clutching at genetic straws for impersonal treatment
Jeremy Laurance in The Independent says that a study led by Hugh Gurling has opened up the prospect of so-called personalised treatment of bipolar disorder with drugs targeting the metabotropic glutamate receptor 3 (mGluR3). This is based on a finding that the Kozak sequence variant of the glutamate receptor 3 (GRM3) gene, which encodes for mGluR3, was overrepresented in a sample of bipolar disorder cases compared with controls. As the paper concludes, confirmation of this finding is needed before accepting this potential marker. It could just be a chance finding based on screening until a significant result is found.
As the paper also points out, "The GRM3 gene has been investigated in bipolar affective disorder as part of several genome-wide association studies (GWASs) but failed to reach genome-wide significance in any of these investigations." Still research goes on with this gene because it is assumed the failure to find genetic association is "probably the result of the presence of low-frequency disease alleles and the high degree of etiologic genetic heterogeneity". Actually it's more likely that there's no genetic link.
I haven't forgotten Hugh Gurling's false claim in Nature in 1988 that he'd found strong evidence for the involvement of a single gene on chromosome 5 in the causation of schizophrenia. Jeremy Laurance shouldn't be so easily taken in by claims for so-called personalised (actually there's nothing personal about it in the sense of relating to patients) psychiatry.
Clutching at genetic straws for impersonal treatme...


Crest of the Anishinaabe people.
Anishinaabe or Anishinabe—or more properly Anishinaabeg or Anishinabek, which is the plural form of the word—is the autonym often used by the Odawa, Ojibwe, and Algonkin peoples, who all speak closely related Anishinaabemowin/Anishinaabe languages.
The meaning of Anishnaabeg is "First-" or "Original-Peoples". Another definition refers to "the good humans", or good people, meaning those who are on the right road/path given to them by the Creator or Gitchi-manitou (Great Spirit).
Not all Anishinaabemowin speakers, however, call themselves Anishinaabeg. The Ojibwa people who moved to what are now the prairie provinces of Canada call themselves Nakawē(-k) and their branch of the Anishinaabe language as Nakawēmowin. (The French ethnonym for the group was the Saulteaux). Particular Anishinaabeg groups have different names from region to region.
There are many variant spellings of the Anishinaabe name, depending on the transcription scheme and also on whether the name is singular or plural. Therefore, different spelling systems may indicate vowel length or spell certain consonants differently (Anishinabe, Anicinape); meanwhile, variants ending in -eg/ek (Anishinaabeg, Anishinabek) come from an Algonquian plural, while those ending in an -e come from an Algonquian singular.
The name Anishinaabe is realised as Nishnaabe, in some parts of North America, most prominently among the Odawa. The cognate word Neshnabé comes from the Potawatomi, a people long allied with the Odawa and Ojibwe in the Council of Three Fires. Identified as Anishinaabe, but not part of the Council of Three Fires, are the Nipissing, Mississauga and Algonkin. The Algonkin may use the word Omàmiwinini to distinguish themselves from other Anishinaabe.
According to Anishinaabeg tradition, and from records of wiigwaasabak (birch bark scrolls), the people came from the eastern areas of North America, and from along the East Coast. In myth, the homeland was called Turtle Island.
Oral traditions among the Anishinaabeg tell a variety of creation stories. According to the oral history, seven great miigis (radiant/iridescent beings in human form) appeared to the Anishinaabe peoples in the Waabanakiing (Land of the Dawn, i.e. Eastern Land) to teach the people about the midewiwin life-style. One great miigis was too spiritually powerful and would kill people in the Waabanakiing whenever they were in its presence. This being later returned to the depths of the ocean, leaving the six great miigis to teach the people.
Each of the six miigis established separate doodem (clans) for the people. Of these doodem, five clan systems appeared: i) Awaazisii (Bullhead), ii) Baswenaazhi (Echo-maker, i.e., Crane), iii) Aan'aawenh (Pintail Duck), iv) Nooke (Tender, i.e., Bear), and v) Moozoonii (Little Moose). Later a sixth was added. vi) Waabizheshi (Marten).
The powerful miigis returned in a vision relating a prophecy to the people. It said that the Anishinaabeg needed to move west to keep their traditional ways alive, because of the many new settlements and people not of Anishinaabe blood who would soon arrive. The migration path of the Anishinaabe peoples would become a series of smaller Turtle Islands, confirmed by the miigis shells (i.e., cowry shells). After receiving assurance from the their "Allied Brothers" (i.e., Mi'kmaq) and "Father" (i.e., Abnaki) of their safety in crossing other tribal territory, the Anishinaabeg moved inland. They advanced along the St. Lawrence River to the Ottawa River and through to Lake Nipissing, and then to the Great Lakes.
The first of these smaller Turtle Islands was Mooniyaa, where Mooniyaang (Montreal, Quebec) now stands. Here the Anishinaabeg divided into two groups: one who travelled up and settled along the Ottawa River, and the core group who proceeded to the "second stopping place" near Niagara Falls.
By the time the Anishinaabeg established their "third stopping place" near the present city of Detroit, the Anishinaabeg had divided into six distinct nations: Algonquin, Nipissing, Mississauga, Ojibwe, Odawa and Potawatomi. While the Odawa established their long-held cultural centre on Manitoulin Island, the Ojibweestablished their centre in the Sault Ste. Marie region of Ontario, Canada. With expansion of trade with the French and later the British, fostered by availability of European small arms, members of the Council of Three Fires expanded southward to the Ohio River, southwestward along the Illinois River, and westward along Lake Superior, Lake of the Woods and the northern Great Plains. In their western expansion, the Ojibwa again divided, forming the Saulteaux, the seventh major branch of the Anishinaabeg: .
As the Anishinaabeg moved inland, through both alliances and conquest, they incorporated various other closely related Algonquian peoples into the Anishinaabe Nation. These included, but were not limited to, the Noquet (originally part of the Menomini Tribe) and Mandwe (originally part of the Fox). Other incorporated groups can generally be identified by the individual's Doodem (Clan). Migizi-doodem (Bald Eagle Clan) generally identifies those whose ancestors were Americans and Ma'iingan-doodem (Wolf Clan) as Santee Sioux. Other Anishinaabe doodem migrated out of the core Anishinaabeg groupings, such as the Nibiinaabe-doodem (Merman Clan), which is now the "Water-spirit Clan" of the Winnebagos. Anishinaabe peoples now reside throughout North America in both the northern United States and southern Canada, chiefly around the Great Lakes and Lake Winnipeg.
After this migration, and the immigration of European newcomers to North America, many Anishinaabeg groups later entered into treaties with the governments of the Dominion of Canada and the United States. Treaty 3 (of the Numbered Treaties) in Canada was signed in 1873 between the Anishinaabe (Ojibwa) people west of the Great Lakes and the government of Canada . Through other treaties and resulting relocations, some Anishinaabeg now reside in the states of Kansas, Oklahoma, South Dakota, and Montana in the United States, and the provinces of Saskatchewan, Alberta and British Columbia in Canada.
The first of the Anishinaabeg to encounter European settlers were those of the Three Fires Confederation, within the states of Illinois, Indiana, Michigan, Ohio, and Pennsylvania in the territory of the present-day United States, and southern Ontario and Quebec of Canada Although there were many peaceful interactions between the Anishinaabeg and the European settlers, there were also times of turmoil and war. Warfare cost many lives on both sides.
The Anishinaabe dealt with Europeans through the fur trade, intermarriage, and performance as allies. Europeans traded with the Anishinaabe for their furs in exchange for goods, and also hired the men as guides throughout the lands of North America. The Anishinaabeg (as well as other Aboriginal groups) began to intermarry with fur traders and trappers. Their descendants would later create the Métis ethnic group. Fur traders were generally capitalists with significant backing. They tended to marry daughters of chiefs, with both sides forming high-status alliances. The explorers and trappers married or had unions with other Anishinaabeg women, and their descendants formed the Métis.
The earliest Europeans to encounter native peoples in the Great Lakes area were the French voyageurs. They were mainly trappers rather than settlers. Such explorers gave French names to many places in present-day Minnesota and Wisconsin.
The ethnic identities of the Ojibwa, Ottawa, and Potawatomi did not develop until after the Anishinaabeg reached Michilimackinac on their journey westward from the Atlantic coast. Using the Midewiwin scrolls, Potawatomi elder Shup-Shewana dated the formation of the Council of Three Fires to 796 AD at Michilimackinac.
In this Council, the Ojibwe were addressed as the "Older Brother," the Odawa as the "Middle Brother," and the Potawatomi as the "Younger Brother." Consequently, when the three Anishinaabe nations are mentioned in this specific order: Ojibwe, Odawa, and Potawatomi, it is an indicator implying Council of Three Fires as well. Each tribe had different functions: the Ojibwa were the "keepers of the faith," the Odawa the "keepers of trade," and the Potawatomi are the "keepers/maintainers of/for the fire" (boodawaadam). This was the basis for their exonyms of Boodewaadamii (Ojibwe spelling) or Bodéwadmi (Potawatomi spelling).
Though the Three Fires had several meeting places, they preferred Michilimackinac due to its central location. The Council met for military and political purposes. The Council maintained relations with fellow Anishinaabeg nations: the Ozaagii (Sac), Odagaamii (Meskwaki), Omanoominii (Menominee), Wiinibiigoo (Ho-Chunk), Naadawe (Iroquois Confederacy), Nii'inaawi-Naadawe (Wyandot), Naadawensiw (Sioux), Wemitigoozhi (France), Zhaaganaashi (England) and the Gichi-mookomaan (the United States). After the Europeans came into the country, the French built Fort Michilimackinac in the 18th century. After the Seven Years War, the victorious English took over the fort, also using it as a trading post.
Through the totem-system (a totem is any entity which watches over or assists a group of people, such as a family, clan or tribe .) and promotion of trade, the Council generally had a peaceful existence with its neighbours. However, occasional unresolved disputes erupted into wars. The Council notably fought against the Iroquois Confederacy and the Sioux. During the Seven Years' War, the Council fought against England.
The Anishinaabeg established a relationship with the British similar to that they had with the French. They formed the Three Fires Confederation in reaction to conflict with encroaching settlers and continuing tensions with the British Canadian government, as well as that of the new United States.
William Warren, a politician of part-Anishinaabe descent, published work in 1885 that identified the Crane and Loon clans as the two Chief clans. Crane Clan was responsible for external governmental relationships, and Loon Clan was responsible for internal governance relationships. Warren stated his belief that the British and Americans had intentionally destroyed the clan system, or the polity of governance. Further, he believed such destruction led to many wars among the Anishinaabe. He also cited the experiences of other Native Nations in the U.S. (such as the Creek, Fox and others). His work in its entirety dealt with the significance of the clan system.
The Three Fires Confederacy had conflict with the new United States after the American Revolution, as settlers kept encroaching on their territory. The Council became the core member of the Western Lakes Confederacy (also known as "Great Lakes Confederacy"), joining together with the Wyandots, Algonquins, Nipissing, Sacs, Meskwaki and others.
Those who remained in the east were subjected to the 1830 Indian Removal policy of the United States; among the Anishinaabeg, the Potawatomi were most affected. The Odawa had been removed from the settlers' paths, so only a handful of communities experienced removal. For the Ojibwa, removal attempts culminated in the Sandy Lake Tragedy and several hundred deaths. The Potawatomi avoided removal only by escaping into Ojibwa-held areas and hiding from US officials.
Population estimates indicate that the Anishinaabeg population in the United States is more numerous than that of Canada, but census reports are criticized as being inaccurate.
The Canadian Anishinaabeg are descended from the northern Lake Superior Anishinaabeg, whose original homeland was probably in the vicinity of the eastern upper peninsula of Michigan. They separated, with one group going down into Michigan, Illinois, Indiana, Ohio, southern Ontario and Pennsylvania, while another group migrated straight westward. The ancestors of the Canadian Anishinaabeg moved to the north, and then to the west. They migrated to eastern British Columbia in the 19th century.
Scholars of the Anishinaabeg will eventually learn if all Anishinaabeg are descended from those Anishinaabeg of the eastern upper peninsula of Michigan, or if they are descended from the Algonquin Anishinaabeg of Quebec. The people's history points to the upper peninsula of Michigan as their land of formation.
The Anishinaabeg of Canada have managed to withstand the efforts of the European settlers and hold onto their languages. An estimated 50,000 Canadian Anishinaabeg speak their native tongue. From Quebec to the eastern lands of British Columbia, the Anishinaabeg reserves are, for the most part, smaller in size than those in the US, a factor which may have helped them preserve the languages.
There are many Anishinaabeg reserves and reservations; in some places the Anishinaabeg share some of their lands with others, such as the Cree,the Dakota, Delaware, and the Kickapoo, among others. The Anishnabek who "merged" with the Kickapoo tribe may now identify as being Kickapoo in Kansas and Oklahoma. The Prairie Potawatomi were the Ojibwe, Odawa and Potawatomi of Illinois and Wisconsin who were relocated to Kansas during the 19th century.
The relationships between the various Anishinaabe communities with the United States government have been steadily improving since the passage of the Indian Reorganization Act. Several Anishinaabe communities still experience tensions with the state governments, county governments and with non-Native American individuals and their groups.
social disparity: many Anishinaabeg suffer poor education, high unemployment, substance abuse/addiction and domestic violence at rates higher than the general population.
A fictional Anishinaabe clan in Ontario, the Mtigwaki, were featured in the comic strip For Better or For Worse from 2005-2006.
^ Alexander Morris, The Treaties of Canada with the Indians, Belfords , Clarke & Co., Toronto (1880).
Bento-Banai, Edward (2004). Creation- From the Ojibwa, The Mishomis Book.
Warren, William W. History of the Ojibway People. Borealis Books (St. Paul, MN: 1984).
White, Richard (July 31, 2000). "Chippewas of the Sault", The Sault Tribe News.
"The Anishinaabe-Ojibwe people of the Great Lakes", Ojibwe Waasa-Inaabidaa, (US-focused), Public Broadcasting Service/PBS.
Anishinaabeg means "first people" or "original people".


Today I just have a brief post to unveil an exciting and big new project for my blog: the Vintage Knitting College!
I was chewing on the idea for the Land Girl sweater knit-along after I proposed it ever a couple of weeks ago. There was some interest, but not as a much as I was hoping for, with at least one or two people interested in the idea of the KAL but wanting to do a different pattern. That got me to thinking that it might be better to do something more like a series of posts and tutorials about vintage knitting and colorwork. Out of that (plus my obsession with campus-inspired fashion this Fall) hatched the idea for the Vintage Knitting College!
Basically, it will be a long-term project of instructional posts about knitting, geared towards knitters who knit from vintage patterns. I'm not going to set any specific deadlines, but figure I'll continue to periodically post tutorials and informational posts for the Vintage Knitting College over the next year or longer (possibly including fun things like guest posts and giveaways now and again, too!).
Because my Fall-focused mind is on colorwork knitting right now (fair isle and other stranded projects, as well as intarsia picture knitting), and because it did seem like several of you liked the idea of a fair isle knit-along, I'm going to focus first on vintage colorwork patterns.
In a series of posts over the next several months in a "class" called Colorwork: 101, I'll cover techniques that can be applied to vintage patterns that involve colorwork. Such as converting patterns to knit in the round, steeking, and charting colorwork when the pattern doesn't include charts (many vintage patterns don't include charts for where you change colors, which slows you down like molasses). I'll also probably wax poetic now and again on picking colors with a vintage look in mind, yarn suggestions, and more. And while it looks like from my opening logo that I'll be focusing only on sweaters, I won't! Expect accessories to crop up now and again, too.
Many of the posts on a given topic will follow through the life of a project. So even though it won't technically be a knit-along, I'll let you know ahead of time what I plan on knitting and instructing on, so if you'd like to follow along with that or a similar project, you are more than welcome to. And depending on interest level, I may start a general Vintage Knitting College Flickr group for people to post inspired-by creations to! I'm hoping over time for this to become a nice resource for vintage knitting, and will probably devote a separate page on the blog just for the knitting college, once it gets rolling.
Phew, I've finally shared my little (well, big!) idea. Do you like the idea of the Vintage Knitting College as an ongoing knitting resource? Let me know via comment or email if you have any suggestions (for colorwork or other knitting). I'd love to hear from you.
This idea is brilliant. I can't wait.
Hi Tasha! this sounds great!
I just started my 'Datemaker' from the pattern you posted earlier. My first knit from a vintage pattern.
I'm into stranded fair isle knitting also at the moment. It is hanging in the air this autumn!
Sounds like a fun adventure to me 🙂 And definitely a worthwhile endeavour! I still say I want to do a fair isle, so I shall wait and see what you have coming up the pipe and get ready to have fun!
I think that's a great idea, Tasha – thanks! I'm a pretty poor knitter, so am reluctant to join KALs, but something I could look at, get inspired by and return to when my skills improve would be really valuable.
Wicked idea. I love it! There is certainly an audience for it!
These are some great thoughts. Being relatively new to knitting and vintage knitting, I will really appreciate these upcoming posts. I still do like the idea of another KAL hosted by you. Perhaps you could try having several patterns and having a vote again. I think that because knitting is such a process, you have to fall in love with a pattern, or else it's too difficult to commit to. But, I can also see that you have your heart set on LandGirl or something similar, so it is important for you to love the pattern of a KAL as well.
this is a wonderful idea! i could do with all of the help i can get with both vintage patterns and colourwork.
I'm really looking forward to this! Thanks!
This idea is just what i need, being just a beginner, I need to familiarize myself with many other things, i know for a fact that i will learn so much from you.
Fabulous!!! I loved all your informative KAL posts and I'm desperately ignorant of all but the very very basics of colorwork. Looking forward to your classes!
This is such a great idea, Tasha! I couldn't (unfortunately… boo-hoo!) join in on the knit-along idea (as I think I said in that post), so I'm excited you'll be still doing instructionals on vintage knitting–just more in general now. I'm so looking forward to your series on colorwork!
I love it! Can't wait for your first, real post.
Oh I like this a lot. I don't have the patience to make a whole sweater, but I have been wanting to try to learn fair isle, intarsia, etc. and hadn't found a good resource. Hooray!