I slice the carrots with a mandoline slicer similar to this with a medium knife attachment. Then I grind the coriander seed (I used to use already ground one, but it's not as aromatic), salt it, crush garlic and add lemon. Then I heat up the oil and pour it over, mix let it stand overnight.
Mix all together except the fritos. Refrigerate and just before serving, add the Fritos.
Kraft Cucumber Ranch dressing, add just enough to barely coat the salad ingredients.
I added chunks of apple and pineapple tidbits right before serving.
1. Spread the spinach out on a serving plate. Place the slices of tomato on the bed of spinach. Top each slice of tomato with a slice of fresh mozzarella. Sprinkle the basil over the salad and drizzle with olive oil and balsamic vinegar.


text:
Risque JRPG 'Blossom Bosoms' Blocked From Western Release
November 30, 2018 - News - Tagged: Video Games
The upcoming game release of Blossom Bosoms was canceled for its launch in North America following trouble with getting the title approved. The title launched in Japan earlier this year…
Could Announcement that 'Neon Genesis Evangelion' is Coming to Netflix be an Evangelion Reference?
November 28, 2018 - News - Tagged: Evangelion, Neon Genesis Evangelion
Earlier this week, Netflix announced several new anime that will be added to their streaming library in 2019, including the 1996 seminal classic Neon Genesis Evangelion. Keen-eyed observers were quick to…
Local Man Dies of Heart Attack of the Cards
November 26, 2018 - News - Tagged: Games, Local, Yu-Gi-Oh!
ROSEMONT, IL — Tensions were high at last night's Yu-Gi-Oh! Championship Series tournament as over 1300 men and three women competed for a chance to participate in next year's national…
Area Man Struggles to Find Acceptable Conversation Topic at Family Dinner Table
November 22, 2018 - News - Tagged: family, Local, thanksgiving
MILFORD, CT – A Connecticut man found himself unable to find a suitable topic of conversation at the family dinner table earlier this evening. Reports indicate that Andrew Klein, 29,…
'Goblin Slayer' to Feature More Shots of Birds in Order to Seem More "Artistic", Less "Edgy"
November 21, 2018 - News - Tagged: Anime, Goblin Slayer, Industry
Latest episode of Goblin Slayer confused, and amused, audiences with its closing scene — a near 2-minute close-up of a bird's eye, followed by the credit roll. Though this sparked speculation about…
Studio Pierrot Refuses to Release New 'Tokyo Ghoul: Re' Blu-rays to "Filthy Anime-Onlys"
November 19, 2018 - News - Tagged: Industry, Tokyo Ghoul
Controversy emerged today in Mitaka, Tokyo as Studio Pierrot has officially declared that Blu-ray discs for all future seasons of its once-popular anime Tokyo Ghoul: re will only be made…
PETA Initiates 'Goblin Lives Matter' Movement
November 15, 2018 - News - Tagged: Goblin Slayer
People for the Ethical Treatment of Animals (PETA), an international animal rights group, recently launched its 'Goblin Lives Matter' campaign, an activist movement opposing the indiscriminate killing of goblins. The…
Grave Robbing Rates Skyrocket Following Popularity of 'Zombieland Saga'
November 13, 2018 - News - Tagged: Zombieland Saga
The Japanese local police have reported this week an alarming spike of recorded crimes against corpses over the last month and a half, which many are chalking up to the…
Local Man Learns Hiragana, Adds "Nihongo de ok" to Twitter Bio
November 8, 2018 - News - Tagged: Japan, Local, Twitter
ATHENS, GA — After declaring that he has mastered hiragana, Lance Williams, 19, has accordingly updated his twitter profile by appending "にほんごでOK". In doing this Williams hopes to inform his…
Kurisu Makise Leads Initial Ballot Count in Minnesota Congressional Race
November 6, 2018 - News - Tagged: Election, Politics, Steins;gate
In a shocking upset, preliminary ballot counts from Minnesota's hotly contested 1st congressional district show write-in candidate Kurisu Makise leading by a small margin. With incumbent Tim Walz (D) not…
« Previous 1 … 6 7 8 9 10 11 12 13 14 15 16 … 106 Next »

redpajama_set_name: RedPajamaCommonCrawl


Not that we really needed TVB to tell us that Natalie had feelings for Kevin, but this scene was cute XD When Natalie overheard Kevin mention he is already married she becomes upset. When she realised he was joking, she couldn't help but show a sweet smile on her face.
The above scene was pretty sad and touching. Love the bond the two of them share!
hehehe, oh wow, more Fistful of Stances moments ^^ What happened in the last shot?
That was the hospital scene after Natalie attempted to hang herself after getting rapped by the bald guy…the scene before this was pretty sad too. Natalie went to each of her family member's rooms that night to bid farewell to them 🙁 Luckily Kenneth and Kevin both felt the need to visit Natalie that night and saved her just in time.
I agree, Fistful of Stances is definitely the best series of 2010 so far.
I didn't like how Kevin's character was constantly screaming and yelling. I do think that he was over shone by Kenneth who did such a wonderful job portraying his character.
I was hoping this would be Selena's breakthrough role, but her acting is still quite awkward to me, her crying scenes are great though. I love her and have been rooting for her this Just Love but she definitely still has room for improvement.
I think Selena really needs to work harder for awards, since at this stage I believe her acting is still petty medicore..
Yups, it's nice to see Natalie with a different type of role, she usually gets those stubborn, young girl sort of role. But in Fistful, she was caring, soft and such. It's a nice change, and she did a great job.


text:
News > Medscape Medical News
Hypomethylating Agents in Children With Relapsed/Refractory AML
Neil Osterweil
Hypomethylating agents are generally considered to be agents of choice for older adults with acute myeloid leukemia who cannot tolerate the rigors of more intensive therapies, but HMAs also can serve as a bridge to transplant for children and young adults with relapsed or refractory acute myeloid leukemia.
That's according to Himalee S. Sabnis, MD, MSc and colleagues at Emory University and the Aflac Cancer and Blood Disorders Center at Children's Healthcare of Atlanta.
In a scientific poster presented during the annual meeting of the American Society of Pediatric Hematology/Oncology, the investigators reported results of a retrospective study of HMA use in patients with relapsed or refractory pediatric AML treated in their center.
Curative Intent and Palliation
They identified 25 patients (15 boys) with a median age of 8.3 years (range 1.4 to 21 years) with relapsed/refractory AML who received HMAs for curative intent prior to hematopoietic stem cell transplant (HSCT), palliation, or in combination with donor leukocyte infusion (DLI).
Of the 21 patients with relapsed disease, 16 were in first relapse and 5 were in second relapse or greater. Four of the patients had primary refractory disease. The cytogenetic and molecular features were KMT2A rearrangements in six patients, monosomy 7/deletion 7 q in four patients, 8;21 translocation in three patients, and FLT3-ITD mutations in four patients.
The patients received a median of 5.3 HMA cycles each. Of the 133 total HMA cycles, 87 were with azacitidine, and 46 were with decitabine.
HMAs were used as monotherapy in 62% of cycles, and in combination with other therapies in 38%. Of the combinations, 16 were with donor leukocyte infusion, and 9 were gemtuzumab ozogamicin (Mylotarg).
Of the 13 patients for whom HMAs were used as part of treatment plan with curative intent, 5 proceeded to HSCT, and 8 did not. Of the 5 patients, 1 died from transplant-related causes, and 4 were alive post transplant. Of the 8 patients who did not undergo transplant, 1 had chimeric antigen receptor T- cell (CAR T) therapy, and 7 experienced disease progression.
The mean duration of palliative care was 144 days, with patients receiving from one to nine cycles with an HMA, and no treatment interruptions due to toxicity.
Of 5 patients who received donor leukocyte infusions, 3 reached minimal residual disease negativity; all 3 of these patients had late relapses but remained long-term survivors, the investigators reported.
They concluded that "hypomethylating agents can be used effectively as a bridge to transplantation in relapsed and refractory AML with gemtuzumab ozogamicin being the most common agent for combination therapy. Palliation with HMAs is associated with low toxicity and high tolerability in relapsed/refractory AML. Use of HMAs with DLI can induce sustained remissions in some patients."
The authors propose prospective clinical trials using HMAs in the relapsed/refractory pediatric AML setting in combination with gemtuzumab ozogamicin, alternative targeted agents, and chemotherapy.
HMAs in Treatment-Related AML
Shilpa Shahani, MD, a pediatric oncologist and assistant clinical professor of pediatrics at City of Hope in Duarte, Calif., who was not involved in the study, has experience administering HMAs primarily in the adolescent and young adult population with AML.
"Azacitidine and decitabine are good for treatment-related leukemias," she said in an interview. "They can be used otherwise for people who have relapsed disease and are trying to navigate other options."
Although they are not standard first-line agents in younger patients, HMAs can play a useful role in therapy for relapsed or refractory disease, she said.
The authors and Shahani reported having no conflicts of interest to disclose.
This article originally appeared on MDedge.com, part of the Medscape Professional Network.
Cite this: Hypomethylating Agents in Children With Relapsed/Refractory AML - Medscape - Jun 25, 2021.
5 Things: Books I Can't Put Down Right Now
Health Costs Over 25 Times Higher for Patients With Hemophilia B
Medical Meetings May Be Forever Changed
Improved Survival in ALL Patients Who Relapse After Transplant
20022006661-overview
Diseases & Conditions Acute Lymphoblastic Leukemia Staging
Diseases & Conditions Acute Myeloid Leukemia (AML) Staging
2002197802-overview
Diseases & Conditions Acute Myeloid Leukemia (AML)
Pathology of Acute Leukemias of Ambiguous Lineage
Acute Lymphoblastic Leukemia Staging
Acute Myeloid Leukemia (AML) Staging
Pediatric Acute Myelocytic Leukemia
Acute Promyelocytic Leukemia
8 Oncologic Emergencies You Need to Know
Highlights in Acute Lymphoblastic Leukemia From ASH 2021
COVID-19 Vaccine Responses Vary Among Hematological Malignancies
Reduced-Intensity Transplant Benefits Older Patients With AML
Sex Differences in Pediatric B-ALL Outcomes Persist
Acute Leukemia News & Perspectives


text:
Did Trump Save Coal?
September 26, 2017 – Amstutz, Electricity, General, Mattingly, Natural Gas : Choice, Choice Energy Services, Coal, Natural Gas, Trump – Facebook Twitter LinkedIn
by Chris Amstutz / Matthew Mattingly
It was around this time last year that in the heat of the presidential campaign, we witnessed a full committal to the coal industry by Donald Trump. "A lot of people are going to be put back to work, a lot of coal miners are going back to work," was stated by Donald Trump, both as the presidential nominee and later as POTUS. Fast forward to today, less than a year after Trump being in office, and coal is back as the number one fuel source for electricity generation in the United States. So did Trump keep his promise? Here at Choice Energy Services we will take a break on the war on "fake news" and look at the facts behind coal's revival.
For decades the domestic abundance of coal made it a profitable fuel source for electricity generation, but with the innovation of hydraulic fracturing for natural gas and realities of Obama-Era regulations on carbon emissions, the profits disappeared. In 2016 many analysts were all ready to anoint natural gas as the new king of electricity generation in the United States. But natural gas' reign as the leading fuel source didn't last long. As of August 2017, Coal has reclaimed its' place as the top electricity generating fuel source, and has been projected by the EIA to remain as such for the rest of 2017 and through 2018.
So does this mean we're witnessing a Trump effect in real time!? To that we answer, no. It is true that some executive regulations have been halted and awareness for coal miners increased but the data has other explanations. To put it simply, it's all about the Benjamins. Utilities operate like the rest of us, ones that have the ability to switch from coal to natural gas will make their decision based on price. Just as a driver will always pick the gas station that is $0.10/gallon cheaper, utilities will make the conscious decision to generate electricity from the cheapest fuel source. For example, to start the summer the May 2017, Henry Hub natural gas spot prices was $3.22/MMBtu as compared to $2.25/MMBtu for Appalachian Coal. The result is that the EIA expects natural gas fired generation to fall from 34% of total generation in 2016 to 31% in 2017, while they expect coal fired generation to increase from 30% in 2016 to 32% in 2017. The percentage difference is being made up from 2017 decrease in demand for electricity and an increase in renewable generation.
While many see new life in the coal industry, the reality is that without drastic policy change, no amount of "Pittsburgh, not Paris" chants will divert the road to the grave of coal as an energy source. Carbon emission policies are certain to change with political parties, which make 30-40 year investments in coal fired power plants unlikely. A net loss of 8,992 MW of coal powered generation capacity is slated to come off the grid in the next 12 months, as opposed to a net addition of 20,966 MW of Natural Gas powered generation capacity. So while Trump has delayed the death of coal; the safer bet is on natural gas and renewables for generation growth in the United States.


A man has published a book about his quest to prove that he's William Shatner's child.
In "The Search," Peter Sloan — who goes by pen name Peter Shatner — details a quest that led him from Canada to Switzerland in hopes of proving the "Star Trek" actor is his dad.
Shatner has vehemently denied he is related to Sloan.
Adopted Sloan tells Page Six he first learned of his real father after locating his mother, Canadian actress Kathy McNeil, in 1984 and says he got the scoop on his paternity from his mom's former roommate.
"I found her in Switzerland in 1984 and recorded the conversation," Sloan said, claiming that the roomie confessed she walked in on McNeil and Shatner. "Two months later, she finds out she is pregnant and comes to New York to get an abortion," claims Sloan, who says McNeil was talked out of it by a Catholic friend.
"I want closure, but a DNA test would be the best," he said. "I do not think that William will take one. I have asked for it repeatedly. He may be afraid that I am going to go after his estate." But he holds out hope Shatner will claim him.


The southernmost continent of the world, Antarctica, has long been a source of fascination for explorers, scientists, geologists, and, of late, tourists, especially those travelling on cruise liners. The extreme remoteness of the place combined with the exotic life forms that live there and the mysteries that lie concealed under the numerous layers of ice pose an enticing challenge to many. According to the International Association of Antarctica Tour Operators (IAATO), 51,707 people visited the cold continent during 2017-2018. This was an increase of 17 percent from the previous year and is in keeping with the rising trend seen since 2011-2012.
The latest team of adventurers setting out on this journey of discovery to the inhospitable continent is heading out from Edmonton soon. And their journey is definitely not going to be as simple as boarding a cruise liner and sailing up to the glaciers!
Known as Xpedition 90x, this 2,400-kilometer (1,491-mile) return voyage is expected to last up to 30 days and put to test man and machine alike.
The three-member team led by innovator-inventor Darryl Weflen, President of Airworks Compressors Corp, will be credited with quite a few firsts. If the team succeeds, it will be the first Canadian-led Antarctic expedition. This voyage also has the distinction of being the first single vehicle expedition to the South Pole as well as the first hybrid vehicle expedition to the South Pole.
The team will be testing out the ruggedness of Weflen's patent-pending hybrid drive system known as G-Force, which will be used in Rockhopper, a modified International MXT. The vehicle now features a custom-made Cummins 6BT 12-valve turbo diesel with 210 horsepower engine! The Rockhopper will use Jet A as the extreme temperatures of Antarctica are bound to turn diesel to gel. Meanwhile, G-Force is a green technology intended for Class 4 and larger vehicles, which Weflen wants to introduce to the commercial trucking industry and the military after it is tested successfully.
One distinguishing factor about G-Force is that besides propelling the vehicle, it also functions as an auxiliary power unit (APU) supplying heat, electricity, and air conditioning to maximize cab comfort. Its versatility in terms of the ability to recharge while driving or even idling without the need for plugging in, reduced heat signature, and ease of fitment make it suitable for a range of military endeavours. Work on fine tuning the hybrid drive-train system has been ongoing for more than a year.
The Rockhopper will first be packed into a sea container with its tracks removed and be transported by truck to Vancouver, where it will travel by ship to Punta Arenas, Chile. It will then be moved on to a transport jet heading for Union Glacier where it will get its tracks back and embark on ground travel to its final destination. The team plans on travelling 12 to 14 hours and sleeping in tents during the course of their journey.
The Rockhopper has been fitted with a state-of-the-art tracking system from, you guessed it, GFI! Equipped with the best reporting times in the industry, GFI is a veteran in the fleet management segment and works in adherence to industry regulations and specifications. With options like geofencing and history replays, we like to think we've made a name for ourselves in the northern hemisphere. We created a custom website to track the progress of the truck as it undertakes this exciting journey from Edmonton to Antarctica.
Weflen is a refrigeration mechanic by trade who also holds quite a few patents. An avid race car driver, he has built quite a few race cars himself! He is also credited with driving the fastest Go-Kart in the world at 158.8 M.
The other members of the team are Jerry Fuentes, who will manage communications and navigation, and Greg Weflen, who will be entrusted with collecting data and carrying out mechanical repairs.
The aim of this Antarctic expedition is to encourage future explorers to reduce the carbon footprint of this type of vehicle and to aim for future commercial operations, with the overall end goal that it will reduce fuel. It is expected that G-Force technology will help reduce emissions by nearly 20 percent while helping minimize engine strain.
Besides doing their bit for the environment through this journey of a lifetime, the enthusiastic team hopes to get across the message that the world is a huge place with vast, unexplored territory that people should make time to go discover!


text:
\section{Introduction}
Chatbots or human-like dialogue systems play an important role in connecting machine-learning-powered platforms with users.
A chatbot that converses like a human should ideally (1) be capable of fluent language generation, and (2) be driven by long-term goals in a multi-turn conversation. In this paper, we develop a practical unified framework for \textbf{G}oal-\textbf{o}riented \textbf{Chat}bots (GoChat in short), which directly addresses both challenges.

Previous work on chatbots can be roughly categorized into two types, i.e., open-domain chatbots and task-oriented dialogue systems. Open-domain chatbots are not task-specific, therefore can be end-to-end trained on large corpus. With the development of deep sequence-to-sequence~(Seq2Seq) learning, they are modeled as a source to target sequence prediction task. However, they cannot support specific tasks by pursuing the long-term goals. On the other hand, task-oriented dialogue systems~\cite{liu2017iterative,yan2017building} are capable of dealing with domain-specific tasks, e.g., recommending products~\cite{yan2017building} or booking restaurant~\cite{liu2017iterative}.

Recently, some dialogue systems have been proposed to bargain on goods~\cite{lewis2017deal,he2018decoupling}, or recommend products~\cite{kang2019recommendation}.

However, designing such task-oriented dialogue systems often requires lots of hand-crafting, for example, by decomposing the task into sub-tasks with manually-designed rules, or using heavily labelled datasets to build a finite-state machine. Such ad-hoc methods are difficult to scale to complex real-world problems.

In light of these observations, in this paper, we propose the GoChat framework for end-to-end training the chatbot to maximize its long-term return without specific design of the dialogue problem.
We leverage hierarchical reinforcement learning~(HRL) that simultaneously learns a high-level policy to guide the conversation and a low-level policy to generate the response according to the guidance from the high-level policy.

In particular, at each turn of the multi-turn conversation, the high-level policy~(the \emph{manager}) observes the previous conversation as its state, determines a sub-goal as its action, and waits for a reward from the environment representing whether the final goal is accomplished or not. The low-level policy~(the \emph{worker}), on the other hand, observes the state together with the sub-goal from its manager, generates the corresponding utterance for response as its action to fulfill the sub-goal, and receives a reward from the manager.
In this way, the goal is decomposed as a hierarchy of goals, so the chatbot can be trained efficiently in an end-to-end manner.

Our dialogue agent is trained with Advantage Actor-Critic (A2C) \cite{mnih2016asynchronous}, an algorithm widely used in modern reinforcement learning and applications \cite{pan2019pgcr}. We conducted experiments on a goal-oriented dialogue corpus from real-world financial data, where each dialogue has a binary label for whether the goal is achieved or not.
Quantitative indicators as well as human evaluations demonstrate that GoChat outperforms previous methods significantly on both conversation quality and success rate of achieving the goals.

\section{GoChat: Goal-oriented Chatbots}

\begin{figure*}[t
    \centering
    \includegraphics[width=0.88\textwidth]{figure/GoChat}
    \caption{The overall GoChat framework for goal-oriented chatbots with hierarchical reinforcement learning. The manager decides the sub-goals for the worker, and the worker generates the response according to the observations and the sub-goal.}
\label{fig:model}
\end{figure*}

\subsection{Problem formulation}\label{sec:setting}

Recall that our chatbot is intended to accomplish some goal via multi-turn conversations.
Here we denote a complete dialogue with $T$ turns as $\{u_{h_1},u_{c_1},\dots,u_{h_T},u_{c_T}\}$, where $u_{h_t}$ and $u_{c_t}$, $t=1,\dots,T$ stand for the utterances generated by human and chatbot at the $t$-th turn, respectively. Each utterance is a sequence of $n$ tokens~(for sentences with less than $n$ words, we use zero-paddings to complete the sequence).
At the $t$-th turn, given the dialogue of the previous $t-1$ turns as well as the $t$-th utterance produced by human, the chatbot needs to generate $u_{c_t}$ as its response.
The target of the chatbot is to accomplish the goal through the whole conversation.

We formulate our problem as a Markov Decision Process~(MDP).
At time step $t$, the state is defined as $s_{t}=\{u_{h_1},u_{c_1},\dots,u_{h_t}\}$, and the action is the chatbot's response sequence $u_{c_t}$. For simplicity, we suppose that the reward of accomplishing the goal is given only at the end of the conversation, i.e., $r_T = 1$ if the goal is accomplished, and $r_T = -1$ otherwise. The intermediate rewards are always $0$, i.e., $r_t = 0$ for all $t<T$.

As the outcome can only be observed after a multi-turn conversation, the decision-making process is challenging.
We adopt Hierarchical Reinforcement Learning~(HRL) to deal with such a long horizon problem with sparse and delayed rewards, and propose the GoChat framework to simultaneously learn the high-level and low-level policy with end-to-end training.

\subsection{The hierarchy of chatbot policies}
The overall framework is illustrated in Figure~\ref{fig:model}.

At the $t$-th turn, given a state $s_{t}=\{u_{h_1},u_{c_1},\dots,u_{h_t}\}$, the high-level policy (the manager) first determines a sub-goal $g_t\in \mathbb{R}^{K}$, which is a one-hot vector drawn from a K-dimensional categorical distribution $g_t \sim \pi_{\psi}(g_t|s_t)$.
Observing the state and the sub-goal, the low-level \emph{worker} outputs the response $u_{c_t}\sim \pi_{\theta}(u_{c_t}| s_t, g_t)$ by an auto-regressive Seq2Seq model, i.e.,
\begin{equation}
\pi_{\theta}(u_{c_t}=(\hat{y}_1,\dots,\hat{y}_n)|s_t, g_t) = \prod_{\tau=1}^n \pi_{\theta}(\hat{y}_\tau|s_t, g_t,\hat{y}_{<\tau})
\end{equation}
Then, the manager receives a reward $r_t$ from the dialogue environment~(described in section~\ref{sec:setting}), while the worker receives a reward $r_t^w$ from the manager~(detailed in section~\ref{sec:worker}).
Next, we introduce the model structures of the manager and the worker.

\subsubsection{Manager}
We adopt the hierarchical attention network (HAN) \cite{yang-etal-2016-hierarchical,luo2018beyond} as the network structure of our manager, which learns the contextual information by sequentially encoding all the previous dialogue turns in the state.
In more detail, each utterance $u_{h_1}, u_{c_1}, \dots, u_{h_t}$ is firstly encoded by a word-level encoder to a hidden vector $h_{h_1}$, $h_{c_1}$, $\dots, h_{h_t}$ with attention mechanism, as the latent representations of utterances.

Next, all the derived sentence vectors $h_{h_1}, h_{c_1}, \dots, h_{h_t}$ are encoded at the dialogue level to form a unified dialogue-level hidden representation $H_t$ also with attention mechanism.
So $H_t$ is a high-level feature to summarize the current conversation, which is used to choose the next sub-goal,
\begin{equation}
g_t \sim \pi_{\psi}(g_t | s_t) = \textrm{Categorical}\big( \mbox{softmax}(W_g^{\top}H_t + b_g) \big),
\end{equation}
where $W_g$ and $b_g$ are trainable parameters.

\subsubsection{Worker}\label{sec:worker}
The worker generates the response $u_{c_t}$ under the high-level guidance from the manager.
We adopt Variational Hierarchical Recurrent Encoder Decoder (VHRED)~\cite{serban2017hierarchical} as the model structure of the worker, which is a commonly used architecture of multi-turn dialogue systems.
As shown in Figure~\ref{fig:model}, the worker consists of three components for generation, namely the encoder, the context RNN, and the decoder.

At time step $t$, the encoder firstly encodes the human utterance $u_{h_t}$ into an utterance representation $h^w_t$, which is then fed to an higher-level encoder, context RNN, to model the long-term aspects of the conversation. Then, context RNN can produce a dialogue representation $h^c_t$ by taking the $t$-th utterance representation $h^w_t$ and the dialogue representation $h^c_{t-1}$ from last conversation as input.
Next, $h^c_t$ is fed to a fully connected layer to predict a mean $\bm{\mu}$ and a variance $\bm{\sigma}$ of a diagonal Gaussian distribution as the variational posterior approximation for a latent variable $z_t$.
Then, the latent $z_t$ is sampled from the posterior using the reparameterization trick, i.e., $z_t = \bm{\mu} + \bm{\sigma} \circ \bm{\epsilon}$, where $\bm{\epsilon}\sim \mathcal{N}(\bm{0}, \bm{1})$. Therefore, the latent can be seen as drawn conditioned on the state: $\bm{z}_t \sim \mathcal{N}(\bm{\mu}(s_t),\bm{\sigma}(s_t))$.

Finally, the sub-goal from the manager influences the generation process by perturbing the latent $\bm{z}_t$.
We get an augmented latent variable by simply concatenating the latent $z_t$ with $g_t$, i.e., $\tilde{z}_t = [z_t, g_t]$.
Finally, the output utterance ${u}_{c_t}=[\hat{y}_1,\dots,\hat{y}_n]$ can be generated by the sequential decoder with the input of $\tilde{z}_t$ and $h^c_t$.

It is challenging to design a reasonable reward function to measures how well the worker has achieved its sub-goal, especially when our agent is trained on off-policy data.
In our framework, we use the following way to directly construct such reward with the offline dataset. First, before the end-to-end training, each (source, target) pair $(s, u_{c})$ is assigned with an \emph{offline} sub-goal vector using unsupervised clustering, so that the dataset becomes a set of tuples in the form of $(s, g, u_{c})$. Then, during training the RL agent, given the state $s_t$ and the sub-goal $g_t$ from the manager, we find the $k$ nearest neighbors of $(s_t, g_t)$ from the offline dataset and get $k$ reference responses. Finally, we compute the average BLEU score between our generated $u_{c_t}$ and the $k$ references as the reward $r_t^w$.

\subsection{End-to-end training the chatbot}\label{sec:training}
In this part, we describe the training methods of our frameworks.

To accelerate learning, we first use a pre-train step to warm start the agent. Recall that we have generated the offline sub-goals for each source-target pair, we can pretrain the manager and the worker with supervised Seq2Seq learning.

Then, we use a simulation-based environment to end-to-end optimize the goal-oriented chatbot. Following previous work on model-based RL \cite{pan2019pome} and RL-for-dialog~\cite{saleh2019hierarchical}, we simulate the interactive simulator with a user model. Each conversation is initialized with some starting utterances randomly sampled from the training set. Then, our agent chats with a learned user model. The user model has the same network structure with our agent, and is trained with supervised Seq2Seq learning on the dataset to predict the human response and is fixed after training. We limit the multi-turn simulation dialogues to $m$ turns~($2m$ utterances), and $m$ is up to the practical application scenarios, to prevent the agents from going off-topic on overlong conversations.

During training the HRL agent, we simultaneously optimize the manager and worker to maximize the following objective function
\begin{equation}\label{eq:actor_loss}
J(\pi_\psi, \pi_\theta) = \mathbb{E}_{g_t\sim \pi_\psi(\cdot|s_t), u_{c_t}\sim \pi_\theta(\cdot|s_t, g_t)}\big[\sum_{t=1}^{T} \alpha r_{t} + \beta  r^w_{t} \big]
\end{equation}
where the expectation is over conversations induced by the policies, and $\alpha, \beta$ are hyper-parameters to balance the two levels of rewards.

To maximize $J(\pi_\psi, \pi_\theta)$, we use the Advantage Actor-Critic~(A2C) \cite{mnih2016asynchronous} algorithm, where the policies are updated according to the estimated advantage function of the chosen actions. E.g., for the manager, $\hat A(s_t, g_t) = R_t(s_t, g_t) - \hat V_\phi(s_t)$, where $R_t=\sum_{\tau=t}^T r_t$ is the long-term return and $\hat V_\phi(s_t)$ is the estimated value function of state $s_t$ with parameters $\phi$.
Then, the surrogate loss for policy learning is
\begin{equation}
    L_{\pi} = -\hat{\mathbb{E}}_t\big[\sum_{t=1}^{T} \alpha \hat A_{t} \log \pi_\psi(g_t|s_t) + \beta  A^w_{t}\log \pi_\theta(u_{c_t} | s_t, g_t)\big]
\end{equation}
where $\hat{\mathbb{E}}_t[\cdot]$ denotes the sample mean over rollout dialogues induced by the policies.

We adopt the hierarchical attention network~(HAN) as the value network $\hat V_\phi(s_t)$, which is trained to minimize the following squared error according to the Bellman equation
\begin{equation}\label{eq:critic_loss}
L_{v} = \frac{1}{2}\big(r_{t+1} + \gamma V_{\phi}^{\textrm{targ}}(s_{t+1}) - V_{\phi}(s_{t})\big)^2
\end{equation}
where $V_{\phi}^{\textrm{targ}}$ is the target value network of A2C and $\gamma=0.99$ is the discount factor.
Finally, we have a unified loss
$\mathcal{L} = L_{\pi} + L_{v}.$

\section{Conclusion}
In this paper, we propose an end-to-end framework GoChat for goal-oriented dialogue systems based on hierarchical reinforcement learning without cumbersome hand-crafting.
It uses a high-level policy to determine the sub-goal to guide the conversation towards the final goal and a low-level worker to generate corresponding responses. Experiments conducted on a real-world dialogue dataset verified the effectiveness of our GoChat.
\section{Experiments}
\subsection{Experimental set-up} \label{data}
We conduct experiments on a anti-fraud dialogue system on a large-scale FinTech platform. Our task is to train a chatbot for assisting the security officers to have a conversation with suspected scammers.
The goal of the chatbot is to find evidences of the scammers' fraud, e.g., to get to know their account ID so that the security officers can take further investigation.
We collect a dataset of over four thousand historical dialogues between the suspected scammers and the security officers, where each dialogue has about 20 turns in average and is only labeled with a binary outcome, indicating whether the goal is accomplished.
The data is anonymized without any physical person's privacy information.

Given such dataset, we compare GoChat against state-of-the-art multi-turn dialog methods, including HRED~\cite{serban2016building}, VHRED~\cite{serban2017hierarchical}, WSeq~\cite{tian2017make}, HRAN~\cite{xing2018hierarchical}, and ReCoSa~\cite{zhang2019recosa}. We do not compare other task-oriented approaches which require detailed annotations on each query-response pairs or dialogue states.

\begin{table}[]
\caption{Quantitative results. D-1,-2 stands for Distinct-1,-2.}
\label{tab:result}\vspace{-1mm}
\begin{tabular}{lcccccc}
\toprule
\multirow{2}{*}{Method} & \multicolumn{3}{c}{v.s. GoChat} & \multirow{2}{*}{BLEU} & \multirow{2}{*}{D-1} & \multirow{2}{*}{D-2} \\ \cline{2-4}
        & Win(\%)   & Tie(\%)  & Lose(\%)   &                        \\ \hline
HRED    &    17.3   &    30.2  &  52.5 &     9.21      &  .051      &  0.407  \\
VHRED   &    22.6   &    40.0  &  37.4 &  9.32      &  .053 &   0.419          \\
WSeq    &    9.5    &    15.3  &  75.2 &      6.43  & .047 &   0.372    \\
HRAN    &    14.9   &    12.8  &  72.3 &    7.18   &  .048 & 0.384    \\
ReCoSa  &    22.1   &    32.6  &  45.3 & 9.01   &  .050 & 0.415 \\
\midrule
GoChat & - & - & - & \textbf{9.70} &  \textbf{.061} &  \textbf{0.479}   \\
\bottomrule
\end{tabular}
\end{table}\label{tab:case}

\begin{table*}[] \small
\caption{Case study: comparison of generated dialogue from our GoChat and the most competitive baseline, VHRED.}\vspace{-2mm}
\label{tab:case}
\begin{tabular}{cll}
\toprule
& \textbf{Agent} & \textbf{Utterance} \\ \hline

\multirow{6}{*}{\textbf{Case~(A)}} & \multirow{2}{*}{Human (Scammer):} & \begin{CJK*}{UTF8}{gkai} 亲,您好!欢迎您来到企业招聘兼职中心 ,请问您是否需要兼职?\end{CJK*} \\
& & (Welcome to the Enterprise Part-Time Recruitment Center. Do you need a part-time job?) \\
& \multirow{2}{*}{VHRED (Officer):} &\begin{CJK*}{UTF8}{gkai} 是的 \end{CJK*} \\
& & (Yes. )\\
& \multirow{2}{*}{GoChat (Officer):} &\begin{CJK*}{UTF8}{gkai} 需要。有什么兼职可以做呢? \end{CJK*} \\
& & (Yes, I need. What kinds of part-time jobs do you have? )\\
\hline

\multirow{6}{*}{\textbf{Case (B)}} & \multirow{2}{*}{Human (Scammer):} & \begin{CJK*}{UTF8}{gkai} 你是谁推荐来贷款的?\end{CJK*} \\
& & (Who recommended you here for the loan?) \\
& \multirow{2}{*}{VHRED (Officer):} &\begin{CJK*}{UTF8}{gkai} 从一个朋友那里 \end{CJK*} \\
& & (From one of my friends.)\\
& \multirow{2}{*}{GoChat (Officer):} &\begin{CJK*}{UTF8}{gkai} 就是打游戏的时候,人家说可以办理贷款的。请问可以办理嘛? \end{CJK*} \\
& & (When I was playing a game, someone said I could apply for a loan. Can I?)\\
\bottomrule
\end{tabular}
\end{table*}

\subsection{Implementation details and evaluation}

In our experiments, we limit the maximum length of dialog to $m=20$ turns~($40$ utterances) and adopt $500$-size word embeddings for all methods.
The hidden size of word-level and dialog-level encoders in HAN~(both the manager and the critic) are $500$ and $50$, respectively.
The configuration of the worker follows VHRED \cite{serban2017hierarchical}.

We firstly pretrain the worker on $(s_t,g_t,u_{c_t})$ tuples detailed in Section~\ref{sec:worker} with supervised Seq2Seq learning, where the offline sub-goal $g_t$ is given by Latent Dirichlet allocation~(LDA) into $14$ categories~(sub-goals).
We set the hyper-parameters in Eq.~\ref{eq:actor_loss} as $\alpha=1,\beta=0.001$.
The agent is optimized with Adam \cite{kingma2014adam}, and trained for $10$ epochs with a learning rate of $0.001$.

We adopt human evaluation to examine the utterances for achieving goals.
In detail, eight annotators are asked to compare the simulated response between our GoChat and each baseline given the same context.
Annotators would give it a tie if both our method and baselines fail/succeed to generate utterances which are beneficial to achieve the goal, otherwise they would annotate a win/lose.
We also use BLEU and distinct~\cite{li-etal-2016-diversity} to evaluate the quality and the diversity of the generated responses, respectively.

\subsection{Quantitative results} \label{sec:res}
We compare our model with five baselines for multi-turn dialogue, and report the quantitative indicators and human evaluation in Table~\ref{tab:result}.
From the perspective of generation indicator, our proposed GoChat get the highest score on BLEU~($9.70$), distinct-1~($.061$) and distinct-2~($0.479$), which demonstrates our GoChat can generate more high-quality and varied utterances than baseline methods.
From the human evaluation, we observe that GoChat outperforms all the baselines on generating utterances for achieving the goal. Specifically, GoChat is preferred at rates of $35.2\%$, $14.8\%$, $65.7\%$, $57.4\%$, $23.2\%$ against HRED, VHRED, WSeq, HRAN and ReCoSa, respectively.
We conjecture it might be the simultaneous training of high-level and low-level policy that our GoChat better models long-term rewards the goals and produces high-quality responses.
\subsection{Qualitative results}

According to Table~\ref{tab:result}, VHRED is the most competitive method. To compare, we present two representative cases in Table~\ref{tab:case}. Observing these cases, both two methods can generate appropriate responses to the inquiries of the scammer given same dialogue contexts.
However, our model is more likely to raise questions and generate more diverse sentences. Specifically, in both cases, the utterances from our GoChat, i.e, ``What kinds of part-time jobs do you have?'', ``Can I?'', are more proactive.
Such responses are more likely towards the ultimate goal in a long run.
While case~(B) demonstrates that our GoChat can generate more diverse and human-like responses, i.e., ``When playing games'', which is very convincing and might help to get rid of the scammers' doubts in anti-fraud dialogue tasks.
\section{Introduction}
Chatbots or human-like dialogue systems play an important role in connecting machine-learning-powered platforms with users.
A chatbot that converses like a human should ideally (1) be capable of fluent language generation, and (2) be driven by long-term goals in a multi-turn conversation. In this paper, we develop a practical unified framework for \textbf{G}oal-\textbf{o}riented \textbf{Chat}bots (GoChat in short), which directly addresses both challenges.

Previous work on chatbots can be roughly categorized into two types, i.e., open-domain chatbots and task-oriented dialogue systems. Open-domain chatbots are not task-specific, therefore can be end-to-end trained on large corpus. With the development of deep sequence-to-sequence~(Seq2Seq) learning, they are modeled as a source to target sequence prediction task. However, they cannot support specific tasks by pursuing the long-term goals. On the other hand, task-oriented dialogue systems~\cite{liu2017iterative,yan2017building} are capable of dealing with domain-specific tasks, e.g., recommending products~\cite{yan2017building} or booking restaurant~\cite{liu2017iterative}.

Recently, some dialogue systems have been proposed to bargain on goods~\cite{lewis2017deal,he2018decoupling}, or recommend products~\cite{kang2019recommendation}.

However, designing such task-oriented dialogue systems often requires lots of hand-crafting, for example, by decomposing the task into sub-tasks with manually-designed rules, or using heavily labelled datasets to build a finite-state machine. Such ad-hoc methods are difficult to scale to complex real-world problems.

In light of these observations, in this paper, we propose the GoChat framework for end-to-end training the chatbot to maximize its long-term return without specific design of the dialogue problem.
We leverage hierarchical reinforcement learning~(HRL) that simultaneously learns a high-level policy to guide the conversation and a low-level policy to generate the response according to the guidance from the high-level policy.

In particular, at each turn of the multi-turn conversation, the high-level policy~(the \emph{manager}) observes the previous conversation as its state, determines a sub-goal as its action, and waits for a reward from the environment representing whether the final goal is accomplished or not. The low-level policy~(the \emph{worker}), on the other hand, observes the state together with the sub-goal from its manager, generates the corresponding utterance for response as its action to fulfill the sub-goal, and receives a reward from the manager.
In this way, the goal is decomposed as a hierarchy of goals, so the chatbot can be trained efficiently in an end-to-end manner.

Our dialogue agent is trained with Advantage Actor-Critic (A2C) \cite{mnih2016asynchronous}, an algorithm widely used in modern reinforcement learning and applications \cite{pan2019pgcr}. We conducted experiments on a goal-oriented dialogue corpus from real-world financial data, where each dialogue has a binary label for whether the goal is achieved or not.
Quantitative indicators as well as human evaluations demonstrate that GoChat outperforms previous methods significantly on both conversation quality and success rate of achieving the goals.

\section{GoChat: Goal-oriented Chatbots}

\begin{figure*}[t
    \centering
    \includegraphics[width=0.88\textwidth]{figure/GoChat}
    \caption{The overall GoChat framework for goal-oriented chatbots with hierarchical reinforcement learning. The manager decides the sub-goals for the worker, and the worker generates the response according to the observations and the sub-goal.}
\label{fig:model}
\end{figure*}

\subsection{Problem formulation}\label{sec:setting}

Recall that our chatbot is intended to accomplish some goal via multi-turn conversations.
Here we denote a complete dialogue with $T$ turns as $\{u_{h_1},u_{c_1},\dots,u_{h_T},u_{c_T}\}$, where $u_{h_t}$ and $u_{c_t}$, $t=1,\dots,T$ stand for the utterances generated by human and chatbot at the $t$-th turn, respectively. Each utterance is a sequence of $n$ tokens~(for sentences with less than $n$ words, we use zero-paddings to complete the sequence).
At the $t$-th turn, given the dialogue of the previous $t-1$ turns as well as the $t$-th utterance produced by human, the chatbot needs to generate $u_{c_t}$ as its response.
The target of the chatbot is to accomplish the goal through the whole conversation.

We formulate our problem as a Markov Decision Process~(MDP).
At time step $t$, the state is defined as $s_{t}=\{u_{h_1},u_{c_1},\dots,u_{h_t}\}$, and the action is the chatbot's response sequence $u_{c_t}$. For simplicity, we suppose that the reward of accomplishing the goal is given only at the end of the conversation, i.e., $r_T = 1$ if the goal is accomplished, and $r_T = -1$ otherwise. The intermediate rewards are always $0$, i.e., $r_t = 0$ for all $t<T$.

As the outcome can only be observed after a multi-turn conversation, the decision-making process is challenging.
We adopt Hierarchical Reinforcement Learning~(HRL) to deal with such a long horizon problem with sparse and delayed rewards, and propose the GoChat framework to simultaneously learn the high-level and low-level policy with end-to-end training.

\subsection{The hierarchy of chatbot policies}
The overall framework is illustrated in Figure~\ref{fig:model}.

At the $t$-th turn, given a state $s_{t}=\{u_{h_1},u_{c_1},\dots,u_{h_t}\}$, the high-level policy (the manager) first determines a sub-goal $g_t\in \mathbb{R}^{K}$, which is a one-hot vector drawn from a K-dimensional categorical distribution $g_t \sim \pi_{\psi}(g_t|s_t)$.
Observing the state and the sub-goal, the low-level \emph{worker} outputs the response $u_{c_t}\sim \pi_{\theta}(u_{c_t}| s_t, g_t)$ by an auto-regressive Seq2Seq model, i.e.,
\begin{equation}
\pi_{\theta}(u_{c_t}=(\hat{y}_1,\dots,\hat{y}_n)|s_t, g_t) = \prod_{\tau=1}^n \pi_{\theta}(\hat{y}_\tau|s_t, g_t,\hat{y}_{<\tau})
\end{equation}
Then, the manager receives a reward $r_t$ from the dialogue environment~(described in section~\ref{sec:setting}), while the worker receives a reward $r_t^w$ from the manager~(detailed in section~\ref{sec:worker}).
Next, we introduce the model structures of the manager and the worker.

\subsubsection{Manager}
We adopt the hierarchical attention network (HAN) \cite{yang-etal-2016-hierarchical,luo2018beyond} as the network structure of our manager, which learns the contextual information by sequentially encoding all the previous dialogue turns in the state.
In more detail, each utterance $u_{h_1}, u_{c_1}, \dots, u_{h_t}$ is firstly encoded by a word-level encoder to a hidden vector $h_{h_1}$, $h_{c_1}$, $\dots, h_{h_t}$ with attention mechanism, as the latent representations of utterances.

Next, all the derived sentence vectors $h_{h_1}, h_{c_1}, \dots, h_{h_t}$ are encoded at the dialogue level to form a unified dialogue-level hidden representation $H_t$ also with attention mechanism.
So $H_t$ is a high-level feature to summarize the current conversation, which is used to choose the next sub-goal,
\begin{equation}
g_t \sim \pi_{\psi}(g_t | s_t) = \textrm{Categorical}\big( \mbox{softmax}(W_g^{\top}H_t + b_g) \big),
\end{equation}
where $W_g$ and $b_g$ are trainable parameters.

\subsubsection{Worker}\label{sec:worker}
The worker generates the response $u_{c_t}$ under the high-level guidance from the manager.
We adopt Variational Hierarchical Recurrent Encoder Decoder (VHRED)~\cite{serban2017hierarchical} as the model structure of the worker, which is a commonly used architecture of multi-turn dialogue systems.
As shown in Figure~\ref{fig:model}, the worker consists of three components for generation, namely the encoder, the context RNN, and the decoder.

At time step $t$, the encoder firstly encodes the human utterance $u_{h_t}$ into an utterance representation $h^w_t$, which is then fed to an higher-level encoder, context RNN, to model the long-term aspects of the conversation. Then, context RNN can produce a dialogue representation $h^c_t$ by taking the $t$-th utterance representation $h^w_t$ and the dialogue representation $h^c_{t-1}$ from last conversation as input.
Next, $h^c_t$ is fed to a fully connected layer to predict a mean $\bm{\mu}$ and a variance $\bm{\sigma}$ of a diagonal Gaussian distribution as the variational posterior approximation for a latent variable $z_t$.
Then, the latent $z_t$ is sampled from the posterior using the reparameterization trick, i.e., $z_t = \bm{\mu} + \bm{\sigma} \circ \bm{\epsilon}$, where $\bm{\epsilon}\sim \mathcal{N}(\bm{0}, \bm{1})$. Therefore, the latent can be seen as drawn conditioned on the state: $\bm{z}_t \sim \mathcal{N}(\bm{\mu}(s_t),\bm{\sigma}(s_t))$.

Finally, the sub-goal from the manager influences the generation process by perturbing the latent $\bm{z}_t$.
We get an augmented latent variable by simply concatenating the latent $z_t$ with $g_t$, i.e., $\tilde{z}_t = [z_t, g_t]$.
Finally, the output utterance ${u}_{c_t}=[\hat{y}_1,\dots,\hat{y}_n]$ can be generated by the sequential decoder with the input of $\tilde{z}_t$ and $h^c_t$.

It is challenging to design a reasonable reward function to measures how well the worker has achieved its sub-goal, especially when our agent is trained on off-policy data.
In our framework, we use the following way to directly construct such reward with the offline dataset. First, before the end-to-end training, each (source, target) pair $(s, u_{c})$ is assigned with an \emph{offline} sub-goal vector using unsupervised clustering, so that the dataset becomes a set of tuples in the form of $(s, g, u_{c})$. Then, during training the RL agent, given the state $s_t$ and the sub-goal $g_t$ from the manager, we find the $k$ nearest neighbors of $(s_t, g_t)$ from the offline dataset and get $k$ reference responses. Finally, we compute the average BLEU score between our generated $u_{c_t}$ and the $k$ references as the reward $r_t^w$.

\subsection{End-to-end training the chatbot}\label{sec:training}
In this part, we describe the training methods of our frameworks.

To accelerate learning, we first use a pre-train step to warm start the agent. Recall that we have generated the offline sub-goals for each source-target pair, we can pretrain the manager and the worker with supervised Seq2Seq learning.

Then, we use a simulation-based environment to end-to-end optimize the goal-oriented chatbot. Following previous work on model-based RL \cite{pan2019pome} and RL-for-dialog~\cite{saleh2019hierarchical}, we simulate the interactive simulator with a user model. Each conversation is initialized with some starting utterances randomly sampled from the training set. Then, our agent chats with a learned user model. The user model has the same network structure with our agent, and is trained with supervised Seq2Seq learning on the dataset to predict the human response and is fixed after training. We limit the multi-turn simulation dialogues to $m$ turns~($2m$ utterances), and $m$ is up to the practical application scenarios, to prevent the agents from going off-topic on overlong conversations.

During training the HRL agent, we simultaneously optimize the manager and worker to maximize the following objective function
\begin{equation}\label{eq:actor_loss}
J(\pi_\psi, \pi_\theta) = \mathbb{E}_{g_t\sim \pi_\psi(\cdot|s_t), u_{c_t}\sim \pi_\theta(\cdot|s_t, g_t)}\big[\sum_{t=1}^{T} \alpha r_{t} + \beta  r^w_{t} \big]
\end{equation}
where the expectation is over conversations induced by the policies, and $\alpha, \beta$ are hyper-parameters to balance the two levels of rewards.

To maximize $J(\pi_\psi, \pi_\theta)$, we use the Advantage Actor-Critic~(A2C) \cite{mnih2016asynchronous} algorithm, where the policies are updated according to the estimated advantage function of the chosen actions. E.g., for the manager, $\hat A(s_t, g_t) = R_t(s_t, g_t) - \hat V_\phi(s_t)$, where $R_t=\sum_{\tau=t}^T r_t$ is the long-term return and $\hat V_\phi(s_t)$ is the estimated value function of state $s_t$ with parameters $\phi$.
Then, the surrogate loss for policy learning is
\begin{equation}
    L_{\pi} = -\hat{\mathbb{E}}_t\big[\sum_{t=1}^{T} \alpha \hat A_{t} \log \pi_\psi(g_t|s_t) + \beta  A^w_{t}\log \pi_\theta(u_{c_t} | s_t, g_t)\big]
\end{equation}
where $\hat{\mathbb{E}}_t[\cdot]$ denotes the sample mean over rollout dialogues induced by the policies.

We adopt the hierarchical attention network~(HAN) as the value network $\hat V_\phi(s_t)$, which is trained to minimize the following squared error according to the Bellman equation
\begin{equation}\label{eq:critic_loss}
L_{v} = \frac{1}{2}\big(r_{t+1} + \gamma V_{\phi}^{\textrm{targ}}(s_{t+1}) - V_{\phi}(s_{t})\big)^2
\end{equation}
where $V_{\phi}^{\textrm{targ}}$ is the target value network of A2C and $\gamma=0.99$ is the discount factor.
Finally, we have a unified loss
$\mathcal{L} = L_{\pi} + L_{v}.$

\section{Conclusion}
In this paper, we propose an end-to-end framework GoChat for goal-oriented dialogue systems based on hierarchical reinforcement learning without cumbersome hand-crafting.
It uses a high-level policy to determine the sub-goal to guide the conversation towards the final goal and a low-level worker to generate corresponding responses. Experiments conducted on a real-world dialogue dataset verified the effectiveness of our GoChat.
\section{Experiments}
\subsection{Experimental set-up} \label{data}
We conduct experiments on a anti-fraud dialogue system on a large-scale FinTech platform. Our task is to train a chatbot for assisting the security officers to have a conversation with suspected scammers.
The goal of the chatbot is to find evidences of the scammers' fraud, e.g., to get to know their account ID so that the security officers can take further investigation.
We collect a dataset of over four thousand historical dialogues between the suspected scammers and the security officers, where each dialogue has about 20 turns in average and is only labeled with a binary outcome, indicating whether the goal is accomplished.
The data is anonymized without any physical person's privacy information.

Given such dataset, we compare GoChat against state-of-the-art multi-turn dialog methods, including HRED~\cite{serban2016building}, VHRED~\cite{serban2017hierarchical}, WSeq~\cite{tian2017make}, HRAN~\cite{xing2018hierarchical}, and ReCoSa~\cite{zhang2019recosa}. We do not compare other task-oriented approaches which require detailed annotations on each query-response pairs or dialogue states.

\begin{table}[]
\caption{Quantitative results. D-1,-2 stands for Distinct-1,-2.}
\label{tab:result}\vspace{-1mm}
\begin{tabular}{lcccccc}
\toprule
\multirow{2}{*}{Method} & \multicolumn{3}{c}{v.s. GoChat} & \multirow{2}{*}{BLEU} & \multirow{2}{*}{D-1} & \multirow{2}{*}{D-2} \\ \cline{2-4}
        & Win(\%)   & Tie(\%)  & Lose(\%)   &                        \\ \hline
HRED    &    17.3   &    30.2  &  52.5 &     9.21      &  .051      &  0.407  \\
VHRED   &    22.6   &    40.0  &  37.4 &  9.32      &  .053 &   0.419          \\
WSeq    &    9.5    &    15.3  &  75.2 &      6.43  & .047 &   0.372    \\
HRAN    &    14.9   &    12.8  &  72.3 &    7.18   &  .048 & 0.384    \\
ReCoSa  &    22.1   &    32.6  &  45.3 & 9.01   &  .050 & 0.415 \\
\midrule
GoChat & - & - & - & \textbf{9.70} &  \textbf{.061} &  \textbf{0.479}   \\
\bottomrule
\end{tabular}
\end{table}\label{tab:case}

\begin{table*}[] \small
\caption{Case study: comparison of generated dialogue from our GoChat and the most competitive baseline, VHRED.}\vspace{-2mm}
\label{tab:case}
\begin{tabular}{cll}
\toprule
& \textbf{Agent} & \textbf{Utterance} \\ \hline

\multirow{6}{*}{\textbf{Case~(A)}} & \multirow{2}{*}{Human (Scammer):} & \begin{CJK*}{UTF8}{gkai} 亲,您好!欢迎您来到企业招聘兼职中心 ,请问您是否需要兼职?\end{CJK*} \\
& & (Welcome to the Enterprise Part-Time Recruitment Center. Do you need a part-time job?) \\
& \multirow{2}{*}{VHRED (Officer):} &\begin{CJK*}{UTF8}{gkai} 是的 \end{CJK*} \\
& & (Yes. )\\
& \multirow{2}{*}{GoChat (Officer):} &\begin{CJK*}{UTF8}{gkai} 需要。有什么兼职可以做呢? \end{CJK*} \\
& & (Yes, I need. What kinds of part-time jobs do you have? )\\
\hline

\multirow{6}{*}{\textbf{Case (B)}} & \multirow{2}{*}{Human (Scammer):} & \begin{CJK*}{UTF8}{gkai} 你是谁推荐来贷款的?\end{CJK*} \\
& & (Who recommended you here for the loan?) \\
& \multirow{2}{*}{VHRED (Officer):} &\begin{CJK*}{UTF8}{gkai} 从一个朋友那里 \end{CJK*} \\
& & (From one of my friends.)\\
& \multirow{2}{*}{GoChat (Officer):} &\begin{CJK*}{UTF8}{gkai} 就是打游戏的时候,人家说可以办理贷款的。请问可以办理嘛? \end{CJK*} \\
& & (When I was playing a game, someone said I could apply for a loan. Can I?)\\
\bottomrule
\end{tabular}
\end{table*}

\subsection{Implementation details and evaluation}

In our experiments, we limit the maximum length of dialog to $m=20$ turns~($40$ utterances) and adopt $500$-size word embeddings for all methods.
The hidden size of word-level and dialog-level encoders in HAN~(both the manager and the critic) are $500$ and $50$, respectively.
The configuration of the worker follows VHRED \cite{serban2017hierarchical}.

We firstly pretrain the worker on $(s_t,g_t,u_{c_t})$ tuples detailed in Section~\ref{sec:worker} with supervised Seq2Seq learning, where the offline sub-goal $g_t$ is given by Latent Dirichlet allocation~(LDA) into $14$ categories~(sub-goals).
We set the hyper-parameters in Eq.~\ref{eq:actor_loss} as $\alpha=1,\beta=0.001$.
The agent is optimized with Adam \cite{kingma2014adam}, and trained for $10$ epochs with a learning rate of $0.001$.

We adopt human evaluation to examine the utterances for achieving goals.
In detail, eight annotators are asked to compare the simulated response between our GoChat and each baseline given the same context.
Annotators would give it a tie if both our method and baselines fail/succeed to generate utterances which are beneficial to achieve the goal, otherwise they would annotate a win/lose.
We also use BLEU and distinct~\cite{li-etal-2016-diversity} to evaluate the quality and the diversity of the generated responses, respectively.

\subsection{Quantitative results} \label{sec:res}
We compare our model with five baselines for multi-turn dialogue, and report the quantitative indicators and human evaluation in Table~\ref{tab:result}.
From the perspective of generation indicator, our proposed GoChat get the highest score on BLEU~($9.70$), distinct-1~($.061$) and distinct-2~($0.479$), which demonstrates our GoChat can generate more high-quality and varied utterances than baseline methods.
From the human evaluation, we observe that GoChat outperforms all the baselines on generating utterances for achieving the goal. Specifically, GoChat is preferred at rates of $35.2\%$, $14.8\%$, $65.7\%$, $57.4\%$, $23.2\%$ against HRED, VHRED, WSeq, HRAN and ReCoSa, respectively.
We conjecture it might be the simultaneous training of high-level and low-level policy that our GoChat better models long-term rewards the goals and produces high-quality responses.
\subsection{Qualitative results}

According to Table~\ref{tab:result}, VHRED is the most competitive method. To compare, we present two representative cases in Table~\ref{tab:case}. Observing these cases, both two methods can generate appropriate responses to the inquiries of the scammer given same dialogue contexts.
However, our model is more likely to raise questions and generate more diverse sentences. Specifically, in both cases, the utterances from our GoChat, i.e, ``What kinds of part-time jobs do you have?'', ``Can I?'', are more proactive.
Such responses are more likely towards the ultimate goal in a long run.
While case~(B) demonstrates that our GoChat can generate more diverse and human-like responses, i.e., ``When playing games'', which is very convincing and might help to get rid of the scammers' doubts in anti-fraud dialogue tasks.
\section*{acknowledgement}
The research work is supported by the National Natural Science

\noindent Foundation of China under Grant No. 61976204, U1811461, the Project of Youth Innovation Promotion Association CAS. This work is also funded in part by Ant Financial through the Ant Financial Science Funds for Security Research.
\bibliographystyle{ACM-Reference-Format}

\section*{acknowledgement}
The research work is supported by the National Natural Science

\noindent Foundation of China under Grant No. 61976204, U1811461, the Project of Youth Innovation Promotion Association CAS. This work is also funded in part by Ant Financial through the Ant Financial Science Funds for Security Research.
\bibliographystyle{ACM-Reference-Format}

redpajama_set_name: RedPajamaArXiv


KARACHI (January 09 2004): The Securities and Exchange Commission of Pakistan (SECP) would pursue stock exchanges for 'demutalisation' and subsequent merger of the country's three bourses.
This was stated by Dr Tariq Hassan, Chairman, SECP, on Thursday. He added that the regulator would soon finalise a report on the demutalisation of the stock exchanges, its objective, rules and other regulation to have a smooth capital market at par with international standards.
"The report will be finalised in one week," said SECP chief. He was addressing the launch of Crosby Asset Management Limited, whose Rs 30 million paid-up capital is held by Crosby Asset Management International Limited – a company incorporated in the British Virgin Islands and is a one of the financial arms of Crosby group.
"We (SECP) have also finalised rules for margin financing and the gradual phase out of badla (carry-over transaction) is likely to take place within next few months," he said.
To a question he said that the government directors on the board of Karachi Stock Exchange for calendar year 2004 were free to use their right of vote to elect the bourse chairman.
On mutual funds, the chairman said, interest of Crosby showed that the confidence of investors not only improved in Pakistan's economy, but also in the integrity of the financial system.
The launch of the fund was also seen as an important factor in the development of the mutual fund sub-sector in Pakistan.
Dr Hassan said that the management of the fund was mostly Pakistani and their foreign counterparts were only providing back end support. "I am pleased to see that we have such skill available in our country, and I am sure that the fund will benefit from their expertise," he said.
The chairman also appreciated the performance of the mutual funds industry, which has shown significant improvement over previous years. The growth is attributable to an effective monitoring and regulatory regime, higher corporate earnings, reduction in interest rates and a buoyant stock market," he concluded.


Over the next few weeks, we'll be gearing up for the upcoming season by creating our official 2018-19 live broadcast schedule. We're currently offering single-game and multi-game packages. Reach out to Mark Igo at migo@myhockeylive.com to discuss our exciting options to reserve your team's games today!
ALL teams in the MIAA, American Collegiate Hockey Association and prep school leagues are eligible to have their games broadcast on My Hockey Live.


text:
Terrence Stubbs (born March 25, 1980 in Manassas, Virginia) is a former Arena Football League defensive back for the Los Angeles Avengers.

High School Years
Stubbs attended Oscar F. Smith High School in Chesapeake, Virginia and was a student and a letterman in football. In football, he won All-City and an All-District honors as a wide receiver.

College years
Stubbs attended Temple University, and was a good student and a letterman in football. In football, he finished his career with 65 receptions for 893 yards (13.74 yards per rec. avg.) and two touchdowns.

External links
Los Angeles Avengers' player profile
AFL stats

1980 births
Living people
Sportspeople from Chesapeake, Virginia
People from Manassas, Virginia
American football defensive backs
American football wide receivers
Temple Owls football players
New York Jets players
Berlin Thunder players
Los Angeles Avengers players

redpajama_set_name: RedPajamaWikipedia


Jyothishmathi Sheejith is an accomplished Carnatic vocalist and composer who received her four-year undergraduate and two-year postgraduate diploma in music from the Kalakshetra Foundation in 1997. Having studied Carnatic vocal from such doyens as Vairamangalam Lakshminarayanan and S Rajaram, she has sung in the lead for many classic Kalakshetra Repertory ensemble dance productions. As an independent artist, she has toured worldwide with illustrious Bharatanatyam dancers such as CV Chandrasekhar, VP and Shanta Dhananjayan, Leela Samson, and Malavika Sarukkai. A graded artist of All India Radio, she has conducted workshops and served as faculty in teaching residencies in India and abroad. Jyothishmathi has composed several musical scores for dance as an individual artist and in partnership with dancer-choreographer husband Sheejith Krishna. Their innovative and critically lauded collaborations include Masquerade (2007), Krishna Bharatam (2009), Parinaamam (2010), Pravaha (2013), Yathadarshe Tathaatmani (2014) and Don Quixote (2015).


text:
Home Vitamin B
Vitamin B Supplements are not a "one size fits all". We offer a large selection of various vitamin b supplements at great prices to help you find the right product to reach your goals. Customer service is available seven days a week from 7:30am to 4:00pm PST through live chat, phone and email.
What Is NMN (Nicotinamide Mononucleotide)? NMN (Nicotinamide Mononucleotide) is a nucleotide that is derived from nicotinamide and ribose. It is pr...
What Is Inositol? Inositol, also referred to as Vitamin B8, is not a true Vitamin at all. It is a naturally occurring sugar that plays several key ...
What Is Niacin (Vitamin B3)? Niacin (vitamin B3) is a water-soluble vitamin created by the body that plays an important role in many bodily functio...
What Is Biotin (Vitamin B7)? Biotin, also known as vitamin B7, is a water-soluble vitamin that plays a vital role in metabolism, gene regulation, a...
What Is Niacinamide? Niacinamide is a form of Vitamin B3 that is found in meat products and does not cause skin flushing. Niacinamide Benefits Niac...
What Is D-Biotin? D-biotin is a form of vitamin B7, a water-soluble vitamin that plays a vital role in metabolism, gene regulation, and cell signal...
What is Methylcobalamin? Methylcobalamin is the natural form of Vitamin B12. Vitamin B12 is an essential, water-soluble vitamin that is abundantly ...
What Is Cyanocobalamin (Vitamin B12)? Cyanocobalamin is a synthetic form of vitamin B12. Vitamin B12 is an essential, water-soluble vitamin that is...
What Is Benfotiamine? Benfotiamine is a synthetic derivative of vitamin B1 (thiamine), and one of its most bioavailable forms. Benfotiamine Benefit...
What is Riboflavin (Vitamin B2)? Riboflavin (vitamin B2) is an important B vitamin that is used in the metabolic process. Riboflavin (Vitamin B2) B...
What Is Pyridoxine Hydrochloride? Pyridoxine hydrochloride (HCl) is a form of vitamin B6 that is bonded with hydrochloride to increase absorption. ...
What Is Methylcobalamin (Vitamin B12)? Methylcobalamin is the natural form of vitamin B12. Vitamin B12 is an essential, water-soluble vitamin that ...
What is Calcium Pantothenate? Calcium pantothenate is a form of vitamin B5, an essential vitamin used in several important body functions such as t...
What Is Thiamine Mononitrate? Thiamine mononitrate is a synthetic, stable nitrate salt form of thiamine (vitamin B1). Thiamine mononitrate is a wat...
What Is Folic Acid Powder? Folic acid powder is a powdered, synthetic form of vitamin B9 (folate) that has higher rates of absorption than naturall...
What is Pyridoxal-5-Phosphate (P5P)? Pyridoxal-5-Phosphate (P5P) is the active form of vitamin B6. P5P is a coenzyme involved in over a hundred pro...
What Is Thiamine HCl? Thiamine HCl is a form of thiamine (vitamin B1) that has been bonded with hydrochloride (HCl) to improve absorption. Thiamine...
What Is Inositol Hexanicotinate (Flush Free Niacin)? Inositol hexanicotinate (flush free niacin), also known as inositol nicotinate, is a form of v...
What is Cyanocobalamin? Cyanocobalamin is a synthetic form of Vitamin B12. Vitamin B12 is an essential, water-soluble vitamin that is abundantly p...
What Is Sulbutiamine? Sulbutiamine is a synthetic derivative of thiamine (vitamin B1). Sulbutiamine Benefits Sulbutiamine may help in maintaining h...


text:
Vt & NY bats winter in Maine to survive white nose
"Noah's Ark" strategy might serve as last ditch effort to save bats
Scientists working to help bats survive the devastating disease known as white nose syndrome say a number of infected bats spent the winter in an abandoned Maine bunker.In an experiment aimed at finding alternative strategies to save infected bats, 30 hibernating little brown bats were taken from caves in Vermont and New York and transported to the bunker on the Aroostook National Wildlife Refuge in Limestone, Maine.Vermont scientists say that of the 30 bats, nine survived, a higher survival rate than expected by bats in the wild. The bats were transported back to Vermont and New York and released in the wild last week.Scientists say the so-called Noah's Ark strategy might serve as a last resort to save some bat species from local extinction.
MONTPELIER, Vt. —
Scientists working to help bats survive the devastating disease known as white nose syndrome say a number of infected bats spent the winter in an abandoned Maine bunker.
In an experiment aimed at finding alternative strategies to save infected bats, 30 hibernating little brown bats were taken from caves in Vermont and New York and transported to the bunker on the Aroostook National Wildlife Refuge in Limestone, Maine.
Vermont scientists say that of the 30 bats, nine survived, a higher survival rate than expected by bats in the wild. The bats were transported back to Vermont and New York and released in the wild last week.
Scientists say the so-called Noah's Ark strategy might serve as a last resort to save some bat species from local extinction.


If your product is connected to the power supply, check if the lights on the controller are on.
A continuous blue LED on shows your product is connected to the local Wi-Fi network. Check to see if your smartphone is connected to the same Wi-Fi network.
If the blue LED flashes it means it is trying to connect to the local Wi-Fi network but something is stopping it. Try to move the Twinkly nearer to the router to solve the problem.


Oral pathology deals with the identification and management of diseases affecting the oral and surrounding regions.
What are some of the conditions ?
This group of diseases presents as vesicles in the oral region. They may be caused by viruses eg herpes or a result of immunological defects such as pemphigoid.
Most ulcers are a result of trauma and will resolve spontaneously. Others occur as a result of immunological dysfunction eg apthous ulcers. Persistent ulcers may be of a sinister nature especially if located at the lateral border of the tongue.
A cyst is defined as a fluid filled cavity lined with cells. Most cysts and tumours in the jaws develop from dormant cells that form the tooth substance and their surrounding tissues. They may be locally aggressive and are capable of destroying large areas of bone. The more common cysts include dentigerous cysts, odontogenic keratocysts and radicular cysts. The most common tumour is the ameloblastoma. Treatment involved marsupialisation of the ameoloblastoma. The lesion shrank to the size of a small grape. This lesion was then excised, thus reducing the morbidity associated with resection.
Mucosal conditions may be a result of immunological defects such as lichen planus. It may also occur as a result of infection eg candidiasis. Candidiasis or fungal infection before treatment. Other conditions may represent a pre-cancerous state as in the case of leukoplakia. A biopsy is often required in order to ascertain the diagnosis before treatment can be prescribed. Candidiasis or fungal infection after treatment.
The most common salivary gland disease is that a salivary stone which can cause much pain and discomfort during eating. Removal of the stone will result in alleviation of the symptoms. (On the right) Stone at the opening of a salivary duct causing pain on eating. Other diseases of the salivary glands include tumours and infections.
The scope of oral pathology is myriad and difficult to list. Should you believe that you are suffering from any of these conditions, please make an appointment for our surgeon to assess your condition.


There will be a $15 fee applied to your account for each additional pet brought in that was not scheduled on the original appointment.
Payment, in full, is required at the time services are rendered.
We accept Visa, MasterCard, Discover, Care Credit, and Scratch Pay.
Unfortunately, like other medical practices, we also have the occasional late arrival or no-show appointment. We do try and manage these as apart from impacting on those who have arrived on time for their appointment; it has a negative impact on the medical team. Often there are cases being treated in hospital and the staff attempts to schedule their day to ensure that they can meet the needs of those patients and scheduled appointments. Arriving late or not showing at all impacts on their ability to manage treatments for these pets. As a result, we do have a few rules and guidelines that the staff work with in terms of late arrivals and no-shows.
Please be aware that our hospital has a policy wherein if you are more than 10 minutes late for your scheduled appointment time, we will be adjusting our schedule and moving you to the walk in/urgent care service when you arrive. While we certainly understand that delays may happen, we feel we have an obligation and responsibility to do our very best to service clients and patients who are here for their appointments on time. We strongly encourage all of our clients to arrive at least 15 minutes early for scheduled appointments if possible, as we do have administrative paperwork that will need to be completed, especially if you are a new client or have a new pet.
When you schedule an appointment and fail to show or call and notify us of your inability to keep this appointment time, it impacts our ability to service clients who may be here already waiting to see a doctor. If you fail to show or call and notify us about difficulty in keeping your appointment time 3 x, you will be prohibited from scheduling an appointment with one of our doctors. We will still see your pet, but it will be as a walk in/urgent care arrival only following three subsequent no-shows for appointments. We recognize that your pet may be in need of medical attention and we do not wish to turn your pet away, but you will be subject to any wait times associated with the walk in/urgent care service.
A $15 fee will be applied to your account for canceling an appointment with less than 24 hours notice.
We ask that you give us 72 hours (3 days) notice if you need to cancel a surgical appointment. This allows us to offer that appointment to another patient and helps our doctors and staff efficiently navigate the day. We understand that cancelled appointments happen for a number of reasons and that some are unexpected and unavoidable. If we determine that this policy is being abused we reserve the right to discontinue reserving our staff's time for that client or we may charge a $25 cancellation fee. We thank you in advance for abiding by these policies and helping us keep our clients, staff, and patients on schedule and safe!!
We have created this policy out of respect for those clients who are waiting to have their pets be seen.


text:
#include "HGQuadEntityScripter.h"
#include "HGQuadEntity.h"
#include "HGGraphicScripter.h"
#include "HGISceneEntityScripter.h"

#include "HGCompositeScripter.h"

#define QUADENTITY_METATABLE    "QuadEntityMetatable"
#define QUADENTITY_LUA_NAME     "QuadEntity"

HGNAMESPACE_START

QuadEntity* quadentity_check(lua_State* L, int idx)
{
    QuadEntity* entity = NULL;

    if (lua_isuserdata(L, idx))
        entity = *static_cast<QuadEntity **>(luaL_checkudata(L, idx, QUADENTITY_METATABLE));

    return entity;
}

// push a QuadEntity to Lua
int quadentity_push(lua_State* L, QuadEntity* entity)
{
    int ret = 0;

    if (entity != NULL)
    {
        QuadEntity** udata = static_cast<QuadEntity **>(lua_newuserdata(L, sizeof(QuadEntity*)));
        *udata = entity;
        luaL_getmetatable(L, QUADENTITY_METATABLE);
        lua_setmetatable(L, -2);

        ret = 1;
    }

    return ret;
}

static int quadentity_set_texture(lua_State* L)
{
    QuadEntity* entity = NULL;
    GLuint texID = 0;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2))
        break;

    texID = luaL_checkunsigned(L, 2);

    entity->setTexture(texID);

    BREAK_END;

    return 0;
}

static int quadentity_set_texture_rect(lua_State* L)
{
    QuadEntity* entity = NULL;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2) || !lua_isnumber(L, 3) || !lua_isnumber(L, 4) || !lua_isnumber(L, 5))
        break;

    float x = (float)luaL_checknumber(L, 2);
    float y = (float)luaL_checknumber(L, 3);
    float w = (float)luaL_checknumber(L, 4);
    float h = (float)luaL_checknumber(L, 5);
    entity->setTextureRect(x, y, w, h);

    BREAK_END;

    return 0;
}

static int quadentity_set_color(lua_State* L)
{
    int i = -1;
    QuadEntity* entity = NULL;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2))
        break;

    if (lua_isnumber(L, 3))
        i = (int)luaL_checkinteger(L, 3);

    uint32_t color = luaL_checkunsigned(L, 2);
    entity->setColor(color, i);

    BREAK_END;

    return 0;
}

static int quadentity_set_vertex_alpha(lua_State* L)
{
    int i = -1;
    QuadEntity* entity = NULL;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2))
        break;

    if (lua_isnumber(L, 3))
        i = (int)luaL_checkinteger(L, 3);

    uint8_t alpha = (uint8_t)luaL_checkunsigned(L, 2);
    entity->setVertexAlpha(alpha, i);

    BREAK_END;

    return 0;
}

static int quadentity_set_blend_mode(lua_State* L)
{
    QuadEntity* entity = NULL;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2))
        break;

    int blend = (int)luaL_checkinteger(L, 2);
    entity->setBlendMode(blend);

    BREAK_END;

    return 0;
}

static int quadentity_set_center_point(lua_State* L)
{
    QuadEntity* entity = NULL;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2) || !lua_isnumber(L, 3))
        break;

    float cx = (float)luaL_checknumber(L, 2);
    float cy = (float)luaL_checknumber(L, 3);
    entity->setCenterPoint(cx, cy);

    BREAK_END;

    return 0;
}

static int quadentity_set_width(lua_State* L)
{
    QuadEntity* entity = NULL;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2))
        break;

    float w = (float)luaL_checknumber(L, 2);
    entity->setWidth(w);

    BREAK_END;

    return 0;
}

static int quadentity_set_height(lua_State* L)
{
    QuadEntity* entity = NULL;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2))
        break;

    float h = (float)luaL_checknumber(L, 2);
    entity->setHeight(h);

    BREAK_END;

    return 0;
}

static int quadentity_get_quad(lua_State* L)
{
    int ret = 0;
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
    {
        Quad* q = entity->getQuad();
        quad_push2lua(L, q);
        ret = 1;
    }

    return ret;
}

static int quadentity_get_texture(lua_State* L)
{
    GLuint texID = 0;
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        texID = entity->getTexture();

    lua_pushunsigned(L, texID);

    return 1;
}

static int quadentity_get_texture_rect(lua_State* L)
{
    int ret = 4;
    float x = 0.0f;
    float y = 0.0f;
    float w = 0.0f;
    float h = 0.0f;
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        entity->getTextureRect(&x, &y, &w, &h);

    lua_pushnumber(L, x);
    lua_pushnumber(L, y);
    lua_pushnumber(L, w);
    lua_pushnumber(L, h);

    return ret;
}

static int quadentity_get_color(lua_State* L)
{
    int vidx = 0;
    uint32_t color = 0;
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
    {
        if (lua_isnumber(L, 2))
            vidx = (int)luaL_checkinteger(L, 2);

        color = entity->getColor(vidx);

    }

    lua_pushunsigned(L, color);

    return 1;
}

static int quadentity_get_vertex_alpha(lua_State* L)
{
    int vidx = 0;
    uint8_t alpha = 0;
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
    {
        if (lua_isnumber(L, 2))
            vidx = (int)luaL_checkinteger(L, 2);

        alpha = entity->getVertexAlpha(vidx);
    }

    lua_pushunsigned(L, alpha);

    return 1;
}

static int quadentity_get_blend_mode(lua_State* L)
{
    int blend = BLEND_DEFAULT;
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        blend = (int)entity->getBlendMode();

    lua_pushinteger(L, blend);

    return 1;
}

static int quadentity_get_center_point(lua_State* L)
{
    float x = 0.0f;
    float y = 0.0f;

    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        entity->getCenterPoint(&x, &y);

    lua_pushnumber(L, x);
    lua_pushnumber(L, y);

    return 2;
}

static int quadentity_init_sprite_animation(lua_State* L)
{
    int ret = 0;

    return ret;
}

static int quadentity_deinit_sprite_animation(lua_State* L)
{
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        entity->deInitSpriteAnimation();

    return 0;
}

static int quadentity_play_sprite_animation(lua_State* L)
{
    QuadEntity* entity = NULL;
    bool success = false;

    BREAK_START;

    entity = quadentity_check(L, 1);
    if (entity == NULL)
        break;

    if (!lua_isnumber(L, 2))
        break;

    uint32_t name = luaL_checkunsigned(L, 2);

    success = entity->playSpriteAnimation(name);

    BREAK_END;

    luaH_pushboolean(L, success);

    return 1;
}

static int quadentity_set_frame(lua_State* L)
{
    int ret = 0;

    return ret;
}

static int quadentity_next_frame(lua_State* L)
{
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        entity->nextFrame();

    return 0;
}

static int quadentity_prev_frame(lua_State* L)
{
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        entity->prevFrame();

    return 0;
}

static int quadentity_stop_sprite_animation(lua_State* L)
{
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        entity->stopSpriteAnimation();

    return 0;
}

static int quadentity_pause_sprite_animation(lua_State* L)
{
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        entity->pauseSpriteAnimation();

    return 0;
}

static int quadentity_resume_sprite_animation(lua_State* L)
{
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        entity->resumeSpriteAnimation();

    return 0;
}

static int quadentity_is_sprite_animating(lua_State* L)
{
    bool isAnimating = false;
    QuadEntity* entity = NULL;

    entity = quadentity_check(L, 1);
    if (entity != NULL)
        isAnimating = entity->isSpriteAnimating();

    luaH_pushboolean(L, isAnimating);

    return 1;
}

static int quadentity_new(lua_State* L)
{
    int ret = 0;

    QuadEntity* entity = new QuadEntity();
    if (entity != NULL)
        ret = quadentity_push(L, entity);

    return ret;
}

static int quadentity_delete(lua_State* L)
{
    QuadEntity* entity = quadentity_check(L, 1);
    delete entity;

    return 0;
}

luaL_Reg sQuadEntityRegs[] =
{
    { "setTexture",     quadentity_set_texture },
    { "setTextureRect", quadentity_set_texture_rect },
    { "setColor",       quadentity_set_color },
    { "setVertexAlpha", quadentity_set_vertex_alpha },
    { "setBlendMode",   quadentity_set_blend_mode },
    { "setCenterPoint", quadentity_set_center_point },
    { "setWidth",       quadentity_set_width },
    { "setHeight",      quadentity_set_height },
    { "getQuad",        quadentity_get_quad },
    { "getTexture",     quadentity_get_texture },
    { "getTextureRect", quadentity_get_texture_rect },
    { "getColor",       quadentity_get_color },
    { "getVertexAlpha", quadentity_get_vertex_alpha },
    { "getBlendMode",   quadentity_get_blend_mode },
    { "getCenterPoint", quadentity_get_center_point },
    { "initAnimation",  quadentity_init_sprite_animation },
    { "deInitAnimation",quadentity_deinit_sprite_animation },
    { "playAnimation",  quadentity_play_sprite_animation },
    { "pauseAnimation", quadentity_pause_sprite_animation },
    { "resumeAnimation",quadentity_resume_sprite_animation},
    { "stopAnimation",  quadentity_stop_sprite_animation },
    { "nextFrame",      quadentity_next_frame },
    { "prevFrame",      quadentity_prev_frame },
    { "isAnimating",    quadentity_is_sprite_animating },
    { NULL, NULL }
};

void ScriptRegisterQuadEntity(lua_State* L)
{
    luaL_newmetatable(L, QUADENTITY_METATABLE);
    lua_pushstring(L, "__index");
    lua_pushvalue(L, -2);
    lua_settable(L, -3);
    luaL_setfuncs(L, sIComponentRegs, 0);
    luaL_setfuncs(L, sISceneEntityRegs, 0);
    luaL_setfuncs(L, sQuadEntityRegs, 0);

    lua_pop(L, 1);

    // register QuadEntity
    lua_newtable(L);

    luaH_setfunc2table(L, "new", quadentity_new);
    luaH_setfunc2table(L, "delete", quadentity_delete);

    lua_setglobal(L, QUADENTITY_LUA_NAME);
}

HGNAMESPACE_END

redpajama_set_name: RedPajamaGithub


Perfect Sense Topmatt - EGGER's award winning solution to your demand for highly resilient, super-matt, incredibly velvet-like surfaces that last.
Deep, rich, and velvet to the touch, the incredibly smooth-touch texture of EGGER Topmatt provides a luxurious finish, adding an air of exclusivity to both residential and commercial spaces – perfect for creating a deeply inviting, high-end look within kitchen, bedroom, bathroom and office environments.
We offer samples of PerfectSense Topmatt so you can use them in presentations to help assist your customers.
The samples are an A4 (297 x 210 x 0,8 mm). They are free to order for business users, all you have to do is login or register, and you will be able to order one.


Get practical tips, guides, and more to help you save money.
Learning about saving energy can be fun with interactive websites and educational activities for the home and the classroom. Explore these latest activities.
Discover how your school can benefit from fundraising opportunities. Explore resources to educate children about energy efficiency and conservation.
Mass Save is an initiative sponsored by Massachusetts' natural gas and electric utilities and energy efficiency service providers. We provide a wide range of energy efficiency services to help residents and businesses manage energy use and related costs.
It's easy to find a trained heating and cooling (HVAC) contractor, and insulation and air sealing specialists.


text:
» Alpha Male
The slut guilt neutralizing defense
Last Updated on Sun, 16 Jun 2019 | Alpha Male
Something you have to learn to reframe in order to become expert seducers is the SGND (Slut Guilt Neutralizing Defense)
This means that most of the women will feel guilty when they feel horniness and they will act this out by a behavioral complex of defensive psychological operations that we call here SGND (Slut Guilt Neutralizing Defense.)
These psychological operations are meant to put the blame for her horniness not on herself but on other people and her environment.
A woman acting under the influence of her SGND will try everything she can to make events seem like she would have no responsibility for her horniness.
She will agree to sex with a man only if she can feel that it happened "just as a case" or "because it just happened" or "because of the circumstances" and so on.
The word "slut" is a depreciative term used to describe a woman who has sexual adventures left and right, to describe a woman who loves sex and loves to have it with all the men she likes whenever she wants.
The term is used by other women and by beta men, who feel a sense of envy and hatred for this woman who has the courage to do something very, very natural and healthy: to make love with all men she feels attracted to and to enjoy the maximum satisfaction of her sexual desires.
Notice that I have just applied the technique of reframing the context in a new frame, by saying: "for this woman who has the courage to do something very, very natural and healthy: to make love with all men she feels attracted to and to enjoy the maximum satisfaction of her sexual desires", I put here the concept of "slut" in a new, very positive frame.
Society has made the women feel guilty for their sexual instinct for centuries. The reasons have always been the same: for women, envy and fear of loosing their own men and for men, the fear of the sexual instinct of the woman and fear of being labeled as unsuitable for reproduction.
Remember? The woman is forced to be a selector by Evolution.
By seducing women and learning to observe their behavior, you will notice that the Slut Guilt Neutralizing Defense affects their behavior and that, in order to seduce them, you have to reframe it in a positive context.
An important detail: when an extremely feminine woman has a strong emotion - in this case a guilty feeling - she doesn't feel a responsibility to keep it inside, as a man does, but she puts it into her environment, without feeling guilty for doing so.
You will find many women distressed by the guilty feeling of being a "slut" if she goes to bed with you, but these women will do anything to discharge the responsibility for their feelings - positive and negative -onto your shoulders!
For this reason, the woman waits for you to reframe her feelings and in particular to reframe her Slut Guilt Neutralizing Defense so that you give her a reason to come to bed with you.
An experienced seducer never talks logically with a woman about her feelings. He limits himself to reframing them in a new frame acceptable to her thereby giving her a reason to go to bed with him.
Reframe the slut guilt neutralizing defense of the woman and all her feelings thereby giving her a reason to enjoy her sexuality, of course, with you my friends, you seducers.
Attract And Seduce Models
How To Easily Approach, Attract And Seduce Models! Finally, you can discover the insider secrets of attracting Models the hottest, sexiest and most desired women in the world...even if you don't have the money, success or looks.
Automatic Sexual Chemistry
Cocky and funny - Alpha Male
The phases of a seduction
Being a single man and living alone
The phase of physical contact
The nice guy syndrome - Alpha Male
Your Shoulders, Seducing Women, Sexual Instinct, Her Sexuality
Real Masculinity Manhood


FortiMail fails to sanitize user input. The vulnerability allows an attacker to bypass its input filtering routine, which could result in the execution of the injected script code.
FortiMail 200D, 400C, VM2K, 2000B and 5002B.
Restrict the user input to allowed characters.


You have not found ever need to believe your partner is actually disloyal in a long time residing with each other in Cadder having faith in, however recently, personal unknown behaviours are going for the reason that path as well as recommending there's need to think the actual partner is actually being unfaithful.
Get in touch with Glasgow Private Detectives Matrimonial Surveillance in Glasgow with regard to pictures as well as video clip evidence if you see your lover is actually using brand new vocabulary, brand new behaviours as well as key telephone calls.
Your spouse has been deceptive upon their pc or even cell phone whenever surrounding you in your home in Easterhouse and that he doesn't reveal their security password along with you any longer.
This new, unexpected attitudes could be an indication that your partner is cheating in Glasgow.
Your spouse will be shelling out more hours in the car if it is sitting outside the dwelling in Easterhouse while chatting on the mobile phone.
Don't ever ignore it when your spouse acts suspiciously and differently at home in Cadder as there is always a reason for their actions and you have to act on worries and uncertainties suffered.
Has your lover altered their selection of phrases to combine along with more youthful individuals in Glasgow region?
A signal of shelling out far more with an all new particular person or perhaps masses in Glasgow is utilizing fresh content.
You are typically known for no clue by what your partner does or even exactly where they're going in Glasgow once they take more time from the house in Baillieston which could make concerned about regardless of whether your partner is actually being unfaithful or otherwise.
Contracting Glasgow Private Detectives we can proceed a Matrimonial Surveillance in Glasgow from your address in Cadder may possibly uncover several details of the actions and also connections your husband or wife in City of Glasgow is.


Yellowfin Sole (53%) (Fish), Watercress Sauce (30%), Breadcrumb and Onion Topping (16%), Dried Potato. Watercress Sauce contains: Fish Stock (Water, Fish Extract, Onion Extract, Salt, Parsley Extract, White Pepper Extract, Bayleaf Extract), Watercress (18%), Water, Cream, Cream Cheese, Milk, Margarine (Palm and Rapeseed Oil, Water, Coconut Fat, Salt, Flavouring, Emulsifier: Mono- and Diglycerides of Fatty Acids; Acidity Regulator: Citric Acid; Vitamin A), White Wine, Wheat Flour, Maize Starch, Onion, Salt, White Pepper, Cayenne Pepper. Breadcrumb and Onion Topping contains: Breadcrumbs (65%) (Wheat Flour, Yeast, Dextrose, Salt), Onion (24%), Margarine (Palm and Rapeseed Oil, Water, Coconut Fat, Salt, Flavouring, Emulsifier: Mono- and Diglycerides of Fatty Acids; Acidity Regulator: Citric Acid; Vitamin A), Salt, White Pepper.
ALTHOUGH EVERY EFFORT HAS BEEN TAKEN TO REMOVE ALL BONES, SOME MAY REMAIN!
Yellowfin sole fillets on a watercress sauce with a breadcrumb and onion topping.
Produced and Packed in Denmark for Iceland Foods Ltd., U.K., CH5 2NW.


The pre-wedding hype was well worth it.
Julianne Hough and Brooks Laich have officially tied the knot! After getting hyped up over her "beachlorette" party and brother Derek Hough's celebration intel, the dancer's big day has finally happened, and it was just as great as it was slotted to be. People shared the exclusive details, and you'll definitely want to read up on the Saturday celebration.
Hough and Laich had an outdoor ceremony in front of over 200 family members and friends. Hough wore a custom Marchesa wedding dress and accessorized with Lorraine Schwart jewelry. Meanwhile, Laich looked expectedly dapper, wearing a Brooks Brothers tuxedo. The celebration was planned by Simply Troy Lifestyle + Events and took place in Idaho. "It was really important to me that we had our ceremony outdoors," Laich told People. "Julianne and I are very adventurous and free and wanted the setting to be in nature."
There were plenty of recognizable guests in attendance, including Hough's close friend Nina Dobrev. "I've never seen a man look at a woman like that," the actress shared of the ceremony, adding "It took my breath away." Among the guests was also Derek Hough, Julianne's close brother and dancing partner, who served as a groomsman for the celebration.
As anticipated, Hough and Laich's first dance was well-planned and memorable. Leave it to the dancing star and her enthusiastic hubby to kill it on the dance floor! DJ Derek Monteiro provided the music, and the antique-mirror dance floor was custom-made. The pair's dogs also made an appearance, just as Hough had promised. Both pups wore flower collars and served as the wedding's ring bearers.
Other notable details included the nuptial flower arrangements, which were designed by Wild Bloom. Roses, tulips, grasses, and more surrounded the outdoor space. For the ceremony, guests watched from wooden pews, and at the reception, munched from a delicious Le Catering menu. Prior to the special day, Hough and Laich hosted a Thursday welcome dinner, followed by a lake day before the big moment on Friday. All in all, the wedding was a success for the couple. "I don't think I ever really necessarily dreamed about my wedding as a kid," Hough shared, "But I know I will always cherish and remember it for the rest of my life."
Hough and Laich got engaged in 2015 when the hockey star proposed. "It was magical and sweet. We were at home, romantic, and relaxed, which is perfectly very 'us,'" Hough recalled on her blog. "We stood on the deck, watching the sun set over the lake and listening to our favorite songs. Not even for a second did I expect that beautiful, serene scene was about to become one of the most meaningful moments of our lives."
Congrats to the happy couple—the anticipation was well worth the wait!