<p>Generally, text generators work by modeling the joint distribution of the text by its Bayesian forward decomposition  </p>

<p><span class="math-container">$
\begin{align*}  
p(w_1, w_2, ..., w_n) &amp;= p(w_1) * p(w_2|w_1) * p(w_3|w_2, w_1) *\ ...\ * p(w_n|\{w_i\}_{i&lt;n})\\
&amp;= \prod_{i=1}^n p(w_i|\{w_k\}_{k&lt;i})\\  
\end{align*}
$</span>  </p>

<p>From a modeling perspective, this looks right up RNN's ally, where you can have a state holding information from <span class="math-container">$\{w_k\}_{k&lt;i}$</span> to learn a representation of <span class="math-container">$w_i$</span>  </p>

<p>Now, in your specific case, you're interested in a conditional text-generator, so you are trying to model <span class="math-container">$p(w_1, w_2, ..., w_n | \{v_j\}_j)$</span>, but this same tactic works.  </p>

<p><span class="math-container">$
\begin{align*}  
p(w_1, w_2, ..., w_n| \{v_j\}_j) &amp;= p(w_1|\{v_j\}_j) * p(w_2|w_1, \{v_j\}_j) * p(w_3|w_2, w_1, \{v_j\}_j) *\ ...\ * p(w_n|\{w_i\}_{i&lt;n}, \{v_j\}_j)\\
&amp;= \prod_{i=1}^n p(w_i|\{w_k\}_{k&lt;i}, \{v_j\}_j)\\  
\end{align*}
$</span>  </p>

<p>So, in your RNN or forward-based model, you can use the exact same approach just additionally embed the conditional inputs you have and somehow infuse it into the model (in practice, I have seen this through attention, concatenation, or some other common approach).</p>

<p>My recommendation (depending on the computational power you have) is to take advantage of the recent fad of pre-trained language models. Specifically, ones trained on next word prediction will probably do the job best. A good example is <a href="https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf" rel="nofollow noreferrer">gpt-2</a>, and, if you check out their GitHub, their code is very readable and easy to adjust for adding conditional input in the ways I have described.  </p>

CC BY-SA 4.0
2019-07-28T17:25:52.490Z
2019-07-28T19:02:22.970Z
2019-07-28T19:02:22.970Z
mshlis
<p>I have a little experience in building various models, but I've never created anything like this, so just wondering if I can be pointed in the right direction.</p>

<p>I want to create (in python) a model which will generate text based on multiple inputs, varying from text input (vectorized) to timestamp and integer inputs.</p>

<p>For example, in the training data, the input might include:</p>

<p><code>eventType = ShotMade</code></p>

<p><code>shotType = 2</code></p>

<p><code>homeTeamScore = 2</code></p>

<p><code>awayTeamScore = 8</code></p>

<p><code>player = JR Smith</code></p>

<p><code>assist = George Hill</code></p>

<p><code>period = 1</code></p>

<p>and the output might be (possibly minus the hashtags):
<code>JR Smith under the basket for 2! 8-4 CLE. #NBAonBTV #ThisIsWhyWePlay #PlayByPlayEveryDay #NBAFinals</code></p>

<p>or</p>

<p><code>JR Smith out here doing #WhateverItTakes to make Cavs fans forgive him. #NBAFinals</code></p>

<p>Where is the best place to look to get a good knowledge of how to do this?</p>

<neural-networks><deep-learning><python><generative-model>
Which approach can I use to generate text based on multiple inputs?
Hdot
# Which approach can I use to generate text based on multiple inputs?

(Asked by: username_0 on 2019-07-28)

I have a little experience in building various models, but I've never created anything like this, so just wondering if I can be pointed in the right direction.

I want to create (in python) a model which will generate text based on multiple inputs, varying from text input (vectorized) to timestamp and integer inputs.

For example, in the training data, the input might include:

`eventType = ShotMade`

`shotType = 2`

`homeTeamScore = 2`

`awayTeamScore = 8`

`player = JR Smith`

`assist = George Hill`

`period = 1`

and the output might be (possibly minus the hashtags):
`JR Smith under the basket for 2! 8-4 CLE. #NBAonBTV #ThisIsWhyWePlay #PlayByPlayEveryDay #NBAFinals`

or

`JR Smith out here doing #WhateverItTakes to make Cavs fans forgive him. #NBAFinals`

Where is the best place to look to get a good knowledge of how to do this?

**Tags:** neural-networks, deep-learning, python, generative-model


---

## 1 answers

### Answer by username_1 on 2019-07-28. Score: 2

Generally, text generators work by modeling the joint distribution of the text by its Bayesian forward decomposition

$
\begin{align\*}
p(w\_1, w\_2, ..., w\_n) &= p(w\_1) \* p(w\_2|w\_1) \* p(w\_3|w\_2, w\_1) \*\ ...\ \* p(w\_n|\{w\_i\}\_{i

From a modeling perspective, this looks right up RNN's ally, where you can have a state holding information from $\{w\_k\}\_{k to learn a representation of $w\_i$

Now, in your specific case, you're interested in a conditional text-generator, so you are trying to model $p(w\_1, w\_2, ..., w\_n | \{v\_j\}\_j)$, but this same tactic works.

$
\begin{align\*}
p(w\_1, w\_2, ..., w\_n| \{v\_j\}\_j) &= p(w\_1|\{v\_j\}\_j) \* p(w\_2|w\_1, \{v\_j\}\_j) \* p(w\_3|w\_2, w\_1, \{v\_j\}\_j) \*\ ...\ \* p(w\_n|\{w\_i\}\_{i

So, in your RNN or forward-based model, you can use the exact same approach just additionally embed the conditional inputs you have and somehow infuse it into the model (in practice, I have seen this through attention, concatenation, or some other common approach).

My recommendation (depending on the computational power you have) is to take advantage of the recent fad of pre-trained language models. Specifically, ones trained on next word prediction will probably do the job best. A good example is [gpt-2](https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf), and, if you check out their GitHub, their code is very readable and easy to adjust for adding conditional input in the ways I have described.


---

