text:
\section{Introduction}

Story-telling is on the frontier of current text generation technology: stories must remain thematically consistent across the complete document, requiring modeling very long range dependencies; stories require creativity; and stories need a high level plot, necessitating planning ahead rather than word-by-word generation \cite{wiseman2017challenges}.

We tackle the challenges of story-telling with a hierarchical model, which first generates a sentence called the \emph{prompt} describing the topic for the story, and then conditions on this prompt when generating the story.
Conditioning on the prompt or premise makes it easier to generate consistent stories because they provide grounding for the overall plot. It also reduces the tendency of standard sequence models to drift off topic.

We find that standard sequence-to-sequence (seq2seq) models \cite{sutskever14} applied to hierarchical story generation are prone to degenerating into language models that pay little attention to the writing prompt (a problem that has been noted in other domains, such as dialogue response generation \citep{li:2015}).
This failure is due to the complex and underspecified dependencies between the prompt and the story, which are much harder to model than the closer dependencies required for language modeling (for example, consider the subtle relationship between the first sentence and prompt in Figure \ref{tbl:wp_example}).

To improve the relevance of the generated story to its prompt, we introduce a fusion mechanism \cite{sriram2017} where our model is trained on top of an pre-trained seq2seq model. To improve over the pre-trained model, the second model must focus on  the link between the prompt and the story.
For the first time, we show that fusion mechanisms can help seq2seq models build dependencies between their input and output.

\begin{figure}[t]
  \centering \small
  \begin{tabularx}{\linewidth}{X}
    \textbf{Prompt: }The Mage, the Warrior, and the Priest \\
    \toprule
       \textbf{Story:} A light breeze swept the ground, and carried with it still the distant scents of dust and time-worn stone. The Warrior led the way, heaving her mass of armour and muscle over the uneven terrain. She soon crested the last of the low embankments, which still bore the unmistakable fingerprints of haste and fear. She lifted herself up onto the top the rise, and looked out at the scene before her. [...] \\
	 \midrule
\end{tabularx}
   \caption{Example prompt and beginning of a story from our dataset. We train a hierarchical model that first generates a prompt, and then conditions on the prompt when generating a story.}
 \label{tbl:wp_example}
\end{figure}

Another major challenge in story generation is the inefficiency of modeling long documents with standard recurrent architectures---stories contain 734 words on average in our dataset.
We improve efficiency using a convolutional architecture, allowing whole stories to be encoded in parallel.
Existing convolutional architectures only encode a bounded amount of context \citep{dauphin17}, so we introduce a novel gated self-attention mechanism that allows the model to condition on its previous outputs at different time-scales.

To train our models, we gathered a large dataset of 303,358 human generated stories paired with writing prompts from an online forum. Evaluating free form text is challenging, so we also introduce new evaluation metrics which isolate different aspects of story generation.

Experiments show that our fusion and self-attention mechanisms improve over existing techniques on both automated and human evaluation measures. Our new dataset and neural architectures allow for models which can creatively generate longer, more consistent and more fluent passages of text. Human judges prefer our hierarchical model's stories twice as often as those of a non-hierarchical baseline.

\section{Writing Prompts Dataset}

We collect a hierarchical story generation dataset\footnote{ \url{www.github.com/pytorch/fairseq}} from Reddit's \textsc{WritingPrompts} forum.\footnote{\url{www.reddit.com/r/WritingPrompts/}} \textsc{WritingPrompts} is a community where online users inspire each other to write by submitting story premises, or prompts, and other users freely respond. Each prompt can have multiple story responses. The prompts have a large diversity of topic, length, and detail. The stories must be at least 30 words, avoid general profanity and inappropriate content, and should be inspired by the prompt (but do not necessarily have to fulfill every requirement). Figure~\ref{tbl:wp_example} shows an example.

\begin{table}[t]
  \centering
  \begin{tabular}{ l c }\hline
    \# Train Stories & 272,600 \\
    \# Test Stories & 15,138 \\
    \# Validation Stories & 15,620 \\
    \hline
    \# Prompt Words & 7.7M \\
    \# Story Words & 200M \\
    \hline
    Average Length of Prompts & 28.4 \\
    Average Length of Stories & 734.5 \\
 \hline
\end{tabular}
   \caption{Statistics of \textsc{WritingPrompts} dataset}
 \label{tbl:wp_dataset}
\end{table}

We scraped three years of prompts and their associated stories using the official Reddit API. We clean the dataset by removing automated bot posts, deleted posts, special announcements, comments from moderators, and stories shorter than 30 words.
We use \textsc{NLTK} for tokenization. The dataset models full text to generate immediately human-readable stories.
We reserve 5\% of the prompts for a validation set and 5\% for a test set, and present additional statistics about the dataset in Table~\ref{tbl:wp_dataset}.

For our experiments, we limit the length of the stories to 1000 words maximum and limit the vocabulary size for the prompts and the stories to words appearing more than 10 times each. We model an unknown word token and an end of document token. This leads to a vocabulary size of 19,025 for the prompts and 104,960 for the stories. As the dataset is scraped from an online forum, the number of rare words and misspellings is quite large, so modeling the full vocabulary is challenging and computationally intensive.

\section{Approach}

The challenges of \textsc{WritingPrompts} are primarily in modeling long-range dependencies and conditioning on an abstract, high-level prompt. Recurrent and convolutional networks have successfully modeled sentences \citep{jozefowicz2016exploring,dauphin17}, but accurately modeling several paragraphs is an open problem. While seq2seq networks have strong performance on a variety of problems, we find that they are unable to build stories that accurately reflect the prompts. We will evaluate strategies to address these challenges in the following sections.

\subsection{Hierarchical Story Generation}

High-level structure is integral to good stories, but language models generate on a strictly-word-by-word basis and so cannot explicitly make high-level plans. We introduce the ability to plan by decomposing the generation process into two levels. First, we generate the premise or prompt of the story using the convolutional language model from \citet{dauphin17}. The prompt gives a sketch of the structure of the story. Second, we use a seq2seq model to generate a story that follows the premise. Conditioning on the prompt makes it easier for the story to remain consistent and also have structure at a level beyond single phrases.

\subsection{Efficient Learning with Convolutional Sequence-to-Sequence Model}
The length of stories in our dataset is a challenge for RNNs, which process tokens sequentially.
To transform prompts into stories, we instead build on the convolutional seq2seq model of \citet{gehring17}, which uses deep convolutional networks as the encoder and decoder. Convolutional models are ideally suited to modeling long sequences, because they allow parallelism of computation within the sequence.  In the Conv seq2seq model, the encoder and decoder are connected with attention modules \cite{bahdanau14} that perform a weighted sum of encoder outputs, using attention at each layer of the decoder.

\begin{figure}
   \centering
   \includegraphics[width=0.35\textwidth]{multihead_attention}
 \caption{Self-Attention Mechanism of a single head, with GLU gating and downsampling. Multiple heads are concatenated, with each head using a separate downsampling function.}
 \label{fig:multihead_attention}
\end{figure}

\begin{figure}
   \centering
   \includegraphics[width=0.4\textwidth]{decoder_attention}
 \caption{Multihead self-attention mechanism. The decoder layer depicted attends with itself to gate the input of the subsequent decoder layer.}
 \label{fig:attention_figure}
\end{figure}

\subsection{Modeling Unbounded Context with Gated Multi-Scale Self-attention}
CNNs can only model a bounded context window, preventing the modeling of long-range dependencies within the output story.
To enable modeling of unbounded context, we supplement the decoder with a self-attention mechanism \citep{sukhbaatar2015end,vaswani2017}, which allows the model to refer to any previously generated words. The self-attention mechanism improves the model's ability to extract long-range context with limited computational impact due to parallelism.

\textbf{Gated Attention:} Similar to \citet{vaswani2017}, we use multi-head attention to allow each head to attend to information at different positions. However, the queries, keys and values are not given by linear projections but by more expressive gated deep neural nets with Gated Linear Unit \citep{dauphin17} activations. We show that gating lends the self-attention mechanism crucial capacity to make fine-grained selections.

\textbf{Multi-Scale Attention:} Further, we propose to have each head operating at a different time scale, depicted in Figure~\ref{fig:multihead_attention}. Thus the input to each head is \emph{downsampled} a different amount---the first head sees the full input, the second every other input timestep, the third every third input timestep,  etc. The different scales encourage the heads to attend to different information. The downsampling operation limits the number of tokens in the attention maps, making them sharper.

The output of a single attention head is given by
\begin{align}
h^{L+1}_{0:t}=\mathrm{Linear}\Big(& v(h^L_{0:t-1})\\&\odot \mathrm{softmax} (q(h^L_{0:t})k(h^L_{0:t})^\top) \Big) \nonumber
\end{align}
where $h^L_{0:t}$ contains the hidden states up to time $t$ at layer $L$, and  $q,k,v$ are gated downsampling networks as shown in Figure \ref{fig:multihead_attention}.
Unlike \citet{vaswani2017}, we allow the model to optionally attend to a 0 vector at each timestep, if it chooses to ignore the information of past timesteps (see Figure~\ref{fig:attention_figure}). This mechanism allows the model to recover the non-self-attention architecture and avoid attending to the past if it provides only noise. Additionally, we do not allow the self-attention mechanism to attend to the current timestep, only the past.

\subsection{Improving Relevance to Input Prompt with Model Fusion}
\label{sect:fusion}

Unlike tasks such as translation, where the semantics of the target are fully specified by the source, the generation of stories from prompts is far more open-ended. We find that seq2seq models ignore the prompt and focus solely on modeling the stories, because the local dependencies required for language modeling are easier to model than the subtle dependencies between prompt and story.

We propose a fusion-based approach to encourage conditioning on the prompt. We train a seq2seq model that has access to the hidden states of a pretrained seq2seq model. Doing so can be seen as a type of boosting or residual learning that allows the second model to focus on what the first model failed to learn---such as conditioning on the prompt. To our knowledge, this paper is the first to show that fusion reduces the problem of seq2seq models degenerating into language models that capture primarily syntactic and grammatical information.

The cold fusion mechanism of \citet{sriram2017} pretrains a language model and subsequently trains a seq2seq model with a gating mechanism that learns to leverage the final hidden layer of the language model during seq2seq training. We modify this approach by combining two seq2seq models as follows (see Figure~\ref{fig:fusion_model}):
\begin{align}
g_t &= \sigma (W [h_t^{\textrm{Training}}; h_t^{\textrm{Pretrained}}] + b )\nonumber\\
h_t &= g_t\circ[h_t^{\textrm{Training}}; h_t^{\textrm{Pretrained}}]\nonumber
\end{align}
where the hidden state of the pretrained seq2seq model and training seq2seq model (represented by $h_t$) are concatenated to learn gates $g_t$. The gates are computed using a linear projection with the weight matrix $W$. The gated hidden layers are combined by concatenation and followed by more fully connected layers with GLU activations (see Appendix). We use layer normalization \citep{ba2016} after each fully connected layer.

\begin{figure}
   \centering
   \includegraphics[width=0.4\textwidth]{fusion_model}
 \caption{Diagram of our fusion model, which learns a second seq2seq model to improve a pre-trained model. The separate hidden states are combined after gating through concatenation.}
 \label{fig:fusion_model}
\end{figure}

\section{Related Work}

\subsection{Story Generation}

Sequence-to-sequence neural networks \citep{sutskever14} have achieved state of the art performance on a variety of text generation tasks, such as machine translation \citep{sutskever14} and summarization \citep{rush2015neural}. Recent work has applied these models to more open-ended generation tasks, including writing Wikipedia articles \citep{liu2018} and poetry \citep{zhang2014chinese}.

Previous work on story generation has explored seq2seq RNN architectures \citep{roemmele2016}, but has focused largely on using various content to inspire the stories. For instance, \citet{kiros2015} uses photos to inspire short paragraphs trained on romance novels, and \citet{jain2017} chain a series of independent descriptions together into a short story. \citet{martin2017event} decompose story generation into two steps, first converting text into event representations, then modeling stories as sequences of events before translating back to natural language. Similarly, \citet{harrison2017toward} generate summaries of movies as sequences of events using an RNN, then sample event representations using MCMC. They find this technique can generate text of the desired genre, but the movie plots are not interpretable (as the model outputs events, not raw text). However, we are not aware of previous work that has used hierarchical generation from a textual premise to improve the coherence and structure of stories.

\subsection{Hierarchical Text Generation}

Previous work has proposed decomposing the challenge of generating long sequences of text into a hierarchical generation task. For instance, \citet{li2015hierarchical} use an LSTM to hierarchically learn word, then sentence, then paragraph embeddings, then transform the paragraph embeddings into text. \citet{yarats2017hierarchical} generate a discrete latent variable based on the context, then generates text conditioned upon it.

\subsection{Fusion Models}

Previous work has investigated the integration of language models with seq2seq models. The two models can be leveraged together without architectural modifications: \citet{ramachandran2016} use language models to initialize the encoder and decoder side of the seq2seq model independently, and \citet{chorowski2016} combine the predictions of the language model and seq2seq model solely at inference time. Recent work has also proposed deeper integration. \citet{gulcehre2015} combined a trained language model with a trained seq2seq model to learn a gating function that joins them. \citet{sriram2017} propose training the seq2seq model given the fixed language model then learning a gate to filter the information from the language model.

\section{Experimental Setup}

\subsection{Baselines}

We evaluate a number of baselines:

(1) Language Models: Non-hierarchical models for story generation, which do not condition on the prompt. We use both the gated convolutional language (GCNN) model of \citet{dauphin17} and our additional self-attention mechanism.

(2) seq2seq: using LSTMs and convolutional seq2seq architectures, and Conv seq2seq with decoder self-attention.

(3) Ensemble: an ensemble of two Conv seq2seq with self-attention models.

(4) KNN: we also compare with a KNN model to find the closest prompt in the training set for each prompt in the test set. A TF-IDF vector for each prompt was created using \textsc{fasttext} \citep{bojanowski2016} and \textsc{faiss} \citep{faiss} was used for KNN search. The retrieved story from the training set is limited to 150 words to match the length of generated stories.

\begin{table}[t]
  \centering
  \begin{tabular}{ l p{18mm} p{18mm} }\hline
    \bf{Model} & \bf{Valid \newline Perplexity} & \bf{Test \newline Perplexity}\\ \hline\hline
    Conv seq2seq & 45.27 & 45.54 \\
    + self-attention & 42.01 & 42.32 \\
    + multihead & 40.12 & 40.39 \\
    + multiscale & 38.76 & 38.91 \\
    + gating & \textbf{37.37} & \textbf{37.94} \\
 \hline
\end{tabular}
   \caption{Effect of new attention mechanism. Gated multi-scale attention significantly improves the perplexity on the \textsc{WritingPrompts} dataset.}
 \label{tbl:attention_ppl}
\end{table}

\subsection{Fusion Training}

To train the fusion model, we first pretrain a Conv seq2seq with self-attention model on the \textsc{WritingPrompts} dataset. This pretrained model is fixed and provided to the second Conv seq2seq with self-attention model during training time. The two models are integrated with the fusion mechanism described in Section~\ref{sect:fusion}.

\subsection{Training}

We implement models with the fairseq-py library in PyTorch. Similar to \citet{gehring17}, we train using the Nesterov accelerated gradient method \citep{sutskever13} using gradient clipping \citep{pascanu13}. We perform hyperparameter optimization on each of our models by cross-validating with random search on a validation set. We provide model architectures in the appendix.

\begin{table*}[!t]
  \centering
  \begin{tabular}{ l c c c }\hline
    \bf{Model} & \bf{\# Parameters (mil)} & \bf{Valid Perplexity} & \bf{Test Perplexity}\\ \hline\hline
    GCNN LM & 123.4 & 54.50 & 54.79 \\
    GCNN + self-attention LM & 126.4 & 51.84 & 51.18 \\
    \hline
    LSTM seq2seq & 110.3 & 46.83 & 46.79 \\
    Conv seq2seq & 113.0 & 45.27 & 45.54 \\
    Conv seq2seq + self-attention & 134.7 & 37.37 & 37.94 \\
    Ensemble: Conv seq2seq + self-attention & 270.3 & 36.63 & 36.93 \\
    Fusion: Conv seq2seq + self-attention & 255.4 & \textbf{36.08} & \textbf{36.56} \\
 \hline
\end{tabular}
   \caption{Perplexity on \textsc{WritingPrompts}. We  dramatically improve over standard seq2seq models.}
 \label{tbl:ppl}
\end{table*}

\subsection{Generation}

We generate stories from our models using a \emph{top-k random sampling} scheme. At each timestep, the model generates the probability of each word in the vocabulary being the likely next word. We randomly sample from the $k = 10$ most likely candidates from this distribution. Then, subsequent timesteps generate words based on the previously selected words. We find this sampling strategy substantially more effective than beam search, which tends to produce common phrases and repetitive text from the training set \cite{vijayakumar2016diverse, shao2017generating}. Sentences produced by beam search tend to be short and generic. Completely random sampling can introduce very unlikely words, which can damage generation as the model has not seen such mistakes at training time. The restriction of sampling from the 10 most likely candidates reduces the risk of these low-probability samples.

For each model, we tune a temperature parameter for the softmax at generation time. To ease human evaluation, we generate stories of 150 words and do not generate unknown word tokens.

For prompt generation, we use a self-attentive GCNN language model trained with the same prompt-side vocabulary as the sequence-to-sequence story generation models. The language model to generate prompts has a validation perplexity of  63.06. Prompt generation is conducted using the top-k random sampling from the 10 most likely candidates, and the prompt is completed when the language model generates the end of prompt token.

\begin{figure}[!t]
   \centering
   \includegraphics[width=0.5\textwidth]{triple_pairing_tight}
 \caption{Human accuracy at pairing stories with the prompts used to generate them. People find that our fusion model significantly improves the link between the prompt and generated stories.
 }
 \label{fig:pairing_task}
\end{figure}

\begin{figure}[!t]
   \centering
   \includegraphics[width=.5\textwidth]{likelihood_ranking_tight}
 \caption{Accuracy of prompt ranking. The fusion model most accurately pairs prompt and stories.
 }
 \label{fig:likelihood_ranking}
\end{figure}

\begin{figure}[!t]
   \centering
   \includegraphics[width=.5\textwidth]{knn_v_fusion_tight}
 \caption{Accuracy on the prompt/story pairing task vs. number of generated stories. Our generative fusion model can produce many stories without degraded performance, while the KNN can only produce a limited number relevant stories.}
 \label{fig:knn_v_fusion}
\end{figure}

\begin{table}[!t]
  \centering
  \begin{tabular}{ l c }\hline
    \bf{Model} & \bf{Human}\\
    & {\bf Preference}\\
    \hline\hline
    Language model &  32.68\% \\
    Hierarchical Model & \textbf{67.32\%} \\
 \hline
\end{tabular}
   \caption{Effect of Hierarchical Generation. Human judges prefer stories that were generated hierarchically by first creating a premise and creating a full story based on it with a seq2seq model.}
 \label{tbl:hierarchical_generation}
\end{table}

\begin{figure*}
   \centering
   \includegraphics[width=.95\textwidth]{example_sentence_gates}
 \caption{Average weighting of each model in our Fusion model for the beginning of the generated story for the prompt \emph{Gates of Hell}. The fused model (orange) is primarily used for words which are closely related to the prompt, whereas generic words are generated by the pre-trained model (green).
 }
 \label{fig:gate_comparison}
\end{figure*}

\subsection{Evaluation}

We propose a number of evaluation metrics to quantify the performance of our models. Many commonly used metrics, such as \textsc{BLEU} for machine translation or \textsc{ROUGE} for summarization, compute an n-gram overlap between the generated text and the human text---however, in our open-ended generation setting, these are not useful. We do not aim to generate a specific story; we want to generate viable and novel stories. We focus on measuring both the fluency of our models and their ability to adhere to the prompt.

For automatic evaluation, we measure \emph{model perplexity} on the test set and \emph{prompt ranking accuracy}. Perplexity is commonly used to evaluate the quality of language models, and it reflects how fluently the model can produce the correct next word given the preceding words. We use prompt ranking to assess how strongly a model's output depends on its input. Stories are decoded under 10 different prompts---9 randomly sampled prompts and 1 true corresponding prompt---and the likelihood of the story given the various prompts is recorded. We measure the percentage of cases where the true prompt is the most likely to generate the story. In our evaluation, we examined 1000 stories from the test set for each model.

For human evaluation, we use Amazon Mechanical Turk to conduct a \emph{triple pairing task}. We use each model to generate stories based on held-out prompts from the test set. Then, groups of three stories are presented to the human judges. The stories and their corresponding prompts are shuffled, and human evaluators are asked to select the correct pairing for all three prompts. 105 stories per model are grouped into questions, and each question is evaluated by 15 judges.

Lastly, we conduct human evaluation to evaluate the importance of \emph{hierarchical generation} for story writing. We use Amazon Mechanical Turk to compare the stories from hierarchical generation from a prompt with generation without a prompt. 400 pairs of stories were evaluated by 5 judges each in a blind test.

\section{Results}
We analyze the effect of our modeling improvements on the \textsc{WritingPrompts} dataset.

\paragraph{Effect of Hierarchical Generation:}  We explore leveraging our dataset to perform hierarchical story generation by first using a self-attentive GCNN language model to generate a prompt, and then using a fusion model to write a story given the generated prompt. We evaluate the effect of hierarchical generation using a human study in Table~\ref{tbl:hierarchical_generation}. 400 stories were generated from a self-attentive GCNN language model, and another 400 were generated from our hierarchical fusion model given generated prompts from a language model. In a blind comparison where raters were asked to choose the story they preferred reading, human raters preferred the hierarchical model 67\% of the time.

\paragraph{Effect of new attention mechanism:} Table~\ref{tbl:attention_ppl} shows the effect of the proposed additions to the self-attention mechanism proposed by \citet{vaswani2017}. Table \ref{tbl:ppl} shows that deep multi-scale self-attention and fusion each significantly improve the perplexity compared to the baselines. In combination these additions to the Conv seq2seq baseline reduce the perplexity by 9 points.

\paragraph{Effect of model fusion:} Results in Table~\ref{tbl:ppl} show that adding our fusion mechanism substantially improves the likelihood of human-generated stories, and even outperforms an ensemble despite having fewer parameters.
We observe in Figure \ref{fig:pairing_task} that fusion has a much more significant impact on the topicality of the stories.
In comparison, ensembling has no effect on people's ability to associate stories with a prompt, but adding model fusion leads improves the pairing accuracy of the human judges by 7\%.
These results suggest that by training a second model on top of the first, we have encouraged that model to learn the challenging additional dependencies to relate to the source sequence. To our knowledge, these are the first results to show that fusion has such capabilities.

\paragraph{Comparison with Nearest Neighbours:}
Nearest Neighbour Search (KNN) provides a strong baseline for text generation.
Figure \ref{fig:pairing_task} shows that the fusion model can match the performance of nearest neighbour search in terms of the connection between the story and prompt.
The real value in our generative approach is that it can produce an unlimited number of stories, whereas KNN can never generalize from its training data.
To quantify this improvement, Figure \ref{fig:knn_v_fusion} plots the relevance of the \emph{k}th best story to a given prompt; the performance of KNN degrades much more rapidly.

\begin{table*}[!t]
  \centering \small
  \begin{tabularx}{\linewidth}{X}
    \textbf{\emph{Generation with Fusion of Conv seq2seq + Self-Attention}} \\
    \toprule
    \textbf{Example Prompt 1: } Aliens start abducting humans \\
    \toprule
       It has been two weeks, and the last of my kind has gone. It is only a matter of time until there will be nothing left. I'm not sure what the hell is going on... I can't think. I can hear a distant scream. I think of a strange, alien sound. I try to speak, but am interrupted by something, something that feels like a drum, I ca not tell. I mean I'm just a little bit older than an average human. But I can, and I can feel the vibrations . I hear the sound of a ship approaching. The ground quakes at the force of the impact, and a loud explosion shatters the silence. \\
	 \midrule
\end{tabularx}
~\\
  \begin{tabularx}{\linewidth}{X}
    \textbf{Example Prompt 2: } The scientists have discovered something terrible .  \\
    \toprule
       The scientist stood there, a little dazed as he stared.

       ``What is it?'' He asked.

       ``This...this...Thing...This is a virus. A chemical that can destroy entire planet and it is a very small, complex, chemical that could destroy any planet.'' The scientist replied. His lab assistant looked down at the tablet.

       ``I've just discovered it. I can't believe it. It looks like it's made of some sort of chemical that's very dangerous.''

       `` Well, there's a virus on the ground. It's very effective...''
       `` I can't believe what it is.'' he said, pointing to the scientist .

       `` We don't know what this thing is. We haven't seen anything like it . We can't even see anything like this. '' Dr. Jones stared at the scientist for a moment.

       ``What do you mean what does it do ?''

       ``It...It 's a monster.'' \\
	 \midrule
\end{tabularx}
~\\
  \begin{tabularx}{\linewidth}{X}
    \textbf{Example Prompt 3: } The silence before the storm comes  \\
    \toprule
        ``I'm sorry,'' she said ``I can't do this anymore.''

        She sat in the grass, the grass blowing behind her as she sat staring at the sky. Her eyes were still closed but she was still staring at it. Her hand was still shaking, her eyes closed as she watched the rain roll in and out of view. She could see the world through her eyelids now. Her hands were still shaking like she hadn't eaten in days. Her fingers were still clenched tight and her fingers were now shaking. She wasn't even sure if she was going to cry or if she just wanted to cry. It wasn't until she was close enough to hear the words.

        ``I ...I just...I need to...I ...'' She was crying now and then, the rain was picking up. Her lips were shaking now and she could see that the storm was getting closer and closer. Her heart stopped when she heard the thunder. She knew it was coming. The storm was \\
	 \midrule
\end{tabularx}
~\\~\\
  \begin{tabularx}{\linewidth}{X}
  \textbf{\emph{Generation with Language Model + Self-Attention Baseline}} \\
    \toprule
    \textbf{Example 1: }\\
    \toprule
       I've always been the curious type. Growing up in the ghetto, I've been around for more than a year now. I still haven't heard it on my own. I'm not sure if I'm a good person. But I'm not the only one. I've been to the hospital, but it's not my first time. I've been to my psychiatrist's journals before. ``Hey, are you ... are you okay?'' I asked. ``Yes'' I said. ``What's wrong?'' she replied, ``I was just ... Doing something.'' She said, ``I've never been a bad person.'' \\
	 \midrule
\end{tabularx}
~\\
  \begin{tabularx}{\linewidth}{X}
    \textbf{Example 2: }\\
    \toprule
    The man was an accountant. He had to be. He had to be the next president. I looked back over the top and saw that his wife was crying in the kitchen. I looked at the clock. It seemed to be coming slower, but I knew if I did it would not be long before I was in my own home. I wasn't sure. I had a hard time finding the right words to say. I was about to leave when he suddenly became angry and began talking to me. ``Hello, sir, I'm John. What is your name?'' ``My name is Manuel and I'm a journalist.'' I said \\
	 \midrule
\end{tabularx}
   \caption{Example stories generated by the proposed hierarchical fusion approach compared to stories generated by a language model. Stories generated by the fusion model relate to the desired prompt and show increased coherence between sentences and ability to stay on one topic compared to the language modeling baseline.}
 \label{tbl:example_stories}
\end{table*}

\section{Discussion}

\subsection{Generation Quality}

Our proposed fusion model is capable of generating unique text without copying directly from the training set. When analyzing 500 150-word generated stories from test-set prompts, the average longest common subsequence is 8.9. In contrast, the baseline Conv seq2seq model copies 10.2 words on average and the KNN baseline copies all 150 words from a story in the training set.

Figure~\ref{fig:gate_comparison} shows the values of the fusion gates for an example story, averaged at each timestep. The pretrained seq2seq model acts similarly to a language model  producing common words and punctuation. The second seq2seq model learns to focus on rare words, such as \emph{horned} and \emph{robe}.

However, the fusion model has limitations. Using random sampling to generate can produce errors. For example, \emph{can't} is tokenized to \emph{ca} \emph{n't}, and the model occasionally produces the first token but misses the second. A similar error is after one line of dialogue, the model may move to another line of dialogue without generating a newline token. A further obstacle is repetition. The model focuses frequently on what it has recently produced, which leads to the generation of similar text multiple times.

In the generation of prompts using the GCNN language model, we find that prompts are fairly generic compared to human prompts. Language models often struggle to model rare words accurately, as the probability distribution over the next word is dominated by more common words. This tends to produce similar prompts, particularly at the start --- we see many prompts that start with \emph{the man}. In contrast, many of the human prompts are very unique (e.g. prompting stories in fantasy worlds such as Harry Potter and Game of Thrones) and the language model rarely produces the specific vocabulary required by these settings.

\subsection{Use of Attention}

We analyze the encoder-decoder attention in the fusion model and find that unlike attention maps in machine translation, where each decoder timestep tends to attend to a different word on the encoder-side, the attention map for each decoder timestep looks similar and focuses mainly on salient words in the prompt. We further look at the usage of the self-attention layers within the decoder. While they could be leveraged to look at words generated very far in the past, at many timesteps the self-attention focuses on the recent past.

\section{Conclusion}

We have collected the first dataset for creative text generation based on short writing prompts. This new dataset pushes the boundaries of text generation by requiring longer range dependencies and conditioning on an abstract premise. Building on this dataset, we show through automatic and human evaluation that novel hierarchical models, self-attention mechanisms and model fusion significantly improves the fluency, topicality, and overall quality of the generated stories.

\bibliographystyle{acl_natbib}

redpajama_set_name: RedPajamaArXiv


text:
\section{Introduction}

The ordinary textbook formulation of quantum mechanics is very abstract. Its starting point: 'The state of a physical system is a normalized vector in a separable Hilbert space' has lead to an extremely rich theory, a theory which has not been refuted by any experiment and whose predictions range over an extremely wide varity of situations. Nevertheless, it is still unclear how this state concept should be interpreted.

Many conferences on quantum foundation have been arranged in recent years, but this has only implied that the number of new interpretations have increased, and no one of the old have died out. In two of these conferences, a poll among the participants was carried out [1, 2]. The result was an astonishing disagreement on several simple and fundamental questions. One of these questions was whether quantum theory should be interpreted as an objective theory of the world (the ontological interpretation) or if it only expresses our knowledge of the world (the epistemic interpretation). According to Webster's Unabridged Dictionary, the adjective 'epistemic' means 'of or pertaining to knowledge, or the conditions for acquiring it'.

In Spekkens [3] a toy model was developed, a toy model which can be taken as a strong argument in favour of an epistemic interpretation of quantum mechanics. Partly based on this and partly on other basic premises, Fuchs [4, 5, 6] and others have argued for various versions of Quantum Bayesianism, a radical interpretation where the subjective observer plays an important part. See also the philosophical discussion by Timpson [7] and the popular account in [8].

The present paper takes as a point of departure that in some sense or other the epistemic interpretation should be important for the fundametal issues of the quantum world. The next question then arises: Can one find a new and more intuitive \textit{foundation} of quantum theory, a foundation related to the epistemic interpretation? It is my view that such a foundation must have some relation to statistical inference theory, another scientific fundament, which gives tools for   wide variety of empirical investigations, and which in its very essence is epistemic. It should also be a kind of decision theory, related to decisions taken in everyday life.

Every epistemic question together with an answer to this question imply at least two decisions: First a decision to focus in the selection of the question to ask. Then data are collected, and one ends up with a decision on which answer to the focused question one should give from these data. This last decision is covered by traditional decision theory, but one needs to pay more attention to the process of focusing.

A very tentative answer to the question on foundation given above, is then yes, but there is still more work to be done before such a foundation can be completely established. In [9] I will aim to carry out the arguments in more detail.

\section{Background and definitions}

The Quantum Bayesianism is founded on an observer's belief, quantified by a Bayesian probability. I want to relate my state concept to the notion of \emph{certain} belief, which I call knowledge. The knowledge will be associated with an agent or with a group of communicating agents, and his/her/their knowledge will be knowledge about what I will call an e-variable.

An epistemic process is any process under which an agent or a group of communicating agents obtain knowledge about a physical system. In general there are many ways by which one can obtain knowledge about the world or about aspects of the world. In a given situation the observer has some background, in terms of his history, in terms of his physical environment, and in terms of the concepts that he is able to use in analyzing the situation. All this may limit his ability to obtain knowledge.

A \emph{conceptual variable} is any variable related to the physical system, defined by an agent or by a group of communicating agents. The variable may be a scalar, a vector or belong to a larger space.

An \emph{e-variable} or epistemic conceptual variable $\theta$ is a conceptual variable associated with an epistemic process: Before the process the agent (or agents) has (have) no knowledge about $\theta$; after the process he/they has/have some knowledge, in the simplest case full knowledge: $\theta=u_k$. Here $u_k$ is one of the possible values that $\theta$ can take. In this paper it is mostly assumed for simplicity that $\theta$ is discrete, which it will be in the elementary quantum setting below. For a continuous variable $\theta$, knowledge on the e-variable will be taken to mean a statement to the effect that $\theta$ belongs to some given set, an interval if $\theta$ is a scalar.

The e-variable concept is a generalization of the parameter concept as used in statistical inference, introduced by Fisher [10], and today incorporated in nearly all applications of statistics. In statistics, a parameter $\theta$ is usually an index in the statistical model for the observations, and the purpose of an empirical investigation is to obtain statements about $\theta$, in terms of point estimation, confidence interval estimation or conclusion from the testing of hypotheses. The parameter is often associated with a hypothetical infinite population. My e-variable will also be allowed to be associated with a finite physical system, a particle or a set of particles. But, in the same way as with a parameter, the purpose of any empirical investigation will be to try to conclude with some statement, a statement expressed in terms of an e-variable $\theta$.

It is important that not all conceptual variables are e-variables. A conceptual variable $\phi$ is called \emph{inaccessible} if there is no epistemic process by which one can get accurate knowledge about it. An example from the area of quantum mechanics is $\phi=(\xi,\pi)$, where $\xi$ is the position of a particle, and $\pi$ is the momentum. Also macroscopic examples abound, for instance connected to counterfactual situations [9]. Another physical example is $\phi=(\lambda^x ,\lambda^y ,\lambda^z )$, where $\lambda^a$ is the component of an angular momentum or a spin for a particle or a system of particles, the component in direction $a$. Here each $\lambda^a$ is an accessible conceptual variable, an e-variable, but the vector $\phi$ is inaccessible.

In such cases, where a vector of e-variables is inaccessible, it is equivalent to say that the components are complementary. Since introduced by Bohr, the concept of complementarity has played a fundamental and important role in quantum mechanics.

It is essential to stress that the inaccessible conceptual variables above are not hidden variables. Variables like $\phi$ are just mathematical variables, but variables upon which group actions may be defined. $\phi=(\xi,\pi)$ may be subject to Galilean transformations, time translations or changes of units, while $\phi=(\lambda^x ,\lambda^y ,\lambda^z )$ may be subject to rotations. This will of course also induce transformation of the components, the e-variables. Important transformations in the group of rotations of $\phi$ are: 1) Those leading to the change in the values of $\lambda^x$ (or of any other fixed component); 2) Those leading to an exchange of $\lambda^x$ and $\lambda^y$ (or any other pair of components).

\section{The maximal symmetrical epistemic setting}

The purpose of this paper is to indicate how one can derive the essence of the quantum formalism from reasonable assumptions in a setting which generalizes the angular momentum setting described above.

Let in general $\phi$ be an inaccessible conceptual variable taking values in some measurable space $(\Phi, \mathcal{F})$, and let $\lambda^a =\lambda^a (\phi)$ be accessible
functions for $a$ belonging to some index set $\mathcal{A}$. I will repeat that a conceptual variable is accessible if it in the given context can be estimated with arbitrary
accuracy by \emph{some} experiment. Technically I will without further mention assume that all functions defined on $\Phi$ are
measurable. To begin with, I will assume that the functions $\lambda^a$ are maximal, and also that they can be transformed into each other by an automorphism in $\Phi$.
\bigskip

\textbf{Assumption 1} \textit{ a) Consider the partial ordering defined by $\alpha <\beta$ iff $\alpha =f(\beta)$ for some function $f$. Under this partial
ordering each $\lambda^a (\phi)$ is maximally accessible.}

\textit{b) For $a\ne b$ there is an invertible transformation $g_{ab}$ such that $\lambda^b (\phi)=\lambda^a (g_{ab}\phi)$.}
\bigskip

Note that the partial ordering in a) is consistent with accessibility: If $\beta$ is accessible and $\alpha=f(\beta)$, then $\alpha$ is accessible. Also, $\phi$ is an
upper bound under this partial ordering. The existence of maximal accessible conceptual variables follows then from Zorn's lemma.

For the angular momentum case, the transformation $g_{ab}$ means exchange of the component $\lambda^a$ and the component $\lambda^b$. In this  process, one can imagine that the vector $\phi$ is decomposed as $\phi=(\lambda^a,\lambda^b,\lambda^c)$, where the axis $c$ is orthogonal to the $(a,b)$-plane, and $g_{ab}$ then changes the sign of $\lambda^c$, moving along a great circle.

Below, I will often single out a particular index $0\in\mathcal{A}$. Then a), given b), can be formally weakened to the assumption that $\lambda^0 (\phi)$ is
maximally accessible, and b) can be weakened to the existence for all $a$ of an invertible transformation $g_{0a}$ such that
$\lambda^a (\phi)=\lambda^0 (g_{0a}(\phi))$.  Take $g_{ab}=g_{0a}^{-1}g_{0b}$.

Even though $\phi$ is inaccessible, it is possible to operate on $\phi$ with functions, in particular group actions. The group of automorphisms on $\Phi$, all
transformations of elements $\phi$, always exists from a mathematical point of view, and one can imagine many subgroups of this group. Some of these
will now be defined.
\bigskip

\textbf{Definition 1} \textit{For each $a$, let $\tilde{G}^a$ be the group of automorphisms on $\Lambda^a$, the space upon which $\lambda^a$ varies.
For $\tilde{g}^a \in \tilde{G}^a$ let $g^a$ be any transformation on $\Phi$ for which $\tilde{g}^a \lambda^a (\phi)=\lambda^a (g^a \phi)$.}
\bigskip

It is easily verified that 1) For fixed $\tilde{g}^a$ the transformations $g^a$ form a set which is invariant under decompositions and inversion. 2) For fixed $a$ the transformations $g^a$ form a group $G^a$.

Thus the group $G^a$ is the group transforming values of $\lambda^a$ into the same or other values of this e-variable, and the corresponding group
$\tilde{G}^a$ is the group of all transformations of these values.

Obvious consequences of Definition 1 are that $\tilde{G}^a$ is transitive over $\Lambda^a$ and that $G^a$ is transitive over $\Phi$.

Now single out a fixed index $0\in \mathcal{A}$.
\bigskip

\textbf{Definition 2} \textit{Let $G$ be the group of transformations generated by $G^0$ and the transformations $g_{0a}$, $a\in \mathcal{A}$.}
\bigskip

It is easily verified that $G^a = g_{0a}^{-1} G^0 g_{0a}.$ Together with $g_{ab}=g_{0a}^{-1}g_{0b}$ this implies that $G$ also is the group generated by
$G^a$, $a\in \mathcal{A}$ and $g_{ab}$, $a,b \in \mathcal{A}$.

Now I want to introduce the further
\bigskip

\textbf{Assumption 2} \textit{a) The group $G$ is a locally compact topological group, and satisfies weak assumptions such that an invariant
measure $\rho$ on $\Phi$ exists.}

\textit{b) The group generated by products of elements of $G^a ,G^b ,...$; $a,b,...\in\mathcal{A}$ is equal to $G$.}
\bigskip

Assumption 2a) is a technical one, needed in the next section. Note that $G$ is defined in terms of transformations upon $\Phi$, so that the topology must be introduced
in terms of these transformations. Technically this can for instance be achieved by assuming $\Phi$ to be a metric space with metric $d$, and letting $g_n\rightarrow g$ if
$\mathrm{sup}_\phi d(g_n \phi, g\phi)\rightarrow 0$. Concerning Assumption 2b), it follows from
$g^a g^b ...=g_{0a}^{-1}g^{0}g_{0a}g_{0b}^{-1}g^{0'}g_{0b},...$, where $g^a \in G^a , g^b \in G^b ,...$ and $g^{0}, g^{0'},...\in G^0$, that the group of
products is contained in $G$. That it is equal to $G$, is an assumption on the richness of the set $\mathcal{A}$ or the richness of $G^0$.

Here is a proof that assuption 2b) holds in the angular momentum/ spin case: Let $c$ be any direction perpendicular to the midline between $a$ and $b$. Let $g^c$ be the group element changing the sign of $\lambda^c$, the $c$-component of $\phi$, moving along a great cicle. Then $g_{ab}=g^c$.

The setting described here, where Assumption 1 and Assumption 2 are satisfied, includes many quantum mechanical situations including angular momenta and spins.
I will call it \emph{the maximal symmetrical epistemic setting}.

An important special case is when each $\Lambda^a$ is discrete. Then $\tilde{G}^a$ is the group of permutations of elements of $\Lambda^a$.

\section{Quantum reconstruction}

Fix $0\in\mathcal{A}$, and let $H$ be the Hilbert space
\[H=\{f\in L^2(\Phi,\rho):\ f(\phi)=\tilde{f}(\lambda^0(\phi))\ \mathrm{for\ some\ }\tilde{f}\}.\]
I will assume that the $\lambda^a$'s are discrete. Then $H$ is separable. If the $\lambda^a$'s take $d$ different values, $H$ is $d$-dimensional. Since all separable Hilbert spaces are isomorphic, it is enough to arrive at the quantum formulation on this $H$.
\bigskip

\textbf{Lemma 1} \textit{The values $u_k^a$ of $\lambda^a$ can always be arranged such that $u_k^a=u_k$ is the same for each $a$ ($k=1, 2, ...$).}
\bigskip

\underline{Proof.} By Assumption 1
\[\{\phi:\lambda^b=u_k^b\}=\{\phi:\lambda^a(g_{ab}\phi)=u_k^b\}=g_{ba}(\{\phi:\lambda^a(\phi)=u_k^b\}).\]
The sets in brackets on the lefthand side here are disjoint with union $\Phi$. But then the sets in brackets on the righthand side are disjoint with union $g_{ab}(\Phi)=\Phi$, and this implies that $\{u_k^b\}$ gives all possible values of $\lambda^a$.
\smallskip

Now I am able to formulate the main result of this paper:
\bigskip

\textbf{Theorem 1} \textit{a) For every $a$, $u_k$ and associated with every indicator function $I(\lambda^a (\phi)=u_k)$ there is a vector $|a;k\rangle\in H$, which is unique in the sense that $|a;k\rangle\ne|b;j\rangle$ for all $a, b, j, k$ except in the trivial case $a=b, j=k$. This inequality is interpreted to mean that there is no phase factor $e^{i\gamma}$ such that $|a;k\rangle=e^{i\gamma}|b;j\rangle$.}

\textit{b) For each $a$ the vectors $|a;k\rangle$ form an orthonormal basis for $H$.}
\bigskip

This gives us the possibility to interprete the vectors $|a;k\rangle$, corresponding to the indicators $I(\lambda^a (\phi)=u_k)$, as follows:
\smallskip

\textit{1) The question 'What is the value of $\lambda^a$?' has been focused on.}

\textit{2) Through an epistemic process we have obtained the answer '$\lambda^a=u_k$'.}
\smallskip

When a ket vector is defined, a corresponding bra vector can be defined. The operator corresponding to $\lambda^a$ can be defined as
\[A^a =\sum_k u_k |a;k\rangle\langle a;k|.\]
In the maximal setting this has non-degenerate eigenvalues.

\section{Proof under an extra assumption}

Let $U$ be the left regular representation of $G$ on $L^2(\Phi,\rho)$: $U(g)f(\phi)=f(g^{-1}\phi)$. It is well known that this is a unitary representation. We will seek a corresponding representation of $G$ on the smaller space $H$.

In the following, recall that upper indices as in $g^a$ indicate variables related to a particular $\lambda^a$, here a group element of $G^a$. Also recall that $0$ is a fixed index in $\mathcal{A}$. Lower indices as in $g_{ab}$ has to do with the relation between two different $\lambda^a$ and $\lambda^b$.
\bigskip

\textbf{Proposition 1} \textit{a) A (multivalued) representation $V$ of $G$ on the Hilbert space $H$ can always be found.}

\textit{b) There is an extended group $G'$ such that $V$ is a univalued representation of $G'$ on $H$.}

\textit{c) There is a homomorphism $G'\rightarrow G^0$ such that $V(g')=U(g^0)$. If $g'\ne e'$ in $G'$, then $g^0\ne e$ in $G^0$.}
\bigskip

\underline{Proof.} a) For each $a$ and for $g^a\in G^a$ define $V(g^a)=U(g_{0a})U(g^a)U(g_{a0})$. Then $V(g^a)$ is an operator on $H$, since it is equal to $U(g_{0a}g^a g_{a0})$, and $g_{0a}g^a g_{a0}\in G^0 =g_{0a}G^a g_{a0}$. For a product $g^a g^b g^c$ with $g^a\in G^a$, $g^b \in G^b$ and $g^c \in G^c$ we define $V(g^a g^b g^c)=V(g^a )V(g^b)V(g^c)$, and similarly for all elements of $G$ that can be written as a finite product of elements from different subgroups.

Let now $g$ and $h$ be any two elements in $G$ such that $g$ can be written as a product of elements from $G^a, G^b$ and $G^c$, and similarly $h$ (the proof is
similar for other cases.) It follows that $V(gh)=V(g)V(h)$ on these elements, since the last factor of $g$ and the first factor of $h$ either must belong to
the same subgroup or to different subgroups; in both cases the product can be defined by the definition of the previous paragraph. In this way we see that $V$ is
a representation on the set of finite products, and since these generate $G$ by Assumption 2b) , it is a representation of $G$.

Since different representations of $g$ as a product may give different solutions, we have to include the possibility that $V$ may be multivalued.
\smallskip

b) Assume as in a) that we have a multivalued representation $V$ of $G$. Define a larger group $G'$ as follows: If $g^a g^b g^c =g^d g^e g^f$, say,
with $g^k \in G^k $ for all $k$, we define $g_1'=g^a g^b g^c $ and $g_2'=g^d g^e g^f$. Let $G'$ be the collection of all such new elements that can be written as a formal product of elements $g^k \in G^k$.
The product is defined in the natural way, and the inverse by for example $(g^a g^b g^c )^{-1}=(g^c )^{-1}(g^b )^{-1}(g^a)^{-1}$. By Assumption 2b), the group $G'$
generated by this construction must be at least as large as $G$. It is clear from the proof of a) that $V$ also is a representation of the larger group $G'$ on $H$, now a one-valued representation.
\smallskip

c)  Consider the case where $g' =g^a g^b g^c$ with $g^k \in G^k$. Then by the proof of a):
\[V(g') = U(g_{0a}) U(g^a)U(g_{a0}) U(g_{0b}) U(g^b)U(g_{b0})U(g_{0c}) U(g^c)U(g_{c0}) \]
\[=U(g_{0a}g^a g_{a0} g_{0b}g^b g_{b0} g_{0c} g^c g_{c0})
=U(g^0),\]
where $g^0 \in G^0$. The group element $g^0$ is unique since the decomposition $g'=g^a g^b g^c$ is unique for $g'\in G'$. The proof is similar for other decompositions. By the construction, the mapping $g'\rightarrow g^0$ is a homomorphism.

Assume now that $g^0 =e$ and $g'\ne e'$. Since $U(g^0)\tilde{f}(\lambda^0(\phi))=\tilde{f}(\lambda^0((g^0)^{-1}(\phi)))$, it follows from $g^0=e$ that $U(g^0)=I$ on $H$.
But then from what has been just proved, $V(g')=I$, and since $V$ is a univariate representation, it follows that $g'=e'$, contrary to the assumption.
\bigskip

Now choose an orthonormal basis for $H$: $f_1,...,f_d$ where $f_k(\phi)=\tilde{f}_k(\lambda^0(\phi))$, and where the interpretation of $f_k$ is that $\lambda^0=u_k$. Write $|0;k\rangle=f_k(\phi)$.

Introduce the assumption that the representation $V$ really is multivalued, in a sense to be made precise below. Let $g_{0a1}'$ and $g_{0a2}'$ be two different elements of the group $G'$, both corresponding to $g_{0a}$ of $G$. Define $g_a'=(g_{0a1}')^{-1}g_{0a2}'$. Then $g_a'\ne e'$ in $G'$. By the homomorphism of Proposition 1c), let $g_a'\rightarrow g_a^0$. Then $g_a^0\ne e$ in $G^0$. Now define
\[|a;k\rangle = \tilde{f}_k(\lambda^0(g_a^0\phi))=U((g_a^0)^{-1})|0;k\rangle.\]

\textbf{Assumption 3}
a) $g_{0a1}'$ and $g_{0a2}'$ can be chosen so that $\tilde{g}_a^0\lambda\ne\lambda$ for all $\lambda\in\Lambda^0$ when $a\ne 0$.

b) $g_{0a1}', g_{0a2}', g_{0b1}'$ and $g_{0b2}'$ can be chosen so that $\tilde{g}_a^0\lambda\ne \tilde{g}_b^0\lambda$ for all $\lambda\in\Lambda^0$ when $a\ne b$.
\bigskip

\underline{Proof of Theorem 1 under Assumption 3.}
Let $j\ne k$. I will first prove that the basis functions $f_1,...,f_d$ can be chosen so that $|a;k\rangle\ne |0;j\rangle$ for all $a$. Here and below, inequality of state vectors is interpreted to mean that they can not be made equal by introducing a phase factor. To this end, choose $\tilde{f}_j$ and $\tilde{f}_k$ in such a way that there exists an $\lambda_0^j$ such that $\tilde{f}_k(\lambda)\ne\tilde{f}_j(\lambda_0^j)$ for all $\lambda\in \Lambda^0$. A possible choice is the indicator function $\tilde{f}_j(\lambda)=I(\lambda=u_j)$, but there are other choices. Then for any fixed $g$, $\tilde{f}_{kg}$ defined by $\tilde{f}_{kg}(\lambda^0(\phi))=\tilde{f}(\lambda^0(g\phi))$ is different from $\tilde{f}_j$, and $|a;k\rangle\ne |0;j\rangle$ for all $a$.

Next fix $k$ and choose $\tilde{f}_k$ in such a way that there exists a $\lambda_1^k$ such that $\tilde{f}_k(\lambda)\ne\tilde{f}_k(\lambda_1^k)$ when $\lambda\ne\lambda_1^k$ and $\lambda\in \Lambda^0$. A possible choice is again $\tilde{f}_k(\lambda)=I(\lambda=u_k)$. For $|a;k\rangle\ne|0;k\rangle$ it is sufficient that $\tilde{f}_k(\lambda^0(g_{a}^0\phi))\ne\tilde{f}_k(\lambda^0(\phi))$ for at least one $\phi$, and this will hold if $\lambda^0(g_a^0\phi)\ne\lambda^0(\phi)$ for all $\phi$. This is equivalent to Assumption 3a).

The proof that $|a;k\rangle\ne |b;j\rangle$ (except in the trivial case $a=b, k=j$) holds under assumption 3b), is similar.

The vectors $|0;k\rangle$ are chosen to be an orthonormal basis for $H$. Since $|a;k\rangle=U|0;k\rangle$ for some unitary $U$, it follows that the vectors $|a;k\rangle$ form an orthonormal basis.
\bigskip

Assumption 3 is not satisfied for the spin/ angular momentum case. Nevertheless it is shown in [16] that Theorem 1 holds also for this case.

\section{The general symmetrical epistemic setting}

Go back to the definition of the maximal symmetrical epistemic setting. Let again $\phi$ be the inaccessible conceptual variable and let
$\lambda^a=\lambda^a (\phi)$ for $a\in \mathcal{A}$ be the maximal accessible conceptual variables satisfying Assumption 1. Let the corresponding
induced groups $G^a$ and $G$ satisfy Assumption 2. Let us assume either the spin/ angular momentum case or that Assumption 3 holds. Finally, let $t^{a}$ for each $a$ be an arbitrary real function on the range of $\lambda^a$,
and assume that we instead of focusing on $\lambda^a$, focus on $\theta^a =t^{a}(\lambda^a)$ for each $a\in \mathcal{A}$. I will call this the
symmetrical epistemic setting; the e-variables $\theta^a$ are no longer maximal.

Consider first how to define the quantum states $|a;k\rangle$. We are no longer interested in the full information on $\lambda^a$, but keep first the Hilbert
space as in Section 4. For definiteness look at the case where Assumption 3 holds. Then let $f_{k}^a (\phi)=I(t^{a}(\lambda^a)=t^{a}(u_{k}))=I(\theta^a =u_{k}^a)$, where $u_{k}^a =t^{a}(u_{k})$.  We let
again $g_{0a1}'$ and $g_{0a2}'$ be two distinct elements of $G'$ such that $g_{0ai}'\rightarrow g_{0a}$, define $g_a'=(g_{0a1}')^{-1}g_{0a2}'$ and let $g_a'\rightarrow g_a^0$ under the homomorphism of Proposition 1c). Then define
\[|a;k\rangle =V((g_a')^{-1})U(g_{0a}) f^a_k(\phi) =U((g_a^0)^{-1})f_{ka}^0(\phi)=f_{ka}^0(g_a^0\phi)\]
with $f_{ka}^0(\phi)=I(t^a(\lambda^0(\phi))=u_k^a)$.
\bigskip

\textbf{Interpretation of the state vector $|a;k\rangle$:} \textit{1) The question: 'What is the value of $\theta^a$?' has been posed. 2) We have obtained
the answer $\theta^a =u_k^a$. Both the question and the answer are contained in the state vector.}
\bigskip

From this we may define the operator connected to the e-variable $\theta^a$:
\[A^a =\sum_k u_k^a |a;k\rangle\langle a;k|=\sum_{k} t^{a}(u_{k})|a;k\rangle\langle a;k|.\]
Then $A^a$ is no longer necessarily an operator with distinct eigenvalues, but  $A^a$ is still Hermitian: $A^{a\dagger}=A^a$.
\bigskip

\textbf{Interpretation of the operator $A^a$:} \textit{ This gives all possible states and all possible values corresponding to the accessible e-variable
$\theta^a$.}
\bigskip

The projectors $|a;k\rangle\langle a;k|$  and hence the ket vectors $|a;k\rangle$ are no longer uniquely determined by $A^a$: They can be transformed
arbitrarily by unitary transformations in each space corresponding to one eigenvalue. As long as the focus is only on $\theta^a$, or $A^a$, I will redefine $|a;k\rangle$ by allowing it to be subject to
such transformations. These transformed eigenvectors all still correspond to the same eigenvalue, that is, the same observed value of $\theta^a$ and they give
the same operators $A^a$. In particular, in the maximal symmetric epistemic setting I will allow an arbitrary constant phase factor in the functions defining the
$|a;k\rangle$'s.

As an important example of the general construction, assume that $\lambda^a$ is a vector: $\lambda^a =(\theta^{a_1},...,\theta^{a_m})$. Then one can
write a state vector corresponding to $\lambda^a$ as
\[|a;k\rangle =|a_1;k_1\rangle \otimes ...\otimes |a_m;k_m\rangle\]
in an obvious notation, where $a=(a_1,...,a_m)$ and $k=(k_1,...,k_m)$. The different $\theta$'s may be connected to different subsystems. This construction can also be made, and is of great interest, in the spin/ angular momentum case.

So far I have kept the same groups $G^a$ and $G$ when going from $\lambda^a$ to $\theta^a =t^{a}(\lambda^a)$, that is from the maximal symmetrical
epistemic setting to the general symmetrical epistemic setting. This implies that the (large) Hilbert space will be the same. A special case occurs if $t^{a}$ is a
reduction to an orbit of $G^a$. Then the construction of this section can also be carried
with a smaller group action acting just upon an orbit, resulting then in a smaller Hilbert space. In the example of the previous paragraph it may be relevant to
consider one Hilbert space for each subsystem. The large Hilbert space is however the correct space to use when the whole system is considered.

These considerations are highly relevant when considering several observers. One single observer may have access to just a few subsystems. In addition he has
his own context. From this context one can define what is his accessible and inaccessible conceptual variables. In the same way a group of several observers may through
verbal communication arrive at a common context, and from this context one can define their accessible and inaccessible  conceptual variables. Imagine that these observers
together observe a particular physical system, and consider the corresponding Hilbert space.
\bigskip

\textbf{Assumption 4.}
 \textit{For most physical system at some particular time one can either imagine an observer or a group of communicating observers
for which the assumptions of the symmetrical epistemic setting are satisfied. In many cases all real and imagined observers agree on the physical observations,
in which case this is considered part of the ontic world.}
\bigskip

It is important that for every physical system we can consider imagined observers.  All real
and  imagined observers may agree on observations like charge or non-relativistic mass of a particle. For these observations we do not need the formalism of
quantum mechanics. For other variables the construction of this section  applies.

At any time we can also imagine non-communicating observers. For each observer then the general symmetrical setting may apply. Particular state vectors in
each observer's Hilbert space might then be linear combinations of primitive state vectors of the form $|a_{1};k_{1}\rangle \otimes ... \otimes |a_{s};k_{s}\rangle$.
As is well known, when linear combinations of primitive state vectors can not be reduced to a primitive form, they are called entangled state vectors. Entangled state vectors play an important role in many
discussions of quantum mechanics.

When the physical system available to an observer $A$ is subject to a non-maximal symmetrical setting resulting in an operator
$A^a =\sum_{k}u_{k}^a |a;k\rangle \langle a;k|$, we can always imagine another observer $B$, communicating with $A$, with an operator
$B^b =\sum_{j}v_{j}^b |b;j\rangle \langle b;j|$, so that the operator
$A^a \otimes B^a =\sum_{k,j}u_{k}^a v_{j}^b (|a;k\rangle \otimes |b;j\rangle)(\langle a;k|\otimes \langle b;j|)$ corresponds to a maximal setting,
i.e., has distinct eigenvalues.

Assumption 4 is assumed to hold for nearly all physical systems, and also for combinations of physical systems. Through the imagined observers the
construction of Section 5 can be carried out, and for each case a Hilbert space can be constructed.

Connected to any general physical system, one may have several e-variables $\theta$ and corresponding operators $A$. In the ordinary quantum formalism there is well-known theorem saying that, in my formulation, $\theta^{1},...,\theta^{n}$ are compatible, that is, there exists an
e-variable $\lambda$ such that $\theta^{i}=t^{i}(\lambda)$ for some functions $t^{i}$ if and only if the corresponding operators commute:
\[[A^{i},A^{j}]\equiv A^{i}A^{j}-A^{j}A^{i}=0\ \mathrm{for\ all}\ i,j.\]
(See Holevo [13].) Compatible e-variables may in principle be estimated simultaneously with arbitrary accuracy.

\section{Further developments}

In [9] I discuss a number of further implications of a possible Hilbert space formulation of epistemic processes.

First, the density operators are defined by
\[\sigma^a=\pi_k^a \sum_k |a;k\rangle\langle a;k|.\]
Here the $\pi_k^a$ are either prior probabilities, posterior probabilities, or derived from confidence distributions. Traditionally, statisticians are divided into Bayesians and frequentists; I want to include both cultures. Prior  distributions and posterion distributions are Bayesian concepts for parameters. Confidence distributions is a recent corresponding frequentist concept for distribution of parameters [14].

For simple epistemic states, Born's formula can be written as
\[ P(\theta^b=u_j^b|\theta^a=u_k^a)=|\langle a;k|b;j\rangle|^2.\]
This formula is proved in [9] from 1) An extension of the likelihood principle from statistics; 2) An assumption of rationality (Dutch book principle); 3) Paul Busch's version of Gleason's theorem [15].

In statistics, the likelihood is the joint density of the observation, and the likelihood principle states that observations with the same likelihood produce the same experimental evidence. This principle can be proved from fairly obvious presuppositions. The Dutch book principle states: For a rational actor, no choice of payoffs in a series of bets shall lead to sure loss for the bettor. Thus in this approach, Born's formula, which is formulated as an independent axiom in most texbooks, can be seen to follow from quite intuitive assumptions.

In [9] also the quantum mechanics for position and momentum is discussed. The Schr\"{o}dinger equation $i\hbar \frac{d}{dt}|\psi\rangle_t =H|\psi\rangle_t$ for one-dimensional position is shown to follow from 1) Position as an inaccessible stochastic process; 2) Conditioning both on past events and on future events; 3) Elements from Nelson's stochastic mechanics.

EPR and violation of Bell's inequality for simple epistemic states can be given interesting interpretations. The conditionality principle from statistics states: When doing a coin toss to choose an experiment, an observer should condition upon the value of the coin toss. Imagine that this principle is used both for Alice and for Bob. This implies that their observations are interpreted as some sort of experiment. That is, observation means inference: Decision on what has been observed. Under this condition the EPR paradox turns out to be no paradox any more, and the violation of Bell's inequality can be simply explained. The assumption of \emph{realism} is violated by this version of quantum mechanics; the assumption of locality is not violated.

It is well known from the Quantum Bayesian literature that the so-called paradox of Wigner's friend is no paradox any more with an epistemic interpretation of the state vector.

\section{Concluding remarks}

Despite its enormous success, there is still no consensus among physicists about what quantum theory is saying about the nature of reality. In my view, its foundation should be related to the process of obtaining knowledge about the world. It seems to be possible to formalize this in a precise mathematical way.

In this theory something resembling the formal state definition of traditional quantum mechanics is derived from a simpler and more natural conceptual basis. This derivation is done under certain technical assumptions. The use of group theory seems to be essential.

It also seems to be important that the approach here and in [9] uses ideas both from the quantum mechanical tradition and from the statistical tradition. In the long run one might perhaps hope for some sort of synthesis in the foundation of the two sciences.

Since space is limited here, I prepare an extended version [16], where I will discuss the validity of Theorem 1 for the spin/angular momentum case, and where the relation to other approaches towards the foundation of quantum mechanics and to the interpretation of quantum mechanics will be discussed.

\section*{References}

\begin{description}

\item[]
[1]	Schlossbauer~M, Koller~J and Zellinger~A 2013  A snapshot of fundamental attitudes toward quantum mechanics (\textit{Preprint} quant-ph/1301.1069)

\item[]
[2]	Norsen~T and Nelson~S 2013 Yet another snapshot of fundamental attitudes toward quantum mechanics (\textit{Preprint} quant-ph/1306.4646)

\item[]
[3]     Spekkens~R~W 2007 \textit{Phys. Rev.} A \textbf{75} 032110

\item[]
[4]  Fuchs~C~A 2010 QBism, the Perimeter of Quantum Bayesianism (\textit{Preprint} quant-ph/1003.5209)

\item[]
[5]  Fuchs~C~A and Schack~R 2011 \textit{Foundations of Physics} \textbf{41}, 345

\item[]
[6] Fuchs~C~A, Mermin~N~D and Schack~R 2013 An introduction to QBism with application to the locality of quantum mechanics (\textit{Prepront} quant-ph/1311.5253)

\item[]
[7] Timpson~C~G 2008 \textit{Studies in History and Philosophy of Modern Physics} \textbf{39} 579

\item[]
[8] von Baeyer~H~C 2013 \textit{Scientific American} \textbf{308} (6), 38

\item[]
[9] Helland~I~S In preparation \textit{Epistemic Processes. A Basis for Statistics and for Quantum Mechanics.} Book manuscript

\item[]
[10] Fisher~ R~A 1922 On the mathematical foundation of theoretical statistics. Reprinted in Fisher~R~A 1950 \textit{Contribution to Mathematical Statistics} (New York: Wiley)

\item[]
[11] Helland~I~S 2004 \textit{International Statistical Review} \textbf{72}, 409

\item[]
[12] Helland~I~S  2010 \textit{Steps Towards a Unified Basis for Scientific Models and Methods} (Singapore: World Scientific)

\item[]
[13] Holevo~A~S 1982 \textit{Probabilistic and Statistical Aspects of Quantum Theory} (Amsterdam: North Holland)

\item[]
[14] Schweder~T and Hjort~N~L 2002 \textit{Scandinavian Journal of Statistics} \textbf{29}, 309

\item[]
[15] Busch~P 2003 \textit{Physical Review Letters} \textbf{91} (12), 120403

\item[]
[16] Helland~I~S 2015 Approach towards the quantum formulation from assumptions of epistemic processes. Article in preparation

\end{description}

\end{document}