text:
\section{Introduction}
The pervasive usage of social media allows users to participate in online activities that produce a large amount of social links. The task of link analysis aims to understand the factors influencing the link formation. The findings from link analysis have been exploited to help a variety of social media mining tasks such as trust prediction~\cite{gbeigi} and community detection~\cite{backstrom2006group}.

Link analysis in unsigned social networks (or networks with only positive links) has been extensively studied. For example, users tend to create positive links to others who share certain similarity with them (Homophily), or two individuals geographically closer are more likely to become friends (Confounding). However, social networks in social media can contain both positive and negative links. Examples of such signed social networks include Epinions with trust/distrust links, and Slashdot with friend/foe links. The existence of negative links in signed networks challenges many existing concepts and properties of unsigned networks. For example, negative links present substantially distinct properties from positive links~\cite{szell2010multirelational} and Homophily of unsigned networks are not directly applicable to signed networks~\cite{tang2014distrust}. Therefore, link analysis for signed networks cannot be performed by simply extending that of unsigned networks, and hence requires further effort.

There is a few recent works dedicated to link analysis in signed networks based on social theories (e.g., balance theory and status theory). On the one hand, these methods suggest that achievements from link analysis can benefit various signed network mining tasks such as link prediction; meanwhile they indicate that theories from social sciences can guide link analysis in signed networks. On the other hand, these methods merely focus on the topological structures, which could suffer from the link sparsity problem while completely ignore other pervasively available resources such as users' personalities and emotional information. Link analysis with leveraging these available resources not only greatly advance current research but also provide deep understandings of user behaviors in signed networks.

The success of link analysis with social theories motivates us to adopt theories from psychology and social science to guide our link analysis in this paper, namely,  {\it Emotional Information}, {\it Diffusion of Innovations} and {\it Individual Personality}.  The first theory suggests that emotions of individuals toward each other are strong indicators of positive and negative links~\cite{bewsell2012distrust,dunn2005feeling}; the second one considers positive and negative link formation as a problem of an individual's probability of adopting a new behavior following her friends' behaviors~\cite{rogers2010diffusion}; and the last theory derived from~\cite{asendorpf1998personality}, suggests that people personality implies individuals' tendency to forming the positive and negative links.  Guided by these social theories, many interesting findings are revealed from our link analysis.
\section{Social Theories}
In this section, we explain the social theories \textit{Emotional Information}, \textit{Diffusion of Innovations} and \textit{Individual Personality}.
\subsubsection{Emotional Information Theory}
Users express their emotions toward each other in various ways. For example, in Slashdot users do so via commenting and replying to the posts; while product-review sites such as Epinions provide the rating mechanisms for their users to express their emotions. As a result, emotional information is pervasively available in social media no matter how they are exposed~\cite{beigi2016exploiting}. According to the sociologist, emotions of individuals toward each other, are strong indicators of positive and negative links~\cite{bewsell2012distrust,dunn2005feeling}. For example, happiness and satisfaction are indicators of positive emotions which could lead to the positive links; while negative emotions such as anger and fear imply the negative relations.
\subsubsection{Diffusion of Innovation Theory}
The problem of positive and negative link creation could relate to the well-studied topic of {\it diffusion of innovation}~\cite{rogers2010diffusion} by treating it as a behavior that spreads through the network. Therefore it turns into a new problem of analyzing an individual's tendency to follow her friends'  behaviors toward other users.
\subsubsection{Individuals Personality Theory}
According to~\cite{golbeck2011predicting}, people's behavior in social media could be a good indicator of their personality and the reasons are two-fold.  First, social media websites allow for free interaction and exposing viewpoints by providing an appropriate platform to satisfy users' basic psychological needs. Second, there is an ample amount of data regarding normative behaviors of individuals which guarantees fair analysis of individual's personality. Moreover, research from sociology suggest that people personality determines individuals' propensity to positive and negative relations~\cite{asendorpf1998personality}. For example, optimistic users are more grateful and receive more social support, and hence have higher chances in establishing and receiving positive links. In contrast, pessimists have negative attitudes, expect the worst of people, and treat positive events as flukes. Consequently, these individuals often receive or give negative links. Hence, considering user's personality information could be very helpful for studying the problem of signed link formation. Although there are many types of personalities~\cite{mccrae1998introduction}, in this paper we only consider two common ones, i.e., optimism and pessimism.
\section{Datasets}
We collect two large online signed social networks datasets from Epinions and Slashdot where individuals can express their opinions toward each other besides creating positive and negative links. Availability of product-rating data in Epinions and individuals' post reviews data in Slashdot, can help approximating user's personality in these websites.  For example optimistic users are likely to give higher rating scores in Epinions and likewise, more positive reviews on posts in Slashdot, whereas pessimistic users tend to give lower scores in Epinions and more negative comments in Slashdot~\cite{asendorpf1998personality}. Accordingly, we can define optimism and pessimism scores, based on the ratings in Epinions, and the users interactions in Slashdot.
\subsection{Epinions} Epinions is a product-review website where users can establish trust/distrust relationships toward each other. We treat each relation as either positive or negative links and construct user-user matrix ${\bf F}$ where ${\bf F}_{ij} = 1$ if user $i$ trusts user $j$, and ${\bf F}_{ij} = -1$ if user $i$ distrusts user $j$. Also, ${\bf F}_{ij} = 0$ where the information is missing. Users can express opinions toward each other by rating how helpful their reviews are, from 1 to 6. From these ratings, we construct the positive and negative emotion matrices ${\bf P}$ and ${\bf N}$ as follows: (1) we consider low helpfulness ratings $\{1,2\}$ as negative emotions, high helpfulness ratings $\{4,5,6\}$ as positive emotions and the rating $3$ as neutral and (2) for each pair of users $(u_i,u_j)$, we compute the number of positive and negative emotions expressed from $u_i$ to $u_j$ to create ${\bf P}_{ij}$ and ${\bf N}_{ij}$.

We define the optimism and pessimism in Epinions as follows. Let $\mathcal{I} = \{I_1, I_2, \ldots, I_M\}$ be the set of $M$ items and assume $r_{ik}$ denotes the rating score from $u_i$ to $I_k$ with $r_{ik} = 0$ indicating that $u_i$ has not rated $I_k$ yet.  Also, consider $\overline{r}_k$ as the average rating score of the $k$-th item rated by users. In this paper, we consider scores in $\{1,2,3\}$ as low and $\{4,5\}$ as high scores. We use $\mathcal{OL}_i$ to denote the set of items with low average rating scores and rated by $u_i$:
\begin{align}
\mathcal{ OL}_i = \{I_k \mid r_{ki}\neq 0 \wedge \overline{r}_k\leq 3\} \nonumber
\end{align}
We further use $\mathcal{OH}_i$ to denote the set of items which are scored high by $u_i$, and meanwhile have low average scores.  $\mathcal{OH}_i$ can be formally defined as:
\begin{align}
\mathcal{ OH}_i = \{I_k \mid I_k \in  \mathcal{ OL}_i \wedge {r}_{ik} > 3\} \nonumber
\end{align}
Intuitively, the more frequently user $u_i$ has rated above the average, the more optimistic she is. Therefore we define the optimism score for $u_i$ as ${\bf o}_i = \frac{|\mathcal{ OH}_i|}{|\mathcal{ OL}_i|}$.

Similarly we use $\mathcal{PH}_i$ to denote the set of items with high average rating scores and rated by $u_i$,
\begin{align}
\mathcal{PH}_i = \{I_k \mid r_{ki} \neq 0 \wedge \overline{r}_k > 3\} \nonumber
\end{align}
Let $\mathcal{PL}_i$ denotes the subset of items from $\mathcal{PH}_i$, which are given low rates by $u_i$:
\begin{align}
\mathcal{ PL}_i = \{I_k \mid I_k \in  \mathcal{ PH}_i \wedge {r}_{ik} \leq 3\} \nonumber
\end{align}
We define the pessimism score $u_i$ as: ${\bf p}_i = \frac{|\mathcal{ PL}_i|}{|\mathcal{ PH}_i|}$.

\subsection{Slashdot} Slashdot is a technology-related news platform which allows users to tag each other as either `friend' or `foe'. Similar to the Epinions, we construct user-user matrix ${\bf F}$ from the positive and negative links in the network. Likewise, users can express their opinions and comments toward each other by annotating the articles posted by each other. In a similar way to the Epinions, using positive and negative opinions, we create user-user positive and negative emotion matrices ${\bf P}$ and ${\bf N}$ by computing the number of positive or negative emotions users express toward each other.

Likewise, we define individual's personality in Slashdot based on user-user emotion matrices ${\bf P}$ and ${\bf N}$. Let $\overline{P}$ and $\overline{N}$ be the average of positive and negative emotions between all pairs of users, respectively. We also define $\overline{{\bf P}}_j$ and $\overline{{\bf N}}_j$ as the average of positive and negative emotions received by $u_j$. Further, we define $\mathcal{OL}_i$, as a set of users $u_j$ who have received positive emotions from $u_i$, but at the same time, have received more negative emotions than the average in the network, i.e. they are worser than the average,
\begin{align}
\mathcal{ OL}_i = \{u_j \mid {\bf P}_{ij}\neq 0 \wedge \overline{{\bf N}}_j> \overline{N}\} \nonumber
\end{align}
We formally define $\mathcal{OH}_i$ to denote the set of users $u_k$ who belong to $\mathcal{ OL}_i$ and have received more positive emotions from $u_i$ than $\overline{{\bf P}}_k$,
\begin{align}
\mathcal{ OH}_i = \{u_k \mid u_k \in  \mathcal{ OL}_i \wedge {\bf P}_{ik} > \overline{{\bf P}}_k\} \nonumber
\end{align}
In other words, the more frequently $u_i$ has given positive emotions to the worst users in the network, the more optimistic she is. Therefore we define the optimism score for $u_i$ as ${\bf o}_i = \frac{|\mathcal{ OH}_i|}{|\mathcal{ OL}_i|}$.

Likewise, we define $\mathcal{PH}_i$, as a set of users $u_j$ who have received negative emotions from $u_i$, but at the same time, have received more positive emotions than the average in the network, i.e. they are better than the average,
\begin{align}
\mathcal{PH}_i = \{u_j \mid {\bf N}_{ij} \neq 0 \wedge \overline{{\bf P}}_j > \overline{P}\} \nonumber
\end{align}
We define $\mathcal{PL}_i$ to denote the set of users $u_k$ who belong to $\mathcal{ PH}_i$ and have received more negative emotions from $u_i$ than $\overline{{\bf N}}_k$,
\begin{align}
\mathcal{ PL}_i = \{u_k \mid u_k \in  \mathcal{ PH}_i \wedge {{\bf N}}_{ik} > \overline{N}_k\} \nonumber
\end{align}
Pessimism of $u_i$ could be similarly defined as: ${\bf p}_i = \frac{|\mathcal{ PL}_i|}{|\mathcal{ PH}_i|}$.

\noindent There might be other ways to construct ${\bf P}$, ${\bf N}$, ${\bf o}$ and ${\bf p}$ from the data which we leave to future work. Table \ref{Tab:DataStat} shows the statistics of our datasets.

\begin{table}[!htbp]

	\centering
	\small
	\caption{\textbf{Statistics of the Raw Data.}}\label{Tab:DataStat}
	\begin{tabular}{l|l|l}
		& Epinions & Slashdot  \\ \hline \hline
		\# of Users & 405,178 & 7,275\\
		\# of Positive Links & 717,677 & 67,705 \\
		\# of Negative Links & 123,705 & 20851 \\
		\# of Positive Emotions & 12,581,748 & 1,742,763\\
		\# of Negative Emotions & 319,908 & 42,260\\ \hline
	\end{tabular}
\end{table}
\section{Data Analysis and Observations}
In this section, we investigate how each social theory is related to the formation of positive and negative links.
\subsection{Emotional Information Theory}
 Here, we investigate (1) the existence of the correlation between emotional information and positive/negative links in signed social networks and, (2) study the impact of emotional strength on the formation of positive and negative relations. Specifically, we aim to answer the two questions:
\begin{itemize}
	\item Are users with positive (negative) emotions more likely to establish positive (negative) relations than those without?
	\item Are users with higher positive (negative) emotion strengths more likely to create positive (negative) links than those with lower positive (negative) emotion strengths?
\end{itemize}

To answer the {\bf first question}, for each pair of users $(u_i, u_j)$ with positive emotions, we randomly select a user $u_k$ where there is no positive emotions from $u_i$ to $u_k$. We then check the existence of positive relations from $u_i$ to $u_j$ and $u_i$ to $u_k$, respectively. We set $vp = 1$ if ${\bf F}_{ij} = 1$ and $vp = 0$ otherwise. Likewise, we set $vr = 1$ if ${\bf F}_{ik}=1$ and $vr = 0$ otherwise. This way, we obtain two vectors, ${\bf v}_p$ and ${\bf v}_r$ where ${\bf v}_p$ is the set of all $vp$s for pairs of users with positive emotions and ${\bf v}_r$ is the set of $vr$s for pairs of users without positive emotions. We conduct a two-sample $t$-test on ${\bf v}_p$ and ${\bf v}_r$ as follows:
\begin{align}
H_0: {\bf v}_p \leq {\bf v}_r,~~~H_1: {\bf v}_p > {\bf v}_r
\end{align}
\noindent where the null hypothesis is rejected at significance level $\alpha = 0.01$ with p-values of $4.32e{-62}$ and $6.17e{-47}$ over Epinions and Slashdot, respectively. A similar procedure can be followed for negative emotions which we omit the details for brevity. Results from $t$-test suggest that {\it with high probability, users with positive (negative) emotions are more likely to establish positive (negative) links than those without.}

To answer the {\bf second question}, we rank all pairs of users $(u_i, u_j)$ with positive emotions according to their emotion strengths (${\bf P}_{ij}$) in a descending order and divide those pairs into $K$ groups $\mathcal{E} = \{\mathcal{E}_1,\mathcal{E}_2,\ldots,\mathcal{E}_K\}$ with equal sizes. The emotion strengths in $\mathcal{E}_i$ are larger than those in $\mathcal{E}_j$ if $i < j$. Then we form $\frac{K(K-1)}{2}$ pairs of groups $(\mathcal{E}_i,\mathcal{E}_j)$ with $i < j$ where $\mathcal{E}_i$ is the group with higher emotional strengths and $\mathcal{E}_j$ is the one with lower emotional strengths. For each pair of groups, we use $hp$ and $lp$ to denote the number of pairs of users with positive relations in $\mathcal{E}_i$ and $\mathcal{E}_j$, receptively. By repeating this over all pairs of groups, we can obtain two vectors ${\bf h}$ and ${\bf l}$ for $hp$s and $lp$s, respectively.

We conduct a two-sample $t$-test on ${\bf h}$ and ${\bf l}$ by defining the null hypothesis $H_0$ as users with weak positive emotion strengths are more likely to establish positive links and the alternative hypothesis $H_1$ as users with strong positive emotion strengths are more likely to create positive:
\begin{align}
H_0: {\bf h} \leq {\bf l} ,~~~ H_1: {\bf h} > {\bf l}.
\end{align}
By choosing $K = 10$, the null hypothesis is rejected at significance level $0.01$ with p-values of $8.47{e-23}$ and $1.72{e-19}$ for Epinions and Slashdot, respectively . We make similar observations when we set $K = 30$ and $K = 50$. Similarly, we observe the impact of the strengths of negative emotions on the formation of negative links. These results suggest that \textit{users with higher positive (negative) emotion strengths are more likely to establish positive (negative) links than those with lower positive emotion strengths}.

\subsection{Diffusion of Innovation Theory}
 Following the diffusion of innovation theory, our goal here is to study if the behavior of user $u_i$ toward user $u_j$ could be influenced by the behavior of $u_i$'s friend $u_k$ toward $u_j$. More specifically, we aim to answer the following question:
\begin{itemize}
	\item Is user $u_i$ with a friend $u_k$ who has a positive (negative) link to user $u_j$, more likely to establish a positive (negative) link with $u_j$ than if he/she does not have such friend?
\end{itemize}
To answer this question, we first find a pair of users $(u_i, u_j)$  where $u_i$'s friend $u_k$ has a positive link to $u_j$. We also randomly select a user $u_r$ without any positive relations with $u_k$. We then check if there are positive links from $u_i$ to $u_j$ and from $u_i$ to $u_r$. We set $fp = 1$ if ${\bf F}_{ij}=1$ and $fp = 0$ otherwise; Similarly, we set $fr = 1$ if ${\bf F}_{ir}=1$ and $fr = 0$ otherwise. We then construct two vectors, ${\bf f}_p$ and ${\bf f}_r$ where ${\bf f}_p$ is the set of all $fp$s and ${\bf f}_r$ is the set of $fr$s.

We conduct a two-sample $t$-test on ${\bf f}_p$ and ${\bf f}_r$ with the null and alternative hypotheses $H_0$ and $H_1$ defined as follows:
\begin{align}
H_0: {\bf f}_p \leq {\bf f}_r,~~~H_1: {\bf f}_p > {\bf f}_r
\end{align}
\noindent where the null hypothesis is rejected at significance level $\alpha = 0.01$ with p-values of $1.84e{-75}$ and $3.56e{-91}$ over Epinions and Slashdot, respectively. Likewise, we repeat the process for friends with negative links; however for brevity we omit the details and directly give the suggestions from the results of the two-sample $t$-test as follows: {\it users are likely to follow their friends' behaviors in terms of positive and negative link creation.}  The theory is likely to encourage triads shown in Figure~\ref{fig:BalanceTriads}, which are balanced according to balance theory.
\begin{figure}[ht]
	\centering
	{\includegraphics[width=0.25\textwidth]{balancetriads.png}}
	\caption{\textbf{Balanced Triads Encouraged by Diffusion of Innovation Theory.}}\label{fig:BalanceTriads}
\end{figure}

\subsection{Individuals Personality Theory}\label{section:IP}
Next, we investigate the impact of user's personality on the formation of positive and negative links via studying the correlation between personality information and positive and negative links. We seek to answer the following questions:
\begin{itemize}
	\item Are users with higher optimism, more likely to establish positive links than those with lower optimism? and
	\item Are users with higher pessimism, more likely to create negative links than those with lower pessimism?
\end{itemize}

To answer the {\bf first question}, we rank all users in a descending order according to their optimism scores and divide them into $K$ levels with equal sizes denoted as $G=\{g_1, g_2,..., g_K\}$. There are $\frac{K(K-1)}{2}$ pairs of $(g_i,g_j)$ where $i<j$. We consider $g_i$ as the group of more optimistic users compared to those in $g_j$. For each pair $(g_i,g_j)$, we use $H$ and $L$ to denote the number of positive links established by users in groups $g_i$ and $g_j$, respectively. Therefore, we have two vectors ${\bf h}$ and ${\bf l}$ for $H$s and $L$s of all pairs of groups.

We conduct a two-sample $t$-test on ${\bf h}$ and ${\bf l}$ where $H_0$ is that users who are less optimistic are more likely to establish positive links and $H_1$ is that users with higher level of optimism are more likely to create positive relations:
\begin{align}
H_0: {\bf h} \leq {\bf l} ,~~~ H_1: {\bf h} > {\bf l}.
\end{align}
We set $K = 20$, and the null hypothesis is rejected at significance level $0.01$ with p-values $3.16{e-19}$ and $1.6029{e-23}$ for Epinions and Slashdot datasets, respectively. We make similar observations by setting $K = 30$ and $K = 50$. These results suggest that \textit{users with high optimistic behavior are more likely to establish positive links than those with low optimism}. Following a similar procedure, we observe that \textit{users who are more pessimistic are more likely to establish negative relations than those with low level of pessimism}.

\section{Conclusion and Future Work}
In this study, we employ three theories from psychology and social sciences, namely \textit{Emotional Information}, \textit{Diffusion of Innovations} and \textit{Individual Personality}, on the additional information available in the networks including user's emotional and personality, for the task of link analysis in signed networks. In future, we plan to replicate this study by exploiting other available information in the network. Another interesting research direction is to study link analysis in dynamic signed networks by deploying the social theories.
\section{Acknowledgments}
This material is based upon the work supported by, or in part by, Army Research Office (ARO) under grant numbers W911NF-15-1-0328 and \#025071.
\bibliographystyle{aaai}


text:
\section{Introduction}
Recent years have witnessed artificial intelligence (AI) systems with remarkable abilities \citep[e.g.,][]{devlin2018bert,silver2016mastering,goyal2021self}, whose success critically depends on having access to huge amounts of training data. Examples include the famous Google BERT language model pre-trained on $800$M words from BooksCorpus and $2,500$M words from Wikipedia \citep{devlin2018bert}, the DeepMind AlphaGo system trained on over $30$M expert moves \citep{silver2016mastering}, the OpenAI GPT-3 model pre-trained on $300$ billion tokens \citep{brown2020language}, and the recent Facebook SEER image recognition model trained on one billion images from Instagram photos \citep{goyal2021self}.

Indeed, an influential subfield of AI, called reinforcement learning (RL), requires AI agents to learn by having interactions with their environment to collect feedback, in the form of rewards \citep{sutton2018reinforcement}. This is especially challenging in settings where the environment consists of human agents, resulting in these interactions being both expensive and time-consuming, thus exacerbating the training process. Could we instead use cognitive models, as a \emph{proxy} for humans, to address this issue?

In this work, we substantiate the idea of using cognitive models as {simulators}, which is to have AI systems interact with, and collect feedback from, cognitive models instead of humans, thereby making their training process both less costly and faster. Here, for the first time in the literature, we leverage this idea in the context of moral decision-making \citep{haidt2007new,lapsley2018moral}, by having RL agents learn about fairness through interacting with a cognitive model of the Ultimatum Game (UG), a well-established game in behavioral and brain sciences for studying fairness \citep[e.g.,][]{sanfey2009expectations,battigalli2015frustration,vavra2018expectations,sanfey2003neural,xiang2013computational,chang2013great}. Interestingly, these RL agents learn to rationally adapt their behavior depending on the emotional state of their UG Responder (see Sec.~1 for an explanation of how UG works). Our work suggests that using cognitive models as simulators of humans is an effective approach for training AI systems, presenting an important way for computational cognitive science to make contributions to the field of AI.

We begin by describing UG and presenting an overview of the relevant psychological findings on the role of emotions in UG (Sec.~2).~We then discuss in Sec.~3 a process model of UG Responder under a variety of emotional states (Lizzotte, Nobandegani, \& Shultz, 2021; Nobandegani, Destais, \& Shultz, 2020), and subsequently present our RL training results under various UG Responder's emotional states (Sec.~4).~We conclude by discussing the implications of our work for the fields of cognitive science and AI, and synergistic interactions between the two (Sec.~5).

\section{UG and the Role of Emotions in UG}
The Ultimatum Game (UG; \citealp{guth1982experimental}) is a canonical task for studying fairness, and has been extensively studied in psychology \citep[e.g.,][]{sanfey2009expectations,battigalli2015frustration,vavra2018expectations}, neuroscience \citep{sanfey2003neural,xiang2013computational,chang2013great}, philosophy \citep[][]{guala2008paradigmatic}, and behavioral economics \citep[e.g.,][]{guth1982experimental,thaler1988anomalies,camerer1995anomalies,fehr1999theory,sutter2003bargaining,camerer2006does}. UG has a simple design: Two players, Proposer and Responder, must agree on how to split a sum of money. Proposer makes an offer. If Responder accepts, the deal goes through; if Responder rejects, neither player gets anything. In both cases, the game is over.

An extensive body of empirical work has established that UG Proposers predominantly respect fairness by offering about $50\%$ of the endowed amount, and that this split is almost invariably accepted by UG Responders \citep[see][]{camerer2011behavioral}.~Relatedly, UG Responders often reject offers below $30\%$,  presumably as retaliation for being treated unfairly \citep[][]{guth1982experimental,thaler1988anomalies,guth1990ultimatum,bolton1995anonymity,nowak2000fairness,camerer2006does}.

A growing body of experimental work has revealed that induced emotions strongly affect UG Responder's accept/reject behavior, with positive emotions increasing the chance of low offers being accepted \citep[e.g.,][]{riepl2016influences,andrade2009enduring}, and negative emotions decreasing the chance of low offers being accepted \citep[e.g.,][]{bonini2011pecunia,harle2010effects,liu2016negative,moretti2010disgust,vargas2019blocking}. Experimentally, these emotions are often induced by a movie clip or recall task.

\section{A Computational Model of UG Responder}
Recently, \citet{nobandegani2020UG} presented a process model of UG Responder, called \emph{sample-based expected utility} (SbEU). SbEU provides a unified account of several disparate empirical findings in UG (i.e., the effects of expectation, competition, and time pressure on UG Responder), and also explains the effect of a wide range of emotions on UG Responder (Lizzotte, Nobandegani, \& Shultz, \citeyear{nobandegani2021EmotionsinGames}).

Nobandegani et al.'s process-level account rests on two main assumptions.~First, UG Responder uses SbEU to estimate the expected-utility gap between their expectation and the offer, i.e., $\mathbb{E}[u(\text{offer})-u(\text{expectation})]$, where $u(\cdot)$ denotes Responder's utility function. If this estimate is positive --- indicating that the offer made is, on average, higher than Responder's expectation --- Responder accepts the offer; otherwise, Responder rejects the offer. This assumption is supported by substantial empirical evidence showing that Responder's expectation serves as a reference point for subjective valuation of offers \citep{sanfey2009expectations,battigalli2015frustration,vavra2018expectations,xiang2013computational,chang2013great}.

The second assumption is that negative emotions elevate loss-aversion while positive emotions lower loss-aversion \citep{nobandegani2021EmotionsinGames}.~Again, this assumptions is supported by mounting empirical evidence \citep[e.g.,][]{de2010amygdala,sokol2015interoceptive,sokol2009thinking} suggesting that emotions modulate loss-aversion --- the tendency to overweight losses as compared to gains (Kahneman \& Tverskey, \citeyear{Kahnemna1979}).

Concretely, SbEU assumes that an agent estimates expected utility:
\begin{eqnarray}
\mathbb{E}[u(o)]=\int p(o)u(o)do,
\label{eq_exp_utility}
\end{eqnarray}
using self-normalized importance sampling (Nobandegani et al., 2018; Nobandegani \& Shultz, 2020b, 2020c), with its importance distribution $q^\ast$ aiming to optimally minimize mean-squared error (MSE):
\begin{eqnarray}
\hat{E}=\dfrac{1}{\sum_{j=1}^s w_j} \sum_{i=1}^s w_i u(o_i), \quad \forall i:\ o_i\sim q^\ast,\ w_i=\dfrac{p(o_i)}{q^\ast(o_i)},
\label{eq_norm_is_estimator}
\end{eqnarray}
\begin{eqnarray}
q^\ast(o)\propto p(o)|u(o)| \sqrt{\dfrac{1+|u(o)|\sqrt{s}}{|u(o)|\sqrt{s}}}.
\label{eq_meta_rational_q}
\end{eqnarray}
MSE is a standard measure of estimation quality, widely used in decision theory and mathematical statistics (Poor, \citeyear{poor2013introduction}). In Eqs.~(\ref{eq_exp_utility}-\ref{eq_meta_rational_q}), $o$ denotes an outcome of a risky gamble, $p(o)$ the objective probability of outcome $o$, $u(o)$ the subjective utility of outcome $o$, $\hat{E}$ the importance-sampling estimate of expected utility given in Eq.~(\ref{eq_exp_utility}), $q^\ast$ the importance-sampling distribution, $o_i$ an outcome randomly sampled from $q^\ast$, and $s$ the number of samples drawn from $q^\ast$.

SbEU has so far explained a broad range of empirical findings in human decision-making, e.g., the fourfold patterns of risk preferences in both outcome probability and outcome magnitude (Nobandegani et al., \citeyear{nobandegani2018meta}), risky decoy and violation of betweenness \citep{nobandegani2019BTW}, violation of stochastic dominance \citep*{nobandegani2022VoSD}, violation of cumulative independence \citep*{nobandegani2022VoCI}, the three contextual effects of similarity, attraction, and compromise (da Silva Castanheira, Nobandegani, Shultz, \& Otto, \citeyear{Castanheira2019value}), the Allais, St.~Petersburg, and Ellsberg paradoxes \citep{nobandegani2020resource,nobandegani2020bSTPP,nobandegani2021AllaisEllsberg}, cooperation in Prisoner's Dilemma \citep{nobandegani2019PD}, and human coordination behavior in coordination games \citep{nobandegani2020CG}.~Notably, SbEU is the first, and thus far the only, resource-rational process model that bridges between risky, value-based, and game-theoretic decision-making.

\section{Training RL Agents in UG}
\label{sec_rl_thompson_sampling}
In this section, we substantiate the idea of \emph{cognitive models as simulators} in the context of moral decision-making, by having RL agents learn about fairness through interacting with a cognitive model of UG Responder \citep{nobandegani2020UG}, as a proxy for human Responders, thereby making their training process both less costly and faster.

To train RL Proposers, we leverage the broad framework of multi-armed bandits in reinforcement learning \citep{katehakis1987multi,gittins1979bandit}, and adopt the well-known Thompson Sampling method \citep{thompson1933likelihood}. Specifically, we assume that RL Proposer should decide what percentage of the total money $T$ they are willing to offer to SbEU Responder. For ease of analysis, here we assume that RL Proposer chooses between a finite set of options: $\mathcal{A} = \{0, \frac{T}{10}, \frac{2T}{10}, \cdots, \frac{9T}{10}, T\}$.
\begin{algorithm}
\caption{{Thompson Sampling for UG Proposer}}\label{TS_proposer}
\begin{algorithmic}[1]
\Statex \textbf{Initialize}. $\forall a\in\mathcal{A}$: $S_{a} = 0$ and $F_{a} = 0$
\State \textbf{for} $i = 1,\ldots, N$
\State $\forall a\in\mathcal{A}$ compute:

$s_{a} = u(T - a)\beta_{a}$,\quad $\beta_{a}\sim$ Beta$(S_{a}+1, F_{a}+1)$
\State $a^\ast = \arg\max_{a}\,\,\, s_{a}$
\State Offer $a^\ast$ to SbEU Responder
\State \textbf{if} SbEU Responder accepts the offer \textbf{then}
\State \hspace*{20pt}$S_{a^\ast} = S_{a^\ast} + 1$
\State \textbf{else}
\State \hspace*{20pt}$F_{a^\ast} = F_{a^\ast} + 1$
\State \textbf{end if}
\State \textbf{end for}
\end{algorithmic}
\end{algorithm}

In reinforcement learning terminology, RL Proposer learns, through trial and error while striking a balance between exploration and exploitation, which option $a\in\mathcal{A}$ yields the highest mean reward. Here, we train RL Proposers using Thompson Sampling, a well-established method in the reinforcement learning literature enjoying strong optimality guarantees \citep{agrawal2012analysis,agrawal2013further}; see Algorithm 1.

Algorithm 1 can be described in simple terms as follows. At the start, i.e., prior to any learning, the number of times an offer $a\in\mathcal{A}$ is so far accepted, $S_a$ (S for success), and the number of times it is rejected, $F_a$ (F for failure), are both set to zero. In each trial (for a total of $N$ trials), an estimate of mean reward for each offer $a\in\mathcal{A}$ is computed by sampling from the corresponding distribution (Line 2), and the offer with the highest mean reward estimate $a^\ast$ (Line 3) is then chosen by Proposer to be offered to SbEU Responder (Line 4). If this offer is accepted by SbEU Responder, the $S_a$ parameter for that offer is incremented by one (Line 6); if rejected, the $F_a$ parameter for that offer is instead incremented by one. In Algorithm 1, $T$ is the total amount of money to be split between Proposer and Responder, $u(\cdot)$ is the subjective utility function of Responder, and Beta$(\cdot,\cdot)$ is the Beta distribution.

\begin{figure}[h!]
\centering
\includegraphics[trim = 178pt 8pt 145pt 10pt,clip,width=0.49\textwidth]{Fig__neutral__combined}
\caption{\textbf{Mean frequency of RL Proposer's offers.} The $y$-axis indicates the mean frequency of each offer made by RL Proposer to SbEU Responder up to current trial ($x$-axis), averaged over 10 RL Proposers. SbEU Responder is in a neutral emotional state. As a visual aid, the dynamics for the first 200 trials are provided in a smaller plot, located at the center.}
\label{fig_neural}
\end{figure}

In Fig.~\ref{fig_neural}, we simulate 10 RL Proposers, and report the mean frequency of an offer being made to SbEU Responder over the past trials, for a total of $N=10,000$ trials. As can be seen, exercising a balance between exploration and exploitation, RL Proposers eventually arrive at the decision that they should be making a fair offer to SbEU Responder, i.e., to split the total sum $T$ equally between themselves and Responder. As such, RL Proposer's making fair offers can be seen as a rational, emergent behavior arising from having sufficient interactions with inequality-averse SbEU Responder --- as a proxy for inequality-averse humans.

\subsection{RL Proposer Meets Emotional Responder}
In this section, we bridge between the idea of \emph{cognitive models as simulators} and emotion research, by letting AI systems interact with a cognitive model of people experiencing various emotional states. Specifically, we pursue this idea in the context of UG, and have RL Proposers interact with SbEU Responders experiencing various emotional states.

A wealth of empirical research has revealed that the effect of emotions on human decision-making is both substantial and systematic \citep[for reviews see, e.g.,][]{phelps2014emotion,lerner2015emotion}. More specifically, in the context of UG, a growing body of empirical studies have shown that induced emotions strongly affect UG Responder's behavior, with positive emotions (e.g., happiness) increasing the chance of low offers being accepted \citep[e.g.,][]{riepl2016influences,andrade2009enduring}, and negative emotions (e.g., disgust, anger, and sadness) decreasing the chance of low offers being accepted \citep[e.g.,][]{bonini2011pecunia,harle2010effects,liu2016negative,moretti2010disgust,vargas2019blocking}. Hence, it would be rational for UG Proposer (from the perspective of maximizing their mean reward) to make larger offers to Responders experiencing negative emotions, and, conversely, to make smaller offers to Responders experiencing positive emotions.
\begin{figure}[h!]
\centering
\includegraphics[trim = 150pt 8pt 165pt 10pt,clip,width=0.48\textwidth]{fig__positive__negative__combined}
\caption{\textbf{Mean frequency of RL Proposer's offers.} The $y$-axis indicates the mean frequency of each offer made by RL Proposer to SbEU Responder up to current trial ($x$-axis), averaged over 10 RL Proposers. In \textbf{(a)} SbEU Responder is under a negative emotional state, while in \textbf{(b)} SbEU Responder is under a positive emotional state. As a visual aid, in each subplot, the dynamics for the first 200 trials are provided in a smaller plot, located at the center.}
\label{fig_positive&negative}
\end{figure}

Interestingly, under the broad and empirically well-supported assumption that emotions modulate loss-aversion \citep[e.g.,][]{de2010amygdala,sokol2015interoceptive,sokol2009thinking}, Nobandegani et al.'s SbEU model explains the effect of a wide range of emotions on human UG Responder \citep{nobandegani2021EmotionsinGames}. Next, we train RL Proposers, using Thompson Sampling (see Algorithm \ref{TS_proposer}), to learn how to interact with SbEU Responders experiencing positive or negative emotional states.

In Fig.~\ref{fig_positive&negative}, we simulate 10 RL Proposers, and report the mean frequency of an offer being made to SbEU Responder over the past trials, for a total of $N=100,000$ trials. In Fig.~\ref{fig_positive&negative}(a), SbEU Responder is under a negative emotional state, and, in Fig.~\ref{fig_positive&negative}(b), SbEU Responder is under a positive emotional state. As can be seen, RL Proposers eventually arrive at the decision that they should be making a larger offer ($60\%$) when Responder is experiencing a negative emotional state (Fig.~\ref{fig_positive&negative}(a)), and, conversely, should be making a smaller offer ($40\%$) when Responder is experiencing a positive emotional state (Fig.~\ref{fig_positive&negative}(b)). As such, RL Proposers learn to adapt their strategy depending on Responder's emotional state.

\section{General Discussion}
To achieve desirable performance, current AI systems often require huge amounts of training data.~This is especially problematic in domains where collecting data is both expensive and time-consuming,~e.g.,~where AI systems require many interactions with humans, collecting feedback from them.~In this work, we substantiate the idea of \emph{cognitive models as simulators}, which is to have AI systems interact with, and collect feedback from, cognitive models as a proxy for humans, thereby decreasing both cost and time for the AI training process.

Here, for the first time in the literature, we leverage this idea in the context of moral decision-making \citep{haidt2007new,lapsley2018moral}, by having reinforcement learning (RL) agents learn about fairness through interacting with a cognitive model of the Ultimatum Game (UG), a canonical task for studying fairness in behavioral and brain sciences \citep[e.g.,][]{sanfey2009expectations,battigalli2015frustration,vavra2018expectations,sanfey2003neural,xiang2013computational,chang2013great}. As a cognitive model, we use {sample-based expected utility} (SbEU), a resource-rational process model explaining a wide range of empirical findings on UG Responders \citep{nobandegani2020UG,nobandegani2021EmotionsinGames}. As an AI system, we train RL Proposers using Thompson Sampling, a well-known method in the multi-armed bandits literature enjoying strong optimality guarantees \citep{agrawal2012analysis,agrawal2013further}.

Given the significant role that emotions play in human decision-making \citep[for reviews see, e.g.,][]{phelps2014emotion,lerner2015emotion}, we further link the idea of \emph{cognitive models as simulators} to emotion research, by having RL Proposers interact with SbEU Responders under various emotional states (i.e., neutral, negative, and positive). Interestingly, RL Proposers learn to rationally adapt their behavior depending on the emotional state of their SbEU Responder, making larger offers when Responder is more likely to reject low offers (due to experiencing negative emotions) and, conversely, making smaller offers when Responder is less likely to reject low offers (due to experiencing positive emotions).

Recent success stories in AI, e.g., AlphaGo and particularly self-play \citep{silver2016mastering,silver2017mastering}, clearly demonstrate the significant role that having access to a simulator of the environment would play in efficient training of AI systems. The idea of \emph{cognitive models as simulators} substantiated in our work is yet another step in the direction of leveraging simulators of the environment --- by using cognitive models as a proxy for people --- in the service of making the training of AI systems both faster and less costly. As such, the idea of \emph{cognitive models as simulators} presents an important way for computational cognitive science to contribute to AI.

Although here we presented the idea of \emph{cognitive models as simulators} as a way of making the training of AI systems more efficient, it could also be seen as a broad \emph{cognitive} framework for how people might be choosing their strategies in multi-agent environments by {mentalizing} about other agents. As such, the idea of \emph{cognitive models as simulators} could potentially serve as a broad {framework} for theorizing about, and mathematically identifying, mental processes by which people choose their strategies when interacting with other agents. Hence, this ``cognitive" reconceptualization of \emph{cognitive models as simulators} has potential to make contributions to computational cognitive science.

Additionally, a strong reading of this cognitive reconceptualization takes the AI systems learning from interacting with mental models as a proposal for how people might be choosing their strategy in multi-agent environments, thus presenting an important way for AI to contribute to computational cognitive science.

From this perspective, the Thompson Sampling algorithm presented in Sec.~\ref{sec_rl_thompson_sampling} for training RL Proposers could serve as a {process-level proposal} for how human Proposers might be choosing their offer: by simulating UG Responder using a mental model of UG Responder and learning from mentally interacting with that model, here implemented by SbEU \citep{nobandegani2020UG}.~Nonetheless, human Proposers might be using a much simpler mental model of their human Responder as compared to SbEU, and would presumably start with much stronger prior beliefs (i.e., inductive biases) about the mean reward of each of their strategies --- instead of the uniformly distributed Beta$(1,1)$ prior used in Algorithm 1.~Future work should more extensively investigate this process-level proposal.

Also, this cognitive reconceptualization is consistent with substantial work on both people's intuitive psychology and human strategic decision-making \citep[e.g.,][]{jern2017people,jara2016naive,nagel1995unraveling,baker2009action,camerer2004cognitive}, broadly assuming that people have a mental model of other agents and use that model to both interpret other agents' behavior and decide how to behave when interacting with those agents.

Finally, as elaborated above, having good potential to contribute to both AI and computational cognitive science, the idea of \emph{cognitive models as simulators} is yet another step in the fruitful direction of having ever more synergistic interaction between the two disciplines. We see our work a step in this important direction.\\

\noindent \textbf{Acknowledgments}. This research was supported in part by an operating grant to TRS from the Natural Sciences and Engineering Research Council of Canada.

\setlength{\bibleftmargin}{.125in}
\setlength{\bibindent}{-\bibleftmargin}

\nocite{nobandegani2018meta}

\bibliographystyle{apacite}

redpajama_set_name: RedPajamaArXiv


text:
\section{Introduction}

Transverse momentum dependent, or equivalently, $k_t$-distributions
are currently object of intense research activity due to
their wide range of applicability in the description of hadron initiated
hard processes~\cite{Hautmann_Jung,WMR}.
The use of $k_t$-distributions is indeed phenomenologically appealing since observables
constructed upon them show a reasonable agreement with data
already in lowest order, which is not the case for predictions based
on collinear factorization at the same accuracy, see for instance the
discussion in Ref.~\cite{Zpt_Kwiecinski}.
Their correct formalization in quantum-chromodynamics appears to be well established only
at high energy~\cite{kt_fact}. Away from this limit the situation is still unclear,
although detailed investigations are present in the literature~\cite{CS,Stefanis,Hautmann}.
The relevant issue of factorization within a $k_t$-dependent approach has also been investigated~\cite{Ji}.
The latter property together with a precise knowledge of the scale dependence of $k_t$-distributions
would allow to relate to each other data coming from
experiments at different energies and, more important, to test factorization
quantitatively.
Although a definitive answer is absent in the literature,
there have been however some attempts, see for example~\cite{Kwiecinski1loop,Kimber}.
In particular we will focus on the equations proposed by us
in Ref.~\cite{CT} which are the space-like version of the ones proposed
in Ref.~\cite{BCM}.
In a subsequent phemonelogical study~\cite{CT2}, performed in
the context of semi-inclusive deep inelastic scattering,
it was shown that a reasonable description of data could be
obtained once unintegrated evolution equations
were solved with suitable, but motivated, initial conditions
and assuming factorization for the cross-sections of interest.
This result stimulated us to apply the same formalism to the description
Drell-Yan type processes in hadronic collisions~\cite{Zpt_Ceccopieri}.
The $p_t$-spectrum of the gauge boson has, in fact, a rich
transverse structure and involves both perturbative and
non-perturbative aspects of the underlying theory.
The resummation of the perturbative series in the multiple soft gluon emission limit
can be accomodated by using properly modified unintegrated evolution equations~\cite{KT}
so that the structure of the non-perturbative form factor can be investigated.

\section{Space-like $k_t$-dependent evolution equations}
Transverse momentum dependent parton distribution function,
$\mathcal{F}_{P}^{i}(x_B,Q^2,\bm{k_{\perp}})$, give
the probability to find, at a given scale $Q^2$, a parton $i$
with longitudinal momentum fraction $x_B$ and transverse momentum $\bm{k}_{\perp}$
relative to the parent hadron momentum.
The evolution equations for $k_t$-distributions proposed in Refs.~\cite{CT,CT2} read
\begin{multline}
\label{dglap_TMD_space}
Q^2 \frac{d \mathcal{F}_{P}^{i}(x_B,Q^2,\bm{k_{\perp}})}{d Q^2}
=\frac{\alpha_s(Q^2)}{2\pi}\int_{x_B}^1 \frac{du}{u^3}
P_{ji}(u) \cdot\\ \cdot \int \frac{d^2 \bm{l}_{\perp}}{\pi}\,\delta(\,(1-u)Q^2-l^2_{\perp})
\,\mathcal{F}_{P}^{j}\Big(\frac{x_B}{u},Q^2, \frac{\bm{k}_{\perp}-\bm{l}_{\perp}}{u} \Big)\,,
\end{multline}
and resum large logarithms associated to the emission
of collinear partons. However, at variance with ordinary evolutions
equations~\cite{AP},  the transverse momentum generated
at each branching, $\bm{l}_{\perp}$, is explicitely taken into accont kinematically
and eventually adds to the non-perturbative one due to
Fermi motion of the parton in the parent hadron.
If we now consider $\mathcal{F}$
to be the bare distribution of a quark in a quark, neglecting flavour indeces,
we may insert  the source term
$\mathcal{F}(x,Q^2,\bm{k_{\perp}})=\delta(1-x)\delta^{(2)}(\bm{k}_{\perp})$
into eq.~(\ref{dglap_TMD_space}) obtaining, up to $\mathcal{O}(\alpha_s)$, the following result
\begin{equation}
\label{one_loop}
\mathcal{F}(x,Q^2,\bm{k}_{\perp})=\delta(1-x)\delta^{(2)}
(\bm{k}_\perp)
+\frac{\alpha_s}{2\pi}\frac{1}{\pi}\frac{1}{k_{\perp}^2} P(x)\,,
\end{equation}
which indeed exposes the singularity associated to the emission of a collinear parton
weighted by splitting functions $P(x)$~\cite{AP}.
In order to clarify the role of the new ingredients appearing
in eq.~(\ref{dglap_TMD_space}), it proves useful to
consider just one emission, see Fig.~(\ref{fig1}).
The small blob at the bottom left of the figure then symbolizes the iteration of emissions
in the parton ladder of whose
only the last, $\widetilde{k} \rightarrow k +l$, is explicitely shown.
Transverse momenta $\widetilde{\bm{k}}_{\perp}$ and $\bm{k}_{\perp}$ are defined
with respect to the incoming hadron direction, while $\bm{l}_{\perp}$ is defined
with respect to $\widetilde{k}$ direction.
The transverse momentum $\bm{\widetilde{k}}_{\perp}$
and longitudinal momentum $\widetilde{x}$ of the branching parton $\widetilde{k}$
are written as a function of the relative transverse momentum $\bm{l}_{\perp}$
and the fractional momentum $u$ appearing in the splitting.
They can be expressed as $\bm{\widetilde{k}}_{\perp}=(\bm{k}_{\perp}-\bm{l}_{\perp})/u$
and $\widetilde{x}=x/u$, respectively. The former is a result of the transverse boost
from $\widetilde{k}$ to hadron direction~\cite{GSW}. Both can be found in
the distribution under convolution in the right hand side of eq.~(\ref{dglap_TMD_space}).
The additional integration $d^2 \bm{l}_{\perp}$and the $\delta$-function
 appearing in eq.~(\ref{dglap_TMD_space})
are associated with the phase space and mass-shell contraint of the emitted partons.
\begin{figure}[t]
\begin{center}
\includegraphics[width=2.5in]{is_art_3new.eps}
\caption{Pictorial representation of the proposed evolution equation.
The longitudinal and transverse component of the partons involved in the splitting
$\widetilde{k} \rightarrow k +l$ are indicated. \protect\label{fig1}}
\end{center}
\end{figure}
The latter can be written as
\begin{equation}
\label{mass}
l^2_{\perp}=-(1-u)k^2+u(1-u)\widetilde{k}^2\,,
\end{equation}
where the emission of massless partons has been assumed ($l^2=0$).
As is well known, logarithmically enanched contributions arise
when the interacting parton virtuality increases along the ladder.
Therefore in the limit $\widetilde{k}^2 \ll k^2$ we may neglect $\widetilde{k}^2$ with respect
to $k^2$ and set $-k^2=Q^2$, obtaining
$l^2_{\perp}=(1-u)Q^2$, which can be found in eq.~(\ref{dglap_TMD_space}).
Furthermore, we assume $k_t$-distributions to fulfil the normalization condition:
\begin{equation}
\label{spacelike_norm}
\int d^2 \bm{k}_{\perp} \mathcal{F}_{P}^{i}(x_B,Q^2,\bm{k}_{\perp})=
f_{P}^{i}(x_B,Q^2)\,,
\end{equation}
where $f_{P}^{i}$ are ordinary parton distributions.
It is important to remark that integrating eq.~(\ref{dglap_TMD_space}) over $d^2 \bm{k}_{\perp}$
and by using eq.~(\ref{spacelike_norm}) we recover the ordinary
evolution equations for $f_{P}^{i}$~\cite{AP} .
\begin{figure}[t]
\centerline{\psfig{file=kt2space.eps,width=7.0in}}
\vspace*{8pt}
\caption{Left and middle panel: average transverse momentum $\langle k_{\perp}^2\rangle $
for up-quark and gluon at three different scales:
$Q_0^2=5$  Ge$V^2$ ($-$), $Q^2=10$ Ge$V^2$ ($--$) and $Q^2=20$ Ge$V^2$ ($-\cdot-$)\,.
Right panel: the transverse spectrum of the up quark distributions at fixed $x_B=0.1$
at the previously indicated three different scales.
\protect\label{fig2}}
\end{figure}

\section{Numerical solutions}
The unintegrated evolution equations, eq.~(\ref{dglap_TMD_space}), are numerically solved by
a finite difference method on a discrete grid in $(x_B,\bm{k}_{\perp})$ space.
The initial conditions at the starting scale $Q_0^2= 5$ Ge$V^2$
are chosen to have the factorized form
\begin{equation}
\label{ic}
\mathcal{F}_P^i(x_B,Q_0^2,\bm{k}_{\perp})=  f_P^i(x_B,Q_0^2) \frac{1}{\pi<k_{\perp,i}^2>}
\,e^{\frac{-k_{\perp}^2}{<k_{\perp,i}^2>}} \,\,\,
\;\;\; i=q,\bar{q},g
\end{equation}
where $f_P^i(x_B,Q_0^2)$ are ordinary parton distributions~\cite{GRV}.
The choise of a energy ($x_B$) independent guassian transverse factor
in eq.~(\ref{ic}) is supported, for example, by recent semi-inclusive deep inelastic scattering
data at low energy~\cite{Misha}.
The width $\langle k_{\perp,i}^2\rangle $ are fixed to a value of $0.25$ Ge$V^2$
both for quarks and gluons~\cite{CT2}.
Since the initial conditions, eq.~(\ref{ic}), by construction fulfil eq.~(\ref{spacelike_norm}),
the latter can be checked for every values of $x_B$ and $Q^2$ to estimate and, eventually, to improve
the numerical accuracy of the evolution. For this check to be meaningful we adopt
the same flavour scheme and coupling values used in Ref.~\cite{GRV}.
The evolved distributions show indeed some interesting properties.
In the left and middle panel of Fig.(\ref{fig2})
we present the average transverse momentum, $\langle k_\perp^2 \rangle$,
as a function of $x_B$ calculated at
three different scale for up quark and gluon, respectively.
In both cases, the averaged transverse momentum at the final scale,  $Q^2= 20$ Ge$V^2$,
increases with decreasing $x_B$. The effect is more
evident in the gluon case, due to the singular behaviour of $P_{gg}(u)$ at small $u$.
In the opposite limit, $x_B \rightarrow 1$, the averaged
transverse momentum approaches the value given in the initial condition and indicated by
the horizontal line in left and middle panel of Fig.(\ref{fig2}).
In this limit, phase space only allows the emission of soft partons
which generate negligible trasverse momentum.
In the right panel of Fig.~(\ref{fig2}) it is shown the up-quark distribution at fixed $x_B=0.1$
as a function of $k_\perp^2$.
Interestingly, the factorized form of eq.~(\ref{ic}) is not preserved under evolution. The guassian
dependence on $k_\perp^2$ at $Q_0^2$ is turned into a inverse power law $1/(k_\perp^2)^a$
at the final scale $Q^2$, the latter dependence in qualitative agreement with matrix
elements behaviour at large $k_\perp^2$, eq.~(\ref{one_loop}).
We also observe that, as $Q^2$ increases, the distributions start to \textit{populate}
the higher $k_\perp^2$ region, as an effect of the leading logarithmic approximation
built-in the evolution equations.
\section*{Conclusions}
In this contribution we have reviewed some of the features
of the unintegrated evolution equations  proposed by us and discussed
some peculiar properties of the evolved $k_t$-distributions obtained by numerical solution.
The use of unintegrated distributions is indeed phenomenologically appealing~\cite{CT2,Zpt_Ceccopieri}.
Most interestingly, the analysis of jet observables within this framework may give access
to valuable informations on the unintegrated gluon distributions
which describes hadron structure as probed in high energy collisions.
However, the departure from a pure collinear scheme
introduces a number of non trivial problems which are at present
under investigation by many groups. This implies that, to date, any conclusive statement
on the validity of $k_t$-distributions approach in general,
and on the proposed evolution equations in particular, must wait for further theoretical developments.