<p>A useful way to view the model is as follows. For each observation $i$, there are $K$ different possible observations (choices). So, there is a random variable $\varepsilon_{i,k}$ for each possible choice. We observe which choice is made and the values of the explanatory variables $\boldsymbol X_i$. Then, for a given distribution of the random variables $\varepsilon_{i,k}$ (given parameter values of the Type-1 EV), we know the conditional probability of picking choice $k$ conditional on $\boldsymbol X_i$.</p>

<p>This is explained in the following part of the relevant Wikipedia <a href="https://en.wikipedia.org/wiki/Multinomial_logistic_regression#As_a_latent-variable_model" rel="nofollow noreferrer">article</a>:</p>

<blockquote>
  <p>It is also possible to formulate multinomial logistic regression as a
  latent variable model, following the two-way latent variable model
  described for binary logistic regression.  This formulation is common
  in the theory of discrete choice models, and makes it easier to
  compare multinomial logistic regression to the related multinomial
  probit model, as well as to extend it to more complex models.</p>
  
  <p>Imagine that, for each data point $i$ and possible outcome $k$,
  there is a continuous latent variable
  $Y_{i,k}^*$ (i.e. an unobserved random
  variable) that is distributed as follows:</p>
  
  <p>$$\begin{align} Y_{i,1}^{\ast} &amp;= \boldsymbol\beta_1 \cdot
\mathbf{X}_i + \varepsilon_1 \, \\ Y_{i,2}^{\ast} &amp;=
\boldsymbol\beta_2 \cdot \mathbf{X}_i + \varepsilon_2 \, \\ \cdots &amp;
\\ Y_{i,K}^{\ast} &amp;= \boldsymbol\beta_K \cdot \mathbf{X}_i +
\varepsilon_K \, \\ \end{align} $$</p>
  
  <p>where $\varepsilon_k \sim \operatorname{EV}_1(0,1),$ i.e. a
  standard type-1 extreme value distribution.</p>
  
  <p>This latent variable can be thought of as the utility associated
  with data point $i$ choosing outcome $k$, where there is some
  randomness in the actual amount of utility obtained, which accounts
  for other unmodeled factors that go into the choice.  The value of the
  actual variable $Y_i$ is then determined in a non-random
  fashion from these latent variables (i.e. the randomness has been
  moved from the observed outcomes into the latent variables), where
  outcome $k$ is chosen if and only if the associated utility (the
  value of $Y_{i,k}^{\ast}$) is greater than the utilities of
  all the other choices, i.e. if the utility associated with outcome
  $k$ is the maximum of all the utilities. Since the latent variables
  are continuous, the probability of two having
  exactly the same value is 0, so we ignore the scenario. That is:</p>
  
   \begin{align} \Pr(Y_i = 1) &amp;= \Pr(Y_{i,1}^{\ast} &gt;
 Y_{i,2}^{\ast} \text{ and } Y_{i,1}^{\ast} &gt; Y_{i,3}^{\ast}\text{ and
 } \cdots \text{ and } Y_{i,1}^{\ast} &gt; Y_{i,K}^{\ast}) \\ \Pr(Y_i = 2)
 &amp;= \Pr(Y_{i,2}^{\ast} &gt; Y_{i,1}^{\ast} \text{ and } Y_{i,2}^{\ast} &gt;
 Y_{i,3}^{\ast}\text{ and } \cdots \text{ and } Y_{i,2}^{\ast} &gt;
 Y_{i,K}^{\ast}) \\ \cdots &amp; \\ \Pr(Y_i = K) &amp;= \Pr(Y_{i,K}^{\ast} &gt;
 Y_{i,1}^{\ast} \text{ and } Y_{i,K}^{\ast} &gt; Y_{i,2}^{\ast}\text{ and
 } \cdots \text{ and } Y_{i,K}^{\ast} &gt; Y_{i,K-1}^{\ast}) \\
 \end{align} 
  
  <p>Or equivalently:</p>
  
   \begin{align} \Pr(Y_i = 1) &amp;=
 \Pr(\max(Y_{i,1}^{\ast},Y_{i,2}^{\ast},\ldots,Y_{i,K}^{\ast})=Y_{i,1}^{\ast})
 \\ \Pr(Y_i = 2) &amp;=
 \Pr(\max(Y_{i,1}^{\ast},Y_{i,2}^{\ast},\ldots,Y_{i,K}^{\ast})=Y_{i,2}^{\ast})
 \\ \cdots &amp; \\ \Pr(Y_i = K) &amp;=
 \Pr(\max(Y_{i,1}^{\ast},Y_{i,2}^{\ast},\ldots,Y_{i,K}^{\ast})=Y_{i,K}^{\ast})
 \\ \end{align} 
</blockquote>

<p><strong>Edit:</strong>
Sometimes you'll see notation like this:
$$
Pr( i \rightarrow k) = \Pr(\max(Y_{i,1}^{\ast},Y_{i,2}^{\ast},\ldots,Y_{i,K}^{\ast})=Y_{i,k}^{*}),
$$ 
where $i \rightarrow k$ means that individual $i$ chooses choice $k$, is fairly concise once the appropriate notation is defined. 
If you're looking for a useful way to put this into a likelihood function, you might see something like this:
$$
P_i = \prod_{k=1}^K Pr( i \rightarrow k)^{\mathbb 1_{i \rightarrow k}}, 
$$
where $\mathbb 1_{i \rightarrow k}$ is an indicator function that is equal to 1 when $i$ chooses choice $k$ and zero otherwise and $P_i$ is the PMF of the observation associated with individual $i$.</p>

<p><strong>Edit 2:</strong></p>

<p>For an explicit formula, see the same Wikipedia article. Check out, for instance, the section with the following:</p>

<blockquote>
  <p>As a result, it is conventional to set $C = -\boldsymbol\beta_K$ (or alternatively, one of the other coefficient vectors).  Essentially, we set the constant so that one of
  the vectors becomes 0, and all of the other vectors get transformed
  into the difference between those vectors and the vector we chose. 
  This is equivalent to "pivoting" around one of the $K$ choices, and
  examining how much better or worse all of the other $K-1$ choices
  are, relative to the choice are pivoting around.  Mathematically, we
  transform the coefficients as follows:</p>
  
   \begin{align} \boldsymbol\beta'_1 &amp;= \boldsymbol\beta_1 -
 \boldsymbol\beta_K \\ \cdots &amp; \cdots \\ \boldsymbol\beta'_{K-1} &amp;=
 \boldsymbol\beta_{K-1} - \boldsymbol\beta_K \\ \boldsymbol\beta'_K &amp;=
 0 \end{align} 
  
  <p>This leads to the following equations:</p>
  
   \begin{align} \Pr(Y_i=1) &amp;= \frac{e^{\boldsymbol\beta'_1 \cdot
 \mathbf{X}_i}}{1 + \sum_{k=1}^{K-1} e^{\boldsymbol\beta'_k \cdot
 \mathbf{X}_i}} \, \\ \cdots &amp; \cdots \\ \Pr(Y_i=K-1) &amp;=
 \frac{e^{\boldsymbol\beta'_{K-1} \cdot \mathbf{X}_i}}{1 +
 \sum_{k=1}^{K-1} e^{\boldsymbol\beta'_k \cdot \mathbf{X}_i}} \, \\
 \Pr(Y_i=K) &amp;= \frac{1}{1 + \sum_{k=1}^{K-1} e^{\boldsymbol\beta'_k
 \cdot \mathbf{X}_i}} \, \\ \end{align} 
</blockquote>

CC BY-SA 3.0
2017-10-16T13:49:15.377Z
2017-10-16T16:27:18.533Z
2017-10-16T16:27:18.533Z
jmbejara
<p>I know what the probability mass function of a multinomial logit model is.  However, I do not know what the probability mass  function of an individual observation i, f(yi|xi), is for a multinomial logit model.  I would really appreciate if someone could explain how to get the answer and knows the correct answer.  </p>

CC BY-SA 3.0
2017-10-16T11:54:28.323Z
I'm voting to close this question as off-topic because this belongs to [Cross-Validated](http://stats.stackexchange.com/).
luchonacho
CC BY-SA 3.0
2017-10-16T12:05:06.917Z
I disagree, I considered what category my question should be asked to.  I am studying econometrics and the topic of multinomial logit models is being discussed in all of my economics textbooks and by my professor. It is correct that this question could also be asked in the cross validated forum, but I would ask you not to delete my question in case anyone in econometrics or quantitative economics can help me. Thanks
Niam45
CC BY-SA 3.0
2017-10-16T13:24:02.607Z
I agree with @Amy that this question is on-topic. As explained in the help section, econometrics/statistics is on-topic here, even though there is overlap with cross-validated.se: https://economics.stackexchange.com/help/on-topic
jmbejara
CC BY-SA 3.0
2017-10-16T14:40:00.073Z
@jmbejara we are just fostering inefficiency. As it stands, the question is not specific about economics, and its likely to benefit more people in CV. With such policy, every single econometric CV question that has not been asked here is on-topic. Clearly not the best policy to pursue.
luchonacho
CC BY-SA 3.0
2017-10-16T15:09:24.163Z
@luchonacho Allowing the overlap is by design and is not unique to economics.SE. As evidenced by this post (https://meta.stackexchange.com/a/4713/299907), this is something that occurs throughout stack exchange, broadly. I like the current policy.
jmbejara
<econometrics>
How do I find the probability mass function of an individual observation for a multinomial logit model?
Niam45
# How do I find the probability mass function of an individual observation for a multinomial logit model?

(Asked by: username_0 on 2017-10-16)

I know what the probability mass function of a multinomial logit model is. However, I do not know what the probability mass function of an individual observation i, f(yi|xi), is for a multinomial logit model. I would really appreciate if someone could explain how to get the answer and knows the correct answer.

**Tags:** econometrics

> **Comments:**
> **username_1**: I'm voting to close this question as off-topic because this belongs to [Cross-Validated](http://stats.stackexchange.com/).  
> **username_0**: I disagree, I considered what category my question should be asked to.  I am studying econometrics and the topic of multinomial logit models is being discussed in all of my economics textbooks and by my professor. It is correct that this question could also be asked in the cross validated forum, but I would ask you not to delete my question in case anyone in econometrics or quantitative economics can help me. Thanks  
> **username_2**: I agree with @Amy that this question is on-topic. As explained in the help section, econometrics/statistics is on-topic here, even though there is overlap with cross-validated.se: https://economics.stackexchange.com/help/on-topic  
> **username_1**: @username_2 we are just fostering inefficiency. As it stands, the question is not specific about economics, and its likely to benefit more people in CV. With such policy, every single econometric CV question that has not been asked here is on-topic. Clearly not the best policy to pursue.  
> **username_2**: @username_1 Allowing the overlap is by design and is not unique to economics.SE. As evidenced by this post (https://meta.stackexchange.com/a/4713/299907), this is something that occurs throughout stack exchange, broadly. I like the current policy.  

---

## 1 answers

### Answer by username_2 on 2017-10-16. Score: 1 | Accepted answer

A useful way to view the model is as follows. For each observation $i$, there are $K$ different possible observations (choices). So, there is a random variable $\varepsilon\_{i,k}$ for each possible choice. We observe which choice is made and the values of the explanatory variables $\boldsymbol X\_i$. Then, for a given distribution of the random variables $\varepsilon\_{i,k}$ (given parameter values of the Type-1 EV), we know the conditional probability of picking choice $k$ conditional on $\boldsymbol X\_i$.

This is explained in the following part of the relevant Wikipedia [article](https://en.wikipedia.org/wiki/Multinomial_logistic_regression#As_a_latent-variable_model):

> It is also possible to formulate multinomial logistic regression as a
> latent variable model, following the two-way latent variable model
> described for binary logistic regression. This formulation is common
> in the theory of discrete choice models, and makes it easier to
> compare multinomial logistic regression to the related multinomial
> probit model, as well as to extend it to more complex models.
>
> Imagine that, for each data point $i$ and possible outcome $k$,
> there is a continuous latent variable
> $Y\_{i,k}^\*$ (i.e. an unobserved random
> variable) that is distributed as follows:
>
> $$\begin{align} Y\_{i,1}^{\ast} &= \boldsymbol\beta\_1 \cdot
> \mathbf{X}\_i + \varepsilon\_1 \, \\ Y\_{i,2}^{\ast} &=
> \boldsymbol\beta\_2 \cdot \mathbf{X}\_i + \varepsilon\_2 \, \\ \cdots &
> \\ Y\_{i,K}^{\ast} &= \boldsymbol\beta\_K \cdot \mathbf{X}\_i +
> \varepsilon\_K \, \\ \end{align} $$
>
> where $\varepsilon\_k \sim \operatorname{EV}\_1(0,1),$ i.e. a
> standard type-1 extreme value distribution.
>
> This latent variable can be thought of as the utility associated
> with data point $i$ choosing outcome $k$, where there is some
> randomness in the actual amount of utility obtained, which accounts
> for other unmodeled factors that go into the choice. The value of the
> actual variable $Y\_i$ is then determined in a non-random
> fashion from these latent variables (i.e. the randomness has been
> moved from the observed outcomes into the latent variables), where
> outcome $k$ is chosen if and only if the associated utility (the
> value of $Y\_{i,k}^{\ast}$) is greater than the utilities of
> all the other choices, i.e. if the utility associated with outcome
> $k$ is the maximum of all the utilities. Since the latent variables
> are continuous, the probability of two having
> exactly the same value is 0, so we ignore the scenario. That is:
>
> \begin{align} \Pr(Y\_i = 1) &= \Pr(Y\_{i,1}^{\ast} >
> Y\_{i,2}^{\ast} \text{ and } Y\_{i,1}^{\ast} > Y\_{i,3}^{\ast}\text{ and
> } \cdots \text{ and } Y\_{i,1}^{\ast} > Y\_{i,K}^{\ast}) \\ \Pr(Y\_i = 2)
> &= \Pr(Y\_{i,2}^{\ast} > Y\_{i,1}^{\ast} \text{ and } Y\_{i,2}^{\ast} >
> Y\_{i,3}^{\ast}\text{ and } \cdots \text{ and } Y\_{i,2}^{\ast} >
> Y\_{i,K}^{\ast}) \\ \cdots & \\ \Pr(Y\_i = K) &= \Pr(Y\_{i,K}^{\ast} >
> Y\_{i,1}^{\ast} \text{ and } Y\_{i,K}^{\ast} > Y\_{i,2}^{\ast}\text{ and
> } \cdots \text{ and } Y\_{i,K}^{\ast} > Y\_{i,K-1}^{\ast}) \\
> \end{align}
>
> Or equivalently:
>
> \begin{align} \Pr(Y\_i = 1) &=
> \Pr(\max(Y\_{i,1}^{\ast},Y\_{i,2}^{\ast},\ldots,Y\_{i,K}^{\ast})=Y\_{i,1}^{\ast})
> \\ \Pr(Y\_i = 2) &=
> \Pr(\max(Y\_{i,1}^{\ast},Y\_{i,2}^{\ast},\ldots,Y\_{i,K}^{\ast})=Y\_{i,2}^{\ast})
> \\ \cdots & \\ \Pr(Y\_i = K) &=
> \Pr(\max(Y\_{i,1}^{\ast},Y\_{i,2}^{\ast},\ldots,Y\_{i,K}^{\ast})=Y\_{i,K}^{\ast})
> \\ \end{align}

**Edit:**
Sometimes you'll see notation like this:
$$
Pr( i \rightarrow k) = \Pr(\max(Y\_{i,1}^{\ast},Y\_{i,2}^{\ast},\ldots,Y\_{i,K}^{\ast})=Y\_{i,k}^{\*}),
$$
where $i \rightarrow k$ means that individual $i$ chooses choice $k$, is fairly concise once the appropriate notation is defined.
If you're looking for a useful way to put this into a likelihood function, you might see something like this:
$$
P\_i = \prod\_{k=1}^K Pr( i \rightarrow k)^{\mathbb 1\_{i \rightarrow k}},
$$
where $\mathbb 1\_{i \rightarrow k}$ is an indicator function that is equal to 1 when $i$ chooses choice $k$ and zero otherwise and $P\_i$ is the PMF of the observation associated with individual $i$.

**Edit 2:**

For an explicit formula, see the same Wikipedia article. Check out, for instance, the section with the following:

> As a result, it is conventional to set $C = -\boldsymbol\beta\_K$ (or alternatively, one of the other coefficient vectors). Essentially, we set the constant so that one of
> the vectors becomes 0, and all of the other vectors get transformed
> into the difference between those vectors and the vector we chose.
> This is equivalent to "pivoting" around one of the $K$ choices, and
> examining how much better or worse all of the other $K-1$ choices
> are, relative to the choice are pivoting around. Mathematically, we
> transform the coefficients as follows:
>
> \begin{align} \boldsymbol\beta'\_1 &= \boldsymbol\beta\_1 -
> \boldsymbol\beta\_K \\ \cdots & \cdots \\ \boldsymbol\beta'\_{K-1} &=
> \boldsymbol\beta\_{K-1} - \boldsymbol\beta\_K \\ \boldsymbol\beta'\_K &=
> 0 \end{align}
>
> This leads to the following equations:
>
> \begin{align} \Pr(Y\_i=1) &= \frac{e^{\boldsymbol\beta'\_1 \cdot
> \mathbf{X}\_i}}{1 + \sum\_{k=1}^{K-1} e^{\boldsymbol\beta'\_k \cdot
> \mathbf{X}\_i}} \, \\ \cdots & \cdots \\ \Pr(Y\_i=K-1) &=
> \frac{e^{\boldsymbol\beta'\_{K-1} \cdot \mathbf{X}\_i}}{1 +
> \sum\_{k=1}^{K-1} e^{\boldsymbol\beta'\_k \cdot \mathbf{X}\_i}} \, \\
> \Pr(Y\_i=K) &= \frac{1}{1 + \sum\_{k=1}^{K-1} e^{\boldsymbol\beta'\_k
> \cdot \mathbf{X}\_i}} \, \\ \end{align}


---

