Answers.Body: <p>This question is better suited for StackOverflow has it is more about coding rather than AI per se, but I did something like this recently, so here is what I ended up with:</p>
  <p>First, you generate a random scalar for each node, then you identify the maximum value for each graph. This will point to a random node, one per graph. Here is a rough sketch:</p>
  <ol>
  <li>Randomly sample a vector <span class="math-container">$z \sim U(0, 1) \in \mathbb{R}^N$</span>, with <span class="math-container">$N$</span> being the number of nodes</li>
  <li>Perform <code>scatter</code> with <code>max</code>-reduction: <code>graph_choose = scatter(z, batch, reduction='max')</code></li>
  <li>Perform gather: <code>graph_choose = graph_choose[batch]</code></li>
  <li>Compare the random vector with the graph-maximums: <code>choice_mask = z == graph_choose</code>. <code>choice_mask</code> will have a single <code>True</code> entry per graph, which is your randomly selected node.</li>
  <li>(optionally) Apply mask to collect the random nodes the nodes: <code>X_select = X[choice_mask]</code></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-28T16:27:32.463Z
Answers.Text: Thank you so much. Choosing the maximum over uniform distributions should lead to a Discrete Uniform distribution over the nodes. Nonetheless, I'm not looking for a DUniform. I have the sampling probabilities given to me for each node. Are you aware of a sampling method that uses the same logic you used by perhaps augmenting z by p (given probabilities).
Answers.UserDisplayName: moe asal
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-28T16:29:17.083Z
Answers.Text: I tried multiplying z*p element-wise and choosing the maximum. Nonetheless, this proved to be mathematically wrong.
Answers.UserDisplayName: moe asal
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-28T17:46:20.413Z
Answers.Text: Ah, I misread, sry. Yes, you can extend the method to probabilities. Instead of the uniform distribution, you can use a Gumbel distribution: 1) Convert your probabilities to logits 2) sample a vector from the [Gumbel distribution](https://pytorch.org/docs/stable/distributions.html#torch.distributions.gumbel.Gumbel) centered at your logits `G(<your logits>, 1)`, continue with step 2) of my answer. This should work as the Gumbel distribution can be used as a substitute for categorical sampling.
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-01T14:38:17.770Z
Answers.Text: I'll leave this here for any future reader: https://arxiv.org/abs/1411.0030
Answers.UserDisplayName: moe asal
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-02T07:04:24.487Z
Answers.Text: Nice, I hope this question will not be closed, I will update my answer later and also reference this article :)
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-28T14:34:54.770Z
Answers.LastActivityDate: 2024-04-28T14:34:54.770Z
Answers.UserDisplayName: Chillston
Body: <p>I am working with pygeometirc Batch objects. But, for those unfamiliar with it, I will pose my question as follows:</p>
  <p>I am given a 1d tensor <code>Y</code> and a 1d tensor <code>batch</code>.
  <code>Y</code> has the shape <code>(num_of_nodes, )</code> and is the nodes' probabilities concatenated over the different graphs.
  <code>batch</code> has the shape <code>(num_of_nodes, )</code> and specifies the graph from which each node comes. Example:</p>
  <p><code>Y = [0.2, 0.8, 1.]</code></p>
  <p><code>batch = [0, 0, 1]</code></p>
  <p>The first two nodes come from the first graph and the last from the second graph in the batch. Note that <code>batch</code> is always sorted.</p>
  <p>I want to sample one node from each graph in the batch in a <strong>vectorized way</strong>. I am aware of a sequential way by splitting the vector and creating Categorical distributions, but this is too inefficient.</p>
  <p>Example output:
  <code>out = [0, 2]</code> or <code>out = [1, 2]</code></p>
  <p>Restarting the count at each graph (<code>out=[0, 0]</code> or <code>out=[1, 0]</code>) is fine, but not desirable.</p>
  
Tags: <graph-neural-networks>
Title: sample one node from each graph in a batch with pytorch
UserDisplayName: moe asal
ThreadText: # sample one node from each graph in a batch with pytorch
  
  (Asked by: username_0 on 2024-04-27)
  
  I am working with pygeometirc Batch objects. But, for those unfamiliar with it, I will pose my question as follows:
  
  I am given a 1d tensor `Y` and a 1d tensor `batch`.
  `Y` has the shape `(num_of_nodes, )` and is the nodes' probabilities concatenated over the different graphs.
  `batch` has the shape `(num_of_nodes, )` and specifies the graph from which each node comes. Example:
  
  `Y = [0.2, 0.8, 1.]`
  
  `batch = [0, 0, 1]`
  
  The first two nodes come from the first graph and the last from the second graph in the batch. Note that `batch` is always sorted.
  
  I want to sample one node from each graph in the batch in a **vectorized way**. I am aware of a sequential way by splitting the vector and creating Categorical distributions, but this is too inefficient.
  
  Example output:
  `out = [0, 2]` or `out = [1, 2]`
  
  Restarting the count at each graph (`out=[0, 0]` or `out=[1, 0]`) is fine, but not desirable.
  
  **Tags:** graph-neural-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-04-28. Score: 1 | Accepted answer
  
  This question is better suited for StackOverflow has it is more about coding rather than AI per se, but I did something like this recently, so here is what I ended up with:
  
  First, you generate a random scalar for each node, then you identify the maximum value for each graph. This will point to a random node, one per graph. Here is a rough sketch:
  
  1. Randomly sample a vector $z \sim U(0, 1) \in \mathbb{R}^N$, with $N$ being the number of nodes
  2. Perform `scatter` with `max`-reduction: `graph_choose = scatter(z, batch, reduction='max')`
  3. Perform gather: `graph_choose = graph_choose[batch]`
  4. Compare the random vector with the graph-maximums: `choice_mask = z == graph_choose`. `choice_mask` will have a single `True` entry per graph, which is your randomly selected node.
  5. (optionally) Apply mask to collect the random nodes the nodes: `X_select = X[choice_mask]`
  
  > **Comments:**
  > **username_0**: Thank you so much. Choosing the maximum over uniform distributions should lead to a Discrete Uniform distribution over the nodes. Nonetheless, I'm not looking for a DUniform. I have the sampling probabilities given to me for each node. Are you aware of a sampling method that uses the same logic you used by perhaps augmenting z by p (given probabilities).  
  > **username_0**: I tried multiplying z*p element-wise and choosing the maximum. Nonetheless, this proved to be mathematically wrong.  
  > **username_1**: Ah, I misread, sry. Yes, you can extend the method to probabilities. Instead of the uniform distribution, you can use a Gumbel distribution: 1) Convert your probabilities to logits 2) sample a vector from the [Gumbel distribution](https://pytorch.org/docs/stable/distributions.html#torch.distributions.gumbel.Gumbel) centered at your logits `G(<your logits>, 1)`, continue with step 2) of my answer. This should work as the Gumbel distribution can be used as a substitute for categorical sampling.  
  > **username_0**: I'll leave this here for any future reader: https://arxiv.org/abs/1411.0030  
  > **username_1**: Nice, I hope this question will not be closed, I will update my answer later and also reference this article :)  
  
  ---
  
  