Answers.Body: <p>Just for completeness, here is one simple formalization of a diffusion GCN (<a href="https://proceedings.neurips.cc/paper/2019/file/23c894276a2c5a16470e6a31f4618d73-Paper.pdf" rel="nofollow noreferrer">Gasteiger et al.</a>):</p>
  <p><span class="math-container">$\text{D-GCN}(X) = \sum_{k=1}^K A^k X W_k$</span></p>
  <p>You have a diffusion factor <span class="math-container">$k \in [1 .. K]$</span> and you apply different GCNs with parameters <span class="math-container">$W_k$</span> to the <span class="math-container">$1$</span>-hop, <span class="math-container">$2$</span>-hop, .., <span class="math-container">$K$</span>-hop neighborhood and sum the results together. <span class="math-container">$A^k$</span> denotes the <span class="math-container">$k^{th}$</span> power of the adjacency matrix.</p>
  <p>There are two principles in visual CNNs that come to my mind that are comparable to the diffusion applied in graph models.</p>
  <p>Firstly, <a href="https://iq.opengenus.org/dilated-convolution/" rel="nofollow noreferrer"><strong>dilated convolutions</strong></a> apply a kind of diffusion (they are also referred to as Atrous convolution). Specifically, the convolutional kernel is spread out based on a dilation factor <span class="math-container">$d$</span>. The dilation factor determines the number of pixels that are skipped when applying the kernel. For example, if you have a 3x3 kernel, and apply a dilation rate of <span class="math-container">$d=1$</span>, you get the standard convolution, a dilation rate of <span class="math-container">$d=2$</span> applies the 3x3 kernel to a 5x5 area of the image by using only every 2nd pixel (see the Image below). So in that case <span class="math-container">$d$</span> determines which pixel neighborhood the kernel is applied to, which is comparable to the <span class="math-container">$k$</span> in the diffusion GCN.
  Dilated convolution has a history in segmentation models (see <a href="https://arxiv.org/abs/1511.07122" rel="nofollow noreferrer">Yu et al.</a> and <a href="https://arxiv.org/abs/1606.00915" rel="nofollow noreferrer">Chen et al.</a>). Apparently, it helps with segmentation, because the dilated convolutions can capture the context of pixels a bit better, so segmentation maps become cleaner.
  It has also been applied with 1-dimensional convolutions in the <a href="https://arxiv.org/abs/1609.03499" rel="nofollow noreferrer">WaveNet</a> architecture. In that case dilation was used to cover greater receptive fields without increasing the number of parameters, i.e. model complexity.</p>
  <p><a href="https://i.sstatic.net/Symgz.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Symgz.png" alt="enter image description here" /></a></p>
  <p><em>The green is the output, blue is the input and the grayed pixels are the pixels where kernel weights are applied to. Image taken from <a href="https://iq.opengenus.org/dilated-convolution/" rel="nofollow noreferrer">here</a></em></p>
  <p>Second, the <strong>Inception architecture</strong> by <a href="https://arxiv.org/abs/1409.4842" rel="nofollow noreferrer">Szegedy et al.</a> applies something that can be interpreted as a diffusion as well: Inception computes convolutions of different kernel sizes in parallel and stacks their feature maps (<a href="https://deepai.org/machine-learning-glossary-and-terms/inception-module" rel="nofollow noreferrer">this blog post</a> has a nice visualization). Here, the different kernel sizes would correspond to the <span class="math-container">$k$</span> and stacking is used instead of a sum.</p>
  <p>I think these two examples are definitely related to the idea of diffusion GCN.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-07T15:25:14.227Z
Answers.LastActivityDate: 2022-07-07T17:38:41.443Z
Answers.LastEditDate: 2022-07-07T17:38:41.443Z
Answers.UserDisplayName: Chillston
Body: <p>I'm pretty new to graph neural networks, so please forgive me if this is a silly question.</p>
  <p>Diffusion is a method used to improve graph CNNs, however it seems to me that general CNNs can also benefit from taking into consideration diffusion-like processes (for example, in CV, one may want information to diffuse from a small neighborhood, then to a bigger neighborhood, etc.). Also, PDE-based methods have been used in CV traditionally (before deep learning became a thing), so there's at least some evidence that this might work. But I did search extensively, and I can't find any information on using diffusion for general CNNs!</p>
  <p>Therefore, what I'm curious about is: why didn't I find any information about this? Is it already explored under a different name? Or does it not really work because of some reason I don't know (or didn't think about)? Or maybe it does work, but there are some technical difficulties?</p>
  
Tags: <convolutional-neural-networks><geometric-deep-learning><graph-neural-networks>
Title: Why don't we use diffusion for non-graph CNNs?
UserDisplayName: xuq01
ThreadText: # Why don't we use diffusion for non-graph CNNs?
  
  (Asked by: username_0 on 2022-07-03)
  
  I'm pretty new to graph neural networks, so please forgive me if this is a silly question.
  
  Diffusion is a method used to improve graph CNNs, however it seems to me that general CNNs can also benefit from taking into consideration diffusion-like processes (for example, in CV, one may want information to diffuse from a small neighborhood, then to a bigger neighborhood, etc.). Also, PDE-based methods have been used in CV traditionally (before deep learning became a thing), so there's at least some evidence that this might work. But I did search extensively, and I can't find any information on using diffusion for general CNNs!
  
  Therefore, what I'm curious about is: why didn't I find any information about this? Is it already explored under a different name? Or does it not really work because of some reason I don't know (or didn't think about)? Or maybe it does work, but there are some technical difficulties?
  
  **Tags:** convolutional-neural-networks, geometric-deep-learning, graph-neural-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-07. Score: 2 | Accepted answer
  
  Just for completeness, here is one simple formalization of a diffusion GCN ([Gasteiger et al.](https://proceedings.neurips.cc/paper/2019/file/23c894276a2c5a16470e6a31f4618d73-Paper.pdf)):
  
  $\text{D-GCN}(X) = \sum\_{k=1}^K A^k X W\_k$
  
  You have a diffusion factor $k \in [1 .. K]$ and you apply different GCNs with parameters $W\_k$ to the $1$-hop, $2$-hop, .., $K$-hop neighborhood and sum the results together. $A^k$ denotes the $k^{th}$ power of the adjacency matrix.
  
  There are two principles in visual CNNs that come to my mind that are comparable to the diffusion applied in graph models.
  
  Firstly, [**dilated convolutions**](https://iq.opengenus.org/dilated-convolution/) apply a kind of diffusion (they are also referred to as Atrous convolution). Specifically, the convolutional kernel is spread out based on a dilation factor $d$. The dilation factor determines the number of pixels that are skipped when applying the kernel. For example, if you have a 3x3 kernel, and apply a dilation rate of $d=1$, you get the standard convolution, a dilation rate of $d=2$ applies the 3x3 kernel to a 5x5 area of the image by using only every 2nd pixel (see the Image below). So in that case $d$ determines which pixel neighborhood the kernel is applied to, which is comparable to the $k$ in the diffusion GCN.
  Dilated convolution has a history in segmentation models (see [Yu et al.](https://arxiv.org/abs/1511.07122) and [Chen et al.](https://arxiv.org/abs/1606.00915)). Apparently, it helps with segmentation, because the dilated convolutions can capture the context of pixels a bit better, so segmentation maps become cleaner.
  It has also been applied with 1-dimensional convolutions in the [WaveNet](https://arxiv.org/abs/1609.03499) architecture. In that case dilation was used to cover greater receptive fields without increasing the number of parameters, i.e. model complexity.
  
  [![enter image description here](https://i.sstatic.net/Symgz.png)](https://i.sstatic.net/Symgz.png)
  
  *The green is the output, blue is the input and the grayed pixels are the pixels where kernel weights are applied to. Image taken from [here](https://iq.opengenus.org/dilated-convolution/)*
  
  Second, the **Inception architecture** by [Szegedy et al.](https://arxiv.org/abs/1409.4842) applies something that can be interpreted as a diffusion as well: Inception computes convolutions of different kernel sizes in parallel and stacks their feature maps ([this blog post](https://deepai.org/machine-learning-glossary-and-terms/inception-module) has a nice visualization). Here, the different kernel sizes would correspond to the $k$ and stacking is used instead of a sum.
  
  I think these two examples are definitely related to the idea of diffusion GCN.
  
  
  ---
  
  