Answers.Body: <p>When I try to understand convolutional layer, I always look at the shape of filters. I try to stack matrices in my mind and visualize the convolution.</p>
  <p>If we consider that each point has 3 features (x, y, z), than k will be equal 3 in this particular layer <code> self.conv1 = torch.nn.Conv1d(k, 64, 1)</code>. The weight matrix of this layer will have the shape (64, 3, 1). Meaning we have 64 filters in total, each with shape (3,1).</p>
  <p>You can think of 1 filter as the weight in a neuron. The number of neurons will be equal to the number of filters. The input to your neuron would be the point, that has shape(1,3). This point will then get multiplied by the weight matrix ( aka. X * W + b, bias is optional). Each point will be go through each filter, that is why I think they call it &quot;shared&quot;.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-29T12:04:32.493Z
Answers.LastActivityDate: 2024-05-29T12:04:32.493Z
Answers.UserDisplayName: Mihnea Aleman
Body: <p>I am studying the architecture of the network <em>pointnet</em>, specifically the MLPs stages of the pipeline highlighted in red in the following image (taken from the author page <a href="http://stanford.edu/%7Erqi/pointnet/" rel="nofollow noreferrer">here</a>):</p>
  <p><a href="https://i.sstatic.net/PaEdl.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PaEdl.jpg" alt="pointnet architecture" /></a></p>
  <p>It is strange to find out that this part was implemented with 2d convolutional layers by the authors, here is the snippet where they create the MLP(64,64) part (<a href="https://github.com/charlesq34/pointnet/blob/master/models/pointnet_cls.py" rel="nofollow noreferrer">here you'll find the complete code base</a>):</p>
  <pre><code> net = tf_util.conv2d(input_image, 64, [1,3],
                       padding='VALID', stride=[1,1],
                       bn=True, is_training=is_training,
                       scope='conv1', bn_decay=bn_decay)
  net = tf_util.conv2d(net, 64, [1,1],
                       padding='VALID', stride=[1,1],
                       bn=True, is_training=is_training,
                       scope='conv2', bn_decay=bn_decay)
  </code></pre>
  <p>Same thing happened when I looked into the pytorch implementation, but now with 1d convolutional layers! (<a href="https://github.com/fxia22/pointnet.pytorch/blob/master/pointnet/model.py" rel="nofollow noreferrer">code base here</a>):</p>
  <pre><code> self.conv1 = torch.nn.Conv1d(k, 64, 1)
      self.conv2 = torch.nn.Conv1d(64, 128, 1)
      self.conv3 = torch.nn.Conv1d(128, 1024, 1)
  </code></pre>
  <p>The extent of my knowledge is that MLPs are densely connected networks (<a href="https://www.youtube.com/watch?v=aircAruvnKk" rel="nofollow noreferrer">MLP definition</a>), even more so, in <a href="https://www.mdpi.com/1999-4907/12/2/131" rel="nofollow noreferrer">this application of pointnet</a> the authors depict the MLP stages as fully connected networks:</p>
  <p><a href="https://i.sstatic.net/J96c8.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/J96c8.png" alt="enter image description here" /></a></p>
  <p>I am lost here, for what I can gather from the images there should be a dedicated MLP to transform every point independently, how does convolution help in this situation? Your insight will be greatly appreciated.</p>
  <p>ERRDATA</p>
  <p>I have just figured that the kernel size in the original tensorflow implementation is [1,3], meaning, that is also a 1D convolution, are they feeding the convolution of the three channels to the MLP?</p>
  <p>Regards</p>
  
Tags: <keras><pytorch><feedforward-neural-networks><multilayer-perceptrons>
Title: How can an MLP be implemented with convolutional layers?
UserDisplayName: Jacob Morales Gonzalez
ThreadText: # How can an MLP be implemented with convolutional layers?
  
  (Asked by: username_0 on 2023-05-01)
  
  I am studying the architecture of the network pointnet, specifically the MLPs stages of the pipeline highlighted in red in the following image (taken from the author page here):
  
  It is strange to find out that this part was implemented with 2d convolutional layers by the authors, here is the snippet where they create the MLP(64,64) part (here you'll find the complete code base):
   net = tf_util.conv2d(input_image, 64, [1,3],
                       padding='VALID', stride=[1,1],
                       bn=True, is_training=is_training,
                       scope='conv1', bn_decay=bn_decay)
  net = tf_util.conv2d(net, 64, [1,1],
                       padding='VALID', stride=[1,1],
                       bn=True, is_training=is_training,
                       scope='conv2', bn_decay=bn_decay)
  
  Same thing happened when I looked into the pytorch implementation, but now with 1d convolutional layers! (code base here):
   self.conv1 = torch.nn.Conv1d(k, 64, 1)
      self.conv2 = torch.nn.Conv1d(64, 128, 1)
      self.conv3 = torch.nn.Conv1d(128, 1024, 1)
  
  The extent of my knowledge is that MLPs are densely connected networks (MLP definition), even more so, in this application of pointnet the authors depict the MLP stages as fully connected networks:
  
  I am lost here, for what I can gather from the images there should be a dedicated MLP to transform every point independently, how does convolution help in this situation? Your insight will be greatly appreciated.
  ERRDATA
  I have just figured that the kernel size in the original tensorflow implementation is [1,3], meaning, that is also a 1D convolution, are they feeding the convolution of the three channels to the MLP?
  Regards
  
  **Tags:** keras, pytorch, feedforward-neural-networks, multilayer-perceptrons
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-05-29. Score: 1
  
  When I try to understand convolutional layer, I always look at the shape of filters. I try to stack matrices in my mind and visualize the convolution.
  
  If we consider that each point has 3 features (x, y, z), than k will be equal 3 in this particular layer  `self.conv1 = torch.nn.Conv1d(k, 64, 1)`. The weight matrix of this layer will have the shape (64, 3, 1). Meaning we have 64 filters in total, each with shape (3,1).
  
  You can think of 1 filter as the weight in a neuron. The number of neurons will be equal to the number of filters. The input to your neuron would be the point, that has shape(1,3). This point will then get multiplied by the weight matrix ( aka. X \* W + b, bias is optional). Each point will be go through each filter, that is why I think they call it "shared".
  
  
  ---
  
  