Answers.Body: <p>So I've done some more research after posting this and it looks like treating things as separate neural nets and connecting them later in subsequent hidden layers is a good idea. If anyone has other ideas that might be helpful, I'd love to hear, though!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-29T06:14:08.730Z
Answers.LastActivityDate: 2018-10-29T06:14:08.730Z
Answers.UserDisplayName: Pınar Demetci
Answers.Body: <p>In case that you want to connect a placeholder to a new layer you can do as below</p>
  
  <pre><code>x = tf.placeholder(shape=[None, 784], dtype=tf.float32) # mnist for example
  
  x = tf.reshape(x, (-1, 784, 1)) # change to new shape (784 ,1)
  
  x = tf.unstack(self._input,axis=1) # getting a list with 784 elements of 1
  
  con = tf.concat(x[1,2,3,4,5]) # for instance you want only from 1 to 5 inputs/neurons
  </code></pre>
  
  <p>you can feed your new layer with above con which is only corresponds from the input 1 to 5.
  you can apply same technique to a layer instead of tf.placeholder</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-07T17:48:56.173Z
Answers.LastActivityDate: 2019-10-26T19:46:35.140Z
Answers.LastEditDate: 2019-10-26T19:46:35.140Z
Answers.UserDisplayName: Fivos Delemis
Body: <p>I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer. So I want the architecture to be highly specified/customized from the get-go and I want the neural network to optimize the weights of the specified connections, while keeping everything else 0 during the forward pass AND the backpropagation (connection does not ever exist).</p>
  
  <p>I am a complete beginner in neural networks. I have been recently working with tensorflow &amp; keras to construct fully connected deep networks. Is there anything in tensorflow (or something else) that I should look into that might allow me to do this? I think with tf, I should be able to specify the computational graph such that only certain connections exist but I really have no idea yet where to start from to do this...</p>
  
  <p>I came across papers/posts on network pruning, but it doesn't seem really relevant to me. I don't want to go back and prune my network to make it less over-parameterized or eliminate insignificant connections. </p>
  
  <p>I want the connections to be specified and the network to be relatively sparse from the initialisation and stay that way during the back-propagation.</p>
  
Tags: <neural-networks><machine-learning><deep-learning><deep-neural-networks><feedforward-neural-networks>
Title: How to create Partially Connected NNs with prespecified connections using Tensorflow?
UserDisplayName: Pınar Demetci
ThreadText: # How to create Partially Connected NNs with prespecified connections using Tensorflow?
  
  (Asked by: username_0 on 2018-10-05)
  
  I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer. So I want the architecture to be highly specified/customized from the get-go and I want the neural network to optimize the weights of the specified connections, while keeping everything else 0 during the forward pass AND the backpropagation (connection does not ever exist).
  
  I am a complete beginner in neural networks. I have been recently working with tensorflow & keras to construct fully connected deep networks. Is there anything in tensorflow (or something else) that I should look into that might allow me to do this? I think with tf, I should be able to specify the computational graph such that only certain connections exist but I really have no idea yet where to start from to do this...
  
  I came across papers/posts on network pruning, but it doesn't seem really relevant to me. I don't want to go back and prune my network to make it less over-parameterized or eliminate insignificant connections.
  
  I want the connections to be specified and the network to be relatively sparse from the initialisation and stay that way during the back-propagation.
  
  **Tags:** neural-networks, machine-learning, deep-learning, deep-neural-networks, feedforward-neural-networks
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2018-10-29. Score: 0
  
  So I've done some more research after posting this and it looks like treating things as separate neural nets and connecting them later in subsequent hidden layers is a good idea. If anyone has other ideas that might be helpful, I'd love to hear, though!
  
  
  ---
  
  ### Answer by username_1 on 2019-04-07. Score: 0
  
  In case that you want to connect a placeholder to a new layer you can do as below
  
  ```
  x = tf.placeholder(shape=[None, 784], dtype=tf.float32) # mnist for example
  
  x = tf.reshape(x, (-1, 784, 1)) # change to new shape (784 ,1)
  
  x = tf.unstack(self._input,axis=1) # getting a list with 784 elements of 1
  
  con = tf.concat(x[1,2,3,4,5]) # for instance you want only from 1 to 5 inputs/neurons
  
  ```
  
  you can feed your new layer with above con which is only corresponds from the input 1 to 5.
  you can apply same technique to a layer instead of tf.placeholder
  
  
  ---
  
  