<p>So I've done some more research after posting this and it looks like treating things as separate neural nets and connecting them later in subsequent hidden layers is a good idea. If anyone has other ideas that might be helpful, I'd love to hear, though!</p>

CC BY-SA 4.0
2018-10-29T06:14:08.730Z
2018-10-29T06:14:08.730Z
Pınar Demetci
<p>In case that you want to connect a placeholder to a new layer you can do as below</p>

<pre><code>x = tf.placeholder(shape=[None, 784], dtype=tf.float32) # mnist for example

x = tf.reshape(x, (-1, 784, 1)) # change to new shape (784 ,1)

x = tf.unstack(self._input,axis=1) # getting a list with 784 elements of 1

con = tf.concat(x[1,2,3,4,5]) # for instance you want only from 1 to 5 inputs/neurons
</code></pre>

<p>you can feed your new layer with above con which is only corresponds from the input 1 to 5.
you can apply same technique to a layer instead of tf.placeholder</p>

CC BY-SA 4.0
2019-04-07T17:48:56.173Z
2019-10-26T19:46:35.140Z
2019-10-26T19:46:35.140Z
Fivos Delemis
<p>I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer. So I want the architecture to be highly specified/customized from the get-go and I want the neural network to optimize the weights of the specified connections, while keeping everything else 0 during the forward pass AND the backpropagation (connection does not ever exist).</p>

<p>I am a complete beginner in neural networks. I have been recently working with tensorflow &amp; keras to construct fully connected deep networks. Is there anything in tensorflow (or something else) that I should look into that might allow me to do this? I think with tf, I should be able to specify the computational graph such that only certain connections exist but I really have no idea yet where to start from to do this...</p>

<p>I came across papers/posts on network pruning, but it doesn't seem really relevant to me. I don't want to go back and prune my network to make it less over-parameterized or eliminate insignificant connections. </p>

<p>I want the connections to be specified and the network to be relatively sparse from the initialisation and stay that way during the back-propagation.</p>

<neural-networks><machine-learning><deep-learning><deep-neural-networks><feedforward-neural-networks>
How to create Partially Connected NNs with prespecified connections using Tensorflow?
Pınar Demetci
# How to create Partially Connected NNs with prespecified connections using Tensorflow?

(Asked by: username_0 on 2018-10-05)

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer. So I want the architecture to be highly specified/customized from the get-go and I want the neural network to optimize the weights of the specified connections, while keeping everything else 0 during the forward pass AND the backpropagation (connection does not ever exist).

I am a complete beginner in neural networks. I have been recently working with tensorflow & keras to construct fully connected deep networks. Is there anything in tensorflow (or something else) that I should look into that might allow me to do this? I think with tf, I should be able to specify the computational graph such that only certain connections exist but I really have no idea yet where to start from to do this...

I came across papers/posts on network pruning, but it doesn't seem really relevant to me. I don't want to go back and prune my network to make it less over-parameterized or eliminate insignificant connections.

I want the connections to be specified and the network to be relatively sparse from the initialisation and stay that way during the back-propagation.

**Tags:** neural-networks, machine-learning, deep-learning, deep-neural-networks, feedforward-neural-networks


---

## 2 answers

### Answer by username_0 on 2018-10-29. Score: 0

So I've done some more research after posting this and it looks like treating things as separate neural nets and connecting them later in subsequent hidden layers is a good idea. If anyone has other ideas that might be helpful, I'd love to hear, though!


---

### Answer by username_1 on 2019-04-07. Score: 0

In case that you want to connect a placeholder to a new layer you can do as below
x = tf.placeholder(shape=[None, 784], dtype=tf.float32) # mnist for example

x = tf.reshape(x, (-1, 784, 1)) # change to new shape (784 ,1)

x = tf.unstack(self._input,axis=1) # getting a list with 784 elements of 1

con = tf.concat(x[1,2,3,4,5]) # for instance you want only from 1 to 5 inputs/neurons

you can feed your new layer with above con which is only corresponds from the input 1 to 5.
you can apply same technique to a layer instead of tf.placeholder


---

