I am trying to implement the following general NN model (Not CNN) using Pytorch.

Boki3HCcAbmFrCHLfyLEOUaQOQlTffYSGi5iAzGyne_dG3YUbonkl5O8n0iUcWSDQQHdxiZON-l2SYksF8L2RI5hjO4MQTRGQPdbtJqn0Z1EIQAf8YnhSvVNnqNwJXLUGpoQRMvowvQ768×584

Here, 3rd, 4th, 5th layers are fully connected-- and Network 1,2, 3 itself are fully connected but they are not fully connected to each other.
I don’t know how to implement this kind of selected (Not Random) sparse connection in Pytorch.
Any help/comments on this are much appreciated.