Speeding up inferencing in a neural network by copying output from one layer of the neural network to another computing resource based on dependencies among layers in the network. A processor causes two subsequent layers of neural networks 106 to be performed on separate computing resources 108, 110 from a previous layer of the neural network, based on whether a same output of a previous layer is to be provided to both subsequent layers. For example, an output from an encoder is duplicated: the output being provided to a second encoder on a processor, the duplicated output being copied to a decoder on a different processor. The processor can be a graphic processing unit GPU. This way, modified neural networks to reduce dependencies of layers are partitioned to be trained using pipeline parallelism. Applications comprise pipeline-based parallel U-Net based on a designed sequential U-Net, where dependency in skip connection is broken facilitating automated partition in automated parallelism while using synchronous stochastic gradient descent and pipeline parallelism technique. Used in medical image segmentation.
Techniques for modifying and training a neural network