Thank you for your reply!
Yes, my outside program cannot generate valid gradients. Instead, it only spits out numbers. I decided to connect a second NN after the first NN to approximate my outside program. If the approximation is good enough, then I can train my original NN with the gradients provided by the second NN.