Normally you do not need to unit-test the model itself. You can use assert statements to check that the outputs of different layers are as expected, but normally any error there would cause the next layer to fail. Just make sure that the architecture of the network is implemented as in article.
I was referring to unit testing all the other code, 95% of cases you will get such errors from:

erroneous ground truth data
erroneous loss function
erroneous conversion (i.e., model does the job well but you have an error on using the result: swapped some axis or channel, mapped something wrongly etc)

Randomly trying out 1000 things is not a good idea, rigorously testing each and every line of code may look tedious but actually is much faster and leads to better results.