i guess from the the looks of it the implementation seems fine to me, maybe someone else can comment. the network “not doing so well” as a description is a very sparse to make a comment, i’ll make a general comment. networks with more parameters are prone to overfitting, where their train error goes down but the validation error does not do as well as compared to a network with less parameters. if this is the case , you can search for regularization as a keyword.
good luck