One of the things is that why are you not doing all the ops on GPU, might give you some speed up.
And again try to remove the for loop and vectorize the operations. Will require some tricks like you will have to unroll the matrix like for each ctx.width create a corresponding row or column of all ctx.weight instead of looping over.