Memory for network parameters:
(256*5000 + 5000) * 4 * 2 = 10 Mbytes, where the factor of 2 is because the network has 1 tensor for weights and 1 tensor for gradients, and the additional 5000 is for biases.
Memory for data:
8192 * 512 * 4 * 2 = 32 Mbytes
So by those rough calculations, the memory consumption for the softmax layer is roughly 42 Mbytes.