The batch size should be in dim0, so if you are passing a batch with 4 samples, your output shape would be correct.
Each feature of this batch has 512 values, yes.
Let me know, if I misunderstood your question.