Aureate:

Is there a difference between having our input shape as (1,50,384) or (50,1,384).


Yes



 Aureate:

Does the latter mean that each datapoint is basically treated independently of the others?


Yes. Basically, transition is the same, but it is applied to different vectors (accumulated context + step input concatenated). What you refer to as “memory” is a transformation matrix, applied multiple times, roughly speaking.