Well, the charm of neural networks is that you’re basically calculating quite complex functions with mostly linear + pointwise nonlinearities.
In a bit more detail: Feeding the coordinates often, but not always, works well. For example, the “positional encoding” bit in the famous Attention is all you need paper introducing transformers is a case when people chose to not just put in the index.
So the gist here is: try simple, be fancy if it doesn’t work.
Another tangentially related occasion where just position doesn’t work is the loss function, where one might naively use mean squared error, but doesn’t whenever one has a credible alternative (like cross-entropy).
Best regards
Thomas