What is the final output you want? It is dependent on that as well. If you apply the same linear layer to the two features, there would be symmetry if the ordering of the output did not matter. What is the output that you are expecting from your model?