It seems the invocation order of hooks on tensors is determined by the order of calling regsiter_hook()

Yes they are called in the order they were installed.

Is there a way to alter the invocation order after registration?

There is no official way to do this no, you will need to uninstall and reinstall them.
The way these hooks are implemented is a bit complex as it depends on where the Tensor is in the graph so I don’t think you can hack your way around that right now. Or you will have to read the code related to hooks in the engine but I don’t think we expose the internal structures to python anyways.