The necessary course of action depends solely on how accessible are the other universes. How universes behave should also be considered.
Inaccessible Alternate Universes
If you know something is most probably true somewhere else, but have no way of knowing, then you have nothing to worry about how true that something is, not even where that something is. You simply have to stop caring, because you can't do anything. Really, you can't affect them and they can't affect you. Let each other be for each other's sake. If you can't make peace with that, declare war on the compartmentalized multiverse find out how to have access.
Accessible Alternate Universes
This is the problem. There are other universes, and there are a lot of ways we can affect them. Just remember they can affect us too. The ultimate problem here is defining the rules of the multiverse, so that we'll know if we can (and then how to) circumvent them.
Some Types of Access (and possible courses of action based on each)
1. Unlimited Parallel Universes: Our universe exists alongside countless others
This means that the cosmos is truly infinite. We can be anything, anyone, anywhere, rules does not really exist. Order comes after chaos. God is also Satan, somehow. 283
Do whatever you like, or not. Your choice. Anarchy reigns.
2. Limited Parallel Universes: Our universe exists alongside countable others
The cosmos is not infinite, but there are a lot more within it. This means rules between universes can be set to the bare minimum of not destroying each other. Universes are like people. Treat each other with respect and caution, should you choose to access another and still desire to safely return to your own.
3. Possible Universes: Our universe exists with others
The multiverse obeys probability and statistics, and some universes receive the anthropic bias of being the best among the rest, wherein the worst ones are eliminated by the best or some shit. Religion now, is to become the best universe, whatever that means.
4. Altered Universes: Other universes are not really universes
The laws of physics in other universes are not the same as with ours. We are not allowed to exist there as we exist here. We can go there, but we need interuniversal law translation devices. Religion now, is to create the best such device, and conquer all the others.
Note that all some of these types might actually be just one, or that the ones listed here are just what I can extemporaneously design.
Summary
Like all memetic structures, religion is alive. Depending on how one universe can affect another and how the meta-universe can affect all universes, religion shall adapt and evolve accordingly for it to survive. 283