Question: In *Fringe*, there is the "Over Here" universe and the "Over there" universe that the characters interact with.

Why do only these two universes exist? Dr. Bishop mentioned that there were an unlimited number of universes existing at the same time.

I thought it was because it was the other universe that Walter went into that caused all the Fringe incidents, but in Walters experiments, he created a wall that only looked into the other universe, not any of the other universes.

Why are these two universes so important and not all the others?

Answer: 
The universes are now connected because of the first cross over by Walter, so that's why it's possible to cross over between them in the present (although it continues not being easy for "normal people")

I think that Walter first created the window to that universe by chance, and the connection only intensified once he crossed over to save Peter.
To expand upon @Juan Manuel's answer a bit:

Walter originally built a device that could view other parallel universes, based on the Fringe universe's version of the many worlds theory. He used that device to watch alternate versions of himself try to solve the problem of Peter's fatal disease, something that he was unable to do in time. Walter found many such universes, none of them sucessfully curing Peter. He was focused in on one particular world where the alternate-him was close to solving the problem when things went sideways.

He just happened to see Walternate's experiment succeed, at the same moment that Observer September distracted Walternate, causing him to miss the result. Walter then decided that he needed to actually *go* to that other universe and save Peter. (Fully intending, I believe, to return Peter when he was done.) That's when Walter built the actual bridge device to travel to that universe and retrieve Peter, cure him, and return him.

When he finally went through with his plan, the end result was that Peter stayed in our world, and now there was an imbalance between the two worlds. That tied those two specific universes together in a way that other universes were not. Walter closed the bridge, but it was still "there" in some sense that the two universes were connected and "moving together".

In theory, Walter's bridge device could take someone to any other universe, assuming they were able to "find" it and calibrate the machine correctly. It's simply that our universe and the prime-universe were the easiest to bridge (having been done once), and there was little incentive for any anyone from either universe to find a third one.