The problem here is terminology: physicists speaking informally often say "universe" when they really mean "observable universe" - that is, the part of the whole universe that we've so far been able to see.