Transformer models create embeddings based on context. If the model doesn’t know where a sentence or document begins and ends, it’s harder for it to determine context, and the resulting embeddings could be affected. If information from an irrelevant context bleeds into the embedding for a word, it can only be a bad thing. As for whether it’s significant, it’s a huge “it depends”. But, unless by some luck things turn out just right, the model will always be a little bit worse.