Yes, those bullet holes have always been there since the glass was manufactured. From the inverted perspective, the building is eventually dismantled and the glass is driven backwards in a truck back to a factory then is melted down into sand.
According to the principle of "whatever happened, happened", the fact that the bullet holes are there, it must mean that they were installed that way. If there was, for example, a fastidious contractor who would have noticed the holes and thrown it out, then the original shots could not have ever happened. This is similar to how they looked through the window to see if their inverted selves were going into the turnstile. If you decided not to go into the turnstile, then you wouldn't see your inverted self going into the turnstile either. Or similarly with the inverted rounds jumping up into the characters' hands. It seems implausible but the construction of the room is not shown to the audience and you could come up with any explanation for why those holes are there.
There's a great novelette called "The Merchant and the Alchemist's Gate" by <PERSON>:
! In it, a character has a hoop where one side is delayed relative to the other. 900
If you put your hand in one side in you see your hand come out the other side a minute later. If you put your hand in the other side your hand comes out the other side a minute earlier. If you see your hand come out that side of hoop you will inevitably be putting your hand in the other side in one minute. If you decide you would not do that then your hand wouldn't have shown up in the first place.
If you manage to shoot someone and the bullet is embedded in their body, then you follow that bullet back to the person's birth, eventually that bullet is too big to fit inside that person. Either you have a paradox since the bullet is going to kill that person before they're old enough to get shot by that bullet (and one way to look at it is that the paradox could not happen therefore it's impossible for the original shooting event to have occurred), or the bullet falls out at some point without killing the person. So in the forward perspective you have a child walking along when a bullet suddenly jumps up off the ground and embeds itself in their body. This is just a contrived example; it's not intended as a full explanation. 174