Remove unwanted person from background of a shot? I am working on a project for someone's video series about pain relief during child birth. During the whole scene, every shot takes place from this perspective. There's a couple people walking through the stairs on the right and behind the window there. I understand that by use of 'rotoscoping' and 'masks' I should be able to cover up the background with a still image and allow the blonde woman in the foreground to walk over the still images, but I don't know where I'd start that.

I've already made this background layer in Photoshop.

I'd really appreciate your help! :)
If the camera's static, then it's totally doable. Start by making a clean plate image of the whole area, as I think you have done. 

Put that on a layer above (on top of) your original footage. Whenever the problem people wander in to shot, cover them up by putting a masked area of your clean plate over them. 

As you mentioned, sometimes this means the desirable foreground subjects get covered up by the clean plate. But there is a solution to this, (even though it is time consuming):

Duplicate your original footage, and put the duplicated track on top of your clean plate, so it's the uppermost layer. Whenever you have some "desirable foreground" subject overlapping your clean plate, animate a masked area of it on top of the plate. Use lots of feathering and careful key framing, and you should be able to get an acceptable result.  

Here's a (non-video) tutorial showing how to remove an unwanted object:
http://www.surfacedstudio.com/blog/after-effects-how-to-remove-an-object

Here's the excellent video copilot series, showing how to animate a mask:
http://www.videocopilot.net/basic/tutorials/04.Keying/

Neither show the idea of putting another foreground track over the top, but should get you along the way...