I am using the pyglet (OpenGL) library and I want to be able to change the virtual resolution without changing the size of the window. For example a 2x2 box would be drawn as 4x4 pixels on the screen. I know I can find everything that is being draw and scale it individually, but this would probably be costly.
I could not find a solution for this online (if it even exists), so any help would be greatly appreciated.
Clarification: I am thinking along the lines of the resolution settings in most games. The window stays in full screen at the same size, but the scale changes.
Clarification: I am thinking along the lines of the resolution settings in most games. The window stays in full screen at the same size, but the scale changes.

No, the window does change the size, because the screen resolution is changed, and the window follows.
However what you intend to do it perfectly possible: First render your image to a FBO of the desired smaller size, then render the contents of that FBO covering the full window. In the case a 3D engine uses some form of post processing (like for depth of field, color grading, compositing effects), this comes virtually for free.
Scale Resolution in Pyglet