Question: Firefox has the Sandbox and evalInSandbox(). Chrome has sandboxed execution in their content scripts (they call it isolated execution). I'm looking for the same thing in an IE browser extension.

I can load a javascript file, then call evalScript(), but the code executes in the same environment as javascript that exists on the page. I need a way to run my library (which includes and is based on jQuery) in an sandboxed/isolated environment, but still allow it to modify the DOM as if it were running on the page.

Jint looks promising, but cannot currently evaluate jQuery. (They can parse it.)

How can I do this?

Answer: 
Looks like you are in a pickle with this one.

* Some Microsoft people say that IE doesn't offer this functionality
* Some who say they have been able to pull this off hold it close and protect it as their secret sauce
* "Professional" IE extension shops, when contacted about building what you are talking about turn down the job

I really wish I had better news but it looks like it might take a small miracle to accomplish what you are looking for... or maybe a lot of money. : )

Your best bet is probably going to be finding one of the few who claim they have been able to do it and pay them a lot to share the secret or reconsider why you really want what you want and see if you can't accomplish it another way.
Thing you are looking for is ActiveScript engine (<http://en.wikipedia.org/wiki/Active_Scripting>)

I can`t find any usefull links now, MSDN contains only interfaces definition (<http://msdn.microsoft.com/en-us/library/ccd0zt2w(v=vs.85).aspx>)
Try to Google "Active Scripting" (not "Action scripting"!)

You need to implement interface IActiveScriptHost in your extension, create an object of «JScript», call SetSite and pass your host object, then you can load your js code into this engine and run it.

This technique is a quite difficult due to lack of documentation about it.
If you are still interested in this, I can send you some examples in C++/ATL.