Is it possible to upgrade Mail without upgrading the whole operating system? I have Apple Mail version 9.2, but I would like to have the version 10.3. However, I don't want to download the 10.12 mac OS (Sierra) if I don't have too. I currently have mac OS 10.11.3 (El Capitan).

There is a discussion here involving a much older OS (tiger) in 2010:
https://discussions.apple.com/thread/2502383?tstart=0

I can't find many other resources discussing upgrading mail.
Sorry but my guess would be that this is very unlikely if not impossible.

Mail isn't a single self-contained piece of code like a third-party application. Big chunks of the functionality that make up Mail as stored in frameworks that reside in places like /System/Library/Frameworks for example Message.framework.

So as well as the binary that sits in the /Applications folder you'd also need to upgrade any relevant frameworks from the /System/Library/Frameworks folder, and you wouldn't be able to just swap them out because that would break other functionality in the operating system that expected the native El Capitan versions.

There is likely a way to do it, but it's such a ridiculously complex task I wouldn't recommend pursuing it.

Sorry.