I want to write a program that ssh's into remote boxes and runs jobs there if the remote computer is not actively being used. I'll be logging in as clusterJobRunner@remoteBox, and the other user will be logged in as someLocalUser@remoteBox. 
Is there a way to see if a remote user is actively using the box using either Python or Java?
I second the answer by @Eero Aaltonen -- you should run your stuff under nice.  A Linux computer can run at 100% CPU busy, yet feel nice and fast for the user, if the extra tasks are all under nice; the scheduler will only run the nice tasks when the main user's tasks are idle.
But if you want to figure out if the machine is being used, I suggest you look into the w command.  Try man w at your prompt.  The w command prints the load average for the machine, and a list of users and how much time they have been using (a combined time that includes any background tasks they are running, plus a time for their main task).
How can I find out if someone is actively using a Linux computer in Python or Java?