Google App Engine Server to Server OAuth Python
A service account isn't you its it's own user.  Even if you could access Gmail with a service account which I doubt you would only be accessing the service accounts GMail account (Which I don't think it has) and not your own.
To my knowledge the only way to access Gmail API is with Oauth2. 
Service accounts can be used to access some of the Google APIs for example Google drive.  The service account his its own Google drive account files will be uploaded to its drive account.  I can give it permission to upload to my google drive account by adding it as a user on a folder in Google drive.
You cant give another user permission to read your Gmail Account so again the only way to access the Gmail API will be to use Oauth2.
I can't find a solution to authorize server-to-server authentication using Google SDK + Python + MAC OSx + GMAIL API.
I would like testing GMail API integration in my local environment, before publishing my application in GAE, but until now I have no results using samples that I have found in GMail API or OAuth API documentation. During all tests I received the same error "403-Insufficient Permission" when my application was using GCP Service Account, but if I convert the application to use User Account everything was fine.