Currently I'm in the process of tagging every S3 bucket I have using boto3. Compared to a resource like Lambdas, doing s3.put_bucket_tagging overwrites any previous tags, compared to Lambdas which only add extra tags while keeping old ones. Is there a way to only add tags, rather than overwrite them?
Secondly, I have created a method to take the current tags, add the new tags on, and then overwrite the tags with those values, so I don't lose any tags. But some of these S3 buckets are created by CloudFormation and thus are prefixed with aws: which gives me the error Your TagKey cannot be prefixed with aws: when I try to take the old tags and re-put them with the new tags. 
A fix for either of these to give me the ability to automate tagging of every s3 bucket would be the best solution.
You are out of luck. If the S3 bucket was created by a CFT, then

You cannot add new tags or 
Add new tags and lose the tags created by CFT (then your delete stack will fail unless you exclude that S3 resource from deletion)

You can try updating the stack with new tags as suggested by @jarmod
Using boto3 to add extra tags to S3 buckets