Do I ever have to decrypt S3-encrypted files?
The "server-side" encryption you have enabled turns on encryption at rest. Which means the file is encrypted while it's sitting on S3. But S3 will decrypt the file before it sends you the data when you download the file.
So there is no change to how you handle the file when downloading it if the file is encrypted or not.
This type of encryption does not protect the file if the file is downloaded via valid means, such as when using the API. It only protects the file from reading if someone were to circumvent the S3 data center or something like that. 
If you need to protect the file, such that it must be decrypted when downloaded, then you need to encrypt it client-side, before uploading it to S3. 
You can use any client-side encryption scheme you deem worthy: AES256, etc. But S3 won't do it for you.
I'm using S3 instead of KMS to store essentially a credentials file, and Python to read the file's contents.
I manually set the file encrypted by clicking on it in S3, going to Properties - Details - Server Side Encryption:AES-256
And in my Python script, I read the key without making changes from when I read the file when it was unencrypted. And I was also able to download the file and open it without having to do anything like decrypting it. I was expecting to have to decrypt it, so I'm a little confused.
I'm just unable to understand what server-side encryption protects against. Would anyone already with access to S3 or the S3 bucket with the key/file be able to read the file? Who wouldn't be able to open the file?