I'm writing code (Python 3) that collects data on a small IOT device and uploads it to a database on AWS. Currently the data is sent by parsing it into a json string and sending it via post request. There can be quite a lot of data at times, and I'm wondering if I can send it in compressed form using a post request.
What I don't want to do is take the data, compress it to a file, then read that file's raw data into a string, and place that string in the JSON. It would be a waste to save a file and immediately read from it.
Is there a way to compress data directly into a string of raw data, and send the compressed string as opposed to compressing into a file and reading it?
I need a lossless compression format, hopefully something that's not too resource intensive to compress/decompress. A .npy compression would be especially nice.
Your not going to POST anything if  your device is connecting via MQTT to AWS IoT core I assume?  That's normally a lighter weight setup then HTTP and MQTT is preferred in real IoT dev.  The best way to handle these things unless you want to program the compression algorithm on the device, is to send your data through AWS IoT Core and connect a Lambda action to that incoming message.  Then program the Lambda to do any file manipulation or compression before dispatching the info to DynomoDB or S3 directly from Lambda.
Compressing data to string for json post without saving files