I have a Python script which produces some some data. I would like to stream it to an HTTP server using POST. That is, I don't want to accumulate the data in a buffer and then send it -- I just want to send it as it's created. There will be a lot.
The apparently obvious way to do this would be to open the HTTP connection in some way that return a writeable file object, write the data to that object, and then close the connection.
However, it's not obvious to me that this is supported in any of the libraries I looked at (urllib2, httplib, and requests).
How can I accomplish this?
You wont get a writeable file object via urllib2 / urllib. The return value is a "file like" object which supports iteration and reading. If you can read the contents and create your own file object for writing.
How can I get a writeable file object for an HTTP POST in Python?