processing large cloud storage files in app engine
The data file doesnt "need to be in memory" and if you try that you will run oom.
If you can process it sequentially open it as a filestream. Ive done that with blobstore, should be similar
I'm required to process large files, up to 2GB, in GAE (I'm using Python).
of course I'll be running the code on a backend, however since a local storage isn't available the data will need to be in memory.
is there a file descriptor like wrapper for boto or other cloud storage supported protocol?
or other recommended technique?
Thanks,
Shay