Parsing parts of a Large XML File in App Engine using Blobstore?
It really sounds like App Engine is not the right platform for this project.
I'm working on an google app engine app that will have to deal with some largish ( <100 MB) XML files uploaded from a form that will exceed GAE's limits -- either taking longer than 30 seconds to upload the file, or exceeding the 10 MB request size.
The current solution I'm envisioning is to upload the file to the blobstore, and then bring it into the application (1 MB at a time) for parsing.  This could also very well exceed the 30 second limits for a request, so I'm wondering if there's a nice way to handle large XML documents in chunks, as I may end up having to do it via task queues in 30 second bursts.
I'm currently using BeautifulSoup for other parts of the project, having switched from minidom.  Is there a way to handle data in chunks that would play nice with GAE?