Does anyone know of a memory efficient way to generate very large xml files (e.g. 100-500 MiB) in Python? 
I've been utilizing lxml, but memory usage is through the roof.
Obviously, you've got to avoid having to build the entire tree ( whether DOM or etree or whatever ) in memory. But the best way depends on the source of your data and how complicated and interlinked the structure of your output is.
If it's big because it's got thousands of instances of fairly independent items, then you can generate the outer wrapper, and then build trees for each item and then serialize each fragment to the output. 
If the fragments aren't so independent, then you'll need to do some extra bookkeeping -- like maybe manage a database of generated ids & idrefs. 
I would break it into 2 or 3 parts: a sax event producer, an output serializer
eating sax events, and optionally, if it seems easier to work with some independent pieces as objects or trees, something to build those objects and then turn them into sax events for the serializer. 
Maybe you could just manage it all as direct text output, instead of dealing with sax events: that depends on how complicated it is. 
This may also be a good place to use python generators as a way of streaming the output without having to build large structures in memory.
Generating very large XML files in Python?