I am using the python package mailbox, and I am trying to extract the messages and clean the data. I am running into the problem that for large databases, I can call the constructor with my sample file, but when I try to print any messages my program hangs. I assume it is because the file I am trying to read is over 7GB. How can I deal with this problem?
Consider splitting the mailbox manually. The format is fairly easy to process (as long as you only need read-only access) by reading it line-per-line; and you can use the existing classes for the actual parsing of individual messages.
Look up the definition of the mbox format - lines beginning with "From" start a new mail. You can split the huge file at these markers, then use the mailbox package to read only one file at a time.
Python mailbox on large mbox datasets