I am trying to build a proxy to buffer some packets according to some schedules.
I have two TCP connections, one from host A to the proxy and the other one from the proxy to host B.
The proxy forwards the packets between A and B. The proxy will buffer the packets according to the scheduled instructions. At certain time, it will buffer the packets. After the buffering period is over, it will forward the packets in the buffer and also do its normal forwarding work.
I am using python. Which module would be the best in this situation? I tried pickle but it is difficult to remove and append elements in the file. Any suggestions? Thanks!
I recommend you join the two scripts into one and just use memory.
If you can't join the scripts for some reason, create a unix-domain socket to pass the raw, binary data directly from one to the other.  These fifos have limited size so you'll still have to do in-memory buffering on one side or another, probably the B side.
If the data is too big for memory, you can write it out to a temporary file and re-read it when it's time to pass it on.  It'll be easiest if the same script both writes and reads the file as then you won't have to guess when to write, advanced to a new file, or deal with separate readers and writers.
Build a buffer in file for stream of TCP packets