title: RRD library for handling time series data in a Python application
body: <p>I am working on a simulation engine using Python where I collect a lot of metrics. The simulation runs at a high speed and generates around 100K events/second (I can do some processing by consolidating these events on a per second basis). I am looking for a mechanism to record these metrics as a time series.</p>
  
  <p>My requirements are:</p>
  
  <ol>
  <li><p>I would like to have this logging mechanism in the same process
  as the simulation as opposed to an external process such as Graphite</p></li>
  <li><p>The mechanism must be able to handle 100K events/second without slowing down the simulation.</p></li>
  <li><p>I would like to store data as follows: Every metric related data should be stored with 1 second granularity for 60 minutes, 1 minute granularity for 1 day, 5 minute granularity for two days, 1 hour granularity for 6 months and 1 day granularity for 3 years of duration. I would like this mechanism to handle the consolidation of data as per the ranges specified.</p></li>
  <li><p>Ideally, I want to maintain one file that holds the metrics information for one simulation run. For another run of the simulation a separate file would have to be created.</p></li>
  <li><p>It would be nice to have a well-tested library/module that is readily available :)</p></li>
  </ol>
  
  <p>BTW, I took a cursory look at RRDTool but from what I understand it seems like the Python library is a thin wrapper around the RRDTool binary. I'm looking for a tighter integration if possible.</p>
  
  <p>TIA</p>