I'm looking to write a script in Python 2.x that will scan physical drive (physical and not logical) for specific strings of text that will range in size (chat artifacts). I have my headers and footers for the strings and so I am just wondering how is best to scan through the drive? My concern is that if I split it into, say 250MB chunks and read this data into RAM before parsing through it for the header and footer, it may be the header is there but the footer is in the next chunk of 250MB.
So in essence, I want to scan PhysicalDevice0 for strings starting with "ABC" for example and ending in "XYZ" and copy all content from within. I'm unsure whether to scan the data as ascii or Hex too.
As drives get bigger, I'm looking to do this in the quickest manner possible.
Any suggestions?
Your problem can be formulated as "how do I search in a very long file with no line structure." It's no different from what you'd do if you were reading line-oriented text one line at a time: Imagine you're reading a text file block by block, but have a line-oriented regexp to search with; you'd search up to the last complete line in the block you've read, then hold on to the final incomplete line and read another block to extend it with. So, you don't start afresh with each new block read. Think of it as a sliding window; you only advance it to discard the parts you were able to search completely.
Do the same here: write your code so that the strings you're matching never hit the edge of the buffer. E.g., if the header you're searching for is 100 bytes long: read a block of text; check if the complete pattern appears in the block; advance your reading window to 100 bytes before the end of the current block, and add a new block's worth of text after it. Now you can search for the header without the risk of missing it. Once you find it, you're extracting text until you see the stop pattern (the footer). It doesn't matter if it's in the same block or five blocks later: your code should know that it's in extracting mode until the stop pattern is seen.
Extracting strings from a physical drive