I have a potentially big list of image sequences from nuke. The format of the string can be:
/path/to/single_file.ext
/path/to/img_seq.###[.suffix].ext
/path/to/img_seq.%0id[.suffix].ext, i being an integer value, the values between [] being optional.
The question is: given this string, that can represent a sequence or a still image, check if at least one image on disk corresponds to that string in the fastest way possible.
There is already some code that checks if these files exist, but it's quite slow.
First it checks if the folder exists, if not, returns False
Then it checks if the file exists with os.path.isfile, if it does, it returns True.
Then it checks if no % or # is found in the path, and if not os.path.isfile, it returns False.
All this is quite fast.
But then, it uses some internal library which is in performance a bit faster than pyseq to try to find an image sequence, and does a bit more operations depending if start_frame=end_frame or not.
But it stills take a large amount of time to analyze if something is an image sequence, specially on some sections of the network and for big image sequences.
For example, for a 2500 images sequence, the analysis takes between 1 and 3 seconds.
If I take a very naive approach, and just checks if a frame exist by replacing #### by %04d, and loop over 10000 and break if found, it takes less than .02 seconds to check for os.path.isfile(f), specially if the first frame is between 1-3000.
Of course I cannot guarantee what the start frame will be, and that approach is not perfect, but in practice many of the sequences do begin between 1-3000, and I could return True if found and fallback to the sequence approach if nothing is found (it would still be quicker for most of the cases)
I'm not sure what's the best approach is for this, I already made it multithreaded when searching for many image sequences, so it's faster than before, but I'm sure there is room for improvement.
If there are potentially so many files that you're worried about wasting memory for a dictionary that holds them all, you could just store a single key for each img_seq.###[.suffix].ext pattern, removing the sequence number as you scan the directory. Then a single lookup will suffice. The values in the dictionary could either be "dummy" booleans because the existence of the key is the only thing you care about, or counters in case you ever want to know how many files you have for a certain sequence.
Fastest way to check if an image sequence string actually exists on disk