Are you applying a lot of preprocessing?
If you are using PIL, you might use Pillow-SIMD, which is a drop-in replacement for Pillow with SIMD.
Could you post the __getitem__ method so that we can have a look at potentially speedups?