lmdb can use multiple workers for loading, have you tried that?
See how I use lmdb for my LSUN dataset here: https://github.com/pytorch/vision/blob/master/torchvision/datasets/lsun.py#L19-L20 2.9k
I keep readers at 1, but I use multiple workers to load from lsun (and hence lmdb):

  
      github.com
  
  
    pytorch/examples/blob/master/dcgan/main.py#L83-L84 524

transform=transforms.ToTensor())
assert dataset