So what happens is that the default collate function will get a list batch of one list per batch item and the use zip(*batch) to get a list of batches. This is clearly not what you want, so you need to implement your own collate function and pass it to the dataloader.
You can use the default_collate function 1 as inspiration for how to write one and handle the dicts.
Best regards
Thomas