I tried the sparse matrix approach today, but I couldn’t get it to work because apparently there is no way to extract a batch from it like so:
weight_batch = sparse_weight_matrix[batch]
which I would need to get one set of weights per group_id in the batch.