And what case scenarios (i.e. not scalars) can I not use ignite if I’m using distributed training?

There are some abstractions that are not working yet in DDP: EpochMetric, LRFinder, all contrib regression metrics that use EpochMetric.

so is there a difference between EpochMetric and custom metrics with respect to distributed training?

Yes, in the way of how to collect data across processes. By default we collect internal metric’s data via all reduce operation (sum across processes). In case of non-scalar mertic’s data, it is not defined how to perform a sum of those objects and certainly it wont give a correct result (think about computing median for example).

For example, I’m assuming the following won’t work?

No, it wont work, as you need to collect _list_of_ETC and _list_of_errors accross processes, so for example total _list_of_errors should be a concat of _list_of_errors by rank, right ?
You have to use all gather op to do that
import torch.distributed as dist

class Correlation(Metric):

   def compute(self):
       # roughly something like that
       list_of_ETC = torch.tensor(_list_of_ETC).to(this_current_device)
       dist.all_gather(_list_of_ETC) 
       self._list_of_ETC = _list_of_ETC.tolist() 
       # etc

PS: btw if you would like to help us with this issue : https://github.com/pytorch/ignite/issues/978
such that everyone could benefit of using EpochMetric in DDP…
If so, you can submit a draft PR and we could guide you on how to better implement and test it…