Hi Cherubim!



 CherubimHD:

I have tried putting all the heads in a ModuleList and then iterating over the ModuleList, each time appending the new result to some existing tensor. However, this is really slow.


I saw a suggestion in an answer to a related question:  Consider adding
each result to a regular python list, and then run torch.cat() on that
list.  This way you don’t incur the expense of a tensor-append for every
loop iteration.
Best.
K. Frank