Hi Alban,
Thanks for clarifying.  Does this mean that this parallelization utilizes locking, ensuring that each GPU model updates its weights from the “main” model before moving on to the next mini-batch?