&gt; Do they have automagic model parallelization (not data parallelization) as well?<p>Not that I know of.<p>Regarding claims of novelty, I don&#x27;t think the Caffe maintainers are claiming that their multi-gpu update method is novel or even very good.  I think it was just the easiest thing someone could think of. I think Flickr originally wrote the multi-gpu extensions and the maintainers simply accepted the pull request.<p>If anything, I think the maintainers are more than willing to listen to people in the scientific computing community with experience.  Even better if they have a pull request in hand.  But otherwise, they probably won&#x27;t know about better methods and won&#x27;t care.