It&#x27;s more than that, and it&#x27;s in use in production at Amazon.  8 TitanX GPUs can contain networks with up to 6 billion weights.  As Geoffrey Hinton once said:<p>&quot;My belief is that we’re not going to get human-level abilities until we have systems that have the same number of parameters in them as the brain.&quot;<p>And you&#x27;re right that it&#x27;s a specialized framework&#x2F;engine.  But IMO making it more general purpose is a matter of cutting and pasting the right cuDNN code or we can double down on emphasizing sparse data.  Amazon OSSed this partially IMO to see what people would want here.