Title: Modular&#x2F;Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Modular_neural_network<p>In principle, the hierarchical organization could allow us to make sense of more complex problem spaces and reach a higher functionality, but it seems difficult to find examples of concrete research done in the past regarding this. I&#x27;ve found a few sources:<p>https:&#x2F;&#x2F;www.teco.edu&#x2F;~albrecht&#x2F;neuro&#x2F;html&#x2F;node32.html<p>https:&#x2F;&#x2F;vtechworks.lib.vt.edu&#x2F;bitstream&#x2F;handle&#x2F;10919&#x2F;27998&#x2F;etd.pdf?sequence=1&amp;isAllowed=y<p>A few concrete questions I have:<p>Are there any tasks where MNNs have shown better performance than large single nets?<p>Could MNNs be used for multimodal classification, i.e. train each net on a fundamentally different type of data, (text vs image) and feed forward to a higher level intermediary that operates on all the outputs?<p>From a software engineering perspective, aren&#x27;t these more fault tolerant and easily isolatable on a distributed system?<p>Has there been any work into dynamically adapting the topologies of subnetworks using a process like Neural Architecture Search?<p>Generally, are MNNs practical in any way?<p>Apologies if these questions seem naive, I&#x27;ve just come into ML and more broadly CS from a biology&#x2F;neuroscience background and am captivated by the potential interplay.<p>I really appreciate you taking the time and lending your insight!
Upvote: