Deep Learning framework MXNet joins Apache Software Incubator
Deep learning framework MXNet, which has been chosen by Amazon as its deep learning framework, has been accepted into the Apache Software Foundation (ASF) Incubator. MXNet is focused on complex applications such as speech recognition and computer vision. MXNet also offers support to a variety of programming languages and is scalable across CPUs and GPUs. Amazon is likely to develop the framework to a commercial scale and may use MXNet's expertise to develop deep learning techniques for languages other than Python. ASF Incubator already sponsors Apache Mahout to provide scalable machine learning resources on Hadoop clusters, while ASF-supported Apache Spark has machine learning functionality.

MXNet, chosen by Amazon as its "deep learning framework of choice," has been accepted into the Apache Software Foundation (ASF) Incubator to support its future development.

This is not the only machine learning project with ASF sponsorship. Apache Mahout provides scalable machine learning resources on Hadoop clusters; Apache Spark has machine learning functionality as well. However, MXNet is specifically aimed at deep learning, which is used in complex applications like speech recognition or computer vision.

Amazon singled out MXNet, one of InfoWorld's Open Source Rookies of the Year for 2016, as compact, with broad support available in a variety of languages, and highly scalable across CPUs and GPUs.

As part of the ASF, MXNet is under more general governance and spurs both contributions and uptake from a beyond Amazon. In practice, some ASF projects continue to be driven mainly by one commercial outfit -- for example, DataStax and Apache Cassandra, or Databricks and Apache Spark.

That said, the ASF will scrutinize the project management committees for projects where a single entity provides the bulk of contributions. Projects can't leave the incubation stage unless they can show a high degree of independence from any one contributor (a high bus factor), company, or entity.

In that respect, MXNet has a good chance of moving rapidly through the incubation stage. Current key contributors to MXNet hail from a variety of organizations: Microsoft, Stanford University, Carnegie Mellon University, the University of Washington, and New York University, among others.

How might Amazon's contributions influence the project? The cloud giant probably will do what most every other cloud giant has done when dealing with open source: Make it more amenable to running at scale and offered as a service on its servers. Most big-name backing for major open source projects -- the aforementioned Spark, Google's Kubernetes, Microsoft's contributions to the Linux kernel -- has been in that vein.

Amazon could also improve MXNet's documentation and offer more examples for languages other than Python. In his MXNet review for InfoWorld, Martin Heller flagged both areas as needing work. Amazon has offered few details about what it'll contribute, other than to say there is "a significant team at Amazon working with the MXNet community to continue to evolve it."