Spark

From Glitchdata
Jump to: navigation, search

Apache Spark™ is a fast and general engine for large-scale data processing. Spark provides improvement of 100x over MapReduce by leveraging in-memory processing.

Apache Spark is an open source cluster computing framework originally developed in the AMPLab at University of California, Berkeley but was later donated to the Apache Software Foundation where it remains today. In contrast to Hadoop's two-stage disk-based MapReduce paradigm, Spark's multi-stage in-memory primitives provides performance up to 100 times faster for certain applications.[1] By allowing user programs to load data into a cluster's memory and query it repeatedly, Spark is well-suited to machine learning algorithms.[2]

Spark is written in Scala


Links

Related
{"cc_re__cc_re__cc_by_4_0": ["rmation about this page\">Page information</a></li><li id=\"t-cite\"><a href=\"/index.php?title=Special:CiteThisPage&amp;page=Spark&amp;id=21613\" title=\"Information on how to cite this page\">Cite this page</a></li>\t\t\t\t\t</ul>\n\t\t\t\t\t\t\t</div>\n\t\t</div>\n\t\t\t\t</div>\n\t\t</div>\n\t\t<div id=\"footer\" role=\"contentinfo\">\n\t\t\t\t\t\t\t<ul id=\"footer-info\">\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-info-lastmod\"> This page was last modified on 9 May 2018, at 23:36.</li>\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-info-copyright\">Content is available under <a class=\"external\" rel=\"nofollow\" href=\"https://creativecommons.org/licenses/by/4.0/\">Creative Commons Attribution</a> unless otherwise noted.</li>\n\t\t\t\t\t\t\t\t\t</ul>\n\t\t\t\t\t\t\t<ul id=\"footer-places\">\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-places-privacy\"><a href=\"/index.php?title=Glitchdata:Privacy_policy\" title=\"Glitchdata:Privacy policy\">Privacy policy</a></li>\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-places-about\"><a href=\"/index.php?title=Glitchdata:About\" title=\"Glitchdata:About\">About Glitchdata</a></li>\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-places-disclaimer\"><a href=\"/index.php?title=Glitchdata:General_disclaimer\" ti", "?title=Glitchdata:Privacy_policy\" title=\"Glitchdata:Privacy policy\">Privacy policy</a></li>\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-places-about\"><a href=\"/index.php?title=Glitchdata:About\" title=\"Glitchdata:About\">About Glitchdata</a></li>\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-places-disclaimer\"><a href=\"/index.php?title=Glitchdata:General_disclaimer\" title=\"Glitchdata:General disclaimer\">Disclaimers</a></li>\n\t\t\t\t\t\t\t\t\t</ul>\n\t\t\t\t\t\t\t\t\t\t<ul id=\"footer-icons\" class=\"noprint\">\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-copyrightico\">\n\t\t\t\t\t\t\t<a href=\"https://creativecommons.org/licenses/by/4.0/\"><img src=\"/resources/assets/licenses/cc-by.png\" alt=\"Creative Commons Attribution\" width=\"88\" height=\"31\"/></a>\t\t\t\t\t\t</li>\n\t\t\t\t\t\t\t\t\t\t\t<li id=\"footer-poweredbyico\">\n\t\t\t\t\t\t\t<a href=\"//www.mediawiki.org/\"><img src=\"/resources/assets/poweredby_mediawiki_88x31.png\" alt=\"Powered by MediaWiki\" srcset=\"/resources/assets/poweredby_mediawiki_132x47.png 1.5x, /resources/assets/poweredby_mediawiki_176x62.png 2x\" width=\"88\" height=\"31\"/></a>\t\t\t\t\t\t</li>\n\t\t\t\t\t\t\t\t\t</ul>\n\t\t\t\t\t\t<div style=\"clear:both\"></div>\n\t\t</"], "copyright__copyright__copyright": ["d in the AMPLab at University of California, Berkeley but was later donated to the Apache Software Foundation where it remains today. In contrast to Hadoop's two-stage disk-based MapReduce paradigm, Spark's multi-stage in-memory primitives provides performance up to 100 times faster for certain applications.[1] By allowing user programs to load data into a cluster's memory and query it repeatedly, Spark is well-suited to machine learning algorithms.[2]\n\nSpark is written in Scala\n\n\nLinks\n\nRelated", "d in the AMPLab at University of California, Berkeley but was later donated to the Apache Software Foundation where it remains today. In contrast to Hadoop's two-stage disk-based MapReduce paradigm, Spark's multi-stage in-memory primitives provides performance up to 100 times faster for certain applications.[1] By allowing user programs to load data into a cluster's memory and query it repeatedly, Spark is well-suited to machine learning algorithms.[2]\n\nSpark is written in Scala\n\n\nLinks\n\nRelated", "d in the AMPLab at University of California, Berkeley but was later donated to the Apache Software Foundation where it remains today. In contrast to Hadoop's two-stage disk-based MapReduce paradigm, Spark's multi-stage in-memory primitives provides performance up to 100 times faster for certain applications.[1] By allowing user programs to load data into a cluster's memory and query it repeatedly, Spark is well-suited to machine learning algorithms.[2]\n\nSpark is written in Scala\n\n\nLinks\n\nRelated"]}