title: How to suppress Spark logging in unit tests?
body: <p>So thanks to easily googleable blogs I tried:</p>
  
  <pre><code>import org.specs2.mutable.Specification
  
  class SparkEngineSpecs extends Specification {
    sequential
  
    def setLogLevels(level: Level, loggers: Seq[String]): Map[String, Level] = loggers.map(loggerName =&gt; {
      val logger = Logger.getLogger(loggerName)
      val prevLevel = logger.getLevel
      logger.setLevel(level)
      loggerName -&gt; prevLevel
    }).toMap
  
    setLogLevels(Level.WARN, Seq("spark", "org.eclipse.jetty", "akka"))
  
    val sc = new SparkContext(new SparkConf().setMaster("local").setAppName("Test Spark Engine"))
  
    // ... my unit tests
  </code></pre>
  
  <p>But unfortunately it doesn't work, I still get a lot of spark output, e.g.:</p>
  
  <pre><code>14/12/02 12:01:56 INFO MemoryStore: Block broadcast_4 of size 4184 dropped from memory (free 583461216)
  14/12/02 12:01:56 INFO ContextCleaner: Cleaned broadcast 4
  14/12/02 12:01:56 INFO ContextCleaner: Cleaned shuffle 4
  14/12/02 12:01:56 INFO ShuffleBlockManager: Deleted all files for shuffle 4
  </code></pre>