title: Is it possible to change Spark parameters at runtime?
body: <p>I am currently working on Spark and trying to suggest an adaptive execution plan. However, I am wondering whether it is possible to modify the parameters of the Spark engine at runtime. For example, Can I use different compression codecs for two separate stages, or can I modify the memory fractions reserved for shuffling and computation at runtime? Say for the map phase, I diminish the memory fraction allocated for shuffling, to increase it later when the shuffling occurs?</p>
  
  <p>Thanks</p>