title: Set spark configuration in aws glue pyspark
body: <p>I am using AWS Glue with pySpark and want to add a couple of configurations in the sparkSession, e.g. <code>'&quot;spark.hadoop.fs.s3a.impl&quot;, &quot;org.apache.hadoop.fs.s3a.S3AFileSystem&quot;</code>,<code>spark.hadoop.fs.s3a.multiobjectdelete.enable&quot;,&quot;false&quot;</code>, <code>&quot;spark.serializer&quot;, &quot;org.apache.spark.serializer.KryoSerializer&quot;</code>, <code>&quot;spark.hadoop.fs.s3a.fast.upload&quot;,&quot;true&quot;</code>. The code I am using to initialise the context is the following:</p>
  <pre><code>glueContext = GlueContext(SparkContext.getOrCreate())
  spark = glueContext.spark_session
  </code></pre>
  <p>From what I understood from the documentation is that I should add these confs as job parameters when submitting the glue jobs. Is that the case or can they also be added when initializing the spark?</p>