title: Databricks job is canceled when azure-cosmos-spark maven library is installed
body: <p>I've used com.azure.cosmos.spark:azure-cosmos-spark_3-1_2-12:4.0.0 installed on cluster with runtime 8.3.x-scala2.12 for a long time. But it suddenly stopped working and databricks jobs that are run on cluster with this library are canceled.
  <a href="https://i.stack.imgur.com/RKhMu.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/RKhMu.png" alt="canceled databricks job" /></a></p>
  <p>Cluster driver logs stderr file contains following error:
  ANTLR Tool version 4.7 used for code generation does not match the current runtime version 4.8</p>
  <p>I've tried to update library and cluster runtime versions and also installed jar library instead of maven, but it didn't help.</p>
  <p>Now my cluster has following configuration:</p>
  <pre><code>{
  &quot;autoscale&quot;: {
      &quot;min_workers&quot;: 1,
      &quot;max_workers&quot;: 2
  },
  &quot;cluster_name&quot;: &quot;test-clstr002&quot;,
  &quot;spark_version&quot;: &quot;9.1.x-scala2.12&quot;,
  &quot;spark_conf&quot;: {
      &quot;spark.databricks.delta.preview.enabled&quot;: &quot;true&quot;
  },
  &quot;azure_attributes&quot;: {
      &quot;first_on_demand&quot;: 1,
      &quot;availability&quot;: &quot;ON_DEMAND_AZURE&quot;,
      &quot;spot_bid_max_price&quot;: -1
  },
  &quot;node_type_id&quot;: &quot;Standard_F4s&quot;,
  &quot;driver_node_type_id&quot;: &quot;Standard_F4s&quot;,
  &quot;ssh_public_keys&quot;: [],
  &quot;custom_tags&quot;: {},
  &quot;spark_env_vars&quot;: {},
  &quot;autotermination_minutes&quot;: 60,
  &quot;enable_elastic_disk&quot;: true,
  &quot;cluster_source&quot;: &quot;API&quot;,
  &quot;init_scripts&quot;: [],
  }
  </code></pre>
  <p>There is a screenshot of installed azure-cosmos-spark maven library
  <a href="https://i.stack.imgur.com/L640T.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/L640T.png" alt="azure-cosmos-spark maven library" /></a></p>
  <p>Thank you for any help or suggestions!</p>