text: Title: Install latest spark version automatically
  Body: Most of the time we do not pin the version of the software we install.
  We should not have this hardcode for spark as well.
  
  This is where hardcode happens: https://github.com/jupyter/docker-stacks/blob/main/images/pyspark-notebook/Dockerfile
  This is how I got rid of the hardcode for Julia: https://github.com/jupyter/docker-stacks/blob/main/images/minimal-notebook/setup-scripts/setup_julia.py
  
  We will also be able to close https://github.com/jupyter/docker-stacks/issues/1937
  I mean, the issue won't be fixed in upstream, but we will no longer be responsible for updating to a new version when it is released - I think it would make sense to close such issues because this project doesn't fix/patch upstream issues most of the time.