I have three scripts:
config.yaml and script.py and slurm.sh
I am submitting job to the job scheduler in Slurm using the slurm.sh file which calls the script.py file and the script.py files loads the config from the config.yaml file.
It would look like this:
sbatch slurm.sh
Now, I change the config.yaml and submit another job. However, if both jobs are in the queue, the config.yaml file will be overwritten and both jobs will use the same configuration. 
Is there any way to tell the slurm scheduler to cache the config.yaml file as well? I know it is dont for the slurm.sh file.
There is no way to achieve that. You should modify your script (.sh) to point to the proper configuration file before submission, and create as many configuration files as needed.
Another option could be to convert the configuration file entries to command line parameters (if possible) before submission and submit a script with a fully configured command line (so no need to read the .yaml file).
How to tell slurm to cache config file for multiple submissions