I wouldn’t consider myself a TF build expert and there may be a better way to do this. The GRPC dependency is defined here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/workspace2.bzl#L622-L641 10
You can clone the repository and replace the URLs with whatever specific release you want. Make sure you also replace the sha256 with the matching one as well.
Once you change the version, you can follow the directions to build TF from source. No guarantees that changing the GRPC version will work correctly when building TF master.