In arm A, pembrolizumab will be started and SBRT will be given to the 1-3 most clinically significant lesions only at the time of progression on pembrolizumab. Pembrolizumab will be continued after progression to detect if the addition of SBRT leads to stability or response in progressing lesions. After SBRT, untreated measurable disease must remain to assess the activity of continued pembrolizumab. Pembrolizumab and SBRT will continue until progression as per immune related response criteria (irRC). This cohort of patients will allow us to more clearly understand if SBRT does play a part in improving progression free survival when given in combination with pembrolizumab.