INITIAL TREATMENT PHASE: Patients who are progressing on enzalutamide will receive pembrolizumab intravenously (IV) over 30 minutes on day 1. Treatment repeats every 3 weeks for 4 courses in the absence of disease progression or unacceptable toxicity. Patients continue to receive standard of care enzalutamide orally (PO) daily.