Two radiation therapy techniques are commonly used for the treatment of intermediate and high risk prostate cancer: brachytherapy and external beam radiation therapy (EBRT). However, both have limitations. Brachytherapy, in which radioactive seeds are inserted into the prostate, produces excellent outcomes but is invasive and not all patients are eligible or willing to receive this treatment. EBRT, while gentle at the time of delivery, tends to be very prolonged and may have poorer outcomes than brachytherapy. This study will examine the use of stereotactic ablative radiotherapy (SABR), in which patients are given only a few, high dose radiation treatments. Treatments are short, non-invasive, applicable to patients not able to do brachytherapy, and may be more effective than conventional EBRT. This study will compare SABR with EBRT in terms of the rates of acute and late toxicities for each treatment, disease-free survival, and health-related quality of life measures.