Purpose To assess outcomes with external beam radiation therapy (EBRT) and a low-dose-rate (LDR) brachytherapy boost without or with androgen deprivation therapy (ADT) for prostate cancer. Materials and Methods From January 2001 through August 2011, 120 intermediate-risk or high-risk prostate cancer patients were treated with EBRT to a total dose of 4,500 cGy in 25 daily fractions and a palladium-103 LDR brachytherapy boost of 10,000 cGy (n = 90) or an iodine-125 LDR brachytherapy boost of 11,000 cGy (n = 30). ADT, consisting of a gonadotropin-releasing hormone agonist ± an anti-androgen, was administered to 29/92 (32%) intermediate-risk patients for a median duration of 4 months and 26/28 (93%) high-risk patients for a median duration of 28 months. Results Median follow-up was 5.2 years (range, 1.1-12.8 years). There was no statistically-significant difference in biochemical disease-free survival (bDFS), distant metastasis-free survival (DMFS), or overall survival (OS) without or with ADT. Also, there was no statistically-significant difference in bDFS, DMFS, or OS with a palladium-103 vs. an iodine-125 LDR brachytherapy boost. Conclusions There was no statistically-significant difference in outcomes with the addition of ADT, though the power of the current study was limited. The Radiation Therapy Oncology Group 0815 and 0924 phase III trials, which have accrual targets of more than 1,500 men, will help to clarify the role ADT in locally-advanced prostate cancer patients treated with EBRT and a brachytherapy boost. Palladium-103 and iodine-125 provide similar bDFS, DMFS, and OS.