A phyllodes tumor is a rare fibroepithelial breast tumor that can be benign, borderline, or malignant. Its diagnosis is challenging because it shares characteristics with other lesions. The treatment is primarily surgical, with no established role for chemotherapy and limited evidence supporting radiotherapy. There is no evidence of phyllodes tumors expressing endocrine receptors, nor is there an established role for endocrine therapy in treating these tumors. We present a case of a large phyllodes tumor that showed a clinical response to treatment with anastrozole.