Benzophenone-3 (also referred to as oxybenzone) is a putative endocrine disrupting chemical and common ingredient in sunscreens and other personal care products. We previously showed that benzophenone-3 can have both promotional and protective effects on mammary tumorigenesis dependent upon dietary fat. The current study examined diet-dependent effects of benzophenone-3 in mammary ductal development in BALB/c mice. Long-term benzophenone-3 treatment initiated in puberty and its subsequent withdrawal in adulthood resulted in regression of ductal branching in mice fed a low-fat diet. Regression was associated with increased Igf1 gene expression. BP-3 treatment in mice fed a high-fat diet resulted in increased ductal branching, which was reversed by BP-3 withdrawal. Examination of T cell and macrophage populations within the mammary gland under low-fat and high-fat diets found a shift from Th1/M1 to Th2/M2 polarization, respectively. This alteration in the immune environment may underlie the diet-dependence of benzophenone-3 effects. This study suggests potential consequences of benzophenone-3 exposure to reproductive health that warrant further examination. It also highlights the need to examine endocrine disrupting chemicals in a variety of dietary contexts, which may govern toxicological interactions with the immune system.