Middle-aged adults who are parents have better average cognitive performance and lower average brain age compared with middle-aged adults without children, raising the possibility that caregiving slows brain aging. Here, we investigate this hypothesis in two additional groups of caregivers: grandmothers and caregivers for people living with dementia (PLWD). Demographic, questionnaire, and structural Magnetic Resonance Imaging (MRI) data were acquired from n = 50 grandmothers, n = 24 caregivers of PLWD, and n = 37 non-caregiver controls, and BrainAGE was estimated. BrainAGE estimation results suggest that after controlling for relevant covariates, grandmothers had a brain age that was 5.5 years younger than non-grandmother controls, and caregivers of PLWD had brains that were 4.7 years younger than non-caregiver controls. Women who became grandmothers at a later age had lower brain age than those who became grandmothers at an earlier age. Among caregivers of PLWD, stress and caregiving burden were associated with increased brain age, such that the beneficial effect of caregiving on brain age was reduced in caregivers reporting more burden. Our findings suggest that caring for dependents may slow brain aging.