The impacts of air pollution, local climate, and urbanization on human health have been well-documented in recent studies. In this study, we combined magnetic resonance imaging (MRI) brain analysis with a questionnaire survey on the local environment in 141 healthy middle-aged men and women. Our findings reveal that a favorable environment is positively correlated with gray matter volume (GMV) in the frontal and occipital lobes, cerebellum, and whole brain, as well as with fractional anisotropy (FA) in the fornix (including the fornix stria terminalis), posterior thalamic radiation (PTR), sagittal stratum (SS), and whole brain. Among these, significant correlations between the local environment and whole-brain and cerebellar GMV, PTR, and SS FA remained after Bonferroni correction. Additionally, the positive relationship between the local environment and whole-brain GMV was further supported by principal component analysis (PCA). This is the first study to demonstrate that healthy adult brain structure, as indicated by GMV and FA values, can be influenced by the local environment.