Extensive studies have identified potential adverse effects on semen quality of obesity, based on body mass index, but the association between body fat distribution, a more relevant indicator for obesity, and semen quality remains less clear. We conducted a longitudinal study of 4304 sperm donors from the Guangdong Provincial Human Sperm Bank (Guangzhou, China) during 2017-2021. A body composition analyzer was used to measure total and local body fat percentage for each participant. Generalized estimating equations were employed to assess the association between body fat percentage and sperm count, motility, and morphology. We estimated that each 10% increase in total body fat percentage (estimated change [95% confidence interval, 95% CI]) was significantly associated with a 0.18 × 106 (0.09 × 106-0.27 × 106) ml and 12.21 × 106 (4.52 × 106-19.91 × 106) reduction in semen volume and total sperm count, respectively. Categorical analyses and exposure-response curves showed that the association of body fat distribution with semen volume and total sperm count was stronger at higher body fat percentages. In addition, the association still held among normal weight and overweight participants. We observed similar associations for upper limb, trunk, and lower limb body fact distributions. In conclusion, we found that a higher body fat distribution was significantly associated with lower semen quality (especially semen volume) even in men with a normal weight. These findings provide useful clues in exploring body fat as a risk factor for semen quality decline and add to evidence for improving semen quality for those who are expected to conceive.