Fat distribution and macro structure of white adipose tissue are important factors in predicting obesity-associated diseases, but cellular microstructure of white adipose tissue has been less explored. To investigate the relationship between adipocyte size and obesity-related traits, and their underlying disease-driving genetic associations, we performed the largest study of automatic adipocyte phenotyping linking histological measurements and genetics to date. We introduce deep learning based methods for scalable and accurate semantic segmentation of subcutaneous and visceral adipose tissue histology samples (N=2,667) across 5 independent cohorts, including data from 9,000 whole slide images, with over 27 million adipocytes. Estimates of mean size of adipocytes were validated against Glastonbury et al. 2020. We show that adipocyte hypertrophy correlates with an adverse metabolic profile with increased levels of leptin, fasting plasma glucose, glycated hemoglobin and triglycerides, and decreased levels of adiponectin and HDL cholesterol. We performed the largest GWAS (N