Testicular histology based on testicular biopsy is an important factor for determining appropriate testicular sperm extraction surgery and predicting sperm retrieval outcomes in patients with azoospermia. Therefore, we developed a deep learning (DL) model to establish the associations between testicular grayscale ultrasound images and testicular histology. We retrospectively included two-dimensional testicular grayscale ultrasound from patients with azoospermia (353 men with 4357 images between July 2017 and December 2021 in The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China) to develop a DL model. We obtained testicular histology during conventional testicular sperm extraction. Our DL model was trained based on ultrasound images or fusion data (ultrasound images fused with the corresponding testicular volume) to distinguish spermatozoa presence in pathology (SPP) and spermatozoa absence in pathology (SAP) and to classify maturation arrest (MA) and Sertoli cell-only syndrome (SCOS) in patients with SAP. Areas under the receiver operating characteristic curve (AUCs), accuracy, sensitivity, and specificity were used to analyze model performance. DL based on images achieved an AUC of 0.922 (95% confidence interval [CI]: 0.908-0.935), a sensitivity of 80.9%, a specificity of 84.6%, and an accuracy of 83.5% in predicting SPP (including normal spermatogenesis and hypospermatogenesis) and SAP (including MA and SCOS). In the identification of SCOS and MA, DL on fusion data yielded better diagnostic performance with an AUC of 0.979 (95% CI: 0.969-0.989), a sensitivity of 89.7%, a specificity of 97.1%, and an accuracy of 92.1%. Our study provides a noninvasive method to predict testicular histology for patients with azoospermia, which would avoid unnecessary testicular biopsy.