Adolescent Idiopathic Scoliosis (AIS) is a prevalent structural deformity disease of human spine, and accurate assessment of spinal anatomical parameters is essential for clinical diagnosis and treatment planning. In recent years, significant progress has been made in automatic AIS diagnosis based on deep learning methods. However, effectively utilizing spinal structure information to improve the parameter measurement and diagnosis accuracy from spinal X-ray images remains challenging. This paper proposes a novel spine keypoint detection framework to complete the intelligent diagnosis of AIS, with the assistance of spine rigid structure information. Specifically, a deep learning architecture called Landmark and Adjacent offset Detection (LAD-Net) is designed to predict spine centre and corner points as well as their related offset vectors, based on which error-detected landmarks can be effectively corrected via the proposed Adjacent Centre Iterative Correction (ACIC) and Corner Feature Optimization and Fusion (CFOF) modules. Based on the detected spine landmarks, spine key parameters (i.e. Cobb angles) can be computed to finish the AIS Lenke diagnosis. Experimental results demonstrate the superiority of the proposed framework on spine landmark detection and Lenke classification, providing strong support for AIS diagnosis and treatment.