OBJECTIVE: To resolve the underestimation problem and investigate the mechanism of the AI model which employed to predict cardiovascular disease (CVD) risk scores from retinal fundus photos. METHODS: An ordinal regression Deep Learning (DL) model was proposed to predict 10-year CVD risk scores. The mechanism of the DL model in understanding CVD risk was explored using methods such as transfer learning and saliency maps. RESULTS: Model development was performed using data from 34,652 participants with good-quality fundus photographs from the UK Biobank and a dataset for external validation collected in Australia comprised of 1376 fundus photos of 401 participants with a desktop retinal camera and a portable retinal camera. The mean [SD] risk-level accuracies across cross-validation folds was 0.772 [0.008], while AUROC for over moderate risk was 0.849 [0.005] and the AUROC for high risk was 0.874 [0.007] on the UK Biobank dataset. The risk-level accuracy for images acquired with the desktop camera data was 0.715, and the accuracy for portable camera data was 0.656 on the external dataset. CONCLUSIONS: The DL model described in this study has minimized the underestimation problem. Our analysis confirms that the DL model learned CVD risk score prediction primarily from age- and sex-related image representation. Model performance was only slightly degraded when features such as the retinal vessels and colours were removed from the images. Our analysis identified some image features associated with high CVD risk status, such as the peripheral small vessels and the macula areas.