AIM: To develop different machine learning models to train and test diplopia images and data generated by the computerized diplopia test. METHODS: Diplopia images and data generated by computerized diplopia tests, along with patient medical records, were retrospectively collected from 3244 cases. Diagnostic models were constructed using logistic regression (LR), decision tree (DT), support vector machine (SVM), extreme gradient boosting (XGBoost), and deep learning (DL) algorithms. A total of 2757 diplopia images were randomly selected as training data, while the test dataset contained 487 diplopia images. The optimal diagnostic model was evaluated using test set accuracy, confusion matrix, and precision-recall curve (P-R curve). RESULTS: The test set accuracy of the LR, SVM, DT, XGBoost, DL (64 categories), and DL (6 binary classifications) algorithms was 0.762, 0.811, 0.818, 0.812, 0.858 and 0.858, respectively. The accuracy in the training set was 0.785, 0.815, 0.998, 0.965, 0.968, and 0.967, respectively. The weighted precision of LR, SVM, DT, XGBoost, DL (64 categories), and DL (6 binary classifications) algorithms was 0.74, 0.77, 0.83, 0.80, 0.85, and 0.85, respectively
weighted recall was 0.76, 0.81, 0.82, 0.81, 0.86, and 0.86, respectively
weighted F1 score was 0.74, 0.79, 0.82, 0.80, 0.85, and 0.85, respectively. CONCLUSION: In this study, the 7 machine learning algorithms all achieve automatic diagnosis of extraocular muscle palsy. The DL (64 categories) and DL (6 binary classifications) algorithms have a significant advantage over other machine learning algorithms regarding diagnostic accuracy on the test set, with a high level of consistency with clinical diagnoses made by physicians. Therefore, it can be used as a reference for diagnosis.