Retinal image registration is essential for monitoring eye diseases and planning treatments, yet it remains challenging due to large deformations, minimal overlap, and varying image quality. To address these challenges, we propose RetinaRegNet, a multi-stage image registration model with zero-shot generalizability across multiple retinal imaging modalities. RetinaRegNet begins by extracting image features using a pretrained latent diffusion model. Feature points are sampled from the fixed image using a combination of the SIFT algorithm and random sampling. For each sampled point, its corresponding point in the moving image is estimated by cosine similarities between diffusion feature vectors of that point and all pixels in the moving image. Outliers in point correspondences are detected by an inverse consistency constraint, ensuring consistency in both forward and backward directions. Outliers with large distances between true and estimated points are further removed by a transformation-based outlier detector. The resulting point correspondences are then used to estimate a geometric transformation between the two images. We use a two-stage registration framework for robust and accurate alignment: the first stage estimates a homography for global alignment, and the second stage estimates a third-order polynomial transformation to capture local deformations. We evaluated RetinaRegNet on three imaging modalities: color fundus, fluorescein angiography, and laser speckle flowgraphy. Across these datasets, it consistently outperformed state-of-the-art methods, achieving AUC scores of 0.901, 0.868, and 0.861, respectively. RetinaRegNet's zero-shot performance highlights its potential as a valuable tool for tracking disease progression and evaluating treatment efficacy. Our code is publicly available at: https://github.com/mirthAI/RetinaRegNet.