A new metaheuristic optimizer combined with artificial neural networks is proposed for streamflow prediction. Hence, the study aimed to forecast monthly streamflow of the main rivers in Urmia, Iran, by considering data shortage and using artificial neural network (ANN) models. By combining three variables: temperature, precipitation, and streamflow, we formulated five patterns, where 70% of the data were used for model training, and 30% for model testing. To improve the performance of ANN, we evaluated a new optimization algorithm, reptile search algorithm (RSA), and compared the results with combinations of ANN, particle swarm optimization algorithm (PSO), and whale optimization algorithm (WOA) models. The results of the ANN + RSA were promising at most stations and patterns. At Band station streamflow simulation testing gave RMSE, MAE, and NSE of 1.65, 1.21 MCM/month, and 0.80, respectively. At Babaroud station they were 4.01, 3.0 MCM/month and 0.68, respectively, at Nazlo station 5.62, 3.79 MCM/month, and 0.69, respectively, and at Tapik station 5.69, 3.82 MCM/month, and 0.59, respectively. However, the results of the ANN + PSO hybrid model were better than ANN + RSA. The impact of different parameters on the accuracy of streamflow prediction varied depending on model and streamflow station, indicating that the models do not perform consistently across different locations, times, and conditions. The inclusion of lagged monthly streamflow in the model was an influential input parameter. The results demonstrated that the new algorithm consistently improved predictions, enhancing the performance of traditional algorithms. The findings of this study highlight advantage of the ANN + RSA hybrid model for specific areas, suggesting its potential application in other similar hydrological problems for further validation.