HIV/AIDS is among the major viral infectious diseases that are major traits with high human morbidity and mortality impacts and it has detrimental effects on the economies and health of nations worldwide. In the current study, we formulated and analyzed an HIV/AIDS compartmental epidemic model by dividing the entire susceptible human population into two groups, such as the sexually mature and sexually immature stage structure groups with optimal control strategies, since HIV/AIDS has been a cause of death for individuals in different age groups. Qualitatively, we computed the model's unique disease-free and endemic equilibrium points, using the next-generation matrix operator approach we calculated the model basic reproduction number, and we proved the local stabilities of both the model disease-free and endemic equilibrium points using by applying the Routh-Hurwitz stability criteria, we also proved the global stabilities of the model's disease-free and endemic equilibrium points by building the suitable representative Lyapunov function. Using Pontryagin's Maximum Principle, the HIV/AIDS model's optimality system which determines the necessary circumstances to improve the control over the spread of the HIV/AIDS disease is built and examined. We performed numerical simulations to validate the qualitative analysis of the optimal control and explore the effects of the suggested optimal control techniques using the model measure, and the ART treatment measure concurrently, i.e., putting Strategy F into practice is essential to reducing and controlling the spread of HIV/AIDS in the community. Therefore, we urge possible policymakers and public health stakeholders to prioritize the implementation of Strategy F, which aims to reduce and control the prevalence of HIV/AIDS in the population.