OBJECTIVES: The Bosniak classification system (BCS) is integral in clinical decision-making for renal cysts and cystic renal masses, with updates in 2005 and 2019 aiming to enhance diagnostic accuracy and clinical utility. Despite these revisions, challenges in inter-rater agreement and practical applicability persist, underscoring the need for comprehensive evaluation of both versions' effectiveness in guiding patient care. METHODS: This meta-analysis adhered to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and systematically reviewed studies comparing the 2005 and 2019 BCS versions using CT or MRI. We searched PubMed, Embase, Scopus, and Web of Science and used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) for quality assessment. In this study, we focused on diagnostic performance and inter-observer agreement. Statistical analysis involved bivariate random-effects modelling and assessing pooled sensitivity, specificity, and inter-rater reliability. RESULTS: We included 11 articles. The 2019 Bosniak classification showed higher diagnostic performance for CT (sensitivity: 85.7%, specificity: 81.9%) and MRI (sensitivity: 96.2%, specificity: 70.9%) compared to the 2005 version. Inter-rater reliability was better with the 2019 classification (CT kappa: 0.813, MRI kappa: 0.601) than with the 2005 version. The quality assessment indicated a low risk of bias overall, though some studies had high risk in specific areas. CONCLUSION: The Bosniak 2019 classification provides improved diagnostic specificity and inter-rater reliability compared to the 2005 version. Its adoption may enhance clinical decision-making and reduce overtreatment in managing cystic renal masses. ADVANCES IN KNOWLEDGE: This paper is novel in being the first meta-analysis to demonstrate that the Bosniak 2019 classification system significantly enhances inter-rater agreement, and overall diagnostic performance compared to the 2005 version, particularly in CT imaging, thus offering a more accurate and reliable tool for evaluating cystic renal masses.