PURPOSE: This study aims to develop an autonomous robotic ultrasound scanning system (auto-RUSS) pipeline, comparing its reproducibility and observer consistency in image analysis with physicians of varying levels of expertise. DESIGN/METHODOLOGY/APPROACH: An auto-RUSS was engineered using a 7-degree-of-freedom robotic arm, with real-time regulation based on force control and ultrasound visual servoing. Two phantoms were employed for the human-machine comparative experiment, involving three groups: auto-RUSS, non-expert (4 junior physicians), and expert (4 senior physicians). This setup enabled comprehensive assessment of reproducibility in contact force, image acquisition, image measurement and AI-assisted classification. Radiological feature variability was measured using the coefficient of variation (COV), while performance and reproducibility assessments utilized mean and standard deviation (SD). FINDINGS: The auto-RUSS had the potential to reduce operator-dependent variability in ultrasound examinations, offering enhanced repeatability and consistency across multiple dimensions including probe contact force, images acquisition, image measurement, and diagnostic model performance. ORIGINALITY/VALUE: In this paper, an autonomous robotic ultrasound scanning system (auto-RUSS) pipeline was proposed. Through comprehensive human-machine comparison experiments, the auto-RUSS was shown to effectively improve the reproducibility of ultrasound images and minimize human-induced variability.