Evaluation of ChatGPT as a Reliable Source of Medical Information on Prostate Cancer for Patients: Global Comparative Survey of Medical Oncologists and Urologists.

• Tác giả: Andrew J Armstrong, Dheepa Chari, Sam Garas, Jennifer Ghith, Martin Gleave, Dany Habr, Eamonn Rogers, Andrea Sboner, Kristine W Schuler, Lucile Serfass, Arnulf Stenzl, Cora N Sternberg, Ken Truman, Jochen Walz
• Ngôn ngữ: eng
• Ký hiệu phân loại: 363.232 Patrol and surveillance
• Thông tin xuất bản: United States : Urology practice , 2025
• Bộ sưu tập: NCBI
• ID: 215804

Tóm tắt

 INTRODUCTION: No consensus exists on performance standards for evaluation of generative artificial intelligence (AI) to generate medical responses. The purpose of this study was the assessment of Chat Generative Pre-trained Transformer (ChatGPT) to address medical questions in prostate cancer. METHODS: A global online survey was conducted from April to June 2023 among >
  700 medical oncologists or urologists who treat patients with prostate cancer. Participants were unaware that this was a survey evaluating AI. In component 1, responses to 9 questions were written independently by medical writers (MWs
  from medical websites) and ChatGPT 4.0 (AI-generated from publicly available information). Respondents were randomly exposed and blinded to both AI-generated and MW-curated responses
  evaluation criteria and overall preference were recorded. Exploratory component 2 evaluated AI-generated responses to 5 complex questions with nuanced answers in the medical literature. Responses were evaluated on a 5-point Likert scale. Statistical significance was denoted by RESULTS: In component 1, respondents (N = 602) consistently preferred the clarity of AI-generated responses over MW-curated responses in 7 of 9 questions ( CONCLUSIONS: These findings provide insight into how clinicians rate AI-generated and MW-curated responses with evaluation criteria that can be used in future AI validation studies.