Peer Prediction for Peer Review: Designing a Marketplace for Ideas

• Tác giả: Alexander Ugarov
• Ngôn ngữ: eng
• Ký hiệu phân loại: 004.652 Data processing || Computer science
• Thông tin xuất bản: 2023
• Bộ sưu tập: Metadata
• ID: 196740

Tóm tắt

The paper describes a potential platform to facilitate academic peer review with emphasis on early-stage research. This platform aims to make peer review more accurate and timely by rewarding reviewers on the basis of peer prediction algorithms. The algorithm uses a variation of Peer Truth Serum for Crowdsourcing (Radanovic et al., 2016) with human raters competing against a machine learning benchmark. We explain how our approach addresses two large productive inefficiencies in science: mismatch between research questions and publication bias. Better peer review for early research creates additional incentives for sharing it, which simplifies matching ideas to teams and makes negative results and p-hacking more visible.