In recent years, computational methods for quantifying cell type proportions from transcription data have gained significant attention, particularly those reference-based methods which have demonstrated high accuracy. However, there is currently a lack of comprehensive evaluation and guidance for available reference-based deconvolution methods in cell proportion deconvolution analysis. In this study, we introduce Deconvolution Evaluator (Deconer), a comprehensive toolkit for the evaluation of reference-based deconvolution methods. Deconer provides various simulated and real gene expression datasets, including both bulk and single-cell sequencing data, and offers multiple visualization interfaces. By utilizing Deconer, we conducted systematic comparisons of 16 reference-based deconvolution methods from different perspectives, including method robustness, accuracy in deconvolving rare components, signature gene selection, and building external reference. We also performed an in-depth analysis of the application scenarios and challenges in cell proportion deconvolution methods. Finally, we provided constructive suggestions for users in selecting and developing cell proportion deconvolution algorithms. This work presents novel insights to researchers, assisting them in choosing appropriate toolkits, applying solutions in clinical contexts, and advancing the development of deconvolution tools tailored to gene expression data. The tutorials, manual, source code, and demo data of Deconer are publicly available at https://honchkrow.github.io/Deconer/.