INTRODUCTION: Targeting immune checkpoint proteins (ICPs) via small molecules open a new window for cancer immunotherapy. Herein, we summarize recent advances of small molecules with novel chemical structures targeting ICPs, discusses their anti-tumor efficacies, which are important for the development of novel small molecules for cancer immunotherapy. AREAS COVERED: In this review, the latest patents and literature were gathered through the comprehensive searches in the databases of European Patent Office (EPO), Cortellis Drug Discovery Intelligence (CDDI), PubMed and Web of Science using ICPs and compounds as key words. EXPERT OPINION: To develop novel weapons to fight against cancer, small molecules targeting ICPs including CTLA-4, LAG-3, PD-L1, Siglec-9, TIM-3, TIGIT, and VISTA have been synthesized and evaluated in succession. Chief among them are the small molecules targeting PD-L1, which have been intensively investigated in recent years. Various in vitro assays such as ALPHA, HTRF binding assay, NFAT assay have been successfully developed to screen novel IPCs inhibitors. However, the in vivo assay, for example, using double-humanized PD-1/PD-L1 (hPD-1/hPD-L1) mouse as evaluation model, are seldom reported. Novel pharmacophores with new working mechanisms such as proteolysis targeting chimeras (PROTACs) and peptides are needed to enhance the therapeutic efficacy.