CD8+ tissue-resident memory T cells (TRM) are strategically located in peripheral tissues, enabling a rapid response to local infections, which is different from circulating memory CD8+ T cells. Their unique positioning makes them promising targets for vaccines designed to enhance protection at barrier sites and other organs. Recent studies have shown a correlation between CD8+ TRM cells and favorable clinical outcomes in various types of cancer, indicating their potential role in immune checkpoint blockade (ICB) therapies. However, the dual nature of CD8+ TRM cells presents challenges, as their inappropriate activation may lead to autoimmunity and chronic inflammatory conditions. This review highlights significant advancements in the field, focusing on the differentiation pathways and phenotypic heterogeneity of CD8+ TRM cells across different tissues and disease states. We also review their protective roles in various contexts and the implications for vaccine development against infections and treatment strategies for tumors. Overall, this comprehensive review outlines the common features of CD8+ TRM cell differentiation and biological functions, emphasizing their specific characteristics across diverse tissues and disease states, which can guide the design of therapies against infections and tumors while minimizing the risk of autoimmune diseases.