Cancer development involves the co-evolution of cancer cells and their surrounding microenvironment, yet the dynamics of this interaction within the physical architecture remains poorly understood. Here, we present a spatial transcriptomic map at single-cell resolution, encompassing 127 multi-stage fields of view from 43 patients, to chart the evolutionary trajectories of human esophageal squamous cell carcinoma (ESCC). By analyzing 6.4 million cells, we reveal that ESCC progression is driven by a proliferative epithelial cell subpopulation that acquires dedifferentiated and invasive characteristics. At the late precancerous stage, these cells disrupt the epithelial-stromal interface and recruit normal fibroblasts via JAG1-NOTCH1 signaling, transforming them into cancer-associated fibroblasts (CAFs). This interaction leads to the formation of a "CAF-Epi" (CAF and epithelial cell) niche at the tumor edge that shields the tumor from immune surveillance. The CAF-Epi niche formation is a key indicator of progression in ESCC and other squamous cell carcinomas and patient outcomes.