In recent years, the progress of artificial intelligence has greatly advanced computer-assisted intervention, surgical learning, and postoperative surgical video analysis techniques, greatly improving the skill levels of surgeons and overall outcomes. Deep learning based endoscopic surgery phase recognition has a very high dependency on large-scale datasets and annotations. This study introduces the Renji endoscopic submucosal dissection (ESD) video dataset for early gastric cancer (EGC), comprising 20 ESD endoscopic videos and 66,656 phase recognition annotations jointly annotated by three endoscopists. To the best of our knowledge, this is the first publicly available ESD dataset for the treatment of EGC, and we believe this work will contribute to the standardization of ESD dataset construction. The dataset and annotations are publicly available in Figshare.