Locally advanced esophageal adenocarcinoma remains difficult to treat and the ecological and evolutionary dynamics responsible for resistance and recurrence are incompletely understood. Here, we performed longitudinal multiomic analysis of patients with esophageal adenocarcinoma in the MEMORI trial. Multi-region multi-timepoint whole-exome and paired transcriptome sequencing was performed on 27 patients before, during and after neoadjuvant treatment. We found major transcriptomic changes during treatment with upregulation of immune, stromal and oncogenic pathways. Genetic data revealed that clonal sweeps through treatment were rare. Imaging mass cytometry and T cell receptor sequencing revealed remodeling of the tumor microenvironment during treatment. The presence of genetic immune escape, a less-cytotoxic T cell phenotype and a lack of clonal T cell expansions were linked to poor treatment response. In summary, there were widespread transcriptional and environmental changes through treatment, with limited clonal replacement, suggestive of phenotypic plasticity.