For efficient anticancer drug delivery, cascade physiological barriers must be overcome, which requires the drug delivery vehicles to possess different or even opposite properties at different stages. Poly(tertiary amine-oxide) (PTAO) polymers with the zwitterionic feature have distinct antifouling properties in blood circulation, which can be reduced and protonated in hypoxic tumors to promote cellular internalization. Nevertheless, the effects of various PTAO structures have not been studied systemically and optimized. In this report, the side groups of PTAO are proposed to be optimized by modulating the structures. Poly(2-(N-oxide-hexamethyleneimino)ethyl methacrylate) (POC7A) with a cyclic seven-membered ring is screened as the optimized PTAO structure for in vivo applications. Moreover, the block copolymer POC7A-block-poly(ε-caprolactone) (POC7A-PCL) is prepared for the formation of micelles in aqueous solution for delivery of doxorubicin (DOX). The zwitterionic nature of POC7A shells with efficient bioreductive activity and protonation in the tumor microenvironment endows the micelles with excellent antifouling properties for long blood circulation, efficient tumor accumulation, deep penetration, and effective cellular internalization. Thus, the DOX-loaded micelles exhibit potent antitumor efficacy after intravenous administration. Zwitterionic POC7A can be used as antifouling shells of the anticancer drug delivery nanocarriers to overcome the cascade physiological barriers efficiently.