In many scenarios, such as the evaluation of place-based policies, potential outcomes are not only dependent upon the unit's own treatment but also its neighbors' treatment. Despite this, "difference-in-differences" (DID) type estimators typically ignore such interference among neighbors. I show in this paper that the canonical DID estimators generally fail to identify interesting causal effects in the presence of neighborhood interference. To incorporate interference structure into DID estimation, I propose doubly robust estimators for the direct average treatment effect on the treated as well as the average spillover effects under a modified parallel trends assumption. I later relax common restrictions in the literature, such as immediate neighborhood interference and correctly specified spillover functions. Moreover, robust inference is discussed based on the asymptotic distribution of the proposed estimators.