BACKGROUND: Early-life adversity is pervasive worldwide and represents a potent risk factor for increased mental health burden across the lifespan. However, there is substantial individual heterogeneity in associations between adversity exposure, neurobiological changes, and mental health problems. Accounting for key features of adversity such as the developmental timing of exposure may clarify associations between adversity, neurodevelopment, and mental health. METHODS: The present study leverages sparse canonical correlation analysis to characterize modes of covariation between adversity exposure across development and the connectivity of white matter tracts throughout the brain in a sample of 107 adults. RESULTS: We found that adversity exposure during preschool-age and middle childhood (ages 4-5 and 8 in particular) were consistently linked across diffusion metrics with alterations in white matter tract connectivity. Whereas tracts supporting sensorimotor functions displayed higher connectivity with higher preschool-age and middle childhood adversity exposure, tracts supporting cortico-cortical communication displayed lower connectivity. Further, latent patterns of tract connectivity linked with adversity experienced across preschool-age and middle childhood (ages 3-8) were associated with post-traumatic stress symptoms in adulthood. CONCLUSIONS: Our findings underscore that adversity exposure may differentially affect white matter in a function- and developmental-timing specific manner and suggest that adversity experienced between ages 3-8 may shape the development of white matter tracts across the brain in ways that are relevant for mental health in adulthood.