A cornerstone of human intelligence is the ability to flexibly adjust our cognition and behavior as our goals change. For instance, achieving some goals requires efficiency, while others require caution. Different goals require us to engage different control processes, such as adjusting how attentive and cautious we are. Here, we show that performance incurs control adjustment costs when people adjust control to meet changing goals. Across four experiments, we provide evidence of these costs, and validate a dynamical systems model explaining the source of these costs. Participants performed a single cognitively demanding task under varying performance goals (e.g., being fast or accurate). We modeled control allocation to include a dynamic process of adjusting from one's current control state to a target state for a given performance goal. By incorporating inertia into this adjustment process, our model accounts for our empirical finding that people under-shoot their target control state more (i.e., exhibit larger adjustment costs) when goals switch rather than remain fixed (Study 1). Further validating our model, we show that the magnitude of this cost is increased when: distances between target states are larger (Study 2), there is less time to adjust to the new goal (Study 3), and goal switches are more frequent (Study 4). Our findings characterize the costs of adjusting control to meet changing goals, and show that these costs emerge directly from cognitive control dynamics. In so doing, they shed new light on the sources of and constraints on flexibility of goal-directed behavior.