Comment: 92 pagesDynamic decisions are pivotal to economic policy making. We show how existing evidence from randomized control trials can be utilized to guide personalized decisions in challenging dynamic environments with budget and capacity constraints. Recent advances in reinforcement learning now enable the solution of many complex, real-world problems for the first time. We allow for restricted classes of policy functions and prove that their regret decays at rate n^(-0.5), the same as in the static case. Applying our methods to job training, we find that by exploiting the problem's dynamic structure, we achieve significantly higher welfare compared to static approaches.