Students in protracted crisis settings often face a range of challenges which combine to yield low education outcomes. This paper presents the results from a randomised controlled trial of weekend and holiday booster classes for 7th and 8th grade girls in Kakuma refugee camp in Kenya, that aimed to improve girls' education outcomes and increase transition rates from primary to secondary school. While qualitative results suggested numerous advantages of the booster classes, including more freedom to ask questions, smaller class sizes, and kinder teachers, the program did not yield statistically significant effects on learning outcomes, school attendance or noncognitive skills. Mixed-methods research suggests that the limited impacts may stem from implementation challenges including irregular booster class attendance and a lack of appropriate teaching materials. More broadly, the results show the importance of accounting for implementation challenges in the reporting of impact evaluation results.