The integration of artificial intelligence (AI) into medical education presents both unprecedented opportunities and significant challenges, epitomized by the "alignment paradox." This paradox asks: How do we ensure AI systems remain aligned with our educational goals? For instance, AI could create highly personalized learning pathways, but this might conflict with educators' intentions for structured skill development. This paper proposes a framework to address this paradox, focusing on four key principles: ethics, robustness, interpretability, and scalable oversight. We examine the current landscape of AI in medical education, highlighting its potential to enhance learning experiences, improve clinical decision making, and personalize education. We review ethical considerations, emphasize the importance of robustness across diverse healthcare settings, and present interpretability as crucial for effective human-AI collaboration. For example, AI-based feedback systems like i-SIDRA enable real-time, actionable feedback, enhancing interpretability while reducing cognitive overload. The concept of scalable oversight is introduced to maintain human control while leveraging AI's autonomy. We outline strategies for implementing this oversight, including directable behaviors and human-AI collaboration techniques. With this road map, we aim to support the medical education community in responsibly harnessing AI's power in its educational systems.