Recent advances in artificial intelligence (AI) tools and techniques can revolutionize the discovery, development, manufacturing, and delivery of new medicines by reducing the time and cost involved. The biopharmaceutical industry is a highly regulated sector where robust regulatory oversight is essential to ensure human therapeutics' quality, efficacy, and safety. This Perspective examines the challenges of regulating AI-driven technologies in drug discovery and development. As AI is anticipated to play an unprecedented role in transforming drug development, the critical question is not whether to regulate these advancements but how to do so effectively. Here, we evaluate current global drug regulatory practices, discuss gaps and unknowns, and provide recommendations on addressing these issues to facilitate effective regulations of AI-driven innovation in the biopharmaceutical industry.