Designing microbiomes for applications in health, bioengineering, and sustainability is intrinsically linked to a fundamental theoretical understanding of the rules governing microbial community assembly. Microbial ecologists have used a range of mathematical models to understand, predict, and control microbiomes, ranging from mechanistic models, putting microbial populations and their interactions as the focus, to purely statistical approaches, searching for patterns in empirical and experimental data. We review the success and limitations of these modeling approaches when designing novel microbiomes, especially when guided by (inevitably) incomplete experimental data. Although successful at predicting generic patterns of community assembly, mechanistic and phenomenological models tend to fall short of the precision needed to design and implement specific functionality in a microbiome. We argue that to effectively design microbiomes with optimal functions in diverse environments, ecologists should combine data-driven techniques with mechanistic models - a middle, third way for using theory to inform design.