Herbivores are commonly classified as host specialists or generalists for various purposes, yet the definitions of these terms, and their intermediates, are often imprecise and ambiguous. We quantified host breadth for 240 non-native, tree-feeding insects in North America using phylogenetic diversity. We demonstrated that a partitioning of host breadth: (1) causes 67% of non-native insects to shift from a generalist to specialist category, (2) displays a reduction in host breadth from the native to introduced range, (3) identifies an inflection point in a model predicting the likelihood of non-native insect ecological impact, with a corresponding change in behaviour associated with specialists versus generalists, and (4) enables three models for strong prediction of whether a non-native forest insect will cause high impacts. Together, these results highlight the primacy of how herbivore host recognition and plant defences mediate whether novel host interactions will result in high impact after invasion.