Unassembled and partially assembled subunits of multi-protein complexes have emerged as major quality control clients, particularly under conditions of imbalanced gene expression such as stress, aging, and aneuploidy. The factors and mechanisms that eliminate such orphan subunits to maintain protein homeostasis are incompletely defined. Here, we show that the UBR4-KCMF1 ubiquitin ligase complex is required for the efficient degradation of multiple unrelated orphan subunits from the chaperonin, proteasome cap, proteasome core, and a protein targeting complex. Epistasis analysis in cells and reconstitution studies in vitro show that the UBR4-KCMF1 complex acts downstream of a priming ubiquitin ligase that first mono-ubiquitinates orphans. UBR4 recognizes both the orphan and its mono-ubiquitin and builds a K48-linked poly-ubiquitin degradation signal. The discovery of a convergence point for multiple quality control pathways may explain why aneuploid cells are especially sensitive to loss of UBR4 or KCMF1 and identifies a potential vulnerability across many cancers.