Stochastic frontier models have attracted significant interest over the years due to their unique feature of including a distinct inefficiency term alongside the usual error term. To effectively separate these two components, strong distributional assumptions are often necessary. To overcome this limitation, numerous studies have sought to relax or generalize these models for more robust estimation. In line with these efforts, we introduce a latent group structure that accommodates heterogeneity across firms, addressing not only the stochastic frontiers but also the distribution of the inefficiency term. This framework accounts for the distinctive features of stochastic frontier models, and we propose a practical estimation procedure to implement it. Simulation studies demonstrate the strong performance of our proposed method, which is further illustrated through an application to study the cost efficiency of the U.S. commercial banking sector.