To realize the accurate identification of coal-gangue in the process of underground coal transportation and the low-cost deployment of the model, a lightweight coal-gangue detection model based on the parallel depth residual network, called P-RNet, is proposed. For the problem of images of coal-gangue taken under complex conditions, the feature extraction module (FEM) is designed using decoupling training and inference methods. Furthermore, for the problem of the nearest neighbor interpolation upsampling method being prone to produce mosaic blocks and edge jagged edges, a lightweight upsampling operator is used to optimize the feature fusion module (FFM). Finally, to solve the problem, the stochastic gradient descent algorithm is prone to local suboptimal solutions and saddle point problems in the error function optimization process, numerous experiments are carried out on selecting the initial learning rate, and the Lookahead optimizer is used to optimize parameters during backpropagation. Experimental results show that the proposed model can effectively improve the recognition effect, with a corresponding low deployment cost.