Accurate assessment of soluble solid content (SSC) in blueberries is crucial for quality evaluation. However, in real production lines, blueberries are usually in random placement and the biological heterogeneity of blueberry parts can lead to spectral distortion, which affects the accuracy of SSC prediction models in various placement situations. Therefore, it is crucial to investigate an appropriate modeling method to minimize these negative effects. In this paper, we propose an approach that combines hyperspectral imaging (HSI) technique, residual multilayer perceptron, and transfer learning to build a universal model capable of detecting blueberry SSC in various placement situations. The study acquired SSC values of 1150 blueberry samples and hyperspectral data at different surfaces (stem end, calyx end, and two parts of the equatorial plane), used a residual multilayer perceptron to build a local model, and fine-tuned the model by transfer learning to improve its generalization ability. The results show that the optimized model has significantly improved prediction accuracy on different surfaces, especially the model based on equatorial surface data (enhanced-equator-1) performs well. In the external validation set, the model achieved correlation coefficients of prediction (r