X-ray inspection is a crucial technique for identifying defects in castings, capable of revealing minute internal flaws such as pores and inclusions. However, traditional methods rely on the subjective judgment of experts, are time-consuming, and prone to errors, which negatively impact the efficiency and accuracy of inspections. Therefore, the development of an automated defect detection model is of significant importance for enhancing the scientific rigor and precision of casting inspections. In this study, we propose a deep learning model specifically designed for detecting small-scale defects in castings. The model employs an end-to-end network architecture and features a loss function based on the Wasserstein distance, which is tailored to optimize the training process for small defect targets, thereby improving detection accuracy. Additionally, we have innovatively developed a dual-layer Encoder-Decoder multi-scale feature extraction architecture, BiSDE, based on the Hadamard product, aimed at enhancing the model's ability to recognize and locate small targets. To evaluate the performance of the proposed model, we conducted a series of experiments, including comparative tests with current state-of-the-art object detection models such as Yolov9, FasterNet, Yolov8, and Detr, as well as ablation studies on the model's components. The results demonstrate that our model achieves at least a 5.3% improvement in Mean Average Precision (MAP) over the existing state-of-the-art models. Furthermore, the inclusion of each component significantly enhanced the overall performance of the model. In conclusion, our research not only validates the effectiveness of the proposed small-scale defect detection model in improving detection precision but also offers broad prospects for the automation and intelligent development of industrial defect inspection.