Waste management handles all kinds of waste, including household, industrial, municipal, organic, biomedical, biological, and radioactive wastes. People still face challenges in proper disposal methods for different types of waste, including landfill-bound items, recyclable materials, and biodegradable waste. Inadequate waste management poses a significant and multifaceted global challenge. The conventional method of segregating waste is a time-consuming and ineffective method that wastes human power and money. To address this issue in real time, sophisticated and sustainable waste management systems need to be implemented. The latest advancements in computer vision and deep learning offer efficient solutions for effective recycling and waste management. Existing deep learning models exhibited various limitations, such as detection accuracy and computational inefficiency, particularly when dealing with objects of varying sizes and exhibiting high degrees of visual similarity. These limitations generate various challenges in effectively capturing and representing the nuanced features of visually similar objects. To address this problem, we proposed the stacking of an enhanced Swin Transformer, improved ConvNeXt, and a spatial attention mechanism. The enhanced Swin transformers incorporate two key components- hierarchical feature extraction and shifting window mechanism to extract the global features from the garbage images effectively. The shifting window mechanism extracts the most important features from various regions of the images to identify the objects. In contrast, the hierarchical feature extraction captures long-range dependencies within the image to effectively identify different types of garbage. The improved ConvNext block with optimized parameterization extracts the local features of the image. This enhanced feature extraction capability enables the model to effectively discern fine-grained details of individual garbage particles, such as shape, texture, and subtle variations in color and appearance, leading to more accurate classification results. When we evaluated the performance of the proposed model using the publicly available Garbage Classification dataset, it attained 98.97% accuracy, 98.42% Precision, and 98.61% Recall. Due to its lightweight and low computational time and power, the proposed model surpasses the existing state-of-the-art deep learning models.