Accurate semantic segmentation of histopathological images plays a crucial role in accurate cancer diagnosis. While fully supervised learning models have shown outstanding performance in this field, the annotation cost is extremely high. Weakly Supervised Semantic Segmentation (WSSS) reduces annotation costs due to the use of image-level labels. However, these WSSS models that rely on Class Activation Maps (CAM) focus only on the most salient parts of the image, which is challenging when dealing with semantic segmentation tasks involving multiple targets. We propose a two-stage weakly supervised segmentation framework (MSRMMP) to resolve the above problems, the generation of pseudo masks based on multi-scale residual networks (MSR-Net) and the semantic segmentation based on multi-layer pseudo-supervision. MSR-Net fully captures the local features of an image through multi-scale residual module (MSRM) and generates pseudo masks using image-level label. Additionally, we employ Transunet as the segmentation backbone, and uses multi-layer pseudo-supervision algorithms to solve the problem of pseudo-mask inaccuracy. Experiments performed on two publicly available histopathology image datasets show that our proposed method outperforms other state-of-the-art weakly supervised semantic segmentation methods. Additionally, it outperforms the fully-supervised model in mIoU and has a similar result in fwIoU when compared to fully-supervised models. Compared with manual labeling, our model can significantly save the labeling time from hours to minutes.