Electroencephalogram (EEG) signals are time series data containing abundant brain information. However, EEG frequently contains various artifacts, such as electromyographic, electrooculographic, and electrocardiographic. These artifacts can change EEG waveforms and affect the accuracy and reliability of neuroscientific studies. Recent research has demonstrated that end-to-end deep learning approaches are highly effective in removing artifacts. Despite the widespread use of convolutional neural networks (CNN) for this task, their inability to capture multi-scale and time-dependent features impacts overall denoising performance. Therefore, we propose a hybrid network based on multi-scale CNN and bidirectional gated recurrent unit (MSCGRU) to address those issues. MSCGRU, an improved generative adversarial network, comprises a generator and a discriminator. Firstly, we design a multi-scale convolution module to extract different frequent features from EEG signals. Next, we employ a channel attention mechanism to selectively emphasize important channels and suppress irrelevant ones, enhancing extracted features' discriminative capability. Then, BiGRU is utilized to extract time-dependency features. The discriminator, a multi-layer convolutional structure, measures the similarity between generated EEG and clean EEG, further improving denoising performance. We compare MSCGRU with other denoising models on publicly available datasets. For electromyographic artifacts, MSCGRU achieves a relative root mean square error of 0.277±0.009, a correlation coefficient of 0.943±0.004, and a signal-to-noise ratio of 12.857±0.294. Results demonstrate that MSCGRU outperforms other models. This paper provides a new method to reconstruct clean EEG and may further benefit the EEG-based diagnosis and treatment.