Deep learning-based image synthesis for medical imaging is currently an active research topic with various clinically relevant applications. Recently, methods allowing training with misaligned data have started to emerge, yet current solution lack robustness and cannot handle other corruptions in the dataset. In this work, we propose a solution to this problem for training synthesis network for datasets affected by mis-registration, artifacts, and deformations. Our proposed method consists of three key innovations: meta-learning inspired re-weighting scheme to directly decrease the influence of corrupted instances in a mini-batch by assigning lower weights in the loss function, non-local feature-based loss function, and joint training of image synthesis network together with spatial transformer (STN)-based registration networks with specially designed regularization. Efficacy of our method is validated in a controlled synthetic scenario, as well as public dataset with such corruptions. This work introduces a new framework that may be applicable to challenging scenarios and other more difficult datasets.