Accurate and timely diagnosis of brain tumors is critical for patient management and treatment planning. Magnetic resonance imaging (MRI) is a widely used modality for brain tumor detection and characterization, often aided by the administration of gadolinium-based contrast agents (GBCAs) to improve tumor visualization. Recently, deep learning models have shown remarkable success in predicting contrast-enhancement in medical images, thereby reducing the need of GBCAs and potentially minimizing patient discomfort and risks. In this paper, we present a study aimed at investigating the generalization capabilities of a neural network trained to predict full contrast in brain tumor images from noncontrast MRI scans. While initial results exhibited promising performance on a specific tumor type at a certain stage using a specific dataset, our attempts to extend this success to other tumor types and diverse patient populations yielded unexpected challenges and limitations. Through a rigorous analysis of the factor contributing to these negative results, we aim to shed light on the complexities associated with generalizing contrast enhancement prediction in medical brain tumor imaging, offering valuable insights for future research and clinical applications.