Patients with ultra rare cancers have a high unmet medical need for the development of safe and effective treatments. To advance cancer drug development is often considered economically unattractive, and usually infeasible with the use of traditional paradigms. Compounding the challenges, evolving scientific understanding of the molecular biology of cancers has resulted in further subdivision of rare cancers into small molecularly defined subsets that may be eligible for targeted therapies. Indeed, research in oncology has undergone an evolution due to advances in biomarker discovery and drug target innovation moving towards a more personalized medicine and effective approach to cancer treatment. These therapies have shown remarkable efficacy with better disease management and brought a higher quality of life for cancer patients. Given the rarity of the diseases, standard randomized controlled trials may not be feasible, and innovative study designs and statistical methods should be applied to evaluate new treatments. To this aim, regulatory agencies have developed guidelines to introduce flexibility in planning of clinical trials, including new adaptive designs, use of real-world data, and surrogate endpoints. This commentary aims at reporting challenges on the evaluation of new treatments for ultra rare cancers with a focus on innovative trial designs, statistical methods, and managing of patients as these cancers are often poorly understood, have limited clinical data, and may require specialized treatment approaches.