There are severe conflicts between grain production and ecological benefits, and how to explore the critical configurations of agricultural landscapes and natural habitats to clarify sustainable scenarios remains unclear. Thus, this study explored a transferable approach to generating the production possibility frontiers of trade-offs between grain production and ecological benefits (biodiversity, carbon sink, and water consumption) under unconstrained, ecological constraint, and agricultural and ecological constraint scenarios, and identified the threshold and safety area for landscape optimization in the Beijing-Tianjin-Hebei (BTH) region of China. When reaching the Pareto optimality of trade-off frontiers, the grain yield and biodiversity increased by 10%-17%, the grain yield and carbon sink increased by 15%-48%, and the grain yield and water consumption improved by 4%-25%. The grain production and ecological benefits were outside the safety area in the BTH region, and the landscape optimization strategy was different for each trade-off. Both the food and biodiversity security can be further improved through increasing by 2.7% of cropland in the BTH region. The land use strategy of converting 6.8% of the cropland of the BTH region into forest land can promote carbon sink security. Although the land use strategy of converting 2.3% of the cropland of the BTH region into grassland can promote water security, more effort should be focused on technological innovation. This study highlights that landscape optimization will promote landscape multifunctionality and provides quantitative landscape optimization thresholds and safety boundaries for improving grain production and ecological benefits.