Adequate revegetation of abandoned farmland acts as a defence against desertification and soil loss, and can help remove carbon dioxide in the atmosphere, thereby playing an important role in regulating regional climate change. Legume, a nitrogen-fixation species, which could effectively improve vegetation coverage to control soil erosion, was widely used for revegetation. However, the dynamics of soil and plant development after legume introduction on abandoned farmland remain unclear. A 16-year in situ experiment including three treatments, natural abandonment (fallow), planting of alfalfa (Medicago sativa L.), and sweet clover (Melilotus officinalis L.) was conducted on bare farmland of the Loess Plateau in 2003-2018. The results showed that initially introduced species significantly affected the potential succession patterns in the community. Alfalfa introduction decreased plant community stability (CS) and hindered plant species establishment in early successional stages due to inter/intraspecific competition caused by high aboveground biomass (AB). Plant CS was affected by species evenness, AB, revegetation time and revegetation methods. Sweet clover facilitated succession process by rapidly improving soil conditions (organic carbon, nitrogen, and phosphorus) and quickly exiting from the community after its life span to avoid further competitive effects. During 2003-2018, the soil (water storage, organic carbon, nitrogen, and phosphorus), plant (AB, CS), and ecological related variables (plant diversity and soil carbon sequestration) contributed 60.1%, 15.7% and 20.2%, respectively, to the ecosystem health. Alfalfa planting increased ecosystem health index (EHI) in the long-term while sweet clover favours plant diversity, providing less overall EHI but recover faster than natural abandonment community. We concluded that alfalfa introduction, which provides the greatest AB, is a good option for comprehensively improving ecosystems (e.g., soil nutrient sequestration and control soil erosion) if the site in question suffers from few disturbances. Sweet clover introduction, however, is recommendable for restoring native biodiversity effectively if disturbances are frequent.