Dietary inflammatory index (DII) was demonstrated to be correlated to quality of life (QoL) in people with osteoarthritis (OA), but there is lack of an exploration of the relationship between the longitudinal trajectories of QoL and DII in OA people. This study aims to investigate the relationship between DII and longitudinal trajectories of QoL, and the moderating effect of physical activity on this relationship in OA people. Data from the Osteoarthritis Initiative database includes baseline measurements and follow-up data at 1, 2, 3, 4, 5, 6, and 8-year intervals for the 12-item Short-Form Health Outcome Survey questionnaire (physical and mental dimensions) and baseline sociodemographic characteristics utilized in this study. Dual Group-Based Trajectory Modeling analysis was employed to identify the profiles of physical and mental QoL trajectories. Multiple logistic regression analysis was used to determine the relationship between DII and QoL profiles. A structural equation model was conducted to assess the moderating effect of physical activity on the relationship between DII and QoL in OA people. A total of 1127 OA people were categorized into three profiles based on their physical and mental QoL: low physical-low mental QoL (14.5%), low physical-high mental QoL (27.1%), and high physical-high mental QoL (58.4%). People with higher DII were more likely to be in the low physical-low mental QoL profile (OR = 1.163, P = 0.014) and low physical-high mental QoL (OR = 1.131, P = 0.013) compared to the high physical-high mental QoL profile. Additionally, a higher level of physical activity could mitigate the relationship between a higher DII and lower QoL (β = 0.064, P = 0.029). A higher DII was associated with a greater likelihood of impaired QoL. Increasing physical activity levels can help mitigate the negative impact.