The association between low serum testosterone levels and all-cause mortality in male and female patients with cardiovascular disease (CVD) was investigated. This study extracted data on CVD patients from the National Health and Nutrition Examination Survey (NHANES) database (1999-2000, 2003-2004, 2011-2012, and 2013-2014). The association between low serum testosterone levels (≤ 300 ng/dL) and all-cause mortality in male and female CVD patients was evaluated using univariate and multivariate Cox regression analyses, with hazard ratios (HR) and 95% confidence intervals (CI). A total of 1,177 participants (689 males) with a mean age of 66.01 ± 12.52 years were included in the study. The median follow-up time was 55 (44, 71) months. Low serum testosterone levels occurred in 487 (70.68%) males and 394 (80.74%) females. Additionally, 202 (29.32%) male patients and 94 (19.26%) female patients with CVD were dead. After adjusting for covariates, low serum testosterone levels were associated with an increased risk of all-cause mortality in male CVD patients (HR = 1.48, 95% CI 1.08-2.02, P = 0.013), while the association was not significant in females. Low serum testosterone levels may be associated with an increased risk of all-cause mortality in male CVD patients, but not in female patients.