In plant-sediment microbial fuel cells (P-SMFCs), the anode serves as the primary site for biochemical reactions. In this study, different carbon nanomaterials (graphenes (GNs), carbon nanotubes (CNT), hydroxylated-carbon nanotubes (CNT-OH), and carboxylated-carbon nanotubes (CNT-COOH)) were used to modify the anode of the P-SMFCs to explore the enhancement of phenanthrene (Phe) degradation. The devices were operated for 131 days, CNT-COOH-modified P-SMFCs (P-CNT-COOH) exhibited a shorter start-up period and higher voltage during the stable operation stage. The voltage of P-CNT-COOH during the stationary phase was approximately 250 mV higher than that of the control device. The voltage and Phe removal of P-CNT-COOH were higher than those of CNT-COOH (without plants in the SMFC), which achieved 67.5% Phe removal, which was 1.25 times higher than the P-CNT, whereas CNT (without plants in the SMFC) showed higher performance than P-CNT. The anode modified with P-CNT-COOH became enriched with small-molecule volatile fatty acids (VFAs) (e.g., acetic acid) and degrading bacteria (e.g., Thiobacillus and Desulfobulbus) attributed to the higher hydrophilicity. The removal of Phe was positively correlated with dehydrogenase activity (DHAA).