Polycyclic aromatic hydrocarbons (PAHs) represent a class of carcinogenic, teratogenic, and mutagenic aromatic organic pollutants that are ubiquitous in the environment. The rapid and on-site detection of PAHs remains a challenge. This study proposes point-of-use (POU) surface-enhanced Raman spectroscopy (SERS)-based strategies for the qualitative and quantitative analyses of PAHs in environmental water and air. The results demonstrate clear correlations between the signal intensity and the logarithmic concentration of PAHs in water (ranging from 2.5 to 100 ppb), with satisfactory recovery and reproducibility. A similar trend was observed for PAHs on glass fiber filters modified with silver nanoparticles (AgNPs@GF filter). Specifically, the limits of detection (LOD) for fluoranthene, phenanthrene, and pyrene in water were 0.7, 1.0, and 0.1 ppb, respectively, while the LOD for fluoranthene, phenanthrene, and pyrene on the AgNPs@GF filter were 9.11, 18.18, and 14.59 ppb. Recovery rates in spiked real water and filters ranged from 83% to 126%, and the entire detection process was completed within 1 min. These findings highlight the significant potential of this method as a powerful tool for rapid on-site analysis of PAHs in various environmental matrices.