Plasmonic ultrafast all-optical switching holds great promise for advancing next-generation optical communication and optical computing technologies. However, achieving subpicosecond all-optical switching with a high on-off ratio remains challenging due to the slow dynamics of electron-phonon scattering in plasmonic materials. Here, we report an innovative method that utilizes the negative signal induced by plasmonic excited hot electrons in the transient spectrum and the positive signal caused by hot electrons excited by off-resonant pumping, both designed at the same wavelength to effectively offset slow dynamics. This approach enables plasmonic ultrafast all-optical switching with a 500 fs response time and a superior on-off ratio exceeding 20 within 1 ps. The strategy offers a promising path for high-performance all-optical modulation and can be widely applied across various plasmonic materials.