The recent extension of topological ideas to continuous systems with broken time-reversal symmetry, such as magnetized plasmas, provides new insights into the nature of topologically protected surface plasma waves (TSPWs). We demonstrate a unique characteristic of TSPWs propagating above the electron cyclotron frequency: their collisionless damping via coupling to the continuum of resonant modes localized inside a smooth plasma-vacuum interface. We show that damped TSPWs retain their unidirectional nature and immunity against backscattering, and that these properties can be used to generate highly localized energy sinks using sheared magnetic field.