Rigid polyurethane-based foam is an ideal choice for sandwich-panel-filling materials due to its high strength, low thermal conductivity, high adhesion, and high chemical resistivity. Since sandwich panel materials often face cyclic mechanical loads during their service, it is significant to study the design methods of fatigue-resistant rigid polyurethane foam and its fatigue failure mechanism to improve the performance of sandwich-panel-filling materials. In this study, a fatigue-resistant rubber powder/polyurethane composite material was prepared by introducing rubber powder, and its fatigue failure mechanism was systematically studied. The static mechanical test results indicate that with the introduction of 20% rubber powder, the compressive strength (at 85% strain) increased to 588 kPa. Additionally, thanks to the excellent energy absorption and dissipation properties of rubber powder, it can effectively dissipate mechanical energy during cyclic loading. The fatigue test results show that after the introduction of rubber powder, the fatigue life of the polyurethane foam material increases from 10,258 cycles (for PU, stress ratio 0.6) to 45,987 cycles (for 20R-PU, stress ratio 0.6). This study not only proves the fact that rubber powder can improve the fatigue performance of foam materials but also provides a potential option for the design of high-performance filling materials.