Though it has been recognized that the sensitivity of hydrophones depends on the temperature of the water they are used in, the amount of specific data that is available is very limited. This is possible because the measurements are technically challenging, laborious, and time-consuming. A broadband primary hydrophone calibration setup was extended to implement stable calibration conditions at different temperature settings. Four hydrophones of different types commonly used in ultrasound exposimetry were then characterized in the range from 17 °C to 29 °C, and average change rates of the sensitivity with temperature were determined for different ultrasonic frequencies. Two different membrane hydrophones showed an increase in sensitivity with increasing temperature in the range from 0.55% to 1.10% per 1 °C temperature rise within their bandwidths. The results for a capsule-type and a needle-type hydrophone were different in the sense that a decreasing sensitivity with increasing temperature was also observed. For the capsule-type hydrophone, a small increase was observed up to 15 MHz and a decrease for higher frequencies. The needle-type hydrophone provided a decrease at all frequencies, and the results were more noisy. Data, as determined in this study, can be applied to correct acoustic output measurements of medical ultrasonic equipment if the water temperature of the hydrophone application differs from that during calibration. Alternatively, it may suffice in some applications to consider a sensitivity change with temperature within the uncertainty estimation of the exposure measurement, in particular, if the temperature difference is limited to 1 °C or 2 °C.