This paper considers inference in a linear instrumental variable regression model with many potentially weak instruments and heterogeneous treatment effects. I first show that existing test procedures, including those that are robust to only either weak instruments or heterogeneous treatment effects, can be arbitrarily oversized in this setup. Then, I propose a valid inference procedure based on a score statistic and a leave-three-out variance estimator. To establish this procedure's validity, this paper proves that the score statistic is asymptotically normal and the variance estimator is consistent. The power of the score test is also close to a power envelope in an empirical application.