Much remains unknown about the computations that allow animals to flexibly integrate across multiple timescales in natural sounds. One key question is whether multiscale integration is accomplished by diverse populations of neurons, each of which integrates information within a constrained temporal window, or whether individual units effectively integrate across many different temporal scales depending on the information rate. Here, we show that responses from neurons throughout the ferret auditory cortex are nearly completely unaffected by sounds falling beyond a time-limited "integration window". This window varies substantially across cells within the auditory cortex (~15 to ~150 ms), increasing substantially from primary to non-primary auditory cortex across all cortical layers, but is unaffected by the information rate of sound. These results indicate that multiscale computation is predominantly accomplished by diverse and hierarchically organized neural populations, each of which integrates information within a highly constrained temporal window.