Contextual information plays a significant role in shaping our perception of speech, yet it remains uncertain at which level of processing such information integrates with acoustic cues. A key area of debate is whether top-down information influences acoustic encoding within the lower levels of the speech-processing hierarchy. This study employed a machine learning algorithm to decode the voice onset time (VOT) of speech and investigated how the gender of the speaker of a precursor sentence impacted subsequent speech perception. Using EEG recordings, we examined neural responses to a VOT continuum following male and female voices. Our results reveal that a linear representation of the VOT continuum emerged at an early EEG time window and that gender-based contextual cues modulated speech perception at this stage. Notably, since context information was not involved in the decoding procedure itself, we conclude that this modulation reflected the true effects of context on the perception of VOT. Moreover, the contextual influence extended across the entire VOT continuum, not just at specific sounds, suggesting a broad and consistent modulation of speech perception by gender-based context. These findings support the idea of a general acoustic-level mechanism through which contextual information influences the early stage of speech processing, contributing to ongoing debates about the interaction between top-down and bottom-up processes in speech perception.