The study examined the role of two slave systems of working memory (WM), the phonological loop and the visuospatial sketchpad, in the speechreading performance of Chinese students with hearing loss (HL). It was motivated by the question whether the visual speech information is processed in the phonological loop as linguistic information or as visuospatial information in visuospatial sketchpad. Seventy-three young adults with HL completed Chinese speech-reading tests (targeting monosyllabic words, disyllabic words, and sentences), the WM test batteries, and a cognitive processing speed test. The hierarchical regression analyses showed that the articulatory rehearsal process and phonological store were the most important predictors of all of the components of WM for speechreading. By contrast, performance on visual spatial working memory tasks was not significantly correlated with speechreading performance. These results demonstrated that speechreading in Chinese students with HL relies more on the efficiency of high-level phonological storing and articulatory rehearsal in the phonological loop, rather than the shallow processing of pure visual features in the visuospatial sketchpad.