Human genome analyses have revealed that abnormal BAF (BRG1/BRM-associated factor) complex is highly associated with hearing loss. However, the underlying pathogenesis remains largely unknown. Disrupted structure and function of the organ of Corti is the most prevalent cause of sensorineural hearing loss in mammals. Here, we investigated the role of Brg1-based BAF complex during the differentiation and development of the auditory sensory epithelium, a crucial period for the formation of the organ of Corti. Our findings indicate that deletion of Brg1 leads to premature hair cell (HC) differentiation by inactivating Sonic hedgehog (Shh) signaling. Despite the formation of HCs, subsequent differentiation of inner hair cells (IHCs) and outer hair cells (OHCs) was impaired. Additionally, we observed that the mosaic-like arrangement of HCs and supporting cells (SCs) was disrupted resulting in abnormal sensory epithelium patterning. Furthermore, we found the planar cell polarity of the Brg1-deficient cochlea was abnormal. Our study demonstrates the pivotal role of Brg1 in the differentiation and patterning of the organ of Corti.