The nature of memory deficits in mesial temporal lobe epilepsy (mTLE) can depend on the side of the epileptogenic zone (EZ). Although left mTLE has been consistently associated with impairments in verbal memory, the association between right mTLE and non-verbal memory is less clear, which may be attributable to methodological limitations of existing visuospatial memory tests (e.g., possible use of verbalization strategies, involvement of psychomotor or visuospatial skills, task structure which differs between verbal and non-verbal memory tests). The Brown Location Test (BLT) was developed to evaluate visuospatial memory by addressing such limitations, and displays a good potential value for lateralizing the EZ. To our knowledge, no independent study has been conducted to assess the contribution of the BLT in lateralizing and localizing the EZ among patients undergoing pre-surgical assessment for drug-resistant focal epilepsy. This study aims to do so by comparing visuospatial and verbal memory performance of patients according to EZ lateralization (left vs. right) and localization (mesiotemporal vs. non-mesiotemporal). Eighty-nine adult patients with TLE were divided into four groups based on EZ (left mTLE, right mTLE, left non-mTLE, right non-mTLE) and were administered the BLT and the Rey Auditory Verbal Learning Test (RAVLT) pre-operatively. Memory performance was associated with sex, education and number of antiseizure medications. Only delayed recognition hits on the RAVLT showed the expected material-specific impairment associated with left mTLE, whereas BLT performance did not differ between groups. Therefore, caution is required when interpreting material-specific memory performances to localize and lateralize the EZ among patients undergoing pre-surgical assessment for drug-resistant epilepsy.