BACKGROUND: The RECOVER trial randomized 493 patients with markedly treatment-resistant major depressive disorder to treatment-as-usual with or without active vagus nerve stimulation (VNS Therapy). While the primary outcome measure did not statistically separate the treatment conditions, the field may lack optimal metrics for quantifying symptom improvement in markedly treatment-resistant patients. METHODS: This study examined the impact of three factors on sensitivity to clinical improvement across the total RECOVER sample and to differences in the effectiveness of the randomized conditions, systematically varying outcome classification (remission, response, and partial response), observation period (3-12 months, 6-12 months, 10-12 months and last observation), and depression rating scale. RESULTS: Effect sizes for detecting therapeutic change across the total sample and the difference in effectiveness between the randomized groups were markedly higher for partial response than response or remission classifications. Longer observation periods produced larger therapeutic effects across the sample, but the effect sizes for the randomized treatment differences were substantially higher in the final 10-12 month period. The MADRS showed the least sensitivity to change across the sample and between the treatment groups. Using the partial response classification and the 10-12 month observation period, a significant difference between the groups was obtained for 3 of 4 depression scales. LIMITATIONS: The findings derive from a retrospective assessment of alternative outcome metrics. CONCLUSION: In a large randomized controlled trial of VNS for markedly treatment-resistant depression, the magnitude of therapeutic effects and separation of treatment groups differed as a function of outcome classification, measurement period, and rating scale.