IMPORTANCE: Norepinephrine is the first-line vasopressor for patients with septic shock. When and whether a second agent, such as vasopressin, should be added is unknown. OBJECTIVE: To derive and validate a reinforcement learning model to determine the optimal initiation rule for vasopressin in adult, critically ill patients receiving norepinephrine for septic shock. DESIGN, SETTING, AND PARTICIPANTS: Reinforcement learning was used to generate the optimal rule for vasopressin initiation to improve short-term and hospital outcomes, using electronic health record data from 3608 patients who met the Sepsis-3 shock criteria at 5 California hospitals from 2012 to 2023. The rule was evaluated in 628 patients from the California dataset and 3 external datasets comprising 10 217 patients from 227 US hospitals, using weighted importance sampling and pooled logistic regression with inverse probability weighting. EXPOSURES: Clinical, laboratory, and treatment variables grouped hourly for 120 hours in the electronic health record. MAIN OUTCOME AND MEASURE: The primary outcome was in-hospital mortality. RESULTS: The derivation cohort (n = 3608) included 2075 men (57%) and had a median (IQR) age of 63 (56-70) years and Sequential Organ Failure Assessment (SOFA) score at shock onset of 5 (3-7 [range, 0-24, with higher scores associated with greater mortality]). The validation cohorts (n = 10 217) were 56% male (n = 5743) with a median (IQR) age of 67 (57-75) years and a SOFA score of 6 (4-9). In validation data, the model suggested vasopressin initiation in more patients (87% vs 31%), earlier relative to shock onset (median [IQR], 4 [1-8] vs 5 [1-14] hours), and at lower norepinephrine doses (median [IQR], 0.20 [0.08-0.45] vs 0.37 [0.17-0.69] µg/kg/min) compared with clinicians' actions. The rule was associated with a larger expected reward in validation data compared with clinician actions (weighted importance sampling difference, 31 [95% CI, 15-52]). The adjusted odds of hospital mortality were lower if vasopressin initiation was similar to the rule compared with different (odds ratio, 0.81 [95% CI, 0.73-0.91]), a finding consistent across external validation sets. CONCLUSIONS AND RELEVANCE: In adult patients with septic shock receiving norepinephrine, the use of vasopressin was variable. A reinforcement learning model developed and validated in several observational datasets recommended more frequent and earlier use of vasopressin than average care patterns and was associated with reduced mortality.