Grape pomace is a by-product abundant in phenolic compounds that can be used in the food, cosmetic, and pharmaceutical industries. For the efficient extraction of such compounds, an aqueous solution of non-ionic surfactant Brij S20 was applied as a green extraction medium, and the optimization was performed using surface response methodology. The effects of four independent factors (surfactant concentration, time, pH, and solvent-to-material ratio) were evaluated, and total phenolic content (TPC), DPPH radical inhibition, and selected polyphenol compound concentrations were analyzed as responses. Using response surface methodology (RSM), five regression equations were derived and good adequacy of the models was confirmed. The solvent-to-material (SM) ratio was the most influential factor. Surfactant concentration of 3% (m/V), extraction time of 120 min, pH value of 4.06, and SM ratio of 50 mL/g were determined as optimum conditions to maximize all responses. Under the optimal conditions, the mean validated values obtained for TPC, DPPH, gallic acid, catechin, and quercetin concentrations were 968.50 ± 37.06 mg GAE/L, 61.41 ± 7.13%, 5.10 ± 0.05 mg/L, 10.62 ± 0.79 mg/L, and 6.04 ± 0.10 mg/L, respectively. Furthermore, the established conditions were applied for the extraction of phenolic compounds from grape pomace of four grape varieties. The proposed extraction method proved effective, providing extracts rich in polyphenols suitable for further applications.