A surrogate marker is a biological measurement in a clinical trial that aims to replace the primary outcome in evaluating the treatment effect, and can be measured earlier, with less cost, or with less patient burden. In theory, once a surrogate is validated, future studies can evaluate treatment efficacy using only the surrogate. While there are many methods to evaluate a surrogate, these methods rarely account for heterogeneity in surrogacy, that is, when a surrogate is valid for only certain people. We propose a general framework for the assessment of complex heterogeneity in the strength of a surrogate marker, as well as corresponding parametric and semiparametric estimation procedures. Our framework defines the proportion of the treatment effect on the primary outcome that is explained by the treatment effect on the surrogate, as a function of multiple baseline covariates,