Faces-the most common and complex stimuli in our daily lives-contain multidimensional information used to infer social attributes that guide consequential behaviors, such as deciding who to trust. Decades of research illustrates that perceptual information from faces is processed holistically. An open question, however, is whether goals might impact this perceptual process, influencing the encoding and representation of the complex social information embedded in faces. If an individual were able to factorize information so that each dimension is separately represented, it might enable flexibility. Having a goal, for example, might mean that only goal-relevant dimensions are leveraged to inform behavior. Whether people are able to build such factorized representations remains unknown, largely due to natural correlations between social attributes. We overcome these confounds using a new statistical face model that orthogonalizes perceived facial attractiveness and trustworthiness. Across three experiments (