Social touch is critical for communication to impart emotions and intentions. However, certain autistic individuals experience aversion to social touch. Here, we used Neuropixels probes to record neural responses to social vs. non-social interactions in somatosensory cortex, tail of striatum, and basolateral amygdala. We find that wild type mice show aversion to repeated presentations of an inanimate object but not of another mouse. Cortical neurons are modulated especially by touch context (social vs. object), while striatal neurons change their preference depending on whether mice could choose or not to interact. In contrast, Fmr1 knockout (KO) mice, a model of autism, find social and non-social interactions equally aversive, especially at close proximity, and their cortical/striatal neurons are less able to discriminate social valence. A linear model shows that the encoding of certain avoidance/aversive behaviors in cortical neuron activity differed between genotypes. Thus, a reduced capacity to represent social stimuli at the circuit level may underlie social avoidance in autism.