In the natural environment, we are attracted to pleasant stuffs such as fur and jewels, and turn away from unpleasant stuffs such as decaying matter and gatherings of small insects. Increasing evidence for the dependence of human preferences and aesthetic judgments on simple image features implies that such affective responses to natural surfaces and textures are related to specific image statistics. Using 194 diverse natural surfaces and their statistically synthesized images, the present study examined if and how ratings of pleasantness - unpleasantness are related to low-level image statistics. The results showed that human observers gave similar emotional ratings for natural images and for Portilla-Simoncelli synthesized images. The perceived surface properties such as glossiness, bumpiness, and translucency showed no or little correlation with the emotional ratings and were degraded in synthesized stimuli. The analysis of surface images revealed that the emotional ratings were related to a small set of low-level image statistics, such as excessive power at specific spatial frequencies and cross-orientation energy correlation. The results suggest the critical role of low-level image statistics in affective responses to natural surfaces and textures.