Innate immune memory (also termed trained immunity) is defined in part by its ability to cross-protect against heterologous pathogens, and can be generated by many different stimuli, suggesting a "universal" trained state. However, different stimuli could form distinct memories, leading to stimulus-specific trained responses. Here, we use primary human monocyte-derived macrophages to demonstrate phenotypic and epigenetic stimulus specificity of innate immune memory six days after initial exposure. Quantification of cytokine production with single-molecule RNA imaging demonstrates stimulus-specific patterns of response to restimulation at the single cell level. Differential licensing of inflammatory transcription factors is associated with encoding of specificities in chromatin. Trained cells show stronger responses to secondary stimuli that are more similar to the initial stimulus they experienced, suggesting a functional role for these stimulus-specific memories. Rather than activating a universal training state, our findings demonstrate that different stimuli impart specific memories that generate distinct training phenotypes in macrophages.