Cohort studies consistently show that a high intake of cereal fiber and whole-grain products is associated with a decreased risk of type 2 diabetes (T2DM), cancer, and cardiovascular diseases. Similar findings are also reported for infectious and chronic inflammatory disorders. All these disorders are at least partially caused by inflammaging, a chronic state of inflammation associated with aging and Metabolic Syndrome. Surprisingly, insoluble (cereal) fiber intake consistently shows stronger protective associations with most long-term health outcomes than soluble fiber. Most humans consume soluble fiber mainly from sweet fruits, which usually come with high levels of sugar, counteracting the potentially beneficial effects of fiber. In both observational and interventional studies, high-fiber diets show a beneficial impact on inflammation, which can be attributed to a variety of nutrients apart from dietary fiber. These confounders need to be considered when evaluating the effects of fiber as part of complex dietary patterns. When assessing specific types of fiber, inulin and resistant starch clearly elicit anti-inflammatory short-term effects, while results for pectins, beta-glucans, or psyllium turn out to be less convincing. For insoluble fiber, promising but sparse data have been published so far. Hypotheses on putative mechanisms of anti-inflammatory fiber effects include a direct impact on immune cells (e.g., for pectin), fermentation to pleiotropic short-chain fatty acids (for fermentable fiber only), modulation of the gut microbiome towards higher levels of diversity, changes in bile acid metabolism, a differential release of gut hormones (such as the glucose-dependent insulinotropic peptide (GIP)), and an improvement of insulin resistance via the mTOR/S6K1 signaling cascade. Moreover, the contribution of phytate-mediated antioxidative and immune-modulatory means of action needs to be considered. In this review, we summarize the present knowledge on the impact of fiber-rich diets and dietary fiber on the human inflammatory system. However, given the huge heterogeneity of study designs, cohorts, interventions, and outcomes, definite conclusions on which fiber to recommend to whom cannot yet be drawn.