Diabetes is one of the most common non-communicable diseases in both developed and underdeveloped countries with a 9.3% prevalence. Unhealthy diets and sedentary lifestyles are among the most common reasons for type 2 diabetes mellitus (T2DM). Diet plays a crucial role in both the etiology and treatment of T2DM. There are several recommendations regarding the carbohydrate intake of patients with T2DM. One of them is about reducing the total carbohydrate intake and/or changing the type of carbohydrate to reduce the glycaemic index. Cereals are good sources of carbohydrates in the diet with a significant amount of soluble and non-soluble fiber content. Apart from fiber, it has been shown that the bioactive compounds present in cereals such as proteins, phenolic compounds, carotenoids, and tocols have beneficial impacts in the prevention and treatment of T2DM. Moreover, cereal by-products especially the by-products of milling processes, which are bran and germ, have been reported to have anti-diabetic activities mainly because of their fiber and polyphenols content. Considering the potential functions of cereals in patients with T2DM, this review focuses on the roles of cereal bioactive compounds in the prevention and treatment of type 2 diabetes.