In principle, we can use many difierent characteristics of a probability distribution. However, in practice, a few of such characteristics are mostly used: mean, variance, moments, correlation, etc. Why these characteristics and not others? The fact that these characteristics have been successfully used indicates that there must be some reason for their selection. In this paper, we show that the selection of these characteristics can be explained by the fact that these characteristics are invariant with respect to natural transformations - while other possible characteristics are not invariant.In principle, we can use many difierent characteristics of a probability distribution. However, in practice, a few of such characteristics are mostly used: mean, variance, moments, correlation, etc. Why these characteristics and not others? The fact that these characteristics have been successfully used indicates that there must be some reason for their selection. In this paper, we show that the selection of these characteristics can be explained by the fact that these characteristics are invariant with respect to natural transformations - while other possible characteristics are not invariant.