In this paper, the concepts of designing the automatic classification system are described. This system classifies and evaluates based on parameters of external characteristics of mangoes as well as their weights. External features are mentioned such as the length, width, and height of each mango, the defect, and the color of the mango’s exocarp. The recognition of parameters is based on vision machine and machine learning techniques to evaluate the appropriate quality of mangoes. The automatic mango classification system is designed for moving mangoes through grading systems, measuring, collecting image data, weighing, and combining the data for grading. The system classifying mangoes by color, size, and weight is designed in modular form to be convenient for moving, assembling, and operating. After the design, the system was developed and evaluated through an experimental process. The system meets the capacity requirement of classification 3 tons/hour. The results of the system classification are compared with the manual sorting and evaluation, which shows that the classification results by the system are more effective and efficient than the manual one. Besides, the system can also attach a labeling module to be able to participate in the blockchain to identify and enhance the value of local mangoes, namely Dong Thap province.