Nowadays, driver-assistance system is much considered by many automotive companies for manufacturing moderns’ cars. In that system, traffic sign detection and recognition algorithm is a challenge problem that many researchers try to solve. This paper presents a design and implementation of the portable real-time traffic sign detection and recognition (TSDR) to help drivers notify the traffic signs in the streets. The main features of the TSDR system are real-time processing capability and high accuracy. To achieve these targets, a fusion method which is combination of advanced techniques including adaptive chromatic color segmentation, shape matching, and support vector machine (SVM) is proposed. Besides, a multi-threading programming technique is applied to enhance the real-time processing capability of the system. The TSDR system is implemented on a portable embedded system board with ARM Cortex- A9 processor. The TSDR system has been tested on the streets of Ho Chi Minh city. The experiments show that the proposed system can detect and recognize the traffic signs with accuracy of 93% at 15 frames per second.