In table tennis, there are various movements involved in hitting a ball, which are called strokes, and these are an important factor in determining the contents of a game. Therefore, research has been conducted to classify these types of strokes using video gameplay data or inertial sensor information. However, the classification of strokes from actual videos of table tennis is more difficult than general action recognition tasks because many strokes display strong similarity. Therefore, this study proposes a multi-label stroke classification method, assigning multiple classes per stroke. Specifically, multi-labeling is performed by assigning two types of labels-namely the player's posture and the rotation and velocity of the ball-to one stroke. By changing the head of the action recognition model to adopt multiple outputs for stroke classification, the difficulty in each classification task is reduced and the accuracy is improved. As a result, when performing multi-labeling classification with a conventional action recognition model, the accuracy of the validation data was improved by up to 8.6%, and the accuracy of the test data was improved by up to 18.1%. In addition, when two types of input-namely video and 3D joint coordinates-were used, the accuracy of the validation and test data was higher by 17.1 and 5.4% for 3D joint coordinates, respectively, confirming that 3D joint coordinates are effective.