INTRODUCTION: The lack of a standardized evaluation method for cleft palate speech makes it difficult to exchange information at the domestic and global levels. We aimed to develop and validate a Japanese outcome tool for the perceptual assessment of speech in patients with cleft palate. METHODS: The tool we developed was based on the Cleft Audit Protocol for Speech-Augmented (CAPS-A) with modifications to some speech parameters. We developed Japanese speech samples and parameters for the Japanese Cleft Speech Outcome Tool (CAPS-A-Japan: CAPS-A-JP). This study was conducted in the Japancleft speech working group at the Japanese Cleft Palate Association. In Phase 1, we constructed and developed the speech sampling protocol. In Phase 2, After the revision of the definitions and criteria for several cleft speech characteristics and the speech sample, a total of 20 audio recordings were tested. In Phase 3, criterion validity was assessed by comparing the CAPS-A-JP resonance outcomes reported for 40 cases with the outcomes of the clinical investigations. In Phase 4, a listening experiment with six Japanese speech-language-hearing therapists was conducted twice to test the intra- and inter-rater reliabilities for 20 cases. Intra-class correlation coefficients (ICCs) were used to analyze reliability. RESULTS: The mean percentage agreement on criterion validity for resonance was 71% on high-pressure sentences but 54% on low-pressure sentences. The inter-rater reliability was rated moderate/good (ICC: 0.45-0.80), except for developmental immaturities/errors that were rated fair. The ICC was very low or incalculable for hyponasality and syllable error patterns because there were few cases of hyponasality and syllable errors. The intra-rater reliability was rated as good/very good. The ICC was very low or incalculable in hyponasality, nasal emission, and syllable error patterns. CONCLUSIONS: The standardized outcome tool, CAPS-A-JP, was developed for the first time in Japan. Future research is required to optimize this tool based on the inclusion of a training protocol.