Generative models have diverse applications, including language processing and birdsong analysis. In this study, we demonstrate how a statistical test, designed to prevent overgeneralization in sequence generation, can be used to infer minimal models for the syllable sequences in Bengalese finch songs. We focus on the partially observable Markov model (POMM), which consists of states and the probabilistic transitions between them. Each state is associated with a specific syllable, with the possibility that multiple states may correspond to the same syllable. This characteristic differentiates the POMM from a standard Markov model, where each syllable is linked to a single state. The presence of multiple states for a syllable suggests that transitions between syllables are influenced by the specific contexts in which these transitions occur. We apply this method to analyze the songs of six adult male Bengalese finches, both before and after they were deafened. Our results indicate that auditory feedback plays a crucial role in shaping the context-dependent syllable transitions characteristic of Bengalese finch songs.