Outside acoustic communication, little is known about how animals coordinate social turn taking and how the brain drives engagement in these social interactions. Using Siamese fighting fish (Betta splendens), we discover dynamic visual features of an opponent and behavioral sequences that drive visually driven turn-taking aggressive behavior. Lesions of the telencephalon show that it is unnecessary for coordinating turn taking but is required for persistent participation in aggressive interactions. Circumscribed lesions of the caudal dorsomedial telencephalon (cDm
the fish pallial amygdala) recapitulated the telencephalic lesions. Furthermore, ventral telencephalic regions and the thalamic preglomerular complex, all of which project to cDm, show increased activity during aggressive interactions. Our work highlights how dynamic visual cues shape the rhythm of social interactions at multiple timescales. The results point to the vertebrate pallial amygdala as a region with an evolutionarily conserved role in regulating the persistence of emotional states, including those that promote engagement in social interactions.