Layer-specific brain computations depend on neurons synapsing with specific partners in distinct laminae. In the Drosophila lobula plate, axons of the four subtypes of T4 and T5 visual motion direction-selective neurons segregate into four layers, where they synapse with distinct subsets of postsynaptic neurons. Here, we identify a layer-specific expression of different receptor-ligand pairs of the Beat and Side families of cell adhesion molecules between T4/T5s and their postsynaptic partners. Developmental genetic analysis demonstrate that Beat/Side-mediated interactions are required to restrict innervation of T4/T5 axons and the dendrites of their partners to a single layer. We show that Beat/Side interactions are not required for synaptogenesis. Instead, they contribute to synaptic specificity by biasing cellular adjacency, causing neurons to segregate in discrete layers, restricting partner availability before synaptogenesis. We propose that the emergence of synaptic specificity relies on a competitive dynamic among postsynaptic partners with shared Beat/Side expression to adhere with T4/T5s.