Droplet motion can occur due to interaction with the surrounding vapor phase. We examined experimentally the motion of two adjacent droplets, either pure liquid or a binary mixture, without direct contact. A droplet is repelled or attracted by the (pinned) adjacent droplet, which acts as a vapor source, depending on its initial concentration as well as the composition in the vapor, even for a pure liquid. The observation is explained by a theoretical model that combines evaporation and adsorption processes, which unifies the mechanism for both directions of motion (attraction and repulsion) and, more importantly, for both binary mixtures and pure liquid droplets. Good agreement is achieved between the theoretical model and the experimental observations. A critical concentration is proposed to determine the transition between attractive and repulsive motion, this being a criterion to predict droplet motion.