The human-animal relationship is an important component of farm-animal welfare. It is often assessed using 'avoidance distance', i.e. the distance at which animals avoid humans approaching them. However, this avoidance distance may be influenced by factors other than the human-animal relationship such as genetic background or health status. The objective of this cross-sectional epidemiological study was to assess the size of the effect of animal factors and testing conditions on the results of an avoidance test in dairy cows. Five observers applied a standardised test of avoidance distance at the feeding rack on 2 306 lactating cows from 103 dairy farms. We collected data on the independent variables: breed, parity and lameness, and height at withers, and calculated the height difference between the observer standing on the walking alley and each cow restrained at the feeding table. The avoidance distance of the cows, expressed in cm, was modelled using a linear mixed-effect model, with the above-mentioned independent variables as fixed factors, and the farm nested in observer as a random factor. The avoidance distance expressed in four categories as in the Welfare Quality protocol (2009) was modelled using a mixed effects ordinal regression with the same fixed and random factors. For the avoidance distance expressed in cm, we observed that intra-farm variability was twice as large as inter-farm variability (SD: 33.9 cm intra-farm vs 16.7 cm inter-farm). Avoidance distance increased by 7.0 cm (95%CI [3.8-10.2]) when the cow was multiparous and increased by 4 cm (95%CI [0.2-0.6]) when the height difference between observer and cow increased by 10 cm. Avoidance distance decreased by 9.9 cm (95%CI [-19.6 - -0.3]) when the cow was lame and decreased by 6.4 cm (95%CI [-23.4 - -0.4]) when the cow breed was Montbéliarde vs Holstein. For the avoidance distance expressed in categories, there was a significant effect of the height difference and the parity. A greater height difference as well as multiparous cows had a slightly increased likelihood of reaching a higher avoidance distance category. We conclude that avoidance-test results are influenced by cow-related factors (parity, height difference to experimenter, lameness status, and breed). Thus, to reliably use the avoidance distance test to assess the human-animal relationship at farm level, we recommend careful sampling of cows for testing based on these factors. The measured avoidance distance can also be corrected with a coefficient based on animal-specific factors and height difference between observer and cows.