During the active transport of cellular cargo, forces generated by cargo-associated molecular motors propel the cargo along cytoskeletal tracks. However, the forces impact not only the cargo, but also the underlying cytoskeletal filaments. To better understand the interplay between cargo transport and the organization of cytoskeletal filaments, we employ coarse-grained computer simulations to study actin filaments interacting with cargo-anchored myosin motors in a confined domain. We show that cargo transport can lead to the segregation of filaments into domains of preferred filament polarity separated by clusters of aggregated cargoes. The formation of polarity-sorted filament domains is enhanced by larger numbers of cargoes, more motors per cargo, and longer filaments. Analysis of individual trajectories reveals dynamic and heterogeneous behavior, including locally stable aggregates of cargoes that undergo rapid coalescence into larger clusters when sufficiently close. Our results provide insight into the impact of motor-driven organelle transport on actin filaments, which is relevant both in cells and in synthetic environments.