To investigate the adhesion properties of asphalt binder-aggregate interfaces, contact angle tests were performed on base, aged, and SBS# asphalt with limestone and basalt aggregates. A molecular dynamics simulation model was established to analyze interfacial adhesion characteristics. The results indicate good consistency between the experimental and molecular dynamics simulation findings. SBS# asphalt exhibited superior surface free energy and adhesion properties compared with base asphalt, whereas aging reduced the adhesion performance. The interfacial energy between the asphalt and aggregates is closely related to their molecular compositions. When moisture penetrates the asphalt mixture, the interfacial energy between limestone and asphalt significantly increases, compared with that of basalt, with SBS# asphalt exhibiting stronger adhesion than base asphalt. The modifier enhanced the adhesion between the asphalt and aggregates, thereby providing resistance to moisture damage. The findings of this study possess referential value for the selection, modification, and performance optimization of asphalt pavement materials.