Stimuli-responsive circularly polarized luminescence (CPL) metal clusters hold significant potential in high-security encryption and sensing applications, yet the exploration of hydrogen-bond-driven CPL-active metal clusters remains limited. Here, we report the synthesis of an enantiomeric pair of rhomboid Au4 clusters utilizing chiral R/S-4-hydroxymethyl-5-methyloxazole-2-thione (R/S-HMMT) ligands. Two enantiomeric pairs of self-assembled metal clusters R/S-Au4(HMMT)4-blue and R/S-Au4(HMMT)4-red were obtained, by constructing distinct intercluster hydrogen bonds through the use of different crystalline solvents. In R/S-Au4(HMMT)4-blue, 1,4-dioxane guest molecules were observed to form a hydrogen-bond network with the hydroxyl groups of the cluster surface ligands. In contrast, a different hydrogen-bond network involving the hydroxyl groups of the surface ligands was identified in R/S-Au4(HMMT)4-red, resulting in a distinct stacking pattern. The unique intercluster couplings mediated by hydrogen bonds result in R/S-Au4(HMMT)4-blue exhibiting a blue CPL emission at 466 nm, while R/S-Au4(HMMT)4-red shows a dual CPL emission at 446 and 727 nm. Theoretical calculations reveal that hydrogen-bond driven intercluster couplings in R-Au4(HMMT)4-red are significantly stronger than in R-Au4(HMMT)4-blue. Additionally, both solid R/S-Au4(HMMT)4-blue and R/S-Au4(HMMT)4-red undergo reversible CPL transformations in response to organic vapors, temperature, or mechanical stimuli, due to the destruction and reconstruction of hydrogen-bond networks. These characteristics make them promising materials for information encryption applications.