High-dimensional quantum gates not only extend the existing quantum computing framework but also serve as a vital component in a range of quantum technologies. In the study, a 4 × 4-dimensional controlled-NOT (CNOT) gate is presented based on the assistance of a weak cross-Kerr medium, utilizing only two degrees of freedom (DoFs) with two photons. Specifically, the control qudits are encoded in the photonic polarization DoF, while the target qudits are encoded in the path DoF of the two photons. Notably, the circuit is constructed using only two times of the Kerr medium, outperforming a previous design that required six times, which simplifies the entire circuits. Furthermore, the fidelity and success probability of the proposed protocol are near 1 when the influence factors take the appropriate value.