Despite the advantages of lithium niobate (LN)-based photonic integration platforms in various applications, the inherently weak third-order nonlinear optical response of conventional materials limits the miniaturization and energy efficiency of nonlinear optical devices in compact optical systems. Localized surface plasmons (LSPs) provide a promising solution to this miniaturization challenge by confining and enhancing light fields at deep subwavelength scales. However, due to nanofabrication limitations, strongly coupled single-crystal LN-LSP structures have yet to be realized. Here, it is demonstrated Au nanoparticle-LN hybrid plasmonic structures are assembled via plasmonic nanolithography. By leveraging plasmon-mediated energy deposition and photon momentum transfer under femtosecond laser irradiation, the nanoparticles within the single-crystalline region are formed from implanted elements. Nanoparticles are transformed into nanorods elongated along the polarization of the laser. With plasmons excited in distinct axial directions, the resulting nanorod-LN hybrid plasmonic material exhibits polarization-dependent nonlinearity, with the nonlinear absorption coefficient for long-axis polarized light augmented by five orders of magnitude compared to the pure LN. Utilizing this feature, a Q-switched laser is developed exhibiting pronounced polarization-dependent behavior. This research offers a viable approach for directly incorporating plasmonic nanostructures into crystals, enhancing and tuning optical properties, and thereby broadening the optical and photonic applications of LN-based platforms.