The study of dense degenerate plasmas at extreme temperatures and densities motivates the investigation of relativistic and ultrarelativistic regimes. In this paper, we are presenting a comprehensive study of the linear and nonlinear propagation characteristics for kinetic Alfven acoustic waves (KAAW) at relativistic and ultra relativistic Fermi energies with small temperature corrections. The linear wave propagation of fast and slow modes for relativistic KAAW are studied and a comparative analysis of both the modes is plotted for relativistic, ultrarelativistic and nonrelativistic regimes. It has been investigated that for constant magnetic fields, the phase velocity of these waves decreases due to an increase in both the specific heat and thermal pressure at low plasma beta values and KAAW propagate more efficiently in outer regions with lower specific heat than in denser inner regions of neutron stars. Various theoretical approaches like Sagdeev pseudopotential approach, dynamical analysis, and finite amplitude expansion method are employed to analyse the nonlinear relativistic and ultrarelativistic KAAW. Phase space trajectories have been plotted to study the nonlinear structures corresponding to arbitrary amplitude perturbations. The existence regions of solitary kinetic Alfven acoustic waves (SKAAW) and the physical reasons for their dependence on different parameters have also been shown. Possible applications are envisaged in the intense laser-matter interactions, plasma photonics, dense astrophysical objects like the interior state of terrestrial planets, neutron stars, and white dwarfs where relativistic effects dominate the microscopic scale and may have macroscopic consequences.