Brain-computer interfaces (BCIs) based on electroencephalogram (EEG) enable direct interactions between the brain and external environments, with applications in medical rehabilitation, motor substitution, gaming, and entertainment. Traditional methods that model the non-Euclidean characteristics of EEG signals demonstrate robustness and high performance, but they suffer from significant computational costs and are typically restricted to a single BCI paradigm. This article addresses these limitations by utilizing a diffeomorphism from Riemannian manifolds to the Cholesky space, which simplifies the solution process and enables application across multiple BCI paradigms. Our proposed Cholesky space-based model, CSNet, achieves state-of-the-art (SOTA) performance in motor imagery (MI) decoding and emotion recognition and demonstrates competitive performance in error-related negativity (ERN) decoding, all without the need for data preprocessing. Furthermore, our runtime comparison shows that the Cholesky space method is more efficient than the method based on the Riemannian manifold as the matrix dimension increases. To enhance the interpretability of CSNet, we perform t-distributed stochastic neighbor embedding (t-SNE) visualization for MI, frequency band energy visualization for emotion recognition, and temporal importance visualization for ERN. The results indicate that CSNet effectively learns discriminative features, identifies important frequency bands, and focuses on important temporal features. The CSNet effectively captures the non-Euclidean characteristics of EEG signals across various BCI paradigms, while mitigating high computational costs, making it a promising candidate for future BCI algorithms. The code for this study is publicly available at: https://github.com/XingfuWang/CSNet.