Effective exploration and analysis tools are vital for the extraction of insights from single-cell data. However, current techniques for modeling single-cell studies performed across experimental conditions (e.g., samples) require restrictive assumptions or do not adequately deconvolute condition-to-condition variation from cell-to-cell variation. Here, we report that reduction and insight in single-cell exploration (RISE), an adaptation of the tensor decomposition method PARAFAC2, enables the dimensionality reduction and analysis of single-cell data across conditions. We demonstrate the benefits of RISE across distinct examples of single-cell RNA-sequencing experiments of peripheral immune cells: pharmacologic drug perturbations and systemic lupus erythematosus patient samples. RISE enables associations of gene variation patterns with patients or perturbations while connecting each coordinated change to single cells without requiring cell-type annotations. The theoretical grounding of RISE suggests a unified framework for many single-cell data modeling tasks while providing an intuitive dimensionality reduction approach for multi-sample single-cell studies across biological contexts. A record of this paper's transparent peer review process is included in the supplemental information.