Advances in methods of biological data collection are driving the rapid growth of comprehensive datasets across clinical and research settings. These datasets provide the opportunity to monitor biological systems in greater depth and at finer time steps than was achievable in the past. Classically, biomarkers are used to represent and track key aspects of a biological system. Biomarkers retain utility even with the availability of large datasets, since monitoring and interpreting changes in a vast number of molecules remains impractical. However, given the large number of molecules in these datasets, a major challenge is identifying the best biomarkers for a particular setting. Here, we apply principles of observability theory to establish a general methodology for biomarker selection. We demonstrate that observability measures effectively identify biologically meaningful sensors in a range of time series transcriptomics data. Motivated by the practical considerations of biological systems, we introduce the method of dynamic sensor selection (DSS) to maximize observability over time, thus enabling observability over regimes where system dynamics themselves are subject to change. This observability framework is flexible, capable of modeling gene expression dynamics and using auxiliary data, including chromosome conformation, to select biomarkers. Additionally, we demonstrate the applicability of this approach beyond genomics by evaluating the observability of neural activity. These applications demonstrate the utility of observability-guided biomarker selection for across a wide range of biological systems, from agriculture and biomanufacturing to neural applications and beyond.