In this study, we present a comprehensive pipeline to train and compare a broad spectrum of machine learning and deep learning brain clocks, integrating diverse preprocessing strategies and correction terms. Our analysis also includes established methodologies which have shown success in prior UK Biobank-related studies. For our analysis we used T1-weighted MRI scans and processed de novo all images via FastSurfer, transforming them into a conformed space for deep learning and extracting image-derived phenotypes for our machine learning approaches. We rigorously evaluated these approaches both as robust age predictors for healthy individuals and as potential biomarkers for various neurodegenerative conditions, leveraging data from the UK Biobank, ADNI, and NACC datasets. To this end we designed a statistical framework to assess age prediction performance, the robustness of the prediction across cohort variability (database, machine type and ethnicity) and its potential as a biomarker for neurodegenerative conditions. Results demonstrate that highly accurate brain age models, typically utilising penalised linear machine learning models adjusted with Zhang's methodology, with mean absolute errors under 1 year in external validation, can be achieved while maintaining consistent prediction performance across different age brackets and subgroups (e.g., ethnicity and MRI machine/manufacturer). Additionally, these models show strong potential as biomarkers for neurodegenerative conditions, such as dementia, where brain age prediction achieved an AUROC of up to 0.90 in distinguishing healthy individuals from those with dementia.