We study estimation and inference on causal parameters under finely stratified rerandomization designs, which use baseline covariates to match units into groups (e.g. matched pairs), then rerandomize within-group treatment assignments until a balance criterion is satisfied. We show that finely stratified rerandomization does partially linear regression adjustment by design, providing nonparametric control over the stratified covariates and linear control over the rerandomized covariates. We introduce several new forms of rerandomization, allowing for imbalance metrics based on nonlinear estimators, and proposing a minimax scheme that minimizes the computational cost of rerandomization subject to a bound on estimation error. While the asymptotic distribution of GMM estimators under stratified rerandomization is generically non-normal, we show how to restore asymptotic normality using ex-post linear adjustment tailored to the stratification. We derive new variance bounds that enable conservative inference on finite population causal parameters, and provide asymptotically exact inference on their superpopulation counterparts.