Memory effects are ubiquitous in small-scale systems. They emerge from interactions between accessible and inaccessible degrees of freedom and give rise to evolution equations that are nonlocal in time. If the characteristic timescales of accessible and inaccessible degrees of freedom are sharply separated, locality can be restored through the standard Markov approximation. Here, we show that this approach can be rigorously extended to a well-defined weak-memory regime, where the relevant timescales can be of comparable order of magnitude. We derive explicit bounds on the error of the local approximation and a convergent perturbation scheme for its construction. Being applicable to any nonlocal time evolution equation that is autonomous and linear in the variables of interest, our theory provides a unifying framework for the systematic description of memory effects.