Climate change-related risk mitigation is typically addressed using cost-benefit analysis that evaluates mitigation strategies against a wide range of simulated scenarios and identifies a static policy to be implemented, without considering future observations. Due to the substantial uncertainties inherent in climate projections, this identified policy will likely be sub-optimal with respect to the actual climate trajectory that evolves in time. In this work, we thus formulate climate risk management as a dynamic decision-making problem based on Markov Decision Processes (MDPs) and Partially Observable MDPs (POMDPs), taking real-time data into account for evaluating the evolving conditions and related model uncertainties, in order to select the best possible life-cycle actions in time, with global optimality guarantees for the formulated optimization problem. The framework is developed for coastal adaptation applications, considering a wide variety of possible action types, including various forms of nature-based infrastructure. Related environmental impacts of carbon emissions and uptake are also incorporated, and social cost of carbon implications are discussed, together with several future directions and supported features.