Effective connectivity can describe the causal patterns among brain regions. These patterns have the potential to reveal the pathological mechanism and promote early diagnosis and effective drug development for cognitive disease. However, the current methods utilize software toolkits to extract empirical features from brain imaging to estimate effective connectivity. These methods heavily rely on manual parameter settings and may result in large errors during effective connectivity estimation. In this paper, a novel brain imaging-to-graph generation (BIGG) framework is proposed to map functional magnetic resonance imaging (fMRI) into effective connectivity for mild cognitive impairment (MCI) analysis. The proposed BIGG framework is based on the diffusion denoising probabilistic models (DDPM), where each denoising step is modeled as a generative adversarial network (GAN) to progressively translate the noise and conditional fMRI to effective connectivity. By introducing the diffusive factor, the denoising inference with a large sampling step size is more efficient and can maintain high-quality results. Evaluations of the ADNI dataset demonstrate the feasibility and efficacy of the proposed model. The proposed model not only achieves superior prediction performance compared with other competing methods but also predicts MCI-related causal connections that are consistent with clinical studies.