Neural Machine Translation (NMT) is a new paradigm in machine translation (MT) powered by recent advances in sequence to sequence learning frameworks. With the advance of Neural Networks, NMT has become the most promising MT approach in recent years. Despite the apparent success, NMT still suffers from one significant drawback in integrating syntactic knowledge into neural networks. This paper proposes an extension of the NMT model to incorporate additional syntactic information from constituency trees. We represent the constituency trees under graph forms encoded by a graph encoder to enhance the attention layer, which allows the decoder to focus on both sequential and graph representation at each decoding step. The experiments show promising results of the proposed method on English-Vietnamese datasets, proving the effectiveness of our syntax-enhanced NMT method.