This paper investigates the correlation amongst transparency, accountability, corruption, and public administration performance in Vietnam using data from the Vietnam Provincial Governance and Public Administration Performance Index survey in 2012. The Generalised Canonical Analysis is applied to evaluate the meaning of the "Don't Know" answer which often exists in response to questions on perception of corruption. The results reveal that "Don't Know" implies corruption. The paper then, shows that a high level of transparency is accompanied with a low level of perception of corruption while impacts of accountability on corruption are mixed. Furthermore, corruption is a critical factor that deteriorates the administration performance whereas transparency and accountability are ineffective in being translated into the quality of administrative services. The results raise the need to closely examine the de-facto forms of transparency and accountability as well as the political will in the fight against corruption to improve the quality of public administrative services in Vietnam.