This study investigated the effects of caffeine in coffee extract (CE) prepared under various roasting and brewing conditions on non-volatile characteristics (antioxidant activity, caffeine contents, chlorogenic acid contents, and taste patterns) and volatile compounds. The representative taste of coffee is bitter taste, and it was mainly influenced by roasting processes compared with the contents of caffeine. Non-volatile compounds were influenced by the bean types (regular and decaffeinated beans) and brewing methods, whereas volatile compounds were mainly affected by the brewing methods and roasting processes. In chemometrics approaches, the correlation between caffeine and bitterness (-0.003) and sourness (0.549) was identified. The correlation between chlorogenic acid and bitterness (0.399) and sourness (0.693) was identified. This study is expected to serve as a basis for understanding changes in bitterness and the generation of volatile compounds according to the contents of caffeine.