Differences in the brain sensitivity to color responses may cause significant differences in the latency and amplitude of the electroencephalographic (EEG) component. This paper investigated the electroencephalography features of binocular color fusion and binocular color rivalry when watching stereoscopic three-dimensional (3D) displays. EEG experiments were conducted on a conventional 3D display platform. Eight subjects were involved to analyze differences in the event-related potential (ERP) and power spectrum when the brain perceived binocular color fusion and binocular color rivalry. Results show that: 1) the latencies of ERP components N1 and P2 of binocular color fusion were shorter than that of binocular color rivalry, 2) the amplitudes of the ERP components P2 and P3 of binocular color fusion were greater than that that of color rivalry, and 3) the left hemisphere was dominant for binocular color rivalry while the right hemisphere was greater involved in binocular color fusion. These results indicate that during the initial and mid-term cognitive processing, the brain response to binocular color fusion is faster than binocular color rivalry. Both binocular color fusion and rivalry involve visual post-processing, but binocular color fusion requires a greater allocation of neural resources. Power spectrum analysis revealed the cerebral lateralization in binocular color fusion and rivalry, it suggested that the way the brain processes this binocular input can have effects on its function.