To obtain an endoscope image pickup device constituted so that a suitable color matrix is freely switched according to the type of an observing part and the type of a using light source and excellent color reproducibility corresponding to the type of the observing part and the type of the using light source is obtained.

A single plate type color CCD is used as an image pickup means. Based on an instruction by the operation of a change-over switch 34, the coefficient of the corresponding color matrix is read out from a ROM 35 by a CPU 31. Then, the coefficient is outputted to a color separation circuit 21. By the separation circuit 21, matrix arithmetic operation based on the above color matrix is executed and a luminance and color difference signal is converted into primary color signals.