To solve the problem that a substrate leakage current increases due to an interface level formed by plasma damage and thereby the image clarity of a CMOS image sensor declines in the case of omitting a high temperature heat treatment process.

For the CMOS image sensor, an epitaxial wafer is used as an element substrate. After forming a contact hole connecting an element in the element substrate and wiring, a tungsten layer is formed, the tungsten layer at a part other than the contact hole is removed and then annealing in a nitrogen and hydrogen atmosphere or annealing in a hydrogen atmosphere is conducted.