PURPOSE: To avoid the possibility of the erroneous detection of a motion vector by defining a point taking the minimal value of the matching evaluation function in each hierarchical image as plural detection motion vector candidates.

CONSTITUTION: A low-pass filter processing and a sub-sampling are performed for a motion detection object block image 5, and the motion detection object block image 6 of the hierarchical level 1 which is reduced into a size of 8 picture elements × 8 picture elements and the motion detection object block image 7 of a hierarchical level 0 which is reduced into a size of 4 picture elements × 4 picture elements are generated. The hierarchical image 9 of the hierarchical level 1 and the hierarchical image 10 of the hierarchical level 0 are generated by the reference search area image 8. The motion vector detectors 11 to 13 of the hierarchical levels 0 to 2 detect motion vectors for every hierarchy from the highest order layer to the lowest order layer. All the points taking the minimal value are detected, detection motion vector candidates are outputted and they are use for the search initial location in each next hierarchical image.