To detect the movement with small arithmetic quantity and with high accuracy in an image encoding mode and to deal with detection of the movement in the encoding order.

When the back movement of a B1 frame of (b) is detected, an offset vector 8 whose size is changed according to the ratio between the I0-P3 frame space and the P3-B1 frame space is calculated based on the motion vector 5 of P3 that is calculated in (a). Then the area of a fixed range that is needed when the frame space is equal to 1 is defined as a search area 9 around the position that is moved by an extent equal to the vector 8 from a position 7 of the P3 frame corresponding to the position of a searched image 6 of the B1 frame. Then an image that has the highest correlation with the image 6 is searched out of the area 9. Thus the back motion vector of the image 6 is detected.