To attain the smooth gradation interpolation by checking the inclusive relation of border lines and extracting easily a border line set that defines a border between an object area of gradation interpolation and other areas.

Border lines L1 to L3, showing the borders of areas R1 to R3, are detected from a multi-gradation image. The area R2 that has a level A and is adjacent to an area of a level B is selected as an object area of gradation interpolation, and a border line set (L2, L3) is extracted to show the border between the area R2 and other areas. Thus, the border lines of the multi- gradation image are sorted into plural types (A+, A-, B+, B-), and the inclusive relation is decided among the border lines. As a result, at least a pair of border line sets is extracted to show the border between an object area and other areas. When the intermediate image levels M1 to MN are interpolated between the levels A and B, N pieces of intermediate image levels are assigned in the order of higher levels in (N+1) pieces of segment areas. Thus, a smooth image is obtained.