To provide an image processing apparatus capable of suppressing reduction in the encoding efficiency.

A decoded pixel classifier 152 and an inputted pixel classifier 153 classify each macro block at a decoded image or an inputted image to a class, based on an orthogonal transformation size read from an orthogonal transformation size buffer 151. A 4×4-block coefficient calculator 154 and an 8×8-block coefficient calculator 155 calculate a filter coefficient so that residual error may be at a minimum in each orthogonal transformation block. A pixel classifier 161 at a loop filter 113 classifies each macro block at the decoded image to a class for every orthogonal transformation block size. A filter (4×4) 162 and a filter (8×8) 163 apply a suitable filter coefficient to an orthogonal transformation block associated with each of the filters and perform filter processing. This invention is applicable to an image processing apparatus, for example.