To shorten time for encoding and decoding, and to perform motion compensation sufficiently in consideration of a shadow part of an image by encoding and decoding a color image without using DCT.

An encoding device 1 for a moving picture consisting of multicolor images is provided with a thinning-out means 2c which thins out at least UV components among YUV components where RGB components are converted, a quantizing means 2d which quantizes the Y component which is not thinned out or slightly thinned out, and an encoding means which performs encoding by compressing the UV components that are largely thinned out and the quantized Y component. When motion compensation for the moving picture is performed, motion compensation is performed for the UV components and not performed for the Y component.