PURPOSE: To provide a scale eliminating device suitable for eliminating scale adhering on a heat treated roll surface.

CONSTITUTION: Scales produced on the surface of a heat treated roll 4, which is rotationally cooled by a pair of turning rollers 3, 3, are eliminated by using a chipping hammer 1. A scale eliminating device consists of rails 11 arranged parallelly with the tuning rollers 3, 3, a frame 13 mounted on the rails 11, a moving cylinder freely moving the frame 13 on the rails 11, and a pressing cylinder 15, which is installed on the frame 13 and holds the chipping hammer 1 at the preset inclination angle freely movably forward/backward. In this way, the scales produced on the surface of the roll 4 can be automatically eliminated.