To save resources, preserve the environment and suppress the cost by heat-treating an Al dross residual ash, then screening the heat-treated Al dross residual ash, providing an Al dross residual ash having a specific value or below of particle diameter and using the resultant Al dross residual ash as a refractory raw material.

An Al dross residual ash is heat-treated to thereby remove nitrogen components and chlorine components remaining in the residual ash. The Al dross residual ash is thereby chemically stabilized and detoxicated. The heat-treating temperature therefor is preferably within the range of 1,300-1,600°C. The heat-treated Al dross residual ash is pulverized and screened to provide an Al dross residual ash having ≤74 μm particle diameter, which is used as a refractory raw material. The resultant refractory raw material has physical properties nearly comparable to those of a chamotte, a baked bauxite and a sintered alumina which are natural materials. The heat-treated Al dross residual ash can be pulverized and then heat-treated again to thereby carry out further stabilization of the physical properties and detoxication.