To obtain a pile fabric useful as a carpet and a sheet fabric for automobile, excellent in cushioning properties and shape stability free from removal of pile yarns, excellent in handle, by mixing a nonelastic polyester-based crimped short fiber with a specific elastic conjugate short fiber, fusing and heat-treating the mixed fiber to give a spun yarn and constituting a woven fabric as a base fabric from the spun yarn.

(A) A nonelastic polyester based crimped short fiber is mixed with (B) an elastic conjugate short fiber composed of a thermoplastic polyester elastomer (a melting point 40-110°C lower than that of the component A) at least exposed to the surface of fiber and a nonelastic polyester to give a spun yarn, which is heat-treated to melt only the thermoplastic elastomer to form flexible heat-set points. A woven fabric made of the spurn yarn is used as a base fabric to form flexible thermally fixed points between pile yarn planted in the woven fabric and the thermoplastic elastomer by heat fusion.