To obtain a polyester fiber having excellent raising property when producing napped fabric and suitable for producing napped fabric having both of napping property and soft feeling and excellent in flame retardance and a napped fabric using the fiber.

This polyester fiber is mainly composed of a polyester and simultaneously satisfies the following characteristics (1): 500≤P≤50,000; (2): 2%≤0.2≤30%; (3): ISa/ISb≥0.9 [wherein P is phosphorus content (ppm); 0.2 is integrated parameter of a structure; ISa is an initial tensile modulus (cN/dtex) after boiling water/dry heat treatment; ISb is an initial tensile modulus (cN/dtex) before boiling water/dry heat treatment].