PURPOSE: To always maintain a fuel pressure in a fuel passage at a proper value by a method wherein an auxiliary relief valve is mounted in a fuel passage connected to a pressure regulator, and when a pressure exceeds a set value, fuel is discharged to a fuel tank.

CONSTITUTION: When a fuel pump 12 is driven with the aid of a motor part 14, fuel in a fuel tank 11 is pumped up with the aid of a fuel pump part 13 and is fed with a pressure from a delivery port 16 through a fuel passage 18 to a fuel injection device 20. When a pressure in the fuel passage 18 exceeds a set value, the valve of a pressure regulator 21 is released, a part of fuel in the fuel passage 18 is returned to the fuel tank 11 through a return passage 19. In this case, when, under a state in which a check valve 17 is closed and an engine is brought into a stop, a pressure in the fuel passage 18 exceeds a set value, an auxiliary relief valve 23 is released. Fuel in the fuel passage 18 is fed through a branch passage 22 to the fuel tank 11.