To constantly maintain fuel pressure by continuously connecting pressure chambers of pressure accumulating units, providing one of pressure accumulating units of a system with a valve and connecting the pressure chamber of the pressure accumulating unit and other pressure chambers connected with the pressure chamber at the same time with a fuel tank via the valve.

Pressure accumulating units 12 are communicated with each other via a pipe 13 for reducing a pressure difference which can be generated between the pressure accumulating units 12. One pressure accumulating unit 12 within a system is provided with an auxiliary valve 16 controlled by a front control valve 17 arranged in a servo oil circuit of an engine. The pressure accumulating unit 12 of the system is connected with a fuel tank 1 through a pipe line 19 via the auxiliary valve 16. Therefore, the pressure accumulating unit 12 can be depressurized via the auxiliary valve 16. In addition to this, the pressure accumulating unit 12 can be circulated to heat fuel before the engine is started, for instance. It required, a control valve 6 is alternately opened and closed and fuel can be surely allowed to flow to all the pressure accumulating units 12 via the auxiliary valve 16.