To reduce the volume of boosting fuel so as to inject fuel with high pressure instantly even in the case of a low speed rotating condition when an engine is started by directly supplying pressurized fuel to an injector by a booster when the engine is started.

When an engine is started, an operating circuit 35 is connected to an AP, and simultaneously a return circuit 42 is connected to a BR. Fuel is forcibly supplied from a low pressure pump 5 is supplied to the operating chamber 39 of a booster 33, and fuel in a return chamber 41 is returned to a fuel tank 3, and thereby, a piston 47 is moved to a right side in the booster 33, fuel of a boosting circuit 27 is pressurized, and its fuel is directly supplied to an injector 17a. Return of fuel from a first injector 17a to a common rail 9 is blocked by a third check valve 32. Boosted fuel by a booster 33 is supplied to the common rail 9 without supplying to the common rail 9, and thereby, high pressure fuel is directly injected from the injector 17a.