To improve responsiveness in an engine restarting time by supplying fuel previously stored in a fuel high pressure storage container under a high pressure in an engine brake operation time to a fuel injection valve until a fuel pressure reaches a lower limit without driving any fuel pump when a standard operation is started again.

A high fuel pressure pump 3 is operated so as to pressurize fuel sucked from a fuel storage container 1 so as to forcibly feed it to a high fuel pressure storage container (accumulator) 6, and the accumulated high pressure fuel is jetted from a fuel injection valve 12 toward respective cylinders. In this process, a detection value of a pressure sensor 7 detecting a fuel pressure inside the high fuel pressure storage container 1 is compared with a target value by means of a controller 9, and according to the deviation between them, a pressure controller 10 is controlled. In an engine brake operation time, a pressure inside the high fuel pressure storage container 1 is regulated to an engine brake operation target value. If the fuel pressure exceeds a lower limit when a driving operation of the internal combustion engine is started again, the fuel inside the fuel high pressure storage container 1 is supplied to the fuel injection valve 12 while the fuel high pressure pump 3 is prevented from being driven.