To perform fuel pressure control at the injection valve side for improv ing reliability by relieving fuel discharged from the fuel pump to the intake side using a normally closed solenoid valve.

A fuel within a fuel tank 1 is discharged from a low pressure pump 2 to a low pressure pipe 3. The fuel is further discharged from a high pressure pump 8 to a high pressure pipe 39 such that it is injected to the respective cylinders of an engine through injection valves 40 of a common rail 38. The high pressure pump 8 is provided with a plurality of cylinders 18 and a fuel pressure control valve 28. By opening the fuel pressure control valve 28 from the intermediate stage to the end of the discharge stroke of the high pressure pump 8 by means of a control unit 41, the fuel in each cylinder 18 is relieved from a relief oil passage 25 to the low pressure pipe 3 on the later side of the discharge stroke. Then the pressure of the fuel injected through the injection valves 40 can be variably controlled.