PURPOSE: To attain high reliability in a parallel computer by restarting program execution from an optional check point.

CONSTITUTION: An access history from respective processing elements 11-1 to 11-n to a shared memory 13 is recorded by a processor number at the time of program execution and access history information indicating the order of the processing element which is accessed to the shared memory 13 is recorded in an access history area 133. At the time of restarting program execution, the plural processing elements 11-1 to 11-n respectively request the access of the shared memory but the access is permitted in the order of the processor number which is recorded at the time of program execution. Thus, program execution by the plural processing elements 11-1 to 11-n is restarted in the same order as that at the time of proceding program execution.