To provide an unauthorized vehicle detection system provided with a means for surely detecting an unauthorized vehicle even when the lowering factor of a vehicle recognition rate such as the stain of a number plate is present and preventing a normal automatic charge collection system (ETC) vehicle from being erroneously recognized as the unauthorized vehicle inversely.

Since the unauthorized vehicle 2 imitating the normal ETC vehicle 3 by utilizing communication between a road and the vehicle transmits control information 7 transmitted from a road side transmission/reception antenna 1 to a following ETC vehicle as the control information 9 by communication replay, the control information 9 is received by a set of two position detection antennas 4, and the position of the transmission source of a control radio wave 9 is detected by the incident angle of the control radio wave 9 to the position detection antenna by a radio wave transmission position detector 22. In the meantime, the image information of a passing vehicle from a camera 5 is image-processed and recognized in a vehicle position image detector 23 and the positions of all the vehicles are obtained. A radio wave monitoring device 24 compares the position of the transmission source of the control radio wave with the positions of all the vehicles and judges the matching vehicle as the unauthorized vehicle 2.