The present invention provides an improvement in or relating to a system for detecting the presence of an object of interest such as contraband, or an illicit threat item within a piece of baggage and notifying an operator such as a Security Officer (or an operator of security detection equipment) of its concealed presence, for instance in a piece of baggage. In particular, a system and method can be provided for notifying the operator of any potential contraband or illicit or threatening items using X-ray Phase Contrast Imaging (XPCI) and Deep Learning models.

Law Enforcement, Border Force and Customs Authorities struggle to identify contraband / illicit items such as explosives, drugs and/or banned wildlife products such as elephant ivory or rhino horn ivory at security and border checkpoints in a timely and efficient manner. Detection of contraband / illicit items at security checkpoints can be very difficult especially when these are concealed in hand- luggage, baggage and containers.

It is common practice to use technology in the form of sensors, detectors and/or imaging equipment at a security checkpoint to identify objects of interest. For instance, X-ray images of hand-baggage are taken and one or more operators review these in search of suspicious shapes, textures or colours that cause a secondary check to be undertaken. This process is cumbersome and manpower intensive, impacting on the quality of life of all travellers. Additionally, the current security arrangements at security checkpoints may also be sub-optimal, due to the insensitivity of the equipment to pick up fine details of banned items, such as Improvised Explosive Devices, within the baggage with the risk that Improvised Explosive Devices can then be smuggled onto transportation such as aircraft, boats or cars with catastrophic loss of life. As a result, many operators at security checkpoints understandably take a more cautious approach during baggage screening and often flag up baggage for further checks where suspicion is low or negligible. This leads to a significant number of false positives and may also cause significant delays at security checkpoints. The current security arrangements at airports and elsewhere usually require electrical items to be removed from bags before being screened by an X-ray scanner, causing further delay and frustration for travellers. Therefore, there is a substantial global need for better security systems at airports and other checkpoints to detect such items with a high degree of reliability and sensitivity.

It is against this background that the present invention has arisen.

According to the present invention, there is provided a system for notifying an operator of the presence of an object of interest within an entity, the system comprising, an image scanner configured to image the entity containing at least one object to provide an image dataset of the entity and the object(s) contained therein, wherein the entity and object(s) contained therein comprise a plurality of parameters; a storage location configured to maintain a dataset comprising a plurality of signature profiles of objects of interest and a set of threshold values indicative of acceptable levels of correlation with each signature of an object of interest, wherein the signature profile comprises a plurality of parameters; a decision-making module configured to receive the dataset from the image scanner and correlate the image dataset of the object(s) within the entity with the plurality of signature profiles of objects of interest, wherein the decision-making module is further configured to determine whether the correlation between at least one parameter of the scanned image and at least one parameter of any one of the signatures of objects of interest exceeds the corresponding threshold value; and an output module associated with the decision-making module configured to provide a notification to the operator; wherein the notification is either one of the following: notification type A indicating the presence of an object of interest based on the correlation between at least one parameter of the scanned image and at least one parameter of the signature profile of an object of interest exceeding the threshold, or notification type B indicating the absence of an object of interest based on the correlation between at least one parameter of the scanned image and at least one parameter of the signature profile of all objects of interest being below the corresponding threshold value. Examples of an object of interest may be one or more of the following: explosives, narcotics or a pharmaceutically relevant drug, contraband items such as ivory items or elephant tusks or a tumour within the human body, or a biopsy of human cells to detect suspicious cancerous or pre-cancerous cells.

The entity may be a physical entity. Examples of an entity may be one or more of the following; a piece of baggage, a goods container such as a shipping container or an aircraft container, a human or animal body or a drug formulation.

In some embodiments, determined perhaps by cost, the system could display the image or related data in such a way as to locate and highlight the presence of the suspected object(s) within the entity. In some embodiments, the threshold values may be adjusted to accommodate real world circumstances that might change or impact workflows. For example, the threshold values can be reduced or lowered to detect more items if a high-profile person is scheduled to go through a security checkpoint. This might be appropriate because the risk of a missed object of interest may be greater than the inconvenience of a higher than usual number of false positives, i.e. detections of possible object of interest that turn out to be benign.

A further example of threshold values being fine-tuned may be on the basis of the geographic location of the system i.e. to accommodate and detect certain objects or items at the borders of specific source-countries (e.g. elephant ivory from African countries) or countries where demand for illicit goods is high (e.g. China re ivory).

In some embodiments, the image dataset can be derived from one or more of the following: Raman spectra, Gamma Ray or X-ray detection. This can be advantageous as it may provide additional distinctive images and/or parameters of an object of interest to be discovered by Deep Learning Models or other signature profiles and/or parameters to be stored in the storage location i.e. a“library”. For example, Raman Spectroscopy is a detection technique that can be used for finding volatile liquids such as hydrogen peroxide hidden inside a drinks bottle such as Lucozade ^® because volatile liquids that could be used in an Improvised Explosive Device can be difficult to distinguish from less dangerous counterparts. By using data derived from Raman spectroscopy, this may eliminate or reduce the need to limit the size of liquid containers carried onto transportation such as aircrafts.

In another example, Gamma-Ray Detectors can be deployed to search for dirty bombs / radioactive material being smuggled through ports, but it is not always easy to distinguish genuine threats from the signal noise. In some embodiments, the input data comes in the form of one or more images is generated by an X-ray scanner, with advances in technology allowing for more information to be extracted (for instance through X-Ray Phase Contrast Imaging).

In some embodiments, the signature profile of objects of interest can be based on one or more of the following parameters: micro-structure and/or texture, shape, intensity, contrast, colour of the item. For instance, the micro-structure and/or texture of a material can provide distinctive features that are associated with an explosive or an illicit item. It is an advantage of the system of the present invention to be able to detect these distinctive features of an object of interest or item because these distinctive textures are not readily recognisable to the human eye. Furthermore, in some embodiments, the one or more parameters of the signature profile of objects of interest may be compared with a plurality of different thresholds to form data comprising a plurality of elements. As a result, each element (e.g. for density) may be“yes/no/maybe” - rather than a binary yes/no, when compared to the threshold value. The system may then combine a plurality of elements from within the data to make an overall determination of the presence or absence of a threat item.

Comparing a plurality of elements within the data is advantageous as it reduces the number of false positives by running multiple observations simultaneously and making a more informed and reliable decision based thereon. In some embodiments, the presence and/or absence of an object of interest may be based on the correlation between a plurality of parameters of the scanned image and the signature profile of an object of interest exceeding the threshold. For example, the presence and/or absence of an object of interest may be based on the correlation between 1 , 2, 3, 4, 5, 10, 20, 50, 100, 1000 or more than 1000 parameters of the scanned image and the signature profile of an object of interest exceeding the threshold.

In some embodiments, the decision-making module can informed by a Deep Learning Model that has been trained to identify tell-tale signatures that are difficult or almost impossible for the human eye to detect, or which can be missed through fatigue-induced errors as the attention span of humans begins to diminish after a relatively short period of time.

In some embodiments, the decision-making module further comprises a feedback mechanism which adds image data, threshold correlation and outcomes to the storage location. The feedback may be checked by a human operator to enhance the DNN threat library. This ensures that the system is adapted for active or continuous learning from real life security processes. This can lead to regular updates to the library of signature profiles of the object of interest informed, for instance, by instances of False Positives, improving the effectiveness of the system to disambiguate objects of interest to those that are of no interest. In addition, the feedback mechanism enables the decision-making module to improve its performance in identifying True Positives. It can be envisaged that individual systems feeding back these lessons learned to a central repository allowing a universal release of updated signature profiles to all systems, ensuring the benefits of Active Learning are shared globally.

In some embodiments, the system may further comprise a processing module associated with the x-ray scanner, wherein the processing module may be configured to process the x-ray image of the object to remove background noise. The X-ray image may further be processed to remove background noise prior to inputting the image into the DNN model. The invention will now be further and more particularly described, by way of example only, and with reference to the accompanying drawings, in which:

Figure 1 provides a schematic demonstration of a system for detecting an object of interest at a security checkpoint according to the present invention.

Referring to Figure 1 , there is provided a schematic showing a system of the present invention for notifying an operator of the presence of an object of interest within an entity such as a piece of baggage. The system comprises inputs from a scanner or detector at a security checkpoint configured to gather data (for instance imaging a piece of baggage); a storage facility, which can be a virtual library, configured to maintain a dataset comprising a plurality of signature profiles of objects of interest and a set of threshold values indicative of acceptable levels of correlation with each signature of the object of interest. A decision-making module, which may also be referred to as a Virtual Assistant, may be configured to receive the dataset from the image scanner or detector and correlate the image dataset of the object(s) within the entity with the plurality of signature profiles of the object of interest, where the decision-making module may be further configured to determine whether the correlation between at least one parameter of any one of the signatures of the object of interest exceeds the corresponding threshold values for at least one parameter of that object of interest signature profile. The decision-making module will typically be algorithms that interrogate one or more Deep Learning Models for instance, based on a Deep Neural Network or DNN trained externally to learn to identify tell-tale signatures. An output module i.e. a user interface associated with the decision-making module may be configured to provide a notification to the operator; where the notification is either one of the following: notification type A indicating the suspected presence of an object of interest based on the threshold values of at least one parameter of the signature profile of the object of interest being exceeded, or notification type B indicating an absence of an object of interest based on the probability of at least one parameter of the object being below the threshold value of at least one parameter of the signature profile of the object of interest. Notification type A and/or B can be a visual and/or an audible notification, which may be used to alert the operator of the object of interest. An output could also take the form of an automated instruction to the security scanner or detector to divert the bag (or other entities being scanned) into a channel for secondary screening. The decision-making module, also referred to as the Virtual Assistant, may be a computer-implemented system connected to an image scanner such as an X-ray scanner that uses the input of distinctive textures derived from the image e.g. XPCI images filtered through DNN Models based on objects or targets of interest e.g. explosives/ivory. The DNN model can be used to process the received image and determine whether the object is an object of interest or not based on the probability that at least one parameter of the object exceeds the pre-determ ined threshold values of at least one parameter of the signature profile of the object of interest.

The DNN model is able to flag up in real-time the suspected presence of contraband and/or illicit items to personnel operating at security checkpoints. Furthermore, the system of the present invention may be able to accommodate a“library” of DNN Models of target items, capable of being updated to reflect improvements in machine-learning and imaging. For instance, advances in x-ray technology include Computed Tomography, 3-D imaging, Multi-View Imaging and X-ray Phase Contrast Imaging.

In some embodiments, the system may enable an operator to set one or more thresholds value for a signature profile of the object of interest or at least one parameter thereof, with the ability for the operator to also adjust the threshold values, in order to reflect changes of risks or the object of interest profile for instance, it might be acceptable to set the output probability to 0.25 for explosives when scanning bags going onto the flight of a very very important person (WIP) who is the target of assassination, or the same for elephant ivory at a place and time when intelligence suggests this is being smuggled out of the airport.

In addition, a feedback mechanism can be provided with the current system in order to support active training of the decision-making module i.e. the DNN model. A feedback mechanism can be used to add image data, threshold correlation and outcomes to the storage location, or any other additional information deemed useful by an operator. This ensures that the system is adapted for continuous learning from real life processed images. An example as to when a feedback mechanism is required can be when the decision-making module determines a false positive i.e. where the decision-making module wrongly suspects the presence of an object of interest.

Following a secondary security check usually by an operator, the feedback can be submitted to a Deep Learning facility for retraining the DNN Models and upgrades are fed back to the library ensuring continuous improvement. In this case, the feedback mechanism helps the decision-making module to continually “learn” in order to reduce or eliminate any false positives. This feedback can ultimately be transferred to a universal Active Learning facility, leading to the performance of all systems to be improved through the release of revised signature profiles. In order to demonstrate the ability of the DNN model in identifying objects of interest such as explosives within an entity i.e. a piece of baggage, a double-blind trial can be carried out, as an example, in which the combination of x-ray images and DNN models can be used to detect explosive in X-ray images. In the study conducted by the inventors of the present invention, the performance of the DNN model in detecting an object of interest e.g. threat items such as explosives within piece of baggage was compared to the performance of humans (professional security officers from airports and volunteers). The results are shown in Table 1 .

Table 1 show that the DNN model receiving XPCI images is able to process the images and provide a 100% accuracy rate in identifying explosive found in baggage. In comparison, the operators using current airport procedures i.e. manual analysis to determine whether a baggage had the explosives or not from the X-ray images had accuracy rate of 45.8%. Volunteers which used an algorithm to help identified the explosive in baggage had an accuracy rate of 70%. Table 1 provide results of a trial for detecting explosives in X-ray images.

* Airport operators/reviewers worked with one (Absorption X-ray) image format for each of the 80 targets.

* ^* Volunteers and the DNN Model worked with multiple image formats per 80 targets.

As demonstrated by the results in Table 1 , the system of the present invention may be intended to speed up the process of identifying an object of interest within an entity and to notify the operator. Based on the results from Table 1 , airport security personnel took on average 6.56 seconds to review an x-ray image whereas the Virtual Assistant took 1.475 seconds. The statistics are even more significant if it is recognised that airport reviewers examined one image per format whereas the Virtual Assistant can examine an x-ray image with multiple formats. Through Active Training of the DNN model, it can be anticipated that there would be fewer and fewer False Positives, significantly reducing delays caused by secondary scanning. The combined application of XPCI and Deep Learning is intended to eliminate the need for electronic items to be removed from hand-luggage, which makes a small, positive impact on the speed of an individual’s journey through the system, but in aggregate makes a substantial impact when there are on average about 8 million air-passenger journeys a day (over 3 billion a year).

Various further aspects and embodiments of the present invention will be apparent to those skilled in the art in view of the present disclosure.

“and/or” where used herein is to be taken as specific disclosure of each of the two specified features or components with or without the other. For example“A and/or B” is to be taken as specific disclosure of each of (i) A, (ii) B and (iii) A and B, just as if each is set out individually herein. Unless context dictates otherwise, the descriptions and definitions of the features set out above are not limited to any particular aspect or embodiment of the invention and apply equally to all aspects and embodiments which are described.

It will further be appreciated by those skilled in the art that although the invention has been described by way of example with reference to several embodiments. It is not limited to the disclosed embodiments and that alternative embodiments could be constructed without departing from the scope of the invention as defined in the appended claims.