METHOD AND SYSTEM

Field of the Invention

This invention relates to methods, computer software and apparatus for processing electromagnetic scanning machine data.

Background of the Invention

Electromagnetic based scanning machines are increasingly being used in a wide range of applications. These machines generate large data sets - commonly referred to as scan images - which can be analysed to detect the presence of a material of interest. These same data sets can also be used to detect the presence of objects of interest based on expected shapes and the combinations of components embodied.

It is possible to detect the presence of many different types of materials and objects of interest from a single scan data set. This data can be provided as an input parameter to a range of different analysis processes, each being executed to focus on the presence of a particular material or object.

Due to size considerations it is preferable to limit handling or transmission of these data sets while also have the capability of identifying limited numbers of data sets of particular interest. In practice it may not be feasible to undertake a large number of analysis processes using such data sets, particularly where detection results are needed in real time.

For example hyperspectral scanning machines are particularly useful in real time applications such as transportation security and border security. As stand-alone units these machines incorporate data processing capabilities, but are limited in terms of how many discrete analyses may be run on a single scan data set in a real-time setting. With these current forms of scanning machines fixed decisions need to be made as to howthe machine is configured and what materials or objects it targets in real-time.

Furthermore the scan data generated by various scanning machines can also be used in computationally intensive analyses which exceed the capabilities of the machine's on-board processing. Current forms of these machines also do not readily facilitate the execution of analysis processes which use two or more different scan data sets as inputs - be they obtained from different scanning machines, or the same scanning machine over a period of time.

It would be of advantage to have improvements to the current state-of-the- art which improved the results provided by the processing or analysis of scan data generated by such machines. In particular, it would be of advantage to provide for flexibility in where and/or how scan data is processed, preferably in real time. It would be of advantage to provide for flexibility in how scan data is processed based on variable characteristics of the article or object used to generate the scan data, or the environment or characteristics of the scanning machine when scan data was generated. It would also be of advantage to provide for flexibility in how scan data is stored, and provide different storage options based on variable

characteristics of the article or object used to generate the scan data, or the environment or characteristics of the scanning machine when scan data was generated.

Disclosure of the Invention

According to one aspect of the invention there is provided a method of processing electromagnetic scanning machine data characterised by the steps of:

• receiving a scan data set generated by an electromagnetic scanning machine scanning an object

• receiving either or both of key data and environmental data, said key data related to information associated with the object used to generate the scan data set and said environmental data related to the location, environment and/or configuration of the scanning machine used to generate the scan data set,

• at any following step of this method, associating either or both of key data or environmental data with the scan data,

• using either or both of the key data or environmental data to select at least one analysis process adapted for the detection, identification and/or measurement of the presence at least one target material or target object

• executing said at least one selected analysis process with the scan

data set as an input to detect identify and/or measure the presence of said at least one target material or the presence of at least one target object.

According to another aspect of the present invention there is provided a method of processing electromagnetic scanning machine data substantially as described above further characterised by the additional step of using either or both of the key data or environmental data to select one or more storage criteria, and

storing the scan data set based on the selected storage criterion or criteria.

According to yet another aspect of the invention there is provided a method of processing electromagnetic scanning machine data substantially as described above further characterised by the additional step of receiving returned results of the at least one selected analysis process, and

communicating the returned results to at least one recipient, wherein at least one of the key data and/or environmental data is used to select at least one recipient to communicate the returned results to.

According to a further aspect of the invention there is provided computer readable storage media storing computer executable instructions that when executed by a computer are configured to implement a method of processing electromagnetic scanning machine data by executing the steps of: :

• receiving a scan data set generated by an electromagnetic scanning machine scanning an object

• receiving either or both of key data and environmental data, said key data related to information associated with the object used to generate the scan data set and said environmental data related to the location,

environment and/or configuration of the scanning machine used to generate the scan data set,

• at any following step, associating either or both of key data or environmental data with the scan data set,

• using either or both of the key data or environmental data to select at least one analysis process adapted for the detection, identification and/or measurement of the presence at least one target material or target object • executing said at least one selected analysis process with the scan data set as an input to detect, identify and/or measure the presence of said at least one target material or the presence of at least one target object.

According to another aspect of the invention there is provided a system for processing electromagnetic scanning machine data which includes computer storage media storing computer executable instructions configured to implement the method of processing electromagnetic scanning machine data defined by the steps of:

• receiving a scan data set generated by an electromagnetic scanning machine scanning an object,

• receiving either or both of key data and environmental data, said key data related to information associated with the object used to generate the scan data set and said environmental data related to the location,

environment and/or configuration of the scanning machine used to generate the scan data set,

• at any following step, associating either or both of key data or environmental data with the scan data set,

• using either or both of the key data or environmental data to select at least one analysis process adapted for the detection, identification and/or measurement of the presence at least one target material or target object,

• executing said at least one selected analysis process with the scan data set as an input to detect, identify and/or measure the presence of said at least one target material or the presence of at least one target object.

In various aspects the present invention facilitates the processing of electromagnetic scanning machine data. The invention encompasses a method of processing electromagnetic scanning machine data, a system for processing electromagnetic scanning machine data, and stored computer executable instructions used to implement this system and execute this method. Reference predominantly throughout this specification will be made to the invention providing a method of processing, while those skilled in the art will appreciate that the invention encompasses additional aspects.

Reference in general throughout this specification will also be made to the execution of the method of the invention being undertaken locally to a scanning machine which generates the electronic scan data set used. For example in various embodiments of the invention may be executed through processor resources provided integral to or on board with such a scanning machine. In other implementations a dedicated processor may be located local to and physically connected to an electromagnetic scanning machine. However skilled in the art will also appreciate that in various embodiments the processing resources used to execute the method of the invention may be located remote from a scanning machine used to generate the scan data set handled by the invention.

The processing facilitated by the invention utilises an electronic scan data set as an input. Electronic scan data sets can be generated through the operation of electromagnetic scanning machines of various forms, such as - for example - X-ray, visible, NIR, UV, THz, millimetre-wave, microwave or radio-wave based machines. Those skilled in the art will appreciate that the invention may be used with a range of different types of scanning machine.

Electronic scan data sets are generated through the action of an electronic scanning machine scanning an object. Those skilled in the art will appreciate that a large range of different types of objects may be scanned by such machines and the examples of these objects used in this specification are in no way exhaustive. For example, objects in the form of people seeking to use transport networks may be scanned, as can materials such as posted packages transmitted by postal systems. Those skilled in the art will also appreciate that a scan data set may provide a base spectral data set, may be raw, partially processed or fully processed and may include layers conveying results of the scanning operation.

The present invention facilitates the execution of at least one analysis process which has the objective of detecting or identifying the presence of one or more specific materials, or the presence of a particular type of target object. Furthermore the analysis processes and scan data sets utilised by the invention may also be used to measure various characteristics of such materials or target objects.

In preferred embodiments the scan data set may be generated by a hyperspectral electromagnetic scanning machine. For example in various embodiments scan data set may be generated by a spectral electromagnetic scanning machine with a plurality (>2) of energy bands and/or frequencies and/or wavelengths and/or time steps. Such a scanning machine could be spectral, multi-spectral, multi-energy, multi-frequency, multi-wavelength, or hyperspectral and/or combination of these. Furthermore the data set generated by such machines could be recorded as any combination of one or more of spectral, multi-spectral, multi-energy, multi-frequency, multi wavelength, or hyperspectral.

In further preferred embodiments the scan data set may be generated by a multispectral or hyperspectral x-ray energy scanning machine. For example in various embodiments a scan data set may be generated by a spectral X- ray energy scanning machine with a plurality (>2) of energy bands. Such a X-ray scanning machine could be spectral, multi-spectral, multi-energy or hyperspectral. Furthermore the data set generated by such machines could be recorded as any combination of one or more of spectral, multi-spectral, multi-energy, multi-frequency, multi-wavelength, or hyperspectral.

In some embodiments an electronic scan data set may be received from an electromagnetic scanning machine. If the processor resources utilised by the invention are located locally to the same scanning machine the invention does not experience significant data propagation delays in the receipt of electronic scan data sets.

In various embodiments electronic scan data sets may also be received from a data archive. Such data archives can be formed by computer implemented systems incorporating appropriate communications interfaces and which are capable of storing and retrieving large amounts of data over extended periods of time. In addition embodiments a data archive may be formed by computer systems which can provide a short or medium term buffer storage facility. For example, in some embodiments a data archive may be formed by the receiving components of a communications system which can buffer transmitted scan data sets for periods of time.

In yet further embodiments electronic scan data sets may also be received by way of a manual upload. In such instances a manual operator may have identified that a particular scan data set should be processed in accordance with the invention, with this operator uploading or otherwise providing the scan data set.

The method of the present invention also incorporates the receipt of either or both of key data and environmental data. This key data relates to information associated with the object used to generate the scan data set while environmental data relates to the location, environment and/or configuration of the scanning machine used to generate the scan data set.

The form of key data utilised by the invention will therefore vary based on the type of object which is scanned and the information available in respect of the object which can be supplied as key data. Furthermore in various embodiments key data related to an object may be derived or obtained through intermediary relationships to other objects. For example in some embodiments key data may be obtained from the results of a sample being taken from the object and scanned using a further electronic scanning machine.

Similarly, environmental data will vary depending on the application in which a scanning machine is used and the environment in which the scanning machine is used. References made to the environment of a scanning machine should be interpreted to include conditions such as security alert levels or warnings issued in respect of a geographical area in which the scanning machines located, in addition to local variables such as

temperature or humidity values in the vicinity of the scanning machine . As such the environment of the scanning machine can be considered to be wide ranging and not necessarily limited to the particular address, building or site at which a scanning machine is physically located.

Those skilled in the art will appreciate that in various embodiments key data may include without limitation any one or more of passport number, flight number, seat number, biometric information, courier tracking number, customs declaration number and/or package identification information.

Those skilled in the art will appreciate that in various embodiments environmental data may include without limitation any one or more of scanning machine ID, date/time of scan, GPS data for scan machine location, machine reference scan data, machine standardisation data, machine encryption codes, or machine data compression codes.

In a variety of additional embodiments the results of one or more analysis processes undertaken by the invention may be returned and associated with the input scan data set as key data or environmental data. In such

embodiments the scan data set may therefore maintain a record of the results of prior analytical processes undertaking using the data set.

Reference in general throughout this specification will also be made to the invention receiving both key and environmental data for a particular scanned data set. Those skilled in the art will however appreciate that in other embodiments key data only may be received, as may be environmental data only.

In a preferred embodiment key and/or environmental data may be received by the invention through a communications interface which delivers this data separate to a received scanned data set. For example in some embodiments the invention may receive a reference or address which facilitates the retrieval of key or environmental data, while in other embodiments this data may be transmitted directly to the system of the invention separate from the scan data set.

In yet other embodiments, and as identified above, key and/or

environmental data may have previously been associated with a scan data set. In such embodiments this data may be extracted from the data set itself, and in some instances may also be combined with additional key and/or environmental data received independent from the scan data set

In yet further embodiments key and/or environmental data may be received by way of a manual upload. In such instances a manual operator may be aware of specific key or environmental data which should be associated with a particular scan data set, with this operator uploading, inputting or otherwise providing the data involved.

The present invention also operates to associate either or both of key data or environmental data with the scan data set. This association operation can be completed with some flexibility at various stages of the methodology implemented by the invention. For example once the processor utilised by the invention has access to this information the key or environmental data may be associated with a scan data set immediately. Alternatively this association operation may be undertaken after the results of one or more analytical processes which use the scan data set have returned their results, thereby allowing these return results to also be associated with the scan data set.

Reference throughout this specification to the association of key and/or environmental data with a scan data set may encompass a variety of techniques undertaken so that this data that is linked, concatenated, either prepended or appended, embedded or otherwise joined or connected to the scan data set.

For example in some embodiments key data and/or environmental data may be associated with the scan data by appending and/or prepending the key and/or environmental data to the scan data. In addition, in yet other embodiments, key data and/or environmental data may be associated with the scan data by embedding the key and/or environmental data in the scan data.

The present invention also operates to use either or both of the key data or environmental data to select at least one analysis process adapted for the detection, identification and/or measurement of the presence at least one target material or target object. In this way the scan data set's key or environmental data can be used to control how the data set is processed and in particular select for the execution of one or more relevant analysis processes considering the application in which the invention is used.

An analysis process executed in conjunction with the invention may provide a variety of different forms of information. In some instances an analysis process may detect the presence of a target object or material through providing a general indication of an anomalous result being obtained from an assessment of the scan data set. Such detection based returns may highlight that further investigation of an object is warranted. In other instances an analysis process may identify the presence of the target object or material, with these processes explicitly confirming that the target object or material is present. In yet other instances and analysis process may complete measurements in respect of a target object or material, returning one or more specific values which indicate the magnitude of particular characteristic of the target material or object.

Those skilled in the art will appreciate that a wide variety of analysis processes may be utilised in conjunction with the present invention and therefore the examples given throughout this specification should in no way be seen as exhaustive. Furthermore the application in which the invention is employed will dictate the types of analytical processes used, and whether there is a desire to provide returned results in real time, or to allow for the processing of archived historical scan data sets.

For example in various embodiments an analysis process executed in conjunction with the invention may include any one or more of target object pattern recognition, target material multivariate analysis, target object image processing, target material statistical analysis, or target material relational analysis.

In yet further embodiments an analysis process executed in conjunction with the invention may receive a plurality of scan data sets as inputs. In such embodiments the processing resources used may receive input scan data sets from multiple different sources, be they a plurality of separate

electromagnetic scanning machines, manual uploads or data archives.

In a preferred embodiment the identity of a selected analysis process determines the location where the process will be executed. Those skilled in the art will appreciate that different analytical processes require different computational resources and may require that the results of the process under executed are returned in real time. The invention therefore allows for the selection of a variety of processing resources for the execution of analytical processes, these resources potentially being located remotely from one another.

For example in some embodiments at least one analysis process may be executed in real time by an electromagnetic scanning machine. These analyses may return results which are time critical, such as those desired in security or border control applications.

In yet other embodiments at least one analysis process may be executed as a low priority batch process by an electromagnetic scanning machine. The results of these forms of processes may not necessarily be time critical, but may produce useful information via more computationally intensive

processing operations.

Furthermore in other embodiments at least one analysis process may be executed by the scan data set being transmitted to a local processor. In such embodiments this local processor may not be integrated into an

electromagnetic scanning machine but may be physically connected to one or more such machines in a particular location or site. This separate local processor may provide increased processing capacity to that provided by an electromagnetic scanning machine if required in various embodiments.

In yet other embodiments at least one analysis process may be executed by the scan data set being transmitted to a remote processor. Although this approach may suffer from propagation delays due to the size of scan data sets being transmitted, a remote processor may be used to provide

significant increases in processing capacity. In addition a remote processor may also be configured to receive multiple scan data sets from a variety of sources and use these multiple data sets as inputs to an analysis process.

In a preferred embodiment the invention may be configured to receive the returned results of one more analysis processes. As indicated above these returned results may also optionally be associated with the scan data set as key or environmental data. Returned analysis process results may also be communicated to additional locations or entities using the present invention. In such embodiments at least one of the key or and/or environmental data associated with the scan data set may be employed to select at least one recipient to which these results are communicated. These recipients may be computer systems or personnel which may be located local to or remote from the processing resources used to execute the invention. In such embodiments the actual return results themselves may also be used as a factor in the selection of recipients for such communications. In a variety of embodiments the invention may also be used to control if or how a scan data set is stored after the execution of any selected analysis process. In such embodiments at least one parameter from the key data and/or environmental data may be used to select one or more storage criteria. Preferably a storage criteria may identify a recipient or location to which the scan data set is to be transmitted, and scan data sets may be stored by transmission to said recipients or locations. These recipients may be computer systems or personnel which may be located local to or remote from the processing resources used to execute the invention.

The present invention may provide a number of advantages over the prior art.

In various embodiments the invention may provide for flexibility in how scan data is processed, and preferably in real time. The invention may allow for the dynamic adjustment of target identification thresholds based on the key or environmental data provided. Furthermore the invention can be used to deploy new analysis processes for novel materials and/or objects of interest as they are identified.

The invention may also facilitate multiple different types of analysis processes being run simultaneously or sequentially on one scan data set. The scan data may be analysed multiple times for different target objects or materials depending on the key or environmental data received in

conjunction with the scan data.

In various embodiments the invention may provide for flexibility in how scan data is stored, and may provide different storage options based on variable characteristics of the article or object used to generate the scan data, or the environment or characteristics of the scanning machine when scan data was generated. The storage options can also facilitate the retrieval of stored historic data sets for subsequent forensic examination, and allowing for the application of new analysis processes to this historic data.

The invention may also facilitate the execution of analysis processes which collate together and use a large number of scan data sets as inputs. In this way the invention may facilitate the provision of information inferred or drawn from the results obtained from multiple scanning machines, and/or results from scanning machines over a selected period of time. In such applications the key and/or environmental data provide in accordance with the invention can be used to select and also index these data set for this type of analysis.

Brief description of the drawings

Additional and further aspects of the present invention will be apparent to the reader from the following description of embodiments, given in by way of example only, with reference to the accompanying drawings in which:

Figure 1 illustrates data flows and communications undertaken in the

processing of a scan data set in accordance with one embodiment of the invention.

Figure 2 provides a visual representation the execution of an analysis

process using scan data sets as an inputs in accordance with another embodiment of a further aspect of the invention.

Figure 3 provides a component diagram illustrating the configuration of

computer executable instructions provided in a further embodiment to process scan data.

Further aspects of the invention will become apparent from the following description of the invention which is given by way of example only of particular embodiments.

Best modes for carrying out the invention

Figure 1 illustrates data flows and communications undertaken in the processing of a scan data set in accordance with one embodiment of the invention. This process starts with the receipt of a scan data set 1. Key data and environmental data 2 pertinent to the object being scanned is also received. The scan data set and key and environmental data are next associated together, where in the embodiment shown the key and environmental data is appended to the scan data set in a common data structure 3. In this embodiment the data structure is formatted so selected bytes are reserved for key data and environmental data and other bytes are used for scan data.

Next a process selection operation 4 occurs. Key and environmental data is retrieved from the common data structure 3 and used to determine which of one or more subsequent analysis processes 5 are run using the scan data set 1 as an input. These analysis processes are adapted to detect, identify or measure the presence of at least one target material or target object of interest.

The process selection operation determines which of these analysis

processes are run with the scan data set. The key and environmental data is used to indicate an increased likelihood of particular target materials and/or particular target objects being present in an object which was scanned, and to trigger processing of the scan data set to detect, identify or measure such particular materials and/or objects.

The outcomes or returned results of the selected analysis processes are communicated locally and remotely 6. In the embodiment show the recipients of this communication are selected based on the key and

environmental data associated with the input scan data set.

In this embodiment the returned results of the selected analysis processes are appended to the scan data as additional key and environmental data 7. The total appended key data and environmental data is used to select one or more storage criteria, and the scan data set is stored at local and remote locations based on the selected storage criterion or criteria 8.

Figure 2 illustrates the analysis processing of a scan data set in accordance with a further embodiment where the analysis process receives a plurality of scan data sets as inputs.

In this embodiment a network of scanning machines (represented as

"rectangles" for Hyperspectral X-ray and a "triangle" or "circle" for other types of machine) automatically send data sets to a remote processor provided by a data centre (be it local, national, international or global).

The data sets are merged into a data matrix where each column holds the sequential data of a specific machine and each row is a time period. In other embodiments different parameters drawn from key or environmental data associated with the data set may alternatively be used to index this matrix.

An analysis process is then applied to the data matrix to automatically look for relationships. For example, we observe in figure 2 two cells that are highlighted in closely related time periods. This is identified as a potential link and calibrations are used to assess the likelihood of the events being related (e.g. persons travelling together, or sub-components of banned substances, or envelopes with drug pre-cursers).

The analysis applied could be pattern recognition, multivariate, image processing, statistical analysis, relational analysis and so forth. If the returned results of this analysis indicates a possible or probable relationship between the events these results can be communicated to interested parties or can trigger a further analysis process.

Figure 3 provides a component diagram illustrating the configuration of computer executable instructions provided in a further embodiment to process scan data. The diagram defines the functions and communications links of various components of the system implemented by these

instructions.

A communications interface A is provided to receive both key and

environmental data at the same time as communications interface B receives a scan data set related to this key and environmental data.

Decision and control logic module C receives this information and undertakes a variety of procedures in accordance with the invention.

Logic module C assesses the received key and environmental data and selects one or more analysis processes which are to be executed using the scan data set as an input. The logic module can transmit the scan data to one or more of a variety of communication interfaces D1-D4 which provide links to processing resources tasked with executing such analysis processes. In the embodiment shown these communications interfaces include:

• Dl, a communications interface linked to a local electromagnetic

scanning machine configured to return results in real time.

• D2, a communications interface linked to a local electromagnetic

scanning machine configured to complete low priority batch processing of scan data sets outside of real-time applications.

• D3, a communications interface linked to a local processor configured to return results in real time.

• D4, a communications interface linked to a remote processor

configured to receive scan data sets from a number of sources.

Logic module C also receives the returned results of the analysis processes undertaken through these communications interfaces D. Logic module C transmits the returned results of the analysis process or processes

undertaken to one or more recipients using a further communications interface E. The recipients for these communications are selected by logic module C using the key and environmental data provide in association with the scan data set.

A scan data set formatting module F receives the scan data set, key and environmental data, and the returned results of any analysis processes. This module integrates the returned results into the key and environmental data and associates the updated key and environmental data with the scan data set by appending it to the scan data set.

A storage communications interface G is also linked to the scan data set formatting module F. This communications interface assesses the appended key and environmental data of a scan data set that it receives from the formatting module F to select one or more storage criteria for the scan data set. The selected storage criteria(s) identifies a recipient or location to which the storage communication interface transmits the scan data set.

In the preceding description and the following claims the word "comprise" or equivalent variations thereof is used in an inclusive sense to specify the presence of the stated feature or features. This term does not preclude the presence or addition of further features in various embodiments.

It is to be understood that the present invention is not limited to the embodiments described herein and further and additional embodiments within the spirit and scope of the invention will be apparent to the skilled reader from the examples illustrated with reference to the drawings. In particular, the invention may reside in any combination of features described herein, or may reside in alternative embodiments or combinations of these features with known equivalents to given features. Modifications and variations of the example embodiments of the invention discussed above will be apparent to those skilled in the art and may be made without departure of the scope of the invention as defined in the appended claims.