Technical Field The present disclosure relates to the field of physiological monitoring apparatus, associated methods, computer programs and apparatus. Certain disclosed aspects/embodiments relate to electronic devices.

Background

Identifying persons having a hostile, subversive, deceptive or malicious intent is a highly complex task, particularly when trying to identify terrorists and prevent any impending terrorist attacks. Security personnel may not be of sufficient deterrence and serve only to put additional lives at risk. If a terrorist can be identified remotely and covertly, then measures can be put in place that may mitigate the risks to civilians.

Research suggests that deceptive behaviour can be identified by analysing psychological and physiological measurements. Unfortunately much of this research requires contact with the individual, either human to human or machine to human. This may not be a desired property of a detection system as this could alarm a person being monitored, which may have adverse consequences.

The listing or discussion of a prior-published document or any background in this specification should not necessarily be taken as an acknowledgement that the document or background is part of the state of the art or is common general knowledge. One or more aspects/embodiments of the present disclosure may or may not address one or more of the background issues.

Summary

In a first aspect, there is provided a response detection system comprising:

stimulation means for delivering a visual and/or an auditory stimulus to a target subject; a remote non-contact sensor configured to monitor a physiological indicator in the subject, the physiological indicator including at least one of heart rate and temperature; and

processing means configured to detect a change in the monitored physiological indicator in response to the stimulus and to determine if the change exceeds a predetermined threshold.

Embodiments may provide a means of distinguishing between hostile and non-hostile subjects. Hostile subjects may comprise individuals who are carrying out or associated with a hostile action. The hostile action may comprise one or more of smuggling, terrorist activities, immoral activities, illegal activities, clandestine activities, and/or money laundering. Non-hostile subjects may comprise individuals who are not carrying out or otherwise associated with a hostile action. The stimulation means may comprise one or more display screens (e.g. flat screen displays, CRTs or overhead projectors), one or more static displays (e.g. poster boards, wall paintings) and/or one or more speakers. The stimulation means may be configured to present an inducing presentation (e.g. intended to stimulate a response in a hostile subject) and/or a non-inducing presentation (e.g. intended not to engender a response in a hostile subject). A presentation is essentially a series of at least one stimulus configured to be presented to one or more subjects.

The non-inducing presentation may comprise a combination of one or more of an image, a video (e.g. advertisement), textual data, and subliminal images. The inducing presentation may comprise a combination of one or more of an image (e.g. smuggled item, drugs, weapons, money), a video, textual data (e.g. warning), and subliminal images (e.g. short-duration (e.g. 50ms-150ms) images).

A visual inducing/non-inducing presentation may comprise one or more of an image, a photograph, video footage, and/or a word display. An audio inducing/non-inducing presentation may comprise speech recordings, music and/or other sounds.

An inducing stimulus may be intended to induce an involuntary response in a hostile subject. An inducing stimulus may be intended to induce a subconscious and/or a conscious response in a hostile subject. An inducing stimulus may be configured to engender a response (e.g. a threat response) in hostile subjects and a different or no response (e.g. a no threat response) in non-hostile subjects. For example, a hostile subject may exhibit a raised heart rate when looking at an inducing stimulus whereas a non-hostile subject may not exhibit a raised heart rate when looking at the same inducing stimulus.

The non-inducing presentation may be configured to engender a voluntary response (e.g. such that the subject looks at the stimulation means) in both hostile and non-hostile subjects. The non-inducing presentation may be configured not to engender a response in either hostile or non-hostile subjects. The non-inducing presentation may be configured to engender the same response in both hostile and non-hostile subjects.

The predetermined threshold may be an absolute value or a relative value. Physiological indicator values exceeding the predetermined threshold may be indicative of a hostile subject and/or a threat response. An absolute threshold may, for example, comprise values associated with heart-rate and/or change in heart rate. A relative value may be a fractional or percentage change in the monitored physiological indicator. For example a heart-rate threshold value may be predetermined to be an 8% to 15% increase in heart- rate. A threshold value may be, for example, defined in terms of a number (or percentage) of subjects (e.g. ordered by corresponding physiological indicator value). The threshold may be defined using calculated parameters (e.g. mean and/or standard deviation). The threshold may comprise one or more indicator (e.g. an indicator corresponding with heart rate and an indicator corresponding with temperature). The response detection system may have one or more physical components. The physical components may be in wired or wireless communication. The response detection system may comprise a GHz radar, a thermal imaging camera and/or a computer (e.g. for analysis). The response detection system may be configured to determine a pulse rate, detect a heart-beat, detect a blush response, detect gaze direction and/or detect breathing rate. For example, any one or more combination of the following may indicate a strong response to the system - pulse rate changes over a threshold value of 8%, facial temperature changes over a threshold value of 10%, gaze direction changes showing avoidance once, breathing rate changes over a threshold value of 20%. The inducing presentation may comprise a stimulus configured to induce an involuntary reaction in subjects associated with (e.g. carrying out) one or more hostile action. The one or more hostile action may comprise smuggling, terrorist activities, immoral activities, illegal activities, and/or money laundering. It will be appreciated that the response detection system may be configured to correlate the monitored response to the nature of the inducing stimulus (e.g. those subjects exhibiting a reaction to stimulus associated with smuggling may be distinguished from those subjects exhibiting a reaction to stimulus associated with terrorist activities). It will be appreciated that subject's response to the non-inducing and/or inducing presentation may be selective depending upon the subject's intention or preference. For example, while using erotic pictures as non-inducing stimuli, similar attentional effects are apparent only for nudes of the opposite sex in heterosexual subjects. The non- inducing presentation may be intended to attract the subject's attention (e.g. to encourage the subject to look at the stimulus means).

It will be appreciated that the pulse rate and heart-beat may be considered to be physiological indicators of stress. Hostile subjects may be expected to exhibit a raised pulse rate and/or an increase in pulse rate in response to appropriate cues (e.g. inducing stimulus). The heart-beat may be remotely sensed using GHz radar at a distance of, for example, 1-2 meters.

Thermal imaging cameras may detect the temperature of imaged objects (e.g. the face of a subject). A thermal imaging camera may be used (e.g. the P7225 high resolution camera, manufactured by e2v technologies). The blush response is largely unconscious/involuntary, and cannot easily be controlled. A thermal imaging camera (e.g. the same camera used to detect the temperature of imaged objects) may be used to perform gaze-aversion detection. For example, facial temperature can change about 0.05-0.15 °C in a blush response, and 1 to 2 degrees of gaze direction change can be observed in gaze aversion depending on distance to visual stimuli. It will be appreciated that these values may be used as threshold values in determining whether a blush response or gaze direction change has occurred.

The response detection system may be configured to detect gaze aversion. Gaze aversion may be considered to be an unconscious/involuntary response to an unexpected and unwelcome psychological cue (e.g. inducing stimulus), which involves a measurable change of gaze direction. The threshold for gaze aversion may comprise values for change in gaze direction, rate of change of gaze direction and direction of gaze direction change. Gaze aversion threshold values may depend on the size of the screen to display the visual stimulus, and the position of the stimulus on the screen. For example, for a 21 inch screen, where the stimulus is displayed about 5-10cm from centre, a gaze aversion threshold of between 4-10 degrees gaze direction change may be used. The gaze-detection remote, non-contact sensor may be rendered more effective by illuminating the target with an appropriate light source (e.g. infrared), which is reflected by the pupils.

The response detection system may be configured to detect breathing (by detecting the expulsion of warmed air from the nostrils using, for example, a thermal imaging camera).

The response detection system may be deployed at a natural "choke point," where subjects are forced to pass within the range of the sensor system individually in a relatively controlled environment (e.g. a ticket barrier). The response detection system may be deployed in an enclosed space where the subjects are stationary (e.g. in a theatre, cinema, bus, train). The response detection system may be deployed inside or outside.

Further measurements may be initiated based on initial measurements. For example, the heart-beat may be detected first to identify apparently-stressed subjects. Then an inducing stimulus may be deployed to detect other physiological indicators for those identified subjects (e.g. blush response, gaze aversion and change of heart-beat).

Visual stimulus may be used to induce responses, as these are language independent, and likely to provoke changes of gaze direction in addition to pulse rate and blush response. Static visual cues (posters) and active visual cues (video screens), and/or audio stimulus may be used.

The remote non-contact sensor may comprise a gaze sensor. The gaze sensor may be configured to determine and/or track the position of the eyelids and/or canthus. The gaze deternnining means may be configured to permit gaze tracking without calibration or to alter setup to allow calibration without the subject realising it is in progress. Calibration may take into account the relative size of subject's head and facial features. Calibration may help to determine the value (e.g. measured in degrees) of gaze aversion.

In order to monitor a subject's gaze covertly, the camera may be hidden and/or positioned at a distance. The remote, non-contact sensor may be configured to determine the pose estimation, the orientation of the subject's head and/or the orientation of the subject's eyes within their sockets.

The gaze sensor may be configured to detect the location of the pupil. The gaze sensor may use a pupil-corneal reflection technique. This technique uses a light source (which may be infra-red), which the cornea reflects and manifests as a bright spot in the pupil of an eye. Then by employing image processing techniques (e.g. segmentation) the bright spot may be used to determine the location of the pupil (e.g. which may be in real time).

Template-based, appearance-based and feature-based methods may also be used. Once the pupils, and if necessary, head pose have been detected, the gaze may then be estimated. Ancillary features about the eye (i.e. eyelid shapes and corners) may also be extracted to assist in estimating the gaze direction. From these features the gaze direction may be computed using feature-based gaze estimators. These gaze estimators may be regression-based and/or model-based.

The regression-based methods may assume that the mapping from image features to gaze coordinates have a parametric or non-parametric form, whereas the model-based approaches compute the gaze direction from a geometric model of the eye and the positions of the cameras, light sources and object. Both types of feature-based gaze estimators may use calibration in order to increase accuracy for each individual. The response detection system may enable face detection, eye region extraction (e.g. template-matching), pupil extraction (e.g. based on elliptical/circle Hough transforms) and/or canthus detection (e.g. using heuristic based corner detection and eyelid extraction (e.g. using Active Contour Models)). The canthus and eyelids serve as bounds in which the pupil must be located. If the pupil is determined to lie outside these bounds, the response detection system may be configured to recognise the determination as an error. In this case, the gaze estimation may be terminated and restarted with the next frame. Furthermore, if the distance between bottom and top eyelid is below a threshold, the eye may be deemed to be closed. The canthus may also be used as points of reference in the face which allow rudimentary gaze detection for a relatively free, moving head.

The gaze sensor may be configured to locate one or more of the face, pupils, canthus and eyelids. The remote, non-contact sensor may comprise a temperature remote, non-contact sensor. The temperature remote, non-contact sensor may be configured to detect the temperature and/or colour of a region of interest. The region of interest may comprise the face, cheeks (e.g. a blush response), ears, forehead and/or any other body part. Detecting the colour may be performed by taking a spectrum of the radiation emitted by the subject (e.g. visible or invisible electromagnetic radiation). Detecting the colour may be performed by measuring a spectrum of the radiation reflected by the subject (e.g. visible or invisible electromagnetic radiation) and/or measuring the intensity of a particular wavelength (or range of wavelengths) of electromagnetic radiation. The source of the scattered radiation may be ambient light. The response detection system may be configured to illuminate the subject (e.g. with infrared light).

Detecting the temperature of the subject may be performed using a thermal imaging camera, an infrared thermometer, a pyrometer, a bolometer and/or a microbolometer. The response detection system may be configured to automatically detect the region of interest (or regions of interest). The regions of interest may be defined with reference to reference points located on the body (e.g. face, canthus, ears, eyes, glasses, glasses' lenses, arms, legs, limbs etc.). The eyes may used as reference points for positioning of the region of interest on the face. The eye positions may be obtained from blinking (e.g. by detecting rapid movements of the eyelids) and/or by shape recognition.

Blinking may be considered to be a spatio-temporal characteristic. The movement relating to blinking is localized in time and space (with respect to the head). This permits the blinking motion to be separated from head movement. A pre-existing blink detection algorithm may determine the second-order change in order to filter out the pixel changes attributed to head movement. The remote, non-contact sensor may be configured to analyze the temperature or colouration of the skin. The remote, non-contact sensor may comprise one or more cameras. The one or more cameras may comprise a combination of one or more of an infrared camera and a visible light camera. The region of interest may be the supraorbital vessels of the forehead. The supraorbital vessels of the forehead may be determined by employing the Hough transform in the forehead region to determine the approximate location of the vessels. An active contour method may be applied to determine the central lines of the vessels before extracting the boundaries. The periorbital region may be determined using the eye location as a point of reference to define the initial periorbital region. The initial periorbital region may be then tracked throughout the footage.

The region of interest may be the region around the maxilla and/or cheeks. The maxilla region may also be visible in people with full beards. An advantage of using the maxilla and/or cheeks may be that this region may not be covered by hair or clothing.

The eyes or lens centroids may be used as landmark points to consistently place the region of interest. The region of interest may be determined using face segmentation (e.g. by image processing). Face segmentation may comprise employing Otsu's threshold algorithm (e.g. followed by 3 dilations and 3 erosions) or other image processing techniques. Face segmentation may use a binary morphological process and/or spatial convolution. The response detection system may be configured to detect whether the subject wears glasses. If the subject is wearing glasses the centroid of each lens may be determined as a reference point.

The processing means may be configured to analyse a plurality of permutations of the adjustable parameters. These parameters may include the number of standard deviations (e.g. for the temperature threshold, h), the threshold for the duration above the threshold, τ, (e.g. τ may be between 0.5s to 3s) and/or the duration of the period from which the baseline data was extracted (e.g. baseline period may be between 5s to 19s). The threshold, h, may be based on the assumption that blushing and baseline temperatures belong to separate distributions and provides a contextual element to the calculation. The subject's internal temperatures and blush responses may be assumed to be unique.

Movement of the subject may cause reduced overlapping between regions of interest of different video frames.

The Lucas-Kanade tracker (or other optical flow estimation method) may be used to estimate the warping of the image and apply to the shape of the region of interest. Alternatively/in addition, the thermal image may be fused with an image from a video camera. Then Active Appearance Models (AAM) can be used to model the warping of the face under movement and the region of interest can simply be the triangle, in the triangulated mesh, that represents the upper cheek.

The Active Appearance Models approach may use landmarks to position the mesh. The camera may move so the subject is always directly facing the camera. Alternatively/in addition more than one camera may be used.

The response detection system may be configured to detect the lenses of glasses. In order to detect the centroid of a lens (e.g. of spectacles), a search for the largest solid ellipse, which does not extend into the background of the image, may be performed. The ellipse may be constrained to conform to a certain size, orientation and/or eccentricity.

This search may be conducted using an elliptical Hough transform or a genetic algorithm. Advantages of using a genetic algorithm may include that the number of generations can be restricted, which may reduce execution time.

Only the foreground pixels of the image may be used as candidate centroids. The genetic algorithm may use elitist selection (e.g. 15% elitist rate, retaining the best 15%) in order to preserve the best fitting ellipses in subsequent populations. Previously successful ellipse parameters (e.g. those shapes and sizes which successfully corresponded to lenses for previous subjects) may be retained to be used in subsequent lens finding.

Once the largest, solid ellipse is found, the search may be performed again on other regions of the face. If only one ellipse that conforms can be found, then the subject may be wearing glasses but may not be directly facing the camera. If two ellipses conform then the user may be deemed to be wearing glasses and the centroids may be used for region of interest placement. The subject may be deemed to not be wearing spectacles if no ellipses that conform are found. If more than two ellipses are found then the frame may be skipped. Segmenting the glasses and finding the centroids of the lenses on every frame can be omitted. The Lucas-Kanade tracker may track the initially detected centroids.

The remote, non-contact sensor may comprise a heart-beat sensor. The pulse-rate is the rate of arterial palpation of an individual's heart (heart-beat) and changes to the periodicity of this signal is often attributed to either an emotional response or a physical activity. The frequency of the human heart rate generally beats between 60 (1Hz) and 100 (1.67Hz) times per minute at rest. The heart-beat may rise when presented with visual stressors

Radar may be used to obtain heart-beat (as well as other vital signs). Heart-beat sensors may comprise active sensors. Radar radiates a microwave signal (e.g. generated by a probe beam generator) towards a target and the strength of the back-scattered signal is measured. Radar-based methods may be configured to detect Doppler shifts from the subject's movement (this includes heart-beat and respiration), which will induce a phase difference between the transmitted and received signal.

A peak in the Fourier transform spectrum (a well-known method of decomposing a signal into its sine and cosine components) of a buffered signal may lie between a frequency of 1 Hz and 2Hz. The response detection system may be configured to detect peaks within the Fourier transform of a window consisting of a number (e.g. 512) of the most recent samples.

The greatest peak within the frequency window may be deemed to correspond to the heart-beat. A measure of confidence of the greatest peak may also be determined. This confidence measure may be calculated to be the threshold of the ratio between the greatest and the second greatest peak. If the ratio exceeds a value (e.g. 1.4), then the frequency is deemed to be the heart rate. The heart-beat signal may be isolated from other movement signals within range of the sensor. The heart-beat sensor may comprise an emitter (e.g. GUNN or IMPATT diodes), a receiver (e.g. Shotkey microwave mixer diodes or Spatial Diversity Detector Array) and/or a processer. The heart-beat sensor may be configured to optimise signal quality, obstacle penetration, effective distance, package size and power consumption.

The emitter may be configured to provide waves having one or more modulations (e.g. pulsed or continuous wave), bandwidths (e.g. narrowband or wideband such as Micropower Impulse Radar), frequencies (e.g. microwaves at 2.45GHz, 5GHz, 24GHz, 35GHz, 60GHz or millimetre waves).

The receiver may comprise signal processing methods in hardware and/or software.

Passive sensors, until recently, were mainly limited to infrared (thermal) images where the thermal activity of the carotid artery was investigated. This is due to the periodic temperature variability in this region as blood is pumped around the body. However, automatic segmentation of the region of interest can be problematic at distance and with excessive movement. Furthermore, the region can be occluded by headwear, hands and other objects, so the region was segmented manually. Therefore a more controlled environment would be required for this approach to be effective.

The remote, non-contact heart-beat sensor may comprise a Laser Doppler sensor. Photoplethysmography (PPG) may also be employed for heart rate detection. In this approach, near-infrared light is focussed on an area of skin and the amount of light absorbed is dependent on the volume of blood in this region. Microwave Doppler sensing may also be employed for contactless heart rate detection.

The remote, non-contact heart-beat sensor may comprise a remote electric potential probe. These probes may permit remote electrocardiograms (ECG) and electroencephalograms (EEG). The response detection system may be configured to determine if the tracking has been lost. When tracking is lost, the feature values are being extracted from the incorrect region causing inconsistencies in the data. Many things can cause the loss of tracking, such as fast movements that have a high frequency of direction changes or occlusion.

Determining whether the tracking has been lost may be performed by computing the Euclidean distance between the newly determined template and the previous template. If loss of tracking occurs, the templates may differ greatly as they will be from different regions of the image. A simple threshold may determine this. Alternatively / in addition, some heuristic rules based on the old and new coordinates may be implemented. These rules may include a threshold on the distance between points in successive frames and the angle and distance between the two eye positions.

The output of the remote detection system may comprise statistical data (e.g. average heart rate, average temperature), subject specific data (peak heart rate for a given subject) and/or identifying information (e.g. height, eye colour, hair colour, a photograph).

The present disclosure includes one or more corresponding aspects, embodiments or features in isolation or in various combinations whether or not specifically stated (including claimed) in that combination or in isolation. Corresponding means for performing one or more of the discussed functions are also within the present disclosure.

Corresponding computer programs for implementing one or more of the methods disclosed are also within the present disclosure and encompassed by one or more of the described embodiments.

The above summary is intended to be merely exemplary and non-limiting.

Brief Description of the Figures

A description is now given, by way of example only, with reference to the accompanying drawings, in which:-

Figure 1 is a schematic diagram of a response detection system comprising a processor, a memory, a stimulation means, and a remote non-contact sensor. Figure 2a is a schematic plan view of a response detection system situated in a corridor of a building.

Figure 2b is a perspective view of the response detection system of figure 2a, from the perspective view of a subject moving through the corridor.

Figure 3 is a schematic plan view of a response detection system situated in a bus.

Figure 4 is a schematic of a person's face showing parameters used to position a region of interest.

Figure 5 is a flow chart of a procedure for determining a location of a region of interest on a subject's face.

Figure 6 shows a number of experimental images where a region of interest has been determined.

Figure 7 shows experimental results.

Figure 8 shows further experimental results. Description of Example Aspects/Embodiments

Other embodiments depicted in the figures have been provided with reference numerals that correspond to similar features of earlier described embodiments. For example, feature number 1 can also correspond to numbers 101 , 201 , 301 etc. These numbered features may appear in the figures but may not have been directly referred to within the description of these particular embodiments. These have still been provided in the figures to aid understanding of the further embodiments, particularly in relation to the features of similar earlier described embodiments. Identifying terrorists and any impending terrorist attacks is a very complex task. Attacks can occur without warning and security presence may not provide sufficient deterrence. However, deceptive behaviour may be identified by analysing psychological and physiological measurements. However, previous methods have relied on direct contact with the individual to obtain these measurements. However, these procedures may alarm a terrorist which may have disastrous consequences.

Figure 1 depicts a remote response detection system comprising at least one memory (107), at least one processor (108), at least one stimulation means (105) (which in this case comprises a display (105a) and may also comprise, for example, speakers), at least one remote, non-contact sensor (106) and at least one communication unit (103) (e.g. in communication with an antenna (102) for transmitting data, or a user interface (e.g. monitor, not shown)). The expression "remote" sensor is intended to encompass any sensor which can take measurements of the requisite physiological indicator without requiring physical contact with the subject, and more preferably encompasses any sensor which can operate over sufficient distance that its existence and/or purpose need not be readily evident to a subject and could be concealed. A typical such distance could be, for example, 1 meter or more.

Figure 2a depicts an overhead view of a response detection system such as that shown in figure 1. The system is arranged in a corridor of a building (e.g. airport, train station, underground station) through which one or more subjects (111) (e.g. commuters, travellers, workers) may pass. It will be appreciated that other embodiments may be situated outside (e.g. on the pavement) or within rooms or other enclosed spaces (e.g. in buses, trains or other vehicles; waiting rooms, lecture theatres, cinemas etc.). The stimulation means in this case comprises two display screens (105a). In this case the display screens (105a) are configured to display a non-inducing presentation such as an advertisement; and an inducing presentation such as a video clip and a textual announcement. In this example the inducing video clip comprises an anti-smuggling video clip and the inducing textual announcement comprises a general security announcement. Figure 2b illustrates the same system as in figure 2a but viewed from the perspective of subjects 111 walking through the corridor.

It will be appreciated that other embodiments may comprise one or more display screens and/or one or more audio speakers. It will be appreciated that in other embodiments the display screen may show video clips and or images of inducing objects (e.g. smuggled objects, drugs, money, weapons) and/or non-inducing objects. Inducing and/or non- inducing objects may be displayed simultaneously (e.g. in four quadrants of a screen) or sequentially. The embodiment of figure 2 also comprises a remote, non-contact temperature sensor ( 06a) and a remote, non-contact heart-beat sensor (106b).

The non-contact temperature sensor (106b) is configured to monitor a physiological indicator of a subject within the remote, non-contact sensor's field of view (126b). In this case the physiological indicator comprises the temperature of the face, e.g. by measuring blush. In this case the non-contact sensor comprises an infrared camera (e.g. with a microbolometer). It will be appreciated that for some embodiments, the sensor would be concealed from the subject. In this case, the embodiment is configured to recognise the face of the subject as a region of interest, e.g. by detecting the shape of the face or facial features such as eyes, nose and/or mouth. In order to establish a baseline temperature value, the temperature of the region of interest may be determined before the inducing presentation is displayed on the stimulation display means (e.g. when a non-inducing advertisement presentation was displayed). It will be appreciated that other embodiments may be configured to present no stimulus on the stimulation display means during the baseline period or configured not to use a baseline measurement. The temperature from each pixel corresponding to the region of interest is, in this case, determined by the remote, non- contact temperature sensor. The average (e.g. mean) temperature of the pixels making up each region of interest may be calculated in order to determine the baseline temperature.

When the data has been recorded for the baseline temperature calculation, the response detection system (101) is configured to display, on the displays (106), the inducing presentation of the inducing video clip and the inducing textual announcement. During the inducing presentation and/or within a period of the inducing presentation being presented (e.g. within 2-120 seconds), the response detection system (101) is configured to re-measure the (face) temperature of the subject using the remote, non-contact temperature sensor (105a). As when measuring the baseline temperature, the temperature from each pixel corresponding to the region of interest is first determined by the infrared camera non-contact sensor. The average (e.g. mean) temperature of the pixels making up each region of interest is then calculated in order to determine the induced facial temperature. It will be appreciated that other embodiments may be configured to determine the temperature of the region of interest over a period of time (e.g. for multiple frames). The average value in this case would encompass values for multiple frames. Other embodiments may be configured to detect the peak temperature (e.g. spatial and/or temporal). It will be appreciated that some embodiments may be configured to process multiple subjects simultaneously. The response detection system is then configured to calculate the difference in temperature between the measured baseline temperature and the measured induced temperature. The temperature difference is then compared to an absolute threshold value. It will be appreciated that other embodiments may compare the temperature difference to a relative, or proportional, threshold. For example, the relative threshold for the temperature difference could be in the range between 8% and 15%. Those subjects which exhibit a temperature change greater than the temperature change threshold value are output by the response detection system. It will be appreciated that this output may allow the authorities/security personnel to carry out further checks on those subjects.

It will be appreciated that, for other embodiments, the baseline value may be determined to be the average temperature measured for one or more subjects. It will be appreciated that, for other embodiments, the response detection system may be configured to calculate the mean and standard deviation of values obtained during the baseline period. It will be appreciated that the threshold value may be predetermined in terms of the calculated mean and standard deviation (e.g. the threshold may be predetermined to be one standard deviation above the mean). The embodiment of figure 2 also comprises a remote, non-contact heart-rate sensor (106a) configured to monitor the heart rate of a subject within the heart-rate sensor's field of view (126a). The remote heart rate sensor (106a), in this case, comprises a Doppler sensor (e.g. a 2.45GHz microwave sensor). In this case, the response detection system comprises a probe generator (109a), configured to generate a directional beam (129a) of microwaves. The directional beam of microwaves are configured to be produced by the microwave probe generator, be reflected by the subject and detected by the heart-beat remote, non-contact sensor (106a). The heart-beat remote, non-contact sensor (106a) is, in this case, configured to detect the intensity of the reflected microwave radiation as a function of time. The response detection system is configured to process this temporal data (e.g. using a Fourier transform) to produce the corresponding frequency spectrum. The subject's heart-rate is then, in this case, estimated from the frequency spectrum.

Similarly as for the temperature determination, this embodiment is configured to determine a baseline value for the heart-rate before the inducing presentation is shown on the displays (105a), and an induced heart-rate value is determined when or after the inducing presentation is shown.

The heart-rate value difference (between the baseline value and the induced value for each subject) is then determined by the response detection system (101). The heart-rate value difference is then compared against a threshold heart-rate value difference. In this case, the threshold heart-rate value difference could be 8%. Subjects with heart-rate value differences above the heart-rate value difference threshold are output by the response detection system.

For this embodiment, as described above, subjects with either a heart-rate value difference above the heart-rate value difference threshold or a temperature change greater than the temperature change threshold value are output by the response detection system. It will be appreciated that other embodiments may output only subjects where both the heart rate and the temperature exceed their respective thresholds.

Figure 3 depicts a second embodiment (201) of a response detection system. This embodiment, unlike the first embodiment which is configured to detect heart rate and gaze direction, is configured to determine temperature and gaze direction. Unlike the first embodiment, which was configured to detect the response of subjects as the moved through a corridor of a building, this embodiment is configured to detect the response of subjects (211) as they sit in a bus. This embodiment comprises stimulation means comprising a speaker (205b) and a display screen (205a). This embodiment also comprises a remote, non-contact sensor (206c), which in this case, comprises a microbolometer based infrared camera.

The remote, non-contact sensor, in this case, is an Argus P7225 from e2v, which is a LWIR (long wavelength infrared) camera and operates near the plateau of the blackbody curve for temperatures around that of a human body. It will be appreciated that a MWIR (medium wavelength infrared) camera may also be used. In this case the remote, non- contact sensor is used to determine both the gaze direction and the temperature of each subject. Unlike the previous embodiment, in which the entire face provided a region of interest, the regions of interest, for this embodiment, are positioned on the cheek below the eyes.

To determine the position of the region of interest, this embodiment is configured to recognise reference points on the face and determine the position of the regions of interest with reference to these reference points (a flow chart of the procedure is shown in figure 5). In this embodiment, the eyes (or if the person is wearing glasses, the lenses of the glasses) serve as reference points from which the regions of interest are positioned. This embodiment is configured to determine the initial eye locations when a subject blinks. If the subject is wearing glasses, the lens centroid locations are determined by searching for an ellipse of a certain size and/or shape.

After determining the initial eye/lens locations, this embodiment is configured to track the eyes/lenses through subsequent frames. It will be appreciated that the eye location, in this embodiment, is used to determine gaze direction (as well as providing a reference point). It will be appreciated that other embodiments may determine, independently for each frame, the set of coordinates representing the eye/lens positions.

This embodiment is configured to position templates (e.g. of a certain shape and size) with respect to the determined eye positions (obtained from the image). These templates are used, in this embodiment, for tracking by the Lucas-Kanade tracker. In this embodiment, the template has a width and height equal to the distance between the two eyes (e.g. to minimise the movement during blinks). Using the eye position, the device is configured to also determine the gaze direction. Using the gaze direction, the embodiment is configured to detect whether or not the subject has looked at the inducing display, and whether the subject has exhibited gaze aversion behaviour. In this case, if it is determined that the subject has not looked at the inducing presentation, the subject is not output by the response detection system.

This embodiment is configured to determine the location of the region of interest (341) using the determined location of the eyes. A schematic diagram of some of the relevant parameters is depicted in figure 4. This embodiment is configured to produce a set of reference point coordinates for both eyes using the remote, non-contact sensor (226c). Firstly, the angle (from image horizontal), Θ, and the distance, D, between both eyes is calculated using the equations:

where E _R ^X and E _R ^y are the x and y coordinates of the right eye as measured on the image; and

E _L ^X and E _L ^y are the x and y coordinates of the left eye as measured on the image.

It will be appreciated that for other embodiments, other features may be detected in order to establish reference points (e.g. mouth, head, nose).

In this case the angle perpendicular to θ, θ _ρ , (pointing down from the eyes) is then calculated by adding π/2, using the equation:

^P 2

From these equations, this embodiment (201) is configured to determine the centre of the region of interest by moving along the perpendicular from each eye. These positions are determined using the equations:

C = E _S ^X + n cos 0 _p , c = E ^> + wsin0 _p ,

where C _s ^x and C _s ^y are the x and y coordinates of the region of interest centre;

s is used to denote whether left or right eye; and

n is the distance to the centre of the region of interest from the eye (e.g. 0.75*D/2). It will be appreciated that n may be scaled to the inter-eye distance to take into account the size of the face. It will be appreciated that an absolute value (e.g. 3cm) or some other relative value (e.g. 0.1*the subject's face height) may be used

It will be appreciated that for other embodiments, the region of interest may be positioned differently with respect to the reference points. The region of interest is, in this case, an oriented, square window. This may provide robustness to roll rotation (tilt head either side) as this has been identified (empirically) as the most common rotation as most subjects tended to keep their eyes on the screen. The coordinates contained within the region of interest are calculated using:

where d and k range from 0 to sz;

sz is the dimension size of the window (e.g. 15 pixels, although to make scale invariant the size may be a factor of the distance between the eyes); and

P _s ^x and P _s ^y are the x and y coordinates of a pixel location belonging to s region of interest (s denotes left or right).

It will be appreciated that for other embodiments, the region of interest may be of a different size and shape.

This algorithm may provide a means to produce reproducible regions of interest from images obtained by both video and thermal images from infrared cameras. Therefore it can also be used to segment a region of interest for determining change in coloration. It will be appreciated that other image processing techniques may be used.

A selection of screen dumps, as a result of the region of interest segmentation algorithm, is provided in Figure 6. In each image, the eyes have been detected. The construction lines indicating the line between the eye centres and running perpendicular to the eye centres are shown in white. The determined regions of interest are shown as white squares.

Once the region of interest has been segmented, this embodiment is configured to determine the mean temperature value using the equation:

where:

l(x,y) is the grayscale value/temperature at the pixel located at (x, y);

{P _s ^x , P _s ^y } are the set of coordinates belonging to s region of interest; N is the total number of pixels in the region of interest; and

μ ₅ (τ) is the mean value of s region of interest (s denotes left or right) for frame f.

The mean value (e.g. of temperature) may be more robust to slight transformations of the region of interest (than, for example, peak value).

The mean of both regions of interest, in this embodiment (201), is determined for each frame of the video. As the base level temperature for each subject will vary, the embodiment employs an initial period from which baseline values can be determined. During the baseline period, the mean, tandard deviation,

of each region of interest for all frames within the baseline period is calculated. For all subsequent frames, the mean value obtained from each region of interest is normalized using z-score normalization:

where f _b is the number of frames in the baseline period.

This embodiment (201) assumes that the temperatures during the baseline period belong to one distribution and the temperatures during a blush response belong to another. The threshold, h, in this case, is based on the number of standard deviations from the baseline mean (e.g. two standard deviations above the baseline mean). However, a blush response may not finish as soon as the inducing presentation has completed. Blushing may occur up to 2 seconds after presentation of the stimulus and may last up to 15 minutes. The median duration of the blush response may be intermediate between these extremes (e.g. 20 seconds). In this embodiment (201), the temperature of the regions of interest is determined for a period of time. The detected blush temperature is compared to a threshold value using the equation: Where dT is the time (continuous) above the standard deviation threshold (z _s (f) > h) for either the left or the right region of interest; and

τ is the duration threshold. In this case, the duration threshold could be in the range between 0 and 20 seconds.

At the same time as determining the temperature and blush response, this embodiment (201) is configured to determine the gaze direction as a function of time, using the remote, non-contact sensor (206c). This embodiment is configured to determine, using the eye locations (determined as described above), where the subject is looking. When the inducing stimulus is presented (via the speakers (205b) and the display screen (205a)), the remote detection system (201) is configured to track the eye positions as a function of time (with respect the initiation of the inducing presentation). This embodiment is configured to determine the rate of change of gaze direction (e.g. how quickly the subject (211) looks away).

It will be appreciated that other embodiments may be configured to determine the direction of gaze direction change (e.g. whether he looks up, down right or left) and/or magnitude of gaze direction change. The gaze-direction threshold value in this case is pre-determined to select the top 2% of subjects ordered by rate of change of gaze direction.

This embodiment is configured to output those subjects which are above the gaze- direction value and above the temperature threshold value. The output in this case is a photograph taken by the camera of the subjects exceeding the thresholds.

It will be appreciated that in other embodiments the stimulation (display) means may present a non-inducing clip (or other non-inducing stimulation). It will be appreciated that in other embodiments, when the subject's gaze is determined to be directed towards the centre of the screen (e.g. by the thermal camera (206c)), an inducing presentation of inducing objects (drugs) will be briefly presented at the corner of screen (e.g. for between 50 and 150 ms).

It will be appreciated to the skilled reader that any mentioned apparatus/device/server and/or other features of particular mentioned apparatus/device/server may be provided by apparatus arranged such that they become configured to carry out the desired operations only when enabled, e.g. switched on, or the like. In such cases, they may not necessarily have the appropriate software loaded into the active memory in the non- enabled (e.g. switched off state) and only load the appropriate software in the enabled (e.g. on state). The apparatus may comprise hardware circuitry and/or firmware. The apparatus may comprise software loaded onto memory. Such software/computer programs may be recorded on the same memory/processor/functional units and/or on one or more memories/processors/functional units.

It will be appreciated that the any mentioned apparatus/circuitry/elements/processor may have other functions in addition to the mentioned functions, and that these functions may be performed by the same apparatus/circuitry/elements/processor. One or more disclosed aspects may encompass the electronic distribution of associated computer programs and computer programs (which may be source/transport encoded) recorded on an appropriate carrier (e.g. memory, signal).

It will be appreciated that any "computer" described herein can comprise a collection of one or more individual processors/processing elements that may or may not be located on the same circuit board, or the same region/position of a circuit board or even the same device. In some embodiments one or more of any mentioned processors may be distributed over a plurality of devices. The same or different processor/processing elements may perform one or more functions described herein.

With reference to any discussion of any mentioned computer and/or processor and memory (e.g. including ROM, CD-ROM etc), these may comprise a computer processor, Application Specific Integrated Circuit (ASIC), field-programmable gate array (FPGA), and/or other hardware components that have been programmed in such a way to carry out the inventive function.

The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole, in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that the disclosed aspects/embodiments may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the disclosure. While there have been shown and described and pointed out fundamental novel features of the invention as applied to preferred embodiments thereof, it will be understood that various omissions and substitutions and changes in the form and details of the devices and methods described may be made by those skilled in the art without departing from the scope of the invention as defined by the claims. For example, it is expressly intended that all combinations of those elements and/or method steps which perform substantially the same function in substantially the same way to achieve the same results are within the scope of the invention. Moreover, it should be recognized that structures and/or elements and/or method steps shown and/or described in connection with any disclosed form or embodiment of the invention may be incorporated in any other disclosed or described or suggested form or embodiment as a general matter of design choice. Furthermore, in the claims means-plus-function clauses are intended to cover the structures described herein as performing the recited function and not only structural equivalents, but also equivalent structures. Experiments

For preliminary experiments, the Argus P7225 from e2v was employed as a remote, non- contact sensor. This is a LWIR camera and operates near the plateau of the blackbody curve for temperatures around that of a human body. As it operates near the plateau, a small change in temperature will have less affect on the spectral radiance than that of a MWIR camera, which operates on a steep ascent. Therefore, MWIR cameras are better for measuring small changes where LWIR cameras are commonly used for firefighting activities. One reason for this is because the dynamic range automatically readjusts to the range of temperatures within the scene, which facilitates easy segmentation of extremely hot objects. LWIR cameras are also much cheaper than the MWIR cameras and it would be advantageous if the cheaper cameras can also detect changes in facial temperature.

A. Experimental Setup As the Argus P7225 is a microbolometer based camera, all temperatures within the field of view are used to determine the range. The camera then readjusts to the new range every so often. Therefore in the experiments it was paramount to ensure that no external heat sources can enter the field of view. This step is essential as the camera used only outputs image data and not temperature data. If the range changes too much, the grayscale values will not represent the same temperature as before. It is assumed that the grayscale values represent some unknown range of temperatures, which change when the range readjusts. In order to test the proposed algorithm, the technique was implemented in conjunction with a psychophysiological experiment. The psycho-physiological experiment attempted to determine the act of deceit by using visual stimuli to elicit a response. Prior to the experiment, individuals decided, in secret, whether they would try to deceive the system by concealing a banned object. In an attempt to elicit a response, the subject watched a video, which initially displayed calming images and audio (in order to obtain baseline values) prior to presenting images of many banned objects (the stressors), while the facial temperature was monitored throughout. Incentives and disincentives were also provided in an attempt to ensure the subjects made concerted efforts of deception and increased the emotional attachment to the object they were concealing.

It is hypothesized that subjects, who chose to conceal a banned object, will have an emotional attachment to an image of the object and potentially to the threat of the disincentive if they fail. At the point at which these stressors are presented, it is also hypothesized that the emotional response attributed to these stressors will manifest as an increase in facial temperature.

B. Results

For the experiments, there were 53 subjects from which 27 did not try to smuggle whereas 26 did. The subjects did not reveal whether they had attempted to smuggle until after the experiment. The analysis was performed off line, although it can be adapted to be performed during the test.

The analysis consisted of a multitude of permutations of the adjustable parameters. These parameters included the number of standard deviations (h - from 0.5 to 10 standard deviations) for the temperature threshold, the threshold for the duration above the temperature threshold (t - 0.5s to 3s) and the duration of the period from which the baseline (blp - 5s to 19s) data was extracted. The threshold, h, is based on the assumption that blushing and baseline temperatures belong to separate distributions and provides a contextual element to the calculation. This is because the subject's internal temperatures and blush responses are unique. In the event that the assumption seems wrong, additional tests were conducted whereby the values obtained during stimuli presentation were not z-score normalized but simply mean adjusted to the baseline mean. An incrementing threshold on the positive deviation from the mean was used to determine blushing.

The optimal receiver operating characteristic (ROC) curve, for both the z-score and mean adjusted threshold, is presented in Figures 7 and 8. Figure 7 is the ROC curve obtained when 15 seconds of baseline values were used and a continuous duration above the threshold of 2 seconds. Figure 8 is the result of extracting 10 seconds of values to be used as baseline and a continuous duration above the threshold of 2.5 seconds.

Both ROC curves show that the results lay on the preferred side of the linear function x=y (as depicted on the graphs). This indicates that the classifier provides better classification than random selection. The optimum threshold from Figure 7 was 3 standard deviations, which resulted in ~77% true positive rate (TPR) and ~60% true negative rate (TNR). This gives an accuracy of the classifier as ~67%. This is slightly better than the optimal results from Figure 8 (-65% accuracy).

The accuracy is encouraging as it is better than expected. Previous trials with the P7225 thermal camera indicated that blushing can be detected, although this was when blushing was elicited through embarrassment. This resulted in a strong response.