SMITH, Norman, Ronald (Vision RT Limited, Daws House33-35 Daws Lane, London NW7 4SD, GB)
RUTO, Anthony, Christopher (Vision RT Limited, Daws House33-35 Daws Lane, London NW7 4SD, GB)
MEIR, Ivan, Daniel (Vision RT Limited, Daws House33-35 Daws Lane, London NW7 4SD, GB)
SMITH, Norman, Ronald (Vision RT Limited, Daws House33-35 Daws Lane, London NW7 4SD, GB)
RUTO, Anthony, Christopher (Vision RT Limited, Daws House33-35 Daws Lane, London NW7 4SD, GB)
CLAIMS:
1. A method of monitoring a patient comprising: obtaining data representative of a measurement indicative of movement of at least part of a patient during at least one breathing cycle; generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data; obtaining further data representative of measurement indicative of position of the at least part of a patient during a subsequent period of breathing; modelling a representation of the position of the at least part of a patient using the generated breathing model; and comparing the obtained further data with the modelled representation of the position of the at least part of a patient.
2. The method of claim 1, wherein obtaining data representative of a measurement indicative of movement of at least part of a patient during at least one breathing cycle and obtaining further data representative of measurement indicative of position of the at least part of a patient during a subsequent period of breathing comprises: obtaining images of the at least part of a patient; and processing the obtained images to determine the position of at least part of a patient appearing in the obtained images.
3. The method of claim 2, further comprising: placing a marker on the part of a patient to be monitored, wherein processing the obtained images to determine the position of at least part of a patient appearing in the obtained images comprises processing obtained images to determine the location of the marker appearing in the images.
4. The method of claim 3, further comprising: obtaining timing data indicative of the relative times at which the images of the at least part of a patient are obtained; wherein generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data comprises generating a model indicative of the motion of said marker during the at least one breathing cycle.
5. The method of claim 4, wherein generating a model indicative of the motion of said marker during the at least one breathing cycle comprises generating a model of the variation in timing and position of said marker, wherein said model is generated using a component analysis model selected from a principal component analysis model and an independent component analysis model.
6. The method of claim 2, wherein processing the obtained images to determine the position of at least part of a patient appearing in the obtained images comprises: processing obtained images to generate a 3D wire mesh model of the surface of the at least part of a patient appearing in the obtained images; and utilising the generated 3D wire mesh model to determine the locations of one or more points on the surface of the patient.
7. The method of claim 6, further comprising: obtaining timing data indicative of the relative times at which the images of the at least part of a patient are obtained; wherein generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data comprises generating a model indicative of the locations of one or more points on the surface of the patient during the at least one breathing cycle.
8. The method of claim 7, wherein generating a model indicative of the locations of one or more points on the surface of the patient during the at least one breathing cycle comprises generating a model of the variation in timing and position of said locations of one or more points on the surface of the patient, wherein said model is generated using a component analysis model selected from a principal component analysis model and an independent component analysis model.
9. The method of claim 6, further comprising: obtaining timing data indicative of the relative times at which the images of the at least part of a patient are obtained; wherein generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data comprises generating a model indicative of the instantaneous motion of one or more points on the surface of the patient during the at least one breathing cycle.
10. The method of claim 1, wherein generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data comprises: processing obtained data to divide the data into data associated with different breathing cycles; and generating a breathing model of the variation in position of the at least part of the patient during the course of a breathing cycle.
11. The method of claim 1 , wherein obtaining further data representative of measurement indicative of position of the at least part of a patient during a subsequent period of breathing comprises obtaining further data representative of measurement indicative of position of the at least part of a patient during a complete breathing cycle, wherein generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data comprises generating a model of the variation in position of the at least part of a patient for a complete breathing cycle; and comparing the obtained further data with the modelled representation of the position of the at least part of a patient comprises comparing the extent to which obtained further data matches a representation of the position of the at least part of a patient for a complete breathing cycle.
12. The method of claim 1, wherein obtaining further data representative of measurement indicative of position of the at least part of a patient during a subsequent period of breathing comprises obtaining further data representative of a single measurement indicative of position of the at least part of a patient, wherein generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data comprises generating a model of the variation in position of the at least part of a patient for a complete breathing cycle; and comparing the obtained further data with the modelled representation of the position of the at least part of a patient comprises comparing the extent to which obtained further data matches a representation of the position of the at least part of a patient for a part of a breathing cycle.
13. The method of claim 1, wherein comparing the obtained further data with the modelled representation of the position of the at least part of a patient comprises: generating a distance measure indicative of the extent to which the representation of the position of the at least part of a patient using the generated breathing model corresponds to the obtained further data; and comparing the generated distance measure with a threshold.
14. The method of claim 13, further comprising generating a signal to cause a treatment apparatus to interrupt treatment if said distance measurement exceeds said threshold.
15. The method of claim 1, wherein generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data comprises generating a breathing model indicative of the manner in which the at least part of the surface of a patient varies during a breathing cycle, wherein comparing the obtained further data with the modelled representation of the position of the at least part of a patient comprises: determining a surface representative of the at least part of the patient using the obtained further data; comparing the determined surface with a surface generated using the breathing model; and identifying whether a patient has moved by determining a transformation required to match the generated and determined surfaces.
16. The method of claim 15, further comprising generating a signal to cause a mechanical couch to reposition a patient on the basis of transformation determined to match the generated and determined surfaces.
17. A computer readable medium storing computer interpretable instructions which when executed by a programmable computer cause the computer to: obtain data representative of a measurement indicative of movement of at least part of a patient during at least one breathing cycle; generate a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data; obtain further data representative of measurement indicative of position of the at least part of a patient during a subsequent period of breathing; model a representation of the position of the at least part of a patient using the generated breathing model; and compare the obtained further data with the modelled representation of the position of the at least part of a patient.
18. A patient monitor comprising: a 3D position determination module operable to determine measurements indicative of positions of at least part of a patient; a model generation module operable to generate a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using measurements determined by said 3D position determination module; and a comparison module operable to utilise a breathing model generated by said model generation module to model a representation of the position of the at least part of a patient and compare a measurement of the position of the at least part of a patient determined by said 3D position determination module with a modelled representation.
19. The patient monitor of claim 18, further comprising: a clock module operable to associate measurements indicative of positions of at least part of a patient determined by said 3D position determination module with timing data, wherein said model generation module is operable to generate a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using measurements determined by said 3D position determination module and said timing data.
20. The patient monitor of claim 18, further comprising: a phase detection module operable to process measurements indicative of positions of at least part of a patient obtained by said 3D position determination module to associate measurements with different breathing cycles, wherein said model generation module is operable to generate a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using measurements determined by said 3D position determination module associated with different breathing cycles by said phase detection module.
21. The patient monitor of claim 18, further comprising: a stereoscopic camera operable to obtain images of a patient to be monitored; wherein said 3D position determination module is operable to process image obtained by said stereoscopic camera to determine measurements indicative of positions of at least part of a patient being monitored. |
PATIENT MONITOR
The present invention relates to patient monitoring. More particularly, embodiments of the present invention relate to modelling breathing in order to detect breathing abnormalities (e.g. coughing) and also to enable the movement of patients to be detected. The invention is particularly suitable for use with radio therapy devices and computed tomography (CT) scanners and the like where detection of patient movement or irregular breathing is important for successful treatment.
Radiotherapy consists of projecting onto a predetermined region of a patient's body, a radiation beam so as to destroy or eliminate tumours existing therein.
Such treatment is usually carried out periodically and repeatedly. At each medical intervention, the radiation source must be positioned with respect to the patient in order to irradiate the selected region with the highest possible accuracy to avoid radiating adjacent tissue on which radiation beams would be harmful.
When a tumour is located in the thoracic or abdominal chest cavities, the position of a tumour can vary considerably (e.g. by as much as a few centimetres) throughout the breathing cycle. In order to obtain reliable CT scanning data it is therefore important to obtain data consistently at the same point within a breathing cycle.
Certain modern CT scanners are able to process CT scanning data acquired at
different times within a breathing cycle to generate a representation of the 3D movement of internal organs during breathing. Such "4D" representations of organ movement are invaluable in determining a course of treatment for irradiating a cancer. Again in order for quality planning data to be generated, the timing within the breathing cycle when individual CT scans are acquired must be known so that 4D representation accurately represents the movement of the internal organs.
When applying radiation to a patient, the gating of treatment apparatus should be matched with the breathing cycle so that radiation is focused on the location of a tumour and collateral damage to other tissues is minimised. If movement of a patient is detected the treatment should be halted to avoid irradiating areas of a patient other than a tumour location. Also if irregular breathing such as coughing is detected, treatment should be halted as such irregular breathing may cause a tumour to change location relative to the focus of a radiation beam.
For this reason a number of monitoring systems for monitoring a patient's breathing during radiotherapy have therefore been proposed. Thus for example, US 6621889 discloses a system in which the gating of a radiation apparatus is linked to a determination that the periodicity of a patients breathing has deviated from an earlier measurement. Further examples are US 6980679 and US 7123758 which disclose a system for monitoring breathing of infants and reporting irregularities based on the periodicity of a patients breathing deviating from an
earlier measurement.
An improved system for monitoring and measuring patient breathing and movement is, however desirable.
In accordance with one aspect of the present invention there is provided a method of monitoring a patient comprising: obtaining data representative of a measurement indicative of movement of at least part of a patient during at least one breathing cycle; generating a breathing model of the variation in position of the at least part of a patient during the at least one breathing cycle using the obtained data; obtaining further data representative of measurement indicative of position of the at least part of a patient during a subsequent period of breathing; modelling a representation of the position of the at least part of a patient using the generated breathing model; and comparing the obtained further data with the modelled representation of the position of the at least part of a patient.
In accordance with another aspect of the present invention there is provided a patient monitor comprising: a 3D position determination module operable to determine measurements indicative of positions of at least part of a patient; a model generation module operable to generate a breathing model of the variation in position of the at least part of a patient during the at least one
breathing cycle using measurements determined by said 3D position determination module; and a comparison module operable to utilise a breathing model generated by said model generation module to model a representation of the position of the at least part of a patient and compare a measurement of the position of the at least part of a patient determined by said 3D position determination module with a modelled representation.
Further aspects and embodiments of the present invention will become apparent with reference to the following description and accompanying drawings in which: Figure 1 is a schematic diagram of a patient monitor in accordance with a first embodiment of the present invention;
Figure 2 is a flow diagram of the processing the processing undertaken by the patient monitor of Figure 1 to generate a breathing model for detecting irregular breathing; Figure 3 A-B are a diagram of an example of a breathing signal and a sectioned breathing signal generated by the patient monitor of Figure 1;
Figure 4 is a flow diagram of the processing involved in dividing a breathing signal into a number of sections;
Figure 5 is a flow diagram of the processing of a sectioned breathing signal to generate a breathing model;
Figure 6 is a flow diagram of the processing undertaken by the patient monitor of Figure 1 to identify a period of irregular breathing using a generated
breathing model;
Figure 7 is a flow diagram of the processing involved in comparing a breathing signal with a stored breathing model;
Figure 8 is a flow diagram of the processing the processing undertaken by the patient monitor of a second embodiment to generate a breathing model for detecting irregular breathing and patient movement; and
Figure 9 is a flow diagram of the processing involved in comparing a breathing signal with a stored breathing model in the patient monitoring system of Figure 8.
First Embodiment
Figure 1 is a schematic diagram of a patient monitor in accordance with a first embodiment of the present invention. In accordance with this embodiment, there is provided a stereoscopic camera 10 that is connected by wiring 12 to a computer 14. The computer 14 is also connected to treatment apparatus 16 such as a linear accelerator for applying radiotherapy or an x-ray simulator for planning radiotherapy. A mechanical couch 18 is provided as part of the treatment apparatus upon which a patient 20 lies during treatment. The treatment apparatus 16 and the mechanical couch 18 are arranged such that under the control of the computer 14 the relative positions of the mechanical couch 18 and the treatment apparatus 16 may be varied, laterally, vertically, longitudinally and rotationally.
Provided on the surface of the patient 20 is a marker 22.
In use, the cameras of the stereoscopic camera 10 obtain video images of a patient 20 lying on the mechanical couch 18. These video images are passed via the wiring 12 to the computer 14. The computer 14 then processes the images of the marker 22 on the patient 20 to generate a model of the patient's breathing. Subsequently the patient's breathing is monitored and compared with the stored model and if abnormal breathing is detected, the treatment apparatus 16 is shut off until normal breathing resumes.
In order for the computer 14 to process images received from the stereoscopic camera 10, the computer 14 is configured by software either provided on a disk 23 or by receiving an electrical signal 24 via a communications network into a number of functional modules 25-36. It will be appreciated that the functional modules 25-36 illustrated in Figure 1 are purely notional in order to assist with the understanding of the working of the claimed invention and may not in certain embodiments directly correspond with blocks of code in the source code for the software. In other embodiments the functions performed by the illustrated functional modules 25-36 may be divided between different modules or may be performed by the re-use of the same modules for different functions.
In this embodiment, the functional modules 25-36 comprise: a 3D position determination module 25 for processing images received from the stereoscopic camera and determining a breathing signal on the basis of a detected 3D position for the marker 22; a clock module 26 for providing a timestamp for portions of
breathing signals determined by the 3D position determination module 25; a signal cache for storing time stamped breathing signals; a phase detection module 30 for processing the signals stored in the signal cache to divide the signals into individual breathing cycles; a model generation module 32 and breathing model store 34 for generating and storing a principal component analysis breathing model; and a signal comparison module 36 for comparing received breathing signals with a stored breathing model to identify abnormal breathing.
In use, as images are obtained by the stereoscopic camera 10, these images are processed by the 3D position determination module 25 which determines the 3D position for the position of the marker 22. The 3D position determination module
25 then converts this 3D position signal into a distance measure indicating the relative distance of the marker 22 from a fixed reference point. This distance data is then stored in the signal cache 28 together with timing data indicating the timing at which the image of the marker 22 was obtained.
During a training phase, after distance and timing data for a number of breathing cycles has been stored in the signal cache 28, this data is passed to the phase detection module 30 which divides the stored data into a number of sections where each section corresponds to an individual breathing cycle. Data representing these individual breathing cycles is then passed to the model generation module 32 which proceeds to generate a principal component analysis
model of the usual variation of the patient's breathing which is stored in the breathing model store 34.
In contrast during a monitoring phase, the phase detection module 30 monitors the data stored in the signal cache 28 and as soon as data corresponding to a complete breathing cycle has been stored, the phase detection module passes the data to the signal comparison module 36 which compares the signal with the principal component analysis breathing model stored in the breathing model store 34.
Due to the manner in which the principal component analysis breathing model stored in the breathing model store 34 is generated, the model will identify whether the latest detected breathing cycle corresponds to a breathing cycle within the usual variation of normal breathing for the patient 20. If the stored principal component analysis breathing model is unable to model the latest detected breathing cycle, this indicates that the patient's breathing is abnormal. In this embodiment, if the signal comparison module 36 detects two consecutive abnormal breathing cycles, the signal comparison module 36 sends a signal to the treatment apparatus 16 to interrupt treatment until two consecutive normal breathing cycles are detected. In this way in the event that the patient 20 begins coughing during treatment, the treatment is halted until the patient begins breathing normally once again.
Generation of a breathing model
The processing performed by the patient monitor of Figure 1 to generate a breathing model for detecting irregular breathing will now be described in detail with reference to Figures 2 -5.
Referring to Figure 2 is a flow diagram of the processing undertaken by the processing patient monitor of Figure 1, when a breathing model is to be generated, initially (S2-1), the stereoscopic camera 10 captures images of marker 22 on the patient 20.
These images are then passed to the 3D position determination module 25 which processes (S2-2) the obtained stereoscopic images to identify the location of the marker 22 in the images. The location of the marker 22 in a pair of images is then processed together with stored data regarding the relative location of the cameras in the stereoscopic camera in a conventional way to convert the differences in position of a marker in a pair of captured images into an instantaneous measurement of the 3D location of the marker 22.
The 3D position determination module 25 then compares this 3D location with a reference location and calculates the distance between the two. In this embodiment, the reference location corresponds to a reference position somewhere within the chest cavity of the patient 20. The 3D position determination module 25 then passes the calculated distance measurement to the
signal cache 28 where it is stored (S2-3) together with a timestamp obtained from the clock 26 indicating the time at which the images used to generate the distance measure where obtained by the stereoscopic camera 10.
Figure 3A is an exemplary illustration of a plot of timing and position data for a period of monitored breathing such as may be stored by the 3D position determination module 25 in the signal cache. It will be noted that although the signal is approximately periodic, a typical signal will vary both due to the relative extent of inhale and exhale for an individual breathing cycle and the duration of individual breathing cycles. Further the overall shape of data for breathing cycles will vary as a patient takes individual breaths in slightly different ways. Due to this natural variation in breathing, simple detectors based on detecting variations in the periodicity of breathing can be unsatisfactory as variations in periodicity can arise even when breathing is relatively regular.
When position and timing data for a number of breathing cycles such as is illustrated in Figure 3 A has been stored within the signal cache 28, the phase detection module 30 then proceeds to process (S2-4) the stored signal data to divide the signal data into a number of sections where each section corresponds to an individual breathing cycle so that a breathing model of the normal variations of breathing as exemplified by the stored signal can be generated (S2-5).
The processing undertaken by the phase detection module 30 to divide a stored breathing signal into sections corresponding to individual breathing cycles such as
is illustrated in Figure 3B will now be described in detail with reference to Figure 4.
Phase Detection
Referring to Figure 4 which is a flow diagram of the processing involved in dividing a breathing signal into a number of sections, once position and timing data for a number of breathing cycles have been stored in the signal cache 28, the phase detection module 30 divides the signal into section representing individual breathing cycles by processing the stored signal to identify points corresponding to the middle and beginning/end of each breathing cycle. These points can be identified by determining the maximum and minimum position values associated with a breathing cycle which will correspond to points of maximum inhale and exhale respectively.
Due to the variability of the extent of inhale and exhale between different breathing cycles, this identification cannot be reliably achieved directly through the value of a position measurement itself. The maxima and minima can, however, be identified by identifying the turning points in the signal. In order to do so, it is, however, necessary to be able to distinguish between true maxima and minima and local maxima and minima which can arise due to noise or general irregularity in breathing.
To this end into order to identify points in a breathing signal corresponding to maximum inhale and the end of exhale, initially (S4-1) the signal is matched to a quadratic curve. In this embodiment, the signal is matched to a Legendre quadratic, which is a quadratic curve defined on the interval [-1, 1] and having the form A0+ x Al + (x A 2-l/3) A2. In the case of such a curve the three basis functions {1, x, x λ 2-l/3} have the property that they are orthogonal with respect to integration over this interval. In general however these basis functions will not be orthogonal when using a discrete version of the integration, for example a sum defined over the corresponding range. In this case, a correction needs to be computed for each new set of sampling x values, yielding a curve of the form A0+ x Al + (x λ 2-t) A2, where t is the correction to be estimated and the three basis functions { 1 , x, x A 2-t} are orthogonal with respect to the sum chosen.
The fitting to signal data (xl, yl) can be achieved using the standard technique of solving an over-determined system of equations A0+ xi Al + (xi A 2-t) A2 = yi for the coefficients AO, Al and A2, noting that the orthogonality of {1, x, x A 2-t} means that the system will be very well conditioned and hence resistant to noise and round-off errors
Once the breathing signal has been matched with a suitable Legendre quadratic all the stored data points for a breathing signal can then be classified (S4-2) based on the behaviour of the matched quadratic. More specifically, each of the stored
items of position and timing data in the signal cache 28 can be classified as either increasing if:
a) the corresponding curve is increasing over a range centred the point being considered;
b) the corresponding curve has a maximum over the range centred on the point being considered which lies to the right of the centre of the range; or
c) the corresponding curve has a minimum over the range centred on the point being considered which lies to the left of the centre of the range.
The remaining points are then classified as decreasing.
Signals corresponding to maxima and minima can then be identified (S4-3) using these classifications. In the case of a stored signal, this can be achieved by considering the classifications of sequences of point and identifying the points identified with a change of classification. Maxima can then be identified as those points where classification changes increasing to decreasing and minima identified where classifications change from decreasing to increasing.
Alternatively, potential maxima and minima can be identified merely through a change in classification compared with immediately preceding points. Such an alternative system would be particularly suitable for classification of points in real time where signals are processed as soon as they are stored in the signal cache. In order to avoid detection of mere local maxima and minima the detection of
maxima and minima can be limited to points where a change in classification occurs and at least three consecutive points following a detected maxima or minima maintain the same classifications.
Having processed the breathing signal data in this way, the breathing signal data can be divided (S4-4) into data corresponding to a series of breathing cycles by dividing the signals where ever a minimum signal is detected. The result of such a division of the data is illustrated in Figure 3B. The division of a breathing signal representing normal breathing in this way provides data representing a series of exemplary breathing cycles which is then utilised to generate (S2-5) a breathing model as will now be described in detail.
Generation of a principal component analysis breathing model
Turning to Figure 5 which is a flow diagram of the processing of a sectioned breathing signal to generate a breathing model, when the phase detection module 30 has sectioned a breathing signal into a series of samples each corresponding to a single breathing cycle each sample is processed in turn by the model generation module 32. The absolute timing data stored in the signal cache 28 is first converted into a set of relative timing measurements by subtracting the least timing data associated with a particular breathing cycle from all of the items of timing data for that cycle.
The identified sections in a breathing signal normally contain sections of varying lengths (periods) and as a result contain different numbers of data points. So that a suitable model can be generated, each of the sectioned signals is then resampled (S5-1) into a series of bins each associated with a portion of a complete breathing cycle.
In order to achieve this, first of all a suitable bin size is selected. In this embodiment, this is selected by considering the average number of data points associated with each of the breathing cycles for which data has been stored. For any particular breathing signal each of the stored items of position and timing data is the associated with a particular bin based on the relative position of the item of data within the breathing cycle. Thus for in the case of a breathing sample being re-sampled into 100 bins, the timing and position data associated with the start of a breathing cycle will be associated with the first bin and timing and position data associated with the end of the breathing cycle will be associated with the final bin with the remaining items of position and timing data being associated with intermediate bins depending upon their relative timing relative to the detected start and end of the breathing cycle. In the event that for a particular signal, no data is allocated to a particular bin, suitable timing and position data for a bin is generated by interpolating timing and position data using the data for adjacent timings. Conversely, if for a particular part of a breathing cycle multiple items of position and timing data are assigned to the same bin, an average of the timing
and position values associated with a bin is determined and stored for that bin. The result of this processing is that for each of the detected breathing cycle, a resampled set of data will exist where the number of pairs of timing position of data for each breathing sample is the same.
The resampled signals are then adjusted (S5-2) to account for any differences between start, peak and end amplitudes, hi an ideal setting, all sectioned breathing signals would have similar start, peak and end amplitudes as part of their inhale and exhale phase but this is not the case in practice. The variation in amplitude range is also independent of the actual shape of a sectioned breathing signal. Therefore in order to reduce any adverse effects in modelling the shape of a sectioned breathing signal, it is important to reduce any misalignment that may be present in a training sample.
In this embodiment, this is achieved by identifying as misaligned sectioned breathing signals, any signals whose inhale and exhale sequence lies in a different amplitude range to another. The position data associated with such misaligned signals is then amended to remove this difference. This is achieved by centring the mean amplitude (centroid) of all sections of a sectioned breathing signal, with the first in the sample as a reference. This is equivalent to carrying a least squares minimisation of the amplitude ranges of all sections in a sample relative to a reference section. This registration of the resampled sectioned breathing signals then makes it possible to calculate a mean sectioned breathing signal that
accurately reflects the mean shape of the sectioned breathing signals in a sequence.
Having adjusted the position data to account for any variation in the extent of inhale or exhale in a particular cycle, the sample mean sectioned breathing signal
X is then calculated (S 5-3) using the registered and resampled sectioned breathing signals X 1 . This is achieved by summing all sectioned breathing signal amplitudes and periods, and dividing these by the sample size. This is expressed as:
1 N
where each X 1 is a vector of length M with sectioned breathing signal amplitudes and a period value, and N is the sample size.
A sample covariance matrix S is then calculated (S5-4) using all Xj (columns of a
M x N matrix X) and X prior to using principal component analysis to calculate principal or significant variations in sectioned breathing signal shape that may be present in the sample used. S is therefore calculated as:
s = (x-x\x-x) τ
Principal component analysis is then (S5-5) used to capture principal or significant variations present in the registered and resampled sectioned breathing
signal sample data. These variations indicate the different ways in which the different sectioned breathing signals present in the sample varied from the mean shape. Principal component analysis therefore identifies principal variations/ as a set of eigenvectors P j (directions of variation) and eigen values λ j (variances corresponding to the eigenvectors) through carrying out an eigen decomposition of the sample covariance matrix S. The principal variations are then ordered in a descending order based on the eigen values λ j . The directions P j which have the most significant variations can then be identified. These significant variations will be the variations indicating variations in shape and period which contribute the most to the variations in the observed sectioned breathing signals stored in the signal cache 28.
Finally, a principal component analysis model for the sample breathing signal stored in the signal cache 28 is then stored (S5-6) in the breathing model store 34
by storing data identifying the sample mean X and the principal variations P j and λ j calculated using principal component analysis. This stored model therefore defines a linear generative model where sectioned breathing signal Y can be generated as:
where all W j are weights used to vary the contributions of the different principal variations, and T is the number of principal variations used in the model.
In this embodiment the weights w, are given a value of ±2 when individual principal variations j are displayed, while T is chosen as the number of variations that capture 90-100% of the sample variance, as calculated from the ordered X 1 .
This linear generative model also makes it possible to fit a sectioned breathing signal PCA model to newly captured sectioned breathing signal data as discussed in the following section.
Comparison of live data with breathing model
The use of a generated breathing model to identify periods of abnormal breathing will now be described with reference to Figures 6 and 7.
Referring to Figure 6 is a flow diagram of the processing undertaken by the patient monitor to identify a period of irregular breathing using a generated breathing model, the initial processing of undertaken by the patient monitor is essentially identical to that undertaken when obtaining an initial training sample to generate a breathing model. As described in relation to the generation of a breathing model, initially image data is obtained (S6-1) by the stereoscopic camera 10 and processed to determine (S6-2) position data for the position of a marker 22 present on the surface of a patient 20. This position data is then stored (S6-3) together with a time stamp in the signal cache 28.
The phase detection module 30 then processes (S6-4) the data stored in the signal cache 28 to identify the beginning and end points of a breathing cycle. This is
achieved in the manner previously described in relation to identifying the start and end points of a breathing cycle in a stored signal when a breathing model is being generated. When the phase detection module 30 identifies that data for a complete breathing cycle has been stored in the signal cache 28, the stored signal is then passed to the signal comparison module 36 which analyses (S6-5) the signal by comparing the signal with the stored breathing model. If the result of the analysis performed by the signal comparison module 36 determines (S6-6) that two successive signals cannot be modelled by the breathing model stored in the breathing model store 34, the signal comparison module 34 sends (S6-7) a signal to the treatment apparatus 16 to halt treatment until normal breathing is resumed and two successive breathing cycles which can be modelled by the breathing model have been detected.
The fitting procedure under taken by the signal comparison module 36 to compare an obtained breathing signal with a breathing model stored in the breathing model store 34 to identify abnormal breathing will now be described in detail with reference to Figure 7.
Turning to Figure 7 which is a flow diagram of the processing involved in comparing a breathing signal with a stored breathing model, initially (S7-1) the captured breathing signal data is resampled in the same way as has previously been discussed in relation to the generation of a breathing model. This resampling is therefore such to generate a set of pairs of position and timing data representing
the captured breathing signal where the number of pairs of data corresponds to the number of bins used to resample breathing signals when generating a breathing model.
The amplitude (position) measurements of resampled breathing signal are then adjusted (S7-2) in the same way that samples of sectioned breathing signal are adjusted to account for the differing extent of inhale and maximum exhale in different breathing cycle as has previously been described in step S5-2. This adjustment ensures that the subsequent fitting process can fit the stored breathing model to the breathing signal, without having to compensate for any misalignment in the amplitude range of both.
A set of weights w,- for modelling the resampled adjusted breathing signal are then (S7-3) calculated as follows:
where X is the sectioned breathing signal model's mean sectioned breathing signal, P j its eigenvectors and λ j its eigenvalues.
Once the stored breathing model has been used to calculate a representation of the new breathing signal, the quality of the fit is assessed (S7-4) to determine whether the shape achieved is a plausible fit.
In this embodiment, a plausible fit is identified as one whose amplitude shape and period are very similar to those in the sectioned breathing signal model's training data. A plausible fit measure D is therefore calculated as the Mahalanobis distance between the fitted sectioned breathing signal model and the sectioned breathing signal model's mean. The weights w j that are calculated as part of the sectioned breathing signal model fitting process are therefore used to calculate D
such that This distance represents the deviation of the fitted
principal variations). Therefore, D mαX = Jx p 2 (r) and a plausible fit is one that
satisfies D<D max .
When detecting bad sectioned breathing signals based on a model built with good sectioned breathing signal data, a plausible fit assessment gives an indication of a sectioned breathing signal being good (like one in the model's training set) or bad
(unlike one in the model's training set) than an accurate fit assessment. The degree to which a sectioned breathing signal is marked as bad can also be quantified by assessing how much greater D is from D max . If the accuracy of fit is determined to be less than a threshold value, this indicates that the breathing cycle
is abnormal and in the event that two successive breathing cycles are detected, treatment can be halted until normal breathing resumes.
Second Embodiment
A second embodiment of the present invention will now be described. In the first embodiment determination of the regularity of a breathing cycle the apparatus is described based upon a comparison of sectioned breathing signals acquired by the patient monitor system. An improved system will now be described which is able to detect both irregular breathing and patient movement. In addition, the system in accordance with this embodiment is not dependent upon the placement of a marker 22 on the surface of a patient 20.
The overall architecture of a patient monitoring system in accordance with this embodiment of the present invention is similar to the patient monitoring system of Figure 1 except that no marker 22 on the patient 20 need be provided and the 3D position determination model 25 is modified so as to process images obtained by the stereoscopic camera 10 to generate a 3D wire mesh model of the surface of the patient as viewed by the camera 10. Suitable means for processing images of stereoscopic cameras to generate 3D wire mesh models of patients are described in detail in the applicant's prior pending US patent application US 10/516400 published as PCT application WO2004/004828, which is hereby incorporated by reference.
As in the first embodiment, the result of processing by the 3D position determination module 25 is stored in a signal cache 28 together with a time stamp. Thus in this way a series of 3D wire mesh surfaces and associated time stamp data is stored in the signal cache 28. When data for a number of normal breathing cycles has been stored, this data is then processed by a modified phase detection module 30 and model generation module 32 to generate a principal component analysis breathing model modelling the manner in which a patient's surface varies throughout a breathing cycle. This breathing model is then stored in the breathing model store 34.
Subsequently during a monitoring phase, images obtained by the stereoscopic camera 10 are processed by the 3D determination module 25 and 3D surface data and timing data are stored in the signal cache 28. The phase detection module 30 then identifies when data for a complete breathing cycle has been stored. The signal comparison module 36 then compares the obtained signal with the stored principal component analysis model and identifies when the model is unable to match the generated surface data. In doing so, the signal comparison module 36 also determines a required motion to bring a detected surface back into alignment with the stored model. If it is found that the stored breathing model is unable to model a current breathing cycle, this indicates that irregular breathing or patient movement has occurred. A signal to halt the treatment can then be sent to the treatment apparatus 16 and where movement has occurred suitable instructions to
take remedial action can be determined and sent to the mechanical couch 18 to realign the patient 20 so that treatment can continue.
The processing undertaken by the phase detection module 30, the model generation module 32 and the signal comparison module 36 in this embodiment will now be described in detail with reference to Figures 8 and 9.
Generation of Surface Based Breathing Model
Due to the manner in which surface data is generated by processing images obtained by a stereoscopic camera 10, inconsistencies such as holes and outliers arise between successive surfaces. In order to overcome these inconsistencies a mesh deformation and hole filling approach is required to accommodate the varying surface characteristics including. To achieve this in this embodiment, the data representing surfaces is first (S8-1) processed using an energy minimisation process in which energy terms are defined for: (i) the point-to-point relationship between an initial reference surface being the surface for an initial pair of images obtained by the stereoscopic camera 10 and a surface generated for a subsequent pair of stereoscopic images, and (ii) the edge properties of the surface mesh of the initial reference surface. These energy terms are defined for corresponding vertices on both surfaces. The minimisation of these terms results in the transformation of reference vertices towards a subsequent surface such that the shape of the subsequent surface and the surface properties of the reference surface are preserved.
The data (point-to-point) energy ε dat a and smoothness (mesh) energy ε smoot ι mess for corresponding vertices i on a reference and subsequent surface are defined as follows:
^0)=(^.(0-^.0
smoothness ( ϊ)= ^ E±^ { χ , M)-XAj)) 1
where X n is the reference surface, X n+\ is the projected surface for a subsequent surface, / is the vertex index for both reference and projected surfaces, andy is the vertex index for all E vertices connected to vertex i on the reference surface.
The combined energy ε for a vertex / that is then minimised in order to calculate a smooth projected surface that preserves the shape of a subsequent surface while maintaining the surface properties of the reference surface is defined as follows:
*(') = a£ da,a (0+ β £ smoothness (0
where ε is the weighted sum of Zdaia and z smool h ness with α,βe R. α and β are used to control the deformation of a reference surface to match a subsequent surface, therefore resulting in a smooth projected surface with the shape of the subsequent surface but with the smooth mesh of the reference surface. Values for α and β are chosen as 1.0 and 0.2 for gated surfaces as these give a good trade off between shape preservation and surface smoothness.
When the combined energy is minimised for a given vertex i, a minimising translation / is calculated that minimises ε. This translation is calculated as follows:
A smooth projected surface Z n+1 will therefore have a set of vertices i defined as follows:
A set of smooth projected surfaces Z were used as the resampled set of gated surfaces for a captured sequence and this resampled set is described as follows:
Z = [X 1 , Z 1 + 1 2 , Z n _, + t H ,..., Z N _ λ + t N }
where an initial reference surface is the first in the resampled set i.e. Zi = Xi, and all subsequent resampled surfaces are calculated as the smooth surface projection of the previous resampled surface onto a generated subsequent surface i.e. Z n = Z n .,+t n .
The phase detection module 30 then (S8-2) processes the data representing smoothed 3D wire mesh surfaces captured by the stereoscopic camera 10 to obtain for each smoothed surface data identifying the location of a number of
points on the surface of the patient 20 where those points are tracked consistently from surface to surface.
In this embodiment, this is achieved by making an initial selection of points on the initial surface for the first pair of images obtained by the stereoscopic camera 10. These points are selected so that the points are spread out relatively evenly across a region of interest to be monitored corresponding to a patient's chest.
In order to identify the new positions corresponding to these points on the surface generated for the next pair of images obtained by the stereoscopic camera 10, each point is considered in turn. The intersection of a normal vector at a point with the surface generated for the next pair of images captured by the camera 1 is then determined. This is then repeated for all of the other points being monitored. The result of this processing is to identify the locations of the points being monitored at the time represented by the new images. This can then be repeated for each successive surface model stored in the signal cache.
Thus at this stage, the computer 14 will have stored for each of the surfaces generated by the 3D position data, a set of data {tj, Xi 1I ..., Xj ,n } where tj indicates the timing associated with an image i captured by the stereoscopic camera 10 and Xi 1I ..., Xj n indicate the 3D positions of points l ...n on the surface of a patient at the time tj.
The phase detection module 30 then (S8-3) utilises this data to derive a breathing signal which enables each set of data to be associated with a particular breathing cycle.
In order to do this, in this embodiment, the phase detection module 30 calculates a distance measure indicating the distance between each of the monitored points for which data is stored for a particular pair of images relative to the corresponding positions of those points in the surface for the initial pair of images. In this embodiment, this is achieved by calculating a distance measure:
i u d = -∑r(i)
where r(i) is a signed distance between corresponding surfaces X n and X n+/ i.e.
The distance measure between the initial surface and all subsequent surfaces is then used to form a breathing signal which is then used to divide the stored data into data for a number of different breathing cycles in the same way as has previously been described in relation to the first embodiment.
Having divided the sets of data {t,, x,,i..., Xi, n } into groups of data each associated with a different breathing cycle, the model generation module 32 then proceeds (S8-4) to generate a principal component analysis model using the data in a similar way to the generation of a principal component analysis model in the first embodiment.
More specifically a sample mean surface Z is calculated by summing the position data for the positions being tracked and dividing these sums by the sample size. This is expressed as:
where N is the sample size and all Z n and Z are vectors of the form (Xi 1I ..., x, , ,,}
The sample covariance matrix W is then calculated using Z n and Z prior to using principal component analysis to calculate principal or significant variations in surface shape that are present in the sample surfaces.
W is therefore calculated as: W = \Z - Zj [Z - Z)
The Eckart- Young Theorem is then used to ensure that suitable eigenvalues and eigenvectors are calculated from W
A surface based principal component analysis breathing model for a given sample
is then defined by the sample mean Z and the principal variations PQ) and IQ)
calculated using principal component analysis. This principal component analysis model defines a linear generative model where surfaces Y can be generated as:
where all w(j) are weights used to vary the contributions of the different principal variations, and T is the number of principal variations used in the model. The weights w(j) are normally given a value of ±2 when individual principal variations j are displayed, while T is chosen as the number of variations that capture 90-
100% of the sample variance, as calculated from the ordered λ(j).
Comparison of live data with surface based breathing model
After a surface based principal component analysis breathing model has been stored in the breathing model store 34, newly captured surface data is then matched with the stored model to identify abnormal breathing and/or patient movement as will now be described with reference to Figure 9 which is a flow diagram of the processing involved in comparing a breathing signal with a stored breathing model.
Initially (S9-1), newly captured surface data is resampled to match configuration (surface topology and vertex count) of a surface model's mean surface. This is necessary in order to fit the surface model to the captured surface as it allows
calculations to be made to determine the correct weights to use when fitting the model to the new surface.
In this embodiment, this resampling process is carried out using a two step projection and registration approach that ensures the resulting resampled surface accurately matches the captured surface.
First, the stored surface breathing model's mean surface is projected to a captured surface in order to establish an initial correspondence between both surfaces. The mean surface and projected surface are then registered using a Generalised Procrustes approach resulting in the mean surface being positioned closer to the captured surface. The mean surface is then projected to the captured surface and the resulting projection used as the resampled surface for the model fitting process. Then mean surface is restored to its original position prior to the model fitting.
Once the resampled captured surface has been established to match the surface model's mean, an accurate model is calculated (S9-2) using the model's linear generative model. This results in the determination of a set of weights w, being calculated for matching the new surface Z' to the model where the weights W j are as follows:
Aj)= -^p τ {jtz'-z)
where Z is the gated surface model's mean surface, PQ) its eigenvectors and XQ) its eigenvalues.
Once the principal component surface model had been fit to a captured surface, the quality of the fit is assessed (S9-3) to determine whether the shape achieved was an accurate fit, or a plausible fit.
In this embodiment an accurate fit is identified as one whose position and shape are very similar to a captured gated surface. A distance measure ε between the fitted model and the resampled gated surface is therefore calculated to determine the accuracy of a fitted model, ε is calculated as the mean distance between the vertices of the fitted model and captured surface i.e. ε = \Z'-Y\. A threshold ε max defining a "good" fit such that ε < ε max can be calculated by fitting the gated surface model to its training data, and selecting either ε max = max π (ε n ) or E 1113x =
1 N T7∑^, ' where ε « = l Z »- } 1-
A plausible fit is identified as one whose position and shape is very similar to that observed in the gated surface model's training data. A plausible fit measure D is therefore calculated as the Mahalanobis distance between the fitted gated surface model and the model's mean. The weights w,- that are calculated as part of the
model fitting process are therefore used to calculate D such that D = |∑(w(y)) 2 .
This distance represents the deviation of the fitted model from the sample mean
and is compared with a value £> max . D max is chosen based on a Chi-squared value χ p {r) for the number of degrees of freedom r contained in the model number of principal variations r = T) and the probability p represented by these degrees (percentage of the total variation captured by T principal variations). Therefore,
Z) max = ^χ p 2 (r) and a plausible fit is one that satisfies D < D max .
When detecting normal and abnormal breathing using a surface based breathing model built with normal breathing data (captured surfaces of a normal breathing sequence), an accurate and plausible fit assessment gives a reasonable indication of whether a newly captured surface has a normal position and shape (like one in the model's training set) or abnormal position and shape (unlike one in the model's training set). The degree to which a captured surface is marked as abnormal can also be quantified by assessing how much greater ε is from ε ma χ or D is from D max .
When comparing the accuracy of a fitted gated surface model, ε max is chosen as max π (ε π ) as this gives a better indication of an "outlier" position and shape when newly captured surfaces are assessed. The plausibility assessment is carried out using the defined Z) max value taken from the Chi-squared distribution. It has been determined that preferably both accuracy and plausibility assessments are made as together they offer a concise means of determining whether a subject's captured
position and shape is consistent with that observed during a period of normal breathing.
In the event that the surface differs from the model by more than a threshold value, this indicates that a good match to the sample data cannot be achieved. In this embodiment, if this is determined to occur the signal comparison module 36 sends (S9-4) a signal to the treatment apparatus 16 to halt treatment. Additionally, the signal comparison module 36 calculates the translation required to realign the captured surface data with the expected position of a surface as represented by the stored breathing model. If for a period of abnormal breathing this corrective translation is approximately consistent, this will indicate that the patient 20 has moved. An appropriate set of corrective instructions can then be sent to the mechanical couch 18 to realign the patient 20 in the correct position so that treatment can continue.
Further Modifications and Amendments
Although in the above described second embodiment, the patient monitoring system has been described waiting until surface data for a complete breathing cycle has been obtained and processed before identifying irregular breathing, it will be appreciated that patient movement can be detected without waiting for data for a complete breathing cycle to be captured. In such a system, a surface based breathing model could be generated using captured surface data in the absence of timing data. A model generated using such an approach would then
model the variation in the shape of a patient surface as the patient breathes. Instantaneous surface data could then be compared with the stored model and if a newly captured surface could not be represented by the model, this would indicate patient movement. If the processing of a number of surfaces identified that a patient had moved, appropriate corrective action could then be taken in the manner described above.
In the first embodiment a system is described which monitors the variation of the motion of a single point on the surface of a patient with time. In contrast, in the second embodiment, a system has been described which matches surface data to a model of the manner in which the surface of a patient varies during normal breathing. Essentially, this second embodiment is equivalent to monitoring the movement of a plurality of different points on the patient's surface over time.
Thus in the case of the first embodiment data of the form {t,x} is captured for each image frame whereas in the case of the second embodiment data of the form {t,xi....x n }is captured where the system monitors the variation of n points on the surface of a patient. In a further embodiment, instead of considering just the positions of points on the surface of a patient, the motion and velocity of points being monitored could be modelled. In such a system, consecutive images could be processed to determine data of the form {vi....v n } where vi....v n are vectors indicative of the position and velocity of points on the surface of a patient being
monitored. Such a system would enable both patient movement and abnormal breathing to be detected without waiting for a breathing cycle to be completed.
Although in the first embodiment, a system involving the placement of a marker 22 on the surface of a patient 20 has been described, it will be appreciated that the use of such a marker 22 merely simplifies the tracking of the motion of a point on the surface of a patient 20 and that a system such as that described in the second embodiment could be utilised to track the movement of a single point on the surface of a patient in the absence of such a marker 22.
It will also be appreciated that in other embodiments, rather than tracking the motion of points on the surface of a patient 20 using a stereoscopic camera other means could be used to obtain a breathing signal. Thus for example a marker 22 outputting a radio signal could be utilised and tracked. Alternatively a flow meter could be used to obtain a direct measurement of the through put of air into a patient's lungs and a breathing model generated using the obtained data.
Although in the first embodiment a single plausibility measure has been described to determine whether data for a breathing cycle can be modelled using a stored breathing model, it will be appreciated that other measurements of the extent that captured data can be modelled could be used.
Thus for example in order to identify whether a captured breathing cycle has an amplitude similar to those represented by a stored model, a distance measure ε
between the fitted model and the resampled and registered new sectioned breathing signal could be calculated to determine the accuracy of a fitted model. The mean distance E between sectioned breathing signal amplitudes and periods of the fitted model and new sectioned breathing signal i.e. ε=|X'-7] could then be determined and the result compared with a threshold t defining a "good" fit such that ε<t selecting either
Although in the above described embodiments, systems have been described in which treatment apparatus is deactivated when irregular breathing is detected, it will be appreciated that in other systems or embodiments, other actions could be undertaken. Thus for example in the case of a CT scanner, rather than deactivating a scanner, scans obtained whilst abnormal breathing was occurring could merely be labelled as such and a subsequent 4D model of the movement a patient's internal organs during a breathing cycle could then be generated using only scans obtained during normal breathing.
A further alternative would be to utilise the breathing monitor in the absence of any treatment apparatus such as might occur during breathing training where a patient attempts to regulate their breathing as much as possible to assist with the reliable application of radio-therapy. During breathing training a patient is typically shown a training signal that may be in the form of a bar graph which
rises and falls in a manner representative of an idealised breathing signal and the patient then attempts to regulate their breathing accordingly. The above described system could then be used to generate a model of the regulated breathing and a patient could then be given feedback as to how well regulated the breathing was in terms of the variability of their breathing as modelled by the system. In such an embodiment or indeed any embodiment, the system could be connected to a display an output a warning when irregular breathing or movement was detected.
It will be appreciated that a breathing model could be created on one occasion during treatment, say for example during CT acquisition, and the same model for normal breathing reused on subsequent occasions. Alternatively, a new model could be generated from a period of normal breathing at the beginning of each treatment session.
Although in the above described embodiments, systems have been described which halt the application of radiation when abnormal breathing or patient movement is detected, it will be appreciated that embodiments of the system could also be used to select portions of a breathing cycle when radiation is to be applied. Thus for example in the case of the system of the second embodiment, the system could be arranged to irradiate a patient 20 only when a captured surface is no more than a threshold distance from a predefined reference surface.
Further it will be appreciated that once abnormal breathing had been detected, the abnormal breathing itself could be further classified before making any decision
to halt treatment. Thus for example a system might be arranged to identify abnormal breathing and if the breathing merely represented rapid breathing, treatment might be continued whereas breathing classified as corresponding to coughing could cause the system to halt treatment.
Although in the above described embodiments, systems have been described in which principal component analysis models of breathing are generated, it will be appreciated that any suitable form of modelling which enabled the variability of movement corresponding to normal breathing could be used. Thus for instead of principal component analysis, an independent component analysis model could be generated for a breathing sample and abnormal breathing could then be identified by comparing data captured subsequently with the generated independent component analysis model.
Although the embodiments of the invention described with reference to the drawings comprise computer apparatus and processes performed in computer apparatus, the invention also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of source or object code or in any other form suitable for use in the implementation of the processes according to the invention. The carrier can be any entity or device capable of carrying the program.
For example, the carrier may comprise a storage medium, such as a ROM, for example a CD ROM or a semiconductor ROM, or a magnetic recording medium,
for example a floppy disc or hard disk. Further, the carrier may be a transmissible carrier such as an electrical or optical signal which may be conveyed via electrical or optical cable or by radio or other means.
When a program is embodied in a signal which may be conveyed directly by a cable or other device or means, the carrier may be constituted by such cable or other device or means.
Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted for performing, or for use in the performance of, the relevant processes.
Next Patent: KIT FOR ASSEMBLY INTO A PLATFORM STRUCTURE