DETECTION IN BREATHING SIGNALS

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the priority to U.S. Provisional Pal Application Serial No. 61 /61 1 ,265, filed on March 15, 2012, which is incorpora by reference herein. FIELD

[0002] The present disclosure relates to food intake monito system and method. More particularly, the disclosure relates to a food int monitoring system that uses sensors to detect apnea in the subject monitoring breathing signals and that generates a stored record of the subje eating patterns, useful in treating obesity, eating disorders and a varietv diseases that are linked to obesity.

BACKGROUND

[0003] This section provides background information related to present disclosure which is not necessarily prior art.

[0004] According to the data from World Health Organizat worldwide obesity has increased over 200% since 1980, and in US 68% of population is considered to be overweight or obese (i.e. Body Mass lm greater than 25). It has been proven that obesity has causal relationship wil number of diseases such as coronary heart disease, type-2 diabetes, ι various types of cancers (e.g. endometrial, breast and colon). With s findings, obesity control and management became an important medi social, and policy issues in recent years. Two important components of obe management are diet control and physical exercise.

[0005] Traditionally, researchers and health practitioners h used self-reported questionnaires for estimating both food intake and phys activity levels for high-risk individuals. In recent years, accelerometry-ba instrumentation techniques are starting to emerge as alternatives questionnaires for physical monitoring. For food intake monitoring, howe ¹ not many instrumented efforts were reported in the literature.

[0006] In most questionnaire based studies, participants h; shown a tendency of intentionally or unintentionally underreporting the amc of their food intake. Additionally, data self-reported by elderly people are ol unreliable since amnesia is quite common among that population. It was∑ found that women are more likely to underreport their fat consumption w over-report their protein intake. It was experimentally shown that due to above and other similar reasons, questionnaire based self-reporting syste are often too unreliable to be successfully used towards food intake monitoi for obesity management. An instrumented system, if available, could elimin such subjectivity attached to questionnaire based systems.

[0007] Current solutions can be divided into two categor invasive and non-invasive. Invasive methods, such as videofluoroscopy ; functional magnetic resonance imaging (fMRI), cannot be used for even monitoring and dietary analysis. Many of the non-invasive methods surface electromyography (SEMG) or movement sensors to measure movement of larynx and the activity of the muscles associated with the swal event. But these sensors are put at the neck, which would lead to reluctance of the subjects to wear them. Some use Respiratory Inducta Plethysmography (RIP) to measure the movement of the chest, but th devices are quite expensive and are driven by other modules thus not suite for wearable solutions. SUMMARY

[0008] This section provides a general summary of the disclosi and is not a comprehensive disclosure of its full scope or all of its features.

[0009] The present disclosure provides an instrumented sysl that can automatically monitor the duration of each instance of food/d intake, which can in turn be used for estimating an individual's eating ; drinking habits. Taking the self-reporting error out of the analysis, the sysl can have enormous significance in terms of the way today's obesity ; disease management programs are implemented. For example, by analy2 data from a type-2 diabetes patient who also suffers from obesity, his or health care provider may suggest the patient to eat less and do more exercise

[0010] Disclosed here is a system and method that usei wearable food and drink intake monitoring system that is capable of monitoi food and drink intake for remote health monitoring applications. In i embodiment a wearable sensor placed around the torso of the subject genera breathing signals that are processed by a classifier to identify apnea patte indicating that the subject has swallowed. In another embodiment a temp matching mechanism is deployed to more accurately perform swallow analyt Post-processing algorithms, such as Consecutive Swallow Amalgamation ; averaging reference cycles, are used in some embodiments to improve accuracy and effectiveness of the system. Other embodiments further impr system accuracy by automatically analyzing patterns of sequence. The disclo system and method can be deployed, if desired, in a food and drink intake ι time monitoring system, which can be used for various purposes, such as he improvement, assisted living and disease monitoring.

[0011 ] The disclosed system thus provides a system and met for monitoring food or beverage intake of a living subject, utilizing a wears breathing sensor adapted to be worn around the torso of the subject and be responsive of inhale-exhale movement of the subject's torso to produc< breathing signal expressed as electrical data. The electrical data are analyzec a classifier receptive of the electrical data and operative to classify the electr data according to a predefined set of breathing patterns that include at least i apnea pattern indicating that the subject has swallowed during a breathing cy If desired, a swallow pattern analyzer operates upon the output of the class to recognize food and beverage intake patterns associated with eating ; drinking different types of food and beverage and optionally associate th patterns with other data, such as eating log data entered by the subj geolocation data automatically captured, and time and date data corresponc to the detected intake patterns. [0012] Further areas of applicability will become apparent from the description provided herein. The description and specific examples in this summary are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.

DRAWINGS

[0013] The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present disclosure.

[0014] Figure 1 is a wearable wireless food intake monitoring system;

[0015] Figure 2 is a resistive belt;

[0016] Figure 3 is a graphical diagram that shows the static response of resistive belts;

[0017] Figures 4a and 4b are graphical diagrams that show the transient response of the resistive belts;

[00 8] Figure 5 is a piezo-respiratory belt;

[0019] Figure 6 is a graphical diagram that shows the status response of piezo-respiratory belts;

[0020] Figure 7 is graphical diagram showing the transient response of piezo-respiratory belts;

[0021] Figure 8 is a signal shaping circuit;

[0022] Figures 9a-9d (collectively Fig. 9) show examples of respiratory and swallow signals;

[0023] Figures 10a- 0b (collectively Fig. 10) show the matched filter operation;

[0024] Figures 1 a-11c (collectively Fig. 11) show and example of BC-IS detection;

[0025] Figure 12 shows the data processing architecture for swallow detection; [0026] Figures 13a, 13b and 13c (collectively referred to as Fig. show a receiver operating characteristic (ROC) distribution with arbiti template selection for different subjects (subject 1 , subject 2 and subject 3);

[0027] Figure 14 is a graphical diagram showing examples of modulation by adjacent swallows;

[0028] Figures 15a, 15b and 15c (collectively referred to as Fig. show ROC with breathing cycle modulation removed;

[0029] Figures 16a, 16b and16c (collectively referred to as Fig. show ROC with templates formed using two controlled cycles;

[0030] Figures 17a, 17b and 17c (collectively referred to as Fig. show ROC with templates using three controlled cycles;

[0031] Figure 18 is a system block diagram illustrating i embodiment of a food intake monitoring system in accordance with the disclo system and method;

[0032] Figure 19 are a collection of graphs providing example: respiratory and swallow signals, useful in comparing swallowing of liquids solids; and

[0033] Figure 20 is a data processing scheme for detecting ; differentiating swallows of liquids vs. solids.

[0034] Corresponding reference numerals indicate corresponc parts throughout the several views of the drawings.

DETAILED DESCRIPTION

[0035] Example embodiments will now be described more fully \ reference to the accompanying drawings.

[0036] The disclosed system and method employs a wears wireless sensor system for food/drink intake monitoring with an associa computer-implemented analytic framework. The system and method wc based on the key observation that a person's otherwise continuous breath process is interrupted by a short apnea when the person swallows as a par solid or liquid intake process. Using a wearable wireless chest-belt, the sysl first detects and records a swallow sequence by detecting the associa apneas extracted from breathing signal captured by the chest-belt. After swallow sequence is recorded, computer-automated swallow pattern analysi used to identify non-intake swallows (or empty swallows), solid int swallows, and drinking swallows. These patterns may then be used to d inferences about the subject's eating and drinking habits.

[0037] One presently preferred embodiment employs swal detection sensor modality based on piezo-respiratory chest-belts, embedded hardware and software platform for wirelessly collecting the sen output, and signal processing mechanisms for swallow detection with r accuracy and low false positives.

[0038] In such a presently preferred chest-belt based systerr breathing signal is extracted via monitoring the expansion-contraction patten the chest. Since the belt can be worn inside, outside, or between garment: does not need skin contact), it has the potential for prolonged comfortable us; without raising any cosmetic issues. Extensive experimental evaluation she the system's robustness in terms of its ability to monitor swallowing by detec apnea with high accuracy and low false positive rates.

[0039] Key Concepts: Breathing, Swallow, and Apnea, breathing process is divided into two phases, inhalation and exhalat Inhalation is initiated by diaphragm and various muscles, leading to expansion of chest cavity. At the same time, the lungs expand due to inhaled air through nose (or mouth), throat, larynx, and trachea. Then oxyge extracted and carbon dioxide is released by pulmonary alveoli. When exhalation starts, diaphragm and muscles get relaxed, thus pushing air ou lungs through trachea, larynx, throat, and nose (or mouth).

[0040] The process of swallowing is normally divided into th phases. When swallow happens, food is first chewed into a bolus (term u: for describing a block of food or liquid) and then propelled into the orophar by the tongue, which is called the oral phase. In the pharyngeal phase, palate bocks the back wall of the throat preventing the bolus from goinc nasal cavity, and the vocal cords close, blocking the trachea. While the be goes down, epiglottis covers the vocal cords, and then the upper esophac sphincter relaxes allowing the bolus to enter the esophagus. Finally, in esophageal phase, the bolus enters esophagus and finally arrives at stomach.

[0041] During swallowing, because the trachea is blocked person is not able to breath, thus causing a temporary apnea during breath A key concept of the proposed system is to extract this apnea as the fingerp for detecting swallows.

[0042] System Architecture. In one embodiment, an embed< wearable sensor system is worn on the chest for collecting the breathing sic and for transmitting it to a PC or other processing device through a 900k wireless link or access point. In this embodiment the embedded wearable ser system comprises: 1 ) a piezo respiratory belt for converting the changes tension during breathing to a voltage signal, 2) an amplifier and signal shap circuit for formatting the raw voltage signal to a format compatible for the A chip, 3) a processor and radio subsystem (Mica2 motes) running Tiny operating system, and 4) two AAAA batteries. The entire package is q lightweight, and the two 600mAh AAAA batteries are able to support the sysl for more than 30 hours of continuous operation.

[0043] After the signal is received by the 900MHz access point, fed via USB port to a PC (e.g., processing server) for detecting swallow events in either real time (runtime) or offline. Swallow sequence patl analysis for food/drink intake estimation is also executed in this external macr (PC). The advantage of using an embedded wireless link is that the develoi swallow sensor can be networked with other physiological and physical acti sensors to develop a networked sensing/detection system to provide a comp instrumentation package for obesity management in future.

[0044] Of course other processors and radio subsystems can used. For example, if desired the processing can be performed on a mo smartphone that communicates with the wearable sensor system via ra communication such as BlueTooth. In such an embodiment the processor board the smartphone performs the data capture and optionally also the c analysis. If desired, the smartphone can communicate either the captured data, or preprocessed data, or final analytical results to a server via WiF cellular communication.

[0045] Sensor Selection and Characterization. This sec presents the properties and experimental characterization of different type; chest-belt materials for obtaining respiratory signals in a non-invasive mani Based on the analysis in this section, a piezo-respiratory belt is prese preferred, although the apnea detection in breathing signal technique disclo here can also be practiced using other wearable sensor systems, including not limited to those discussed below.

[0046] Inductance Belt. An inductance belt relies on phenomenon that a magnetic field is generated when current flows throug loop of wire. Any change in the area enclosed by the loop can create a currer the loop in the opposite direction proportional to the change in the area. W using an inductance belt for respiration monitoring, a low amplitude carrier w of -20 mV at -300 KHz is injected through the loop on the belt. The inhala and exhalation process change the area enclosed by the belt and introduces opposing current, which in turn modulates the original carrier wave. The signi then demodulated to recover a signal that reflects the change of the area du< breathing. It is reported [16] that the output of such belts usually changes line with the enclosed cross-sectional area.

[0047] Required instrumentation for inductance belts consists c frequency generator, an analog to digital (ADC) converter, and a sic processing unit. Inclusion of that many active electronic devices make the pric point of the inductance belts to be currently much higher than the other type: respiration monitoring belts as described below. Future improvements inductance belt technology may make it cost effective.

[0048] Resistive belt. Resistive belts use an elastic mate whose resistance per unit length depends on the amount of stretch tha created during the breathing process. Resistance change can be easily captu by measuring the voltage drop across the belt. Such belts can be worn eithe the abdomen or in the chest area for capturing the expansion-contrac sequence created by the breathing dynamics. [0049] We have created a prototype resistive belt using stre sensors manufactured by Scientific Instruments as shown in Fig. 2. The diami of the sensor is 1 .5mm, and the length is 1 5cm.

[0050] Static and transient characteristics of the sensor analyzed by stretching it to different lengths. Fig. 3 demonstrates the st property of the sensor. In this experiment, the resistance of the sensor at e sample point is read 1 minute after it is stretched so that the impact of trans response is minimized. From Fig. 3, the sensor demonstrates good linearity static experiments.

[0051 ] To analyze the transient property, the belt is first stretc for 5cm and kept for 1 minute to make it tight and stable so that the impac slack is minimized. Then it is stretched by another 5 cm, and the responsi recorded. After 1 minute, when the resistance of the sensor becomes stable, released by 5 cm. Fig. 4 shows the resistive dynamic response of the belt a function of time. Fig 4 (a) illustrates the response after it is stretched, and (t the response after it is released.

[0052] Fig. 4a demonstrates that when the belt is stretchec surge in resistance followed by decay can be observed. Fig. 4b depicts 1 when the belt is released, the resistance surges up a little before decaying tc terminal value. Results above demonstrate that although the static behavio the resistive belts maintain excellent linearity, their transient behavior is hie non-liner over time constants that are comparable to human breathing peric As a result, despite their cost advantages current resistive belts are considered as the best sensing tool for the swallow detection mechan presented in this work. Future improvements in resistive belt materials r produce better linearity over time constants comparable to human breatr periods. In such case, resistive belts may become suitable for detecting apne breathing signals.

[0053] Piezo-respiratory Belt. A piezo-respiratory belt contain piezoelectric sensor placed between two elastic strips. Stretching the belt exi a strain on the sensor, which generates a voltage proportional to the strengtl the force. Comparing with other transduction principles, such as capacit inductive and piezoresistive effects, piezoelectric phenomenon curre provides the highest sensitivity and excellent linearity over a wide ampliti range.

[0054] Fig. 5 shows the piezo-respiratory belt from ADI Instrume used in a presently preferred system. The static response of the belt is show Fig. 6, which demonstrates good linearity of produced voltage as a functior belt elongation.

[0055] Fig. 7 demonstrates the voltage signal captured oscilloscope when the piezo-respiratory belt is first stretched by 15mm and t released after some time. It is notable that unlike in the transient response of resistive belt in Fig. 4, the output of the piezo-respiratory belt closely follows mechanical inputs (i.e. stretch and release) without the surges. It is becaus< its static response and clean transient response we chose the piezo-respiral belts for our presently preferred swallow detection system. Of course, as nc above, as materials improve, other types of sensors may become useful in future.

[0056] Signal Shaping Hardware. The voltage output from piezo-respiratory belt is shaped as follows. First, the voltage output from the needs to be amplified since the peak-to-peak voltage variation during inh; exhale cycle is only 10mV which is smaller than what majority of the off-the-s ADC converters can discern with acceptable accuracy. Second, depending the wearing tightness of the belt there is a DC component in the output volta which can vary from person to person depending on how tightly the belt is w< Sometimes the DC component can even vary for the same person on diffei wearing instances. The situation is further aggravated when the DC va changes over time as the belt becomes loose after a subject wore it for sc time. The shaping circuit thus needs to take care of such person-, wearing-, ; time- dependent variability of the DC component of the output voltage from piezo-respiratory belt.

[0057] The shaping circuit shown in Fig. 8 serves the ab shaping needs by first removing any DC voltage component (i.e. low pass fil in the signal, and then by amplifying the AC-only signal which is fed into analog-to-digital convertor (ADC) of the processor and radio card as showr Fig. 1 .

[0058] The shaping circuit in Fig. 8 has six distinct stages, althoi other types of shaping circuits may alternatively be used. The first st; represents the equivalent circuit of the piezo-electric sensor device wr contains a voltage generator and a capacitor. The second stage is a drift-cor module which is meant for damping DC voltage drifting when the belt is worr different subjects. The third stage is for voltage shifting. It sets the default voltage to be 63mV, so that the output signal always stays positive. The foi stage is for impedance matching by isolating the previous stages of the sic shaping circuit from the amplification stage. The fifth stage is a low-pass f that filters out any noise produced by the previous stages. Finally, the sixth st; is the amplifier, whose amplification gain is controlled by an adjustable resisto

[0059] The output of the shaping circuit as depicted in Fig. 8 is into a 10-bit ADC channel of a processor and radio card in Fig. 1 . With a power supply (i.e. two AAAA batteries) the resolution of the 10-bit ADC is 3r ADC sampling rate is set to be 30Hz. As shown in Fig. 1 , the collected dat sent wirelessly to an access point via 900MHz radio link. The access point fe the data to a processing server machine for executing the swallow detec algorithms.

[0060] Breathing Signal and Apnea Analysis. Fig. 9 demonstrE a number of experimentally obtained breathing signal segments from diffei human subjects using the system as shown in Fig. 1 . The ADC readings in figure are directly proportional to the tension on the piezo-respiratory t meaning the rising edge in the graph corresponds to inhalation and the fal edge corresponds to exhalation of a breathing cycle.

[0061] As shown in Figure 9 a breathing cycle can be either nor (i.e. NBC) or elongated due to a momentary apnea caused due to a swal event. A cycle that is elongated due to an apnea at the beginning of an exr (see Fig. 9a for subject- 1 , session-1 ) is termed as Breathing Cycle with Exl Swallow (BC-ES). Both Fig. 9a and Fig. 9b depict BC-ES events recorded subject-1 in two different sessions. For a second subject, Fig. 9c shows swallc (i.e. apnea) during the inhale process which are termed as Breathing Cycles ι Inhale Swallow (BC-IS). The same second subject was instructed to perform I ES during a second session as presented in Fig. 9d.

[0062] One key objective of the disclosed system is to classify three types of breathing cycles, namely, NBC, BC-ES, and BC-IS, with r accuracy and low false positive rates. The challenges in this task stem from fact that there are significant amounts of variability in: 1 ) breathing wavefoi across different subjects, 2) across different measurement instances for same subject, and 3) most importantly, the placement and duration of the api with respect to its home breathing cycle. This depends a great deal on swallowing habits and the specific material (i.e. solid, liquid, etc.) that is be swallowed. During the course of our experimentation with a large numbei subjects and multiple sessions with each subject, it was found that the majc of swallow-triggered apneas were found to have happened either in the middle the inhale or right at the beginning of the exhale as shown in Fig. 9.

[0063] Processing for Swallow Detection. This section introduce matched filter based template matching mechanism for swallow detection.

[0064] Matched filter. In one presently preferred embodimer matched filter is used for detecting the presence of a known signal, referred tc template, by correlating the template signal with an unknown signal. In embodiment, the template signal is chosen from different types of breatr cycle waveforms, namely, Normal Breathing Cycle (NBC), Breathing Cycle ι Exhale Swallow (BC-ES), and Breathing Cycle with Inhale Swallow (BC-IS), that the an arbitrary breathing cycle can be classified as one of those three observing the similarity score produced by the matched filter.

[0065] As shown in Fig. 10a, the matched filter in this contex parameterized by input x(n) and template s(n), representing wavefoi corresponding to a single breathing cycle consisting of T samples (i.e. 0 < Γ). Note that the durations of both the waveforms are assumed to be equal. ^" output of the matched filter is represented as y(n) which consists of 27samp Finally, the similarity score of the filter is parameterized as μ, which is the hig value of y(n) over the interval 0 < n < 2T. [0066] Output of the filter y(n) is computed as time delayed convolut y(n) = ^ x(k)h(n— k)

k=-∞

where h(ri) = s(T - n) is the impulse response of the filter for a given temp s(n). As the similarity between input x(n) and template s(n) becomes stron< the amplitude of the output y(n) increases, thus producing larger similarity sc μ. Since the quantity μ is computed as the highest value of y(n) over the inte 0 < n < 2T, the impact of time shift between x(n) and s(n) is removed.

[0067] Fig. 1 0b shows an example operation of matched filter. ^' figure shows four operation instances with different input waveforms x _k (ri) ( 1,2,3,4) but with the same template waveform. Observe that x^n) is same as template waveform without any time-shift, x ₂ (n) and x ₃ (ri) are different from template, and x ₄ (n) is derived by time-shifting x^n). The waveforms y _fe (n) the outputs of the filter for each of the correspondingx _fe (n). Note that similarity μ ₁ and μ ₄ are equal to 1 , which are greater than μ ₂ and μ ₃ . Thi: because χ^ ) and x ₄ (n) share the same shape of reference template fed the filter. The impact of time shifting (i.e. from x^n and x ₄ (n)) is removed selecting the maximum value of y(n) as the similarity score.

[0068] Fig. 1 1 depicts example matched filter operations wit Breathing Cycle with Inhale Swallow (BC-IS) as the template input. Observe 1 in Fig. 1 1 a, when the input to the filter is a Normal Breathing Cycle (NBC), output μ is 0.93, which is smaller than the output (i.e. 0.99) in Fig. 1 1 b in withe input is a BC-IS waveform. The output is smaller (i.e. 0.975) in Fig. 1 1 i which the input is a BC-ES waveform. This example shows how the mate filter is able to produce a higher output when the input is of the same breatr cycle type as the template waveform, even though the specific BC-IS ir waveform in Fig. 1 1 b is different from the actual template waveform. Sirr behavior was also recorded when NBC and BC-ES waveforms were used template waveforms.

[0069] Processing Processing Architecture. Fig. 1 2 depicts processing modules used for classifying breathing cycles towards swal detection. The first step is to pass the ADC output through a low-pass filter removing quantization caused by the A-to-D conversion process. The sea step is to run the filtered data stream through a peak and valley detec module in order to extract the individual breathing cycles. The next process module is used for normalizing the extracted cycles in both time and ampliti dimensions. The objective of such normalization is to make sure that althoi different cycles may have different time and amplitude ranges (person-to-per or cycle-to-cycle for the same person) they can be effectively correlated for template matching purposes. In other words, the normalization process ensi that both x(n) and s(n) waveforms are of same duration (i.e. Γ samples).

[0070] The normalized breathing cycle waveforms are then fed three separate matched filters, each with a specific type of reference temp waveform. The matched filters use reference waveforms corresponding normal breathing cycle (NBC), Breathing Cycle with Exhale Swallow (BC-E and Breathing Cycle with Inhale Swallow (BC-IS), as shown in Fig. 9. similarity score output of all three filters are finally compared to classif breathing cycle waveform as one of the above three types of BCs.

[0071] Performance. Extensive experiments using the systerr Fig. 1 were carried out for swallow detection with seven subjects. Results three male subjects are presented in this section.

[0072] Experimental Method. Each subject performed th sessions, five minutes each. The subject was asked to wear the instrumer chest-belt and drink water from a flask with a swallow instruction given onci every 20 seconds. Each session resulted in approximately 80 Normal Breatr Cycle (NBC) and approximately 15 breathing cycles with swallows (t Breathing Cycle with Exhale Swallow (BC-ES) and Breathing Cycle with In! Swallow (BC-IS)). The resulting swallow signals are collected over the 900I wireless link as shown in Fig. 1 . For each session, approximately 90 breatr cycles are recorder in total.

[0073] A small microphone was also attached to the front par the neck for recording the swallow sound. This audio signal, which was tii synchronized with swallow data extracted from the chest-belt, provided a cor that was used for both training and verification of the proposed swallow detec mechanism. Two male and one female subject performed the above procedur

Table 1 : Breathing cycle breakdown

[0074] Breathing Cycle Statistics. Table 1 summarizes breakdown of different types of breathing cycles observed during th experiments. For these three specific and other subjects it was consiste observed that Breathing Cycles with Exhale Swallow (BC-ES) are gener more prevalent than Breathing Cycles with Inhale Swallow (BC-IS). In ol words, most instances people swallow immediately after inhalation followed exhalation.

Table 2: Durations of different breathing cycle types

[0075] Table 2 summarizes the duration of different types breathing cycles. In addition to the spread of the cycle durations, it should observed that the cycles with swallows (i.e. both BC-ES and BC-IS) consistently longer than the normal breathing cycles. This is mainly due to short apnea introduced by the swallow events.

[0076] Performance Parameters . The system performance reported as ROC (Receiver Operating Characteristic) statistics of True Posi Rate vs. False Positive Rate. These two rates are computed using performa indices as reported in Table 3. True Positive Rate is defined as the numbe correctly detected BC-IS and BC-ES as a fraction of the total number of BC and BC-ES. This can be expressed as TP /(TP + FN). False Positive Rah defined as the number of erroneously detected BC-IS and BC-ES as a fractioi the total number of NBCs.

Table 3: Detection performance indices

[0077] Arbitrary Template Waveforms . As a first step, we have evaluated the system's detection performance with arbitrary combinations NB BC-IS and BC-ES waveforms as the reference input s(t) as shown in Fig. 12. The evaluation process is as follows.

[0078] First, for each subject, approximately 3500 different reference combinations of NBC, BC-IS and BC-ES waveforms are created fro approximately 300 breathing cycle waveforms collected from three experimen sessions. Second, for each reference combination, all 300 breathing cycle waveforms are detected to be one of NBC, BC-IS and BC-ES using the systei described herein. Third, by comparing these detection results with the actual breathing cycle types observed from the neck-attached microphone for all 30C cycles a True Positive Rate and a False Positive Rate (from the parameters ir Table 1 ) are computed. Using the above process an ROC pair ( True Positive Rate, False Positive Rate) is computed for each of the 3500 reference combinations. Fig. 13 shows the ROC { True Positive Rate, False Positive Rat distribution of 3500 such pairs for three different subjects.

[0079] The following observations can be made from Fig. 13. Fir the cluster of high value columns in the diagram indicate that even with arbitrs chosen template waveform combinations, majority of the combinations offer h True Positive Rate and low False Positive Rate. The spread in the distribution indicate that there exist NBC, BC-IS, and BC-ES waveforms which, if chosen templates, can indeed bring down the system performance. As a result, this approach of random template selection is not practically feasible. The task of appropriate template selection will be dealt with in the following sections.

[0080] Breathing Cycle Modulation by Adjacent Swallows. It wa; experimentally observed that sometimes a swallow event in a breathing cycle can modulate the signal for cycles that are immediately before or after. Such modulations are demonstrated in the example trace in Fig. 14. It was determir that most of the affected cycles before actual swallows were caused by a mini change of breathing in subconscious anticipation of an impending swallow. Tr affected slots after actual swallows were caused due a similar reason. The subjects also reported that sometimes they executed and very minor second swallow for drinking any remaining liquid in the throat region.

[0081] It turns out that the matched filter based system is often a to detect the above modulation and reports such occurrences as two swallow; consecutive breathing cycles, thus contributing to the false positive rate. Givei that such modulations by adjacent swallows are always involuntary, it is reasonable to filter them out. Considering that it is extreme rare for people to have real swallows in consecutive cycles, we count any two swallows that happen in consecutive breathing cycles as one.

[0082] Fig. 15 reports the ROC distribution when the above filter scheme is applied to the results obtained from Fig. 13. As expected, filtering c the breathing cycle modulation does reduce the false positive rates while maintain the true positive rates as obtained in the without filtering scenario. Unless otherwise stated, all subsequent results in the paper correspond to thi: modulation filter enabled.

[0083] Globally Averaged Template Waveforms. Results in this section correspond to a detection scenario in which for a given subject, all normalized NBC waveforms are averaged on a sample-by-sample basis to create the reference template for the NBC filter. Similarly, all normalized BC-I and BC-ES waveforms are averaged for the creating the templates for the BC and BC-ES filters respectively. This arrangement is expected to provide a bes case performance indication of the system. Table 4 summarizes the detection performance with such globally averages template waveforms. True Positive Rate False Positive Rate

Subject 1 0.933 0

Subject 2 0.933 0.089

Subject 3 0.911 0.010

Table 4: Performance with globally averaged templates

[0084] While providing a performance upper bound, mechanism of template formation is not practically feasible since the breat cycles need to be classified a priori in order to prepare the templE themselves. The next two sections details two practically feasible detec mechanisms.

[0085] Template Formation using Controlled Breathing Cyc Results in this section correspond to template waveforms that are compi based on average of known cycle types during a brief controlled phase. At beginning of data collection for each subject, a number of NBC, BC-IS, and I ES cycles are recorded on instruction. Meaning, data recorded from each s cycle, referred to as controlled cycle, is known to be of its specific type. After control phase, a sample by sample average NBC waveform is created from recorded controlled cycles of type NBC. This average waveform is used as template for the NBC matched filter. Templates for BC-IS, and BC-ES filters computed using the controlled cycles of the respective types. Unlike the glob computed average, this mechanism can be practically feasible.

[0086] Figure 16 reports detection performance using controlled cycles. To capture the effects of variability present in the contro cycles, for each cycle type we arbitrarily choose two breathing cycles from entire pool of collected cycles, and use them as the controlled cycles. compute the true positive and false positive rate for each such combinatior two controlled cycles for three cycle types. A distribution of 3500 s combinations is reported in Fig. 16.

[0087] As expected, the performance distribution in this c shows relatively better true and false positive performance compared to th corresponding to single arbitrarily selected template.

[0088] Fig. 17 reports the case when 3 controlled cycles are u for creating the reference waveforms for the matched filters. Comparing with 16 it can be observed that the overall performance is further improved whic indicated by stronger clustering of the histogram columns near 0% false posi rates and 100% true positive rates.

[0089] Discussion of Additional Embodiments

[0090] While the system and method of food intake monitoi using apnea detection in breathing signals can be implemented in a variet; different ways, refer now to Fig. 18, which shows one way of implementing s a food intake monitoring system. In accordance with the description above, system employs a wearable breathing sensor system 100. In a prese preferred embodiment this wearable breathing sensor system may be a pie respiratory belt, although other types of sensors could be used instead.

[0091] Depending on the sensor system selected, a signal shap circuit 102 is employed, to bring the raw signal levels of the sensor system u| a useable voltage and to filter out any DC component or unwanted nc components in the raw signal. In this regard, the primary function of breathing sensor system is to obtain measurement data from which the wear breathing patterns may be analyzed. Thus the raw signal from the breatr sensor system may be filtered, as discussed above to filter out signal informa that is not varying over a period commensurate with the expected period of human breathing cycle.

[0092] As described above, the presently available sensors that suitable for sensing human breathing patterns produce analog outputs. Thus signal shaping circuit 102 may be implemented, as shown in Fig. 8 as an an£ signal processing circuit. Alternatively, signal processing of the raw senor oul can be performed in the digital domain. In such case the raw sensor outpi digitized using a suitable analog-to-digital convertor to sample the analog ser output and thereby produce digital data indicative of the measured breatr patterns.

[0093] Once the raw signals have been processed to put them a useable, standardized form, they are then fed to the classifier 104 wr functions to classify intervals of the measured breathing patterns accordinc predefined categories. In the presently preferred embodiment the classifie configured to classify breathing patterns into one of three types, namely N (normal breathing cycle), BC-ES (breathing cycle with exhale swallow, and I IS (breathing cycle with inhale swallow). These three types are discussed m fully above, and are used in the presently preferred embodiment to exti information from which the wearer's eating and drinking behavior can be infer through data analysis.

[0094] In the illustrated embodiment featured in Figs. 10 - 12, classifier was implemented as a set of matched filters. These filters can constructed to operate in the analog domain, or in the digital domain, depenc on the implementation. The classifer 104 may also be implemented usin trained stochastic, statistical, neural network or probabilistic model if desired such an embodiment the model may be trained using examples of knc breathing patterns (NBC, BC-ES and BC-IS) to define a set of classifiers, i tuned to recognize each of the breathing patterns.

[0095] In operation, classifier 104 provides as its output a detec result, typically an indication of which breathing pattern was recognized. In regard, because human breathing is basically cyclic, the classifier may configured to parse the raw data into inhale-exhale cycles, and then opei upon each cycle individually to classify it. In practice this is preferably done supplying the inhale-exhale cycle data to each classifier (that is, to the N classifier, the BC-ES classifier and the BC-IS classifier), allowing each class to generate an output reflective of how closely the input cycle matches breathing cycle it is designed to recognize. If desired each classifier may∑ output a similarity score that serves as a measure of how likely each class has performed its recognition task. Thus if the NBC classifier outputs a 9 likelihood score, and the BC-ES and BC-IS classifiers output 12% and 2 scores, respectively, the system can safely conclude that the breathing patl being examined is an NBC pattern. In the case where there is no clear wini the results of analysis for that breathing cycle may be discarded as unreliable.

[0096] As the classifier 104 produces its output, in the form c string or sequence of NBC, BC-ES and BC-IS classifications for indivic breathing cycles, this sequence is fed to the swallow pattern analyzer 106. ^' swallow pattern analyzer performs a computational analysis upon the breat patterns to extract inferences as to whether the wearer is eating or drinking, ; also in some cases what the wearer is eating or drinking. The swallow patl analyzer may be implemented using a computer or processor that has netw communication capability so as to receive geolocation data as well as date ; time data. By associating swallow patterns with date and time of day and \ geolocation, a record of when and where the wearer was eating or drinking. ^' swallow pattern analyzer may be based on a trained model, using patl recognition techniques discussed above, to classify different swallow sequences as corresponding to the eating and drinking of different foods beverages. The pattern recognizer can be trained in use so that it learns c time what a particular food or beverage looks like in terms of swallow pattei Training may be supervised by tying the swallow pattern analyzer to a fi intake analyzer 108 that includes a suitable user interface through which wearer logs in what he or she has eaten at each meal. The recognizer is trail by correlating the entered food and beverage items to different swallow pattei so that over time the swallow pattern analyzer becomes able to distingi between eating a bag of French fries and eating a carrot stick.

[0097] The subsystems that make up the food intake monitoi system described in Fig 18 can be implemented in a variety of ways. In sc embodiments many of the components can be embedded in the wears breathing sensor belt; in other embodiments some of the components may incorporated into a portable device carried by the wearer of the sensor bell still other embodiments some of the components may be remotely located ; accessed by network or telecommunication link, such as by WiFi connection 1 server located on the Internet. Thus the distribution of components illustrate! Fig. 18 can be deployed in a variety of different configurations.

[0098] In one embodiment, the wearable breathing sensor communicates by a wireless link, such as by BlueTooth, to a mobile device s as a smartphone 1 10. The smartphone includes at least one processor 1 12 1 performs program instructions to implement some or all of the sysl components 102 - 108. These program instructions are loaded and operate! memory 1 14 as an App, or as a Web-based application. The smartphone further includes a storage input/output subsystem 1 1 6, which includes onbc storage 1 1 8, such as flash memory 1 1 8, where the App may be stored and ί where data obtained by the food intake monitoring system may be stored. smartphone 1 1 0 also includes a radio communication system 1 20 that prefers includes support for short range BlueTooth communication, mid range V communication and optionally cellular communication, as well as GPS rece equipment to provide geolocating services.

[0099] In most cases, it is less taxing on the battery resource: the smartphone to digitize the raw signals obtained from the wearable breat sensor system 1 00 before they are sent via BlueTooth to smartphone 1 However, if desired, the smartphone can be equipped with analog-to-dic conversion circuitry 1 22, allowing the raw analog data to be sent and proces within the smartphone or by equipment attached to the smartphone.

[00100] The signal shaping 1 02, classifer 1 04, swallow patl analyzer 1 06 and food intake analyzer 1 08 functionality can be performed by processor 1 1 2 onboard the smartphone 1 1 0. However, if desired some of computationally intensive tasks may be offloaded to a server with which smartphone communicates by wireless network connection.

[00101 ] It will of course be appreciated that even using smartphone or other portable device there is still wide variation in hov particular embodiment may be deployed. Thus, while the foregoing has pla most of the computational tasks on the processor of the smartphone, ol division of labor is also envisioned. Thus, for example, the signal shaping functionality may be deployed on the wearable belt device, or on a small sysl attached to the wearable belt device. Also, for example, portions of functionality of the classifier 1 04, swallow pattern analyzer 1 06 and food int analyzer 1 08 can be performed on processor 1 1 2, with other portions of 1 functionality being distributed to other processors via network communication this regard, swallow pattern analyzer data from a plurality of users can uploaded to a server and used in the aggregate to refine swallow patl recognition models, with the results being used to adapt the recognition moc for download back to the individual users' smartphone devices.

[00102] Solid vs. Liquid Detection

[00103] Some examples of solid and liquid swallowing data collec on different subjects are shown in Fig. 19. The rising edge in the gr; corresponds to the inhalation and the falling edge corresponds to the exhalat The arrows in the figure indicate when swallows happen.

[00104] Figure 19 demonstrates the differences in derived signal betwi solid and liquid swallow. Figure 19(a) shows the two cases of one subject, w Figure 19(b) shows that of another subject. According to Figure 19(a), it can observed that the breaths are deeper during solid swallows and shallower dui liquid swallows. While for Figure 19(b), deeper breaths can be generally s< for liquid swallows, but breathing cycles for liquid can be very like the ones solid swallows, such as the second swallow in the right graph in Figure 19(b).

[00105] The scheme shown in Figure 20 is proven to be effective to de swallows from normal breathing cycles and differentiate solid and lie swallows. In this scheme, the data received is first fed into a low pass filte order to remover the steps caused by quantization and artifacts caused motion. Peak-valley detection picks up the peaks fan valleys in order separate and normalize different breathing cycles. Normalization is di because the lengths of breathing cycles are different among people and va from time to time. The normalized breathing cycles are then fed into feati extraction and feature selection modules, which would calculate features \ discriminative power and select part of those for optimum performar Comparing to the detection mechanism based on matched filter in our previ work, this machine learning based algorithm is able to provide be performance.

[00106] Support Vector Machine is used in the classification problem h because the decision boundary only depends on the support vectors wh number is usually small thus providing good generalizability, for which the S model is also simple and robust. We first use polykernel function to increase dimensionality of features in case the classes are not linearly separable. Tl the problem becomes finding the decision boundary f{x) = w ^T x + b, such 1 the geometrical margin of the training set to the boundary is maximiz

Geometrical margin is defined as γ = ^y(w , ^x * ^b function y calculates the func a ' IMI ^J

margin between the decision boundary and the training set, while | normalizes the function margin into geometrical margin. Geometrical marg in this case reflects the confidence of classification. Therefore, by selecting proper decision boundary {w ^T ,b), the SVM model would be able to provide \ classification confidence, which also means higher accuracy.

[00107] Table 5 shows the result of using machine learning algorithm: detecting solid and liquid swallows. In this result, machine learning algorithr trained on 90% of data set from that of each subject, and tested on remaining 10% of data set.

Table 5 Performance of solid/liquid swallow detection

[00108] The foregoing description of the embodiments has bi provided for purposes of illustration and description. It is not intended to exhaustive or to limit the disclosure. Individual elements or features c particular embodiment are generally not limited to that particular embodimi but, where applicable, are interchangeable and can be used in a selec embodiment, even if not specifically shown or described. The same may alsc varied in many ways. Such variations are not to be regarded as a departure fi the disclosure, and all such modifications are intended to be included within scope of the disclosure.