Technical Field

The present invention generally relates to infant monitors.

Background Art

[0001] The best sensor for no contact (no wires or wearables) detecting vitals of a living subject is radar because of its sensitivity and ability to permeate materials such as blankets and clothing. Radar has its own set of problems, however, such as susceptibility to RF reflections, spatial sensitivity limited to aimed antennas which are at least partially sensitive in 4p steradians, and low spatial resolution of static objects.

[0002] The best sensor for detecting presence of a warm-blooded living subject is a LWIR camera which is able to detect blackbody radiation emitted by the target. This sensor, however, is not ideal for detecting vital signs due to its low range resolution and inability to detect motion perpendicular to the sensor.

[0003] The current market is provided only products which rely heavily on visible-range camera information to detect presence, and also for detecting vitals. These systems are compute-heavy, unreliable and easily fooled, are prone to error in dynamic lighting scenarios, and lack the sensitivity to achieve highly accurate vitals detection.

[0004] Additionally, a camera based solution has historically become a privacy pain-point for privacy-minded consumers. Due to the compute-heavy nature of these systems, digital processing is often computed in the cloud which can require sending and often storing personally identifiable information on a remote server.

[0005] Radar has the sensitivity to detect vitals but presence detection of still or small objects can yield results with low confidence. This is due to the radar’s inability to select targets spatially due to the 4p sr sensitivity of an antenna and its susceptibility to multipath and other interference. LWIR is able to detect presence with high confidence but does not have the resolution for vitals detection. Summary Of The Invention

[0006] The best way to mitigate these problems is via the inclusion of a LWIR detector which is able to detect black body radiation emitted by the living subject within the detector’s fixed and restricted field of view. A LWIR detector is not fooled by RF reflections, is not partially sensitive outside of its designed field of view, and is able to observe blackbody radiation emissions even of static objects.

[0007] A radar is orders of magnitude more sensitive for detecting vitals compared to a camera, and LWIR typically has superior discrimination performance for detecting presence of living targets, compared to traditional visible-range image-based solutions.

[0008] Additionally, both these sensors are completely immune to dynamic visible-spectrum lighting conditions. Additionally, the compute requirement is significantly lower, to the point where designing a local processing system is trivial. [0009] Because local processing is trivial, transmission of personally identifiable data (which this system does not need to measure or collect) is not sent to a remote server.

[00010] Using both sensors, the present invention obtains presence detection and vitals detection with high confidence. [00011] The present invention is a system for using a single pixel or imaging long-wave infrared (LW IR, such as a PIR sensor) “thermal camera” and a radar (single or multichannel, CW, FMCW, PD, etc) for detecting presence and vital signs of a warm-blooded living target.

[00012] Additionally, this solution works without the use of a visible camera. [00013] Additionally, visible-spectrum image and video processing for presence and vitals detection is orders of magnitude more compute-heavy compared to low resolution LWIR and radar processing.

[00014] Therefore, this solution does not necessitate sending any data to a remote server for computation.

[00015] One aspect of the present invention is a system for presence and vitals detection of a living subject. The system comprises a passive long wave infrared (“LWIR”) sensor, a radar, a processor and a user interface. The LWIR sensor is utilized to detect block-body radiation originating from a living subject. The radar emits a radiofrequency at a specific frequency, and detects the frequency change of reflections of a plurality of targets which have subtle movements caused by the respiration and/or ballistocardiography from the living subject. The processor is configured to run an algorithm to perform digital signal processing on data provided by the radar and the LWIR sensor to generate presence and vitals information for the living subject for communication to the user interface.

[00016] Another aspect of the present invention is a method for presence and vitals detection of a living subject. The method includes detecting at a LWIR sensor of a monitoring device block-body radiation originating from a living subject. The method also includes emitting from a radar of the monitoring device a radiofrequency at a specific frequency, and detecting the frequency change of reflections of a plurality of targets which have subtle movements caused by the respiration and/or ballistocardiography from the living subject. The method also includes receiving at a processor of the monitoring device the data from the radar and the LWIR sensor. The method also includes running on the processor an algorithm to perform digital signal processing on the data provided by the radar and the LWIR sensor to generate presence and vitals information for the living subject. The method also includes communicating from a first communication module of the monitoring device the presence and vitals information for the living subject to a second communication module of an interface device. The method also includes presenting on a user interface module of the interface device the presence and vitals information for the living subject.

Brief Description Of The Drawings

[00017] FIG. 1 is an illustration of an antenna pattern.

[00018] FIG. 2 is a block diagram of one embodiment of a system for presence and vitals detection of a living subject.

[00019] FIG. 3 is a block diagram of a process for presence and vitals detection of a living subject.

[00020] FIG. 4 is a flow chart of a method for presence and vitals detection of a living subject.

Best Mode(s) For Carrying Out The Invention [00021] The present invention is a system which consists of a LWIR camera or detector bore-sighted with a radar sensor, and a set of algorithms which operate internally on the system, which determine and transmit presence and vitals information remotely to an observer.

[00022] A sensor consists of a passive long wave infrared low-resolution single-detector or imaging sensor is used to detect black body radiation originating from a warm blooded living subject (all of which naturally radiate black body heat). A radar emits RF at a specific frequency, and detects the frequency change of reflections of targets which have subtle movements caused by the respiration and/or ballistocardiography from a living subject. [00023] The field of view of the LWIR sensor and the transmit and receive antenna main lobe sensitivity are roughly collocated, with similar attitude (azimuth and zenith angle). The radar pattern is at least partially sensitive in 4p sr surrounding the radar antenna. An LWIR sensor such as a thermal imager, PIR, or other LWIR detector, is used In order to ensure the spatial sensitivity required to identify the presence of targets, greatly increasing the confidence of a radar system in determining presence of a target.

[00024] FIG. 1 is an illustration of an antenna pattern 100 showing the target of interest 10 (the living subject), the LWIR sensor’s field of view 15, the radar sensitivity pattern 20, and the Azimuth angle 25.

[00025] Digital sensor data from these sensor modules are provided for ingestion by a processor unit, which runs an algorithm internally to perform digital signal processing, feature extraction, decision logic, and preparation for communication with user’s module. Data is then transmitted from sensor system communication module to the corresponding user interface system’s communication module.

[00026] A user interface system consists of a communication module which receives data from its corresponding sensor system. This data is presented to the user via a user interface such as an LED, a display, a speaker, or any other manner of interface.

[00027] A long wave infrared detector determines a presence of living subject by detecting black body radiation at 8pm to 14pm (8,000 to 14,000nm) for example. An example of a commercial product is a FLIR Boson 320, Seek Contact.

[00028] A radar detects the vitals of living subject by detecting reflected RF.

An example of a commercial product is a Xethru X4, Miku Smart Monitor.

[00029] A compute module with memory and communication module performs the entirety of the presence and vitals detection within the sensing system.

[00030] A user interface system with a communication module receives data and presents information to a user.

[00031] A single pixel presence detector performs the following function: monitor sensor output x[n] with sample rate S Hz and store measurement in system memory, a sample rate S could be 1 Hz for example; Apply high pass filter y[n] = x[n] - x[n-k] for time constant k, time constant k could be 10 to represent 10 second delay; and search for y[n] > a & x[n] > b, change threshold a , to detect sudden changes, temperature threshold b, to select temperatures which exceed a threshold.

[00032] An imaging sensor presence detector performs the following function: (1) monitor sensor output X = xi[i,j], X2[i,j], ... Xn[i,j] with sample rate S Hz, (a) i, j represent row and column dimension of sensor image and could be (320, 240) for example, (b) sample rate S could be 1 Hz for example; (2) Compute aggregated background value m[n] of image to estimate background temperature, (a) Aggregated background value could be median of all pixels, image pooling, image segmentation, or other; (3) Subtract background from image using bk[n] and segment result, (a) Threshold-and-mask for example:

(i) Threshold image based on bk[n] to generate image mask, (j) y _n [i,j] = X _n [i,j]

> bk[n] for all i,j in range; (ii) Perform morphological denoise transformation on mask such as erosionl (iii) Sum all values of y _n to calculate mask area a[n]; (iv) If a[n] > k for area threshold k: (1) Apply mask y _n to x _n to generate z _n ; (2) Sum pixel values of z _n to generate v[n]l; (3) Divide v[n] by a[n] to get mean value of segment m[n] (4) Compare m[n] mean value to temperature threshold b.

[00033] An imaging sensor presence detector performs the following function: (1) monitor sensor output X = xi[i,j], X2[i,j], ... Xn[i,j] with sample rate S Hz, (a) i, j represent row and column dimension of sensor image and could be (320, 240) for example, (b) sample rate S could be 1 Hz for example; (2) Compute aggregated background value m[n] of image to estimate background temperature, (a) Aggregated background value could be median of all pixels, image pooling, image segmentation, or other; (3) Subtract background from image using bk[n] and segment result, (a) Threshold-and-mask for example:

(i) Threshold image based on bk[n] to generate image mask, (j) y _n [i,j] = X _n [i,j]

> bk[n] for all i,j in range; (ii) Perform morphological denoise transformation on mask such as erosionl (iii) Sum all values of y _n to calculate mask area a[n]; (iv) If a[n] > k for area threshold k: (1) Apply mask y _n to x _n to generate z _n ; (2) Sum pixel values of z _n to generate v[n]l; (3) Divide v[n] by a[n] to get mean value of segment m[n] (4) Compare m[n] mean value to temperature threshold b.

[00034] FIG. 2 is a block diagram of one embodiment of a system 200 for presence and vitals detection of a living subject 10. As shown in FIG. 2, the system 200 for presence and vitals detection of a living subject 10 comprises a sensor system 30 (monitoring device) and a user interface device 40. The monitoring device 30 comprises a passive long wave infrared low-resolution (“LWIR”) sensor 32, a radar 34, a processor 36, and a first communication module 38. The interface device 40 comprises a second communication module 42 and a user interface module 44. The LWIR sensor 32 is utilized to detect block-body radiation originating from a living subject 10. The radar 34 emits a radiofrequency at a specific frequency, and detects the frequency change of reflections of a plurality of targets which have subtle movements caused by the respiration and/or ballistocardiography from the living subject 10. The processor is configured to run an algorithm to perform digital signal processing on data provided by the radar 34 and the LWIR sensor 32 to generate presence and vitals information for the living subject 10 for communication to the interface device 40.

[00035] The radar can be a pulsed Doppler radar, FMCW radar, a continuous wave radar, or any other type of radar.

[00036] The user interface preferably presents data using a LED, a display or a speaker. The user interface preferably comprises a second communication module for receiving data from a first communication module in communication with the processor.

[00037] The memory is preferably configured to store sensor output from the LWIR sensor. [00038] The LWIR sensor is preferably an imaging sensor presence detector or a single pixel presence detector.

[00039] The living subject is preferably an infant.

[00040] The first communication module and the second communication module preferably operate on a WiFi communication protocol.

[00041] FIG. 3 is a block diagram of a process 300 for presence and vitals detection of a living subject. As shown in FIG. 3, a passive long wave infrared low-resolution (“LWIR”) sensor 332 determines a presence of living subject 10 by detecting black body radiation. A radar 334 detects the vitals of a living subject 10 by detecting reflected RF. The sensor system 330 processes the data and communicates the data from a communication module 338 to a communication module 342 of the user interface system 340, which displays the data.

[00042] FIG. 4 is a flow chart of a method 400 for presence and vitals detection of a living subject. Step 401 starts at a LWIR sensor of a monitoring device, detecting black-body radiation originating from a living subject. Step 402 is emitting from a radar of the monitoring device a radiofrequency at a specific frequency, and detecting the frequency change of reflections of a plurality of targets which have subtle movements caused by the living subject. Step 403 is receiving at a processor of the monitoring device the data from the radar and the LWIR sensor. Step 404 is running on the processor an algorithm to perform digital signal processing on the data provided by the radar and the LWIR sensor to generate presence and vitals information for the living subject. Step 405 is communicating from the monitoring device's communication module 38 the presence and vitals information for the living subject to an interface device’s communication module 42. And step 406 is presenting on an interface device the presence and vitals information for the living subject.

[00043] White et ak, U.S. Patent Number 10825314 for a Baby Monitor, is hereby incorporated by reference in its entirety.