Background of the Invention

More than half of Americans aged 65+ experience a fall each year, costing the US healthcare system over $55B and the nation over 30,000 lives. Falling once doubles the likelihood or a second fall. The system and method of the present invention (termed “FallGAI” (Fall Generative Artificial Intelligence) for commercial purposes) is a ubiquitous, yet unobtrusive network of inexpensive sensors driven by artificial intelligence (Al) to not only predict falls, but to intervene in order prevent them. Designed to be installed in hospitals, senior centers, assisted living centers, and the homes of senior citizens, FallGAI uses deep learning and reinforcement learning Al, supplemented by generative adversarial networks, classical rule and data-driven expert systems, literature extraction, relational, object and big data models, and a historically-aware world model to produce high fidelity Al-driven results.

Older persons are more likely experience falls and fall injuries in particular activities of daily living, and that these may be identified and the risk quantified and then mitigated. Using multimodal sensing, behaviors are segmented by context, and position, posture, center of gravity and velocity and are sampled to characterize each context using machine learning coupled with a generative adversarial network for training and refinement. Edge cases tending to precede falls are identified for each context. This model is then validated against passive observations.

“Nudge” interventions are successfully be used to alter behavior away from those actions that present enhanced risk of fall and injury, and a combination of explicit interventions and nudges is believed to be most effective in reducing fall risk and fall injury in order persons. Effectors using Internet of Things (loT) technologies, as well as purpose-built effectors are deployed, actuated in context and the results evaluated by observation.

The present state of the art in dealing with falls among seniors is best exemplified by the trademark slogan, “I’ve fallen and I can’t get up!” (LifeCall Systems) Rather than prevent falls, current systems concentrate on detecting them (or more commonly, having the person who has just fallen take action) and providing assistance and medical care after they occur. Healthy aging requires that older persons not sustain life-threatening injuries from preventable falls.

One object of FallGAI is to prevent the majority of fall-induced injuries, first in in-patient settings (hospitals, skilled nursing facilities and assisted living settings), and then later in independent living settings and aging in place in seniors’ homes.

Description of the Invention

The system of the present invention uses sensors that detect the position and posture of human subjects without requiring either worn devices or physical contact. Sensors are deployed in healthcare facilities and residential settings and transduced information is used to refine the machine learning models of the system.

FallGAI uses integrated multimodal sensing (RADAR, LiDAR, LowRez Video, IRVideo, Audio, Temperature, Air Quality, force sensing, etc.) and deep machine learning Al to learn human behavior in context. FallGAI applies a postural center of gravity dynamic model to predict the probability of falls and to warn and intervene, based on that probability, before a fall occurs.

Rather than merely detect and respond after a fall occurs, FallGAI intervenes in many ways. It uses visual and auditory warnings, notifies facility staff, employs vibrotactile stimulation, and controls environment factors in real time to prevent falls.

Although FallGAI does not require worn devices of physical contact, it is able to integrate information derived from such devices, and also address them to provide information and feedback to the wearer. For instance, an Apple Watch (Apple, Inc.) may be provided with an “app” (application software module) to enable auditory and haptic cues, as well as to provide positional and acceleration/velocity information to the FallGAI system. It must be recognized, however, that older individuals and those with cognitive impairment often forget to don such devices, or to ensure that their batteries are fully charged, and thus, reliance on these devices may impair the operation of any system that critically depends on their operation.

Implementation of the Al uses PyTorch, an off the shelf machine learning tool that makes it simple to set up generative adversarial networks, and operates in the Amazon Web Services cloud, if directed to do so. AWS cloud is used for training, however the production version of FallGAI uses embedded computing and "edge computing” to avoid network latency that results from large sensor data rates.

A machine vision motion-capture system determines pose, center of gravity and velocity, and using a generative adversarial network classifies both present and likely near term future pose, center of gravity and velocity to identify fall risk. Examples of such systems include FreeMoCap (www.freemocap.org). Chordata (www.chordata.ee), and EasyMocap (http s : // github . com/ zj u3 dv/EasyMocap) . Ideally, markerless capture of motion provides pose data in both static and dynamic regimes to permit both identification of an individual within the field of view of the system, as well as changes in pose in near-real-time to permit extrapolation of stability, frailty and other parameters of motion. Using machine learning, poses representing both normal activities (sitting in a chair, on a bed, on a commode, resting prone in bed, standing) are classified, as well as those poses representing undesired activities (resting prone on a floor, kneeling adjacent to furniture or a wall or door, etc.). Similarly, dynamic changes in pose are also learned and classified (rising from seated to standing position, lowering from standing to seated position, transitioning from seated to prone positing in bed, etc.)

A system executive using data-driven entropy minimization and Baysian processing uses postural and positional data derived from machine vision and motion tracking models to determine human behavioral context (e.g., is the person about to arise from bed or chair, seated on a toilet, entering or exiting a shower, dressing, etc.) This context is used to select from among various Al modules, each having specialized training related to the context activated by the executive.

The executive takes action (warn, increase lighting intensity, vibrationally stimulate, lower or raise bed or chair height, etc.) to reduce the probability of fall and injury through a network of both commercial off the shelf (COTS) loT devices and bespoke actuators and effectors developed specifically for FallGAI.

It is well known that persons experiencing neuropathy in the feet are more likely to fall because of reduced proprioception. Such persons may be aided by good lighting, as visual feedback and cues augment one’s sense of position in space and proper placement of the feet to support standing and ambulation. In such cases, actuation of loT dimmers and switches in response to system-sensed changes in location and position may be employed to reduce the probability of falls and resultant injuries.

Among the system actuators is an interface to a hospital bed that, upon receipt of an actuating signal, engages the height control circuitry of the bed to adjust the bed to a predetermined height above the floor. (For example, the lowest height during sleep, or a height of about 1.3x the length of the bed occupant’s tibia at times when transitions to and from bed are likely.) For the lowest bed position, the system reduces the probability of fall injury should the bed occupant fall from the bed, and also reduces the ability of an older person to exit the bed, thus reducing the likelihood of falls. The same type of interface may be used with a chair having adjustable height, to achieve similar results.

Another bespoke actuator is a vibrotactile actuator embedded in a toilet seat. Because falls during toileting are a significant risk for injury, and many occur because the occupant become drowsy or falls asleep, the system of the present invention may sense lack of stability and incipient sleep. In response, the system may trigger the vibrotactile actuator according to one or more strengths and/or patterns to alert and awaken the occupant, thus reducing the probability of falls and resultant injuries.

Preventing a significant fraction of in-patient fall injuries will shorten length of stay, reduce complications, reduce legal liabilities, and free staff time to attend to primary medical needs. In today’s world of staff shortages and lengthening response times to patient calls for assistance (average is now over 13 minutes), leveraging automation to reduce frequent rounding and enforced toileting benefits care staff directly.

The reduction of in-home fall incidence will encourage aging in place, which has been shown to improve quality of life, reduce cost, and extend longevity.

The postural dynamics dataset generated will inform ancillary health assessments such as impairment from stroke, vestibular instability, positional hypertension, and many other conditions, aiding telemedicine and direct contact providers.

The first embodiment of the present invention uses a commercial 60GHz RADAR (Aqara),, motion capture software (FreeMoCap), and a PyTorch implementation of the system executive, embedding several explicit contexts (bed exit, toileting, chair exit, ambulation) and link to a few effectors (COTS loT devices.) System Components

Single-board computers - Raspberry Pi4B or similar and peripheral hardware

Sensors - 60-90GHz RADAR

20-40GHz Doppler RADAR NearIR Video Camera

Low Resolution Video Cameras

LIDAR

Ultrasonic Standing Wave Detector

Effectors - loT switches, dimmers, etc. Relay controller (purpose built)

Vibrotactiles