CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This claims the benefit of U.S. Provisional Patent Application No. 63/061 ,919, filed on August 6, 2020, the entire contents of which are incorporated by reference.

FIELD

[0002] This relates generally to systems for acquiring and processing data from sensors, and in particular from sensing arrays.

BACKGROUND

[0003] Sensing systems may be used to detect data from the external environment in the immediate vicinity of a sensor. Conventional sensing systems collect sensor data and attempt to analyze sensor data and draw various conclusions. However, sensing systems are limited in a number of meaningful ways. For example, sensing systems can be difficult to incorporate into a physical environment in such a manner as to allow sensors to be placed for accurate sensor readings without great risk of damage to sensors over the course of regular use.

[0004] Moreover, sensor systems tend to analyze one source of data at a time, or attempt to analyze the data without defining the exact relationships between different datasets. This often is the case for wearable devices. As such, present systems do not achieve an accurate or nuanced understanding of environmental data. It would be desirable to improve or more of the above-noted challenges associated with sensing systems.

SUMMARY

[0005] According to an aspect, there is provided a flexible electronic system, comprising: a plurality of sensors; a plurality of integrated circuits for receiving data from said sensors; a sensor tile partitioned into a plurality of portions, each of said portions comprising one of said sensors; a stem signal-carrying line travelling along the sensor tile in a first direction; a plurality of leaf lines branching off from said stem line, said leaf lines connecting to one or more of said sensors and integrated circuits.

[0006] According to another aspect, there is provided a method comprising: arranging a plurality of flexible sensor tiles on a deformable surface, said flexible sensor tiles comprising a plurality of sensors; receiving, at a microcontroller, sensor data from said plurality of sensors; transmitting, via a communications bus, said sensor data to an loT hub; aggregating, at said loT hub, said sensor data into a single cycle; correcting and/or regulating said single cycle data using a fault tolerance algorithm; rotating and/or segmenting said regulated single cycle data; transforming said rotated and/or segmented data into matrix formatted data; storing said matrix formatted data in a database; labelling said stored data; and visualizing said stored data to represent labelled and time-stamped data.

[0007] Other features will become apparent from the drawings in conjunction with the following description.

BRIEF DESCRIPTION OF DRAWINGS

[0008] In the figures, which illustrate example embodiments,

[0009] FIG. 1 is a schematic view of an example sensor tile;

[0010] FIGs. 2A, 2B, and 2C are top, bottom and side views of an example sensor housing unit;

[0011 ] FIG. 3A is a schematic diagram of an outer cover;

[0012] FIG. 3B is a schematic diagram of an inner cover;

[0013] FIG. 4A is an illustration of an example sensor array logic zone mounted to the underside of a mattress; [0014] FIG. 4B is an illustration of example a plurality of flexible sensor tiles mounted to the top surface of a mattress;

[0015] FIG. 5 is a block diagram depicting components of an example computing device;

[0016] FIG. 6 depicts a simplified arrangement of software at a computing device;

[0017] FIG. 7 is a simplified diagram depicting a process for converting and transforming of data from multiple sensors in an external environment into a reduced or single frame of visualization;

[0018] FIG. 8 is an overview of an example architecture for a modular flexible sensing array;

[0019] FIG. 9 is a schematic view of a sensor tile printed circuit board; and

[0020] FIGs. 10A and 10B are an electronic schematic configuration for an example sensor tile.

DETAILED DESCRIPTION

[0021 ] Some embodiments relate to systems and methods for acquisition and processing of data from a modular flexible sensing array. Some embodiments may have a multitude of applications by enabling smart surfaces for data acquisition for improved contextualized decision making. Some embodiments may facilitate provision of a product ecosystem that allows for scalability and modularity in sensing surface and/or and sensing modalities of environmental factors for data processing.

[0022] Some embodiments may be applicable to a variety of applications such as medical, wearables, consumer products, manufacturing, automotive, construction, supply chain and mining, and the like. Example of applications include, but are not limited to, predictive bedfalls, sepsis, infection control, deep vein thrombosis, urinary and fecal incontinence, spinal cord injury management, pain tracking, sleep quality monitoring, movement and posture tracking, snoring, menstruation, heart monitoring, breathing monitoring, skin breakdown, baby monitoring, inventory monitoring, asset tracking, inventory management, occupancy tracking, infrastructure monitoring, to name but a few.

[0023] Flexible electronic devices (hereinafter flexible electronics) have found a multitude of applications benefiting from the freedom of movement in all 3 axes of space associated with flexible electronics. Several major challenges and constraints may arise relating to reliability and form, particularly when the size of the flexible electronics increases. As the size increases, the need for dynamic movement and reliability may limit the use of conventional flexible electronic designs.

[0024] Various embodiments of the invention are described herein with reference to the drawings.

[0025] FIG. 1 is a schematic view of an example sensor tile 100. As depicted, sensor tile 100 is implemented as a printed circuit board (PCB). To mitigate the aforementioned challenges, some embodiments relate to systems and devices which may allow for uniform free movement of flexible electronics with an improved and/or high degree of reliability.

[0026] Some embodiments may include a patterned cut-out design around sensing elements 102 in a stem 104 and leaf 106 design. As used herein, the stem 104 refers to the signal-carrying lines traced across the length of the flexible electronic 100 and branching off across the width of the flexible panel to interface with the components (e.g. sensors 102 and MEMS islands 112). In some embodiments, this design may allow each sensing element to be cut out on 2 or 3 sides to subdivide or partition the flexible electronic 100 into a plurality of portions or “sensor peninsulas” 108a, 108b. As depicted, flexible electronic 100 is partitioned into many sensor peninsulas, some of which have 2 free ends (e.g. 108a), and some of which have 3 free ends (e.g. 108b). It will be appreciated that although FIG. 1 depicts peninsulas which are square or rectangular in shape, other shapes are contemplated. In some embodiments, a sensor peninsula 108a, 108b may allowing for some or all mechanical stress to remain localized to the particular peninsula. Thus, a sensor peninsula design may significantly increase mechanical reliability and the ability for flexible electronics to free-form around objects. In some embodiments, sensor peninsulas 108a, 108b may facilitate free- forming around a surface when subjected to an external force.

[0027] In some embodiments, flexible electronics 100 may be used in order to more fully utilize the space of an enclosure that provides mechanical protection to the electronics. In practice, this often limits the use of flexible electronics within an enclosure, as integrated circuit-specific protection of electronics might not be readily available. With the proliferation of sensors to enable direct monitoring of conditions of soft or complex objects, the protection of ICs in flexible electronics is important to allow free-forming electronics that monitor complex objects which are subjected to the environment.

[0028] In some embodiments, a sensor housing unit 202 may allow for the containment of standard Micro Electromechanical Systems (MEMS) and their termination ICs. FIGs. 2A, 2B, and 2C are top, bottom and side views of an example sensor housing unit, respectively. In some embodiments, housing unit may include inner and outer sloping walls and a slotted vent hole. As depicted, housing unit 202 has 3 mounting legs, but it is contemplated that other embodiments may include fewer than 3 legs or more than 3 legs. Housing unit 202 may reduce mechanical strain of the MEMS and solder pads while significantly reducing impact on profile and performance of the MEMS sensor. Without sensor housing 202, mechanical strain may be directly applied to ICs, and particularly to surface mount pads. In some embodiments, sensor housing unit 202 takes the mechanical loading forces (rather than the IC itself). In some embodiments, a sensor island stiffener may take a portion of mechanical loading forces. Sensor housing unit 202 and sensor island stiffener may together form an enclosure at the sensor island.

[0029] Interfacing sensing electronics for a large sensing area within a textile is technically difficult, as there is little freedom in terms of where solid components can be placed, the mounting requirements of solid components, and accessibility to the electronics for servicing and/or installation. Most approaches to large surfaces (specifically focused on mattress sensor mats) is to have the sensor processing ICs contained within enclosures around the side of a mattress or cushion, and then meshed permanently into one large unit. The fundamental technical drawback of isolating the processing outside the sensing surface is that there is a high likelihood of transmission trace breaks, which may require extensive servicing or replacement of the device when they inevitably occur.

[0030] To mitigate the drawbacks of integrating electronics into textiles, some embodiments include a double cover system. In some embodiments, a double cover includes an inner cover 302 that fits onto the mattress or cushion, and an outer cover 304. FIG. 3B is a schematic view of an example inner cover 302. FIG. 3A is a schematic view of an example outer cover 304. In some embodiments, the outer cover 304 may be stitched inwards to the edge of the inner cover bottom, coming slightly above the bottom edge of the inner cover, to form a pocket around the bottom edge of the mattress or cushion. The outer cover bottom section may be equipped with a zipper 306 around some or all of the perimeter of the mattress, thereby allowing for the outer cover top section to zip with the outer cover bottom section. The electronics may then be fastened onto inner covering 302. In some embodiments, electronics may be fastened onto inner cover 302 using any of hook and loop fasteners, double sided tape, welding, and the like. The use of a double cover system may allow for a number of advantages including but not limited to, the ability to house of solid components under the mattress or cushion, the ability to perform modular servicing, and greater design flexibility and customizability.

[0031] FIG. 4A is an illustration of an example sensor array logic zone mounted to the underside 402b of a mattress 402. In some embodiments, the sensor array logic zone may be mounted to the underside 402b of mattress 402 to maintain surface flexibility. In some embodiments, non-sensing electronics may be contained within housing unit 202. FIG. 4B is an illustration of a plurality of flexible sensor tiles 100 mounted to the top surface 402a of a mattress 402. Though not explicitly depicted in FIG. 4B, it will be appreciated that sensor peninsulas 108a, 108b may house an assortment of sensors 102 which may be in electrically and/or communicably coupled to MEMS islands 112. In some embodiments, flexible sensor tiles 100 are electrically and/or communicably coupled to sensor housing units 202 on the underside 402b of the mattress.

[0032] Some embodiments relate to a modular smart bedsheet. In some embodiments, the smart bedsheet incorporates one or more flexible sensing arrays 100. The smart bedsheet may involve a design of a sensor mat with multiple types of sensors 102 arranged in a patterned array. In some embodiments, sensors 102 may include any of gas, temperature, pressure and humidity sensors. Sensors 102 may be interfaced into a single free-forming flexible electronic array 100 through a localized logic node to mesh with a network of sensor arrays. In some embodiments, flexible electronics arrays 100 may be arranged together to form a larger sensing surface (as shown, for example, in FIG. 4B).

[0033] Some embodiments include one or more computing devices 500 for receiving, processing, storing, and/or transmitting various forms of data. FIG. 5 is a block diagram depicting components of an example computing device 500. As depicted, computing device 500 includes a processor 514, memory 516, persistent storage 518, network interface 520, input/output interface 522, and bus 526. It will be appreciated that various embodiments of computing device 500 may include additional or fewer components than those depicted in FIG. 5 depending on the functionality required.

[0034] Processor 514 may be an Intel or AMD x86 or x64, PowerPC, ARM processor, microprocessor, microcontroller, or the like. Processor 514 may operate under the control of software loaded in memory 516. Network interface 520 connects computing device 500 to other devices via a network. Network interface 520 may support domain-specific networking protocols. I/O interface 522 connects computing device 500 to one or more storage devices and/or peripherals such as keyboards, mice, pointing devices, USB devices, disc drives, display devices 524, sensors, and the like.

[0035] Software may be loaded onto computing device 500 from peripheral devices, from a network, from memory, or the like. Such software may be executed using processor 514. FIG. 6 depicts a simplified arrangement of software at a computing device 500. The software may include one or more of an operating system 628 and application software, such as sensor processing system 626.

[0036] As depicted above, multiple flexible electronic panels 100 may be arranged together into a larger sensing surface through the utilization of multiple communication protocols between a microcontroller 514 and sensors. In some embodiments, microcontrollers 514 and/or sensor processing system 626 may be configured to transmit messages for communication over a communications bus 526 (e.g. a Controller Area Network (CAN) bus). In some embodiments, communications bus 526 may be implemented as a single shared line running across the perimeter of the sensing surface. The data may be received by an Internet of Things (loT) hub that enables two-way communication between the sensor mat and an external server. In some embodiments, the data being transmitted by the loT hub may be processed to reduce size and improve reliability in data streaming prior to transmission.

[0037] In some embodiments, sensor processing system 626 may be configured to, at a microcontroller, perform a sensor calibration algorithm for data cleaning prior to transmission over the CAN bus 526 line. It will be appreciated that microcontroller can be embodied in numerous locations (e.g. as shown in FIG. 9) and configurations suitable for performing the functionality described herein. At the loT hub 702, the data may be intelligently manipulated to represent the location of sensor data in relation to the entire sensing surface, and then aggregated into a single row for low-size transmission and database access and storage. It will be appreciated that loT hub 702 may be embodied by a computing device 500, and may be located in numerous possible locations suitable for accomplishing the functionality described herein.

[0038] In some embodiments, sensor processing system 626 may be conceptualized as two distinct but related systems, loT-driven Communication (l-dC) and loT-driven Artificial Intelligence (l-dA). In some embodiments, l-dC includes communication that is established by a multi-node system that interfaces with multiple loT sensors arranged in an array. l-dC may maintain communication within and between multiple loT sensors with a microcontroller node using multiple communication protocols, including but not limited to, Serial Peripheral Interface (SPI) and analog. Associated microcontroller nodes may be meshed using a distributed communication system (e.g. CAN) to interface with an internet-connected loT hub. This may enable two-way communication between the CAN and external devices (e.g. the internet).

[0039] In some embodiments, 1-dA involves artificial intelligence (Al) algorithms that enable the pre-processing of incoming data at the level of the CAN and loT Hub. In some embodiments, 1-dA may regulate and correct incoming sensor data. Such regulation and correction may include, for example, calibration of missing data and/or data values below or over a specified or expected threshold. In some embodiments, I- dA regulation and correction may ensure balancing of incoming data in real-time.

[0040] FIG. 7 is a simplified diagram depicting a process for converting and transforming of data from multiple sensors in an external environment into a reduced or single frame of visualization. At block A, data from sensors in an external environment is gathered, collected and extracted. Such data may be gathered, collected and extracted from some or all sensors to CAN bus 526. In some embodiments, CAN bus 526 may transport raw sensor data to an loT Hub (i.e. MASTER in FIG. 7). In some embodiments, loT hub 702 may be implemented on a computing device 500 as described herein. At block B, loT hub 702 may be configured to aggregate the raw data from some or all sensors into a single cycle that represents multiple sensor readings from block A.

[0041] At block C, once the raw data has been aggregated into a single cycle, a fault tolerance algorithm (FLA) may be applied to ensure the regulation of the single cycle raw data. In some embodiments, FLA detects and corrects any missing, absent, below desired threshold and/or above desired threshold data values within the single cycle raw data. In some embodiments, corrected and regulated data may be stored (e.g. in storage 706) before subsequent data processing.

[0042] At block D, within loT Hub 702, corrected and regulated post-FLA data may be rotated and segmented (referred to as algorithm-driven data positioning) to ensure representation of the external environment as well as the desired outcome. Rotation and segmentation may be important, as it may allow software and desired outcome to be independent of the orientation of sensors in the external environment.

[0043] At block E, following data rotation and segmentation, the data may then be transformed into matrix format. In some embodiments, the data may be transformed to a single matrix. Transformation of single cycle data to a single matrix may allow a single matrix to represent a single row in the database. The data in matrix format may then be stored in storage 704.

[0044] At block F, stored data may be labelled to represent, for example, posture detection and use for one or more other purposes.

[0045] At block G, the data stored as a single row in the database may undergo a visualization process to represent a single frame of an object on the sensors. A single frame of the visualization may represent data that was labelled and time-stamped, so as to represent historical events that can be reviewed or replayed, as well as predict unique values within the frame per cycle that may be useful for multiple technical purposes, including but not limited to modeling and diagnostics.

[0046] Some embodiments relate to a smart bedsheet. A smart bedsheet may function by converting and transforming data from multiple sensors of an external environment into a single frame of data visualization. At block A, data may be extracted from sensors and sent to the CAN bus. At block B, data is received at loT hub 702 from CAN bus 526, and said received data is aggregated in a single cycle which may represent multiple climate sensor readings. At block C, FLA may be used to detect, for example, missing sheets in a cycle (e.g. missing sensor values or value thresholds within the sheet). At block D, due to sensor orientation layout, data in a cycle is rotated to represent the actual layout of the external environment. This may allow for sensors to be in any orientation because the algorithm can rotate for certain configurations of the external environment. At block E, data is transformed into a matrix to represent one row in a Database. At block F, data may be labelled or otherwise prepared for multiple purposes (robust data set), such as posture detection. At block G, visualization of one row in the database may represent a single frame of an object on sensors. In some embodiments, each cycle may include of microclimate data that was labelled and time- stamped to represent a unique visualization per climate type, in which historical events may be reviewed and replayed, as well as to predict unique values within the frame per cycle that may be useful for multiple technical purposes (e.g. modeling, diagnostics, or the like).

[0047] Some embodiments may use historical data sets to determine localized skin response. For example, historical information of patients may be used to further enable the development of a pressure injury (PI) prevention artificial intelligence (Al) system. The data sets may be unique, and unique data sets may be used for one or more of: 1 ) helping clinicians assess and document overall patient pressure ulcer (Pll) risk, 2) helping clinicians assess and address modifiable pressure injury (PI) risk factors, and 3) guiding clinicians toward an evidence-based protocol of care for patients with impaired skin integrity. To that end, artificial intelligence (Al) may facilitate broader learning by leveraging historical datasets and combining them with patient-specific data to enable clinicians to better form a diagnosis. Al’s deep learning algorithms and analysis capabilities may be deployed to traverse massive amounts of data and detect a few variables across hundreds of thousands of data points that may be specific to certain conditions, such as Pls. It may be possible to use this deep learning to analyze electronic health records (EHRs) to enable prediction of Pls.

[0048] In some embodiments, systems and methods described herein may be used to detect localized responses in skin. Current technology uses sensors with only a single measurement parameter as a predictor of skin integrity. Examples include: interface pressure and movement orientation (relative to x,y,z axis). It is trite that skin that is in contact with an object under sustained mechanical load is susceptible to degradation. However, many of the technologies currently available in the marketplace are unable to (1 ) monitor the integrity of the skin without direct contact with the skin, (2) monitor the entire contact area of the skin without direct contact with the skin, and (3) mapping sensor data and comparing that data to patient-specific historical data. Current technologies do not take a multifactorial approach to assessing or determining the integrity of the skin. Some embodiments of the systems and methods described herein may use a combination of the skin microclimate (e.g. temperature, humidity next to the skin surface, and the like) as an indirect pressure ulcer risk factor, in combination with interface pressure. Some embodiments of the system may weigh some or all possible risk factors (extrinsic) and compare those to intrinsic risk factors and extrapolate the risk of the skin relative to where the location is on the patient’s body.

[0049] Conventional systems utilizing multiple sources of data often analyze one source of data at a time, or attempt to analyze the data without defining the exact relationships between different datasets. This often is the case for wearable devices, in which the data is not related by its relevance to time and space to achieve a more accurate understanding of the complete situation when analyzing.

[0050] Some embodiments were developed with the framework of creating a new dataset which may allow for an improved understanding of temporal and/or spatial relationships between arrays of different sensor types. This approach may be achieved by unifying data sets using a master reference point. In so doing, a more complete understanding may be achieved using associated different sensors together to understand the impact of each variable on one another. For example, the association of temperature sensors with pressure sensors may allow for a better understanding of the causes of temperature changes being attributable to conduction or convection.

[0051] Of course, the above described embodiments are intended to be illustrative only and in no way limiting. The described embodiments are susceptible to many modifications of form, arrangement of parts, details and order of operation. The invention is intended to encompass all such modifications within its scope, as defined by the claims.

WHAT IS CLAIMED IS:

1. A flexible electronic system, comprising: a plurality of sensors; a plurality of integrated circuits for receiving data from said sensors; a sensor tile partitioned into a plurality of portions, each of said portions comprising one of said sensors; a stem signal-carrying line travelling along the sensor tile in a first direction; a plurality of leaf lines branching off from said stem line, said leaf lines connecting to one or more of said sensors and integrated circuits.

2. The system of claim 1 , wherein the sensor tile is implemented as a printed circuit board.

3. The system of claim 1 , wherein at least one of said portions has 3 free ends.

4. The system of claim 1 , wherein at least one of said portions has 2 free ends.

5. The system of claim 1 , wherein said portions are configured to free-form around a surface when subjected to a force.

6. The system of claim 1 , further comprising a plurality of sensor housing units, each of said sensor housing units comprising inner and outer sloping walls and a vent aperture.

7. The system of claim 1 , further comprising: an inner cover configured to fit onto a mattress or cushion; and an outer cover configured to form a pocket around a bottom edge of said mattress or cushion.