Field of the Invention

This invention relates generally to a computer-implemented system, and method for monitoring an environment and identifying therein a predetermined condition and/or a characteristic or feature of an occupant thereof.

Background of the invention

Thermal sensors operate in the infrared band in the electromagnetic spectrum, i.e. , it senses light waves invisible to the human eye. The infrared spectrum is divided into three subbands: (1 ) NIR, (2) MIR and (3) FIR. All three bands are used to passively capture a heat map (i.e., temperature profile) of the camera's field of view. However, the three bands are used for monitoring different temperature ranges, and hence require different operating imaging technology.

NIR imaging operates between the 0.7 to 2.5pm wavelengths range and can monitor temperature ranging from 600 degrees C to 1 ,000 degrees C. NIR is typically used in the industrial space, given its temperature and price range. MIR imaging operates in wavelengths from 1 ,3pm to 8pm and observes temperature ranges from 5 degrees C to 300 degrees C. Thermal cameras operating in the FIR spectrum with wavelengths between 7.5 and 13pm can capture temperature ranges between -20 degrees C and 900 degrees C. FIR thermal sensors are considered to be the most commercially available thermal cameras in terms of small size and cost effectiveness.

In more recent years, attempts have been made to incorporate thermal sensors into everyday appliances such as televisions and remote controls.

CN1 13422993 describes a television remote controller with a body temperature monitoring function, including a contact-based body temperature sensor for determining user body temperature, which is then displayed to the user. Room temperature data can also thus be generated and displayed. KR20090075175A describes a system for controlling power to a television based on whether or not there is a person in a field of view. Such person identification is achieved using an infrared camera located in front of the television, which transmits thermal data to a signal processing unit. The signal processing unit determines, using the temperature data in the image, a temperature distribution; and user presence is detected by using the difference between the ambient temperature and the temperature of a user in the field of view. If the presence of a user is so detected, the system acts to switch the power to the television on. When no such user is detected, the power is switched off.

CN204206375U describes a system for controlling the air quality and temperature in a room, using data generated by various sensors provided on a television. A body infrared sensor can determine the body temperature of room occupants and, in response to changes therein, the system can control an air conditioning and humidifier system.

These systems utilse raw temperature data acquired from the field of view of an infrared sensor to, either generate a respective informational display or control another system. There is no further analysis or utilization of the raw temperature data.

CN1 11385650A describes a system and method for monitoring body temperature of the users of a television. An RGB camera is provided on the television for capturing images of the users. An infrared sensor captures body temperature data in respect of the users, and maps the body temperature measurements onto the images captured by the RGB camera, so that an image of the users, together with their individual body temperatures, can be displayed on the television.

WO201 4/007555 similarly describes an RGB camera and facial recognition system, which captures images of users and identifies them. An infrared sensor captures thermal data in the field of view and maps that thermal data onto the image data captured by the camera, so that the body temperature of the users can be monitored. This body temperature data is stored in a database and periodically sent for monitoring by a medical institution.

In these two systems, the raw temperature data is simply mapped onto an RGB image, to provide locational body temperature information. However, there is no further analysis of the temperature data, nor is there any other output. Due to the operation, nature and price history of thermal imaging, previous applications have been limited to those intended for very specific domains such as medical, firefighting, and industrial settings. Summary of the Invention

It is an object of aspects of the present invention to utilise thermal sensors in a novel way by deploying and integrating them into ubiquitous computing devices such as smart TVs and loT devices. Examples of the invention envisage the collection and utilisation of temperature data without endangering users’ privacy, and thus enabling the improvement of users’ home experience, to make it smarter, safer, more seamless, personalised and adaptive to users’ profiles and behaviours.

In accordance with a first aspect of the invention, there is provided a computer implemented system comprising at least one thermal sensor embedded, or otherwise integrated into, an appliance intended for use in an environment, a processor and a memory, the thermal sensor being configured for contactless capture of thermal image data from within its field of view in a said environment in which said appliance is located, and the system being configured, under control of the processor, to execute instructions stored in the memory to: receive a thermal image data feed captured by said thermal sensor; process said thermal image data feed to generate thermal images of said field of view; identify, within said thermal images, at least one occupant or region of interest within said environment; analyse said thermal images and determine therefrom, in respect of said identified at least one occupant or region of interest, one or more thermal conditions, characteristics or features; and output data identifying said occupant or region together with data representative of said determined one or more thermal conditions, characteristics or features.

Other aspects and optional and/or advantageous features of the invention are set out in the appended claims. Brief Description of the Drawings

Embodiments of the present invention will now be described, by way of examples only, and with reference to the accompanying drawings, in which:

Figure 1 A is a schematic diagram illustrating integration of a thermal sensor in a television remote control and/or a smart TV;

Figure 1 B is a schematic block diagram illustrating a computer-implemented system according to an exemplary embodiment of the present invention;

Figure 2A illustrates schematically the concept of capturing thermal reflectivity from a display screen using a thermal sensor embedded in a remote control;

Figure 2B illustrates the imaging result using a thermal sensor facing a television capturing a user sitting behind the sensor through thermal reflection;

Figure 3 illustrates user identification using thermal pattern recognition in relation to the veins of the hand;

Figure 4 shows a view of a floor captured by RGB camera (left) and thermal sensors (right), showing the advantage of using thermal sensors over their RGB counterparts as the thermal sensor is capable of capturing traces of someone who has passed by in the near past;

Figure 5 shows an image captured by a thermal sensor of an overheated cooking pot;

Figure 6 is a schematic diagram illustrating the operation of a health monitoring system according to an exemplary embodiment of the present invention;

Figure 8 illustrates nose and forehead extraction within thermal images at different stages of the feature extraction algorithm;

Figure 9 is a table illustrating a sample of detectable emotions based on changes in facial temperature; Figure 9 is a schematic diagram illustrating the operation of an emergency monitoring system according to an exemplary embodiment of the invention;Figure 10 is a schematic illustration of a system according to an exemplary embodiment of the invention in use, wherein a thermal sensor embedded in the TV is used to detect viewers thereof; and

Figure 11 is a schematic illustration of a system according to an exemplary embodiment of the invention, in use, wherein a thermal sensor embedded in the TV is used to detect and count viewers thereof.

Detailed Description

Thus, it is an object of aspects of the invention to utilize thermal sensors in a novel way, by deploying and integrating thermal sensors in our daily ubiquitous computing devices, such as smart TVs and smart homes Internet of Things (loT). These devices already collect personal user data, where users' data can aid to build adaptive interfaces that achieve more effective/efficient input, and more personal output. However, most of the data is either collected explicitly, hence, interrupting the users’ experience, for instance, asking user every time to enter their Netflix password to authenticate, or the data being collected is privacy invasive e.g., installing cameras in smart TVs.

On the other hand, it is envisioned that embodiments of the invention will use thermal sensors that collect temperature information from their field of view without endangering users’ privacy. In an exemplary embodiment, it is proposed to establish a platform with a dedicated focus on leveraging thermal sensors to improve users’ home experience, to make it smarter, safer, more seamless, personalised and/or adaptive to users' profile and behaviour. This may be achieved by utilizing concepts, modules, and applications, which utilise thermal sensors to monitor, transfer and reuse behaviour data and knowledge across diverse domains. Thermal sensors have become commercially viable in terms of cost and size, which enables the novel collection of non-visible spectrum.

One exemplary embodiment focuses on the integration of thermal sensors in smart TVs and their accessories (e.g., remote control). Even for the remote control integration, thermal sensors embedded therein can be used to capture thermal reflection of the display, i.e., the users usually point the remote controller towards the TV; hence, the TV would act as a thermal mirror as illustrated schematically in Figures 1 A, 2A and 2B of the drawings.

Referring to Figure 1 B of the drawings, a computer-implemented system according to the invention comprises at least one thermal sensor 10 embedded, or otherwise integrated into, an appliance 12 (e.g. a television or remote control) intended for use in an environment, a processor 14 and a memory 16, the thermal sensor 10 being configured for contactless capture of thermal image data from within its field of view 18 in a said environment in which said appliance is located, and the system being configured, under control of the processor 14, to execute instructions stored in the memory 16 to: receive a thermal image data feed captured by said thermal sensor 10; process said thermal image data feed to generate thermal images of said field of view; identify, within said thermal images, at least one occupant or region of interest within said environment; analyse said thermal images and determine therefrom, in respect of said identified at least one occupant or region of interest, one or more thermal conditions, characteristics or features; and output data identifying said occupant or region together with data representative of said determined one or more thermal conditions, characteristics or features.

In an example, the environment may be monitored periodically and, if the average temperature exceeds a predetermined threshold, the room may be scanned to identify the “hottest spot” in the field of view of the camera (thermal sensor) to identify an overheating object. The owners may then be notified of the location in the environment of the overheating object so that they can take any necessary action to prevent a fire hazard before it occurs. In another example, the system may be configured for user identification. All users have a unique thermal profile, whether that be their face, body temperature distribution or vein patterns. Thus, they would be able to set up their TV with their own unique thermal profile, and a database would store the thermal profiles of registered users. The system could be configured to periodically compare the users in front of the TV and the database and, if there is a non-matching profile, the system generates an alert to the owners. Different selectable responses could be provided. For example, ‘activate guest mode’ if the owners declare the unidentified profile as a guest, or ‘notify the authorities’ if the owners declare an intruder. In this case, the thermal image data feed can be analysed in real time and in a fully encrypted manner to identify threats as well as apply the early alarm system. One approach for realising privacy and safety use cases is to identify users based on thermal readings from their veins (e.g. in their hands) using thermal pattern recognition by combining thermal imaging with computer vision algorithms, as illustrated in Figure 3 of the drawings.

Examples of potential applications

1 . Privacy, security and safety

Home safety and surveillance are highly important features of the current modem homes. Specifically with smart homes, where the home acts as a hub for users' devices and data. Currently, most homeowners have a surveillance RGB or an Infrared camera, however, such cameras require special lighting sources or conditions.

An embodiment of the present invention aims to integrate thermal sensors in domestic setups, to enable the robust monitoring of a certain environment, regardless of the environmental conditions. The benefits of using thermal sensors as opposed to other imaging sensors are:

1 . Seeing in the Dark and Challenging Environments: Thermal sensors operate in the far infrared; hence, it does not require any external source of light. In other words, it can operate in complete darkness.

2. Seeing in the Past: Another advantageous feature of thermal sensors, is the ability to maintain/ keep the information over time. For instance, the heat traces left behind form walking over the floor. As thermal sensors capture all thermal information and properties, it could capture heat transfer i.e. , heat traces left behind. As illustrated in Figure 4 of the drawings, thermal sensors could detect if someone passed by this area, even if they are not there anymore. Unlike existing technologies that can only view live streaming of what is currently happening, thermal sensors can also view what happened in the past without the need of explicitly rewinding the surveillance video.

3. Seeing in the Future: Thermal sensors can act as early fire alarm, where they capture over-heating objects in the environment and alert the users for any potential risks. Figure 5 shows an unattended over heating cooking pot, this information cannot be captured by RGB cameras and the ordinary fire alarm will only detect the danger from the smoke, which is in many cases; too late. This feature can scale up to early leakage detection, electricity over consumption and so many heat-related in-home threats.

In summary, embodiments of the invention introduce novel and enhanced home safety features. Where thermal sensors are integrated in the users’ TVs, they can detect trespassers regardless of the environment conditions, and the system could be configured to generate an alert or other data informing a user that a trespasser has been detected. Alternatively, or in addition, the system could be configured to monitor the environment for other threatening conditions, such as over heating, and generate an alert as required.

In an exemplary embodiment, the thermal image data captured by the thermal sensor could be analysed in real-time in a fully encrypted manner to identify temperature-based threats, as well as apply an early alarm system. Furthermore, one approach for realising privacy and safety use cases is to identify the users based on thermal readings from their veins, using a thermal pattern recognition pipeline and an authentication part. For example, by combining thermal vision and computer vision algorithms, users can be reliably identified using the veins on the back of their hands.

2. Health and wellbeing

Thermal sensors can provide information about the observed body temperature; hence why they can detect any health issues reflected in the body temperature. For instance, they can remotely measure the users’ temperature and notify them if any irregularities were found. Over the past two years, due to the spread of Covid-19, both public spaces e.g., airports, have been equipped with thermal sensors to remotely capture users’ body temperature. Examples of the invention aim to utilize the cost and size reduction of thermal sensors and integrate them in domestic setups to have them accessible in private spaces as well e.g., homes and offices, as illustrated schematically in Figure 6 of the drawings. Aside from Covid- 19, there are many health-related issues that could be detected by observing the body temperature, including but not limited to:

• Heart Rate & Blood pressure: since the blood flow causes changes in the skin temperature, any irregulates in the blood pressure could be detected by monitoring the fluctuation of the body temperature and heat rate.

• Breathing Rate: monitoring the nasal area can detect the temperature change due to breathing in and out, hence, the ability to monitor the breath rate of the users.

• Irregular skin temperature: users might experience irregular skin temperature profile for multiple reasons e.g., inflammation, or tumours.

Furthermore, thermal sensors could be deployed in appliances and loT devices within care homes and configured to monitor users and detect any fainting incidents. Embodiments of the invention enable having such a technology accessible to the elderly people at their own homes. Where our system seamlessly monitors the user, and if a faint condition is detected, it automatically notifies a previously assigned emergency contact, as illustrated schematically in Figure 9 of the drawings.

An example system consists of a thermal sensor and image processing software that recognises and analyses the user’s face temperature. A face recognition application allows the automatic face and region of interest, the forehead and nose tip extraction from the thermal sensor feed. For example, the OpenCV Iibrary4 can be used for image processing and facial points extraction. To enhance the face recognition, a series of pre-processing steps may be performed for each retrieved frame:

Frame extraction: the image can be extracted from the feed of the thermal sensor. This happens at frequency of 120 fps, based on the frame rate of the sensor model used. Noise filtering: A 5x5 median filter may be applied to smooth the image. The output may then be converted to grayscale and a 2D Gaussian filter is applied to further remove high frequency noise as performed in.

Face Recognition: The viola-jones classifier, which will be known to a person skilled in the art, may be used for detecting faces. ROI Identification: In this step, the nose tip and forehead may be identified as the ROI. These ROI are computed relative to the face coordinates extracted as shown below, and illustrated in Figure 7 of the drawings. A 5x5 pixels window can be identified to represent the ROI. A simple ROI identification approach may be utilised to maintain fast operation of the algorithm. Embodiments of the invention aim to rely solely on the thermal sensor, without any wearable tracking headsets. xForehead = xFace + (4 * face. Width 17); yForehead = yFace + (face. Height 16); xNasal = xFace + (4 * face. Width 19); yNasal = yFace + (face. Height 12);

Temperature Analysis: the average temperature of the 5x5 window may be analysed to identify the user’s health and mental status, e.g. stress, high blood pressure, etc.

Thermal sensors can provide information about the observed body temperature; hence they can detect any health issues reflected in the body temperature. For instance, it can remotely measure the user’s temperature and notify them if any irregularities are found. Additionally, temperature changes can be correlated with different affective and emotional states.

3. Emotional Awareness and Wellbeing

While thermal sensors can detect physical health issues, they can also detect the emotional, physiological and cognitive state of the users in an unobtrusive manner. Our affective status strongly influences how our blood flows through our bodies. When we are scared, blood flows to our legs in reaction to the fight or flight response; when we are embarrassed, blood flows to our face, making us blush. Because blood carries heat, as it flows through our bodies, it changes the temperature distribution on our skin, underlying tissues, and vessels. Therefore, monitoring changes in this distribution can give us an insight about the changes in our status or arousal that caused them.

Thermal sensors can provide information about the observed body temperature, which can be used to infer the physiological and cognitive state of users in an unobtrusive manner by, for example, evaluating their stress levels. The reason why this is possible, is because our skin temperature is modulated by the Autonomic Nervous Systems (ANS) activity. ANS controls the organs of our bodies, such as heart, stomach, and intestine. It is responsible for activating the glands and organs to secrete hormones and other substances such as Adrenaline to defend the body from potential threats. Its activation might be accompanied by many body reactions, such as an increase in heart rate, rapid blood flow to the muscles, activation of sweat glands, and increase in the respiration rate. Embodiments of the invention may deploy thermal sensors in daily devices e.g., smartphones, smart TVs, and smart homes appliance, to continuously capture the users' states and personalize their experience accordingly.

Several points in the body can be used to measure this temperature fluctuation, such as the nose, the cheeks, the areas around the eyes (periorbital and supraorbital), the jaw, the neck, the hands (fingers and palm), the lips and the mouth. The face is thought to be particularly advantageous for this task for several reasons. First, it is often exposed, making it easy to observe with a thermal camera. Second, it features a thin layer of tissue, making temperature changes more pronounced. Therefore, in this work we explore how facial temperature fluctuations can give us an insight into changes in cognitive load. We focus on two of the points of interest suggested in the literature — the forehead and the nose — as these can be monitored even if the user is wearing glasses or the head is covered (by a cap for instance).

Temperature changes on the forehead have been shown to be linked to the changes in the brain temperature. There is a direct relationship between workload and facial temperature based on the involvement of the Autonomic Nervous System (ANS): increased brain activity causes a surge in blood supply. Hence, higher workloads lead to blood flowing from the adjacent facial areas to the brain causing the facial temperature to vary. Different facial areas have been shown to be effective temperature indicators, namely the tip of the nose, above the eyes, and at the centre of the forehead.

The use of thermal sensors for observing users’ mental states has been explored, and some embodiments of the invention may build upon previous work that assessed stress based on the variations in the forehead and nose temperature. Emotions like stress, fear, startling, empathy, anxiety, and guilt, could be detected by monitoring facial temperature chain summary, embodiments of the invention introduce novel and enhanced health and emotional awareness in-home system. Where thermal sensors integrated in the users’ TV, it can detect users’ physical and emotional health states. Additionally, utilizing smart homes loT, emergency measures could be deployed when health issues are detected according to the seventy of the situation, either notify the user or directly contact the emergency. This is crucial to enhance users’ well-being while maintaining their privacy, as all data is processed at the users’ side.

Examples of detectable emotions based on changes in facial temperature are summarized in the Table shown in Figure 8 of the drawings, as explained by lonnou et al, “Thermal infrared imaging in Psychophysiology: potentialities and limits; Psychophysiology 51 (10):951 -963, 2014.

4. User Recognition

Another interesting thermal feature is how every user has a distinctive temperature profile. For instance, considering our veins patterns, each user has a unique veins pattern just like their fingerprints, since veins’ pattern can’t be replicated and most likely can’t be damaged (unless a serious damage to the users’ skin and body), this makes it way more robust and secure. As thermal sensors can capture the veins pattern (as the blood vessels have different temperature than the skin), in a seamless and remote way, users could be implicitly identified without the need to explicitly enter a password or touch a device, they can be identified by their “thermal profile”.

Embodiments of the invention, as illustrated schematically in Figure 10 of the drawings, aim to personalise and improve users’ in-home experience. For instance: 1 . Personalized Content: one potential use case would be the users walk into their living room and the TV automatically switches on to their Netflix account or their favourite TV Program.

2. Parental guidance: since ThermoMedia can identify the users’ it can also adapt the content based on the existence of parents or not.

3. Guests or unidentified users’ detection: Another security measure, ThermoMedia can detect unregistered users, and accordingly turn on Guest Mode (e.g., adjust the displayed content or the background images or hide private content).

5. Viewer Counter

Embodiments of the invention may find utility in the media industry, providing a very wide sample of the viewers and their behaviour towards any content that is being displayed on the smart TV screen. Body temperature is detectable from the surrounding environment using thermal sensors embedded in the TV and/or other loT devices within the environment.

As illustrated schematically in Figure 11 of the drawings, the system can infer the number of viewers while viewing a certain content and identify their gender and age group. Moreover, our system would correlate the content displayed on the TV screen, whether the source of the content is a broadcasted TV channel (e.g., BBC, ITV, Channel 4, etc.), a TV app (e.g., Netflix, BBC iPlayer, BT Sport, etc..) or an external device connected to the TV whether it’s wired or not (e.g., watching YouTube on a mobile phone device and broadcasting it on TV).

The system (software and hardware) can connect the data of the content that is being displayed on the smart TV to the data collected by the thermal sensor for the surrounding environment including the viewers and their profiles and analyse their states towards such content (it can detect whether they are happy, sad, terrified, excited, or showing no interest). The system is intended to provide accurate data about the consumer/ viewer reactions and can identify their genders and age groups and relate this to the content that they frequently or rarely watch.

The system may be configured to be interactive with the viewers and could be set up to send them notifications to warn them about the unsuitability of certain content for certain age groups (e.g., Adult content is not suitable for children), and also can early alarm them for increased heart rate, or blood pressure when they watch certain content (e.g., Horror or sever violent content is not suitable viewers with heart diseases).

A system according to some embodiments of the invention could be used to change the way media production companies and tv channels select and invest in production of any visual or entertainment content, as they will be able to accurately measure the success or failure of similar content, rather than the current assumptions which are based on inaccurate market research findings, which are usually based on a selected small sample of people and multiplying it to get the research figures.

Currently the way of counting viewers is based on some technology support of satellite, cable and internet figures based on small samples from different groups of people with selection based on a pre-set profile for the viewers in a certain location and then multiplying it in a manner that we will explain later in this document, to give a false impression of a wide sample with accurate figures which is not the case. This means that the advertisers are never 100% sure how effective was their advert and whether they succeeded to penetrate their target market or not. The opportunity lies in examples of the invention, that would allow much more accurate numbers to be recorded, as the sample will be all the people who are using smart TVs with an exemplary embodiment of the invention.

The current reports from the market research companies are based on a very small sample of population in each country as they compile a sample of 3,500 to 5,000 people (these are the average UK figures) who are chosen based on certain criteria (such as gender, age group, socioeconomic class, income, education, etc..). Each person of this sample of 5,000 people have been chosen from a larger sample of average 1 ,500 people based on a survey and individual consents from the people to have the TV and a people meter device to detect their viewership (Qualitative Research approach). Then the market research company multiplies this sample of 5,000 people by 1 ,500 and issues their reports claiming that their sample was based on 7.5 million people which is utterly inaccurate and doesn't reflect the reality at all. It will be self-evident that the 5,000 people sample out of 67.8 million people which is the population of the United Kingdom is a false reflection of the consumer viewership nor the behaviour towards the content produced and viewed by this population. The advertisers and TV channels are currently relying on the market research companies such as Epsos and AC Nelsen to provide them with reports revealing the viewership of certain programs or any other entertainment or sports content on certain tv channels. The media buying companies and the TV channels are selling the airing time to the advertisers and even classifying the time slots into prime times and regular times based on these inaccurate reports which is misleading for both parties (the advertisers and TV Channels).

A system according to an embodiment of the invention could be configured to analyse the viewers’ responses in a real time manner, and the data is beneficially encrypted end to end to preserve the privacy of the viewers and we don’t store the actual images as our system is capable of converting this visual data to numeric data.

Thus, the system could guarantee the maximum reach and the highest viewership for the advertisers and content producers and also TV channels. They will not only save a lot of money on the inaccurate and false reports that they are purchasing from the market research companies but will also put their investments in the right place and to the right target audience and right times. TV Channels would change and adjust their grids and content based on the new data.

An embodiment of the invention could be a very efficient tool for governments to choose the most effective media channels, as well as timing to communicate with the public in the times of wars and pandemics and also assess the success of awareness campaigns that could be crucial in the times of wars and pandemics. It will be much easier for them in terms of the targeting based on the data that we can provide.

As illustrated in Figure 11 of the drawings, an embodiment of the invention is configured to, not only detect the number of viewers, but additionally, it can detect their states during viewing. This information enables better understanding of the users’ behaviour, hence, a more personalised and tailored user experience.

Body temperature is distributable from the environment using one or more thermal sensors embedded in, for example, a smart TV. This enables the precise counting of viewers. For the purposes of recognising people, Open CV library could be used, for example, for image processing and features extraction. Form each retrieved frame from a thermal sensor, pre- processing is required before extracting features. The pre-processing may include the following steps: noise filtering, background subtraction, thresholding and, subsequently, people detection.

People detection is a challenging task using ordinary RGB cameras. In contrast, since the body has a different temperature to its background, it can be robustly segmented in any lighting conditions using thermal sensor image data. An algorithm may be used to compute the convex hull and convexity defects of the body contour.

Thus, the present invention the first domestic deployment of thermal sensors in smart homes to allow smart, personalized in-home experience. Embodiments of the invention offer novel opportunities including but not limited to (1) personalised watching experience, (2) health- aware and well-being in-home system, (3) seamless users’ identification, (4) early fire alarm and overheating detector. The aforementioned opportunities can be realised while considering the latest privacy and users' data protection techniques e.g., GDPR compliant.

The invention has been given by way of example only, and various other modifications of and/or alterations to the described embodiment may be made by persons skilled in the art without departing from the scope of the invention as specified in the appended claims.

The above-mentioned use cases, directly impact diverse target groups, starting from the end users to the governments. Embodiments of the invention could be used to directly influence the following:

1 . The End user & the Society: Embodiments of the invention could benefit the end user, as it can act as an early fire alarm system as it detects changes in temperature of the place and the smart TV device could be connected to the mobile phone of the user to alarm them about a possible fire. This device can also be connected to the home phone and programmed to call the firefighting department service. Some embodiments can also act as a surveillance/ security camera as it could be connected to the mobile phone of the user and alarming them if any strangers get to the house or the workspace. Additionally, it will increase the users’ awareness to their health and emotional states by tracking their states. Lastly, the content and environment could adapt to the users’ states and behaviour, hence, tailor and enhance the user experience. Health Care Systems: Some embodiments of the invention could reduce the costs and burden to the health care system, they could act as the personal monitor to, for example, elderly people at their own homes. Additionally, the early detection of disease reflected in body temperature fluctuations would alarm the users to seek medical help and diagnosis of a potential diseases at early stages. TV Channels, Media Industry, Artists & Players currency: Accurately determining the number of viewers of any content that is being displayed on the smart TV screen, can capture the success or failure of broadcasted certain content including but not limited to Films, TV series, documentaries, or TV shows/ programs, etc. This would change the budget allocation for content production companies, which will be based on much more accurate statistics of viewers, that will vastly change the spending of the media organisations (e.g., BBC, ITV, etc..). Rescheduling the grids of the TV channels for showing certain content in a certain time (rearranging the prime and regular times). As for the advertisers and market research companies, it will change the currency of media map and will give these companies the opportunity to benefit from their advertising budgets to the max, as they will have more accurate figures for the viewers of each tv show, entertainment content, etc., with more specific data about the age groups and genders which are related to the geolocation. This will change how each tv show or any other type of content is priced for the advertisers to advertise their products/ services as it will be based on a much larger sample of population (millions not a few thousands like the current situation). That will also certainly help in more accurate and easier targeting for the customers and more personalised marketing materials for each customer in the near future (since most of tv channels have their content currently on demand). It might also help in assessment of positive or negative reactions towards certain advertising content which will help in assessment of the effectiveness and success of the advertising campaign. Some embodiments could also affect the currency of participation of the football players and sports people in advertising as many players have contracts with major advertisers that could exceed the 100 million dollars, however with ThermoMedia, advertisers can find out if a relatively new player like Kylian Mbappe is more effective than an older and more famous player like Cristiano Ronaldo or not because they can have more accurate tool to measure the numbers of fans and their reactions when they see the players on TV. In brief examples of the invention could completely change the game of advertising and all related industries, mainly media content production and the use of sports players in advertising.