The present invention relates to mobile electronic devices that comprise display state controls.

BACKGROUND OF THE INVENTION

Smartphones become more and more popular in the consumer market. To support various applications and games, the power consumption of the device becomes a very critical factor affecting user experience given very limited battery improvement in terms of capacity density over the years. For example, most smartphones in the market today need to be charged every day or every two days . To overcome these issues on power consumption, a great deal of efforts has been put into choosing more power efficient hardware components, e.g. certain inductors and designing more power efficient hardware architecture, e.g. power management unit (PMU) , to provide more efficient power supply to the various parts of the hardware system, and intelligent switching on or cutting off power supply to the devices. Also efforts in designing different levels of power saving modes at software level, e.g. different settings for sleep mode have been developed. Furthermore, many mobile devices have a lock mode. The lock mode may be used to prevent inadvertent operation of a touch screen display, e.g., while the device is in a user's pocket or purse or when another object is placed against the device. The lock mode may also be used to prevent an unauthorized person from using the device. Generally, the device is programmed to enter the lock mode when a user presses a specific button or a series of buttons or when it has been idle for a certain period of time. When a user desires to use a device that is locked, the user will typically be required to drag a slide bar, press a specific button or a series of buttons (e.g., to enter a password) to unlock the device. However, a user may find these steps inconvenient and time consuming. For example, a user may be reading a document using the device when the device detects that it has been idle for a certain period of time. In this case, the device will automatically enter the lock mode where it turns off or dims its display screen, and the user will be required to unlock the device before being able to resume reading the document. In another example, a user may be prone to forgetting the password needed to unlock the device. As a result, the user may decide to configure the device so that it does not automatically lock. If she then forgets or chooses not to manually lock her device, which leaves the device susceptible to inadvertent operation or unauthorized use. Furthermore, the delay time before switching off the screen is typically unnecessary since the user have typically already left the mobile device unattended e.g. if the inactivity is determined by movement sensors. Since screens are among the most power consuming components especially in modern smartphones with large screens of high resolution the on-time of the screens are becoming increasingly important in the power consumption of mobile devices. Thus, there is a need to minimize the power consumption from screens in mobile devices . DISCLOSURE OF THE INVENTION

On this background, it is an object of the present invention to provide a mobile electronic device with a lowered power consumption relating to the screen of the device. This object is achieved by providing a mobile electronic device comprising a processor configured to control the operation of said mobile electronic device, a front-facing display screen coupled to said processor and arranged on a front side of said mobile electronic device, said front-facing display screen having an on- state and an off-state, a movement sensor coupled to said processor, said movement sensor being configured to detect movement of the mobile electronic device, a front- facing camera arranged on the front side of the device, said front-facing camera being coupled to said processor and configured to capture images, said processor being configured to control the on-state and off-state of said display screen based on movement detection of said movement sensor and on one or more images captured by said front-facing camera, said processor configured to analyze the one or more images to determine whether a user's face is present in the one or more images. It has now been realized that many mobile electronic devices waste valuable battery capacity on power to displays without the presence of a user, and it has been realized that there is an advantage actively detecting the user presence by analyzing images from a front-facing camera in order to detect the presence of a user in order to control an on-state and an off-state of a front facing display by configuring a processor of the mobile device to control the on-state and off-state of said display screen based on movement detection and on the analysis of one or more images captured by said front-facing camera.

Preferably, the processor is configured to place the front-facing display screen in said on-state when it detects the presence of the face of a user in said one or more images and to place said front-facing display screen in an off-state when the presence of a user is not detected in said one or more images.

Thus, there is a clear distinction from known dimming modes already known from modern mobile devices. In the presence of a user the processor is configured to place the front-facing display screen in said on-state whereas when the presence of a user is not detected in the images the processor is configured to place the front-facing display screen in an off-state such that no power is consumed by the screen in the off-state when a user is not present.

Preferably, the processor is configured to analyze the one or more pictures using an edge detection image processing algorithm. The processor may be configured to analyze the one or more pictures using a high-pass filter image processing algorithm.

Preferably, the processor is configured to analyze the one or more pictures for the presence of a face.

In an embodiment of the invention, the processor is configured to analyze the one or more pictures using a facial recognition image processing algorithm. The processor may be configured to analyze the one or more pictures when an object is within a distance of less than 0.5 meters from the camera.

The control of the off-state of said front-facing display screen may be based on at least two consecutively captured images. The control of the on-state of said front-facing display screen may be based on at least two consecutively captured images.

Since image processing based on algorithms may not always be completely convergent, the redundancy of the control may be enhanced by basing the control on at least two consecutive images to avoid isolated misinterpretations of images e.g. on one falsely analyzed image. The control of the off-state of said front-facing display screen comprises an off-state delay time such that when said front-facing display screen is in an on-state and the presence of a user' s face is not detected anymore the off-state of said front-facing display screen is activated after the off-state delay time.

Since image processing based on algorithms may not always be completely convergent, the redundancy of the control may be enhanced by introducing a delay time in the control of the on-state and/or off-state such that e.g. when the presence of a user' s face is not detected anymore in the on-state, the off-state of said front- facing display screen is activated after a delay time and vice versa. To avoid flickering between the on-state and the off-state a delay time may ensure that the state of the display only changes when intended.

The camera may be a low resolution camera.

The camera may be a low power consumption camera.

The camera may be a low cost camera. In an embodiment of the invention the mobile device further comprises an additional high resolution camera configured to capture images of high resolution camera dedicated for other purposes than to determine whether a user' s face is present in the one or more images and wherein said front-facing camera is dedicated to determine whether a user' s face is present in the one or more images.

In an embodiment of the invention the processor is configured to comprise a toggle of the control of the on- state and off-state of said display camera such that the control of the on-state and off-state of said display may be disabled e.g. to enable an always on-state during playback of movies.

To accommodate special user situations the display control according to the invention may be toggled on or off. It is also an object of the invention to provide a method for automatically controlling an on-state and off-state of a front-facing display screen of a mobile electronic device having a front-facing camera, comprising the steps of: a) capturing one or more images using the front- facing camera,

b) analyzing the one or more images to determine whether a user' s face is present in the one or more images, and

c) controlling the on-state and off-state of said front-facing display screen based on step b such that the front-facing display screen of a mobile electronic device is in an on-state when the presence of a user is detected in said one or more images and an off-state when the presence of a user is not detected in said one or more images. It is another object of the present invention to provide a method for automatically controlling an on-state and off-state of a front-facing display screen of a mobile electronic device having a front-facing camera and a movement sensor, comprising the steps of:

- detecting movement of the device while the front- facing display screen of a mobile electronic device is in the off-state,

- in response to detecting movement of the device, capturing one or more images using the front- facing camera,

- determining that the device moved to a use position by analyzing the one or more images, wherein the use position indicates that a user is likely to want to use the device,

- in response to determining that the device is in the use position controlling the front-facing display screen of a mobile electronic device to be in the on-state. The method may further comprise the step of analyzing the one or more images comprising using an edge detection image processing algorithm. The method may further comprise the step of analyzing the one or more images comprising the use of a facial recognition image processing algorithm.

The method may further comprise the step of analyzing the one or more images comprising the use of a facial recognition image processing algorithm.

The method may further comprise a step of a user entering a radius of 0.5 meters from the camera.

The method may also comprise a step of controlling of the on-state and off-state of said front-facing display screen based on at least two consecutively captured images .

The method may also comprise a step of controlling the off-state of said front-facing display screen comprising an off-state delay time such that when said front-facing display screen is in an on-state and the presence of a user' s face is not detected anymore the off-state of said front-facing display screen is activated after the off- state delay time.

The method may also comprise a step of acquiring a low resolution image using a high resolution camera configured to capture images of low resolution for determining whether a user' s face is present in the one or more images and configured to capture images of high resolution for other purposes than determining whether a user's face is present in the one or more images.

The method may also comprise a step of toggling the control of the on-state and off-state of said display camera such that the control of the on- state and off- state of said display may be disabled e.g. to enable an always on-state during playback of movies. Further objects, features, advantages and properties of the mobile electronic device, method and software product according to the invention will become apparent from the detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

In the following detailed portion of the present description, the invention will be explained in more detail with reference to the exemplary embodiments shown in the drawings, in which

Fig. la is a front view of a mobile electronic device according to an embodiment of the invention,

Fig. lb is a front view of a mobile electronic device according to an embodiment of the invention,

Fig. 2 is a perspective view of a user holding a mobile electronic device according to an embodiment of the invention,

Fig. 3 is a block diagram of a mobile device,

Fig. 4 is a flow chart illustrating a method according to an embodiment of the present invention,

Fig. 5 is a flow chart illustrating a method according to an embodiment of the present invention, and

Fig. 6 is a flow chart illustrating a method according to an embodiment of the present invention. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In the following detailed description, the method and the mobile electronic device according to the invention in the form of a cellular/mobile phone will be described by the preferred embodiments.

Figs, la and lb illustrate a first embodiment of a mobile electronic device 1 according to the invention in the form of a mobile telephone 1. The mobile electronic device 1 comprises a front-facing display screen 2 arranged on a front side 4 of said mobile electronic device 1, and said front-facing display screen 2 having an on-state SI and an off-state S2. Fig la shows the front facing display in the on-state SI whereas Fig. lb shows the front facing display in the off-state S2.

Fig. 2 is a perspective view of a user 10 holding a mobile electronic device 1 according to an embodiment of the invention showing the front-facing display screen 2 arranged on a front side 4 of said mobile electronic device 1 facing the user 10 of the mobile electronic device 1. Also shown in Fig. 2 is a front-facing camera arranged on the front side of the device.

Fig. 3 illustrates in block diagram form a simplified general architecture of the mobile electronic device 1 constructed in accordance with the present invention. A processor module 5 controls the operation of said mobile electronic device 1. The processor module 5 is configured to control the on-state SI and off-state S2 of the front- facing display screen 2 based on movement detection of a movement sensor 6 and on one or more images captured by said front-facing camera 3. The processor is configured to analyze the images acquired by the front-facing camera 3 to determine whether a user's face is present in the images. The processor module 5 also forms the interface for some of the peripheral units of the device, including a (Flash) ROM memory 8 and a battery 9. In Fig. 3 the processor module has an integrated RAM 7.

The movement sensor 6 may be based on various physical devices such as piezoelectric, piezoresistive and capacitive components, which are commonly used to convert mechanical motion into an electrical signal, but there are no limitations as to how the movement sensor 6 of the device is operated. It is, however, advantageous to utilize a movement sensor having low power consumption.

According to the present invention the processor 5 is configured to control the operation of said mobile electronic device 1. The front-facing display screen 2 is coupled to the processor 5 and the front-facing display screen has an on-state SI and an off-state S2. Also a movement sensor 6 is coupled to the processor, the movement sensor being configured to detect movement of the mobile electronic device. On a front side 4 of the mobile device the front-facing camera 3 is arranged. The front-facing camera is coupled to the processor and configured to capture images. According to the invention the processor is configured to control the on-state SI and off-state S2 of said display screen 2 based on movement detection of said movement sensor 6 and on one or more images captured by said front-facing camera 3, said processor 5 configured to analyze the one or more images to determine whether a user' s face is present in the one or more images. A face presence detection using the mobile electronic device according to the invention may be carried out by a procedure of continuously measuring movements of the mobile electronic device 1 using the movement sensor. When movements of the device 1 are detected, even when only detecting small movements corresponding to a user holding the device still, the front-facing camera is activated by the processor to take a photo using the front-facing camera, and thereafter processing the image using a standard edge detection algorithm or a 2D high- pass filter algorithm to get the outline of the image and an outline of the objects in the image. If the algorithm detects the presence of a user in the image e.g. by detecting an egg-shaped curved close loop outline being size-wise bigger than a certain preset threshold, it may be concluded that a human face is presented right in front of the front-facing camera and if not it is concluded that. Otherwise, no face presented in front of the LCD/front-camera.

The computational complexity may be kept at an absolute minimum to minimize computational power thereby lowering power consumption of the device. Complex facial recognition algorithms may be used when the power consumption is maybe less critical. The typical facial recognition algorithm normally goes through several phases including edge detections, segmentations, feature extractions, matching etc. and therefore consumes more power than when using simple face presence detection algorithms such as edge detection.

A typical front-facing display 2 e.g. LCD or AMOLED, typically comprises six LED' s for backlight of the display, each LED consuming about 10mA. The total power consumption of a display screen 2 may therefore be about 70mA, which is an extremely high amount of consumed power when comparing with other components in such mobile electronic devices 1. Current power-saving solutions in mobile electronic devices is to set e.g. a screen timeout to turn off the display after 15 seconds (or 30 seconds, 1 minute, 2 minutes, 10 minutes) of inactivity. The approach of using a screen timeout period has an inherent problem in the trade-off between good user experience and a good power saving mechanism. For example, if the screen timeout is too quick, the device may save power during the off-state of the display screen, but may at the same time require the user to repeatedly touch the screen to avoid entering the off- state thereby annoying the user e.g. when reading. On the other hand, when turning off the display screen too slow, the battery drains quicker.

Based on a mechanism of combining handset motion detection and human head image presence detection, the handset would know if the user is not in front of the display and can immediately turn off the display thereby saving power while keeping an ultimate user experience intact .

The invention also relates to a method for automatically controlling an on-state and off-state of a front-facing display screen of a mobile electronic device having a front-facing camera Fig. 4 is a flow chart illustrating a method according to an embodiment of the present inven ^¬ tion. The method comprises the steps of a) capturing one or more images using the front-facing camera, b) analyz- ing the one or more images to determine whether a user's face is present in the one or more images, and c) con ^¬ trolling the on-state and off-state of said front-facing display screen based on step b such that the front-facing display screen of a mobile electronic device is in an on state when the presence of a user is detected in said one or more images and an off-state when the presence of a user is not detected in said one or more images. As shown in Fig. 5 the invention also relates to another method for automatically controlling an on-state and off- state of a front-facing display screen of a mobile electronic device having a front-facing camera and a movement sensor, comprising the steps of: detecting movement of the device while the front-facing display screen of a mobile electronic device is in the on-state, and in response to detecting movement of the device, capturing one or more images using the front-facing camera. Then determining that the device moved to an idle position by analyzing the one or more images, wherein the idle position indicates that a user is likely to want to stop using the device, and in response to determining that the device is in the idle position controlling the front-facing display screen of a mobile electronic device to be in the off-state.

When the mobile device 1 is turned on, a software executable may be run by the processor 5 detecting if there is a movement of the handset, ignoring small vibration or a steady state. The detection may be done by using a movement sensor 6 such as an accelerometer . The motion detection may be done by using other methods or sensors than accelerometers . When the motion detection confirms the presence of the movement of handset, a software executable starts to detect if there is a human head image present in front of the display. The human head image may be taken using the front camera of the handset, which typically takes short time very time and lower power consumption, e.g. 500ms at 10mA. The image processing function may include a simple head image edge- detection. If the "egg-shaped outline" is bigger than a certain size threshold, it may be concluded that a human head is present in front of the display, even if the head is not looking at the display. By doing this using a dedicated front-facing camera with low power consumption and a simple image processing function in the camera, in an external chip or implemented in SW code, the power consumption of the method may be kept at a minimum. If no head presence is detected, the display will be turned off immediately and if the presence of a head of a user is detected the display screen is kept in the on-state. The invention provides an optimal trade-off between user experience and power saving. It exploits the situations where the user has moved away from the display or put the mobile device away by turning off the display immediately instead of waiting for a screen timeout in 15 seconds or even up to 10 minutes. A typical user pattern is to look quickly at the phone and put it away. This type of action normally happens above 60 times per day, and thus the power potentially saved by the present invention could be estimated to ranging from 18mAh to 70mAh, or even more.

As shown in Fig. 6 the invention also relates to another method for automatically controlling an on-state and off- state of a front-facing display screen of a mobile electronic device having a front-facing camera and a movement sensor, comprising the steps of: detecting movement of the device while the front-facing display screen of a mobile electronic device is in the off-state, and in response to detecting movement of the device, capturing one or more images using the front-facing camera. Then determining that the device moved to a use position by analyzing the one or more images, wherein the use position indicates that a user is likely to want to use the device, and in response to determining that the device is in the use position controlling the front- facing display screen of a mobile electronic device to be in the on-state.

Controlling how the screen is turned off in order to save power may be used for intelligent turning on of the display screen using the same types of algorithms e.g. to improve user experience. If the user by merely picking up the phone and looking on the screen automatically activates the screen, the user may avoid activation of the screen by pushing the screen or any button such as conventional mobile devices require.

For example, although the invention has been described in terms of a mobile phone, it should be appreciated that the invention may also be applied to other types of electronic devices, such as cameras, video recorders, music players, palmtop computers and the like.

The term "comprising" as used in the claims does not exclude other elements or steps. The term "a" or "an" as used in the claims does not exclude a plurality. The single processor or other unit may fulfill the functions of several means recited in the claims.

Although the present invention has been described in detail for purpose of illustration, it is understood that such detail is solely for that purpose, and variations can be made therein by those skilled in the art without departing from the scope of the invention.