Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
ELECTRONIC DEVICE WITH A DISPLAY AND METHOD OF OPERATING SUCH A DEVICE
Document Type and Number:
WIPO Patent Application WO/2018/084734
Kind Code:
A1
Abstract:
The invention relates to an electronic device (100) comprising: a display (101) configured to display electronic image data, wherein the display (101) comprises a plurality of pixels and each pixel is associated with at least one code value, in particular a brightness value, within a range of code values; and a processor (103) configured to determine a tone mapping operator by minimizing a maximum of an absolute value of a lack of contrast function, wherein the lack of contrast function is based on a distribution of the code values of the plurality of pixels over the range of code values and at least one parameter relating to external viewing conditions and/or the display (101), and wherein the processor (103) is further configured to adjust the code values of the plurality of pixels on the basis of the tone mapping operator.

Inventors:
LOGUNOV ALEXEY NIKOLAEVICH (CN)
KHREKIN KONSTANTIN EVGENYEVICH (CN)
ZHONG SHUNCAI (CN)
Application Number:
PCT/RU2016/000744
Publication Date:
May 11, 2018
Filing Date:
November 02, 2016
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
HUAWEI TECH CO LTD (CN)
LOGUNOV ALEXEY NIKOLAEVICH (CN)
International Classes:
G09G3/22; G09G5/02
Foreign References:
US20060209005A12006-09-21
US20140078166A12014-03-20
Other References:
None
Attorney, Agent or Firm:
MITS, Alexander Vladimirovich et al. (RU)
Download PDF:
Claims:
CLAIMS

1. An electronic device (100) comprising:

a display (101) configured to display electronic image data, wherein the display (101) comprises a plurality of pixels and each pixel is associated with at least one code value within a range of code values; and

a processor (103) configured to determine a tone mapping operator by minimizing a maximum of an absolute value of a lack of contrast function, wherein the lack of contrast function is based on a distribution of the code values of the plurality of pixels over the range of code values and at least one parameter relating to external viewing conditions and/or the display (101), and wherein the processor (103) is further configured to adjust the code values of the plurality of pixels on the basis of the tone mapping operator.

2. The electronic device (100) of claim 1, wherein the processor (103) is configured to determine the lack of contrast function, LCF, on the basis of a product of a contrast sensitivity loss function, CSLF, and the distribution of the code values of the plurality of pixels over the range of code values.

3. The electronic device (100) of claim 2, wherein the processor (103) is configured to determine the distribution of the code values of the plurality of pixels over the range of code values on the basis of the following equation:

wherein w and h denote the horizontal and vertical coordinates of a pixel, w* and h* denote the number of horizontal pixels and the number of vertical pixels of the display (101) and Yh w denotes the luma value of the pixel at coordinates

4. The electronic device (100) of claim 2, wherein the processor (103) is configured to determine the distribution of the code values of the plurality of pixels within a subrange of the range of code values over the range of code values on the basis of the following equation:

wherein bin denotes a subrange of the range of code values, w and h denote the horizontal and vertical coordinates of a pixel, w* and h* denote the number of horizontal pixels and the number of vertical pixels of the display (101), YKw denotes the luma value of the pixel at coordinates w and h, binjsize denotes the size of the subrange of the range of code values in pixels and the operator I... J denotes rounding down to the next integer.

5. The electronic device (100) of any one of the preceding claims, wherein the processor (103) is configured to determine the contrast sensitivity loss function the basis of the following equation:

wherein STM0 denotes the slope of the tone mapping operator, CSr denotes a contrast sensitivity function for real viewing conditions, CSi denotes a contrast sensitivity function for ideal viewing conditions, L£ denotes the luminosity of a pixel having a code value cv without reflection, Lamb denotes an ambient luminosity and lr denotes the luminosity of the pixel having a code value cv including reflection.

6. The electronic device (100) of claim 5, wherein the processor (103) is configured to estimate the luminosity of the pixel having a code value cv including reflection, i.e. Lr, on the basis of the following relation between the luminosity of the pixel having a code value cv including reflection, i.e. Lr, and the luminosity of the pixel having a code value cv without reflection, i.e. Lf.

wherein k enotes a reflection coefficient.

7. The electronic device (100) of claim 6, wherein the display (101) is associated with an electro-optical transfer function, EOTF, and the processor (103) is configured to determine the luminosity of the pixel having a code value cv without reflection, i.e. I,-, on the basis of the EOTF of the display (101).

8. The electronic device (100) of any one of claims 5 to 7, wherein the electronic device (100) further comprises a photometer (105) configured to measure the ambient luminosity

9. The electronic device (100) of any one of claims 5 to 8, wherein the processor (103) is configured to determine the contrast sensitivity function for real viewing conditions, i.e. CSr, and the contrast sensitivity function for ideal viewing conditions, i.e. on the basis of the following equation:

wherein ALt denotes an absolute incremental threshold being a function of the ambient luminosity Lamb and the luminosity L, which stands for the luminosity L; or the luminosity Lr.

10. The electronic device (100) of claim 9, wherein the processor (103) is configured to determine the absolute incremental threshold ALt as a function of the luminosities Lamb and L on the basis of the following equation:

wherein the processor (103) is further configured to determine A(Lamb) and

A(L) on the basis of the following equations:

wherein the processor (103) is further configured to determine λ on the basis of the following equation:

wherei 2 denote predefined constants.

11. A method (700) of operating an electronic device (100) comprising a display (101) configured to display electronic image data, wherein the display (101) comprises a plurality of pixels and each pixel is associated with at least one code value within a range of code values, wherein the method (700) comprises the steps of:

determining (701) a tone mapping operator by minimizing a maximum of an absolute value of a lack of contrast function, wherein the lack of contrast function is based on a distribution of the code values of the plurality of pixels over the range of code values and at least one parameter relating to external viewing conditions and/or the display (101); and

adjusting (703) the code values of the plurality of pixels on the basis of the tone mapping operator.

12. The method (700) of claim 11, wherein the method (700) comprises the further step of determining the lack of contrast function, LCF, on the basis of a product of a contrast sensitivity loss function, CSLF, and the distribution of the code values of the plurality of pixels over the range of code values.

13. The method (700) of claim 11 or 12, wherein the method comprises the further step of determining the contrast sensitivity loss function on the basis of the following equation:

wherein STM0 denotes the slope of the tone mapping operator, CSr denotes a contrast sensitivity function for real viewing conditions, C5j denotes a contrast sensitivity function for ideal viewing conditions, I,· denotes the luminosity of a pixel having a code value cv without reflection, Larnb denotes an ambient luminosity and Lr denotes the luminosity of the pixel having a code value cv including reflection.

14. The method (700) of claim 13, wherein the method (700) comprises the further step of determining the contrast sensitivity function for real viewing conditions, i. and the contrast sensitivity function for ideal viewing conditions, i.e on

the basis of the following equation:

wherein t denotes an absolute incremental threshold being a function of the

ambient luminosity b and the luminosity L, which stands for the luminosity ; or the luminosity .

15. The method (700) of claim 14, wherein the method (700) comprises the further step of determining the absolute incremental threshold t as a function of the luminosities and L on the basis of the following equation:

wherein the method (700) comprises the further step of determining amb and on the basis of the following equations:

wherein the method (700) comprises the further step of determining λ on the basis of the following equation:

wherein a enote predefined constants.

16. A computer program comprising program code for performing the method (700) of any one of claims 1 1 to 15 when executed on a computer.

Description:
ELECTRONIC DEVICE WITH A DISPLAY AND METHOD OF OPERATING

SUCH A DEVICE

TECHNICAL FIELD

Generally, the present invention relates to the field of electronic devices having a display for displaying electronic data. More specifically, the present invention relates to an electronic device having a display with a real-time automatic brightness adjustment.

BACKGROUND

The image displayed on a display of an electronic handheld device may be subjectively impaired if the ambient (surround) light significantly exceeds the average luminance of the display. A challenge in the context electronic devices with displays is to keep the perceived quality of a displayed image constant under changing ambient light conditions, while simultaneously minimizing the power consumed by the display.

There are several known approaches to address this issue. A very simple approach assumes the proportional increase of the peak luminance emitted by the display, e.g. a backlight level of a LCD/TFT display. This approach has the disadvantage that the power consumption of the display dramatically depends on the luminance of the emitted light. Additionally, there is a physical restriction on the maximum display brightness defined by the display manufacturer and the display technology used.

Another approach is to combine an increase of the brightness of a display and a non-linear processing (generally referred to as tone mapping) of the image data that is being displayed. This approach is more sophisticated as the image processing may improve the perceived quality of the displayed image and may decrease the additional power consumption. This approach, however, has the same disadvantage as the first approach, as it still supposes a peak luminance control.

The currently most sophisticated approach is to process the image data only without actively controlling the peak luminance of the display. This approach does not consume extra power but has significant disadvantages with respect to the estimation of an optimal tone mapping operator. Existing solutions use different human vision models and multiband (spatial) processing of the image data. Such a spatial processing, however, increases the complexity of the solution as well as the input-to-output signal delay, which is important for real-time processing. Another disadvantage of existing approaches is that they compensate the perceived contrast and brightness of the image concerning the dark room luminance adaptation as a reference that does not allow finding the optimal tone mapping operator for very low ambient light levels. One more disadvantage of these approaches is that they define the optimal tone mapping operator minimizing the mean square difference between the responses of a human vision system in a reference viewing condition and given viewing condition, which does not comply with a real human vision system.

Thus, there is a need for an improved electronic device having a display allowing for a real-time automatic brightness adjustment as well as a corresponding method.

SUMMARY

It is an object of the invention to provide an improved electronic device having a display allowing for a real-time automatic brightness adjustment as well as a corresponding method.

The foregoing and other objects are achieved by the subject matter of the independent claims. Further implementation forms are apparent from the dependent claims, the description and the figures.

According to a first aspect the invention relates to an electronic device comprising: a display configured to display electronic image data (or short "images"), wherein the display comprises a plurality of pixels and each pixel is associated with at least one code value, in particular a brightness value, such as a luma value, within a range of code values; and a processor configured to determine a tone mapping operator, i.e. tone mapping operator values, by minimizing a maximum absolute value of a lack of contrast function, wherein the lack of contrast function is based on a distribution of the code values of the plurality of pixels over the range of code values and at least one parameter relating to external viewing conditions of the electronic device and/or the display, and wherein the processor is further configured to adjust the code values of the plurality of pixels on the basis of the tone mapping operator.

Thus, an improved electronic device is provided having a display allowing for a real-time automatic brightness adjustment. The electronic device can be implemented, for instance, in form of a smartphone, a smartwatch, a tablet computer or a laptop computer.

In a first possible implementation form of the electronic device according to the first aspect as such, the processor is configured to determine the lack of contrast function, LCF, on the basis of a product of a contrast sensitivity loss function, CSLF, and the distribution of the code values of the plurality of pixels over the range of code values. In an implementation form, the processor is configured to determine the lack of contrast function, LCF, on the basis of the following equation:

wherein cv denotes a code value within the range of code values and H denotes the distribution of the code values of the plurality of pixels over the range of code values. In another implementation form, the processor is configured to determine the lack of contrast function, LCF, on the basis of the following equation:

wherein bin denotes a subrange of the range of code values and H denotes the distribution of the code values of the plurality of pixels within the subrange of the range of code values over the range of code values.

In a second possible implementation form of the electronic device according to the first implementation form of the first aspect, the processor is configured to determine the distribution of the code values of the plurality of pixels over the range of code values on the basis of the following equation:

wherein w and h denote the horizontal and vertical coordinates of a pixel, w * and h * denote the number of horizontal pixels and the number of vertical pixels of the display and Y h w denotes the luma value of the pixel at coordinates w and h.

In a third possible implementation form of the electronic device according to the first implementation form of the first aspect, the processor is configured to determine the distribution of the code values of the plurality of pixels within a subrange of the range of code values over the range of code values on the basis of the following equation:

wherein bin denotes a subrange of the range of code values, w and h denote the horizontal and vertical coordinates of a pixel, w' and h' denote the number of horizontal pixels and the number of vertical pixels of the display, Y h w denotes the luma value of the pixel at coordinates w and h, bin_size denotes the size of the subrange of the range of code values in pixels and the operator [.,. J denotes rounding down to the next integer, i.e. the floor function.

In a fourth possible implementation form of the electronic device according to the first aspect as such or any one of the first to third implementation form thereof, wherein the processor is configured to determine the contrast sensitivity loss function on the basis of the following equation:

wherein S TM0 denotes the slope of the tone mapping operator, CS r denotes a contrast sensitivity function for real viewing conditions, i.e. with reflection,, CSi denotes a contrast sensitivity function for ideal viewing conditions, i.e. without reflection, Li denotes the luminosity of a pixel having a code value cv without reflection, L amb denotes an ambient luminosity and L r denotes the luminosity of the pixel having a code value cv including reflection. In an implementation form, the processor can be configured to determine the slope of the tone mapping operator values on the basis of the following equation:

In a fifth possible implementation form of the electronic device according to the fourth implementation form of the first aspect, the processor is configured to estimate the luminosity of the pixel having a code value cv including reflection, i.e. L r , on the basis of the following relation between the luminosity of the pixel having a code value cv including reflection, i.e. L r , and the luminosity of the pixel having a code value cv without reflection, i.e. If

wherein denotes a predefined reflection coefficient.

In a sixth possible implementation form of the electronic device according to the fifth implementation form of the first aspect, the display is associated with an electro-optical transfer function, EOTF, and the processor is configured to determine the luminosity of the pixel having a code value cv without reflection, i.e. Lj, on the basis of the EOTF of the display. Thus, in an implementation form the processor is configured to determine the luminosity of the pixel having a code value cv without reflection, i.e. L„ on the basis of the following equation:

wherein denotes the code value adjusted by the tone mapping

operator.

In a seventh possible implementation form of the electronic device according to any one of the fourth to sixth implementation form of the first aspect, the electronic device further comprises a photometer configured to measure the ambient luminosity

In an eighth possible implementation form of the electronic device according to any one of the fourth to seventh implementation form of the first aspect, the processor is configured to determine the contrast sensitivity function for real viewing conditions, i.e. with reflection, i.e. C and the contrast sensitivity function for ideal viewing

conditions, i.e. without reflection, i.e. C on the basis of the following equation: 44

wherein t denotes an absolute incremental threshold being a function of the

ambient luminosity L amb and the luminosity L, which can denote the luminosity L; of a pixel under ideal viewing conditions or the luminosity L r of a pixel under real viewing conditions, i.e. including reflection.

In a ninth possible implementation form of the electronic device according to the eighth implementation form of the first aspect, the processor is configured to determine the absolute incremental threshold &L t as a function of the luminosities and L on the basis of the following equation:

L f(L L) ( ) ( )

wherein the processor is further configured to determine on the basis of the following equations:

wherein the processor is further configured to determine A on the basis of the following equation:

wherein a denote predefined constants.

According to a second aspect the invention relates to a corresponding method of operating an electronic device comprising a display configured to display electronic image data, wherein the display comprises a plurality of pixels and each pixel is associated with at least one code value, in particular a brightness value within a range of code values, wherein the method comprises the steps of: determining a tone mapping operator, i.e. tone mapping operator values, by minimizing a maximum absolute value of a lack of contrast function, wherein the lack of contrast function is based on a distribution of the code values of the plurality of pixels over the range of code values and at least one parameter relating to external viewing conditions of the electronic device and/or the display; and adjusting the code values of the plurality of pixels on the basis of the tone mapping operator values.

In a first possible implementation form of the method according to the second aspect as such, the method comprises the further step of determining the lack of contrast function, LCF, on the basis of a product of a contrast sensitivity loss function, CSLF, and the distribution of the code values of the plurality of pixels over the range of code values. In an implementation form the lack of contrast function, LCF, is based on the following equation:

wherein cv denotes a code value within the range of code values and H denotes the distribution of the code values of the plurality of pixels over the range of code values. In another implementation form the lack of contrast function, LCF, is based on the following equation:

wherein bin denotes a subrange of the range of code values and H denotes the distribution of the code values of the plurality of pixels within the subrange of the range of code values over the range of code values.

In a second possible implementation form of the method according to the second aspect as such or the first implementation form thereof, the method comprises the further step of determining the contrast sensitivity loss function on the basis of the following equation:

wherein S TM0 denotes the slope of the tone mapping operator, CS r denotes a contrast sensitivity function for real viewing conditions (i.e. with reflection), CSi denotes a contrast sensitivity function for ideal viewing conditions (i.e. without reflection), L t denotes the luminosity of a pixel having a code value cv without reflection, L ajnb denotes an ambient luminosity and L r denotes the luminosity of the pixel having a code value cv including reflection. In a third possible implementation form of the method according to the second implementation form of the second aspect, the method comprises the further step of determining the contrast sensitivity function for real viewing conditions (i.e. with reflection), i.e. CS r , and the contrast sensitivity function for ideal viewing conditions (i.e. without reflection), i.e. on the basis of the following equation:

wherein AL t denotes an absolute incremental threshold being a function of the ambient luminosity L amb and the luminosity L, which can denote the luminosity of a pixel under ideal viewing conditions or the luminosity L r of a pixel under real viewing conditions, i.e. including reflection.

In a fourth possible implementation form of the method according to the third implementation form of the second aspect, the method comprises the further step of determining the absolute incremental threshold LL t as a function of the luminosities and L on the basis of the following equation:

wherein the method comprises the further step of determining and

A(L) on the basis of the following equations;

wherein the method comprises the further step of determining λ on the the following equation:

wherein denote predefined constants.

According to a third aspect the invention relates to a computer program comprising program code for performing the method according to the second aspect when executed on a computer.

The invention can be implemented in hardware and/or software.

BRIEF DESCRIPTION OF THE DRAWINGS

Further embodiments of the invention will be described with respect to the following figures, wherein:

Fig. 1 shows a schematic diagram illustrating an electronic device with a display for displaying electronic data according to an embodiment;

Fig. 2 shows a flow diagram illustrating a process implemented in an electronic device according to an embodiment;

Figs. 3a and 3b show graphs illustrating tone mapping operator values as a function of pixel code value and bin number as implemented in electronic devices according to embodiments of the invention;

Figs. 4a and 4b show graphs illustrating tone mapping operator values and the slope of the tone mapping operator values as implemented in electronic devices according to embodiments of the invention;

Fig. 5a shows a flow diagram illustrating an initialization stage of an iterative procedure for determining an optimal tone mapping operator as implemented in an electronic device according to an embodiment;

Figs. 5b and 5c show flow diagrams illustrating an iteration stage of an iterative procedure for determining an optimal tone mapping operator as implemented in an electronic device according to an embodiment;

Figs. 6a to 6g shoes respective graphs illustrating different quantities at different iteration stages of the iterative procedure shown in figures 5a-c according to an embodiment; and

Fig. 7 shows a schematic diagram illustrating a method for operating an electronic device having a display according to an embodiment.

In the various figures, identical reference signs will be used for identical or at least functionally equivalent features.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following description, reference is made to the accompanying drawings, which form part of the disclosure, and in which are shown, by way of illustration, specific aspects in which the present invention may be placed. It is understood that other aspects may be utilized and structural or logical changes may be made without departing from the scope of the present invention. The following detailed description, therefore, is not to be taken in a limiting sense, as the scope of the present invention is defined be the appended claims.

For instance, it is understood that a disclosure in connection with a described method may also hold true for a corresponding device or system configured to perform the method and vice versa. For example, if a specific method step is described, a corresponding device may include a unit to perform the described method step, even if such unit is not explicitly described or illustrated in the figures. Further, it is understood that the features of the various exemplary aspects described herein may be combined with each other, unless specifically noted otherwise.

Figure 1 shows a schematic diagram illustrating an electronic device 100 according to an embodiment. The electronic device 100 can be implemented, for instance, in form of a smartphone, a smartwatch, a tablet computer or a laptop computer.

The electronic device 100 comprises a display 101 configured to display electronic image data (or short "images"). In an embodiment the display 101 can be, for instance, a LCD display 101 or an AMOLED display 101.

The display 101 comprises a plurality of horizontal and vertical pixels and each pixel is associated with at least one code value, in particular a brightness value, such as a luma value Y, within a range of code values (also referred to as working range). In an embodiment, the working range can be from 0 to 255 (i.e. for a 8 bit representation of code values).

The electronic device 101 further comprises a processor 103 configured to determine a tone mapping operator, i.e. tone mapping operator values, by minimizing a maximum of an absolute value of a lack of contrast function, wherein the lack of contrast function is based on a distribution of the code values of the plurality of pixels over the range of code values and at least one parameter relating to external viewing conditions of the electronic device 101 and/or the display, and wherein the processor is further configured to adjust the code values of the plurality of pixels on the basis of the tone mapping operator.

In the embodiment shown in figure 1, the electronic device 100 further comprises a photometer 105 configured to determine external viewing conditions by measuring an ambient luminosity.

Further embodiments of the electronic device 100 will be described in the following.

As used herein, an image to be displayed by the display 101 is the array of code values (cv) that is a digital representation of the luminance and color of the pixels of an object of a real scene. Each code value can contain 3 components: a Y-component that represents the luminance of the pixel and two color components Cr and Cb (or U and V) that represent the color of the pixel. Embodiments of the invention are based on the Y-component.

As already mentioned above, the code values cv of an image can have values within a working range of code values. For example, a 8 bit representation assumes the working range WR= [0...255] and the allowed code values are all the integer values of this working range. For a 10 bit representation the working range is WR= [0...1023] and the allowed code values are all the integer values of this working range. Other variants of the representation are possible (floating point representation of the code values, limited working range).

In an embodiment, the processor 103 of the electronic device 100 is configured to determine the lack of contrast function, LCF, on the basis of a product of a contrast sensitivity loss function, CSLF, and the distribution of the code values of the plurality of pixels over the range of code values. Thus, in an embodiment, the processor 103 is configured to determine the lack of contrast function, LCF, on the basis of the following equation:

wherein cv denotes a code value within the range of code values and H denotes the distribution of the code values of the plurality of pixels over the range of code values.

In an embodiment, the processor 103 is configured to determine the distribution of the code values of the plurality of pixels over the range of code values in the form of a histogram on the basis of the following equation:

wherein w and h denote the horizontal and vertical coordinates of a pixel, w * and denote the number of horizontal pixels and the number of vertical pixels of the display and Y h w denotes the luma value of the pixel at coordinates w and h. For example, for a 8 bit representation the histogram is a one-dimensional array with 256 components.

Embodiments of the invention are based on a reduced version of the above image histogram H using bins. For example, the working range WR=[0...255] may be split into 32 subranges or bins, such as (with 8 code

values in each bin). Thus, in such an embodiment, the processor 103 is configured to determine the lack of contrast function, LCF, on the basis of the following equation:

wherein bin denotes a subrange of the range of code values and H denotes the distribution of the code values of the plurality of pixels within the subrange of the range of code values over the range of code values.

In an embodiment, the processor 103 of the electronic device 100 is configured to determine the distribution of the code values of the plurality of pixels within a subrange of the range of code values over the range of code values in the form of a histogram on the basis of the following equation:

wherein bin denotes a subrange of the range of code values, w and h denote the horizontal and vertical coordinates of a pixel, w * and h * denote the number of horizontal pixels and the number of vertical pixels of the display, Y h w denotes the luma value of the pixel at coordinates w and h, bin_size denotes the size of the subrange of the range of code values in pixels and the operator l„. J denotes rounding down to the next integer, i.e. the floor function.

Embodiments of the invention make use of the electro optical transfer function (EOTF) of the display 101 of the electronic device 100. The EOTF is the function that converts the code values of a pixel to the luminance value, i.e.

where / may be a standardized function or defined by the manufacturer of the display 101. In embodiments of the invention based on bins of code values, the processor 103 can be configured to use a reduced version of the EOTF, i.e.:

The reduced version of the EOTF returns the average luminance value for the code values from the bin.

Often the surfaces of displays surface can reflect the ambient light L am b- Thus, in an embodiment, the processor 103 is configured to take into account a reflection coefficient k r fl associated with the display 101. In an embodiment, the processor 103 is configured to estimate the luminance of a pixel as the sum of the own light of the display 101 and the reflected light, i.e.:

In an alternative embodiment, the processor 103 is configured to estimate the luminance of a pixel as the sum of the own light of the display 101 and the reflected light on the basis of the following equation:

wherein denotes the code value adjusted by the tone mapping

operator.

Embodiments of the invention are based on the assumption that the surround luminance level dramatically affects the perception of the image. As mentioned above, a surround luminance L am b may cause visible reflections on surface of the display 101 and impair the perceived brightness and contrast of the image.

Additionally, the very low and high surround luminance definitely causes a decrease of the contrast sensitivity of the human eye even if there are no visible reflections. This effect is known as a luminance adaptation.

The key parameter of the external viewing conditions is the average luminance of the ambient light L am b. This parameter is neither an image nor display feature and can be estimated independently, e.g. using the photometer or lux-meter 105 of the electronic device.

As used herein, "ideal viewing conditions" are conditions when the human eye has the maximum of contrast sensitivity. The maximum of the contrast sensitivity of the observer can be achieved if the average luminance of the ambient light is equal to the luminance of the object that the observer tries to recognize on the display 101. So the "ideal viewing condition" for the observer means that the observer can manually change the ambient light in order to recognize all the objects in image: to decrease the ambient light recognizing objects in the dark area and to increase ambient light recognizing the objects in the bright area of the image. Additionally, "ideal viewing conditions" imply that there is no reflection from the surface of the display 101, i.e. k r ji

= 0.

It is virtually impossible to achieve ideal viewing conditions in practice, because a user of a smartphone or a tablet cannot change the ambient light. Thus, it is much more realistic to assume "real viewing conditions", where the ambient light arrives from an uncontrollable source. This implies that there is a contrast sensitivity loss in the areas of the image where the luminance of the objects of the image on the display 101 differs from the average ambient light luminance level Lamb.

Embodiments of the invention take into account the changes of the human eye contrast sensitivity when the viewing conditions are varying. In an embodiment, the processor 103 is configured to determine the contrast sensitivity function for real viewing conditions, i.e. with reflection, herein denoted as CS r , i.e. the contrast sensitivity function CS applied to the luminosity L r including reflection, and the contrast sensitivity function for ideal viewing conditions, i.e. without reflection, herein denoted as C5 ; , i.e. the contrast sensitivity function CS applied to the luminosity Li without reflection, on the basis of the following equation: wherein AL t denotes an absolute incremental threshold being a function of the luminosities L amb and L and wherein I stands for the luminosity L t of a pixel under ideal viewing conditions or the luminosity L r of a pixel under real viewing conditions, i.e. including reflection.

The contrast sensitivity ) is not a scalar (average) characteristic of

the human eye for the given viewing conditions, it is a function and it defines the set of contrast sensitivity values for each combination of a background luminance L (i.e. the luminance produced by the display 101) and the ambient luminance L am b. The absolute incremental threshold AL t is the minimal required difference between the luminance of the background L and the luminance of the object L+AL that the observer tries to recognize. If AL < AL t the object is invisible, if Δί, > AL t the object is visible.

In an embodiment, the processor 103 of the electronic device 100 is configured to determine the absolute incremental threshold AL t as a function of the luminosities

Lamb and L on tne basis of the following equation:

wherein the processor 103 is further configured to determine and

A(L) on the basis of the following equations: and

wherein the processor 103 is further configured to determine λ on the basis of the following equation:

wherein denote predefined constants. A

corresponding flow diagram is shown in figure 2 (where L amb has been abbreviated as La). In a processing block 201, the processor 103 determines A on the basis of L a . In the processing blocks 203, 205 and 207, the processor 103 determines A on the basis of A and L. In the processing block 209, the processor 103 determines &L t on the basis of A and A. In an exemplary embodiment, the predefined constants can have the following values:

The tone mapping operator (TMO) as used by the processor 103 of the electronic device 100 processes the code values of the pixels of an image to be displayed on the display 101. Thus, the TMO can be considered to define how to calculate an output code value for each input code value within the range of code values, i.e. the working range. In an embodiment, the TMO can be represented and used by the processor 103 in the form of a table. For example, in an embodiment, a 8bit representation of a TMO can look as follows:

In embodiments of the invention based on bins of code values, the processor 103 can be configured to use a reduced or binned version of the TMO. The reduced version of the TMO operates with the subranges (bins) of code values. It is a table where the output code values are defined only for the biggest code value of each bin, not for all. For calculating the remaining code values of a bin bilinear interpolation can be used. For example, the reduced TMO 8bit representation may look like

bin Output CV

1 12

For example, for the bin=l (CV = [0...7]) the output CVs can be reconstructed in the following way:

For the bin=2 (CV = [8...15]) the output CV can be reconstructed following way:

The graph of full TMO may look like a smooth curve as shown in figure 3a. The graph of a reduced TMO (bin size =16) looks like piece- wise curve as shown in figure 3b.

A key feature of the TMO is the set of values of the slope of the TMO curve for all code values or bins from the working range of the image data. Thus in an embodiment, the processor 103 is configured to determine the slope of the tone mapping operator values on the basis of the following equation:

Figures 4a and 4b show graphs of the values of the reduced TMO (figure 4a, which is identical to figure 3b) and the slope of the reduced TMO (figure 4b) for a 16 bits size of the bins. If in figure 4b the slope is greater than one, i.e. S(bin) > 1, it means that the TMO increases the contrast of the details with the background luminance L e bin. If the slope is smaller than one, i.e. S(bin) < 1, it means that the TMO decrease the contrast of the details with the background luminance L e bin. Thus, according to embodiments of the invention the slope value is the parameter of the TMO that allows controlling the contrast transformation and therefore compensating (partly or fully) a given contrast loss.

As already described above, embodiments of the invention make use of the tone mapping operator (TMO) as an image enhancement tool. As will be described in more detail in the following, embodiments of the invention allow determining an optimal TMO, which takes into account the given viewing conditions (Lamb), given display parameters and given displayed image statistics (i.e. distribution

or image histogram H).

Embodiments of the invention are based on the general approach of optimizing the TMO by compensating a visible contrast loss. The visible contrast loss may be calculated considering the given viewing conditions and the ideal viewing conditions as a reference.

Embodiments of the invention make use of the Contrast Sensitivity Loss Function (CSLF) taking into account the display parameters, viewing conditions and the TMO that is being applied to the image. Thus, in an embodiment the processor 103 is configured to determine the contrast sensitivity loss function on the basis of the following equation:

In embodiments of the invention based on bins of code values, the processor 103 can be configured to use a reduced version of the above CSLF.

As already described above, the processor 103 of the electronic device 100 is configured to determine the optimal tone mapping operator, i.e. the optimal tone mapping operator values, by minimizing the lack of contrast function, in particular the absolute value of the lack of contrast function. Thus, in an embodiment, the processor 103 is configured to determine the optimal TMO (oTMO) as the tone mapping operator that minimizes the maximum absolute value of the lack of contrast function, i.e.

In an embodiment, the processor 103 of the electronic device 100 is configured to determine the optimal TMO in an iterative manner, as will be described in the following in the context of figures 5a-c.

Figure 5 a shows a first initialization stage according to an embodiment, which starts at block 500 and ends at block 509. In a block 501 the values of the TMO are initialized by setting the TMO value for each code value equal to the code value (as the output cv are equal to the input cv the initial TMO does not apply any image processing). In a block 503 the TMO slope values are all set to 1 , i.e. S TMO (cv) = 1

(all values are equal to 1 so the contrast transformations are not assumed by the initial TMO). In a block 505 the background luminance values of the "ideal" display are defined by the EOTF only, as the "ideal" display does not include the reflected light. In a block 507 the reference, i.e. ideal, maximum achievable contrast sensitivity values of the contrast sensitivity function are set. As described above, ideal viewing conditions assume that the ambient luminance is equal to the background luminance that provides the best possible viewing conditions.

In an embodiment, the processor 103 is configured to repeat the following steps shown in figures 5b and 5c until an optimal TMO has been achieved, including a TMO slope correction routine, which starts at block 510 and ends at block 522, as well as a TMO correction routine, which starts at block 530 and ends at block 532.

In a block 51 1 the bbackground luminance values of the display 101 under real viewing conditions are determined as the sum of the EOTF applied to the code value processed by the tone mapping operator and the reflected light. In a block 513 the real contrast sensitivity values provided by the contrast sensitivity function under real viewing conditions are determined (as described above "real viewing conditions" depend on the given ambient luminance L am b). In a block 515 the contrast sensitivity loss function values are calculated. In a block 517 the lack of contrast function values are calculated. In a block 519 a contrast correction coefficient C c is determined on the basis of the following equation:

wherein r denotes a speed of correction coefficient. An exemplary value for the speed of correction coefficient is r— 0.001. The contrast correction coefficient C c leads to a correction of the TMO slope values in step 521 using the contrast correction coefficient C c , in case LCF(cv) < 0, which corresponds to a contrast loss.

In the TMO correction routine, the corrected TMO values are reconstructed using the corrected values of the TMO slope (see block 531 of figure 5b).

In a block 533 shown in figure 5c, the processor 103 checks whether the TMO is optimal. If this not the case yet, the processor 103 returns to step 51 1 of figure 5b. According to embodiments of the invention three versions of the optimal TMO are possible in different scenarios. The contrast loss can be partly compensated, i.e.

The contrast loss can be fully compensated, i.e. The contrast loss can be fully compensated and the contrast

can be improved, i.e

The block 535 of figure 5c indicates that the previous processing steps are performed for all code values defined by the image to be processed.

In the following an example will be provided to illustrate the performance of embodiments of the invention. For the example the following assumptions have been made. The image to be displayed by the display 101 of the electronic device 100 is represented by a plurality of 8 bit code values defining the Y-component of the image. The range of code values, i.e. the working range WR = [0 - 255] is split into 32 bins, wherein each bin has a size bin size = 256/32 = 8. The distribution of code values, i.e. the histogram H has the following exemplary normalized values for the 32 bins: H(l) ... H(6) = 0.1333; H(7) = 0.0667; H(8) = 0.0533; H(9) = 0.04; H(10) = 0.0267; H(l l) = 0.0133; H( 12) ... H(32) = 0. The exemplary ambient light luminance is L am b = 400 nit. The EOTF of the display 103 of the electronic device 100 is assumed to be the EOTF standardized by ITU-R.BT-709 with the black level Lb = 0 nit and the white level Lw = 250 nit. The reflection coefficient of the display 101 is assumed to be kr/i - 0.03. The speed of correction coefficient, i.e. r, described in the context of step 519 of figure 5b is assumed to have the value 0.001 (corresponding to a slow correction).

In the figures 6a-g the different graphs show the following quantities as a function of the bin number (in the order from left to right and from top to bottom):

H - histogram of code values (which will not change through the iterative procedure);

TMO - initial TMO is a straight line;

S - initial TMO slope is constant and equal to 1 ;

Lr - luminance of the real display, curve starts from non-zero because of reflected ambient luminosity, which in this example is 12 nit;

CSi - ideal contrast sensitivity function/value is about 100 for the whole working range;

CSr - real contrast sensitivity function/value is significantly decreased for the dark part of dynamic range (i.e. smaller bin numbers) because the ambient light is high (400 nit);

CSLF - contrast sensitivity loss function is high for the dark part of the dynamic range;

LCF - lack of contrast function is big as the image histogram is clustered in the dark range mostly (i.e. at smaller bin numbers);

Cc - slope correction coefficient (as defined in step 519 of figure 5b) is significantly larger than 1.0.

After the first slope correction shown in figure 6b, the TMO became stronger for the dark range. Therefore, the contrast sensitivity loss function has been decreased and the lack of contrast function has been decreased as well. However, they are still negative, so that an additional iteration is needed. 4

22

After 4 iterations (shown in figure 6c) the lack of contrast function has decreased by 4 times, but is still negative

After 8 iterations (shown in figure 6d) the lack of contrast function has decreased by 10 times, but is still negative. Although the lack of contrast function has almost been compensated, additional iterations will correct it to zero accurately. Figures 6e and 6f show the different quantities as a function of the bin number after 16 and 24 iterations, respectively.

After 80 iterations (shown in figure 6g) the TMO has become optimal (LCF=0 for the whole working range) and the contrast loss function has been fully compensated.

Figure 7 shows a schematic diagram illustrating a method 700 of operating the electronic device 100 comprising the display 101 configured to display electronic image data, wherein the display 101 comprises a plurality of pixels and each pixel is associated with at least one code value within a range of code values. The method 700 comprises the steps of: determining 701 a tone mapping operator, i.e. tone mapping operator values, by minimizing a maximum of an absolute value of a lack of contrast function, wherein the lack of contrast function is based on a distribution of the code values of the plurality of pixels over the range of code values and at least one parameter relating to external viewing conditions and/or the display 101 ; and adjusting 703 the code values of the plurality of pixels on the basis of the tone mapping operator.

While a particular feature or aspect of the disclosure may have been disclosed with respect to only one of several implementations or embodiments, such a feature or aspect may be combined with one or more further features or aspects of the other implementations or embodiments as may be desired or advantageous for any given or particular application. Furthermore, to the extent that the terms "include", "have", "with", or other variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term "comprise". Also, the terms "exemplary", "for example" and "e.g." are merely meant as an example, rather than the best or optimal. The terms "coupled" and "connected", along with derivatives thereof may have been used. It should be understood that these terms may have been used to indicate that two elements cooperate or interact with each other regardless whether they are in direct physical or electrical contact, or they are not in direct contact with each other.

Although specific aspects have been illustrated and described herein, it will be appreciated that a variety of alternate and/or equivalent implementations may be substituted for the specific aspects shown and described without departing from the scope of the present disclosure. This application is intended to cover any adaptations or variations of the specific aspects discussed herein.

Although the elements in the following claims are recited in a particular sequence with corresponding labeling, unless the claim recitations otherwise imply a particular sequence for implementing some or all of those elements, those elements are not necessarily intended to be limited to being implemented in that particular sequence.

Many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the above teachings. Of course, those skilled in the art readily recognize that there are numerous applications of the invention beyond those described herein. While the present invention has been described with reference to one or more particular embodiments, those skilled in the art recognize that many changes may be made thereto without departing from the scope of the present invention. It is therefore to be understood that within the scope of the appended claims and their equivalents, the invention may be practiced otherwise than as specifically described herein.