The invention relates to the processing of photographic images and more particularly the correction of red eyes in a digital photographic image taken with a flash.

The digital cameras use systems that produce preparatory preliminary flashes before the taking of a photographic image with a flash. The intense light emitted in this way by the preparatory flashes is used to make the pupils of the people being photographed contract.

In this way, the surface of the eye reflecting the light emitted by the flash, that is to say the pupil, is minimized, consequently minimizing the red-eye effect that appears in a photographic image taken with flash.

Although the reflecting surface is minimized, the surface of the pupil is never nil. Consequently, the red-eye effect is never completely non-existent in a photograph.

Moreover these systems often fail. In practice, these systems do not operate, for example, in the case where the subject looks in a direction other than that of the lens of the camera.

It is also known to correct the red eyes by adjusting the colours of the pixels of the eyes in the photographic image after the latter has been captured using software internal to a digital camera, or using image processing software on computers.

Unfortunately, these solutions generally produce unsatisfactory results with looks lacking in expression. These looks lacking in expression are due to pupils whose colour is an opaque grey or an unreal aberrant colour. In the case of an inconsistent or aberrant pupil colour, the dilated pupil is completely corrected by desaturation of the colours, that is to say by reducing the saturation of the colours, sometimes inducing unreal results.

It is proposed to overcome the drawbacks mentioned hereinabove by replacing, for example, the red pixels of a pupil of an eye in a photographic image taken with flash with pixels of the pupil of the corresponding eye in a photographic image taken without flash, and thus restore the true texture of the pupil of the eye.

According to one aspect, there is proposed a method o f correcting a representation of a central region of an eye in a photographic image of said eye taken with a flash.

According to a general characteristic of this aspect, the method comprises a taking of a supplementary photographic image of the eye without flash, and a replacement in the photographic image taken with flash of said representation of the central region of the eye with the corresponding representation of said central region of the eye in the supplementary photographic image, in case of detection of a chromatic anomaly in the representation of the central region of an eye in the photographic image taken with flash. A detection of a chromatic anomaly in the representation of the central region of an eye can, for example, correspond to a detection o f a red eye. This detection can be done by analysing the chrominance o f the representation of the central region of the eye. If a colour, such as red for example in the case of red eyes, is present in a significant number of pixels, up to a predetermined threshold for said colour, then the representation of the central region of the eye can be considered to exhibit a chromatic anomaly.

The predetermined thresholds for the different chrominances (the different colours) are well known to those skilled in the art for a central region of the eye and can be adjusted according to the optical and digital characteristics of the digital camera used.

The central region of the eye comprises, for example, the pupil and/or the iris.

The taking of the supplementary photographic image can comprise a taking of several additional photographic images taken without flash and a selection of said supplementary photographic image from the additional photographic images taken without flash.

The taking of photographic images can be done in a close time window before and/or after the taking of the photographic image taken with a flash. As an indication, the close time window can be between -0.5 seconds and +0.5 seconds.

The selection of the supplementary photographic image from the additional photographic images can be done based on the aperture ratio of the eye.

By comparing the aperture ratio of an eye in the photographic image taken with a flash with the aperture ratio of the corresponding eye in each additional photographic image taken without flash, it is possible to determine the additional photographic image in which the deviation between the aperture ratio of the eye in the additional photographic image and that of the photographic image taken with flash is the smallest, and thus select this additional photographic image as the supplementary photographic image.

In the digital photographic systems, reduced resolution preview images make it possible to view on a control screen of the digital camera the photographic images preceding the triggering of the capture.

Some systems are not capable of recording several additional images at full resolution, because that requires a large bandwidth for transferring the images and a large memory space for storing them.

Such a limitation does not exist for the preview images, because, given their reduced resolution, they are less memory and bandwidth intensive. In this case, a number of preview images can be recorded before the taking of the photographic image taken with a flash and it is possible to use these images as additional images.

These preview images must be oversampled to overcome the resolution difference with the photographic image taken with flash.

Thus, by oversampling a preview image, the supplementary photographic image has the same resolution, and therefore the same number of pixels, as the photographic image taken with flash. In practice, the oversampling of a photographic image makes it possible to increase its resolution and therefore the number of pixels defining it.

The replacement of the representation of the central region o f the eye in the photographic image taken with a flash with the central representation of the corresponding eye in the supplementary photographic image preferably includes a motion compensation between the photographic image taken with flash and the supplementary photographic image. The expression "motion compensation" is meant here in the broad sense and can be applied to the whole of the image or to a portion only, for example, by compensating at least a portion of the supplementary photographic image surrounding the representation of the central region of the eye. According to one implementation, if the coordinates of the representation of the central region of an eye in the supplementary photographic image differ from the coordinates of the corresponding representation of the central region of the eye in the photographic image taken with flash, a correction translation vector is applied to the representation of the central region of the eye in the supplementary photographic image in order to make the representations of the central regions of the eye detected in the photographic images correspond. In this way, a pixel-by-pixel replacement can be carried out.

Thus, if the subject or the eye of the subject has shifted between the two photographic images taken, that is to say between the photographic image taken with flash and the supplementary photographic image taken without flash, a motion correction of the representation of the central region of the eye makes it possible to replace the corresponding pixels at the corresponding places in the photographic image taken with flash.

The replacement of the representation of the central region o f the eye in the photographic image taken with a flash with the corresponding central representation of the eye in the supplementary photographic image can advantageously comprise a correction of the brightness of at a least a portion of the supplementary photographic image surrounding the representation of the central region of the eye.

In this way, the brightness defect of the supplementary photographic image taken without flash is overcome. There is thus obtained a supplementary photographic image taken without flash in which the brightness is equivalent to the brightness of the photographic image taken with flash. In this way, the replacement of the pixels of the pupil of the eye in the photographic image taken with flash with the pixels of the pupil of the eye of the photographic image taken without flash is done with an equivalent brightness of the pixels.

According to another aspect, there is proposed a digital camera comprising means of correcting a representation of a central region of the eye in a photographic image of said eye taken with a flash.

The correction means comprise means of capturing a supplementary photographic image capable of taking a supplementary photographic image of the eye without flash, means of detecting a chromatic anomaly capable of detecting a chromatic anomaly in the representation of a central region of the eye in the photographic image with flash and of delivering a detection signal in case of such a detection, and replacement means capable of replacing, in response to the detection signal, in the photographic image taken with a flash, said representation of a central region of the eye with the corresponding representation of said central region of the eye in the supplementary photographic image. Advantageously, the correction means can comprise control means capable of controlling a taking of several additional photographic images taken without flash, and selection means capable of selecting said supplementary photographic image from the additional photographic images taken without flash. The selection means preferably comprise means of determining the aperture ratio of the eye.

Advantageously, the correction means comprise oversampling means capable of increasing the resolution of a photographic image.

The replacement means can advantageously comprise motion compensation means capable of compensating for the motion between the photographic image taken with flash and the supplementary photographic image, for example, by replacing at least a portion of the supplementary photographic image surrounding the representation o f the central region of the eye. Preferably, the replacement means comprise brightness compensation means capable of correcting the brightness of at least a portion of the supplementary photographic image surrounding the representation of the central region of the eye. According to another aspect, there is proposed a mobile telephone including a digital camera as defined hereinabove.

Other benefits and features of the invention will become apparent on studying the detailed description of the embodiments and implementations, which are by no means limiting, and the appended drawings, in which:

Figure 1 diagrammatically represents an embodiment of a mobile telephone including a digital camera including means of correcting a representation of a central region of the eye in a photographic image o f said eye taken with a flash;

Figure 2 diagrammatically represents an embodiment of the means of correcting a representation of a central region of the eye in a photographic image o f said eye taken with a flash; - Figure 3 diagrammatically represents an embodiment of the means of selecting the supplementary photographic image;

Figure 4 illustrates an embodiment of the replacement means; - Figure 5 represents a synopsis of an implementation o f a method of correcting a representation of a central region of an eye in a photographic image of said eye taken with a flash.

Figure 1 shows a mobile telephone T including a digital camera P. The digital camera P further includes correction means 1 making it possible to correct a chromatic anomaly in a representation of a central region of the eye, such as a red-eye phenomenon, in a photographic image taken with a flash.

Figure 2 illustrates an embodiment of the correction means 1. The correction means 1 further include capture means 2 making it possible to take a photographic image, means 3 of detecting the eyes, means 4 of detecting the representation of the centre of an eye, means 5 of detecting a chromatic anomaly, selection means 6, replacement means 7 and control means 8.

The capture means 2 are controlled by control means 8 which also control means 9 of activating a flash when taking a photographic image with a flash.

Some of these means are well known in themselves to those skilled in the art. These are in particular the capture means 2, detection means 3 and 4. More specifically, and as an indication, the detection means 3 and 4 apply software methods known per se and for example described in the articles "Driver Fatigue Detection Based on eye Tracking and Dynamic Template Matching" and "Real-time pupil detection based on three-step hierarchy" published in the IEEE review.

The photographic image taken with flash that is captured in this way is delivered to the means 3 of detecting the eyes. Using one of the methods known from the state of the art, the means 3 o f detecting the eyes detect the position of the eyes in a face present in the photographic image taken with flash.

The photographic image taken with flash complemented with information on the position of at least one eye of a face that can be seen in the photographic image taken with flash is delivered to the means 4 of detecting the representation of the centre of an eye. The detection means 4 make it possible to detect, based on the information delivered by the means 3 of detecting the eyes, the position of the representation of the centre of the eye for each of the eyes detected in the photographic image with flash. The pupil and/or the iris of each detected eye is thus identified. The photographic image taken with flash complemented with the information on the position of the representation of the centre of the eye is then delivered to the means of detecting chromatic anomalies 5 which also receive as input information originating from the means 9 of activating a flash. The information delivered by the means 9 of activating a flash makes it possible to know whether the flash was used when capturing the photographic image, and whether the image received by the selection means corresponds to the photographic image taken with flash or to an additional photographic image taken without flash.

The detection means 5 make it possible to detect a chromatic anomaly in the representation of a central region of the eye in the photographic image with flash, such as a red eye. The detection by the means 5 is done, in this embodiment, by analysing the chrominance o f a representation of the central region of the eye.

If a colour, such as red chrominance for example in the case o f red eyes, is detected, for example, in pixels contained in a working area including the detected eye, then the representation of the central region of the eye can be considered to exhibit a chromatic anomaly. If the photographic image taken with flash does not have any chromatic anomaly in a representation of a central region of an eye, the detection means 5 deliver the photographic image directly as output from the correction means 1. The photographic image taken with flash thus does not undergo any correction, and becomes the final photographic image.

On the other hand, if a chromatic anomaly in a representation of the central region of an eye is detected in the photographic image taken with flash, the latter, complemented with information on the position of the representation of the centre of the eye, is then delivered to the selection means 6 which also receive as input information originating from the means 9 of activating a flash. The information delivered by the means 9 of activating a flash makes it possible to know whether the flash was used when capturing the photographic image, and whether the image received by the selection means corresponds to the photographic image taken with flash or to an additional photographic image taken without flash.

A number of additional photographic images are captured before and/or after the taking of the photographic image taken with flash using the capture means 2. The control means 8 control the taking of several additional photographic images without activating the flash activation means 9 in order for the latter to be taken without flash.

The additional photographic images can be captured in a time zone situated around the photographic image taken with flash, and these additional photographic images can be previewed images.

The additional photographic images captured in this way by the capture means 2 are delivered to oversampling means 100. If an additional photographic image is a preview image, the latter is oversampled by the means 100 in order to increase its resolution to the resolution of the photographic image taken with flash. After an oversampling of a preview image, the latter has the same resolution, that is to say the same number of pixels, as the photographic image taken with flash. Otherwise, the resolution of the photographic image is unmodified.

Each additional photographic image is then delivered to the means 3 of detecting the eyes which detect the eyes of each face present in the additional photographic image. The additional photographic image complemented with the information on the position of the eyes of each face detected in the supplementary photographic image is delivered to the means 4 of detecting the representation of the centre of an eye.

Based on the information delivered by the means 3 of detecting the eyes, the detection means 4 detect the position of the representation of the centre of the eye for each of the eyes detected in the additional photographic image. The additional photographic image complemented with the information on the position of the representation of the centre of each detected eye present in the additional photographic image is then delivered to the chromatic anomaly detection means 5.

The chromatic anomaly detection means 5 also receive information originating from the means 9 of activating a flash. The means 9 in this case deliver information indicating that the photographic image was taken without flash. The chromatic anomaly detection means 5 do not do any processing or analysing of the photographic image taken without flash and deliver as output the additional photographic image complemented with the information on the position of the representation of the centre of each detected eye present in the additional photographic image as delivered by the detection means 4.

The selection means 6 then select a supplementary photographic image taken without flash from the additional photographic images taken without flash. The selection means 6 then deliver as output the photographic image taken with flash and the supplementary photographic image taken without flash to the replacement means 7.

The replacement means 7 then replace, in the photographic image taken with a flash, said representation of a central region of the eye with the corresponding representation of said central region of the eye in the supplementary photographic image taken without flash for each detected eye.

Figure 3 illustrates an embodiment of the selection means 6. In this embodiment, the selection means 6 comprise means 10 o f determining the aperture ratio of the eye, storage means 1 1 and a selection module 12.

The means 10 of determining the aperture ratio of an eye are capable of determining the aperture ratio of an eye based on, for example, the surface occupancy ratio of the white of an eye in a predetermined area including a detected eye. The means 10 o f determining the aperture ratio of an eye thus measure the aperture ratio of each detected eye for each image that is delivered to them.

Thus, the aperture ratio of each eye detected in the photographic image taken with flash will be determined and the photographic image taken with flash complemented with the information on the aperture ratio of each detected eye will be delivered to the storage means 1 1.

The storage means 1 1 also receive as input the information deriving from the means 8 of generating a flash making it possible to indicate whether the stored photographic image was taken with a flash or not. In this way, the storage means determine whether the stored photographic image is the photographic image taken with flash and therefore the photographic image to be corrected, or an additional photographic image which will be used as a basis for correcting the photographic image taken with flash.

Each additional photographic image taken without flash will also be delivered to the means 10 of determining the aperture ratio of an eye, which then deliver to the storage means 1 1 the additional photographic images complemented with the information on the aperture ratio of each eye detected in each of the photographic images.

The storage means 1 1 then deliver to the selection module 12 the additional photographic images complemented with the information on the aperture ratio of each detected eye, together with the photographic image taken with flash which serves as a reference for the selection of the additional photographic image closest to the photographic image taken with flash. The selection module 12 then selects the supplementary photographic image, in this case, the additional photographic image taken without flash closest to the photographic image taken with flash, that is to say, the photographic image taken without flash having the aperture ratio of each eye closest to the aperture ratio of each corresponding eye in the photographic image taken with flash.

In the case where several faces, and therefore several eyes, are detected in the additional photographic images and that taken with flash, it is possible for example to select the additional photographic image that includes the smallest average aperture ratio difference. For this, the difference is calculated between the aperture ratio of an eye in an additional photographic image and the aperture ratio of the corresponding eye, for each detected eye. The average of the differences calculated in this way is then calculated. The additional photographic image selected as the supplementary photographic image will be that with the smallest deviation average.

The selection module 12 then delivers as output the supplementary photographic image taken without flash corresponding to the additional photographic image taken without flash selected by the selection module 12, which will be delivered to the replacement means 7. The storage means 1 1 also deliver as output the photographic image taken with flash to the replacement means 7. The photographic image taken with flash and the supplementary photographic image are then delivered to the replacement means 7.

Figure 4 diagrammatically illustrates an embodiment of the replacement means 7. The replacement means 7 in this embodiment comprise motion compensation means 20, brightness compensation means 21 , and means 22 of replacing the pixels of the centre of the eye.

The means 20 of compensating the motion of the detected eye make it possible to compensate for the motion of the eye, and more particularly of the representation of the centre of the eye, between the photographic image taken with flash and the supplementary photographic image taken without flash. The motion compensation means 20 compare the coordinates of the centre of an eye in the supplementary photographic image determined previously by the means 4 of detecting the representation of the centre of an eye, with the coordinates of the centre of a corresponding eye in the photographic image taken with flash in a limited spatial window around the position of the eye concerned.

If a difference appears between the coordinates of the representation of the centre of the eye in the supplementary photographic image relative to the photographic image taken with a flash, a motion compensation is done by applying a translation to the representation of the centre of the eye in the supplementary photographic image. The centre of the eye in the supplementary photographic image then has the same coordinates as the representation of the centre of the corresponding eye in the photographic image taken with a flash.

The motion compensation means 20 then deliver as output the photographic image taken with a flash and the supplementary photographic image whose motion has been compensated, to the brightness compensation means 21.

The brightness compensation means 21 make it possible to correct the brightness, for example by increasing it, of at least a portion of the supplementary photographic image surrounding the representation of the central region of the eye so as to overcome the brightness difference between the supplementary photographic image and the photographic image taken with a flash. The brightness of the supplementary photographic image and more particularly the brightness of a portion of the supplementary photographic image surrounding the representation of the central region of the eye, is corrected by an internal digital processing so as to obtain a brightness comparable to the brightness obtained in the photographic image taken with a flash. In this way, when replacing the representation of the central region of the eye in the photographic image taken with a flash with the representation of the central region of the eye in the supplementary photographic image taken without flash, the difference in brightness between the two images will not be perceived and the replacement will be all the more discrete and the rendering all the more realistic.

The brightness compensation means 21 then deliver as output the photographic image taken with a flash and the supplementary photographic image in which the motion has been compensated and in which the brightness has been corrected, to the pixel replacement means 22.

The pixel replacement means 22 will then perform a pixel-by- pixel replacement of the pixels of the representation of the central region of the eye in the photographic image taken with a flash with the corresponding pixels of the representation of the central region of the eye in the supplementary photographic image. The pixel replacement means 22 then deliver as output the corrected photographic image taken with a flash, that is to say the photographic image taken with a flash in which the representation of the central region of each eye has been replaced with the representation of the central region of each corresponding eye in the supplementary photographic image.

Figure 5 represents a synopsis of a method of correcting a representation of a central region of an eye in a photographic image o f said eye taken with a flash, according to one implementation. In a first step 501 , a photographic image is taken with the activation of a flash. In a subsequent step 502, a detection of the eyes is done in the photographic image taken with a flash captured in this way. The detection of the eyes is done, for example, using a known method of varying brightness based on a detected area corresponding to a face. In a third step 503 , the representation of the centre of the eye is detected for each eye detected in the photographic image taken with flash.

In a fourth step 504, a detection of chromatic anomalies in a representation of a central region of an eye is done. If no anomaly is detected, the method goes directly to the final step 510 in which the final image delivered corresponds to the photographic image taken with flash without correction. Otherwise, a determination of the eye aperture ratio is carried out, in a subsequent step 505 , for each detected eye based, for example, on the area ratio that represents the white of the eye for a working area including the detected eye.

Before and/or after the first step 501 of capturing a photographic image with a flash, additional photographic images can be captured without flash in a step 51 1.

In a subsequent step 512, if one of the photographic images is a preview image, the later is oversampled in order to increase its resolution.

In a subsequent step 513 , a detection of the eyes is done in each of the additional photographic images captured in this way. In a step 514, a detection of the representation of the centre of the eye is done for each detected eye for each of the additional photographic images. A determination of the eye aperture ratio is then done, in a subsequent step 515 , for each detected eye in each additional photographic image.

In a step 506, a supplementary photographic image is selected from the additional photographic images taken without flash. The selection is made by comparing the aperture ratio of each eye of an additional photographic image to the aperture ratio of each corresponding eye in the photographic image taken with flash. The comparison is thus done for all the additional photographic images relative to the photographic image taken with flash. The additional photographic image that has the smallest deviation in the aperture ratio of each eye with the aperture ratio of each corresponding eye in the photographic image taken with flash is selected. In a subsequent step 507, a motion compensation is done in the supplementary photographic image in order to correct any displacement of the representation of the centre of an eye between the supplementary photographic image and the photographic image taken with a flash. To compensate for any displacement, a translation is applied at least to the portion of the supplementary photographic image including the representation of the central region of the eye. The translation vector is determined from the difference in coordinates of the representation of the centre of an eye between the photographic image taken with flash and the supplementary photographic image. In a subsequent step 508, a brightness compensation is done on the supplementary photographic image in order to overcome the brightness difference that exists between the photographic image taken with flash and the supplementary photographic image. Thus, the brightness of at least a portion of the supplementary photographic image including the representation of the central region of the eye is increased so as to obtain a brightness that is comparable, even equal, to the brightness of the photographic image taken with a flash.

In a subsequent step 509, a pixel-by-pixel replacement of the representation of the central region of the eye in the photographic image taken with flash with the representation of the central region o f the corresponding eye in the supplementary photographic image is done.

The corrected photographic image taken with a flash is then delivered as final photographic image taken with flash on completeion of the final step 510.

It should be noted that, if no chromatic anomaly is detected in the photographic image taken with flash in the step 504, the steps 5 12 to 515 and 505 to 509 are not carried out.