Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
METHOD AND DEVICE FOR THE TRANSMISSION OF IMAGES TO DEVICES PROVIDED WITH SMALL DISPLAYS
Document Type and Number:
WIPO Patent Application WO/2005/076590
Kind Code:
A1
Abstract:
The present invention concerns a method and a device for transmitting images to devices with displays having small dimensions by selecting the part of the image to be transmitted, detecting the area on which the observer has fixed his attention, processing the selected area and transmitting it to the device.

Inventors:
MARINGELLI FRANCESCO (IT)
HEIKKILAE JUHA-PEKKA (FI)
Application Number:
PCT/EP2005/050530
Publication Date:
August 18, 2005
Filing Date:
February 08, 2005
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
SMALLTECK S N C DI F MARINGELL (IT)
SR LABS S R L (IT)
MARINGELLI FRANCESCO (IT)
HEIKKILAE JUHA-PEKKA (FI)
International Classes:
H04N1/00; (IPC1-7): H04N1/00
Domestic Patent References:
WO2001069585A12001-09-20
Foreign References:
US20020063905A12002-05-30
EP1349364A22003-10-01
US6417867B12002-07-09
US6381339B12002-04-30
Attorney, Agent or Firm:
Gervasi, Gemma (Corso di Porta Vittoria 9, MILAN, IT)
Download PDF:
Claims:
CLAIMS
1. Method for image processing and transmission to receiving devices with small displays characterised in that a predefined size selected portion of the entire image is transmitted to the receiving devices.
2. Method according to claim 1 characterised in that the portion of the entire image is selected according to the point of view of one or more observer gazing at this image.
3. Method according to claims 1 and 2 characterised in that it comprises the following steps: a) the size of the rescaled image is defined in relation to the display dimensions of the receiving device; b) the coordinates of the point of fixation of an observer on the image are obtained; c) a selected area of the image is cropped, said area being defined by the centre of said coordinates related to the observer points of fixation and the dimensions of which are defined according to step a); d) possibly this image portion according to step c) is expanded so that it has greater dimensions in comparison to those of the receiving device display; e) possibly the size of the image according to step d) is rescaled to bring the size of the image back to the same dimensions of the display of the receiving device; f) possibly the parameters of the image portion according to step e) are modified ; g) possibly an intelligent compression of the whole image or of part of it is performed ; h) the portion of image calculated at step c) is transmitted to the receiving device ; i) the previous steps are repeated every time the part of the image on which the attention of the observer is focused changes.
4. Method according to claim 3 characterised in that the characteristic parameters according to step f) above are the colour, the contrast and the brightness.
5. Method according to claims 14 characterised in that said step b) is performed by the observer with the help of a computer mouse.
6. Method according to claims 14 characterised in that said step b) is performed by an eyetracker device.
7. Method according to the previous claims characterised in that the receiving device provided with display belongs to the groups of portable phones, cellular telephones, tablet PCs, Palm/Handhelds, smart phones, mini DVD readers/players, mini Video CD readers/players and laptop computers.
8. Computer program product having a computer readable medium said computer program product comprising means of performing the steps described in claim 3, when said program is ran on a computer.
9. Computer readable means comprising a recorded computer program according to claim 8.
10. Device for images transmission to instruments with small displays characterised in that it comprises : means to display the image or the video that is going to be processed means to allow an observer to select the part of the observed video/image on which the attention of the observer is fixed.
11. means to crop and process this part of the observed video means to transmit this processed part of the observed video to a receiving device.
12. Device according to claim 10 characterised in that said means to display the image or the video that is going to be processed consist of a computer monitor, said means to allow an observer to select the part of the observed video/image on which is fixed the attention of the observer consist of an eyetracking device or of a computer mouse, said means to crop and process said part of the observed video consist of a computer running an appropriate software program, said means to transmit this processed part of the observed video to a receiving device consist of a GSM transmitting device.
13. Method according to daims 17 characterised in that the image that is going to be transmitted is a video image.
14. Device according to claims 1011 characterised in that the image that is going to be transmitted is a video image.
Description:
Method and device for the transmission of images to devices provided with small displays Field of the invention The present invention is related to the field of processing and transmission of images.

State of the art In the last decade,"mobile"technologies have become extremely popular, initially within a rather limited and innovation-friendly group of people, then more and more diffusing to other layers of the population and to less homogeneous areas of the public.

With the terms"mobile"technologies, all those devices characterized by small dimensions and easily transportability are indicated, devices which are used without the need of placing them on a support, a transportation system or connecting them to power supply units that would reduce the simplicity and comfort of mobile use.

Examples of these devices are portable computers (notebooks or laptops), electronic notebooks or PDAs (Personal Digital Assistants), portable DVD readers, cellular telephones.

A common element of all the above described (and similar) devices is the display, the component intended to the representation of information mainly related to the function of the device itself and the communication with the user-observer- consumer.

These displays are characterized and defined by a series of technical parameters such as the dimension, the resolution, and the frequency.

The parameters related to the dimension of the display is particularly important having a direct impact on the quality of the image shown by device to the consumer, that, to a first approximation, is directly proportional to the dimension of the display. Whenever it is necessary to fulfil the processing of a video content designed for computers displays (today typically at least 15") or for television sets (up to 28"-32") to be shown on a display of smaller dimensions such as the display of a cellular telephone, obviously it is necessary to resize the image. Such operation, that typically consists in a scalar shrinking of the original video to suit

the dimensions of the display of the mobile telephone, generally means a substantial loss of quality of the video and of its general legibility and usability, as it implicates a reduction of the image of a scale factor at least 1 to 5 and consequently a diminution of its resolution around the same scale, thus jeopardizing the possibility for the consumer to enjoy the image in satisfactory way.

This is particularly true in cases of wide angular camera shots, particularly common in the reproduction of sport events: in these cases, the dimension of the single elements (for example, the single player or ball) risks to become too small to be seen by the user or even too small to be shown by the device. This problem is increased not only by the fact that the mobile terminals are endowed with a display of notably lower dimensions in comparison to those of a monitor for PCs or TV sets, but they are also characterized by a lower resolution and a lower number of available colours.

On the other hand it is known from neuroscientific studies and, more particularly, from studies in the field of the visual perception and ocular movements, that an observer, in front of a scene, observes only some details of the scene-especially if the scene is dynamic as in the case of a video-skipping in reality the biggest part of the shown elements. The elemente'are not at all selected in a casual way, and this selection is often driven by conscious volitive efforts. Sometimes, the ocular movements are"automatic"and therefore out of the wish of the subject. In other words, the surrounding world is visually explored as if seen through a hole of a mobile keyhole, collecting the pieces of a puzzle gathered on the retina one by one, drawing by these pieces the sense of what is happening. Typically a third of the pieces is sufficient to understand the object of the visual investigation. We can synthesize what it happens through the principle of the"redundancy of the information"as follows : the system is based on the multiplicity of evidences and signals to"decide"about the identity of the perceived information.

The method described in the present invention offers a solution to the problems described above, it guarantees a minimum loss of information in comparison to the original version of the image information and offers, in addition, the possibility of an intelligent compression of the video signal, decreasing in fact the minimum

requirements in terms of necessary bandwidth for the transmission of multimedia content. Finally, the method of the present invention enables the broadcasting of events in real time, particularly important when time is of particular importance, for example in case of sport events.

Summary of the invention The method and the device according to the present invention allow to transmit images to devices provided with a display of reduced dimensions.

Description of the figures Fig. 1-Schematic of the results using the device and the method according to the present invention Fig. 2-Schematic of the architecture of the preferential implementation of the device according to the present invention Fig. 3-Schematic of the method according to the present invention Fig. 4-Software program flowchart realizing the method according to the present invention Description Detailed of the invention The method according to the present invention-from now on referred to as ICS <BR> <BR> (Intelligent Cropping System) -represents a valid and efficient solution to the problem of the downsizing of the video format dimensions or image dimension starting from the standard format of, typically, 15"and transmitting those videos or images to devices with small displays such as cellular telephones or PDAs ( Personal Digital Assistant).

The ICS method according to the present invention uses the principle of information redundancy related to the visual perception illustrated above. In a normal situation typically a large number of indicators is available to the observer "to decide"if what he is seeing is a one thing or another. Not all the available elements will be useful to the"decision", and many of the useful elements will be ignored, simply because the observer will already have reached the decision without the need to observe all elements of the whole image. The more complex the image is the more ocular movements the subject will make and the more pieces of the puzzle he will be collecting to be able to achieve a perceptive "decision".

The functioning principle of the ICS method descends from the above characteristics of human visual perception and from the way in which the subjects determines what is the relevant information on a given display to achieve a "perceptive decision", said method emulates the intelligent selection of the pieces of the puzzle typically relevant to the"perceptive decision"for a human being in a given situation, allowing to ignore and discard (also on the physical level) the redundant pieces of the puzzle, thus allowing a considerable increase of image dimensions and resolution and/or a meaningful reduction of the quantity of data to be transmitted.

The method according to the present invention (ICS) can also be used, for displays of greater dimensions, as some kind of intelligent zoom or to change the properties of a determined portion of the image (for example, increasing in a differential way contrast and brightness).

Besides this it is possible to use ICS to operate an intelligent compression of the signal to reach an acceptable compromise between quality of the image/video and data quantity/available bandwidth.

ICS is an advanced method for video editing in real time, it is based on an intelligent algorithm able to digitally translate the ability to focus the attention and processing a specific selection of visual information usually only available to the human being.

On Fig. 1 it is displayed an illustration of an example of the method subject of the present invention and the way it operates. An image 10 part of a video content normally designed to be visualized on a screen with normal dimensions of, for instance, 15", is being reproduced on a display of small dimensions, for instance of a cellular telephone. The current know how and technology would simply decrease the dimension of the original image from which is obtained an image 11 with less details/information and a much lower quality level. The method according to the present invention achieves a change of the dimensions of the image taking into consideration the point of view of the observer. The ICS method individuates the part of the image on which the attention is focused, in this case the faces of the two people on the right side, it selects the corresponding area and cuts out this area from the rest of the image, maintaining matching dimensions with the target

device/display and getting the image n. 12 with more details and a higher level of resolution and quality.

With reference to Fig. 2 the architecture of a preferred embodiment of the device that realises the method ICS according to the present invention is displayed.

One or more subjects 20 observe a determined image or video content operating the selection of the parts of the observed elements which are object of their attention. In one of the preferred embodiment of the present invention an eye- tracker device 21 is used for this purpose, but the selection can also be made with other techniques such as a mouse. The eye-tracking device acquires the subject/subjects gaze coordinates 21 during the observation of a determined image or video 22 and monitors the position of the eyes of the subject 20 (referred to as"stalker") observing the video content 22, many times per second (typically 50 times per second).

At this point, the coordinates of the subjects fixation point are recorded by the eye- tracking device 21 frame by frame and, together with the digital recording of the observed event, they are sent to the computer 23 provided with a software program that carries out the ICS method according to the present invention. The computer 23 calculates and"cuts out" ("Intelligent Cropping function") frame by frame the portion of the image, ("cropping"), that the subjectiuser has effectively selected with his gaze. The software that carries out the ICS method also contains a function of a digital zoom to various zooming levels comparable to those used by traditional image processing and re-dimensioning systems. In particular, the subject can also change/expand the cropping area in such a way that said area has a larger size than the size of the target display.

For example, when it is necessary include an overall view of a determined scene, the subject/user can expand the cropped area of interest by simply pressing a given key on the keyboard. This action will activate a function for the expansion in scale of the cropping window: in this case, the image will result of a greater dimension in comparison to the target display though, therefore it will need to be rescaled, similarly to the process used today to fit images to the dimension of the target display. The dimension of the"cropping"is available in real time to be transmitted to the target device (in fact there is a small delay, of less than 1/30 of

second, due to the necessary computational process), for instance by a GSM transmitter through the GPRS or UMTS networks to the cellular telephone of the user 24.

Fig. 3 represents a schematic of the ICS method according to the present invention, showing the two inputs channels of the process, the data flow from the eye-tracking device 30 and the digitalized video content 31, the software program 32 that performs the method according to the present invention and the output 33 of this method, which, in a preferred embodiment of the present invention, is represented by a video content that is elaborated and sent to the display of the user cellular telephone.

The software program that performs the method is described by the flowchart of Fig. 4. The data input consists of the single pictures of the source video 40 and of the gaze coordinates of the point of interest related to a specific image 41. The process then cuts out the area of the image 40 the centre and dimension of which is defined by the gaze coordinates of the point of interest 41 and by the dimension of the target display.

The above defined function, it is referred to as"Intelligent Cropping"42. At this point, if the subject-observer ("Stalker") has activated the function of Intelligent ", zooming 43-the function that provides a resizing (enlargement) of the cropping area of the image so that it overcomes the display-target dimensions, afterwards the method according to the present invention operates a resizing of the same image, so that image coincides with the dimensions of the target display. On the opposite, if the stalker doesn't activate the Intelligent Zooming function 43 this process phase is jumped and the"Intelligent Image Processing"phase 44 takes place. This phase of the process is optional, like the"Intelligent Zooming"one.

The"Intelligent Image Processing"phase 44 allows to operate changes to the attributes of the image (for example, the colour, contrast and brightness) and/or to compress the image in a differential way, performing a low compression in the area of the fixation point (and therefore a maximum quality in this area of the image) and a higher compression of other areas of the image that are more distant from the fixation point of the observing subject (obtaining therefore a lower quality). If there is no need to change the image attributes and/or there is no need

to operate an intelligent compression of the image ("Intelligent Compression function"), the method jumps to the final phase of production of the output image 45.

The processing phases described so far are recursive: after the output image has been produced 45, the method according to the present invention captures the upcoming input images as well as the coordinates of the points of interest and the phases from 42 to 45 are repeated. The iteration of the process described above happens in real time (with a processing time per cycle lower than 1/30 of a second) and therefore allows the reproduction of the temporal attributes of the source video.

The present invention can be advantageously realized through a computer program that includes means of coding for the realization of one or more steps of the method, when this program is ran on a computer.

Therefore the protection it is extended to this computer program and to computer readable means including a recorded code, said computer readable means including means of encoding the program for the realisation of one or more steps of the method, when this program is ran on a computer.