Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
SYSTEM FOR MAINTAINING AND BROWSING VERY LARGE IMAGES
Document Type and Number:
WIPO Patent Application WO/2004/102415
Kind Code:
A1
Abstract:
A system and method for maintaining and browsing very large images as well as a software tool for implementing the system are described, A wavelet coded representation of an n-dimensional image is stored in the form of blocks of coded image data relating to different wavelet resolutions. Image data stored as blocks from different wavelet resolutions are retrieved and a part of the n-dimensional image reconstructed by partial inverse wavelet transform on the retrieved data and displayed.

Inventors:
JANSSENS DAVID (BE)
Application Number:
PCT/BE2004/000074
Publication Date:
November 25, 2004
Filing Date:
May 17, 2004
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
UNIV CATHOLIQUE LOUVAIN (BE)
JANSSENS DAVID (BE)
International Classes:
G06F17/30; H04N1/41; H04N7/26; (IPC1-7): G06F17/30; H04N1/41; H04N7/26
Foreign References:
US6546143B12003-04-08
Other References:
TAUBMAN D ED - INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS: "Remote browsing of JPEG2000 images", PROCEEDINGS 2002 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING. ICIP 2002. ROCHESTER, NY, SEPT. 22 - 25, 2002, INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, NEW YORK, NY : IEEE, US, vol. VOL. 2 OF 3, 22 September 2002 (2002-09-22), pages 229 - 232, XP010607302, ISBN: 0-7803-7622-6
TAUBMAN D ET AL: "Embedded block coding in JPEG 2000", SIGNAL PROCESSING. IMAGE COMMUNICATION, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 17, no. 1, January 2002 (2002-01-01), pages 49 - 72, XP004326798, ISSN: 0923-5965
DESHPANDE S ET AL: "Scalable streaming of JPEG2000 images using hypertext transfer protocol", PROCEEDINGS OF THE 9TH. ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA. OTTAWA, CANADA, SEPT. 30 - OCT. 5, 2001, ACM INTERNATIONAL MULTIMEDIA CONFERENCE, NEW YORK, NY : ACM, US, vol. CONF. 9, 30 September 2001 (2001-09-30), pages 372 - 381, XP002286797, ISBN: 1-58113-394-4
MAPPING SCIENCE, INC: "GeoJP2 - An Extended Standard for Raster Image Compression", 22 July 2002 (2002-07-22), XP002298141, Retrieved from the Internet [retrieved on 20040923]
TAUBMAN D: "Proposal and Implementation of JPIP (Jpeg2000 Internet Protocol) in Kakadu v3.3", 9 August 2002, XP002260543
"JPEG 2000 IMAGE CODING SYSTEM - PART 9: INTERACTIVITY TOOLS, APIS AND PROTOCOLS - CD 1.0", ISO/IEC JTC1/SC29/WG1 N2904, XX, XX, 14 March 2003 (2003-03-14), pages 1 - 7,1, XP001150721
Attorney, Agent or Firm:
Bird, Ariane (Klein Dalenstraat 42A, Winksele, BE)
Download PDF:
Claims:
Claims
1. A system for displaying ndimensional images comprising: means for storing a wavelet coded representation of an ndimensional image in the form of blocks of coded image data relating to different wavelet resolutions; means for retrieving image data stored as blocks from different wavelet resolutions, and means for reconstruction of a part of the ndimensional image by partial inverse wavelet transform on the retrieved data.
2. The system according to claim 1, wherein the blocks of coded image data comprise coefficients of the wavelet transform of the ndimensional image.
3. The system according to claim 1 or 2, wherein each block of coded image data is stored associated with coordinates of a related part of the ndimensional image.
4. The system according to any of the previous claims wherein each block of coded image data is stored associated with the relevant resolution level.
5. The system according to any of the previous claims further comprising means for displaying the reconstructed part of the ndimensional image.
6. The system according to any of the previous claims further comprising means for modifying the image data associated with at least one block of the coded ndimensional image.
7. The system according to claim 6, wherein the means for modifying includes a means for executing a partial forward wavelet transform.
8. The system according to any previous claim wherein the means for reconstructing includes a caching memory means.
9. The system according to any of the previous claims, further comprising a network connecting the means for storing and the means for reconstructing.
10. The system according to any previous claim wherein the ndimensional images have infinite resolution.
11. The system according to any previous claim, wherein the system is a computer based system.
12. A method for displaying ndimensional images comprising: storing a wavelet coded representation of a ndimensional image in the form of blocks of coded image data relating to different wavelet resolutions; retrieving image data stored as blocks from different wavelet resolutions, and reconstructing of a part of the ndimensional image by partial inverse wavelet transform on the retrieved data.
13. The method according to claim 12, wherein the blocks of coded image data comprise coefficients of the wavelet transform of the ndimensional image.
14. The method according to claim 12 or 13, wherein each block of coded image data is stored associated with coordinates of a related part of the ndimensional image.
15. The method according to any of claims 12 to 14, wherein each block of coded image data is stored associated with the relevant resolution level.
16. The method according to any of claims 12 to 15, further comprising displaying the reconstructed part of the ndimensional image.
17. The method according to any of claims 12 to 16 further comprising modifying the image data associated with at least one block of the coded ndimensional image.
18. The method according to claim 17, wherein modifying includes executing a partial forward wavelet transform.
19. A display device for displaying ndimensional images comprising: means for retrieving image data stored as blocks from different wavelet resolutions, and means for reconstruction of a part of the ndimensional image by partial inverse wavelet transform on the retrieved data.
20. An image modifying device for modifying a ndimensional image stored as blocks of coded image data relating to different wavelet resolutions comprising: means for generating modified coefficients for blocks of a coded image by applying a forward partial wavelet transform.
21. A software product which implements a system according to any of claims 1 to 11, a method according to any of claims 12 to 17, a display device according to claim 19 or an image modifying device according to claim 20 when executed on processing engine.
22. A machine readable data carrier storing the software product of claim 21.
Description:
SYSTEM FOR MAINTAINING AND BROWSING VERY LARGE IMAGES The present invention relates to a system and a method for maintaining and browsing very large images as well as a software tool for implementing the system.

Technical Background The JPEG-2000 standard specifies a file-format that is used to represent images into a compact form. The standard uses state of the art image compression techniques (EBCOT algorithm, etc. ). However, the JPEG-2000 standard doesn't handle unlimited resolution images, i. e. those having a value at any position within the image, and doesn't specify how to browse or modify very large images in their compressed form.

The Kakadu system was created by one of the authors of the JPEG-2000 standard. The Kakadu system offers an efficient way to navigate into images stored into the JPEG- 2000 format. This system doesn't manage local modifications of images in their compressed form, nor does it handle representation of unlimited-resolution images, i. e. those having a value at any position within the image. Moreover, because the images are stored into a single sequential file, it's not feasible to manipulate efficiently images that are truly very large.

The Terraserver system for Microsoft Corp. USA is a system that is used to visualize very large images. The system uses pyramids of images at different resolutions to allow efficient visualization at different levels of detail. The images are divided in blocks that are stored in a relational database. The method used to represent the images is not optimal because of the redundancy introduced by the image pyramids. What is more, the traffic of information between the server that contains the images and the visualization clients is far from minimal (redundancy, no client-side caching, etc.).

Summary of the Present invention An object of the present invention is to provide a system and method for maintaining and browsing very large images as well as a software tool for implementing the system.

The present invention provides a system for displaying n-dimensional images, n being at least 2 and for example 3 or 4 or more, comprising:

means for storing a wavelet coded representation of an n-dimensional image in the form of blocks of coded image data relating to different wavelet resolutions; means for retrieving image data stored as blocks from different wavelet resolutions, and means for reconstruction of a part of the n-dimensional image by partial inverse wavelet transform on the retrieved data. The blocks of coded image data may comprise coefficients of the wavelet transform of the n-dimensional image. Each block of coded image data may be stored associated with coordinates of a related part of the n-dimensional image. This allows only parts of the image of interest to be retrieved.

Each block of coded image data may be stored associated with the relevant resolution level. This allows the required resolution to be selected so that unnecessary data does not have to be fetched. Means for displaying the reconstructed part of the n-dimensional image may be provided. Means for modifying the image data associated with at least one block of the coded n-dimensional image may be provided. This allows images to be updated easily and locally without replacing the complete image.

The means for modifying may include a means for executing a partial forward wavelet transform. The means for reconstructing may include a caching memory means. This allows data locality to be exploited and to thereby reduce the amount of data transmitted over a network. Hence, a network may be provided for connecting the means for storing and the means for reconstructing.

The present invention also provides a display device for displaying n-dimensional images, n being at least 2 and for example 3 or 4 or more, comprising: means for retrieving image data stored as blocks from different wavelet resolutions, and means for reconstruction of a part of the n-dimensional image by partial inverse wavelet transform on the retrieved data.

The present invention also provides an image modifying device for modifying an n-dimensional image stored as blocks of coded image data relating to different wavelet resolutions comprising: means for generating modified coefficients for blocks of a coded image by applying a forward partial wavelet transform.

The present invention also provides a software product which implements a any of the systems, the methods, a display device or an image modifying device

according to the present invention when executed on processing engine. The present invention also includes a machine readable data carrier storing the software product, e. g. on an optical disc, in a solid state memory, a magnetic disk such as a hard drive, a tape storage device, etc.

The present invention will now be described with reference to the following drawings.

Brief Description of the Drawings Fig. 1 is a schematic block diagram of a system according to an embodiment of the present invention.

Fig. 2 is a schematic representation of manipulation and storage of image data in accordance with an embodiment of the present invention.

Fig. 3 is a representation of how a part of an image is reconstructed by data from different levels of a wavelet transform.

Fig. 4 is a representation of how a part of an image is reconstructed in a progressive manner.

Fig. 5 is a block diagram of a computer system according to an embodiment of the present invention.

Description of the illustrative embodiments The present invention will be described with respect to particular embodiments and with reference to certain drawings but the invention is not limited thereto but only by the claims. The drawings described are only schematic and are non-limiting. In the drawings, the size of some of the elements may be exaggerated and not drawn on scale for illustrative purposes. Where an indefinite or definite article is used when referring to a singular noun e. g."a"or"an","the", this includes a plural of that noun unless something else is specifically stated.

The present invention provides a system and a tool for implementing efficient browsing and manipulation of very large images that have unlimited resolution, i. e. those having a value at any position within the image rather than only having discrete values a certain positions, e. g. in an array. In one aspect system comprises several agents whose behavior is described below.

Fig. 1 shows a schematic representation of a system in accordance with an embodiment of the present invention. The system is made of at least two types of agents: storage and visualization agents 2,3. A third type of agent, a modification agent 4 may be provided if images are to be modified. The storage agents 2 can be used to store very large images that have unlimited resolution, visualization agents 3 are used to browse these images and modification agents 4 are used to perform real- time modification of such images. The three agents 2,3, 4 are connected by any form of network 1, for example a telecommunications network such as the Internet, a Local Area Network, a Wide Area Network, a Personal Area network, the busses of a personal computer, an infra-red, Bluetooth or Wireless network.

Storage agents Storage agents 2 are used to store representations of very large unlimited- resolution images, e. g. a world map 12. Such images are stored by means of subband coding such as the wavelet decomposition of these images. Because the system handles images with unlimited resolution, their decomposition can have an unlimited number of frequency bands or resolution levels. Each band is divided in small fixed- size blocks that contain coefficients resulting from the wavelet decomposition. The sequence of coefficients in each block is entropy-coded using an existing conventional compression technique. For each block, compressed data is obtained that describes the original coefficients of the block.

As shown in Fig. 2, blocks that correspond to the same spatial region of the image 12 in different resolution levels of the coded image 13 are grouped into macro- blocks. One can look at the macro-blocks as the smallest quanta of information about the images which is to be stored and manipulated. For example, macro-blocks that contain relevant information (i. e. at least one non-zero coefficient of the coded image portion) are preferably stored in an associative memory 5. The coefficients are preferably entropy coded to compress the coefficients further. The associative memory 5 stores a link between the regional area of the image and the coefficients of the coded image portion-see table 1. Hence, in this memory 6, the access and storage keys are the coordinates X, Y... of the macro-blocks in the coded image 13 in a suitable coordinate system, e. g. Cartesian, as well as the resolution level R and the corresponding values are the entropy-coded data of the macro-blocks. A means 6 for

accessing the associative memory is provided, i. e. to retrieve the relevant coefficients using as input the coordinates of a region of the image. These access means may be in the form of a user interface 6 for both reading and writing to the associative memory 5.

The associative memory may be any suitable data storage device such as solid state storage, RAM, non-volatile or volatile memory, tape storage, hard disc, an optical disk such as a CD-ROM or a DVD ROM, etc. The images may be n-dimensional, i. e. at least two-dimensional, for example four dimensional whereby the fourth dimension is time.

For two dimensional images, macro-blocks may be composed of three blocks.

For two dimensional images, the coordinates of a macro-block are: the resolution of coefficients it contains (R), and the spatial coordinates of the region it describes (X, Y).

The values corresponding to the keys in the associative memory are the data of the macro-blocks (this data is the entropy-coded sequence of the coefficients of their blocks).

Storage agents may contain information representing one or more images.

Storage agents are capable of containing images that have unlimited resolution. The representation of images are chosen so as to permit modification of the images at any resolution. Preferably, the representation of images is as compact as possible. This way, the system can store a maximum quantity of relevant information for a given available physical memory space. Moreover, it's useful to avoid introducing redundancies in the representation of images to facilitate the maintenance of the stored images. The compact representation of images is entirely reversible, meaning no details are lost when storing the images in compressed form.

The system allows handling images that contain a very large quantity of information. This quantity is limited only by the amount of physical memory available to a storage agent. Therefore storing images in single files is avoided. A relational database management system can be used to provide the organization of the associative memory. The relational database management system may be implemented in software and may run on a processing device such as a personal computer, workstation, server, etc. It easy is easy to perform optimized accesses to the associative memory when the data is stored in a relational database. For example, the accesses can be implemented using simple SQL statements. A relational database management system also offers an efficient means of managing great quantities of

data. A representation for images is chosen that is compatible with storage in a relational database management system for example, so as to efficiently take advantage of the entire available amount of memory.

The workload assigned to storage agents is minimal. Means 6 can be provided so that many visualization and modification agents are allowed to interact with a single storage agent. This can be done without overloading it, e. g. when the storage agent is located on a server and access is provided over a data telecommunications network such as a Local Area network, a Wide Area Network, or over the Internet or similar.

Storage agents are implemented as servers that can handle a great number of simultaneous connections to various visualization and modification agents.

Visualization agents Visualization agents 3 are used to browse into very large unlimited-resolution images contained in storage agents. The visualization agents use the accessing means 6 to retrieve the coefficients for an area of the image 12. A visualization agent 3 preferably has a user-friendly interface 9 allowing continuous and fluid exploration of the image in all its details. The physical memory needs of visualization agents are low and are independent of the browsed image and even independent of the size of the explored region. For example, visualization agents can be implemented that consume no more than 64KB working memory at all circumstances and for all display resolutions.

Visualization agents that reconstruct a region of an unlimited resolution image at a given resolution level need to perform a partial inverse wavelet transform. Hence, the visualization agent has a means 8 for performing the inverse partial wavelet transform. This is because, in the storage agents, the very large images are stored as if a single wavelet transform had been applied on them. This is unlike other systems that partition the original images and then compress and store the resulting smaller images using commonly used techniques.

The computation power needed by visualization agents stays very low. Needed computation power is proportional only to the resolution of the display used by the visualization agent and is independent of the particular image viewed. A visualization agent has been implemented on the PalmOS (tm) platform that runs on a 33MHz processor with a display resolution of 320x320. The user interface 9 offered by the

visualization agent is as user-friendly as possible. It allows the user to browse very large images that have unlimited resolution in an intuitive and fluid way. The visualization agent developed for the PaImOS (tm) platform allows browsing in TeraByte images in an intuitive and continuous way.

Partial inverse wavelet transform To reconstruct the region of an image at a given resolution, a visualization agent carries out information requests to a storage agent that contains a representation of the image. The storage agent has an associative memory in which macro-blocks of a wavelet decomposition are stored. To visualize a portion of an image, a visualization agent must therefore apply a partial inverse wavelet transform. The macro-block requests to the storage agent are determined by the needs of information occurring when applying a partial inverse wavelet transform. This transform allows the visualization agent to reconstruct any region at any resolution of very large image with unlimited resolution. By using a partial inverse wavelet transform, visualization agents can efficiently reconstruct any region of an image at any wanted resolution by carrying out appropriate requests for macro-blocks stored into an associative memory. The macro-block requests to the storage agent are determined by the needs of information occurring when applying a partial inverse wavelet transform. This transform allows the visualization agent to reconstruct any region at any resolution of very large image with unlimited resolution.

For example, in a practical implementation reversible integer 5-3 wavelets are used for the decomposition of images. The shape of the basis functions used by the partial inverse wavelet transform implies that a bilinear interpolation in zones is obtained where the storage agent doesn't contain any information about the image.

This is the case when browsing the image at resolutions for which no detail is known.

An efficient way of performing wavelet transforms is to factor the transforms into lifting steps. This technique is commonly used to perform transforms on images and is described in detail in I. Daubechies and W. Sweldens, Factoring Wavelet Transforms into Lifting Steps, J. Fourier Anal. Appl. For example, this can be used in an open-source J2000 implementation of the JPEG-2000 standard (j2000. org). The system uses this factoring into lifting steps in a novel way to implement partial wavelet transforms. These partial transforms are used to reconstruct any region at any

level of detail of unlimited resolution images so as to be able to display these regions on fixed-resolution displays.

As with conventional inverse wavelet transforms, the image samples at a given resolution are reconstructed by iteratively applying a merging procedure. Given a set of samples that represent the image at a given resolution, and a set of coefficients from frequency bands, the merging procedure constructs samples that represent the image at an higher resolution. One big difference between partial inverse transforms and regular inverse transforms, is that the number and coordinates of the coefficients from the needed frequency bands is not known in advance and depends largely on the specific region that is to be reconstructed.

To be specific, the following code shows precisely how, in a system according to the present inventio, samples from a given resolution are constructed from samples at a lower resolution and additional coefficients that represent the details that make it possible to construct an higher resolution approximation from a lower resolution one (Fig. 3). #define LL (ij) LL_ #define HL #define LH (i, LH lu LH #define #define void dwtmerge LLw, short *LH, short bug long ! lw_, ; long axl, ayl, aw, aw2 ; mtij ; bng short *Ap, *LLp, *HLp, *LHp. ; hxO=xO>>l ;

hy0=y0 ; hxl ; hyl=yl>>l ; lx0=hx0 ; ly0=hy0 ; lxl lyl-hylv- ; hxO_-lxO_-X ; hy0= hxl_-lxl_ ; hyl_-lyl_ ; Iw_-lxl_-lxO_ ; hw==hxl-hx0 axO= (hxO_-« ayO axl=hxl_< ayl===hyl aw=axl-axO aw2=aw<<l for (j=ly0 ; j<lyl y==j « Ap=&A {(hxO_ ; HLp==&HL for *Ap Apq Ap=&A y)-l LLp=&LL for ; i--) *

Ap+=2 Ap=&A for ! =O ; i--) * ; Ap+=7 for O : zz : hyC_ ; j- : hyl Y=O<<'),-, y= Ap HHp=&HH ; forti=hxl_-hxO_ *Ap-*HHp9-+ Ap+=2 ; Ap=tAXlxO_ LHp=&LHIlxO_ ; fo * Ap+=2 } Ap ; foy ; i ; i) * Ap+=2 ; } t for ; xxl ; x++) Ap=&A ; for (j=lyl-ly0 * (ApQ ; Ap+=aw2

[(i) } Ap=&A(x, (hy0#1)+1)-aw; for (j=hyl-hyO ; j t=o ; j) F * (Ap+aw)+=(*Ap+*(Ap+aw2))#1; Ap+=aw2 ; <BR> <BR> <BR> }<BR> <BR> <BR> <BR> t<BR> } The partial inverse wavelet transforms used in the system are thus developed with the goal of reconstructing an unlimited resolution image from an infinite set of coefficients resulting from its wavelet decomposition. Of course, only a finite subset of these coefficients are needed to display a given region of an unlimited resolution image at given level of detail. This is because the devices used to visualize the unlimited resolution images have themselves only displays with limited resolution.

Therefore, the partial transforms used in a system according to the present invention are not the same as the"progressive transforms"that are commonly used in microelectronic systems to perform a regular wavelet transform in such a way that memory accesses are optimized.

The previous code shows how the merging procedure is implemented in the case of a 5-3 inverse wavelet transform. This merging procedure can be extended in a straight foward manner to handle transforms with additional lifting steps, like the 9-7 wavelet transform. With reference to Fig. 3 the reconstruction of part of image 12 is shown. The image 12 is stored as a multiresolutional image 13, having a plurality of wavelet coded levels: LL, HL, HH, LH. In the algorithm above, the ZZ parameter is a pointer to a buffer that contains a region of an image at a given resolution level. The HL, LH and HH parameters designate buffers that contain coefficients that can be used to reconstruct a higher resolution approximation of the image from the low resolution approximation contained in LL_. The A-parameter is a buffer that will contain the higher resolution approximation once it is constructed. The xO, yO, xl, and yl parameters are used to describe exactly which region of the higher resolution approximation is to be reconstructed. Finally, the LLxO and LLyO parameters are used to point the coordinates of the first low-resolution sample contained in the LL_ buffer.

Fig. 3 illustrates the process of applying the merge procedure to reconstruct an higher resolution approximation of an image from a lower-resolution approximation and from coefficients from the frequency bands. The originality here is to allow only a partial reconstruction of the higher resolution approximation limited only to a particular region of interest.

The code described above is a novel implementation of a partial inverse wavelet transform. Compared to conventional wavelet transforms that operate on whole images of fixed resolution, the main part of the problem solved by the algorithm is to handle the intricate dependencies between coefficients that arise when reconstructing only a specific region of an image.

The rendering of the display may be done in a progressive manner as shown schematically in Fg. 4. Because the system is capable of reconstructing efficiently any region of an image at any resolution, the display is rendered block by block, which is a way to minimize memory needs and a way of making the rendering operations more cache-friendly by exploiting the locality of the memory accesses. It also makes the browsing experience more pleasing for the user when browsing images through a low- bandwidth connection, because it's then possible to follow the progress of the rendering process. Further the resolution of each block displayed may be altered by fetching more blocks from different resolution levels. This allows the visualisation of a low resolution images followed by increasing levels of resolution in a smooth manner.

The combination of these two methods in accordance with the present invention means that a large image can be traversed easily without abrupt changes as well as the ability to zoom in or out at will.

Caching system A caching system 7 is preferably used to minimize accesses to the associative memory. Before being sent to a storage agent, the macro-block requests are preferably

processed by a caching system 7. This caching system memorizes the macro-blocks that are most likely to be asked for, so as to have them directly at disposition for future requests. This is a way to minimize the bandwidth usage between visualization agents and storage agents.

Modification agents Modification agents 4 are used to make modifications to very large unlimited- resolution images contained in storage agents. By using a partial forward wavelet transform, modification agents are capable of efficiently modifying any region of images at any wanted resolution by performing appropriate requests and updates to an associative memory. Hence, modification agents have a means 10 for carrying out a partial forward wavelet transform. Modification agents can also be used to create the coded image-effectively the creation of a coded image is an initial modification of a blank space. Modification agents allow users to perform real-time modification of very large images that have infinite resolution. Modifications can be made at any region and resolution of the images contained in a storage agent.

The memory needs of modification agents are only proportional to the number of samples submitted by a user to describe the modification. Memory consumption is thus independent of the particular image modified.

The computation power needed by a modification agent to carry out a modification also only depends on the number of samples submitted and not on the particular image that is modified. This permits implementation of modification agents on a very wide range of devices whose computation power only varies by the size of the modifications it can carry out and by the time constraints on the execution of the modifications.

The bandwidth used between modification and storage agents is minimal, thereby allowing to efficiently carry out large scale modifications remotely and using only low-bandwidth connections.

Partial forward wavelet transform: Unlimited resolution images are stored using their wavelet decomposition on storage agents. Therefore, when a region of an image is modified, a partial forward wavelet transform can be performed to update coefficients of the wavelet

decomposition contained in the storage agent. When samples are submitted to a modification agent by a user, macro-blocks requests and update are thus performed on a storage agent. The modification agent communicates with the user through a user interface 11. The goal of a modification agent is to carry out macro-block updates in the most efficient way so as to make any given modification effective.

The algorithm used to implement the partial forward wavelet transforms used in the modification agents is in many ways similar to the algorithm used for the partial inverse transform desribed previously. The goal of the partial forward wavelet transforms is to be able to efficiently modify any region of any resolution of unlimited-resolution images contained in storage agents. The partial forward wavelet transforms may be implemented as follows: Knowing the resolution and the spatial region of the modification to be performed, the set of all coefficients to be modified is computed for each inferior resolution, using exactly the same method that is used in the case of the partial inverse wavelet transforms.

Once the set of coefficients that need to be modified is known, a set of samples surrounding the region to be modified is computed using the partial inverse wavelet transform. These are the samples that are needed to compute the set of coefficients to be modified, but that are not included in the region to be modified.

For each resolution, the set of modified coefficients is computed using a wavelet splitting step that uses both new samples submitted for insertion and a set of surrounding samples computed in step 2.

The partial forward wavelet transforms used by modification agents result in a series of"GET"queries and"PUT"updates to a storage agent that result in the efficient modification of any region at any resolution of the data contained in the storage agent.

System Operation By storing the image in an associative memory and by using partial wavelet transforms, the system is able to handle images that have infinite resolution. The

storage of an image is limited only by the effective quantity of information submitted and not by overall characteristics of the image like its dimensions.

In accordance with one aspect of the present invention a relational database management system, for example, may be used to provide an associative memory.

This allows efficient storing of images that are very large and that couldn't be handled efficiently if stored in single files.

The only operations assigned to storage agents are the reading and writing of macro-blocs in an associative memory. Therefore, many visualization and modification agents can interact at the same time with a single storage agent without surcharging it.

The information retrieved by a visualization agent is only the coefficients required to visualize an image. The information sent by a modification agent is only the coefficients required to modify a portion of an image. If the visualization and/or modification agents are remote, they may communicate with a storage agent through a network. The bandwidth used on the network may be kept minimal due to the small amount of data which has to be sent. The compact representation of images used by the system and a macro-block caching system assure minimal bandwidth consumption.

The system allows performing real-time modifications of images, when these are stored in their compressed form. The complexity of the operations carried out by a modification agent only depends of the number of samples used to describe the modification and is independent of overall characteristics of the image.

The complexity of operations carried out by a visualization agent only depends on the number of samples used for displaying the image. This complexity is independent of global characteristics of the image like it's dimensions or number of resolutions. Therefore, it's possible to implement visualization and modification agents on a wide range of devices with very different computation power and display capabilities (PDA's, cell-phones, desktops, Tablet PC's, etc.).

Hence a client/server architecture (see Fig. 1) can be used for the implementation of the system. Storage agents are implemented on servers whereas visualization and modification agents are implemented on a wide range of devices (PDAs, cell-phones, desktops, Tablet PC's, etc. ). The bandwidth used between visualization agents and storage agents is minimal. This allows browsing images remotely using only very low bandwidth transmissions.

Advantages of a system according to the present invention can be: 'Efficiency of the image representation used on storage agents (compactness, scalability, etc.).

Minimal workload on storage agents, allowing many simultaneous interactions with visualization and modification agents.

Minimal bandwidth consumption between storage, visualization and modification agents.

Very low computation power and memory needs for visualization and modification agents.

. Intuitive and fluid browsing experience for users of visualization agents The efficiency of the system is due to the use of partial wavelet transforms for manipulating and visualizing very large images that have infinite resolution. It is also due to the storage of information resulting from wavelet decompositions into an associative memory (for example a relational database management system).

A system in accordance with the present invention can be of particular interest for storing, visualizing and modifying three-dimensional volumetric data. The large quantity of information manipulated by applications working with volumetric data would be advantageously processed and displayed in accordance with the present invention. The partial inverse and forward wavelet transforms introduced in the system can be extended to three dimensions for use in applications manipulating volumetric data. Hence, the system can be used effectively in numerous other applications: medical imagery, satellite imagery, etc.

The storage agents 2 visualization agents 3 and modification agents 4 can be implemented in software and can be compiled to run on suitable processing engines with memory such as a microprocessor with associated memory. Fig. 5 is a schematic representation of a computing system 30 which can be utilized with the methods and in a system according to the present invention. The computer system may comprise a server which is in communication with a remote device 21 which may have a lower procesing capacity an memory capability than the server. The images are displayed on the display device 21 and the image data is stored on the server. However, the remote device 21 may also be a powerful computer such as a workstation for displaying

medical images and the medical images may be stored on the server for retrieval over a network.

A computer system 30 is depicted which may include a video display terminal 14, a data input means such as a keyboard 16, and a graphic user interface indicating means such as a mouse 18. Computer 10 may be implemented as a general purpose computer, e. g. a personal computer, a laptop, a UNIX workstation, a server on a telecommunications network.

Computer system 30 includes a Central Processing Unit ("CPU") 15, such as a conventional microprocessor of which a Pentium IV processor supplied by Intel Corp.

USA is only an example, and a number of other units interconnected via system bus 22. The computer system 30 includes at least one memory. Memory may include any of a variety of data storage devices known to the skilled person such as random-access memory ("RAM"), read-only memory ("ROM"), non-volatile read/write memory such as a hard disc as known to the skilled person. For example, computer system 30 may further include random-access memory ("RAM") 24, read-only memory ("ROM") 26, as well as an optional display adapter 27 for connecting system bus 22 to an optional video display terminal 14, and an optional input/output (I/O) adapter 29 for connecting peripheral devices (e. g. , disk and tape drives 23) to system bus 22. Video display terminal 14 can be the visual output of computer system 30, which can be any suitable display device such as a CRT-based video display well-known in the art of computer hardware. However, with a portable or notebook-based computer, video display terminal 14 can be replaced with a LCD-based or a gas plasma-based flat-panel display. Computer system 30 further includes user interface adapter 19 for connecting a keyboard 16, mouse 18, optional speaker 36. A display device 21 including a processing engine and memory such as a PDA, a laptop, a personal computer, a work station, a smartphone, etc. is connected via a network 41 to an input 39 of the computer system 30, e. g. a modem or similar. Network 41 may be a data network such as the Internet, an Intranet a Local or Wide Area network (LAN or WAN) or a CAN. This allows transmission of image data over a telecommunications network. The processing engine of the display device 21 in combination with memory implements a method in accordance with the present invention, for example the visualisation agent 3, the caching system 7 and the user interface 9.

Computer system 30 also includes a graphical user interface that resides within machine-readable media to direct the operation of computer system 30. It can implement the interface 6 for example as well as the modification agent 4, interface 11 ad partial wavelet transform means 10. Alternatively or additionally, the modification agent 4 may be implemented on display device 21. The computer system implements the management of the associative memeory 5. Any suitable machine-readable media may retain the graphical user interface, such as a random access memory (RAM) 24, a read-only memory (ROM) 26, a magnetic diskette, magnetic tape, or optical disk (the last three being located in disk and tape drives 23). Any suitable operating system and associated graphical user interface (e. g. , Microsoft Windows) may direct CPU 15. In addition, computer system"0 includes a control program 51 which resides within computer memory storage 52. Control program 51 contains instructions that when executed on CPU 15 carry out the operations described with respect to any of the methods of the present invention.

Those skilled in the art will appreciate that the hardware represented in Fig. 5 may vary for specific applications. For example, other peripheral devices such as optical disk media, audio adapters, or chip programming devices, such as PAL or EPROM programming devices well-known in the art of computer hardware, and the like may be utilized in addition to or in place of the hardware already described.

In the example depicted in Fig. 5, the computer program product (i. e. control program 51) can reside in computer storage 52. However, it is important that while the present invention has been, and will continue to be, that those skilled in the art will appreciate that the mechanisms of the present invention are capable of being distributed as a program product in a variety of forms, and that the present invention applies equally regardless of the particular type of signal bearing media used to actually carry out the distribution. Examples of computer readable signal bearing media include: recordable type media such as floppy disks and CD ROMs and transmission type media such as digital and analogue communication links.