Copyright Notice:

[0001] A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. Field of Invention:

[0002] The present invention is generally related to computer systems, and is particularly related to a distributed data grid.

Background:

[0003] Modern computing systems, particularly those employed by larger organizations and enterprises, continue to increase in size and complexity. Particularly, in areas such as Internet applications, there is an expectation that millions of users should be able to simultaneously access that application, which effectively leads to an exponential increase in the amount of content generated and consumed by users, and transactions involving that content. Such activity also results in a corresponding increase in the number of transaction calls to databases and metadata stores, which have a limited capacity to accommodate that demand.

[0004] This is the general area that embodiments of the invention are intended to address. Summary:

[0005] Described herein are systems and methods that can support metadata compression in a distributed data grid. The system can store a plurality of keys in compact data structure (such as a radix tree) in a heap, wherein each said key is associated with entry in an off-heap journal. Furthermore, the system can look up the tree for a key, wherein the key is associated with a journal ticket that represents a virtual address of an entry in the journal. Then, the system can use the journal ticket to manage the entry in the journal that is associated with said key.

Brief Description of the Figures:

[0006] Figure 1 is an illustration of a data grid cluster in accordance with various embodiments of the invention.

[0007] Figure 2 shows an illustration of supporting an elastic data structure in a distributed data grid, in accordance with an embodiment of the invention.

[0008] Figure 3 shows an illustration of storing compressed keys on-heap in a distributed data grid, in accordance with an embodiment of the invention. [0009] Figure 4 shows an illustration of supporting a journal ticket in a distributed data grid, in accordance with an embodiment of the invention.

[0010] Figure 5 shows an illustration of supporting an elastic data ticket tree in a distributed data grid, in accordance with an embodiment of the invention.

[0011] Figure 6 shows an illustration of supporting elastic data metadata compression in a distributed data grid, in accordance with an embodiment of the invention.

[0012] Figure 7 illustrates an exemplary flow chart for supporting elastic data metadata compression in a distributed data grid, in accordance with an embodiment of the invention.

[0013] Figure 8 illustrates a simplified structure diagram of an exemplary serverfor supporting metadata compression in a distributed data grid, in which various embodiments of the present invention may be implemented.

[0014] Figure 9 illustrates a system in accordance with an embodiment of the invention.

[0015] Figure 10 illustrates a computer system that can be used to implement embodiments of the invention.

Detailed Description:

[0016] The invention is illustrated, by way of example and not by way of limitation, in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to "an" or "one" or "some" embodiment(s) in this disclosure are not necessarily to the same embodiment, and such references mean at least one.

[0017] The description of the invention as following uses the Oracle Coherence data grid as an example for a distributed data grid. It will be apparent to those skilled in the art that other types of distributed data grids can be used without limitation.

[0018] Described herein are systems and methods that can support elastic data metadata compression in a distributed grid.

Distributed Data Grid

[0019] In accordance with an embodiment, as referred to herein a "distributed data grid", "data grid cluster", or "data grid", is a system comprising a plurality of computer servers which work together to manage information and related operations, such as computations, within a distributed or clustered environment. The data grid cluster can be used to manage application objects and data that are shared across the servers. Preferably, a data grid cluster should have low response time, high throughput, predictable scalability, continuous availability and information reliability. As a result of these capabilities, data grid clusters are well suited for use in computational intensive, stateful middle-tier applications. Some examples of data grid clusters, e.g., the Oracle Coherence data grid cluster, can store the information in-memory to achieve higher performance, and can employ redundancy in keeping copies of that information synchronized across multiple servers, thus ensuring resiliency of the system and the availability of the data in the event of server failure. For example, Coherence provides replicated and distributed (partitioned) data management and caching services on top of a reliable, highly scalable peer-to-peer clustering protocol.

[0020] An in-memory data grid can provide the data storage and management capabilities by distributing data over a number of servers working together. The data grid can be middleware that runs in the same tier as an application server or within an application server. It can provide management and processing of data and can also push the processing to where the data is located in the grid. In addition, the in-memory data grid can eliminate single points of failure by automatically and transparently failing over and redistributing its clustered data management services when a server becomes inoperative or is disconnected from the network. When a new server is added, or when a failed server is restarted, it can automaticallyjoin the cluster and services can be failed back over to it, transparently redistributing the cluster load. The data grid can also include network-level fault tolerance features and transparent soft re-start capability.

[0021] In accordance with an embodiment, the functionality of a data grid cluster is based on using different cluster services. The cluster services can include root cluster services, partitioned cache services, and proxy services. Within the data grid cluster, each cluster node can participate in a number of cluster services, both in terms of providing and consuming the cluster services. Each cluster service has a service name that uniquely identifies the service within the data grid cluster, and a service type, which defines what the cluster service can do. Otherthan the root cluster service running on each cluster node in the data grid cluster, there may be multiple named instances of each service type. The services can be either configured by the user, or provided by the data grid cluster as a default set of services.

[0022] Figure 1 is an illustration of a data grid cluster in accordance with various embodiments of the invention. As shown in Figure 1 , a data grid cluster 100, e.g. an Oracle Coherence data grid, includes a plurality of cluster nodes 101-106 having various cluster services 1 1 1 -1 16 running thereon. Additionally, a cache configuration file 1 10 can be used to configure the data grid cluster 100.

Elastic Data Structure

[0023] In accordance with an embodiment of the invention, a data grid cluster can use an elastic data structure to seamlessly store data across memory and disk-based devices. Furthermore, the elastic data structure can use a journaling technique to optimize the data storage across memory and disk.

[0024] Figure 2 shows an illustration of supporting an elastic data structure in a distributed data grid, in accordance with an embodiment of the invention. As shown in Figure 2, a data grid cluster 200 can use different types of journals 201-202 for recording state changes associated with a sequence of modifications on data storage. For example, the data storage can include both a random access memory (RAM) and various disk-based devices such as solid state disks (SSDs).

[0025] In accordance with an embodiment of the invention, the data grid cluster 200 can take advantage of an elastic data structure, which enables the data grid cluster 200 to store data in the SSDs and/or read data from the SSDs at a near memory speed.

[0026] As shown in Figure 2, the data grid cluster 200 can use a RAM journal 201 for storing data in-memory and can use a flash journal 202 for storing data to the flash-based devices. Additionally, the RAM journal 201 can work with the flash journal 202 to enable seamless data overflow from the RAM storage to the flash disk storage.

[0027] Furthermore, each of the journals 201 -202 used by the data grid cluster 200 can contain multiple entries. For example, the RAM journal 201 can include RAM journal entries 21 1-219, while the flash journal 202 can include flash journal entries 221 -229.

[0028] In accordance with an embodiment of the invention, the system can associate the value stored in each different journal entries 21 1-219 and 221-229 with a different key 220. As the changes are applied to the elastic data storage, each specific key 220 can remain associated with a journal entry 21 1-219 or 221 -229 that contains the latest value.

[0029] Also, stale values may accumulate in the RAM journal 201 and the flash journal 202, since new values may be written for different keys 220 and previous changes in the elastic data structure 200 may become obsolete. The system can evacuate these stale values, e.g. at regular intervals, in order to make room for new values to be written in the journals 201-202.

[0030] In accordance with an embodiment of the invention, the system can maintain a compact data structure in memory (e.g. using a tree structure on a JVM heap 210), for keeping track of which journal entry contains the current value for a particular key. Thus, a user can find a value stored in a journal entry in the elastic data structure 200, by first looking up for a key 220 stored on the JVM heap 210.

[0031] In accordance with an embodiment of the invention, caches that use the RAM journal 201 and the flash journal 202 can be configured as part of a cache scheme definition within a cache configuration file. Additionally, the system allows a user to configure the journaling behavior by overriding the out-of-box configuration.

[0032] Also, the RAM journal 201 and the flash journal 202 can be used in different manners. For example, the RAM journal 201 and the flash journal 202 can be used for supporting backing maps and backup storage in the data grid cluster 200. Furthermore, the RAM journal 201 and the flash journal 202 can be used for supporting composite caches (e.g. a near cache).

Metadata Compression

[0033] In accordance with an embodiment of the invention, the system can use a compact attribute storage for supporting various high-level caching features.

[0034] Figure 3 shows an illustration of storing compressed keys on-heap in a distributed data grid, in accordance with an embodiment of the invention. As shown in Figure 3, the distributed data grid 300 can use a radix-tree 310 for storing different keys 331 -335 on-heap, e.g. on a Java virtual machine (JVM) heap. [0035] For example, the keys 331-335 stored in the radix-tree 310 can include "Albert" 331 , "Alfred" 332, "Andrew" 333, "Antonio" 334 and "Antony" 335. As shown in Figure 3, the root node 301 for the radix-tree 310 stores a character "A", since all keys 331 -335 stored in the radix-tree 310 start with a character "A."

[0036] Furthermore, both the two keys, "Albert" 331 and "Alfred" 332, which contain a character "I," can share the same node 31 1. Additionally, the key "Albert" 331 uses the node 321 , which contains the characters "bert" and the key "Alfred" 332 uses the node 322, which contains the characters "fred." Thus, the radix-tree 310, which is a trie data structure, can be optimized in saving memory space on-heap, since any node in the radix-tree 300 with only one child can be merged with its child.

[0037] Similarly, the key "Andrew" 333 uses the nodes 301 , 312, and 323, while the key "Antonio" 334 uses the nodes 301 , 312, 324 and 325, and the key "Antony" 335 uses the nodes 301 , 312, 324 and 326.

[0038] In accordance with an embodiment of the invention, each compressed key 331-335 stored in the radix-tree 310 can correspond to a value stored in a journal entry off-heap. Additionally, in order to improve efficiency, tiny values (e.g. values with a size up to seven bytes) can be stored directly in the radix-tree 310, instead being stored off-heap.

[0039] Also, the compressed keys 331-335 stored in the radix-tree 310 can support the transparent overflow from a RAM journal to a flash journal when the RAM journal is full.

[0040] Figure 4 shows an illustration of supporting a journal ticket in a distributed data grid 400, in accordance with an embodiment of the invention. As shown in Figure 4, a compressed key 401 , which is stored on-heap (e.g. on a JVM heap 410) can be associated with a journal ticket 411. The journal ticket 411 can represent a virtual address for a value 412 that is stored off-heap, such as in an entry 402 of a journal 420 that corresponds to the key 401 . Additionally, the key 401 can represent additional attributes 413 that are associated with the value 412 stored off-heap (such as an eviction attribute and an expiry attribute).

[0041] Figure 5 shows an illustration of supporting an elastic data ticket tree in a distributed data grid, in accordance with an embodiment of the invention. As shown in Figure 5, an elastic data ticket tree 500 can be used to store compressed keys on-heap based on a radix-tree data structure. Furthermore, each compressed key stored in the elastic data ticket tree 500 can be associated with a journal ticket, or a virtual address for a value in a journal that corresponds to the key.

[0042] For example, the key, "Albert," which involves the nodes 501 , 51 1 , and 521 , can be associated with a journal ticket, "0xll2ca530". The key, "Alfred," which involves the nodes 501 , 51 1 , and 522, can be associated with a journal ticket, "0x38fce788". The key, "Andrew," which involves the nodes 501 , 512, and 523, can be associated with a journal ticket, "0xfd738d80". The key, "Antonio," which involves the nodes 501 , 512, 524 and 525, can be associated with a journal ticket, "0x3819e6c0". Also, the key, "Antony," which involves the nodes 501 , 512, 524 and 526, can be associated with a journal ticket, "0xb381efd0".

[0043] Figure 6 shows an illustration of supporting metadata compression in a distributed data grid, in accordance with an embodiment of the invention. As shown in Figure 6, a distributed data grid 600 can use a radix-tree 601 to store different compressed keys on-heap (e.g. a JVM heap). These compressed keys (i.e. a key-set) can include "Albert", "Alfred", "Andrew", "Antony" and "Antonio".

[0044] Furthermore, each compressed key stored in the radix-tree 601 can be associated with a journal ticket 622, which is stored separately on-heap (such as in a long storage array 602). Additionally, each journal ticket 622 stored in the long storage array 602 can be associated with a slot number 621 (or a row number).

[0045] As shown in Figure 6, the node 61 1 , which is used to store the key "Albert," includes a journal ticket, "0xll2ca530," stored at the slot 1 in the long storage array 602. The node 612, which is used to store the key "Alfred," includes a journal ticket, "0x38fce788," stored at the slot 2 in the long storage array 602. The node 613, which is used to store the key "Andrew," includes a journal ticket, "0xfd738d80," stored at the slot 3 in the long storage array 602. The node 614, which is used to store the key "Antonio," includes a journal ticket, "0x3819e6c0," stored at the slot 4 in the long storage array 602. Finally, the node 615, which is used to store the key "Antony," includes a journal ticket, "0xb381efd0," stored at the slot 5 in the long storage array 602.

[0046] Also as shown in Figure 6, the long storage array 602 can include additional columns 623 that encode other attributes, such as expiry, touch-time, units, index values, etc. The storage format for the long storage array 602 allows for an arbitrary number of additional columns 623.

[0047] Furthermore, these additional columns 623 can be dynamically added into long storage array 602, and the corresponding attributes can be dynamically associated with the key-set in the radix-tree 601 with near-zero fixed space overhead beyond the attribute encoding size. This can be beneficial, e.g. for large data sets, since the system, which can be implemented on the Java VM runtime, would require considerably more space overhead if other representations are used.

[0048] Additionally, the system can further compact multiple small attributes together into a single attribute 623, in order to further reduce the space overhead for the metadata. For example, the journal ticket 622 and each of the other attributes 623 can be a Java "long" values, which takes eight (8) bytes of space. The system can split a single eight-byte attribute into multiple smaller virtual attributes for storing more information.

[0049] Thus, by utilizing such a compact data structure, the system can store metadata information for very large datasets.

[0050] Figure 7 illustrates an exemplary flow chart for supporting elastic data metadata compression in a distributed data grid, in accordance with an embodiment of the invention. As shown in Figure 7, at step 701 , the system can store a plurality of keys in a tree in a virtual machine (VM) heap, wherein each said key is associated with an entry in a journal off-heap. Then, at step 702, the system can look up the tree for a key, wherein the key is associated with a journal ticket that represents a virtual address of an entry in the journal. Furthermore, at step 703, the system can use the journal ticket to manage an entry associated with said key.

[0051] Figure 8 illustrates a simplified structure diagram of an exemplary server 800 for supporting metadata compression in a distributed data grid, in which various embodiments of the present invention may be implemented. The units of the server 800 may be implemented by hardware, software, or a combination of hardware and software to carry out the principles of the invention. It is understood by persons of skill in the art that the units described in Figure 8 may be combined or separated into sub-units to implement the principles of the invention as described above. Therefore, the description herein may support any possible combination or separation or further definition of the functional units described herein.

[0052] As shown in Figure 8, the exemplary server 800 may comprise a first storing unit 801 , a looking up unit 802 and a management unit 803. The first storing unit 801 may be configured to store a plurality of keys in a tree in a compact data structure in a heap, wherein each said key is associated with an entry of a journal off-heap. The looking up unit 802 may be configured to look up the tree for a key, wherein the key is associated with a journal ticket that represents a virtual address of an entry in the journal. The management unit 803 may be configured to use the journal ticket to manage the entry in the journal that is associated with said key.

[0053] In one embodiment, the journal off-heap includes a RAM journal, and a flash journal that operates to transparently handle data overflow from the RAM journal.

[0054] In one embodiment, the heap is a virtual machine (VM) heap.

[0055] In one embodiment, said tree is a radix tree, wherein each key in the radix tree corresponds to a value.

[0056] In one embodiment, the exemplary server further comprises a second storing unit 804 configured to store a value corresponding to a key directly in the radix tree if the size of the value is small.

[0057] In one embodiment, the exemplary server further comprises a third storing unit 805 configured to store one or more journal tickets in the radix tree, wherein each said journal ticket is associated with a key.

[0058] In one embodiment, the exemplary server further comprises a fourth storing unit 806 configured to store one or more journal tickets in a long storage array, wherein each said journal ticket is associated with a key.

[0059] In one embodiment, the exemplary server further comprises an associating unit 807 configured to associate the key stored in the radix tree with a slot number, wherein the slot number indicates a location in the long storage array where said journal ticket stores, and wherein the long storage array stores additional attributes that are associated with the key.

[0060] In one embodiment, the exemplary server further comprises a fifth storing unit 808 configured to store additional attributes that are associated with the key in the long storage array. [0061] In one embodiment, the exemplary server further comprises a compacting unit 809 configured to compact multiple small attributes together into a single attribute stored in the long storage array.

[0062] The units described in Figure 8 may be combined or separated into sub-units to implement the principles of the invention as described above.

[0063] The server and the units thereof described in FIG. 8 can carry out various operations according to the principles of the invention as described in above embodiments according to actual needs.

[0064] Referring Figure 9, a system 900 is illustrated in accordance with an embodiments of the invention. Figure 9 shows an illustration of a functional configuration realized by system 900.

System 900 includes a store module 910, a heap 920, a look-up module 930, and a manager 940.

System 900 is, for example, implemented on a general-purpose computer.

[0065] Store module 910 is configured to store a plurality of keys in a tree in a compact data structure in heap 920. Each key is associated with an entry of a journal off-heap. Look-up module 930 is configured to look up the tree for a key. The key is associated with a journal ticket that represents a virtual address of an entry in the journal. Manager 940 is configured to use the journal ticket to manage the entry in the journal that is associated with the key.

[0066] Figure 10 shows an illustration of a computer system 1000 which includes well-known hardware elements. Namely, computer system 1000 includes a central processing unit (CPU) 1010, a mouse 1020, a keyboard 1030, a random access memory (RAM) 1040, a hard disc 1050, a disc drive 1060, a communication interface (l/F) 1070, and a monitor 1080.

[0067] In an aspect, store module 910, look-up module 930, and manager940 are implemented by CPU 1010. In a further aspect, more than one processors can be used so that store module 910, look-up module 930 and manager 940 is physically remote from each other. Heap 920 is implemented by RAM 1040.

[0068] In yet another aspect, system 900 can be implemented by using a plurality of circuits which function as store module 910, look-up moduel 930 or manager 940.

[0069] The present invention may be conveniently implemented using one or more conventional general purpose or specialized digital computer, computing device, machine, or microprocessor, including one or more processors, memory and/or computer readable storage media programmed according to the teachings of the present disclosure. Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art.

[0070] In some embodiments, the present invention includes a computer program product which is a storage medium or computer readable medium (media) having instructions stored thereon/in which can be used to program a computer to perform any of the processes of the present invention. The storage medium can include, but is not limited to, any type of disk including floppy disks, optical discs, DVD, CD-ROMs, microdrive, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data.

[0071 ] The foregoing description of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations will be apparent to the practitioner skilled in the art. The modification and variation include any relevant combination of the described features. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to understand the invention for various embodiments and with various modifications that are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalence.