TECHNICAL FIELD

[0001] The present invention relates to data backup and recovery, and more particularly to real time data backup and recovery based on blockchain technology.

BACKGROUND ART

[0002] A blockchain is a list of records, grouped into blocks, which are linked together using cryptography. Blockchain systems are used to maintain a reliable record of transactions by means of collective participation and consensus among participants. A blockchain can be understood as a distributed ledger technology, jointly maintained by multiple networked devices called nodes. A blockchain can thus be thought of as a distributed storage system.

[0003] Conventional storage systems utilizing database backup and recovery processes generally require substantial investment in redundant hardware and associated software. Moreover, the actual backup and recovery cycle, if executed, can typically be quite labor intensive. Accordingly, it is often expensive to set up a reliable system capable of real time backup and restoration of data, in an enterprise environment.

[0004] For real time data backup, conventional systems utilizing existing technology often requires the use of a powerful backup servers. However, the utilization rate of such powerful servers is often low, which translates to enormous waste of computing resources that have already been acquired.

[0005] If changes are made to the backup data in the backup server, then when the system is restored, the data in production database will be incorrect. There is thus a need for a reliable, secure, versatile, and inexpensive backup systems that meet the needs of various organizations.

[0006] One of the conventional ways to mitigate the problems associated with backup and restore operations is the use of incremental backup. During incremental backup, transaction logs are utilized to identify changes made since the last backup was taken, and only contents associated with changes that cannot be accounted for in the previous backup operation will be backed up in the next incremental backup procedure.

[0007] As a transaction log records only changes made to the database after a previous database backup or transaction log record backup, incremental backups only record database changes made during a limited period in between backup operations. Accordingly, a full database backup is required before undertaking a transaction record backup or an initial incremental backup.

[0008] There are several problems related to reliability, security and consistency of the aforementioned approaches.

[0009] In terms of reliability, there is the possibility that a central node of the backup system may fail. If there is a problem with the central node or machine associated with the backup transaction log, then data loss or damage may occur and the entire backup process may fail.

[0010] With regard to security and consistency, any unauthorized change to the transaction log could inevitably lead to recovered data that is not trustworthy.

[0011] Accordingly, there is a need for improved systems and methods to mitigate at least some of the aforementioned problems associated with backup and restore systems that utilize conventional hardware devices and technologies.

SUMMARY OF INVENTION

[0012] In accordance with one aspect of the present invention, there is provided a blockchain- based backup and restore system and method. Embodiments of the present invention include blockchain-based backup-and-restore systems that are secure, reliable, and capable of real time operation.

[0013] In accordance with one aspect of the present invention, there is provided a data backup and recovery system, for use with a data store and a blockchain including a plurality of nodes, the system including: a server including one or more processors and memory; a storage adaptation layer executing on the server, the one or more processors in data communication with the blockchain and the data store; wherein the storage adaptation layer stores logs associated with a subset of changes in data stored within the data store, into the blockchain.

[0014] In accordance with another aspect of the present invention, the system may further include a recovery adaptation layer, in data communication with the data store and the blockchain, the recovery adaptation layer configured to retrieve stored data from the blockchain and store data corresponding to the retrieved stored data into the data store.

[0015] In accordance with another aspect of the present invention, there is provided a method of data backup and recovery. The method includes: tracking a subset of changes in the data stored at a data store, by a user, in a log; mapping the user to an account on the blockchain; encrypting the log using a public key of the account; and storing the encrypted data to cache; and storing the encrypted data to the blockchain.

[0016] The method may additional include: retrieving the data from the cache; triggering a blockchain contract in the blockchain for data consensus and global validation using the blockchain adapter; recording a new data change in a one of the plurality of said nodes; performing consensus voting in the blockchain; wherein upon said new data change conflicting with historical change record in the blockchain, the consensus voting fails; and otherwise, storing said new data change in a block in the blockchain.

[0017] In accordance with yet another aspect of the present invention, there is provided a real time data replication system based including: a blockchain; a target data store; a computing device. The computing device includes: a blockchain listening module adapted to listen to all blocks on the blockchain; a transaction filter filtering transactions on the blocks related to data replication; an event generator to convert content of the filtered transactions to data operations for execution on the target data store, the transaction content including pre-modification content, modified content, and operation type; and a data restore module for executing the data operations such that after execution the target data store is modified to correspond to the blockchain. [0018] In accordance with yet another aspect of the present invention, there is provided a data backup and recovery system for a blockchain, including: a server including: a data adaptation layer; and a data storage system; wherein the data adaptation layer is connected to the blockchain, the data storage system comprises a distributed data store having one or more storage devices, and the data adaptation layer is adapted to facilitate communication between the data storage system and the blockchain. The data adaptation layer includes: a data change monitoring module that monitors data change records in the data storage system; a data conversion module adapted to format the monitored change records into a standard data change record; a blockchain contract to record the data to the blockchain.

BRIEF DESCRIPTION OF DRAWINGS

[0019] In the figures, which illustrate by way of example only, embodiments of the present invention,

[0020] FIG. l is a schematic block diagram of a system utilizing a blockchain based backup and restore operation, exemplary of an embodiment of the present invention;

[0021] FIG. 2 is a flow diagram of an exemplary procedure for backing up data using the system of FIG. 1;

[0022] FIG. 3 is a flowchart summarizing real time data synchronization procedures in an exemplary embodiment of the present invention; and

[0023] FIG. 4 is a flowchart of an exemplary procedure for restoring data using the system of

FIG. 1 DESCRIPTION OF EMBODIMENTS

[0024] The present disclosure describes a blockchain-based backup and restore system and method. Embodiments of the present invention include blockchain-based backup-and-restore systems that operate in a manner that satisfy one or more of the requirements for security, reliability, credibility and/or real time operations.

[0025] A description of various embodiments of the present invention is provided below. In this disclosure, the use of the word “a” or “an” when used herein in conjunction with the term “comprising” may mean “one,” but it is also consistent with the meaning of “one or more”, “at least one” and “one or more than one”. Any element expressed in the singular form also encompasses its plural form. Any element expressed in the plural form also encompasses its singular form. The term “plurality” as used herein means more than one, for example, two or more, three or more, four or more, and the like. Directional terms such as “top”, “bottom”, “upwards”, “downwards”, “vertically” and “laterally” are used for the purpose of providing relative reference only, and are not intended to suggest any limitations on how any article is to be positioned during use, or to be mounted in an assembly or relative to an environment.

[0026] The terms “comprising”, “having”, “including”, and “containing”, and grammatical variations thereof, are inclusive or open-ended and do not exclude additional, un-recited elements and/or method steps. The term “consisting essentially of’ when used herein in connection with a composition, use or method, denotes that additional elements, method steps or both additional elements and method steps may be present, but that these additions do not materially affect the manner in which the recited composition, method, or use functions. The term “consisting of’ when used herein in connection with a composition, use, or method, excludes the presence of additional elements and/or method steps.

[0027] A “blockchain” is a tamper-evident, shared digital ledger that records transactions in a public or private peer-to-peer network of computing devices. The ledger is maintained as a growing sequential chain of cryptographic hash-linked blocks. [0028] A “node” in the context of a blockchain, is a device on the blockchain network. The device is typically be a computing device having a processor interconnected to a processor readable medium including memory, having processor readable instructions thereon.

[0029] The terms “first”, “second”, “third” and the like are used for descriptive purposes only and cannot be interpreted as indicating or implying relative importance.

[0030] In the description of the invention, it should also be noted that the terms “mounted”, “linked” and “connected” should be interpreted in a broad sense unless explicitly defined and limited otherwise. For example, it could be fixed connection, or assembled connection, or integrally connected; either hard-wired or soft-wired; it may be directly connected or indirectly connected through an intermediary. For technical professionals, the specific meanings of the above terms in the invention may be understood in context.

[0031] In the drawings illustrating embodiments of the present invention, the same or similar reference labels correspond to the same or similar parts. In the description of the invention, it should be noted that the meaning of “a plurality of’ means two or more unless otherwise specified; The directions or positions of the terms “up”, “down”, “left”, “right”, “inside”, “outside”, “front end”, “back end”, “head”, “tail”, the orientation or positional relationship shown in the drawings is merely for the convenience of describing the invention and simplifying the description rather than indicating or implying that the indicated device or element must have a particular orientation and be constructed and operated in a particular orientation, and therefore cannot be used as a limitation of the invention.

I. System overview

[0032] FIG. 1 schematically depicts a block diagram of a system utilizing a blockchain based backup and restore operation, exemplary of an embodiment of the present invention. The system includes a data source 101, a log tracker module 102, an event filter 103, an event store 104, a smart contract writer 105, a blockchain 106 a smart contract 107, a node 108, a restore management module 109, a target data store 110, data restore module 111, event generator 112, transaction filter 113 and block transfer 114.

[0033] A storage adaptation layer 115 may be defined to include the log tracker module 102, the event filter 103, the event store 104, and the contract writer 105.

[0034] A recovery adaptation layer 116 may comprise the data restore module 111, the event generator 112, the transaction filter 113 and the block transfer 114.

[0035] Data source 101 generates a change log, whenever data in the data source 101 changes. Data source 101 may be a relational database management system (RDBMS) such as Oracle™ database, MySQL™, Microsoft SQL Server™, and IBM DB2™ database. As may be appreciated, MySQL™ database generates bin-log whereas an Oracle™ database generates the redo-log.

[0036] Log tracker module 102 stores data changes based on logs. For different data sources, different data tracker plugins are used to capture data. For example, in a database system, one data tracker may emulate the replication client and connect to the main database and the main database may send the transaction log to the log tracker when a transaction is successful. After capturing the logs in the data source 101, the data tracker module 102 generates an internal structure or format to store the data changes.

[0037] Event filter 103 filters data based on configuration settings. Not all data needs to be backed up to the blockchain. Event filter 103 thus selects data matching a predefined condition or selection criteria for backup or restore and related processing.

[0038] Event store 104 temporarily stores an event. Event store 104 matches the speed between the blockchain and the data source. If the speed of the data change is faster than the blockchain write speed, the data is temporarily stored in a cache or message queue and waits for further processing. The cache or message queue, thus helps match the data rate of the change in the data source, to the data rate of writing to the blockchain. In this case, even when the system is disrupted, the captured data change is not lost and can continue processing when the system is restored. [0039] Smart contract writer 105 encrypts the change event and triggers the contract in the blockchain. It is also a plugin for matching different blockchains.

[0040] Blockchain 106 is deployed in multiple sites and generates a network for consensus. In this exemplary embodiment, only transactions are stored and thus almost all blockchains may be supported.

[0041] Smart contract 107 is a smart contract running on the blockchain 106. Smart contract 107 takes changed data as input, verifies the account that submitted the change data, and checks the data matches with specific rules and formats. After most of the nodes agree on the consensus, the data is stored in a block as a transaction.

[0042] Node 108 is another node on blockchain 106 where the block containing the transaction is synchronized and can be extracted.

[0043] Restore management module 109 manages the data restoration process. The restore management module 109 controls restoration to a precise snapshot of data or a real time restoration to synchronize with the source data.

[0044] Target data store 110 is a data store that may or may not be the same type as the type used in data source 101. For example, data source 101 may be an Oracle™ database, while data store 110 may be MongoDB™. If choosing the same data type of data store, the modules on a given node Node-N may also be applied in Node-1 to restore data to data source 101 if the original data source 101 crashes.

[0045] Data restore module 111 in this embodiment, includes a plugin to match the target database. In the depicted embodiment, data restore module 111 converts a general JavaScript Object Notation (JSON) data to specific database operations. For example, for a relational database, data restore module 111 converts the operation to an SQL command, whereas for a NoSQL database, data restore module 111 uses another execution format. [0046] Event generator 112 is used to generator event data. After filtering out the transactions, as data is encrypted, it is necessary to decrypt the event to get the changed data, and convert the changed data to JSON format.

[0047] Transaction filter 113 filters out transaction data required for backup and restore operations. A block may contain many kinds of transactions and an appropriate subset is filtered for data backup/restore transactions. After extracting the transactions from the block, it is also necessary to filter out events that are based on the restore side interests. For example, some target nodes are only interested in changes to a particular table, while other nodes may have different interests.

[0048] Block tracker 114 captures the target block in the blockchain 106. For real time synchronization, the tracker 114 starts from the current synchronized block and ends with the latest block in the chain. I Backup Procedure

[0049] FIG. 2 depicts a flow diagram of an exemplary procedure 200 for backing up data using a system exemplary of an embodiment of the present invention, such as the system illustrated in FIG. 1. Each step of procedure 200 described below may be carried out by the system 100 which includes one or more processors on one or more server computing devices, connected to memory storing processor executable instructions that when executed cause the processor(s) to perform one or more of the steps recited below.

[0050] At step 201, a transaction takes place in a database where a piece of data is changed. One of the processors associated with running the database records the changed data and generates log in the form of a bin-log or redo-log to record the change.

[0051] At step 202 a backup server emulates the database replication client and connects to the database. The database copies the change log to the backup server.

[0052] At step 203 a monitor oversees the changes on the log. [0053] At step 204, the log, if it is in raw mode, typically contains the “before” data, the “after” data, and the change type (“insert,” “delete,” or “update”).

[0054] At step 205, as resources are limited, and not all the data may need to be backed up, a user configures which database changes or table changes or column changes or row changes should be backed up to the blockchain 106. After extracting the data from the log, the configured filter is applied to filter out unwanted data.

[0055] At step 206, as the raw data format may be different in different data sources, an adapter is used to convert the data format to the desired (e.g., JSON format). An exemplary JSON format may look like: “

(operator:“oliver”, “source-db”: “dbl”, “table”: “salary”, “operation” :“insert”, “before”:[{“id”:l, “name”: “oliver”, “salary”:“1500.00”},{“id”:2, “name”:“frank”,“salary”:“2000.00”}], “after”: [{“id”: 1, “name”:“oliver”, “salary”:“1510.00”},{“id”:2, “name”:“frank”, “salary”:“2010.00”}]}

[0056] As may be appreciated, this does not record the original SQL command, but rather only the changes in data.

[0057] At step 207 the JSON event is placed into a message queue. As the speed of the log generation may fluctuate with the speed to invoke the blockchain contract, the message queue serves as a cache to buffer the changed data.

[0058] At step 208, the message queue temporarily stores the JSON data, and in case of a system crash, unsaved data is not lost. When the system recovers from the crash, it will continue from the point of interruption.

[0059] At step 209 events are retrieved from the message queue. The contract writer retrieves the JSON data from the message queue. If there are multiple sets of data, smart contract writer retrieves them and combines them into one JSON record. [0060] At step 210 an account is needed to operate the blockchain. With the log, we can extract the database user to operate the data and map that user to a blockchain account in a configuration file.

[0061] At step 211 data is encrypted to prevent a third party from reading sensitive information in the database. The JSON data is encrypted using the account’s public key.

[0062] At step 212 the smart contract is invoked in blockchain 106. The input is encrypted JSON data.

[0063] At step 213 the smart contract generates auto increase event identifier (ID). In a blockchain, time is needed to generate a block. During the period, multiple JSON records are stored. In one block, the sequence of transactions recorded is not guaranteed. The smart contract generates an auto increase sequence ID for each JSON record to identify the sequence within the same block.

[0064] At step 214 the smart contract is executed on most nodes in blockchain for consensus. The smart contract validates the account and the correctness of the JSON data. If the majority of nodes vote with the same result, the smart contract is executed successfully.

[0065] At step 215, after the smart contract is successfully invoked, the JSON data is recorded as smart contract input in the block.

[0066] At step 216, the generated block is again voted on for consensus in the blockchain before taking effect.

III. Real time Data Synchronization Procedure

[0067] FIG. 3 depicts a flowchart summarizing real time data synchronization process in an exemplary embodiment of the present invention. In depicted embodiment, changed data is restored to another node in real time using a process 300 whose steps are enumerated below. Each step of procedure 300 described below may be carried out by the system 100 which includes one or more processors on one or more server computing devices, connected to memory storing processor executable instructions that when executed cause the processor(s) to perform one or more of the steps recited below.

[0068] At step 301 blockchain 106 generates a new block which contains encrypted JSON data.

[0069] At step 302 the current block height is obtained or designated as the target block, and the synchronized block is obtained or designated as the start block.

[0070] At step 303 the process monitors the block in the chain and retrieves the block information from the start block to the target block.

[0071] At step 304 the process extracts all transactions, in each block.

[0072] At step 305 a filter is applied. As there also may be other services running on the blockchain, this step filter out transactions generated by the backup/restore contract.

[0073] At step 306 the process decrypts the transaction and retrieve the clear text JSON event record with the account’s private key. As the blockchain 106 is viewable by all nodes, this prevents unauthorized users from retrieving sensitive data in data storage. Only authorized accounts can access the data.

[0074] At step 307, the process sorts JSON events based on the unique ID generated by the smart contract. If multiple JSON events exist in the same block, the sequence in the block storage may not be the same as the sequence of events.

[0075] At step 308, another filter is applied. As the restored target side may not be interested in all changes in the data source, the process filters out only the data in which the target side is interested.

[0076] At step 309 the process uses a plugin to convert the JSON data to a data execution command. For different target data storage, it may use a different command to apply the changes. [0077] At step 310, the process checks the type of JSON data, for a different database operation.

[0078] At step 311, an “insert” operation causes the use of the “after” data section to insert it into the target store.

[0079] At step 312 an “update” operation, causes the use of the “after” section of data in JSON to update the values in the store.

[0080] At step 313 a “delete” operation causes use the “before” section of data in JSON to find the matching record in the target store and remove it.

[0081] At step 314 the target data structure is changed according to the definition in the JSON record, for DDL (Data Definition Language) operations.

[0082] At step 315, after the current block is finished processing, the process goes to the next block in the chain until the latest block is reached. The process then goes to step 303 to start processing the new block.

IV Restoration Procedure

[0083] FIG. 4 is a flowchart of an exemplary procedure for restoring data using the backup and restore system of FIG. 1. Each step of procedure 400 described below may be carried out by the system 100 which includes one or more processors on one or more server computing devices, connected to memory storing processor executable instructions that when executed cause the processor(s) to perform one or more of the steps recited below.

[0084] In one scenario, the target data store is already synchronized to block 10,000. The process restores the target store status to block 8,000. The process extracts all the operations in the blockchain from block 10,000 to block 8,000 and revert the data change in reverse sequence, which is called a roll-back. [0085] In another scenario, the target store is in block 8,000, and it is necessary to move the status to block 10,000 status. The process extracts operations in the blockchain from block 8,000 to block 10,000 and applies the changes in sequence, which is called a roll-forward.

[0086] The operation applied to a target data store is different in these two scenarios.

[0087] For a rollback operation, if the operation is “insert,” first a “delete” is applied for data matching the “after” values; for the “delete” operation, an “insert” is applied with “before” values.

[0088] For a roll-forward operation, the operation is kept the same as defined in JSON.

[0089] An exemplary process 400 is depicted with the aid of the flowchart in FIG. 4 is described below.

[0090] At step 401 the blockchain contains the change log in sequence.

[0091] At step 402 the process gets the current synchronized block height on the local data store.

[0092] At step 403 the process check the current height with target block height. If it is the same, it means it already reached the target status and could exit. If not, the process continues.

[0093] At step 404 the process monitor the block in the chain and retrieve the block information.

[0094] At step 405 the process extracts the transactions in the block.

[0095] At step 406 the process filters out the transactions generated by the backup/restore smart contract.

[0096] At step 407 the process decrypts the transactions in the block with the account’ s private key. [0097] At step 408, if multiple events occur in the same block, the process retrieves the unique ID generated by the smart contract and sort using this ID. If a roll-back is needed, the process uses the reverse sort but otherwise uses the forward sort.

[0098] At step 409, as the target store may contain more than the request is interested in, the process applies the filter again to retrieve only the data in which the target store is interested.

[0099] At step 410, the process converts the JSON record to a data execution command based on the different data store type. For example, relational databases are converted to SQL commands, and MongoDB/Redis is converted to Mongo or Redis command.

[00100] At step 411, if the target block is greater than the current block, the process rolls forward but otherwise rolls back. The process checks the type of data change operation in JSON.

[00101] At step 412, in roll-back mode, if the operation is “insert,” the process applies “delete” action to the target store with data matching the “after” section.

[00102] At step 413, in roll-back mode, to update, the process changes the data back with the “before” section of data.

[00103] At step 414, in roll-back mode, if the operation is “delete,” the process applies the “insert” action with “before” values in JSON.

[00104] At step 415, if the data structure has changed, the process changes the data structure back.

[00105] At step 416, in roll-forward mode, the process checks the data change operations.

[00106] At step 417, in roll-forward mode, the process applies “insert” with the “after” values in JSON.

[00107] At step 418, in roll-forward mode, the process applies “update” with the “after” values in JSON. [00108] At step 419, in roll-forward mode, apply “delete” with the data matching the “before” values.

[00109] At step 420, for DDL, the process changes the structure defined in JSON.

[00110] At step 421, the process gets the next block in the chain.

[00111] Having thus described, by way of example only, embodiments of the present invention, it is to be understood that the invention as defined by the appended claims is not to be limited by particular details set forth in the above description of exemplary embodiments as many variations and permutations are possible without departing from the scope of the claims.