WU, Song (G2-1301, High-level District1037, Luoyu Roa, Wuhan Hubei 4, 430074, CN)
PAN, Xiaodong (Room 124-201, West 2 District1037, Luoyu Roa, Wuhan Hubei 4, 430074, CN)
JIN, Hai (Room 701, Apartment 5High-level District, 1037, Luoyu Roa, Wuhan Hubei 4, 430074, CN)
WU, Song (G2-1301, High-level District1037, Luoyu Roa, Wuhan Hubei 4, 430074, CN)
PAN, Xiaodong (Room 124-201, West 2 District1037, Luoyu Roa, Wuhan Hubei 4, 430074, CN)
| CLAIMS What is claimed is: 1. A method for live migration of a virtual machine of a first host computer to a second host computer, the method comprising: while the virtual machine is operating on the first host computer, continually monitoring and recording (202), by the first host computer, writes by the virtual machine to memory of the first host computer, continually sending (204), by the first host computer, modified data in the memory of the first host computer to the second host computer, and continually determining (206), by the first host computer, whether modified data sent to the second host computer substantially matches the modified data in the memory of the first host computer; and stopping the virtual machine (208) from operating on the first host computer, when the first host computer determines that the modified data sent to the second host computer substantially matches the modified data in the memory of the first host computer. 2. The method of claim 1, further comprising after stopping the virtual machine, sending (210), by the first host computer, one last amount of modified data in the memory of the first host computer to the second host computer. 3. The method of claim 1, wherein continually monitoring and recording, by the first host computer, writes by the virtual machine to memory of the first host computer, comprises initializing (308) one or more bitmaps to track modification of one or more corresponding memory pages of the memory of the first host computer; initializing a first table (302) to track modification rates of the memory pages; or initializing a second table (302) to track modified data of the memory pages. 4. The method of claim 1, wherein continually monitoring and recording, by the first host computer, writes by the virtual machine to memory of the first host computer, comprises updating one or more bitmaps (310) to track modification of one or more corresponding memory pages of the memory of the first host computer; updating a first table (310) to track modification rates of the memory pages; or updating a second table (310) to track modified data of the memory pages. 5. The method of claim 1, wherein continually sending, by the first host computer, modified data in the memory of the first host computer to the second host computer, comprises determining (402) whether a modification rate of a memory page of the memory of the first host computer has exceeded a boundary value. 6. The method of claim 5, wherein continually sending, by the first host computer, modified data in the memory of the first host computer to the second host computer, further comprises sending (404) an entire memory page to the second host computer, when the first host computer determines that the modification rate of the memory page has exceeded the boundary value. 7. The method of claim 5, wherein continually sending, by the first host computer, modified data in the memory of the first host computer to the second host computer, further comprises sending (406) modified data of a memory page to the second host computer, when the first host computer determines that the modification rate of the memory page has failed to exceed the boundary value. 8. The method of claim 7, wherein sending modified data of a memory page to the second host computer comprises packaging (406) the modified data of the memory page, with a modification bitmap of the memory page. 9. A first host computer for hosting a virtual machine, including live migration of the virtual machine to a second host computer, the first host computer comprising: memory (104); one or more processors (106) coupled with the memory, and configured to cooperate with the memory to host the virtual machine; and a virtual machine monitor (112) configured to be operated by the one or more processors to: perform a number of tasks while the virtual machine is in operation on the first host computer, including, a first task to continually monitor and record (202) writes by the virtual machine to the memory, a second task to continually send (204) modified data in the memory to the second host computer, and a third task to continually determine (206) whether modified data sent to the second host computer substantially matches modified data in the memory; and stop (208) the virtual machine after a determination that the modified data sent to the second host computer substantially matches the modified data in the memory. 10. The first host computer of claim 9, wherein the virtual machine monitor (112) is further configured to, after the virtual machine has stopped, send (210) one last amount of modified data in the memory of the first host computer to the second host computer. 11. The first host computer of claim 9, wherein the virtual machine monitor is configured to perform, as part of the continually monitoring and recording task, initializing (308) or updating (310) one or more bitmaps to track modification of one or more corresponding memory pages of the memory of the first host computer; initializing (302) or updating a first table to track modification rates of the memory pages; or initializing (302) or updating a second table to track modified data of the memory pages. 12. The first host computer of claim 9, wherein the virtual machine monitor is configured to perform, as part of the continually sending task, determining (402) whether a modification rate of a memory page of the memory of the first host computer has exceeded a boundary value. 13. The first host computer of claim 12, wherein the virtual machine monitor is further configured to perform, as part of the continually sending task, sending (404) an entire memory page to the second host computer, when the first host computer determines that the modification rate of the memory page has exceeded the boundary value. 14. The first host computer of claim 12, wherein the virtual machine monitor is further configured to perform, as part of the continually sending task, sending (406) modified data of a memory page to the second host computer, when the first host computer determines that the modification rate of the memory page has failed to exceed the boundary value. 15. The first host computer of claim 9, wherein the virtual machine monitor is further configured to perform, as part of the sending of modified data of a memory page to the second host computer, packaging (406) the modified data of the memory page with a modification bitmap of the memory page. 16. An article of manufacture for live migration of a virtual machine of a first host computer to a second host computer, the article comprising: a tangible computer-readable storage medium; and a plurality of programming instructions stored on the tangible computer-readable storage medium, and configured to enable the first host computer, in response to execution of the programming instructions by the first host computer, to perform operations including: while the virtual machine is operating on the first host computer, continually monitoring and recording writes by the virtual machine to memory of the first host computer, continually sending modified data in the memory of the first host computer to the second host computer, continually determining whether modified data sent to the second host computer substantially matches modified data in the memory of the first host computer; and stopping the virtual machine after the first host computer determines that the modified data sent to the second host computer substantially matches the modified data in the memory. 17. The article of claim 16, the operations further comprising, after stopping the virtual machine, sending (210) one last amount of modified data in the memory of the first host computer to the second host computer. 18. The article of claim 16, wherein continually monitoring and recording writes by the virtual machine to memory of the first host computer comprises: initializing (308) or updating (310) one or more bitmaps to track modification of one or more corresponding memory pages of the memory of the first host computer; initializing (302) or updating a first table (310) to track modification rates of the memory pages; or initializing (302) or updating a second table (310) to track modified data of the memory pages. 19. The article of claim 16, wherein continually sending modified data in the memory of the first host computer to the second host computer comprises: determining (402) whether a modification rate of a memory page of the memory of the first host computer has exceeded a boundary value; sending (404) an entire memory page to the second host computer, when the first host computer determines that the modification rate of the memory page has exceeded the boundary value; or sending (406) modified data of a memory page to the second host computer, when the first host computer determines that the modification rate of the memory page has failed to exceed the boundary value. 20. The article of claim 16, wherein sending of modified data of a memory page to the second host computer comprises packaging (406) the modified data of the memory page with a modification bitmap of the memory page. |
BACKGROUND
[0001] In today's virtual data center and large-scale cluster computing environments involving multiple host computers, with each host computer hosting multiple virtual machines, efficient and reliable operation often requires live migration of a virtual machine from one host computer to another host computer. Ideally, the live migration is accomplished without discernable lost of service provided by the virtual machine. At present, virtual machine live migration methodology typically requires a large amount of bandwidth from the network or networks inter-coupling the host computers. This is because today's methodology often involves a large amount of data transmissions between the source host computer, from which a virtual machine is live migrating, and the destination host computer, to which the virtual machine is live migrating to. For example, today's methodology often transmits the memory pages of the source host computer in their entirety to the destination host computer, even for a memory page with only a small number of memory locations having been modified.
[0002] Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
SUMMARY
[0003] This disclosure is drawn, inter alia, to methods, apparatuses and/or articles of manufacture generally associated with live migration of a virtual machine of a first (source) host computer, to a second (destination) host computer. In various embodiments, a method may include, while the virtual machine is operating on the source host computer, the source host computer continually monitor and record writes by the virtual machine to memory of the source host computer, continually send modified data in the memory of the source host computer to the destination host computer, and continually determine whether modified data sent to the destination host computer substantially matches the modified data in the memory of the source host computer. The method may further include the source host computer stopping the virtual machine from operating on the source host computer, when the source host computer determines that the modified data sent to the destination host computer substantially matches the modified data in the memory of the source host computer.
Additionally, the method may include the source host computer sending one last amount of modified data in the memory of the source host computer to the destination host computer, after stopping the virtual machine from operating on the source host computer.
[0004] In various embodiments, the continual monitor and record of writes by the virtual machine to memory of the source host computer may include the source host computer initializing one or more bitmaps to track modification of one or more corresponding memory pages of the memory of the source host computer, initializing a first table to track modification rates of the memory pages of the source host computer, and/or initializing a second table to track modified data of the memory pages of the source host computer.
Additionally or alternatively, the continually monitoring and recording of writes by the virtual machine to memory of the source host computer may include the source host computer updating one or more bitmaps to track modifications of one or more corresponding memory pages of the memory of the source host computer, updating the first table to track modification rates of the memory pages of the source host computer, and/or updating the second table to track modified data of the memory pages the source host computer.
[0005] In various embodiments, the continually sending of modified data in the memory of the source host computer to the destination host computer may include the source host computer determining whether a modification rate of a memory page of the memory of the source host computer has exceeded a boundary value. Additionally, the sending operation may include the source host computer sending an entire memory page to the destination host computer, when the source host computer determines that the modification rate of the memory page of the source host computer has exceeded the boundary value, and/or sending modified data of a memory page to the destination host computer, when the source host computer determines that the modification rate of the memory page of the source host computer has failed to exceed the boundary value. Further, the sending of modified data of a memory page of the source host computer to the destination host computer may include sending a package formed with the modified data of a memory page of the source host computer, and the modification bitmap of the memory page.
[0006] In various embodiments, a computer may be configured to be a source host computer for hosting a virtual machine. The source host computer may be configured to include a virtual machine live migration service equipped to live migrate the virtual machine from the source host computer to a destination host computer. [0007] In various embodiments, the source host computer may include memory, one or more processors, and a virtual machine monitor. The virtual machine monitor may be configured to be operated by the one or more processors to perform a number of tasks while the virtual machine is in operation on the source host computer. The tasks may include a first task to continually monitor and record writes by the virtual machine to the memory, a second task to continually send modified data in the memory to the destination host computer, and a third task to continually determine whether modified data sent to the destination host computer substantially matches modified data in the memory. In various embodiments, the virtual machine monitor may be further configured to stop the virtual machine after a determination that the modified data sent to the destination host computer substantially matches the modified data in the memory of the source host computer.
[0008] In various embodiments, an article of manufacture with a non-transitory, tangible computer-readable storage medium, may be provided with a number of instructions configured to cause an apparatus, in response to execution of the instructions by the apparatus, to perform a number operations associated with live migrating a virtual machine hosted by the apparatus to another apparatus. The operations may be performed while the virtual machine is in operation on the apparatus. The operations may include continually monitoring and recording writes by the virtual machine to memory of the apparatus, continually sending modified data in the memory of the apparatus to the another apparatus, and continually determining whether modified data sent to the other apparatus substantially matches modified data in the memory of the apparatus. In various embodiments, the operations may further include stopping the virtual machine after the apparatus determines that the modified data sent to the other apparatus substantially matches the modified data in the memory of the apparatus.
[0009] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Subject matter is particularly pointed out and distinctly claimed in the concluding portion of the specification. The foregoing and other features of this disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings depict several embodiments in accordance with the disclosure and, therefore, are not to be
considered limiting of its scope, the disclosure will be described with additional specificity and detail through use of the accompanying drawings. Various embodiments will be described referencing the accompanying drawings in which like references denote similar elements, and in which:
FIG. 1 illustrates an overview of a computing environment having a number of host computers hosting virtual machines, and equipped with a virtual machine live migration service that includes a continual monitor and send function for memory writes,
FIG. 2 illustrates a method of virtual machine live migration, employing the virtual machine live migration service of FIG. 1,
FIG. 3 illustrates the monitor and record operation of FIG. 2 in further details;
FIG. 4 illustrates the send operation of FIG. 2 in further details,
FIG. 5 illustrates a modification rate table, a write data table, and a memory page bitmap, suitable for use with the monitor and record, and send operations of FIG. 3 and 4,
FIG. 6 illustrates an example computing device suitable for use as a host computer of FIG. 1, and
FIG 7 illustrates an example program product including an article of manufacture with instructions configured to enable an apparatus to perform the method of FIG. 2, all arranged in accordance with at least some embodiments of the present disclosure.
DETAILED DESCRIPTION
[0011] The following description sets forth various examples along with specific details to provide a thorough understanding of claimed subject matter. However, it will be understood by those skilled in the art that claimed subject matter may be practiced without some or more of the specific details disclosed herein. Further, in some circumstances, well- known methods, procedures, systems, components and/or circuits have not been described in detail in order to avoid unnecessarily obscuring claimed subject matter. In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the Figures, may be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure.
[0012] In the following description, algorithms and/or symbolic representations of operations on data bits and/or binary digital signals stored within a computing system, such as within a computer and/or computing system memory may be presented. An algorithm may generally be considered to be a self-consistent sequence of operations and/or similar processing leading to a desired result where the operations may involve physical
manipulations of physical quantities that may take the form of electrical, magnetic and/or electromagnetic signals capable of being stored, transferred, combined, compared and/or otherwise manipulated. In various contexts such signals may be referred to as bits, data, values, elements, symbols, characters, terms, numbers, numerals, etc. Those skilled in the art will recognize, however, that such terms may be used to connote physical quantities. Hence, when terms such as "storing," "processing," "retrieving," "calculating," "determining" etc. are used in this description they may refer to the actions of a computing platform, such as a computer or a similar electronic computing device such as a cellular telephone, that manipulates and/or transforms data represented as physical quantities including electronic and/or magnetic quantities within the computing platform's processors, memories, registers, etc.
[0013] FIG. 1 illustrates an overview of a computing environment having a number of host computers hosting virtual machines, and equipped with a virtual machine live migration service that includes a continual monitor and send function for memory writes, in
accordance with various embodiments of the present disclosure. As shown, computing environment 100 may include a number of host computers, e.g., host computer A 102 and host computer B 122. Host computer A 102 may include memory 104 and processors 106, while host computer B 122 may include memory 124 and processors 126. Host computer A 102 may host a number of virtual machines 110, while host computer B 122 may host a number of virtual machines 130. Further, host computer A 102 may include virtual machine monitor (VMM) 112, while host computer B 122 may include virtual machine monitor (VMM) 132. VMM 112 may include a virtual machine (VM) Live Migration Service 114 with Continual Monitor and Send Function for memory writes, while VMM 132 may include a virtual machine (VM) Live Migration Service 134 with Continual Monitor and Send Function for memory writes. [0014] Host computers A and B 102 and 122 may be coupled to each other via network 150. Host computers A and B 102 and 122 may also have shared access to various mass storage devices 140.
[0015] VM Live Migration Services 114 and 134, each with Continual Monitor and Send Function for memory writes, may be configured to enable host computer A and B 102 and 122, to practice embodiments of the method of VM live migration of the present disclosure, to be described more fully below. Except for VM Live Migration Services 114 and 134, the other elements of FIG. 1, host computers A and B 102 and 122, memory 104 and 124, processors 106 and 126, virtual machines 110 and 130, the basic functions of VMM 112 and 122, network 150 and mass storage devices 140 are all intended to represent a broad range of these elements.
[0016] Before further describing the method of VM live migration of the present disclosure, and VM Live Migration Services 114 and 134, it should be noted that while for ease of understanding, only two host computers A and B 102 and 122, having memory 104 and 124 and processors 106 and 126, respectively, are illustrated, nonetheless, the present disclosure is not so limited, and may be practiced with many more host computers, and a host computer may include more or less components.
[0017] Referring now to FIG. 2, wherein a method of virtual machine live migration, employing the virtual machine live migration service of FIG. 1, in accordance with embodiments of the present disclosure, is illustrated. For the methods described herein, one or more of the host computer 102 and/or host computer 122 can be configured to perform the various operations, functions or actions described below via a respective one of VMM 112 and VMM 132. An example method 200 may include one or more functions, operations, or actions as is illustrated by one or more of blocks 202, 204, 206, 208, and/or 210. It should be appreciated that in some implementations one or more of the illustrated blocks may be eliminated, combined or separated into additional blocks, or performed in a different order without departing from the spirit of the present disclosure.
[0018] As shown, VM live migration method 200 may include two phases, a pre-copy phase 212 and a shut down and copy phase 214. For ease of understanding, pre-copy phase 212 and shut down and copy phase 214 of VM live migration method 200 will be described in terms of an example live migration of a VM 110 from host computer A 102 to host computer B 122. However, the description is not to be construed as limiting on the present disclosure. Further, host computer A 102 may be referred to as the source host computer, while host computer B 122 may be referred to as the destination host computer. [0019] During the pre-copy phase 212, VM Live Migration Services 114 of source host computer 102 may begin the VM live migration method 200 at block 202 (Monitor and Record Writes to Memory). At block 202, VM Live Migration Services 114 may monitor and record writes by one or more of virtual machines 110 to memory 104. From block 202, the method may continue at block 204 (Send Modified Data). At block 204, VM Live Migration Services 114 may send modified data from source host computer 102 to destination host computer 122.
[0020] As described earlier, the sending of data between source computer and the destination computer may be constrained by the available bandwidth of the inter-coupling network. Thus, after a series of writes by the virtual machines of the source host computer to the memory of the source host computer, the amount of modified data having been sent to and stored on destination host computer, might be less than the amount of modified data on the source host computer. However, when the network bandwidth eases (especially relative to the volume of writes to memory), the amount of modified data having been sent to and stored on destination host computer might substantially match the amount of modified data on the source host computer
[0021] Accordingly, for various embodiments, from block 204, method 200 may proceed to block 206 (Modified Data Sent substantially matches Data Modified). At block 206, VM Live Migration Services 114 may determine whether the amount of modified data sent from the source host computer to the destination host computer substantially matches the amount of data modified on the source host computer.
[0022] What constitutes "substantially matches" may be application dependent.
Typically, the amount of modified data sent from the source host computer to the destination host computer is considered substantially matching with the amount of data modified on the source host computer, when the amount remaining to be sent to the destination host computer can be sent within a desired small time interval in view of the network bandwidth available. The reason shall be apparent from the description to follow.
[0023] If at block 206, VM Live Migration Services 114 determines that the amount of modified data sent from the source host computer to the destination host computer does not substantially match the amount of data modified on the source host computer, method 200 may return to block 202, and then proceed onto blocks 204 and 206 as earlier described. Thus, method 200 continually monitor and record writes by the virtual machine to the memory of the source host computer at block 202, continually send modified data from the source host computer to the destination host computer at block 204, and continually determine whether the amount of modified data sent from the source host computer to the destination host computer substantially matches the amount of data modified on the source host computer, until VM Live Migration Services 114 determines that the amount of modified data sent from the source host computer to the destination host computer does substantially match the amount of data modified on the source host computer.
[0024] If at block 206, VM Live Migration Services 114 determines that the amount of modified data sent from the source host computer to the destination host computer substantially matches the amount of data modified on the source migration host computer, method 200 ends pre-copy phase 212, and enters shut down and copy phase 214.
[0025] On entry into the shut down and copy phase 214, method 200 may proceed to block 208 (Stop Virtual Machine). At block 208, VM Live Migration Services 114 causes the virtual machine to be migrated to stop operating on the source host computer. From block 208, method 200 may proceed to block 210 (Send Final Modified Data). At block 208, VM Live Migration Services 114 causes the final installment of the modified data to be sent from the source host computer to the destination host computer.
[0026] As described earlier, since method 200 enters into the shut down and copy phase 214 after VM Live Migration Services 114 determines that the amount of modified data sent from the source host computer to the destination host computer substantially matches the amount of data modified on the source host computer, subject to a design point selected for substantial matching and the network bandwidth, the virtual machine being live migrated has to be stopped for only a relatively short time. In particular, it is expected that using method 200, the amount of time the virtual machine being live migrated has to be stopped is shorter than prior art live migration methods.
[0027] FIG. 3 illustrates the monitor and record operation of FIG. 2 in further details, in accordance with various embodiments of the present disclosure. For the methods described herein, one or more of the host computer 102 and/or host computer 122 can be configured to perform the various operations, functions or actions described below via a respective one of VMM 112 and VMM 132. An example method 202 may include one or more functions, operations, or actions as is illustrated by one or more of blocks 302, 304, 306, 308, and/or 310. It should be appreciated that in some implementations one or more of the illustrated blocks may be eliminated, combined or separated into additional blocks, or performed in a different order without departing from the spirit of the present disclosure.
[0028] As shown, for the embodiments, VM Live Migration Services 114 starts the monitor and record writes to memory operation 202 at block 302 (Initialize Data Structures). At block 302, VM Live Migration Services 114 may initialize various data structures to be employed for the monitor and record writes to memory operation 202. In various embodiments, VM Live Migration Services 114 may initialize a modification rate table and/or a write data table, to be described more fully below.
[0029] From block 302, operation 202 may proceed to block 304 (Monitor Memory Writes). At block 304, VM Live Migration Services 114 may continue to monitor for writes to memory by the virtual machine to be live migrated. On detection of a write to memory by the virtual machine to be live migrated, operation 202 may proceed from block 304 to block 306 (Bitmap Initialized?). At block 306, VM Live Migration Services 114 may determine whether a memory page bitmap corresponding to the memory page being written into has been previously initialized and in use. If the result of the determination is negative, that is, a memory page bitmap has not been initialized for the memory page being written into, operation 202 may proceed from block 306 to block 308 (Initialize a Bitmap). At block 308, VM Live Migration Services 114 may initialize a memory page bitmap for the memory page being written into, and proceeds to block 310 (Record Memory Write). On the other hand, if the result of the determination at block 306 is affirmative, that is, a memory page bitmap has been initialized for the memory page being written into, operation 202 may directly proceed from block 308 to block 310 (Record Memory Write). At block 310, VM Live Migration Services 114 may record the write to memory by the virtual machine being live migrated. For the embodiments where a modification rate table, a write data table, and memory page bitmaps are employed, Live Migration Services 114 may update the modification rate table, the write data table and a memory page bitmap corresponding to the memory page being written into accordingly.
[0030] FIG. 4 illustrates the send operation of FIG. 2 in further details, in accordance with various embodiments of the present disclosure. For the methods described herein, one or more of the host computer 102 and/or host computer 122 can be configured to perform the various operations, functions or actions described below via a respective one of VMM 112 and VMM 132. An example method 204 may include one or more functions, operations, or actions as is illustrated by one or more of blocks 402, 404, and/or 406. It should be appreciated that in some implementations one or more of the illustrated blocks may be eliminated, combined or separated into additional blocks, or performed in a different order without departing from the spirit of the present disclosure.
[0031] As shown, to send modified data of a memory page, send operation 204 may start at block 402 (Memory writes to a Page greater than a boundary value). At block 402, Live Migration Services 114 may determine whether the number of writes into memory for the particular memory page has exceeded a boundary value. In various embodiments, the boundary value may be configurable.
[0032] On determining at block 402 that the number of writes into memory for the particular memory page has not exceeded the boundary value, send operation 204 may proceed from block 402 to block 404 (Send only changed data). At block 404, Live
Migration Services 114 may send only the changed data of the memory page from the source host computer to the destination host computer. In various embodiments, Live Migration Services 114 may reference the modification rate table to determine whether the number of writes into memory for the particular memory page has exceeded the boundary value.
Further, the sending of only changed data may include forming a package that includes the changed data (using the write data table) and a modification bitmap of the memory page (showing the locations of the changed data), and sending the formed package.
[0033] However, on determining at block 402 that the number of writes into memory for the particular memory page has exceeded the boundary value, send operation 204 may proceed from block 402 to block 406 (Send entire Memory Page). At block 406, Live
Migration Services 114 may send the entire memory page from the source host computer to the destination host computer.
[0034] FIG. 5 illustrates a modification rate table, a write data table, and a memory page modification bitmap, suitable for use with the monitor and record, and send operations of FIG. 3 and 4, in accordance with various embodiments of the disclosure. The various features illustrated in FIG. 5 may be utilized by host computer 102 and/or host computer 122 to facilitate the various methods described herein. It should be appreciated that in some implementations a different arrangement of the same or similar information may be suitable.
[0035] As illustrated, for various embodiments, modification rate table 502 may include two columns, a page id column 504 for storing identifiers of memory pages, and a
modification rate column 506 for storing the number of memory locations of the memory pages having been modified. Additionally, write data table 512 may include three (3) columns, a page id column 514 for storing identifiers of the memory pages, an offset column 516 for storing an offset to a specific location within a memory page for the write data, and a data column 518 for storing the write data themselves. Further, memory page modification bitmap 522 may include a m x n array, where m and n are integers corresponding to the size of a memory page of the source host computer. Each array slot 524 may be employed to store an indicator indicating whether the corresponding memory location has been modified. [0036] FIG. 6 is a block diagram illustrating an example computing device 600, suitable for use as a host computer of FIG. 1, in accordance with the present disclosure. In a very basic configuration 601, computing device 600 typically includes one or more processors 610 and system memory 620. A memory bus 630 may be used for communicating between the processor 610 and the system memory 620.
[0037] Depending on the desired configuration, processor 610 may be of any type including but not limited to a microprocessor (μΡ), a microcontroller (μθ, a digital signal processor (DSP), or any combination thereof. Processor 610 may include one more levels of caching, such as a level one cache 611 and a level two cache 612, a processor core 613, and registers 614. An example processor core 613 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. An example memory controller 615 may also be used with the processor 610, or in some implementations the memory controller 615 may be an internal part of the processor 610.
[0038] Depending on the desired configuration, the system memory 620 may be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. System memory 620 may include a number of hosted virtual machines 622, and a virtual machine monitor 623. Virtual machine monitor 623 may include a virtual machine live migration service 624 that includes a continual monitor and send function, as described earlier.
[0039] Computing device 600 may have additional features or functionality, and additional interfaces to facilitate communications between the basic configuration 601 and any required devices and interfaces. For example, a bus/interface controller 640 may be used to facilitate communications between the basic configuration 601 and one or more data storage devices 650 via a storage interface bus 641. The data storage devices 650 may be removable storage devices 651, non-removable storage devices 652, or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. [0040] System memory 620, removable storage 651 and non-removable storage 652 are all examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which may be used to store the desired information and which may be accessed by computing device 600. Any such computer storage media may be part of computing device 600.
[0041] Computing device 600 may also include an interface bus 642 for facilitating communication from various interface devices (e.g., output interfaces, peripheral interfaces, and communication interfaces) to the basic configuration 601 via the bus/interface controller 640. Example output devices 660 include a graphics processing unit 661 and an audio processing unit 662, which may be configured to communicate to various external devices such as a display or speakers via one or more A/V ports 663. Example peripheral interfaces 670 include a serial interface controller 671 or a parallel interface controller 672, which may be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports 673. An example communication device 680 includes a network controller 681, which may be arranged to facilitate communications with one or more other computing devices 690 over a network communication link via one or more communication ports 682.
[0042] The network communication link may be one example of a communication media. Communication media may typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A
"modulated data signal" may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein may include both storage media and communication media.
[0043] Computing device 600 may be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. Computing device 600 may also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.
[0044] Articles of manufacture and/or systems may be employed to perform one or more methods as disclosed herein. FIG. 7 illustrates a block diagram of an example article of manufacture having a computer program product 700 configured to enable an apparatus to practice the method of FIG. 2, in accordance with various embodiments of the present disclosure. The computer program product 700 may comprise non-transitory computer- readable storage medium 702 and plurality of programming instructions 704 stored in the computer-readable storage medium 702.
[0045] In various ones of these embodiments, programming instructions 704 may be configured to enable an apparatus, in response to execution by the apparatus, to perform operations including:
while a virtual machine is operating on the apparatus,
continually monitoring and recording writes by the virtual machine to memory of the apparatus,
continually sending modified data in the memory of the apparatus to another apparatus, and
continually determining whether modified data sent to the another apparatus substantially matches the modified data in the memory of the apparatus; and stopping the virtual machine from operating on the apparatus on determining that the modified data sent to the other host computer substantially matches the modified data in the memory of the first host computer.
[0046] Computer-readable storage medium 702 may take a variety of forms including, but not limited to, non-volatile and persistent memory, such as, but not limited to, compact disc read-only memory (CDROM) and flash memory.
[0047] Reference in the specification to "an implementation," "one implementation," "some implementations," or "other implementations" may mean that a particular feature, structure, or characteristic described in connection with one or more implementations may be included in at least some implementations, but not necessarily in all implementations. The various appearances of "an implementation," "one implementation," or "some implementations" in the preceding description are not necessarily all referring to the same implementations. Moreover, when terms or phrases such as "coupled" or "responsive" or "in response to" or "in communication with," etc. are used herein or in the claims that follow, these terms should be interpreted broadly. For example, the phrase "coupled to" may refer to being communicatively, electrically and/or operatively coupled as appropriate for the context in which the phrase is used.
[0048] In the preceding description, various aspects of claimed subject matter have been described. For purposes of explanation, specific numbers, systems and/or configurations were set forth to provide a thorough understanding of claimed subject matter. However, it should be apparent to one skilled in the art and having the benefit of this disclosure that claimed subject matter may be practiced without the specific details. In other instances, well-known features were omitted and/or simplified so as not to obscure claimed subject matter. While certain features have been illustrated and/or described herein, many modifications, substitutions, changes and/or equivalents will now, or in the future, occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and/or changes as fall within the true spirit of claimed subject matter.
[0049] There is little distinction left between hardware and software implementations of aspects of systems; the use of hardware or software is generally (but not always, in that in certain contexts the choice between hardware and software may become significant) a design choice representing cost versus efficiency tradeoffs. There are various vehicles by which processes and/or systems and/or other technologies described herein may be effected (e.g., hardware, software, and/or firmware), and that the preferred vehicle will vary with the context in which the processes and/or systems and/or other technologies are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may opt for a mainly hardware and/or firmware vehicle; if flexibility is paramount, the implementer may opt for a mainly software implementation; or, yet again alternatively, the implementer may opt for some combination of hardware, software, and/or firmware.
[0050] The foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, flowcharts, and/or examples.
Insofar as such block diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood by those within the art that individual function and/or operation within such block diagrams, flowcharts, or examples may be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, several portions of the subject matter described herein may be implemented via Application Specific ICs (ASICs), Field Programmable Gate Arrays (FPGAs), digital signal processors (DSPs), or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein, in whole or in part, may be equivalently implemented in ICs, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more
microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein are capable of being distributed as a program product in a variety of forms, and that an illustrative embodiment of the subject matter described herein applies regardless of the particular type of signal bearing medium used to actually carry out the distribution. Examples of a signal bearing medium include, but are not limited to, the following: a recordable type medium such as a floppy disk, a hard disk drive, a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape, a computer memory, etc.; and a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).
[0051] Those skilled in the art will recognize that it is common within the art to describe devices and/or processes in the fashion set forth herein, and thereafter use engineering practices to integrate such described devices and/or processes into data processing systems. That is, at least a portion of the devices and/or processes described herein may be integrated into a data processing system via a reasonable amount of experimentation. Those having skill in the art will recognize that a typical data processing system generally includes one or more of a system unit housing, a video display device, a memory such as volatile and nonvolatile memory, processors such as microprocessors and digital signal processors, computational entities such as operating systems, drivers, graphical user interfaces, and applications programs, one or more interaction devices, such as a touch pad or screen, and/or control systems including feedback loops and control motors (e.g., feedback for sensing position and/or velocity; control motors for moving and/or adjusting components and/or quantities). A typical data processing system may be implemented utilizing any suitable commercially available components, such as those typically found in data
computing/communication and/or network computing/communication systems.
[0052] The herein described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely exemplary, and that in fact many other architectures may be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality may be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated may also be viewed as being "operably connected", or "operably coupled," to each other to achieve the desired
functionality, and any two components capable of being so associated may also be viewed as being "operably couplable," to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly
interacting components and/or logically interacting and/or logically interactable components.
[0053] With respect to the use of substantially any plural and/or singular terms herein, those having skill in the art may translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity.
[0054] It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term
"includes" should be interpreted as "includes but is not limited to," etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to inventions containing one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an" (e.g., "a" and/or "an" should typically be interpreted to mean "at least one" or "one or more"); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, typically means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to "at least one of A, B, or C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, or C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B."