The present invention relates to a method for supporting creation of virtual machines on a virtualization platform.

Furthermore, the present invention relates to a system supporting creation of virtual machines on a virtualization platform. In recent years lightweight virtualization technologies such as Docker retrievable at "https://www.docker.com/" and LXC retrievable at "https:// linuxcontainers.org" are gaining enormous traction not only in the research field but also in terms of real- world deployment. Google, for instance, is reported to run all of its services in containers, retrievable at "http://www.theregister.co.uk/2014/05/23/google_contain erization_two_billion/", and Container as a Service (CaaS) products are available from a number of major players including Azure's Container Service (retrievable at "https://azure.microsoft.com/en-us/services/container-servic e/"), Amazon's EC2 Container Service and Lambda offerings (retrievable at "https://aws.amazon.com/ lambda"), and Google's Container Engine service (retrievable at "https://cloud. google.com/container-engine").

Beyond these services, lightweight virtualization is crucial to a wide range of use cases, including just-in-time instantiation of services (e.g. described in the nonpatent literature of MADHAVAPEDDY, A., LEONARD, T., SKJEGSTAD, M., GAZA-GNAIRE, T., SHEETS, D., SCOTT, D., MORTIER, R., CHAUDHRY, A., SINGH, B., LUDLAM, J., CROWCROFT, J., AND LESLIE, I. Jitsu: Just-in-time summoning of unikernels. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15) (Oakland, CA, 2015), USENIX Association, pp. 559-573) - e.g., filters against Distributed Denial of Service (DDoS) attacks, TCP acceleration proxies, content caches, etc. - and networks functions virtualization (NFV), all the while providing significant cost reduction through consolidation and power minimization. The reasons for containers to have taken the virtualization market by storm are clear. In contrast to heavyweight, hypervisor-based technologies - which represent virtualization platforms - such as VMWare, KVM or Xen, containers provide extremely fast instantiation times, small per-instance memory footprints, and high density on a single host, among other features.

However, no technology is perfect, and containers are no exception. Security, for one, has been and continues to be a thorn on their side. First, their large trusted computing base (TCB), at least compared to type-1 hypervisors, has resulted in a large number exploits. Second, a container that causes a kernel panic will bring down the entire host. Further, any container that can monopolize or exhaust system resources (e.g., memory, file descriptors, user IDs, forkbombs, etc.) will cause a Denial-of-Service (DOS) attack on all other containers on that host. Over the years, a significant amount of effort has resulted in the introduction of mechanisms such as user namespaces and Seccomp that harden or eliminate a large number of these attack vectors. However, a simple misconfiguration can still lead to an insecure system.

Beyond security, another downside of containers is that their sharing of the same kernel rules out the possibility to specialize the kernel and its network stack to provide better functionality and performance to specific applications. Finally, containers do not currently support live migration, although support for it is under development. At least for multitenant deployments, this leaves with a difficult choice between

(1) containers and the security issues surrounding them, and

(2) the burden coming from heavyweight, VM-based platforms.

Clearly, it cannot easily, overnight, be fixed all of the security issues related to containers, nor prevent new ones from arising. Thus, the ability to quickly boot a virtual machine (VM) is crucial to a wide range of application scenarios ranging from networks functions virtualization (NFV) to the ability to instantiate firewalls on a per-connection basis, to dynamically create filters to deal with DOS attacks, to be able to quickly and dynamically boot monitoring services to oversee financial transactions, and to host services whose key performance indicators (KPIs) depend on boot times such as block chain and function-based services such as Amazon's Lambda, among many others.

As previously mentioned, the importance of small creation and boot times is at least partly demonstrated by the rise of containers and their typically faster-than- VMs boot times, although containers trade-off this performance against isolation, which is basic to a number of the application scenarios mentioned above. Known virtualization platforms, and the virtual machines that run on top of them, appears to be inherently and fundamentally heavyweight and slow to boot.

In view of the above, it is therefore an object of the present invention to improve and further develop a method and a system of the initially described type for supporting creation of virtual machines on a virtualization platform in such a way that creation of virtual machines is improved with regard to the virtual machine creation time, preferably while retaining the isolation and security afforded by the underlying virtualization platform.

In accordance with the invention, the aforementioned object is accomplished by a method for supporting creation of virtual machines on a virtualization platform, wherein virtual machine creation is triggered when a virtual machine creation command is issued, the method comprising:

providing a toolstack that separates its functionality into at least two processing phases, wherein said processing phases comprise a prepare phase and an execute phase;

performing the prepare phase independently from virtual machine creation, wherein the prepare phase generates one or more virtual machine shells; and

performing the execute phase based upon issuance of a virtual machine creation command, wherein the execute phase creates a virtual machine by employing a prepared virtual machine shell. Furthermore, the aforementioned object is accomplished by a system for supporting creation of virtual machines on a virtualization platform, wherein virtual machine creation is triggered when a virtual machine creation command is issued, the system comprising:

a toolstack being configured to separate its functionality into at least two processing phases, wherein said processing phases comprise a prepare phase and an execute phase;

means for performing the prepare phase independently from virtual machine creation, wherein the prepare phase generates one or more virtual machine shells; and

means for performing the execute phase based upon issuance of a virtual machine creation command, wherein the execute phase creates a virtual machine by employing a prepared virtual machine shell.

According to the invention it has first been recognized that an enormous improvement with regard to the virtual machine creation time can be achieved by removing overhead from the virtual machine creation and boot process. As such, virtual machine creation is triggered when a virtual machine creation command is received. According to the invention, a toolstack is provided that separates its functionality into at least two processing phases, wherein the processing phases comprise a prepare phase and an execute phase. The prepare phase can be performed independently from virtual machine creation, wherein the prepare phase generates one or more virtual machine shells. Furthermore, according to the invention, the execute phase for virtual machine creation is performed based upon the issuance of a virtual machine creation command, wherein the execute phase creates a virtual machine by employing a prepared/pre-created virtual machine shell that has been previously generated by the prepare phase. Thus, a standard toolstack is redesigned in such a way that a split toolstack is provided, wherein the split toolstack separates functionality that does not need to be carried out when a command such as a virtual machine creation command is issued from that which must be carried out when a command such as a virtual machine creation command is issued. As such, functionality that, for example, may be run periodically and/or offline is separated from that which must be carried out when a command such as a virtual machine creation command is issued.

Hence, a method and a system according to the present invention improve the creation of virtual machines with regard to the virtual machine creation time, while retaining the isolation and security afforded by the underlying virtualization platform.

Embodiments of the invention can significantly reduce virtual machine creation times by splitting a virtualization platform's toolstack into

(1 ) a set of prepare phase functions that can be performed independently - e.g., periodically and offline - and (2) a set of execute phase functions that can only be carried out when a virtual machine needs to be created, in particular upon the issuance of a virtual machine creation command.

This split of the toolstack can remove the overhead arising from the prepare phase functions from the boot process of a virtual machine, thus reducing virtual machine instantiation times.

Embodiments of the invention enable to quickly create virtual machines, e.g. in as little as a few milliseconds, while retaining the isolation and security afforded by the underlying virtualization platform.

Further, it is mentioned that embodiments of the invention may also reduce the actual virtual machine creation time when a virtual machine migration is to be performed, which may also involve a virtual machine creation command.

Aspects of the invention may extend to corresponding systems, methods, and computer program products such as computer readable storage media having instructions stored thereon which are operable to program a programmable computer processor to carry out a method as described in the aspects and embodiments set out below or recited in the claims and/or to program a suitably adapted computer to provide the system recited in any of the claims.

The term "virtualization platform" may refer in particular in the claims, preferably in the description, to an entity comprising software packages that can emulate a whole physical computer machine. Specifically, the entity may provide multiple virtual machines on one physical platform. The virtualization platform can be implemented and provided by a software entity/product such as an emulator and/or hypervisor.

The term "toolstack" may refer in particular in the claims, preferably in the description, to at least one interface for hypervisor management and/or configuration of virtual machines. In particular, the toolstack may comprise a collection of tools and/or libraries, which provide interfaces between the hypervisor and the user/administrator of the system to interact and control the hypervisor. Thus, the toolstack may comprise a number of tools and libraries that are built on top of each other to provide certain functionalities. The toolstack being a collection of tools and libraries may be employed and/or may be configured to handle virtual machine management requests such as requesting, by the toolstack, a builder entity to create a new guest virtual machine in response to a user request.

The term "virtual machine shell" may refer in particular in the claims, preferably in the description, to an empty virtual machine or rather to a partially initialized virtual machine for which all basic and generic steps have been done, and which only needs to be filled with an image and some device-specific configuration, e.g. MAC addresses for network devices, to create a fully functional virtual machine. The virtual machine shell can be pre-created before a virtual machine has to be started. Then, the virtual machine shell can be used when a virtual machine creation command is received, to speed up the creation process by only having to execute the execute phase of the toolstack. Hence, the virtual machine shell is a pre-created virtual machine that can be filled with content. The virtual machine shell is a partially-created virtual machine that waits to be filled with execution logic - that is at a minimum an operating system - in order to form a fully functional virtual machine. The term "hypervisor" may refer in particular in the claims, preferably in the description, to computer software, firmware and/or hardware, which creates and runs virtual machines. A hypervisor may also be designated as a virtual machine monitor (VMM).

According to embodiments of the invention the prepare phase of the toolstack may comprise, preferably mainly or only, code functions that are irrespective of - i.e. not specific to - a predetermined and specific virtual machine that is requested by the issued virtual machine creation command. Thus, a significant portion of the code that does not need to run at virtual machine creation time can be pre- executed and thus off-loaded from the virtual machine creation process. Hence, the actual virtual machine creation time when receiving a virtual machine creation command is reduced.

According to embodiments of the invention the execute phase of the toolstack may comprise, preferably mainly or only, code functions, which directly relate to a predetermined and specific virtual machine requested by the issued virtual machine creation command. In other words, the execute phase may comprise code functions that are not irrespective of - i.e. specific to - a predetermined and specific virtual machine that is requested. Thus, the execute phase performs code function calls that can only be carried out when a virtual machine needs to be created. Hence, the actual virtual machine creation time when receiving a virtual machine creation command is reduced.

According to embodiments of the invention the prepare phase may comprise code functions that performs hypervisor reservation. For example, hypervisor reservation may comprise such jobs as creating the internal control structures, allocating information such as a unique identifier (ID) to the virtual machine to be started, and filling that information into those control structures. Thus, the process of hypervisor reservation can be off-loaded from the execute phase and, therefore, actual virtual machine creation time is reduced when receiving a virtual machine creation command. As such, the actual virtual machine creation time can also be reduced when a virtual machine migration is to be performed, which may also involve a virtual machine creation command.

According to embodiments of the invention the prepare phase may comprise code functions that perform compute allocation. For example, compute allocation may comprise creating virtual CPUs and allocating them to the virtual machine. Thus, the process of compute allocation can be off-loaded from the execute phase and, therefore, actual virtual machine creation time is reduced when receiving a virtual machine creation command. As such, the actual virtual machine creation time can also be reduced when a virtual machine migration is to be performed, which may also involve a virtual machine creation command.

According to embodiments of the invention the prepare phase may comprise code functions that perform memory reservation. For example, memory reservation may comprise setting aside memory for the virtual machine in which it can then run. Thus, the process of memory reservation can be off-loaded from the execute phase and, therefore, actual virtual machine creation time is reduced when receiving a virtual machine creation command. As such, the actual virtual machine creation time can also be reduced when a virtual machine migration is to be performed, which may also involve a virtual machine creation command.

According to embodiments of the invention the prepare phase may comprise code functions that perform memory preparation. For example, memory preparation may comprise setting up the page table to allow paged memory access and scrubbing memory if needed, e.g. if memory has been used by other virtual machines before, to prevent information leakage between virtual machines. Thus, the process of memory preparation can be off-loaded from the execute phase and, therefore, actual virtual machine creation time is reduced when receiving a virtual machine creation command. As such, the actual virtual machine creation time can also be reduced when a virtual machine migration is to be performed, which may also involve a virtual machine creation command.

According to embodiments of the invention the prepare phase may comprise code functions that perform device pre-creation. For example, device pre-creation may comprise creating a console and a virtual network card for eventual interfacing with real network hardware. The process of device pre-creation may create the virtual machine shell. The virtual machine shell is a partially initialized virtual machine for which all basic and generic steps have been done, and which only need to be filled with an image and some device-specific configuration, e.g. MAC addresses for network devices, to create a fully functional virtual machine. The virtual machine shells can be pre-created before a virtual machine has to be started, and are then used when a virtual machine creation command is received, to speed up the virtual machine creation process by only having to execute the execute phase of the toolstack. Thus, the actual virtual machine creation time is reduced when receiving a virtual machine creation command.

According to embodiments of the invention the execute phase may comprise code functions that perform configuration parsing. For example, the process of configuration parsing may comprise that the configuration file that describes the virtual machine is parsed. Parameters which describe the virtual machine may include, but are not limited to: virtual machine image, required memory, required device and their configuration parameters, etc. According to embodiments of the invention the execute phase may comprise code functions that perform device initialization. For example, the device initialization may comprise that the devices are initialized according to the information acquired in the process of configuration parsing. According to embodiments of the invention the execute phase may comprise code functions that load a virtual machine image. For example, an image build, i.e. loading the virtual machine image, may comprise that the image, containing the operating system and potentially a virtualized disk that contains further tools, is opened, its format parsed, and its data loaded into the virtual machine memory according to the image's specification.

According to embodiments of the invention the execute phase may comprise code functions that perform virtual machine boot. For example, the virtual machine boot may comprise that the virtual machine finishes its creation, and starts its boot process. At this point, control can be handed over to the virtual machine. Hence, the toolstack's virtual machine creation process and thus the execute phase terminates. According to embodiments of the invention the prepare phase may be performed on demand and/or temporally decoupled from virtual machine creation, preferably such that the prepare phase and, thus, the code functions of the prepare phase are performed periodically and/or offline. Hence, the code functions of the prepare phase may be decoupled form the virtual machine creation such that their operation does not affect the virtual machine performance.

According to embodiments of the invention multiple virtual machine shells may be generated by repeatedly performing the prepare phase, preferably wherein the generation of the multiple virtual machine shells is managed by a daemon that carries out the prepare phase. Thus, it may be kept track of the prepared/pre- created virtual machine shells by the daemon for later use during virtual machine creation in the execute phase. Furthermore, the daemon can be queried in order to receive such a pre-created/prepared virtual machine shell to enable fast creation of a virtual machine.

According to embodiments of the invention, it may be provided that virtual machine shells are generated and placed in a pool for being available in the execute phase. As such, virtual machine shells with different memory amounts may be generated and be held available in the pool. Thus, a number of virtual machine shells with different kinds of properties each can be available when a virtual creation command is received. With regard to the process of memory reservation, which may be comprised by the prepare phase, it may be possible that it is not known in advance how much memory a virtual machine, which is to be created later, might need. Hence, it may be provided that a set of memory allocation buckets (e.g., in increments of powers of 2, i.e., 16MB, 32MB, 64MB, 128M, etc.) is created in the pool and that a predetermined number of virtual machine shells is pre-created for each bucket of the pool. Later, based on data regarding which memory allocation buckets are the most popular, the daemon of the pre-create phase might be tuned to generate more virtual machine shells for the most popular memory allocation buckets.

According to embodiments of the invention, it may be provided that a virtual machine shell associated with a virtual machine that is to be destroyed due to a virtual machine destroy command is re-used. Thus, virtual machine shells may be efficiently provided and generated for the execute phase.

According to embodiments of the invention, it may be provided that means for performing the prepare phase independently from virtual machine creation may comprise a daemon entity. The daemon entity may trigger, control and/or perform the prepare phase such that the prepare phase is performed independently from virtual machine creation. As such, the daemon entity may represent and/or provide a daemon function - i.e. a program running in the background - whose job it is to run the prepare phase and thus prepare a number of virtual machine shells as well as keeping them ready to be used to create new domains by running the execute phase on them. The daemon may be a background server process running inside the management domain, with which the toolstack can interact. According to embodiments of the invention, it may be provided that means for performing the execute phase may comprise one or more tools and/or one or more libraries of the toolstack that are configured to perform the execute phase based upon the issuance of a virtual machine creation command. There are several ways how to design and further develop the teaching of the present invention in an advantageous way. To this end it is to be referred to the patent claims subordinate to patent claim 1 on the one hand and to the following explanation of further embodiments of the invention by way of example, illustrated by the figure on the other hand. In connection with the explanation of the further embodiments of the invention by the aid of the figure, generally further embodiments and further developments of the teaching will be explained. In the drawings

Fig. 1 is a schematic view illustrating an overview of the Xen architecture including toolstack, the XenStore, software switch and split virtual drivers between the driver domain domO and the guests,

Fig. 2 is a schematic view illustrating an architecture overview for a method or a system according to an embodiment of the present invention,

Fig. 3 is a schematic view illustrating a procedural process for virtual machine creation using a method or a system according to an embodiment of the present invention, and Fig. 4 is a diagram illustrating virtual machine creation times of an embodiment according to the present invention compared to the standard virtualization platform Xen.

Fig. 1 shows a Xen architecture including toolstack, the XenStore, software switch and split virtual drivers between the driver domain (domO) and the guest domains. The term "domain" is primarily used when using the Xen hypervisor and is synonymous with the term "virtual machine". Fig. 1 shows several building blocks which may be described as follows:

Building blocks "DomO" (reference sign 1 ) and respectively "Domll 1 " (reference sign 2) represents a domain, which is a virtual machine in the context of the Xen terminology.

In particular "DomO" is the control domain in context of the Xen terminology, from the fact that the first started domain (having number 0) is awarded special privileges, such as being allowed to instruct the hypervisor to start and stop additional virtual machines. Hence, DomO is the administrative domain of Xen. ln particular "DomU" is an unprivileged domain in context of the Xen terminology, i.e. every domain other than "DomO". Building block "Xen Hypervisor" (reference sign 3) represents a system running on a physical machine and which is in charge of, e.g., creating, destroying, scheduling virtual machines. To put it more simply, the Xen Hypervisor is an operating system for operating systems. The Xen Hypervisor may also be designated as virtual machine monitor.

Building block "xl" (reference sign 4) represents a command-line tool that a system administrator can use to run commands such as "create a domain", "destroy a domain", etc. Building block "libxl" (reference sign 5) represents a library that the xl command- line tool interacts with, and which in turn interacts with libraries libxc and libxs to facilitate xen-related tasks such as domain creation or destruction.

Building block "libxc" (reference sign 6) represents a Xen control library. Specifically, libxc is a library that comprises, among other things, the code to interact with the hypervisor, by providing an interface to the various hypercalls required to control the system, in particular with regard to domain created, destruction, etc. A hypercall is a request from a domain to the hypervisor, for information, or for running a certain action on the domain's behalf. Conceptually similar to a system call, in which an application asks the operating system to, e.g., write data to a hard disk: since - in the context of the Xen architecture - the application must not access the hard disk itself for reasons of security and abstraction, it asks the operating to do so on its behalf. Some operations, such as creating or destroying domains, must not be done by a domain itself; it needs to ask the hypervisor to do so on its behalf (and only domO may typically ask for domain creation, for example; such requests by other domains would be rejected). Building block "libxs" (reference sign 7) represents a XenStore library, in particular a library that provides an application programming interface (API) to interact with the XenStore. Building block "SW switch" (reference sign 8) represents a software switch, which is a software implementation realizing network switching between its different ports. The physical network interface card NIC is connected to the virtual network devices via the software switch to allow control over the data flow between physical and virtual devices and for example also between different virtual devices of virtual machines.

Building block "XenStore" (reference sign 9) represents an implementation of an information storage. While the XenStore is primarily an information store, the fact that entities such as a netback driver can register watches to be informed of new information being put into the XenStore means that the XenStore, in a way, can also trigger events happening. So in the case of the XenStore, the netback registers a watch on the part that deals with network devices, so that whenever the toolstack wants to create a new virtual network device, it can simply write the information (e.g., which MAC address the device should have) into the XenStore, and the watch will trigger an event that tells the netback driver of this, which can then create a new virtual network device.

Building block "NIC" (reference sign 10) represents a network interface card (NIC), which is a network adapter used for network communication. The NIC driver is the driver in charge of controlling the NIC.

Building block "block" (reference sign 1 1 ) represents a block device, which is a term for a device that allows reading and writing to it. A typical example can be a hard disk, on which blocks of data are written and read from.

Building blocks "netback" (reference sign 12) and "netfront" (reference sign 13) represent split virtual drivers. A virtual driver may consist of two split virtual drivers, which can be designated as netback driver and netfront driver, respectively. The netback driver is the back-end of the driver for virtual network devices. This portion of the driver exports a unified network-device interface that can be accessed by any operating system, e.g. Linux (reference sign 14), that implements a compatible front-end. For example, when spawning a virtual machine that needs network access, a virtual network device is created. The virtual machine in charge of the physical network interface card (NIC) uses the back-end to create such a virtual network device. In this regard, it is noted that the virtual machine in charge of the physical network interface is generally the privileged control virtual machine, though this does not necessarily have to be the case. The virtual machine that is being created then uses the netfront driver to interact with that virtual device, much like a normal NIC driver would do with a physical NIC. This way, the data from the newly created virtual machine flows via its netfront driver to the control domain's netback, and from there via the NIC driver out of the physical machine. Similar driver concepts exist for block devices, where a blockfront and blockback model can be created to create a virtual hard disk.

Building block "xenbus" (reference sign 15) represents a driver that provides an interface between the XenStore and the drivers for virtual devices, e.g. netback and netfront, which are illustrated in the Fig. 1 , Fig. 2 and Fig. 3 as examples. Thus, the information from the drivers such as how to access the virtual network device can be stored in the XenStore and be read from there.

Building block "toolstack" (reference sign 16) represents a collection of tools and/or libraries, which provide interfaces between the hypervisor and the user/administrator of the system to interact and control the hypervisor. Thus, the toolstack may be a number of tools and libraries that are built on top of each other to provide certain functionalities. The toolstack being a collection of tools and libraries may be employed and/or may be configured to handle virtual machine management requests such as requesting, by the toolstack, a builder entity to create a new guest virtual machine in response to a user request. In the context of the embodiment illustrated by Fig. 1 the toolstack relates to the part of the control flow required to, e.g., create and/or destroy domains, which resides within the user space of the control domain. ln the context of the Xen architecture, e.g. as illustrated in Fig. 1 , the toolstack is a component of DomO, which is responsible for creating, destroying, and managing the resources and privileges of virtual machines. To create a new virtual machine a user provides a configuration file describing memory and CPU allocations as well as device configurations. Then the toolstack parses this file and writes this information in the XenStore. The toolstack takes advantage of DomO privileges to map guest memory, to load a kernel and virtual BIOS, and to set up initial communications channels with the XenStore and with the virtual console when a new virtual machine is created.

In the example of Fig. 1 , the toolstack comprises building blocks xl, libxl, libxc and libxs.

The Xen hypervisor as illustrated by Fig. 1 is a type-1 hypervisor in charge of managing basic resources such as CPUs and memory. When it finishes booting, it automatically creates a special virtual machine called domO. DomO typically runs Linux and hosts the toolstack, which includes the xl command and the libxl and libxc libraries needed to carry out commands such as virtual machine creation, migration and shutdown.

In addition, domO hosts the XenStore, a proc-like central registry that keeps track of management information such as which virtual machines are running in the system and device information, along with the libxs library containing code to interact with it. The XenStore provides watches that can be associated with particular directories of the store and that will trigger callbacks whenever those directories are read or written to.

Typically, domO also hosts a software switch (Open vSwitch is the default) to mux/demux packets between NICs and the VMs, as well as the (Linux) drivers for the physical devices. Strictly speaking, this functionality can be put in a separate virtual machine. For communication between domO and the other guests, Xen implements a split-driver model: a virtual backend driver running in domO (e.g., the netback driver in the case of networking) communicates over shared memory with a front-end driver running in the guests (the netfront driver). So-called event channels, essentially software interrupts, are used to notify drivers about the availability of data.

Fig. 2 shows an architecture overview for a method or a system according to an embodiment of the present invention.

Some of the building blocks illustrated in Fig. 2 correspond to the building blocks of Fig. 1. However, according to the embodiment of Fig. 2, the toolstack is modified compared to the standard platform architecture of Fig. 1. In particular the embodiment of Fig. 2 shows other building blocks as follows:

Building block "chaos" (reference sign 17) represents command-line tool that replaces the building block xl of Fig. 1. Building block chaos provides the same functionality as xl, however, chaos is just more efficient with regards to, e.g., domain creation time, i.e., its use results in smaller times.

Building block "chaos daemon" (reference sign 18) represents a daemon - i.e. a program running in the background - whose job it is to run the prepare phase and thus prepare a number of virtual machine shells as well as keeping them ready to be used to create new domains by running the execute phase on them.

Building block "libchaos" (reference sign 19) represents a library, which in the case of the split toolstack the command line tool chaos and the chaos daemon interact with, and which in turn interacts with libxc and libxs in order to facilitate xen-related tasks such as domain creation or destruction.

Thus, in the embodiment of Fig. 2 the toolstack comprises building blocks chaos daemon, chaos, libchaos, libxc and libxs for providing the concept of a split toolstack (reference sign 20).

The embodiment of Fig. 2 considers and improves shortcomings of conventional virtualization platforms where a significant portion of the overheads related to virtual machine creation and other operations comes from the toolstack itself. More specifically, it turns out that a significant portion of the code that executes when, for instance, a virtual machine creation command is issued, does not actually need to run at virtual machine creation time. This is because this code is common to all virtual machines, and so can be pre-executed and thus off-loaded from the creation process in accordance with the implementation of the embodiment illustrated by Fig. 2.

Hence, according to the embodiment of Fig. 2, the standard Xen toolstack is replaced with the libchaos library and is split into two phases, namely into a prepare phase and a execute phase. The prepare phase is responsible for functionality common to all virtual machines such as having the hypervisor generate an ID and other management information and allocation CPU resources to the virtual machine. This functionality is offloaded to the chaos daemon, which generates a number of virtual machine shells and places them in a pool. The daemon may ensure that there are always a certain number of configurable virtual machine shells available in the system.

The execute phase then begins when a virtual machine creation command is issued. First, chaos contacts the daemon and asks for one of these virtual machine shells to be removed from the pool. It then completes this shell with virtual machine specific operations such as parsing its configuration file, building initialization its devices and booting the virtual machine.

Fig. 3 illustrates a procedural process for virtual machine creation using a method or a system according to the embodiment as illustrated by Fig. 2.

As already described with regard to the architecture of Fig. 1 , one of the key elements of a virtualization platform such as Xen, KVM, VMWare, etc., is the toolstack, which may comprise a command line tool (e.g., "xl" in Xen or "qemu- kvm" in KVM) and a set of libraries in charge of executing operations such as creating virtual machines, destroying them, migrating them, and accessing their console, etc.

The embodiment illustrated by Fig. 3 optimizes virtual machine creation times by means of splitting the toolstack's functionality into a set of functional processes each containing a number of code functions fn1 ... fnN, and further classifying those into

(1 ) a prepare phase that may be executed periodically and offline, and

(2) an execute phase that is executed when the command to create a virtual machine is received.

The prepare phase of the embodiment illustrated by Fig. 3 comprises five functional processes that are common to virtual machines:

(1 ) Process of hypervisor reservation: Hypervisor reservation comprises such jobs as creating the internal control structures, allocating information such as a unique ID to the virtual machine to be started, and filling that information into those control structures.

(2) Process of compute allocation: Compute allocation comprises e.g. creating virtual CPUs and allocating them to the virtual machine.

(3) Process of memory reservation: Memory reservation comprises e.g. setting aside memory for the virtual machine in which it can then run.

(4) Process of memory preparation: Memory preparation comprises e.g. setting up the page table to allow paged memory access and scrubbing memory if needed (if memory has been used by other virtual machines before, to prevent information leakage between virtual machines).

(5) Process of device pre-creation: Device pre-creation comprises e.g. creating a console and a virtual network card for eventual interfacing with real network hardware. This creates the virtual machine shell which is a partially initialized virtual machine for which all basic and generic steps have been done, and which only need to be filled with an image and some device-specific configuration (e.g., MAC addresses for network devices) to create a fully functional virtual machine.

The virtual machine shells generated by the process of device pre-creation can be pre-created before a virtual machine has to be started, and are then used when a virtual machine creation request is received, to speed up the virtual machine creation process by only having to execute the execute phase.

The execute phase of the embodiment illustrated by Fig. 3 comprises the functional processes as follows:

(6) Process of configuration parsing: Configuration parsing comprises e.g. that the configuration file describing the virtual machine (e.g., image, required memory, required device and their configuration parameters) is parsed.

(7) Process of device initialization: Device initialization comprises e.g. that the devices are initialized according to the information acquired in the process of configuration parsing.

(8) Process of image build: Image build comprises e.g. that the image, which contains the operating system and potentially a virtualized disk containing further tools, is opened, that its format is parsed, and that its data is loaded into the virtual machine memory according to the image's specification.

(9) Process of virtual machine boot: Virtual machine boot comprises e.g. that the virtual machine finishes its creation, and starts its boot process. At this point, the control is handed over to the virtual machine and the toolstack's creation step is terminated.

According to the embodiment of Fig. 3, the pre-created/prepared virtual machine shells are kept track of, so that at any point, it is clear which virtual machine shells have been pre-created/prepared and which can thus be used for creating a virtual machine in them. This information can be kept in numerous ways, from a dedicated, fixed location in the hypervisor's memory over a file on the management domains file system. According to the embodiment of Fig. 2 and Fig. 3, the use of a pre-create daemon is provided. The daemon is a background server process running inside the management domain, with which the toolstack can interact.

The daemon will aim to pre-create a number of virtual machine shells defined by the operator, keeping them available for future use. Whenever a virtual machine is to be created, the toolstack requests such a virtual machine shell from the daemon to fill it with the operating system and start the virtual machine. When the daemon runs low on virtual machine shells, it will aim to fill up its virtual machine shell backlog by pre-creating/preparing new virtual machine shells without impeding the performance and behavior of the already running virtual machines, e.g., by pre- creating virtual machine shells during times of low system load.

Fig. 4 shows a diagram illustrating virtual machine creation times of an embodiment according to the present invention compared to the standard virtualization platform Xen. The diagram of Fig. 4 plots a cumulative probability against a domain creation time in milliseconds.

To quantify faster virtual machine creation times that are provided by an embodiment according to the present invention, Cumulative Distribution Function (CDF) is included when creating 1000 virtual machines as fast as possible on a single server using the Xen virtualization platform and a minimalistic virtual machine such as a unikernel. The standard platform (labeled "Standard" in Fig. 4; reference sign 21 ) is compared against the method according to an embodiment of the present invention (labeled "Fastboot" in Fig. 4; reference sign 22). As illustrated by the diagram of Fig. 4, the embodiment according to the present invention clearly outperforms the standard virtualization platform, with practically all virtual machines being created and booted in about 15 milliseconds or less - and some of them in only a few milliseconds - compared to the standard platform (as exemplarily illustrated in Fig. 1 ) which results in a long tail and boot times of over 30 milliseconds.

Many modifications and other embodiments of the invention set forth herein will come to mind to the one skilled in the art to which the invention pertains having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it is to be understood that the invention is not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

List of reference signs

1 DomO

2 DomU 1

3 Xen Hypervisor

4 xl

5 libxl

6 libxc

7 libxs

8 SW switch

9 XenStore

10 NIC

11 block

12 netback

13 netfront

14 Operating systenn Linux

15 xenbus

16 toolstack

17 chaos

18 chaos daemon

19 libchaos

20 split toolstack

21 Evaluation of Fastboot platform

22 Evaluation of Standard platform