A METHOD COMPUTER PROGRAM AND SYSTEM TO OPTIMIZE MEMORY MANAGEMENT OF AN APPLICATION RUNNING ON A VIRTUAL MACHINE

Field of Invention

This invention generally relates to applications executed in a virtual machine such as application developed in Java (Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates) language and executing in a Java virtual machine; more particularly, the invention aims at providing a method to improve memory management of applications running in a virtual machine .

Background Art

An application running in a virtual machine may encounter several kinds of severe conditions that may come from over use of computing resources and in particular of memory. If the amount of memory used by an application exceeds the memory allocated to the virtual machine or if the virtual machine or if the system hosting the virtual machine exceeds the available memory, this can result to an out of memory' exception making the application or the virtual machine to crash. A crash may be very prejudicial for enterprise applications stopped during production hours or if the application in question manages and control computing activity over multiple locations of a same enterprise. Some application may be able to catch the out of memory' exception events from the virtual machine and include code for recovery. Recovery code is very difficult to write because it depends when the event occurs, at which place in the application code. Furthermore, running recovery code upon an out of memory exception may require additional memory resources which may be no more available to the application at this point. The "Hedge" technique" is presented in the presentation of John Tang Boyle Handling "Out Of Memory" Errors, found at:

http://www.cs.uwm.edu/~boyland/papers/memorv-error-talk.p df

Most of such techniques try to manage these situations by pre- allocating a certain amount of memory at the application start-up and releasing it when an out of memory condition occurs to allow the system to be able to accomplish a recovery action. The main drawback of such techniques is that they are reactive. The operation that gets interrupted because of the out of memory error may not be easily recoverable and may leave data inconsistencies if the recovery handler code is not "near" enough to the code that threw the exception. Additionally, trying a recovery action after an out of memory error may be too late for certain kinds of applications. A standard way to prevent out of memory exceptions is to use a garbage collection mechanism for detecting and freeing memory which is no longer in use. A garbage collector program (GC) running on a virtual machine which provides an API to be monitored is activated each time a predefined percentage of memory is in use. Once activated the garbage collector program reclaims memory no longer needed by a given application running on the virtual machine. One advantage is that the garbage collection relives programmers from freeing allocated memory in the application code. One disadvantage of a garbage collector to be taken into account for optimizing memory usage of an application running on a virtual machine is that its execution adds an overhead that can affect the executing application. The garbage collection activity requires more CPU time than would have been required if the application explicitly itself freed unnecessary memory. Finally garbage collection algorithms may be very sophisticated to avoid the drawbacks and may be not only able to free some memory space for one application but also is able to de-fragment the memory space for the virtual machine. The freeing of memory and de ^¬ fragmenting are different functions of the garbage collector which are both provided because de-fragmenting the memory is usually required for having larger blocks of free memory De ^¬ fragmenting is not done in general at each garbage collector cycle but each time the garbage collector decides it is necessary .

Besides the out of memory exceptions, the patent application US 2007/0136402 of the same assignee detects the Memory leak condition for a virtual machine which is a loss of memory available, over time. Detecting the Memory leaks allows predicting when an out of memory exception is likely to occur. This solution can be used to test if applications running on the virtual machine correctly release memory after it has been used. Once again, the remedies to avoid memory leak or out of memory exceptions are just advising the administrator or stopping a process running on the virtual machine.

Finally, low memory conditions are generally very disruptive for the application functionality. Under low memory conditions, the application performances usually degrade a lot because of the increased garbage collector activity. When a virtual machine is running near its upper memory limit most of the CPU time is spent in the garbage collector activity instead of application processing. So it should be avoided as much as possible to have an application running in such conditions . There is thus a need for a method and a system to dynamically optimize the memory usage of an application running on a virtual machine while minimizing the drawbacks described so far. Such a method could be used to optimize for instance memory management of a Java (Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates) application. The Java Virtual Machine (JVM) provides a set of bytecode instructions, the applications written in Java language being compiled into such bytecodes. The JVM allows interpretation of bytecodes independently from the underlying hardware and operating system.

Summary of the Invention

It is an object of the present invention to provide a method for optimizing memory usage of an application running on a virtual machine.

The object is reached, according to claim 1, with a method for optimizing memory usage of an application running on a virtual machine, said virtual machine providing a monitoring API to communicate with the virtual machine and said virtual machine including a garbage collector, said method comprising:

- self-allocating a virtual machine memory block;

- periodically collecting and storing virtual machine memory usage statistics;

periodically computing an average virtual machine memory usage from the memory usage statistics in a time window;

- if the computed virtual machine memory usage average becomes higher than a first threshold, entering into a recovery mode by releasing the virtual machine memory block and forcing the running application to reduce its processing activity; - if the computed memory usage average becomes lower than a second threshold, entering into a normal mode by re-self- allocating the virtual machine memory block and forcing the running application to resume its normal processing activity

The object is also reached, according to claim 2, with the method of claim 1 further comprising an initial step of setting a virtual machine state indicator to normal mode, and wherein the steps of entering into a recovery mode and entering into a normal mode further comprise:

- if the virtual machine state indicator is set to normal mode and if the computed memory usage average exceeds the first threshold performing the following steps:

- switching said state indicator

from normal mode into a recovery mode;

releasing the self-allocated memory block to the virtual machine;

- notifying the running application that a low memory condition occurs to force said running application to reduce its

processing activity;

- returning to the periodically computing step;

- if the virtual machine state indicator is set to normal mode and if the computed memory usage average does not exceed the maximum threshold returning to the periodically computing step;

- if the virtual machine state indicator is set to recovery mode and if the computed memory usage average descends below a second threshold:

- switching said state indicator from recovery mode into normal mode;

- self-allocating a virtual machine memory block; - notifying the running application that a low memory condition occurs to force said running application to resume its normal processing activity;

- returning to the periodically computing step;

- if the virtual machine state indicator is set to recovery mode and if the computed memory usage average does not descend under the second threshold, returning to the periodically computing step.

The object is also reached, according to claim 3, with the method of claim 1 or 2 further comprising if the virtual machine memory usage average becomes higher than the first threshold, forcing the virtual machine to perform a garbage collector cycle.

The object is also reached, according to claim 4, with the method of any one of claims 2 to 3 wherein the running application performs an initial callback request to receive notification and wherein notifying the running application consists in sending a callback notification to the running application . The object is also reached, according to claim 5, with the method of any one of claims 1 to 4 wherein in the steps of comparing the computed memory usage average to a first threshold and to a second threshold, the first threshold value is greater than the second threshold value.

The object is also reached, according to claim 6, with the method of any one of claims 1 to 5 wherein, in the steps of comparing the computed memory usage average to a first threshold and to a second threshold, the first threshold and the second thresholds are set as configuring parameters.

The object is also reached, according to claim 7, with the method of any one of claims 1 to 3 further comprising:

- testing if the virtual machine is idle; and,

- if the virtual machine is idle, forcing the virtual machine to perform a garbage collector cycle.

The object is also reached, according to claim 8, with the method of claim 7 wherein testing if the virtual machine is idle comprises:

periodically collecting and storing virtual machine CPU usage and garbage collector activity statistics;

- periodically computing an average virtual machine CPU usage and an average garbage collector activity from the collected and stored statistics;

- testing if the computed virtual machine CPU usage average is lower than a configurable third threshold and if the garbage collector activity is null.

The object is also reached, according to claim 9, with the method of any one of claims 1 to 8 wherein the steps are performed in a Java Virtual Machine and the monitoring API is JMX management interface.

The object is also reached, according to claim 10, with a system comprising means adapted to carry out the steps of the method according to any one of the preceding claims.

The object is also reached, according to claim 11, with a computer program comprising instructions for carrying out the steps of the method according to any one of the preceding method claims when said computer program is executed on a computer .

With the solution of the invention, a pro-active system continuously monitors the available memory and the Garbage Collector activity for an application running on a virtual machine including a garbage collector function and providing an API for computing resource monitoring. The main advantage of the solution is that the system not only detects that the virtual machine reaches a low memory condition but also takes appropriate corrective actions to prevent the system from reaching and exceeding its limits.

To achieve this goal the method of the invention first takes advantage of the period when the virtual machine is "idle" to start garbage collection activity. Also, the system detects a low memory condition when the garbage collector activity is increasing for a certain amount of time while the available free memory is continuously decreasing under a certain threshold; it notifies the application which initiates a set of activities aimed at reducing the overall processing activities upon receiving such notification. A "Hedge technique" is used by pre-allocating, at the application start-up, a certain amount of memory to be released and used in case a recovery action is required: when the application is notified of low memory condition, the system releases the memory allocated at startup and forces a recovery action from the application. The system also notifies the application in case the available free memory comes back and re-exceeds a specified threshold, it notifies the application about this condition. Upon receiving such notification the application code resumes its normal activities and the system additionally, re-allocates the memory which was previously released . By combining the release of memory allocated to the application at start-up time to the application activity reduction, the garbage collector will benefit of a larger amount of free memory to optionally de-fragment the system memory space and the application may execute the activity reduction with a larger amount of free memory, thus avoiding- an out of memory exception.

Reference to the drawings

Figure 1 illustrates the system environment in which the method optimizing the memory usage of a Java application is implemented according to the preferred embodiment of the invention;

Figure 2 is the flow chart of the method for optimizing the memory usage of a Java application according to the preferred embodiment;

Figure 3 is a flowchart of an optional process in the method for optimizing the memory usage of a Java application according to the preferred embodiment;

Figure 4 shows the curves representing the values varying with the time of the memory usage and garbage collector activity of a running Java application when its memory usage is optimized according to method of the preferred embodiment.

Detailed Description of the preferred embodiment

Figure 1 illustrates the system environment in which the method optimizing the memory usage of a Java application is implemented according to the preferred embodiment of the invention. Figure 1 illustrates one common example of virtual machine which is a Java virtual machine (100) operating on a computer not illustrated here. The runtime executes bytecodes, compiled Java language code, on said computer of a Java language compiled application (120) . The Java virtual machines comprises a garbage collector (included, not represented in Fig. 1) and provides an API for monitoring computing resources and more particularly, memory and CPU as well as the activity of the garbage collector, which allow implementation of the application memory usage optimization.

The new components which are part of the preferred embodiment of the system of the invention are computer programs respectively executing JVM computing resource monitoring (JVM Monitoring Agent 140) and memory optimization (Memory Optimizer 130) . It is noted that these two components performing the memory optimization process can be located outside the virtual machine in a remote server and can be connected to the virtual machine through the network. Two pieces of code (125, 127) in the application are used to implement notifications by the Memory optimizer (125) and for reducing/restoring the application activity in case of notification of low memory by the Memory optimizer as described hereunder in reference to description of the next figures .

It is noted that this illustration of the system of the preferred embodiment of the invention comprises only one Java application: this is the most common pattern having one application running on a virtual machine. As a matter of fact, running more than one application on the same virtual machine may have several drawbacks in the context of fail over and tolerance. This architecture applies also particularly to the type of applications supporting no unexpected interruption as running in production mode in an enterprise or implementing, for instance, a constant management and control of systems over enterprise locations. These applications have been modified (125, 127) according to the preferred embodiment of the invention, to support the memory management optimization.

By the way, because the garbage collector is part of the virtual machine and does not distinguishes between the memory allocated by one application and by another one, when the memory is freed, it is done not at the level of one application but at the level of the entire virtual machine. More generally the VM monitoring is made at the level of the entire virtual machine. Consequently, in the case there are more than one application running on a virtual machine, the system to optimize memory management may be implemented but its effectiveness is reduced. For instance all the applications running on the JVM may be enabled to interact with the Memory Optimizer component: they will receive the low memory notifications and will take appropriate actions to reduce memory usage. One other option would be to enable only one set of applications for receiving notifications from the Memory Optimizer.

The JVM Monitoring Agent (140) collects the statistics (170) (garbage collector activity) memory and CPU usage ...) required by the Optimizer for its internal processing. In the modern JVM runtimes it may retrieve such data using the JMX management interface that almost all the vendors provide. The Memory Optimizer is the engine that analyzes and processes the data received from the Monitoring Agent and that, according to its configuration, decides about the corrective actions to be taken for maintaining the availability of the monitored application via notifications to the application (150) and releasing to JVM of the pre-allocated memory block. The Memory Optimizer can also force the garbage collector to execute a garbage collector cycle (180) when the JVM is idle to anticipate future use of the memory by the application. Forcing a garbage collector cycle in this circumstance is optional because it may be not necessary.

Figure 2 is the flow chart of the method for optimizing the memory usage of a Java application according to the preferred embodiment. The example of a Java virtual machine is still used in the description of the steps of the method of the preferred embodiment.

A new Java application for which the memory usage will be optimized according to the preferred embodiment gets started along with the Memory Optimizer engine and the JVM Monitoring agent (200) . As soon as the Memory Optimizer engine gets started, it pre-allocates (205) a certain configurable amount of memory to be used during the "recovery" phases. The size of the pre-allocated block of memory may be a configurable parameter of the Memory Optimizer, the value depending on the particular application and system context. The parameter value may be decided by the system administrator upon his experience of the computing environment.

The application executes (210) the code for requesting to be notified by the Memory Optimizer engine about "Low Memory" conditions: in the preferred embodiment, the application registers a callback notification object to the Memory Optimizer engine to be notified. Callback is the typical way of sending notification between two programs.

The JVM Monitoring Engine, at regular time intervals, contacts the JVM using the JVM monitoring API to collect metrics (215) about the available/used memory, the JVM Garbage Collector activity and the process CPU usage; these data are periodically provided by the JVM Monitoring Engine to the Memory Optimizer. There are many possibilities for implementing this process of collecting data as well known in the art. For instance, the Memory Optimizer at regular intervals may ask the Monitoring agent to retrieve JVM metrics. Another possibility could be that the Monitoring agent, independently from the Memory Optimizer, periodically gets the JVM parameters and send them back to the Optimizer through a notification mechanism. The Memory Optimizer when getting these statistics stores them in its internal historical repository. A separate thread into the Memory Optimizer at regular intervals, in general different from the time intervals used by the JVM Monitoring Agent to send the data, infers the resource utilization (memory, CPU and GC activity) from an historical analysis of the memory and GC metrics: it calculates (220) the average values for those parameters using an evaluation window that starts from the current timestamp and goes back in time up to a pre-defined configurable limit, as illustrated later on in this document is relation with the description of Figure 4.

At first run of this flowchart, when the application starts running, the JVM is ^λ ίη normal mode' (answer yes to test 225) . A JVM state indicator (a programming variable for instance) can be set by the Memory Optimizer to a certain value representing ^λ normal mode' . When the Memory Optimizer detects that the calculated average "used" memory exceeds (answer yes to test 230) a pre-defined threshold Th2 (430), at this point the JVM enters into a 'recovery mode' (240) . The JVM state indicator is set to 'recovery mode' corresponding to a certain value.

It is noted that, the average memory usage can be also satisfying and no action is started if the calculated average "used" memory does not exceed (answer No to test 230) the Th2 threshold; in this case, the process goes back to next measurement after new metrics collected (215, 220) . When switching to recovery mode (back to step 240) , the Memory Optimizer releases the pre-allocated memory block (250) to the runtime environment; the Memory Optimizer also triggers a notification (260) of low memory condition to the running application by mean of the registered callback object.

Upon receiving the "low memory" condition notification the application starts a set of "recovery" actions aimed at reducing its internal processing activities until the low memory condition get normalized.

The memory Optimizer has released the pre-allocated memory to the JVM to help the application to perform the activity reduction and to help the garbage collector action of freeing memory. Additionally, the Memory Optimizer as recovery action may force a garbage collector cycle in the JVM if it appears necessary .

Back to the regular collect of metrics and calculation of an average memory usage (215, 220) in an Evaluation window, as the JVM is still in 'Recovery mode' (answer No to test 225) , the efficiency of the recovery actions is tested. A certain number of measurement periods may be necessary (answer No to test 265) , but when, after a certain number of loops, the average "used" memory finally descents (answer Yes to test 265) below the alert threshold Thl (440), the JVM must return to 'Normal mode' (270), the memory optimizer sets the JVM state indicator to normal mode. The Memory Optimizer re-allocates memory block for preparing next recovery in case of low memory condition occurs; it also sends a notification to the running application (290) that the mode is back to normal situation. The use of the two thresholds implements a sort of hysteresis mechanism to avoid immediately reacting when small oscillations of the memory usage occur around the configured limit. Upon receiving the notification that JVM is now in normal mode, the application resumes its normal internal processing activities until the low memory condition get normalized.

As already mentioned before, the application includes some code for supporting the optimization of memory usage which includes code for receiving notifications from the Memory Optimizer component and code for reacting at reception of the notifications by reducing the application activity (reading, writing computing may use a lot of memory) or restoring its expected normal processing upon reception of a notification of mode returned to normal situation sent by the Memory Optimizer component .

Figure 3 is a flowchart of an optional process in the method for optimizing the memory usage of a Java application according to the preferred embodiment. This optional process is executed in parallel (on a separate thread in the JVM) to the general flowchart as described in Fig. 2.

Asynchronously from the previous activities, the Memory Optimizer periodically collects (300) (through the JVM monitoring agent) and computes (310) an average of the CPU usage and garbage collector activity of the JVM in an Evaluation Window. When it detects (answer Yes to test 320) that the application is almost idle (i.e. no garbage collector activity and low CPU usage) , it triggers (330) a preventive Garbage collector cycle to free as much unused memory as possible when it is not detrimental to application activity. If the application is not idle, this parallel process returns to next metrics collect and check to start or not this preventive memory cleaning. The time interval for the check of idle periods is different from the one used to check the low memory conditions and in general it is represented by a larger value (i.e. this kind of check is performed less frequently that the one to check the memory usage) . This specific preventive memory cleaning is optional that is not required in all cases. It mainly depends on how the garbage collector system has been implemented in the virtual machine. The detection of an idle period (i.e. no garbage collector activity and low CPU usage) may be tuned through the use a Memory Optimizer parameter by the system administrator.

Figure 4 shows the curves representing the values varying with the time of the memory usage and garbage collector activity of a running Java application when the memory usage is optimized according to method of the preferred embodiment. The memory usage curve (400) provides the memory usage parameter values varying with the time on the x axis (410) . The max heap size is the maximum value of the memory used by the Java application for creating a condition of out of memory exception (Heap is an area of memory reserved for data that is created when a program is executed) . This max value (420) is never reached with the memory usage optimization of the invention provided that the applications take timely recovery actions when notified about the low memory condition. Furthermore, as illustrated in Fig. 4, the Th2 threshold, which represents a memory usage value for a low memory condition and which starts the process for recovery, is a maximum value of the memory usage, even if, the Th2 value may be in general exceeded during the recovery phase since a few moments have to elapse for the recovery actions done by the application to have a tangible effect. The garbage collector curve (450) represents variation with the time of the garbage collector activity which is also measured. Periodically, the Memory Optimizer computes the Java application memory usage in a window of time, the Evaluation Window (360) . In the Evaluation Window of Fig. 4 the Memory Optimizer computes the average value of the memory usage and as it has reached the Th2 threshold for low memory condition, it will start action for recovering from this situation. It is noted that, as expected, during this Evaluation Window the garbage collector activity becomes very high and needs more memory for freeing the Java application unused memory in this situation.

The recovering action triggered by the Memory Optimizer in case of low memory condition is detected consists in notifying the running application which will reduces its activity to decrease its CPU consumption and memory usage; also the Memory Optimizer releases the pre-allocated memory block for its use by the JVM and particularly by the garbage collector of the JVM. The release of memory helps the garbage collector and the application to more safely and efficiently complete their tasks during the memory peak periods. These recovery actions achieve two goals:

- Helps the garbage collector that, with a larger amount of free memory, is able to more efficiently de-fragment the heap space .

- Helps the application that, with a larger amount of free memory can safely perform its "activity reduction" actions without incurring in out of memory exception.

As an additional proactive action the optimizer may periodically force a garbage collector cycle. This is not strictly needed and in some cases may not be necessary. For example good garbage collectors may directly do this activity by themselves, while others ones do not do this.

When, in a next Evaluation Window the Memory Optimizer computes that the average memory usage for the Java application is under an alert threshold Thl (440), it notifies the application which returns to normal activity. Additionally, the Memory Optimizer re-allocates the block of memory previously released.

The Thl memory usage alert threshold value is lower than the Th2 low memory condition threshold value. Thl and Th2 being determined by the experience of the system administrator .

Not illustrated in Fig. 4, the preventive action of the memory optimizer to optimize memory usage is for forcing deep garbage collection cycle during low CPU activity.

It is noted that the applications which have their memory usage managed by the method of the invention have code included to interface with the Memory Optimizer (callback is the most usual way to implement notification between programs) and code to reduce activity (CPU and memory usage) and code to re-establish normal activity.

Compared to the existing solutions in the field for optimizing memory usage of applications running on a virtual machine providing a monitoring API and including a garbage collector, this system is pro-active and adds the following new elements :

Historical and combined analysis of both CPU Usage, GC activity and free memory availability to decide about the corrective actions to be taken (forced GC cycle, application notification)

- Notification mechanism based on a set of two different pairs of alert thresholds (with an hysteresis mechanism) that allows notifying the application not only when an alert condition arises, but also when the system comes back to a "normal" condition so that the application can resume its full functional state .

- The pre-allocation and release of a memory block to help the garbage collector and the application to more safely and efficiently complete their tasks during the memory peak periods.