STATEMENT REGARDING GOVERNMENT FUNDING

The project leading to this application has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No 814035.

TECHNICAL FIELD

The present disclosure is generally related to improving software security and reliability, and is more particularly related to techniques for securely rolling back execution of software executing in multiple software fault isolation domains.

BACKGROUND

Software that comprises multiple co-operating software modules is susceptible to flaws in individual modules, which may compromise the operational integrity of the entire application. A prominent example of such software flaws is memory vulnerabilities in software written in memory-unsafe programming languages, e.g., C and C++.

There are many different approaches designed to mitigate the impact of vulnerabilities, including software fault isolation (SFI), which aims to isolate the effects of flaws in software modules within a single process. SFI is described in Robert Wahbe, Steven Lucco, Thomas E. Anderson, and

Susan L. Graham, Efficient software-based fault isolation, SIGOPS Oper. Syst. Rev., 27(5):203-216, Dec. 1993. Another approach is the use of run-time defenses, such as control-flow integrity (CFI), which aim to detect specific classes of attacks against software flaws before they can be leveraged to take control of the victim application. CFI is described in Martin Abadi, Mihai Budiu, Ulfar Erlingsson, and Jay Ligatti, 2005, Control-flow integrity, in Proceedings of the 12th ACM conference on Computer and Communications Security (CCS ’05), Association for Computing Machinery, New York, NY, USA, 340-353.

A recently emerging trend is for processor manufacturers to incorporate, into commercial, off-the- shelf processors, hardware-assisted security technologies that harden software against run-time attacks. Examples of deployed technologies include MPK/PKU (Memory Protection Keys / Protection Keys for Userspace) and CET (Control-flow Enforcement Technology) for Intel CPUs, as well as PA (Pointer Authentication), MTE (Memory Tagging Extension), and BTI (Branch Target Indicators) for ARM processors. These technologies mitigate the security impact of vulnerabilities in applications written in memory-unsafe languages, e.g., C and C++. Under the standard threat model for run-time attacks, the attacker is assumed to be able to corrupt arbitrary memory within the application process by leveraging software vulnerabilities. With SFI and defenses based on attack detection, even when such invasive attacks are detected, any memory objects resident in the process memory may have already been corrupted. Consequently, to recover from a detected attack the software process must be terminated and restarted from a known good initial state. This means that even if the application has been hardened against software attacks and an attack is detected, the response can still be leveraged by attackers to create temporary denial-of-service conditions while the application (or system) is restarted. For service-oriented applications in cloud environments, this can translate to service disruptions that affect a large number of clients. In another example, long-running computeintensive applications may lose acquired intermediate computational results because of software flaws that lead to termination and may need to recalculate significant portions of the computation after application restart.

Accordingly, further improvements to software security and resistance to attacks are needed.

SUMMARY

The problems discussed above are addressed by several embodiments of the techniques, apparatuses, and systems described herein. In various embodiments, software modules within an application are compartmentalized into distinct software fault isolation domains that may be hardened against software attacks. Each such domain executes code within an isolated memory compartment inside the application process. When the execution flow of the application transfers between two such domains, the state of execution of the domain being transferred from is recorded, thus creating a recovery point that can later be restored. The application can be securely restored to an existing recovery point because any adverse effects of software flaws triggered within an isolated module are limited to the module's isolated memory compartment. Consequently, the application can leverage the proposed solution to recover from unexpected errors by discarding any affected compartment and resuming execution from a previously created recovery point in a good state in a previously left domain. As a result, the application software can recover from unexpected software flaws that might otherwise be fatal and require a restart of the application process.

An example method, for software that comprises two or more software modules compartmentalized into two or more distinct software fault isolation domains, comprises the step of detecting that execution flow of the software has transferred or is transferring from a first domain of the two or more distinct software fault isolation domains to a second domain of the two or more distinct fault isolation domains. The method further comprises the step of saving data recording a state of execution of the first domain, responsive to said detecting. Subsequently, when execution returns to the first domain, the saved data may be used to restore the state of the first domain to the saved state, depending on whether a fault or other abnormality is detected, e.g., using an attack detection defense.

Variations of the above-described method are detailed below, as are corresponding apparatuses and systems.

BRIEF DESCRIPTION OF THE FIGURES

Figure 1 is a block diagram providing a system overview.

Figure 2 is a flowchart showing an example SFI domain lifecycle.

Figure 3 shows domain nesting, illustrating different possibilities for rolling back execution.

Figure 4 illustrates details of rollback configurations for nested domains.

Figure 5 is a process flow diagram illustrating an example method, according to some embodiments or instances of the techniques detailed below.

DETAILED DESCRIPTION

As briefly discussed above, even if a software application has been hardened against software attacks and an attack is detected, the time and resources necessary to respond to the attack can still be leveraged by attackers to create temporary denial-of-service conditions, e.g., while the application (or system) is restarted. For service-oriented applications in cloud environments, this can translate to service disruptions that affect many clients. In another example, long-running compute-intensive applications may lose acquired intermediate computational results because of software flaws that lead to termination and may need to recalculate significant portions of the computation after application restart.

System designs that aim to provide resilience against invasive attacks typically introduce redundancy by running multiple, artificially diversified, variants of the same application, in tandem, and monitoring each distinct application copy for divergent behavior. Such N-Variant Execution (NVX) systems can either be instantiated on a single host system, through virtualization or emulation technologies, or distributed across multiple discrete host systems. In both cases, however, NVX introduces considerable run-time overhead because of increased system load and the need to replicate input/output across each variant application instance. This makes existing NVX-based solutions impractical for applications that are sensitive to the degradation of performance and/or throughput.

Application checkpoint-restore is a technique that involves saving the state of a running process periodically or before a critical session, so that the process can later be restarted from the checkpoint in case of failure. The cost of this technique depends on how much data is needed for the checkpointing and what the checkpointing interval is. Checkpoint-restore can be performed on the system-level or at the application-level. At the system level, the checkpoint cannot distinguish between persistent and temporary data, which leads to cost increases. At the application level, checkpointing needs additional user effort. The protection of integrity and confidentiality of checkpoint data at rest can also raise additional challenges. While certain cryptographic solutions can be deployed, this may lead to practical challenges and increases in computational overhead based on checkpointing intervals.

Hardware-assisted Software Fault Isolation is another approach to protect the confidentiality and integrity of sensitive data or code within an application. Intel MPK and ARM Memory Domains are two state-of-art hardware features. Based on Intel MPK, ERIM and Donky are two research projects that propose in-process isolation guarantees with low overhead to isolate an application's trusted components from its untrusted components. ERIM is described in Anjo Vahldiek- Oberwagner, Eslam Elnikety, Nuno O. Duarte, Michael Sammler, Peter Druschel, and Deepak Garg, ERIM: Secure, Efficient In-process Isolation with Protection Keys (MPK), in Proceedings of the 28 ^th USENIX Security Symposium (Security), Santa Clara, CA, August 2019. Donky is described in David Schrammel, Samuel Weiser, Stefan Steinegger, Martin Schwarzl, Michael Schwarz, Stefan Mangard, and Daniel Gruss, Donky: Domain keys-efficient in-process isolation for RISC-V and x86, in Proc. USENIX Security, 2020. Besides in-process isolation, these projects include mitigation techniques to prevent adversaries from bypassing the isolation scheme, such as binary inspection and hardware watchpoints. But, while these isolation methods ensure the memory safety of the isolated parts, the recovery mechanism from a memory flaw in any isolated domains is still an open issue. Based-on ARM Memory Domains, pFiles proposes intra-process isolation to protect or share threads' memory compartments both from the untrusted code within itself as well as from any untrusted thread. pFiles is described in Yudan Liu, Raja Nassar, Chokchai Leangsuksun, Nichamon Naksinehaboon, Mihaela Paun, and Stephen L. Scott, An optimal checkpoint/restart model for a large scale high performance computing system, in 2008 IEEE International Symposium on Parallel and Distributed Processing, pages 1-9, 2008. With this approach, however, in case of any attempt to access protected data from the untrusted side, the application must abort execution, even if the execution is multithreaded.

In the techniques described herein, solution software modules within an application are compartmentalized into multiple, distinct, software fault isolation domains that may be hardened against software attacks, e.g., using hardware-assisted SFI technologies. Each such domain executes code within an isolated memory compartment inside the application process.

When the execution flow of the application transfers between two domains, the state of execution of the domain being transferred from is recorded, thus creating a recovery point that can later be restored. The application can be securely restored to an existing recovery point because any adverse effects of software flaws triggered within an isolated module are limited to the module's isolated memory compartment. Consequently, the application can leverage the proposed solution to recover from unexpected errors by discarding any affected compartment and resuming execution from a previously created recovery point in a good state in a previously left domain. Abnormal events, such as the detection of an attack, can be flagged, upon return to the recovery point, so that actions to prevent recurrence of the problem can be taken. As a result, the application software can recover from the unexpected exploitation of software flaws, which might otherwise be fatal and require a restart of the application process.

The software fault isolation domains can be "nested", i.e., code already executing within a domain can itself create new domains to compartmentalize additional code paths. The purpose of nesting is to create multiple possible recovery points that can be the target of a recovery process. Examples of code paths which can be isolated include (but are not limited to):

• subroutines that perform input validation and sanitization of external, untrusted input;

• subroutines in third-party software libraries; and

• subroutines that operate with sensitive data, e.g., cryptographic keys.

At a high-level, the techniques described herein might be characterized as "software exceptions with isolation guarantees."

Figure 1 shows a system overview of the proposed solution. The software fault isolation domains for the application 100 in this example are organized as follows. First, the root domain 110 is the initial security domain the application 100 starts in. The entry point to the application code begins its execution in the root domain 110. Initially all application memory, i.e., the stack 112, heap 114, and global data 116, are assigned to the root domain. Next are subsequent nested domains 120A, 120B, etc., that compartmentalize application functionality. When a nested domain is created it is assigned a substack area 122 and subheap area 124 that are distinct from the regular application stack and heap areas. Software Fault Isolation (SFI) is used to ensure that the code running in a nested domain is limited to operate on the substack and subheap areas assigned to it. In some instantiations, SFI may allow nested domains to be assigned limited (for instance read-only) access to memory areas that belong to other domains or global memory that belongs to the root domain.

The SFI can be realized using hardware-based memory isolation technology, such as Intel PKU or ARM MTE. In some instantiations the SFI may require the incorporation of a reference monitor 150 for domain transitions, which may execute in the root domain 110 or in a distinct monitor domain separate from the root and nested domains. In other instantiations the monitor functionality may be "inlined" and part of the regular application code, rather than a discrete software component.

The reference monitor 150 is responsible for reconfiguring the hardware support for software fault isolation 166 to enforce an access control policy on memory accesses that achieve memory isolation between the application's active domains. The reference monitor 150 is also responsible for creating the saved execution state 130, 140 for the corresponding domains, when transferring execution from one domain to another. The saved execution state is stored within the corresponding domain, e.g., saved execution state 130 is stored in the root domain, and saved execution state 140 is stored in the nested domain. Alternatively, in instantiations that incorporate a separate monitor domain, the saved execution state may be stored in the monitor domain.

Figure 2 shows a flowchart depicting an example lifecycle of nested domains. The illustrated flow begins, at block 210, with the parent domain active. As shown at block 215, a nested domain is created.

Nested domains can be created at any point during the application's execution when the execution flow enters a subroutine that should operate isolated from the application's other functionality. Each nested domain has one parent domain, which is responsible for creating the nested domain. A nested domain d may have zero or more child domains, i.e., additional nested domains created by the nested domain d.

When a nested domain is created it is assigned a substack area and subheap area indicated in the flowchart as Domain Data 220. A nested domain may be designated as transient or persistent when created. Memory areas assigned to transient nested domains persist until the application's execution flow returns from the transient nested domain to the parent domain at which point unused memory areas assigned to the transient nested domain are discarded. Any active allocations in the transient nested domain's heap area are merged back to the parent domain's heap area. Persistent domains, on the other hand, retain any assigned memory areas even after the application's execution flow returns from the persistent nested domain to the parent domain. Another code path may enter the persistent domain again, at which point access is granted to memory areas that belong to the persistent domain.

Block 225 of the flowchart of Figure 2 shows the application entering the nested domain. When the execution flow of the application enters a nested domain, a domain transition occurs. During a domain transition the reference monitor performs the following operations:

• information about the execution state of the parent domain, such as the stack pointer, the instruction pointer, possibly the values of other registers and the signal mask is saved into the parent domain's memory, or in the monitor domain's memory, if a distinct monitor domain is used. This saved parent execution state 230 can later be used by the recovery process to restore the application's state to the point prior to entering the nested domain.

• the memory access control policy enforced through SFI is updated to grant access to memory areas assigned to the newly entered domain, and to prevent access to other memory areas.

• finally, the entry point to the software module to be run within the nested domain is invoked and control is transferred to the nested domain.

As shown at block 235, the nested domain is then active. While the nested domain is active, it executes code that belongs to a software module within the application. Code inside a nested domain can be hardened using software instrumentation to detect unexpected error conditions, such as memory errors.

As shown at block 240, an exit of the nested domain is triggered. A nested domain can end its execution in one of two ways: 1) normal domain exit, and 2) abnormal domain exit. A normal domain exit occurs when the application's execution flow returns naturally from the code invoked inside the domain back to the parent domain. This means that the isolated code has completed successfully and application execution is resumed outside the domain. An abnormal domain exit occurs when an unexpected error is encountered during the execution of code inside the domain. This can mean that the isolated code tried to access memory past the confines of the domain's memory area, or that a possible run-time attack was detected.

During normal domain exit, as shown at block 245, the reference monitor 150, may perform the following operations:

• the information about the execution state of the parent domain is discarded after the return to the parent domain, if the nested domain being exited is a transient domain. In the case of a persistent domain, this execution state may be retained, in some embodiments or instances, to facilitate later responses to an abnormal domain exit.

• the memory access control policy enforced through SFI is updated to grant access to memory areas assigned to parent domain, and to prevent access to other memory areas.

Other operations depend on whether the exited domain is a transient domain or a persistent domain. If the nested domain being exited is a transient domain the following additional operations may be performed by the reference monitor 150:

• discard the substack area assigned to the transient nested domain.

• merge any active allocations in subheap area assigned to the transient nested domain to the parent domain's (sub)heap area, as shown at block 250.

• discard the unused subheap area assigned to the transient nested domain.

If the nested domain being exited is a persistent domain, on the other hand, its substack and subheap areas are retained, as shown at block 255. If another code path enters the persistent domain again, access is granted to memory areas that belong to the persistent domain.

In both cases of normal exiting of the nested domain, i.e., from a persistent domain or a transient domain, execution returns to the parent domain, so that the parent domain is then active, as shown at block 260.

If an unexpected error occurs during the execution of code while in a nested domain, then abnormal domain exit occurs, as shown at block 265. In this case, the reference monitor 150 may perform the following operations:

• execution of that code in the nested domain is halted and a prior state of the parent domain is restored.

• in some cases, the application's control flow is resumed at the parent domain as if the application's execution flow had returned from the nested domain during a normal domain exit. In other words, the state of the parent domain as recorded upon entering the nested domain, is restored. This is the only option for a transient nested domain, but may also apply to a persistent domain.

• in other cases, the application's control flow is resumed at the parent domain at an earlier point, i.e., to a state of the parent domain as recorded at an earlier point, prior to the most recent entry into the nested domain, such as a state recorded upon a previous entry into the nested domain, from which control was returned normally.

• the subheap area and substack area assigned to the nested domain are discarded.

In some embodiments, the calling environment, i.e., the domain to which execution is transferred upon exit from the nested domain, learns of the nested domain's exit status through a return value that indicates whether a normal or abnormal domain exit occurred. On an abnormal domain exit, the application is expected to take some alternate action to avoid the conditions that lead to abnormal domain exit before retrying the operation. For example, a server application can close the connection to a potentially malicious client.

While the flow illustrated in Figure 2 shows that execution after an abnormal exit from a nested domain returns to the parent domain that created that nested domain, i.e., its immediate parent, that need not always be the case. Figure 3 illustrates different configurations for rollback from nested domains.

Again, the root domain is the default domain from which execution begins at application startup. Nested domains are organized as a stack-like structure where the nesting level of a domain indicates the number of nested domains that separated it from the root domain. A nested domain may be referred to as a "higher" domain, relative to the domain from which it was entered, because the nesting level is higher. Likewise, the parent to a nested domain (or the root domain) may be referred to as a "lower" domain, with respect to the nested domain.

A normal domain exit always unwinds the nested domain stack to the previous nested domain, i.e., to the domain from which execution was previously transferred to the domain now being exited. This previous nested domain is the next lower domain.

In the event of an abnormal domain exit, however, the execution state of any nested domain (nested domain #1 - nested domain #n) might be rolled back to its immediate parent, or to an established recovery point in the root domain, or to an established recovery point in a nested domain at a lower nesting level than the immediate parent domain. All of these options are illustrated in the top portion of Figure 3, as dashed lines. The exact targets for an abnormal domain exit will depend on the application and the particulars of the compartmentalized modules. Figure 4 illustrates different example configurations for rollback from nested domains, showing various differences that might, in a given application, depend on the level of nesting and on whether the domains involved are transient or persistent domains.

In view of the detailed examples given above, it will be appreciated that the process flow diagram of Figure 5 illustrates a generalized method in accordance with the techniques outlined above. This process flow is intended to encompass aspects of the techniques described above, so where the terminology used to describe the process flow of Figure 5 differs somewhat from that used above, the terms used below should be understood to be consistent with and to encompass the similar or related terms used in the above examples, unless the context clearly indicates otherwise.

The process flow shown in Figure 5 illustrates steps of a method for executing software that comprises two or more software modules compartmentalized into two or more distinct software fault isolation domains. A software fault isolation domain should be understood as a protection domain for the execution of one or more software modules that is configured to detect and isolate run-time effects of software defects that occur while those modules are executing. These software fault isolation domains may be, but are not necessarily, hardware-assisted software fault isolation domains.

As shown at block 510, the method illustrated in Figure 5 includes the step of detecting that execution flow of the software has transferred or is transferring from a first domain of the two or more distinct software fault isolation domains to a second domain of the two or more distinct fault isolation domains. This first domain may be the root domain or a nested domain at a higher nesting level than the root domain. Thus, the illustrated steps may occur at any time during the execution of an application in which several levels of software fault isolation domains are used.

As shown at block 520, the method further comprises saving data recording a state of execution of the first domain, responsive to the detecting shown in block 510. This saving of data recording the state of execution of the first domain may comprise saving one or more of a stack pointer, an instruction pointer, a signal mask, and a value for each of one more registers, for example. There are different possibilities, however. For instance, saving data recording the state of execution of the first domain may comprise saving an application checkpoint, e.g., as used in application checkpoint-restore approaches, that records the state of execution of the first domain. The second domain referred to above may be nested within the first domain. In some embodiments or instances, this nested domain may be created by the first domain. Thus, in some embodiments, the method may comprise determining that execution flow is entering a subroutine that should operate in isolation from other functionality of the software, and creating the second domain for execution of the subroutine in response to said determining. In the example shown in Figure 5, these steps are shown as sub-steps 512 and 514.

As was discussed above, the second domain may later be exited, either normally or abnormally. As shown at block 530, the method may comprise the step of detecting a normal exit of the second domain to the first domain. In this case, the method may further comprise discarding the saved data recording the state of execution of the first domain, as shown at block 535, in response to detecting the normal exit, as there is no longer any need to restore an earlier operational state. But, as discussed below, this data may be saved in some instances. Then, as shown at block 538, execution flow is transferred back to the first domain.

In some embodiments or instances, the second domain may be a persistent domain. In this case, the method may comprise retaining substack and subheap areas for the second domain, responsive to determining, upon exit of the second domain, that the second domain is a persistent domain. In addition, in some embodiments or instances the saved execution state of the first domain may be retained, when the second domain is a persistent domain. Alternatively, the second domain may be a transient domain. In this case, the method may comprise, responsive to determining, upon exit of the second domain, that the second domain is a transient domain, discarding the substack area for the second domain and merging any active allocations in subheap area for the second domain to a heap area or subheap area for the first domain.

The exit from the second domain might be an abnormal exit, in some instances, e.g., in response to detecting a fault, an attack, or some other irregularity during software execution in the second domain. Thus, the method shown in Figure 5 might include the step of detecting an abnormal exit of the second domain, as shown at block 540, and discarding substack and subheap areas for the second domain and returning execution of the software to the first domain, using the data recording a state of execution of the first domain, as shown at block 545. The saved state of first domain is thus restored, e.g., using setjmp and longjmp C functionality.

The method might further comprise signaling an abnormal exit of the second domain to the first domain. This is shown at block 550. Alternatively, an abnormal exit from the second domain might return execution to a domain other than the first domain. (Examples of this were shown in Figures 3 and 4.) In this case, the step shown at block 545 involves returning execution of the software to a domain other than the first domain, operating at a lower nesting level than the first domain, using data recording a state of execution of that domain. Similarly, the signaling of the abnormal exit, as shown at block 550, is to that domain at which execution is restored.

Returning to Figure 1, it will be appreciated that this figure illustrates an apparatus or system for carrying out a method like that shown in Figure 5, as well as any of the variants of the techniques described above. This apparatus/system comprises system hardware 160, which in turn comprises one or more central processing units (CPUs) 162 or other processors, as well as memory 168. CPUs 162 may be integrated with or coupled to hardware support for SFI 166. System hardware 160 may further comprise one or more hard disks 167 or other long-term storage, as well as other devices 168, such as user interface devices, communication interface, circuitry, etc. It will be appreciated that the system hardware 160 may be implemented in a single, standalone node, as well as in a distributed fashion, where processing and memory resources are shared among several nodes.

The system shown in Figure 1 includes non-hardware elements, too, including operating system 170 and kernel 175, as well as other system software 180. Details of various implementations of these elements, which provide the base or platform upon which application 100 is executed, are well known to software developers, and others. Program code executing upon the software platform provided by the system illustrated in Figure 1 can be configured to carry out methods like that shown in Figure 5, as well as all the variations described above.

The techniques, apparatuses, and systems described herein thus combine the saving of process/application states with software fault isolation, which may be hardware assisted, to securely and efficiently roll back corrupted application states by discarding isolated memory compartments and restoring the application to a previously saved application state. The techniques and systems provide for the nesting of software fault isolation domains, to create additional possible recovery points that may be the target of the recovery process.

With the disclosed techniques, applications can be engineered to rapidly recover from unexpected error conditions that currently necessitate the termination and restart of the affected application. This enables applications to provide resilience against run-time attacks that are isolated to individual memory compartments without requiring costly redundancy mechanisms.