CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority from U.S. Provisional Application Serial No. 62/381 ,930, filed August 31 , 2016, the entire disclosure of which is incorporated herein by reference in its entirety.

FIELD OF THE INVENTION

[0002] The invention relates to network based content inspection (NBCI). More specifically, the invention provides apparatus and methods for NBCI's line-rate detection of malicious software, documents or websites, collectively known as malware.

BACKGROUND

[0003] Computer programs dedicated to malware scanning employ various methods of detecting malware in network content. Such methods include behavior-based sandboxing techniques and content-based signature scanning techniques.

[0004] Behavior-based methods may involve allowing network content to execute in an isolated virtual environment, identifying malicious behavior within such environment, and if identified as malware blocking the transmission and further propagation of the offending network content.

[0005] In content-based methods, the suspected network content is commonly compared to a database of known malware-identifying signatures. If a known malware signature is found in the suspected network content, the network content is tagged as malicious and blocked.

[0006] The industry is now witnessing the resurgence of custom or zero-day malware - malware that targets a vulnerability that is not publicly known, and for which a signature has not yet been developed. Because no signature exists for zero-day malware, it cannot be reliably detected by traditional content-based, signature-based detection techniques. On the other hand, behavioral-based analysis can reveal the malicious nature of zero custom or day malware with thorough sandboxing - an approach through which the inspected network is executed in an isolated virtual environment; however, the limitation being that such sandboxing analysis is typically not possible at network-line rates.

[0007] Other inference-based heuristics analysis technique which uses machine learning schemes - be it through Bayesian weighing, neural networks or deductive reasoning - can reliably map the content's safety to a numerical verdict; out of which range, content is declared not to be safe. Though inference-based techniques could be made to compute within a bounded time though to varying degrees of accuracy, such techniques are considered be less accurate than behavioral techniques given the computational accuracy and certainty of the latter.

[0008] Systems and methods for network based content inspection are disclosed in US Patent 7,630,379, which is incorporated herein by reference in its entirety.

[0009] With the rapid growth of network bandwidth, from 100Mbits/s, to 1 Gbits/s, and to 10Gbits/s and beyond, the importance of NBCI performance is increasingly becoming paramount in the effective management of large, complex networks. As a result, there continues to be a need for NBCI methods that effectively and efficiently process data payloads in order to improve the efficiency, stability while reducing NBCI costs without compromising network speeds.

SUMMARY

[0010] In accordance with the present disclosure, there is provided an apparatus for network-based, malware analysis comprising:

a signature scanner configured to scan incoming network-based content with signature-based scanning to identify known threat content;

an Al scanner configured to:

assign a risk value, using an Al scan, to the network content which has not been identified as a known threat in the signature-based scanning step;

identify content having been assigned a risk score below a safe threshold value as safe content; and

identify content having been assigned a risk score above a threat threshold as threat content; and a controller configured, based on the scans:

to allow safe content; and

to block threat content.

[0011] The network-based content may comprise MIME objects.

[0012] The network-based content may comprise one or more attachments.

[0013] The Al scanner may be configured to scan the content in parallel.

[0014] The signature scanner may be configured to scan the content in parallel. For example, the tasks of scanning the content may be divided up across a number of processors. For example, the division may be based on one or more of: content and scanning tasks. For example, one processor may be dedicated to scanning for a particular one or more signature whereas another processor may be dedicated to another subset of the signatures. Another option is that one portion of the content (e.g. one attachment) is sent to one processor for scanning, whereas another portion (e.g. a second attachment) is sent to another processor for scanning.

[0015] The safe threshold may be equal to the threat threshold.

[0016] The threat threshold may be higher than the threat threshold, and the Al scanner may be configured to:

identify suspicious content, suspicious content having been assigned a risk score below the threat threshold and above the safe threshold.

[0017] The controller may be configured to notify the user of identified suspicious content; and prompt the user for input regarding how to process the identified suspicious content. For example, the controller may be configured to remember the user's selection when presented with suspicious content previously.

[0018] The controller may be configured to: receive information from other users of the system regarding how to process suspicious content; and provide this information to the user to allow the user to base their input on information received from other users. This may allow the "user" input to be remembered or received from other users such that, in the absence of a sandbox, the input of other users can guide the user's decision. This may be considered to be a 'crowd-guided' learning/verdict. [0019] The controller may be configured to notify the user if the user's input is inconsistent with the crowd-guided learning, and/or prompt the user again for input (e.g. confirming or cancelling their original selection).

[0020] For example, the system may say:

"User, we could not conclusively classify this content as clean. However, 59% of users found this to be malware, do you still want to view the file?"

[0021] The apparatus may comprise a behavioural scanner, and the apparatus is configured to run suspicious content in a isolated virtual environment to determine whether the suspicious content can be allowed to run or blocked.

[0022] The content may be processed through the malware analysis at rates of at least 1 Gbps, or at least 10Gbps.

[0023] The apparatus may be configured to identify different types of malware identified as threat content. For example, the apparatus may be configured to identify particular instances of threat content spam, spyware or a virus.

[0024] The apparatus may use signature and Al scanning in an in-line mode. In-line mode In-line mode may be considered to place a scanner directly in the network traffic path, inspecting all traffic at wire speed as it passes through the scanner. In-line mode may enable the scanner to be run in a protection/prevention mode, where packet inspection is performed in real time, and intrusive packets can be dealt with immediately; For example, malicious or threat packets can be dropped immediately. This may enable the prevention of an attack from reaching its target.

[0025] In contrast, tap mode works through installation of an external wire tap or built-in internal taps. A scanner deployed in tap mode monitors the packet information as it traverses the network segment. A downside of tapped mode is that, unlike in-line mode, you cannot prevent attacks. Tap mode is typically passive; the scanner essentially sees malicious traffic as it passes. It generally is difficult to inject response packets back through a tap, so discovering an attack in tap mode triggers a response post-attack.

[0026] The scanner may be configured to inspect the data payload (content) to determine if the content of this payload has been inspected previously or is currently under inspection. If the scanner recognizes that the content of the payload has been previously inspected, the scanner will deliver the recognized payload (together with an inspection result as explained below) without subjecting the payload to content inspection. If the scanner determines the payload is an unrecognized payload (that is, the payload has not been inspected previously), the unrecognized payload is delivered for scanning. The scanner calculates the inspection result and delivers the inspected payload together with the inspection result. If the scanner recognizes that the payload 14 is under inspection, the scanner will delay processing of other payloads containing the same content.

[0027] The apparatus may comprise a policy module configured to apply a set of operations, such as the downstream delivery of the recognized payload, or modify the payload, based on business specific policies. An inspected payload and inspection result is returned to the apparatus in order that subsequent receipt of a similar payload does not pass through the scanner. Generally, an inspection result may include one or more markers that indicate that the content has been inspected and/or classified, and that enable other functions to be performed on the payload according to pre-determined policies.

[0028] Other embodiments may be configured to allow partial recognition of the payload content in order that content inspection is only needed for the portion of the payload content that has not been previously inspected.

[0029] For example, the payload may be decomposed into logical portions and each portion is evaluated to determine if it has been inspected. If the algorithm determines that there are un-inspected portions a message digest is calculated for the un-inspected portions. Each message digest may then be scanned by the scanner as described above.

[0030] Decomposition may be achieved by breaking down a payload into logical portions such as by attachment within an email, or the file content within a zip file.

[0031] The apparatus (e.g. the Al and/or signature scanners) may be configured to schedule the content inspection of multiple inspection tasks in order to prevent system resource exhaustion in the event of the rapid or simultaneous arrival of many different data payloads, many instances of the same content, or in the event of a deny-of-service attack. Such scheduling will ensure that the system resources are efficiently utilized to complete content inspection and are spent on applying the content inspection algorithms to one only instance of any multiple instances. This is achieved by giving much lower priority to time-consuming or system resource demanding content processing tasks. Scheduling is accomplished by utilizing the content inspection state (i.e. un-inspected, under-inspection or inspected) together with information relating to the number of required inspection tasks, the time of receipt of an inspection task and the size of the inspection task.

[0032] For example, in a situation where there are 100 users accessing a network, and 99 of those users are attempting to pass small payloads (e.g. a 1-2 kb file) through the network and 1 user is attempting to pass a larger file (e.g. a 100 Mb file) through the network, the scheduling manager will assign priority and allocate system resources based on both the size of each inspection task and the time taken to complete each content inspection task in order to minimize or prevent system resources being consumed by the single, larger inspection task, thus preventing each of the multiple users passing smaller payloads having to wait for the completion of the single larger inspection. That is, the system is able to prevent "live-lock" of the system resources by lengthy content inspection tasks.

[0033] In accordance with the present disclosure, there is also provided a method for network-based, malware analysis comprising:

scanning incoming network-based content with signature-based scanning to identify known threat content;

assigning a risk value, using Al scanning, to the network content which has not been identified as a known threat in the signature-based scanning step;

identifying content having been assigned a risk score below a safe threshold value as safe content; and

identifying content having been assigned a risk score above a threat threshold as threat content; and

allowing safe content; and

blocking threat content. [0034] In accordance with the present disclosure, there is also provided a computer program configured to, when run on a computer to:

enable scanning incoming network-based content with signature-based scanning to identify known threat content;

enable assigning a risk value, using Al scanning, to the network content which has not been identified as a known threat in the signature-based scanning step;

enable identifying content having been assigned a risk score below a safe threshold value as safe content; and

enable identifying content having been assigned a risk score above a threat threshold as threat content; and

enable allowing safe content; and

enable blocking threat content.

[0035] Malware may be considered to be software configured to disrupt computer operation, gather sensitive information, or gain unauthorized access to computer systems. Malware may comprise one or more of: viruses, worms, spyware and Trojan horses.

[0036] Machine Learning is a type of artificial intelligence (Al) which may be used in conjunction with embodiments of the present disclosure. Machine learning may allow a computer (or computer program) to learn and adapt when exposed to data. For example, an apparatus using machine learning may be configured to determine characteristics which are common so some malware which distinguish them from safe content. Machine learning may comprise using pattern recognition and/or computational learning theory in Al. Machine learning may use heuristic techniques.

[0037] A controller or a scanning module (e.g. the signature, Al or behavioural scanner) may comprise a processor and memory. The memory may store computer program code. The processor may comprise, for example, a central processing unit, a microprocessor, an application-specific integrated circuit or ASIC or a multicore processor. The memory may comprise, for example, flash memory, a hard-drive, volatile memory. The computer program may be stored on a non-transitory medium such as a CD. [0038] A behavioural scanner may comprise a virtual computer sandbox for running or opening the code in isolation.

[0039] The apparatus may comprise an Internet security-centric machine configured to learn and adapt algorithms to uncover new and unknown threats so as to learn from and make predictions on data. This may help protecting our systems from malicious entities.

[0040] The Al and/or signature scanners may be configured to reduce the computation required to perform content inspection on concurrently received network data packet payloads (or other network content) are described. In the context of this description "concurrently" means content items (e.g. data payloads) received by a computer network within a short time period such that the system resources considers the content items to have been effectively received at the same time or within a short time period.

[0041] The method may include:

subjecting a data payload that may be carried via a variety of communication protocols to a content recognition module for determining if the payload or a component thereof has been previously inspected, or is being inspected; and

a) if the content has been previously inspected, associate this payload with a previous inspection result for policy enforcement without inspecting this payload;

b) if a previous instance of the content is being inspected, in order to preserve the NBCI system resources, the inspection of the instance will wait until the inspection of the being inspected instance is completed;

once inspection is completed, delivering the payload and inspection result.

[0042] The method may include subjecting an unrecognized payload to content inspection to produce an inspection result and subsequently storing the inspection result in a content recognition module.

[0043] For a given content inspection task, the NBCI system resource allocation priority may be adjusted with the passage of time so that other communication sessions can have a share of system resources.

[0044] The method may allow several NBCI systems in a network to learn from each other's content inspection results. [0045] The method may provide further functionality including any operative combination of various functions as described below.

[0046] The content recognition module may include a one-way hash function for calculating a message digest of the data payload and wherein the message digest is compared to previously stored message digests from previously inspected data payloads. Message digests from previously inspected data payloads are stored in a look-up table and the content recognition module returns a previously inspected result if the message digest of the data payload is the same as a previously stored message digest. As well, the content recognition module returns a null result if the message digest of the data payload does not correspond to a previously stored message digest. This payload is then subjected to content inspection and is further subjected to a one-way hash function to calculate a message digest of the unrecognized result and the message digest subsequently stored in the content inspection module.

[0047] The knowledge of what content has been inspected or what content is under inspection may be stored in a Content Inspection History Lookup (CIHL) table as a record. Each record, hereafter referred to as CIH record, in the CIHL table is indexed with a unique signature of the content. This signature is in the form of a message digest such as those created with SHA-1 , MD-5, etc. Each entry also contains a field to indicate if the content is currently under inspection.

[0048] The CIHL table may also contain a field for the inspection result which may be instructions to take subsequent action with respect to the data payload. Additional information may be added as fields to a CIH record. In one embodiment, this additional information may be time of inspection that may be part of a system to enhance security such that a data payload is no longer marked as inspected if the time of creation information exceeds a pre-determined value. Other information may include size information.

[0049] The apparatus and method may enable many copies of data payloads of the same content entering a network via a variety of communication sessions and via different communication protocols to be effectively and efficiently inspected. [0050] The system and methods may associate a message digest with supplementary information such as time stamp, payload size, etc., to minimize the risk of message digest collision-based attacks against a NBCI system.

[0051] The payload or content may be data of an application level network protocol and the payload may be decomposed prior to content inspection.

[0052] The apparatus may comprise a system implemented on a computer or a network of computers for enhancing network based content inspection of a number of concurrently received data payloads.

[0053] The apparatus may be comprise:

a content recognition module for recognizing if each data payload has been previously inspected for content or is currently under inspection and a) allowing a recognized data payload to be delivered without content inspection; and b) subjecting an unrecognized data payload to content inspection to produce a content inspection result and subsequently storing the content inspection result in the content recognition module.

[0054] Inspections of multiple data payloads may be scheduled by a content inspection scheduler that assigns and modifies the system resource allocation priority to an inspection task.

[0055] The content inspection module may be a co-processor and/or the content inspection module utilizes CAM (Content-Addressable Memory).

[0056] The system may include at least two content inspection modules operatively connected to a common look-up table, the results of content inspection on at least two content inspection modules is added to the common look-up table and/or the content recognition look-up tables are synchronized with each other.

BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Various objects, features and advantages of the invention will be apparent from the following description of particular embodiments of the invention, as illustrated in the accompanying drawings. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of various embodiments of the invention. Similar reference numerals indicate similar components. Figure 1 is a schematic of a scanning procedure according to a first embodiment.

Figure 2 is a schematic of a scanning procedure according to a second embodiment.

Figure 3 is a schematic showing how the apparatus interacts with other components of a network.

Figure 4 is a schematic block diagram showing the interaction of a content recognition module and a content inspection module within a computer system, a co-processor, or a software module in accordance with one embodiment of the invention;

Figure 5 is a schematic block diagram of an implementation of the content recognition module;

Figure 6 is the structure of a content inspection history (CIH) record;

Figure 7 is a flow diagram of a method of processing a payload in accordance with one embodiment of the invention;

Figure 8 illustrates the state machine of the content inspection state transition;

Figure 9 is a flow diagram of a method of processing a payload including decomposition of the payload and partial recognition of the payload where content inspection is only conducted for portions not previously inspected;

Figure 10 is a schematic block diagram of an implementation of the content inspection module;

Figures 11 and 11A are flow diagrams of a method of using a "Time Quantum Divided" strategy to apply content inspection algorithms to a payload in accordance with one embodiment of the invention;

Figure 12 is a schematic block diagram of the implementation of the system of the invention using a content co-processor in accordance with one embodiment of the invention; and, Figure 13 is a schematic block diagram of a network showing how multiple servers on a network may learn from the content inspection performed by other servers.

DETAILED DESCRIPTION

Introduction

[0058] Various aspects of the invention will now be described with reference to the figures. A number of possible alternative features are introduced during the course of this description. It is to be understood that, according to the knowledge and judgment of persons skilled in the art, such alternative features may be substituted in various combinations to arrive at different embodiments of the present invention.

[0059] A problem with using signature-based scanning is that it requires the lengthy process of finding malware, getting a sample, analyzing it, generating a signature and adding it to the repository that is propagated to users as anti-virus updates.

[0060] Another problem is that some malware (e.g. polymorphic and metamorph malware) is able to mutate its code, making signature generation difficult. A polymorphic virus for example, may make use of a decrypter, encrypter stub, thereby making the creation of a signature more difficult because a different key is used each time the virus propagates to a different computer the body of the virus is completely different every time. Metamorphic viruses may be able to 'evolve', i.e., re-code themselves. This ability makes the creation of a signature difficult.

[0061] Machine learning techniques on the other hand, may identify (or assign a risk to) malware based on their characteristics rather than a pre-determined signature even if it hasn't been detected elsewhere or analyzed by researchers.

[0062] Once the 'intelligent' malware detection system has been trained using known malware activity the system gains the ability to detect previously unknown malware. However, using such 'intelligent' malware detection systems alone may generate an unworkable quantity of false positives, as characteristics of safe files may be incorrectly identified as corresponding to malware.

[0063] The present technology relates to a network-based line-rate method and system for detecting and managing potential malware utilizing a blacklist of possible malware to scan content and detect potential malware content based upon characteristics that match the preliminary signature. The undetected content (e.g. content which does not correspond to previously identified malware) is then subjected to an inference-based or behavioural-based processes and methods to determine whether the undetected content is safe for release. Typical to inference-based processes and method, the verdict is a numerical value within a predetermined range, out of which content is not safe. The network content released if the verdict is within safe range, otherwise, the system provides various options of handling such presumably unsafe content; options including, soliciting user input whether to release, block, or subject said content to further offline behavioral analysis.

[0064] That is, the present technology uses a multi-staged analysis which may offer better accuracy during line-rate content-based scanning. Offline behavioral analysis may be used in conjunction with the present technology. For example, after-the-fact remediation may be used if it was deduced that malware is present.

[0065] With the advent of mobile devices and cloud computing, where the accuracy of network-based solutions has to be high, embodiments use multiple scanners (e.g. arranged sequentially), in the order of signature scanning first, then the inference Al scanning techniques. In this way, a higher accuracy of detecting unknown malware may be achieved.

Overview

[0066] Figure 1 depicts a scanning method for network-based, malware analysis. In this embodiment, incoming network-based content 150 is subjected to a multi-staged scanner. In its first stage 101 , content is subjected to a content-based, signature-based analysis to determine whether the content is known to be malware. Content which corresponds to known malware based on the signature is identified as threat content 113a.

[0067] In this case, failure to be mapped to a known-malware signature results does not mean the content is not malware and hence the undetected content 1 10 is subjected to a second stage, the second stage comprising Al scanning 102. In other embodiments, the second stage may comprise behavioural, sandboxing analysis and/or inference- based techniques. Because of this second stage scanning - after being scanned against known malware - the processing times and accuracy of this inference-based second stage can be greatly enhanced.

[0068] In some embodiments, content that is still uncategorized after the two stages of scanning can either be blocked 113, subjected to user's input to determine whether the content be blocked, released or subjected to a full behavioral scanning.

[0069] The scanning apparatus may be configured to scan received content and/or content for transmission.

[0070] The scanning apparatus may be configured to scan content before or after a firewall. Configuring the scans to take place before the firewall helps relieve firewalls and other systems that may be fatigued.

[0071] The content identified as safe can then be delivered to, for example, applications, Operating Systems, a Network or connected infrastructure (e.g. comprising clients, servers) or used in conjunction with the Internet of Things.

[0072] The first two stages can be performed at network line rates (e.g. one or more of: up to 100Mbits/s; between 100Mbits/s and 1 Gbits/s; between 1 Gbits/s and 10Gbits/s; between 10Gbits/s and 100Gbits/s; and over 100Gbits/s).

[0073] To give an indication of how the present system may be used, in a US school district with 120,000 active internet users, 3.2% are attacked on average generating 3,840 events daily. Signature-based is typically able to stop 3,540 of these. Without Al scanning, the remaining 300 files were submitted to a sandbox company. Among these submitted files, only 2 (1 to 3) is APT (Advanced Persistent Threat). The Second stage is hence expected to handle around 295 of these 300. Percentage-wise, signature scanning accuracy will typically be around 92.1 %, with Al this may increase to 99.8% and with Sandboxing the accuracy may approach 100%. It will be appreciated that these accuracy figures are for guidance only.

[0074] Signature scanning scans against known-malware, and hence provides as a base blacklist. Al scanning scans content, analyzing its structure looking for patterns or sequence thereof, that are common across malware. Al scanning helps protect against malware for which signatures are yet to be developed (unknown malware or custom tailored malware). Because malware is typically made of executable computer instructions, Al scanning techniques can result in false positives especially when scanning 'safe' computer instructions. Al scanning techniques may also be computationally 'expensive' compared with signature scanning.

Signature Based Scanning

[0075] The first stage of scanning in the embodiment of figure 1 is signature scanning 101. Signatures consist of a small sequence(s) of bytes of machine code from within the virus which can be used to recognise the virus. An effective signature may be one that is found in every object infected by the virus, but unlikely to be found in 'normal' code.

[0076] In this embodiment, the scan is performed according to the scheme outlined in Morishta et al. (U. S. Patent 7,630,379). For example, the signature scanner may be configured to carry out the following steps:

a) subjecting each newly arriving data payload to content recognition to determine if the newly arriving data payload content has been previously inspected, has not been inspected or is currently under inspection;

b) allowing a newly arriving data payload recognized as previously inspected to be delivered without content inspection;

c) subjecting a newly arriving data payload recognized as not been inspected to content inspection to produce a new payload inspection result whereby the newly arriving data payload becomes a newly inspected data payload;

d) storing a message digest for the newly inspected data payload with the new payload inspection result in a content history lookup table wherein content recognition includes the steps of:

i) subjecting each newly arriving data payload to a one way hash function to calculate a message digest of the newly arriving data payload;

ii) comparing the message digest of the newly arriving data payload to previously stored message digests in the content history lookup table wherein each previously stored message digest has an associated inspection result; and wherein iii) if the message digest of the newly arriving data payload from step ii) is identical to a previously stored message digest determining: a. if the previously stored message digest is flagged as inspected then i. determining a policy action based on the inspection result; or

b. if the previously stored message digest is flagged as under- inspection then i. waiting a pre-determined time period before repeating step ii).

[0077] In this way, the accuracy of the verdict can be improved by referring to verdicts of previously scanned content.

[0078] In this case, the signature scanning is configured to block known malware 1 13a by identifying known malware as threat content 113; to forward all other traffic 110 to the next scanner 102; and provide coverage for Legacy File Types.

Al Risk Assignment

[0079] The second scanning stage 102 in this case is an Al (Artificial Intelligence) scanning stage which is configured to identify characteristics common to malware but different from safe content.

[0080] Artificial intelligence may use algorithms to classify the behavior of a file as malicious or benign according to a series of file features that are manually extracted from the file itself. The machine may be configured by the manufacturer or user what parameters, variables or features to look at in order to make the decision. In some cases machine learning solutions are used to identify a suspicious situation, but the final decision as to what to do about it is left to a user.

[0081] Artificial Intelligence scanning may use machine learning. Once a machine learns what malicious code looks like, it may be configured to identify unknown code as malicious or benign (by assigning a risk value) and in real-time. Then a policy can be applied to delete or quarantine the file or perform some other specified action.

[0082] Al scanning may be performed in a way similar to CylancePROTECT® software (e.g. as described in US 2015/227741). [0083] Al scanning may comprise analyzing and classifying many (e.g. thousands) characteristics per file to determine a risk value.

[0084] Analysis of a file may be based on a previous analysis of a large number (e.g. hundreds of millions) of files of specific types (executables, PDFs, Microsoft Word® documents, Java, Flash, etc.).

[0085] The goal of this pre-analysis is to ensure that:

• there is a statistically significant sample size;

• the sample files cover a wide range of possible range of file types and file authors (or author groups such as Microsoft, Adobe, etc.); and

• the sample is evenly distributed across the specific file types.

[0086] The pre-analysis involves identifying the files into three categories: known and verified valid, known and verified malicious and unknown.

[0087] The next phase in the machine learning process is the extraction of attributes which are characteristic of known malware. These attributes or characteristics can be as basic as the PE (Portable Executable) file size or the compiler used and as complex as a review of the first logic leap in the binary. The identified characteristics may be dependent on its type (.exe, .dll, .com, .pdf, Java, .doc, .xls, .ppt, etc.).

[0088] By identifying multiple characteristics may also substantially increases the difficulty for an attacker to create a piece of malware that is not detected by the Al scanner.

[0089] In this case, the Al scanner 102 is configured to analyze all Executables; provide risk scores; block threat content 1 13b (content with a risk value above a threat threshold); allow safe content 111 a (content with a risk value below a safe threshold); identify suspicious content 1 12 (content with a risk value between the sage threshold and the threat threshold); and pass suspicious content 112 to the behavioural scanner.

[0090] In some cases, the behavioural scanner may be omitted. It will be appreciated that in some embodiments, the malware analysis apparatus may prompt the user to determine how to process suspicious content. For example, the apparatus may be configured to ask whether suspicious content should be allowed or blocked (e.g. prevented from running). The apparatus may be configured to remember user's choices and prompt or apply them for future identified suspicious content (e.g. based on source, risk score).

Behavioural Scanner

[0091] In this case, the embodiment comprises a behavioural scanner 103 for scanning suspicious content 1 12. The behavioural scanner comprises a malware sandbox. A malware sandbox is a controlled testing environment (sandbox) which is isolated from other computers in which an item of suspicious code can be executed in a controlled way to monitor and analyze its behaviour.

[0092] The malware sandbox may comprise an emulator. An emulator may be considered to be a software program that simulates the functionality of another program or a piece of hardware.

[0093] The malware sandbox may use virtualization. With virtualization, the suspicious content may be executed on the underlying hardware. The virtualization may be configured to only control and mediate the accesses of different programs to the underlying hardware. In this fashion, the sandbox may be isolated from the underlying hardware.

[0094] The behavioural scanner in this case is configured to minimize false positives from the Al-based scanner by providing a safe outlet for content identified as suspicious by the Al scanner.

[0095] The behavioural scanner is configured, based on the suspicious content's 103 behaviour to identify the content as either threat content 1 13c or as safe content 11 1 b.

[0096] Content identified as safe content (e.g. by the Al or behavioural scanning) is allowed. Depending on the context, allowed may mean that the content can continue to its destination on the network, or that the content can be executed. Content identified as threat content (e.g. by the signature, Al or behavioural scanning) is blocked.

Interaction between stages

[0097] In embodiments described herein, because the first step is a signature-based analysis, the threshold for identifying a particular file as a risk may be higher in the Al- scanning stage. This may reduce the instances of false-positives. [0098] Once a file is identified as being a threat in the signature-based scan, this file may be passed to the machine learning algorithm already identified as being a threat for learning (in addition to being blocked). Similarly, in embodiments with a sandbox stage, any files identified as being safe or as being a threat may be passed to the Al algorithm to allow the Al algorithm to learn from this sandbox analysis. This may allow the machine learning algorithm to refine the characteristics which it uses to identify content as a threat.

[0099] Because NBCI systems have a finite number of system resources, by filtering out known malware in a first stage, more of these resources may be dedicated to detecting unknown content using Al scanning. This may result in a more robust and stable NBCI apparatus. In addition, to maintain line rate scanning speeds, it is important that the Al scanning stage is dedicated to unknown threats. That is, because Al can be slower than signature scanning, it may be important to ensure that Al scanning is used only on content which can not be identified as a threat using the more rapid signature scanning.

Second embodiment

[0100] Figure 2 shows an alternative embodiment showing how content may be processed in order to identify threat content and safe content.

[0101] That is, figure 2 provides the logic of how content flows between the scanners - the signature, the Al and the sandbox scanners.

Interaction with Network

[0102] Figure 3 is a schematic showing the location of the malware analysis apparatus within the network. In this case, the network is connected an external network such as the Internet 391 or cloud via one or more servers 392. Content is transmitted to and from this external network from various sources. Dividing this portion of the network and the internal portion of the network is the malware analysis apparatus 300. The internal portion of the network may comprise one or more connected computer devices (e.g. mobile phones 393a, computers 393b, and servers 393c).

[0103] The malware analysis apparatus is configured to scan content passing between the internal and external portions of the network to filter threat content. [0104] As noted above, the malware analysis apparatus may be configured to scan content before or after a firewall. Configuring the scans to take place before the firewall helps relieve firewalls and other systems that may be fatigued.

Controlling Computation

[0105] Regarding optimizing the computation required to perform content inspection on concurrently received network data packet payloads are described. In the context of this description "concurrently" means data payloads received by a computer network within a short time period such that the system resources considers the data payloads to have been effectively received at the same time or within a short time period.

[0106] With reference to Figure 4, a content recognition module (CRM) 12 receives a data payload 14. The CRM 12 inspects the data payload to determine if the content of this payload 14 has been inspected previously or is currently under inspection. If the CRM 12 recognizes that the content of the payload 14 has been previously inspected, the CRM will deliver the recognized payload 14a (together with an inspection result as explained below) without subjecting the payload to content inspection. If the CRM 12 determines the payload is an unrecognized payload (that is, the payload has not been inspected previously), the unrecognized payload 14b is delivered to a content inspection module (CIM) 16. The content inspection module 16 calculates the inspection result and delivers the inspected payload 14c together with the inspection result. If the CRM recognizes that the payload 14 is under inspection, the CRM will delay processing of other payloads containing the same content. This type of analysis may be used for signature and/or Al scanning.

[0107] In this case, a policy module 15 applies a set of operations, such as the downstream delivery of the recognized payload 14a, or modify the payload, based on e.g. business specific policies. An inspected payload 14c and inspection result 14d is returned to the CRM 12 in order that subsequent receipt of a similar payload does not pass through the content inspection module 16. Generally, an inspection result is one or more markers that indicate that the content has been inspected and/or classified, and that enable other functions to be performed on the payload according to pre-determined policies. [0108] With reference to Figures 5, 6, 7 and 8, the functionality of the CRM 12 is described. Initially, the payload 14 is passed through a one-way hash function to calculate a message digest 20 of the payload. The message digest is then passed through a Content Inspection Scheduler (CIS) 22 that compares the message digest 20 with previously stored message digests within a Content Inspection History Lookup (CIHL) Table 24. Each record 42 (Figure 6) within the CIHL Table 24 is uniquely identified with a message digest. If a null matching record is found, meaning that the digest does not correspond to a previously stored digest, the payload content 14b is passed to the content inspection module 16 for inspection by the content inspection module 16. The content is then marked as "Under Inspection" 25 (Figure 7) by adding a CIH record 42c into the CIHL table 24. This record will have its Inspection State 43 set to the value of "Under inspection". The content inspection module 16 scans the payload for content and classifies the content based on pre-determined criteria.

[0109] After inspection, the newly inspected content 14c is passed through the one-way hash-function to calculate a message digest 20a of the newly inspected content 14c. A CIH record 42b is inserted into the CIHL Table 24. This entry has the message digest 20a, the Inspection State "Inspected", the Inspection Result 14d, and optionally other supplementary information as will be explained in greater detail below.

[0110] If the comparison returns a matching CIH record 42 with the Inspection State field 43 being "Under Inspection" (Step 29), meaning a previous payload carrying the same content is currently being inspected, the processing of the latter payload content will wait for a period of time 26 before continuing. When the system determines that the inspection state of the previous payload content (Figure 7, step 27) has changed to "inspected", the latter payload content will be subjected to content recognition.

[0111] If the comparison (step 28) returns a matching CIH record 42 with the Inspection State field 43 being "Inspected", meaning that the digest corresponds to the message digest of previously inspected content, the payload by-passes the content inspection module 16 as recognized payload 14a.

[0112] The one-way hash function may be a known Secure Hash Algorithm (SHA), such as SHA-1 , MD2, MD4, MD5, variations thereof or other hashing algorithms as known to those skilled in the art. [0113] With reference to Figure 9, an alternate embodiment is described that further enhances the efficiency of content inspection. In this embodiment, a methodology allowing partial recognition of the payload content is conducted in order that content inspection is only needed for the portion of the payload content that has not been previously inspected.

[0114] In this embodiment, the payload is decomposed into logical portions 30 and each portion is evaluated to determine if it has been inspected. If the algorithm determines that there are un-inspected portions (step 31), a message digest (step 32) is calculated for the un-inspected portions. Each message digest is then searched within the CIHL table as described above.

[0115] Decomposition may be achieved by breaking down a payload into logical portions such as by attachment within an email, or the file content within a zip file.

Scheduling Manager

[0116] In a preferred embodiment, scheduling the content inspection of multiple inspection tasks is conducted to prevent system resource exhaustion in the event of the rapid or simultaneous arrival of many different data payloads, many instances of the same content, or in the event of a deny-of-service attack. Scheduling of one or more of the Al and signature scanning stages will ensure that the system resources are efficiently utilized to complete content inspection and are spent on applying the content inspection algorithms to one only instance of any multiple instances. This is achieved by giving much lower priority to time-consuming or system resource demanding content processing tasks. Scheduling is accomplished by utilizing the content inspection state (ie un-inspected, under-inspection or inspected) together with information relating to the number of required inspection tasks, the time of receipt of an inspection task and the size of the inspection task.

[0117] Figures 10, 1 1 and 11A describe one embodiment of the Content Inspection Module (CIM) 16a using a "Time Quantum Divided" (TQD) content inspection process scheduling strategy. This process enables the system to enhance speed of service to multiple users accessing a network by decreasing priority for a content inspection process with the passage of time. For example, in a situation where there are 100 users accessing a network, and 99 of those users are attempting to pass small payloads (e.g. a 1-2 kb file) through the network and 1 user is attempting to pass a larger file (e.g. a 100Mb file) through the network, the scheduling manager will assign priority and allocate system resources based on both the size of each inspection task and the time taken to complete each content inspection task in order to minimize or prevent system resources being consumed by the single, larger inspection task, thus preventing each of the multiple users passing smaller payloads having to wait for the completion of the single larger inspection. That is, the system is able to prevent "live-lock" of the system resources by lengthy content inspection tasks.

[0118] As shown in Figures 11 and 1 1A, a content inspection process 80 starts by registering each content inspection task with the TQD CIP manager (Figure 11 , step 82) as each content packet requiring inspection arrives thus defining "n" inspection tasks. Upon registration, the TQD CIP manager 90 (Figure 11 A) periodically adjusts priority for each content inspection task such that each content inspection task is completed in a manner to maintain quality of service to multiple users by balancing the time to complete an inspection as determined by the number of inspection tasks, and the relative size of each inspection task. Once the inspection algorithms have been completed (step 84) based on the TQD CIP manager 90 assigned priority, the CIHL table is updated (step 86) with the inspection result and the content inspection process 80 un-registers a specific content inspect task from the TQD CIP manager (step 88).

[0119] As content inspection tasks are being registered and un-registered from the TQD CIP manager 90, the TQD CIP manager 90 continuously loops through each of the registered content inspection tasks and reviews and updates the status or priority of each content inspection task.

[0120] With reference to Figure 1 1A, initially, the TQD CIP manager 90 determines if there are any registered inspection tasks (step 92). If there are no registered inspection tasks, the TQD manager 90 waits a pre-determined period of time (step 94) until redetermining if there are any inspection tasks. If there are inspection tasks (step 93), the TQD CIP manager 90 will reduce the time-to-live (TTL) value of each inspection task by a certain value (step 98). A content inspection process (CIP) will be aborted (step 102) if its TTL drops below an arbitrary threshold value (step 100). The CIH record of an aborted inspection task will be removed from the CIHL table (step 104). The transmission of the payload may be re-initiated by the sender and/or receiver at a later time.

[0121] If the TTL is not less than zero, the TQD CIP manager 90 will reduce the priority for the / ^' th inspection task (step 106) by a pre-determined value.

[0122] Once the priority has been adjusted or the / ^' th CIP has been aborted, the TQD CIP manager determines if there are any remaining registered inspection and either waits for a period of time (step 94) to check for registered inspection tasks or continues reviewing and adjusting the status of other registered inspection tasks.

[0123] As an example of a possible scheduling scenario, 5 content inspection tasks may have been registered with the TQD CIP manager 90. These registered inspection tasks may include 3 small files (e.g. 3 kb each), 1 medium size file (e.g. 10 Mb) and 1 large file (e.g. 100 Mb) received in any particular order. In processing these inspection tasks, the manager will seek to balance the content inspection in order to maintain efficiency for a desired level of service. For example, scheduling manager parameters may be set to ensure that priority is assigned to inspection of the smaller files first irregardless of the time of receipt. Alternatively, scheduling manager parameters may be set to ensure that priority is assigned strictly based on the time of arrival irregardless of size. As illustrated in Figure 1 1A, the system may assign the same initial priority to all the inspection tasks. The scheduling manager then reduces the priority for each of the task with the passage of the time. Further still, scheduling manager parameters may be set as balance between time of arrival and size. That is, in certain situations, the large file may be processed concurrently with the smaller files based on a particular allocation of system resources. Alternatively, the large file may be processed only for a period of time, until the scheduling manager determines that processing has taken too long and the inspection process is aborted for the large file. It is understood that the number of tasks registered with the scheduling manager may be dynamically changed such that priority may be adjusted up or down based on changes to the number of registered tasks.

[0124] It is understood by those skilled in the art that the determination of priority and the allocation of system resources to effectively manage content inspection based on content size, and time-to-complete an inspection task may be accomplished by a variety of algorithms and that the methodologies described above are only a limited number of examples of such algorithms.

Classification of Inspection Results

[0125] In various embodiments, the content of a data payload, as a recognized payload 14a or an inspected payload 14c can be associated with further information as described below allowing the system to take particular actions with respect to the payload based on the inspection result (Figure 6).

a) Classification of Content

[0126] The inspection result can be classified on the basis of content. For example, it can be a marker indicating that the content is spam, spyware or a virus.

b) Content Instructions

[0127] The inspection result can include a set of instructions to alter the content. In this case, the policy module 15 may use these instructions to take further steps with respect to the payload. For example, if the content is marked as a virus, the instructions may be to warn the recipient that the payload contains a virus and should not be opened. In other examples, the instructions may be to prevent the delivery of payload, but to send information indicating that the delivery has been denied.

c) Supplementary Data

[0128] The inspection result can be associated with supplementary data. Supplementary data provides further functionality including enhanced security to the methods of the invention.

[0129] For example, supplementary data may include the time of creation 44 of the message digest which may be used to provide enhanced security. That is, as it is known that given enough time, an attacker can achieve a collision with the commonly used oneway hash algorithms, by adding time information as supplementary data, a message digest can be retired if the message digest is older than a pre-determined value.

[0130] In another embodiment supplementary data may also or alternatively include the size 45 of the payload wherein the size information can be used to provide finer granularity to also reduce the possibility of a hash code collision. In this example, when conducting the CIHL table search function within the lookup table, both the message digest and the size have to match those of the payload.

Deployment

[0131] The system may be deployed as an adaptive external module of an existing content inspection system, or as an embedded module within a content inspection system.

[0132] In one embodiment, the system is implemented with the content recognition module interacting with an existing content inspection co-processor as shown in Figure 12.

[0133] In another embodiment, the system is a software component embedded into a content inspection module which is a software module, a co-processor or a computer system.

[0134] In a further embodiment, and in order to leverage the computation spent on content inspection, the message digests along with the inspection results can be shared among several instances of content recognition/inspection systems as shown in Figure 13. The sharing can be accomplished by storing the message digests along with the inspection results in a central server shared by these instances of content recognition/inspection systems, or replicating the digests along with the inspection results in each instances of the group. For example, Figure 13 shows four networked servers that each have external Internet connections that are securely linked via a common internal network. Server 3 is shown to represent the highest traffic server, possibly as an enterprise gateway. Servers 1 , 2 and 4 see less traffic but are operatively and securely connected to Server 3. A NBCI database is connected to Server 3. In order to further enhance the efficiency of the system, each server may report the results of their respective payload inspections to Server 3 and hence to database 5 such that each server on the system can "learn" from the experiences of the other Servers, thereby preventing the duplication of content inspection across a larger network. This networked embodiment is particularly beneficial in larger enterprises or service providers where the volume of traffic is sufficiently large that the ability to share such inspection results can greatly enhance the overall efficiency and cost-effectiveness of the system. [0135] The preceding description is intended to provide an illustrative description of the invention. It is understood that variations in the examples of deployment of the invention may be realized without departing from the spirit of the invention.

Conclusions

[0136] As outlined above, the present technology may be better able to reach a verdict using signature and Al-based scanning in realtime (i.e. at line rates). This may be at least partially because of the subsonic scanning (and hashing of previous caches) as well as the Time Quantum based scheduling described in the applicant's previous U.S. Patent 7,630,379. These techniques may be used in both the signature and the Al based scanners.

[0137] Another aspect of the present application may be used inline with traffic. The reason most previous network solutions typically use tap mode is because they cannot 'cope' with the network throughput, and hence being in tap mode means they can pass on some network traffic, without slowing the network. However, disadvantages of such tap mode (not inline) solutions may include that they cannot participate in the handshakes of 'encryption' (SSL for example) and hence cannot 'see' encrypted content. With encryption becoming more the norm, increasing at nearly 16% year over year (nearly 64% now), the usefulness of tap mode solution may be diminished.

[0138] Although the present invention has been described and illustrated with respect to preferred embodiments and preferred uses thereof, it is not to be so limited since modifications and changes can be made therein which are within the full, intended scope of the invention as understood by those skilled in the art.