SYSTEMS AND METHODS FOR CALCULATING AGGREGATION RISK AND SYSTEMIC RISK ACROSS A POPULATION OF ORGANIZATIONS

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This document is a PCT patent application that claims benefit to U.S. provisional application serial number 62/850,431 filed on May 20, 2019, which is incorporated herein by reference in its entirety.

FIELD

[0002] The present disclosure generally relates to predictive cyber technologies; and in particular, to systems and methods for calculating risk and predicting costs to improve cybersecurity.

BACKGROUND

[0003] An increasing number of software (and hardware) vulnerabilities are discovered and publicly disclosed every year. In 2016 alone, more than 10,000 vulnerability identifiers were assigned and at least 6,000 were publicly disclosed by the National Institute of Standards and Technology (NIST). Once the vulnerabilities are disclosed publicly, the likelihood of those vulnerabilities being exploited increases. With limited resources, organizations often look to prioritize which vulnerabilities to patch by assessing the impact it will have on the organization if exploited. Standard risk assessment systems such as Common Vulnerability Scoring System (CVSS), Microsoft Exploitability Index, Adobe Priority Rating report many vulnerabilities as severe and will be exploited to err on the side of caution. This does not alleviate the problem much since the majority of the flagged vulnerabilities will not be attacked.

[0004] NIST provides the National Vulnerability Database (NVD) which comprises of a comprehensive list of vulnerabilities disclosed, but only a small fraction of those vulnerabilities (less than 3%) are found to be exploited in the wild - a result confirmed in the present disclosure. Further, it has been found that the CVSS score provided by NIST is not an effective predictor of vulnerabilities being exploited.

[0005] It is with these observations in mind, among others, that various aspects of the present disclosure were conceived and developed.

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] FIG. 1 is a simplified block diagram showing a general computer- implemented system for aggregating, and computing probabilities of risk of a cyber- related attack in view of predetermined costs.

[0007] FIG. 2A is a simplified block diagram showing a first embodiment of the system of FIG. 1 configured to determine cyber aggregation risk by calculating the probability of a single attack costing a certain amount in terms of damage.

[0008] FIG. 2B is an image of pseudocode illustrating aspects of the embodiment of FIG. 2A.

[0009] FIG. 2C is a graph illustrating an evaluation including depiction of a percentage increase in probability of attack using the functionality of the embodiment of FIG. 2A.

[0010] FIG. 3 is a simplified block diagram of a second embodiment of the system of FIG. 1 configured to determine cyber aggregation risk by calculating the probability of a single attack costing a certain amount in terms of damage with respect to a single or multiple industry vertical.

[0011] FIG. 4A is a simplified block diagram of a third embodiment of the system of FIG. 1 configured to identify organizations to be incentivized to reduce aggregation risk.

[0012] FIG. 4B is an exemplary computer-executable algorithm or pseudocode associated with the third embodiment of FIG. 4A.

[0013] FIG. 4C is an exemplary computer-executable algorithm or pseudocode associated with the third embodiment of FIG. 4A.

[0014] FIG. 5 is a screen shot of an exemplary interface for uploading and identifying sources of aggregation risk.

[0015] FIG. 6 is a simplified block diagram of a general computer- implemented method of applying aspects of the system of FIG. 1 for aggregating, and computing probabilities of risk of a cyber-related attack in view of predetermined costs.

[0016] FIG. 7 is an example simplified schematic diagram of a computing device that may implement various methodologies described herein. [0017] Corresponding reference characters indicate corresponding elements among the view of the drawings. The headings used in the figures do not limit the scope of the claims.

DETAILED DESCRIPTION

[0018] Aspects of the present disclosure relate to embodiments of a computer-implemented system (hereinafter“system”) and methods for predicting and/or determining cyber aggregation risk. In some embodiments, the system determines cyber aggregation risk by calculating the probability of a single attack costing a certain amount in terms of damage. In some embodiments, the system determines cyber aggregation risk by calculating the probability of a single attack costing a certain amount in terms of damage with respect to a single or multiple industry verticals. In some embodiments, the system identifies organizations to be incentivized to reduce

aggregation risk. The system may also include or otherwise be associated with a graphical user interface for uploading and identifying sources of aggregation risk.

[0019] Introduction and Technical Challenges

[0020] Definitions:

[0021] Vulnerability: throughout this document, the term“vulnerability” can be instantiated in a number of ways. Perhaps most obvious is a standard enumeration of software vulnerabilities such as the National Vulnerability Database (NVD), a reference vulnerability database maintained by the National Institute of Standards and Technology ( see nvd.nist.gov). The NVD numbering system defines CVE identifiers.

[0022] The CVE numbering system follows one of these two formats:

• CVE-YYYY-N N N N ; and

• CVE-YYYY-N NNNNNN.

[0023] The“YYYY” portion of the identifier indicates the year in which the software flaw is reported, and the N’s portion is an integer that identifies a flaw (e.g., see CVE-2018-4917 related to https://nvd.nist.gov/vuln/detail/CVE-2018-4917, and CVE-2019-9896 related to https://nvd.nist.gov/vuln/detail/CVE-2019-9896).

[0024] However, other ways to identify or instantiate vulnerabilities are possible - such that the term vulnerability may be used to include any vulnerabilities identified by the software vendor, security firms, within an organization, or determined from a piece of software designed to probe vulnerabilities. Further, the term “vulnerability” can also be used to refer to a class of vulnerabilities and may not only include software flaws (may also include hardware or software/hardware combinations), but other flaws including but not limited to misconfigurations, to organizational practices, hardware, and physical security. It can also be used to describe a class of generalized computer issues that appeal to particular hackers or communities of hackers for purposes of compromising computer systems.

[0025] Software: throughout this document, the term“software” can be instantiated in a number of ways. Perhaps most obvious is a standard enumeration of software vulnerabilities such as NIST’s NVD numbering system defining CPE numbers or identifiers. More specifically, a Common Platform Enumeration (CPE) is a list of software/hardware products that are vulnerable to a given CVE. The CVE and the respected platforms that are affected, i.e. , CPE data, can be obtained from the NVD.

For example, the following CPEs are some of the CPEs vulnerable to CVE-2018-4917:

• cpe:2.3: a: adobe: acrobat_2017: *: *: *: *: *: *:*: *

• cpe:2.3:a:adobe:acrobat_reader_dc: 15.006.30033:*:*:*: classic:

• cpe:2.3:a:adobe:acrobat_reader_dc: 15.006, 30060:*:*:*: classic:

[0026] However, other ways to identify software (vulnerabilities) are possible and may also include components used to create software including libraries, source code snippets, and SaaS-provided services. Further, the term“software” can also be used to refer to a class of software that may be determined by the vendor of the software, the author of the software (especially for custom code), the platform the software runs on, the type of applications, what services the software uses, the language the software is written in, coarsening based on version number, and/or combinations of these methods of classification.

[0027] Technical Challenges: Information technology (IT) administrators lack sufficient technical means for efficiently identifying and practically addressing possible vulnerabilities of a technology configuration associated with an IT system such as determining how to approach a given vulnerability (versus another). A given IT system may be potentially susceptible to thousands of security vulnerabilities (at least those identifiable via the NVD). While the NVD and CVSS provides baseline information about some threats, there is insufficient technology presently available that might allow IT administrators to actually make sense of and intelligently leverage such information to apply responsive measures and prioritize patches or other fixes, and predict actual attacks based on the specifics of a given technology configuration.

[0028] General Specifications of Computer-implemented System

Responsive to Technical Challenges

[0029] Referring to FIG. 1 , an inventive concept responsive to the aforementioned technical challenges may take the form of a computer-implemented system, designated system 100, comprising any number of computing devices or processing elements. In general, the system 100 leverages artificial intelligence to implement cyber predictive methods to e.g., predict risk and potential cost to an information technology (IT) system or environment. While the present inventive concept is described primarily as an implementation of the system, it should be appreciated that the inventive concept may also take the form of tangible, non-transitory, computer- readable media having instructions encoded thereon and executable by a processor, and any number of methods related to embodiments of the system described herein.

[0030] In some embodiments, the system 100 comprises (at least one of) a computing device 102 including a processor 104, a memory 106 of the computing device 102 (or separately implemented), a network interface (or multiple network interfaces) 108, and a bus 110 (or wireless medium) for interconnecting the

aforementioned components. The network interface 108 includes the mechanical, electrical, and signaling circuitry for communicating data over links (e.g., wires or wireless links) within a network (e.g., the Internet). The network interface 108 may be configured to transmit and/or receive data using a variety of different communication protocols, as will be understood by those skilled in the art.

[0031] As indicated, via the network interface 108 or otherwise, the computing device 102 is adapted to access data 112 from a host server 120 or other remote computing device and the data 112 may be generally stored/aggregated within a storage device (not shown) or locally stored within the memory 106. The data 112 includes any information about cybersecurity events across multiple technology platforms referenced herein, information about known vulnerabilities associated with hardware and software components, any information from the NVD including updates, and may further include, without limitation, information gathered regarding possible hardware and software components/parameters being implemented by a given technology configuration associated with some entity such as a company. A technology configuration may include software and may define software stacks and individual software applications/pieces, may include hardware, and combinations thereof, and may generally relate to an overall network or IT infrastructure system or environment including telecommunications devices and other components, computing devices, and the like.

[0032] As shown, the computing device 102 is adapted, via the network interface 108 or otherwise, to access the data 112 from directly and/or indirectly from various data sources 118 (such as the deep or dark web (D2web), or the general Internet). In some embodiments, the computing device 102 accesses the data 112 by engaging an application programming interface 119 to establish a temporary

communication link with a host server 120 associated with the data sources 118.

Alternatively, or in combination, the computing device 102 may be configured to implement a crawler 124 (or spider or the like) to extract the data 112 from the data sources 118 without aid of a separate device (e.g., host server 120). Further, the computing device 102 may access the data 112 from any number or type of devices providing data (or otherwise taking the form of the data sources 118) via the general Internet or World Wide Web 126 as needed, with or without aid from the host server 120

[0033] The data 112 may generally define or be organized into datasets or any predetermined data structures which may be aggregated or accessed by the computing device 102 and may be stored within a database 128. Once this data is accessed and/or stored in the database 128, the processor 104 is operable to execute a plurality of services 130, encoded as instructions within the memory 106 and executable by the processor 104, to process the data so as to determine correlations and generate rules or predictive functions, and compute metrics from these rules or functions based on predetermined inputs to e.g., compute a probability of a cyber-attack, as further described herein. The services 130 of the system 100 may generally include, without limitation, a filtering and preprocessing service 130A for, in general preparing the data 112 for machine learning or further use; an artificial service 130B comprising any number or type of artificial intelligence functions for modeling the data 112 (e.g., natural language processing, classification, neural networks, linear regression, etc.); and a predictive functions/logic service 130C that formulates predictive functions and outputs one or more values suitable for reducing risk, such as a probability of an attack, incident, or exploit of a vulnerability, an overall threat value defining a possible cost predicted from an exploitation of the vulnerability, and the like, as further described herein. The plurality of services 130 may include any number of components or modules executed by the processor 104 or otherwise implemented. Accordingly, in some embodiments, one or more of the plurality of services 130 may be implemented as code and/or machine-executable instructions executable by the processor 104 that may represent one or more of a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, an object, a software package, a class, or any combination of instructions, data structures, or program statements, and the like. In other words, one or more of the plurality of services 130 described herein may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks (e.g., a computer-program product) may be stored in a computer- readable or machine-readable medium (e.g., the memory 106), and the processor 104 performs the tasks defined by the code.

[0034] As shown, the computing device 102 may be in operable

communication with some device associated with at least one of an information technology (IT) system 130 or enterprise network. The IT system 130 may include any system architecture, IT system, or configuration where it is desired to assess possible vulnerabilities to the IT system 130, rank these vulnerabilities, and apply the

functionality described herein to reduce risk to the IT system 130. The IT system 130 may further include data 132 defining some configuration of possible hardware and/or software components (e.g., various software stacks) that may be susceptible to vulnerabilities.

[0035] As further shown, the system 100 may include a graphical user interface (“interface”) 134 which may be presented by way of a portal or gateway embodied as an API, a browser-based application, a mobile application, or the like. The interface 134 may be executable or accessible by a remote computing device (e.g., client device 136) and may provide predefined access to aspects of the system 100 for any number of users. For example, accessing the interface 134, a user may provide information about an external IT system (such as data 132) so that the computing device 102 can process this information according to the plurality of services 130 and return some output value useful for reducing risk of an attack based on a vulnerability to the IT system 130.

[0036] Technical Preliminaries

[0037] Some technical preliminaries shall be described, followed by exemplary embodiments of the system 100 that apply aspects of these technical preliminaries in some form to predict risk and potential costs in cybersecurity. These technical preliminaries may be defined as problem sets or initial models and may be implemented as code and/or machine-executable problem definitions and/or instructions executable by the processor 104.

[0038] As an initial matter, we can assume a population of organizations (i.e. organizations that control critical infrastructure, organizations for which a party may be responsible for damages, or any set of organizations for which one must consider aggregation risk) denoted as set U. Organizations may define or otherwise include at least one IT system or environment defining any number of software and/or hardware components, such as the IT system 130.

[0039] Likewise, we can further assume a sets of all possible pieces of software (denoted S), software vulnerabilities (denoted V), and industry verticals (denoted /). This information may be defined within the data 112 accessed by the computing device 102.

[0040] For each organization o in set U, we assume it is associated with a set of software, denoted S ₀ , and a set of software vulnerabilities V ₀ , and a set of industry verticals, l ₀ For the sake of simplicity, we will generally assume each organization o is mapped to a single industry vertical, but in practice it can be many. There are simple methods to extend the framework to allow for many industry verticals (for example, having a symbol representing multiple industries). Therefore, we will treat l ₀ as a single element of set /. Data 132 in FIG. 1 is an exemplary representation of this data for each organization o. In other words, information about an IT system 130 associated with each organization o, may be represented as data 132, and the data 132 may further define the set of software S ₀ of the IT system 130, a set of software vulnerabilities V ₀ , and a set of industry verticals, l ₀

[0041] Formalism for Attack Cost

[0042] For a given organization o and piece of software s the function cost_sw(o,s) returns the cost of an attack against organization o if software s is exploited (note that we can expand this to sets of software as well). Likewise, for a given organization o and vulnerability the function cost_vuln(o,v) returns the cost of an attack against organization o if vulnerability v is exploited; note that we can expand this to sets of vulnerabilities as well). We note that these functions can be instantiated in multiple ways. Some examples include:

- Estimate of payment to organization o after an attack occurs,

- Estimate of the total cost of damages specific to organization o,

- Standard estimate of damages associated with organization o based on prior history of related attacks to similar software or vulnerabilities,

- A simple model such as“universal” cost (i.e. setting the value to 1.0) or based on the number of systems that have the software and/or vulnerability, and/or

- A combination of the above-described examples.

[0043] Formalism for Victim Susceptibility

[0044] For a given organization o and software s the function

susceptible_sw(o,s) returns the probability that organization o is susceptible to an attack conducted leveraging exploits in software s. Likewise, for a given organization o and vulnerability v the function susceptible_vuln(o,v) returns the probability that organization o is susceptible to an attack conducted leveraging exploits on vulnerability v. We note that these functions can be expanded for sets of software and

vulnerabilities and instantiated in multiple ways. Some examples include:

- Estimating the probabilities based on various aspects of the organization’s security program such as the number of personnel on the security team, their experience level, the annual security budget, adherence to compliance standards, the adoption of certain“best practices,” the setup/segmentation of the organization’s IT infrastructure, and/or the use of specialized security hardware or software,

- Estimating the probabilities based on the results of a survey issued to the organization covering various aspects of their security,

- Historical record of the organization’s past effectiveness in being prepared for an attack,

- Historical records of the probability of an attack when such software and/or vulnerabilities are exploited,

- A simple model, such as the probability being 1 or 0 depending on whether or not the organization employs the software, and/or

- A combination of the above-mentioned examples.

[0045] Formalism for Threat

[0046] We assume the existence of a function threat sw that maps pieces of software (from set S) to a probability. We note that the function th reat sw would be dynamically updated over time and can be instantiated in multiple ways, but the intuition is that it returns the probability that a given piece of software S will be exploited or otherwise leveraged in an attack by a hacker. Some examples of how the function threat sw can be instantiated include:

- Determining the probability of a piece of software being exploited based on

availability of exploit, current vulnerabilities present within that piece of software and/or information collected from the Internet (to include but not limited to darkweb, deepweb, social media, chan sites, paste sites),

- Machine learning or artificial intelligence based approaches based on historical exploitation rates for that software, the vendor of that software, interest by hackers (malicious or otherwise), or other information, and/or

- Estimation of the probability of the software being exploited.

[0047] Likewise, we assume the existence of function threat vuln that behaves in a similar manner to threat sw except that it takes a software vulnerability (from set V) as input and returns the probability that the vulnerability is exploited. This function can also be dynamic and instantiated in a manner similar to threat sw. Exemplary Embodiments of the System (100)

[0048] Given the above Technical Preliminaries, various embodiments of the system 100 are contemplated that are responsive to the technical challenges set forth herein.

[0049] First embodiment: Referring to FIG. 2A, in a first embodiment 150 of the system 100, the system 100 is configured to determine cyber aggregation risk by, generally, calculating the probability of a single attack costing a predetermined certain amount in terms of potential damage.

[0050] Exact calculation based on a software vulnerability. Applying any number of functions, expressions, or logic as represented by risk and cost functions 152 in FIG. 2A (and indicated below from expressions 1.1 to 1.12) and executable by the processor 104, and given a population of organizations denoted as U and illustrated as 154, and a single software vulnerability (denoted v, a member of set V) and illustrated as 156, an overall probability of a payout equal to at least X (represented as 158) can be computed as follows:

(1.1 ) Exact calculation based on a software vulnerability. Given a population of organizations (U) and a single software vulnerability (denoted v, a member of set V), we need to compute the overall probability of a payout equal to at least X.

(1.2) For a given subset U’ of organizations, we can compute the cost of an attack leveraging vulnerability v as:

(1.3) Likewise, assuming independence between the probability of an attacker using a vulnerability and the probability of individual organizations being susceptible, the probability of such an attack occurring that affects precisely the systems in U’ can be computed as:

(1.4) Hence, the overall probability of an attack using vulnerability v with an associated payout of at least X can be calculated as:

Y

. Pv f

v ¹ x

(1.5) Likewise, we can calculate the probability based on an attack on software s in a similar manner, except we modify the calculations as follows:

(1.6) The calculations of 1.5 give rise to an overall“probability of attack with an associated payout of at least X” computation for the scenario where the threat is determined by software as:

(1.7) Note that this problem maps directly to a variant of the“subset sum”

problem, in which the input is a set of numbers ( N ) and a target amount (T), and the output is all subsets of numbers that sum to T or greater.

(1.8) In the mapping to the subset-sum problem, the target T is X (the payout lower bound) and the set of numbers ( N ) is the following (replace

cost vuln with cost sw for the software variant):

(1.9) Standard exponential-time algorithms can be used to solve this problem exactly thru enumeration, as well as simple variants of the dynamic programming algorithm commonly used to solve the subset-sum problem. We note that the result of approximation methods could still yield an exponential number of combinations and the runtime of such an algorithm could also take exponential time. (1.10) We note a simple sampling method can be used to compute a probability bounds (we shall denote them / and u for the lower and upper bounds of probability, respectively. Consider the pseudo-code 160 of FIG. 2B

(shown for vulnerability-based version - note the simple changes in 1.5 and 1.6 for the software-based version). This pseudocode 160 is executable by the processor 104 to compute the probability bounds, / and u. A variation to the pseudo-code of FIG. 2B is contemplated where “TRIED” is omitted.

(1.11 ) Several variants to the above sampling algorithm can be applied - for example where the sampling is biased in a way to produce the subsets where (i.e. at step 2.1 ) is the greatest in the earlier iterations.

(1.12) Referring to FIG. 2C an evaluation was conducted on four organizations, assigning each a unit cost of“1” for an attack and setting susceptible(v,o) as 1 if organization o had vulnerability v and 0 otherwise. For 68 software vulnerabilities, we examined the probability of experiencing an attack that had a cost of 2 units or greater. We compared these probabilities with a baseline method that assumed independence among the organizations. For each software vulnerability, we show the percent increase in the probability of attack using our method in the figure below. In all cases, the probability was at least twice as great as the on yielded by the baseline method.

[0051] Second embodiment: In a second embodiment 200 of the system 100, in general, applying any number of functions, expressions, or logic as represented by risk and cost functions 202 in FIG. 3 (and indicated below from expressions 2.1 to 2.8) and executable by the processor 104, and given at least one industry vertical denoted below as / and illustrated as 204, and software 206 denoted below as software s (and further associated with vulnerability v), the system 100 is configured to determine a cyber aggregation risk by computing a probability of a single attack 208 costing a certain amount in terms of damage with respect to a single or multiple industry verticals, as follows:

(2.1 ) Let P(i) be the probability of an organization in set U being in industry

vertical / (/ is in set /). This is equivalent to the fraction of organizations in U in industry vertical /. Each organization is assigned to one industry vertical (note we can easily extend to allow multiple industry verticals by having symbols that can represent more than one vertical). (2.2) For a given industry vertical / and software s (resp. vulnerability v), let Pi(s) (resp. P,(v)) be the fraction of companies in industry vertical / running software s (resp. having vulnerability v).

(2.3) We will overload the notation susceptible sw, susceptible vuln,

cost sw, and cost vuln, for industry verticals - where each company in that industry vertical has the same susceptibility and cost for a given software or vulnerability. For industry vertical /, this will be denoted susceptible_sw(i,s), susceptible_vuln(i,v), cost_sw(i,s), and

cost_vuln(i,v).

(2.4) Hence, for a single industry vertical / and vulnerability v, we can compute the probability of an attacker targeting m specific organizations in that industry vertical as:

(2.5) Analogously, for a single industry vertical / and software s, we can

compute the probability of an attacker targeting m specific organizations in that industry vertical as:

(2.6) For a given industry vertical, the probability of an attack occurring that utilizes vulnerability v that costs at least X can be computed as follows:

Where ceil is the ceiling function. The following expression computes the probability when we consider software s:

(2.7) When considering multiple industry verticals: |/|, we can compute the probability of an attack costing X amount as follows when considering vulnerability v: Likewise, with respect to software s the probability can be computed as follows:

(2.8) The expressions in 2.7 can be solved in much the same way as the

problem presented in the description of the first embodiment 150 (e.g., expressions 1.9-1.11 ). Specifically, an enumeration approach (exponential time), a dynamic programming method used to solve subset- sum, and the sampling method from 1.10 (along with extensions) can be adapted.

[0052] Third embodiment: In a third embodiment 300 of the system 100, in general, applying any number of functions, expressions, or logic as represented by risk and cost functions 302 in FIG. 4A (and indicated below from expressions 3.1 to 3.52) and executable by the processor 104, and given at least a vulnerability 304

denoted below as v, a cost 306 denoted below as cost X, and select organizations 308 denoted as K organizations, the system 100 is configured to identify one or more organizations 310 to be incentivized to reduce aggregation risk (by addressing problems associated with probability reduction or cost reduction), as follows:

(3.1 ) The discussions in this section will be based on software vulnerability (i.e. using susceptible vuln and cost vuln) but can easily be adapted for use by software (i.e. using susceptible sw and cost sw) without loss of generalization.

(3.2) Also of note, extensions of this framework based on industry verticals is also possible.

(3.3) In this section, there are two key problems that we shall explore:

(3.3.1 ) Probability reduction: Given a vulnerability and cost , select K

organizations such that if they resolve the vulnerability, it reduces the probability of an incident (that is due to v) that costs X or greater.

(3.3.2)Cost reduction: Given a vulnerability v and cost , select K organizations such that if they resolve the vulnerability, it ensures that any incident (due to v) that has a cost X or greater and has a probability close to zero. (3.4) Algorithms for“Probability Reduction”: here we present several methods to address this problem.

(3.4.1 ) Enumeration approach. Note that finding an“optimal” solution to

Probability Reduction is likely a difficult combinatorial optimization problem (i.e. likely NP-hard); hence, approaches for finding such an optimal solution may require exponential time in terms of the input. Further, evaluation of the“probability of attack” (P„ _, u from the first embodiment 150 and second embodiment 200) is also computationally difficult. In FIG. 4B, an algorithm 320 is presented that uses an enumeration approach in combination with a subroutine for computing a probability of attack (in practice, one can substitute algorithms or other expressions of the first embodiment 150 for said subroutine; i.e., this algorithm 320 can assume an oracle-implemented approach using expressions of the first

embodiment 150). If the number of iterations is set at or above then the method illustrated can be used.

(3.4.2) MPW approach. An alternative is to find the“most probable world” (MPW) of the worst-case scenario (where the total cost exceeds X). That is, there are many cases where v is used in an attack that can result in the worst case - but these scenarios are disjoint (hence the sum of the probabilities of each scenario can lead to such a case). The MPW of such a case will identify where a large portion of the probability lies -selecting organizations that reduce this scenario also reduces the overall probability (though it would not necessarily be optimal).

(3.4.2.1 ) Finding an MPW. Finding the MPW that corresponds with a certain minimum cost requirement can be solved with a simple set of integer constraints by maximizing the log-liklihood of p _{V U’} (see first embodiment 150) where U’ is the subset of systems affected by an exploit for the software vulnerability. In this case, there is a variable Y ₀ associated with each organization o in set U. The objective function is associated with the log-liklihood of p _{V U’} while the constraint ensures that the minimum cost is met.

Ve _f Y _a e {0,1} (3.4.2.2) Using the above constraints (in conjunction with an integer program solver like CPLEX or QSOPT) we can find the affected

organizations in the MPW as each associated Y ₀ will be returned as 1 by the solver. We shall refer to the set of all organizations in the MPW as UMPW· Based on this set, we select the organizations to incentivize with the (greedy) algorithm 330 depicted in FIG. 4C, executable by the processor 104.

(3.4.3) With Simplifying Assumption on Susceptibility. Under certain

strong/simplifying assumptions, we can compute exact solutions to the probability reduction problem exactly. For example, a common case is that susceptible_vuln(o,v) can only be either 0.0 or 1.0. In this case, we can apply the MPW approach, but forgo the solving of the integer constraints in step 3.4.2.1 as the MPW is the set of all organizations where susceptible_vuln(o,v)=1.0. Then, we can simply apply GREEDY-PICK from 3.4.2.2 and obtain an exact solution.

(3.5) Algorithms for“Cost Reduction”: here we present several methods to address this problem.

(3.5.1 ) Integer programming method. As with“probability reduction,” we can address“cost reduction” by finding a solution to a set of integer constraints. We will use the same notation as in 3.4.2. Flowever, there is one key difference, we only consider organizations in set U where susceptible_vuln(o,v)>0.0.

Each o such that the corresponding Y ₀ variable is 1 can be considered part of the solution if the result of the objective function has a value less than X. If the value of the objective function is greater than X, then K must be increased, as this would indicate there is no set of K organizations that can reduce the cost to the desired level.

(3.5.2) Other methods. Various combinatorial methods can be used as alternatives to the above-described integer programming approach - for instance, methods used for subset-sum and related

combinatorial problems.

[0053] Referring to FIG. 5, a screenshot 400 illustrating one exemplary embodiment of the interface 134 is shown that may be implemented with any one or more of the embodiments (150, 200, and 300) of the system 100 described herein, and may be leveraged for, e.g., uploading and identifying sources of aggregation risk.

Embodiments of the interface 134 described in this section is based on software vulnerabilities. However, the interface 400 can also apply to software, hardware, or combinations thereof. As previously described, the interface 134 may be presented by way of a portal or gateway embodied as an API, a browser-based application, a mobile application, or the like. In other words, the interface 134 can be web-based, but instantiations as stand-alone software are also possible. In general, the interface 134 accommodates input of data 132 defining aspects of some external IT system 130 associated with an organization, or other external data (which may be specific to organizations and industry verticals) by a user, by establishing at least a temporary communication link between the computing device 102 and the device 136

implementing the interface 134, so that the user can upload or otherwise make the data 132 available to the computing device 102. Upon accessing the data 132 provided through the interface 134, the computing device 102 can perform one or more computations from the data 132 and return some cyber risk output for one or more of an organization, industry vertical, etc. associated with the IT system 130, in the form of a probability of attack for an organization, or some other output or metric as described herein.

[0054] In some embodiments, using the interface 134, as illustrated by the screenshot 400, a user may upload a file 402 or other data structure for access by the computing device 102. In some embodiments, the file 402 is an implementation of the data 132 specific to an IT system 130 associated with some organization.

Embodiments of the system 100 may allow for a user to upload of multiple CSV files or similar spreadsheets using the drag-drop window 404, and then submit files using a submit button 406. This capability can also be instantiated in other ways such as supplying (from the device 136 to the computing device 102) a URL or link to a repository, folder, database, or similar storage facility.

[0055] In particular for example, in some embodiments, each file 402 may represent the previous results of a vulnerability scan from a different organization. As elaborated upon in more detail in FIG. 6, a vulnerability scan is generally a scan of the data 132 by the computing device 102 or otherwise to map a CPE identifier from the data 132 to a common vulnerability enumeration (CVE) identifier. A given mapping of a CPE identifier to a CVE identifier during the vulnerability scan highlights a possible vulnerability to the the target IT system 130.

[0056] The computing device 102 may aggregate any information of the vulnerability scan and present the results in the form of a reporting window or tab 408 of the interface 134 as shown. For large quantities of organizations, this may require the use of software to easily enable parallelization

[0057] Aggregated results of one or more vulnerability scans, and processing applied to vulnerability scans applied to multiple IT systems according to the functionality of the system 100 described herein may be organized and presented via the interface 134 in any number of formats. In some embodiments, the overall result may comprise a single spreadsheet (or series of database entries) bearing columns resembling the following:

• Vulnerability ID (i.e. by CVE number)

• Description of vulnerability

• Probability of exploitation

• Number of affected organizations

• Total number of affected computer systems (i.e. the sum of affected systems across all organizationsO

• Median number of affected computer systems

• Expected number of infected systems (arising from the

probability of exploitation multiplied by the total number of affected systems)

• Median expected number of infected systems

• Source data (i.e. text from actual hacker community

discussions used to derive the probability)

[0058] In some embodiments, the reporting window 408 of the interface 134 may define Summary Statistics, to include number of organizations, number of vulnerabilities affecting more than one organization, and the range of probability of exploitation of the 90 ^th percentile of vulnerabilities occurring in the population.

[0059] It should be appreciated that the embodiments (150, 200, and 300) of the system 100 are not mutually exclusive, such that the system 100 may be configured to include any number of features from one or more of these embodiments. More specifically, the expressions and variations to the general mathematical expressions of each embodiment are related and are not mutually exclusive to one embodiment or another.

[0060] Referring now to a process flow diagram 1000 of FIG. 6, one possible implementation of various embodiments of the system 100 shall now be described. Referring to block 1002, a first dataset, or any number datasets of the data 112 may be accessed, collected, or acquired by the computing device 102 as illustrated in FIG. 1. The first dataset of the data 112 may include information from, by non-limiting examples, dark web forums, blogs, marketplaces, intelligence threat APIs, data leaks, data dumps, the general Internet or World Wide Web (126), and the like, and may be acquired using web crawling, RESTful FITTP requests, HTML parsing, or any number or combination of such methods. The data 112 may further include information originating from the NVD including CPEs, corresponding CVEs, and CVSS scores. In addition, a second dataset may be accessed by the computing device 102 from data 132

associated with the IT system 130. The data 132, as previously described, may include information about a configuration of the IT system 130 including software, hardware, and/or combinations thereof implemented by the IT system 130; the IT system 130 being associated with an organization related to some industry vertical. In some embodiments, a user may provide the second dataset or otherwise make the second dataset available to the computing device 102 by implementing the interface 134

(detailed in FIG. 5) via the client device 136.

[0061] In one specific embodiment, using the API 119, the first dataset may be acquired from a remote database hosted by, e.g., host server 120. In this embodiment, the host server 120 gathers D2web data from any number of D2web sites or platforms and makes the data accessible to other devices. More particularly, the computing device 102 issues an API call to the host server 120 using the API 119 to establish a RESTful Hypertext Transfer Protocol Secure (HTTPS) connection. Then, the data 112 can be transmitted to the computing device 102 in an HTTP response with content provided in key-value pairs (e.g., JSON). [0062] Once accessed, the first dataset and/or the second dataset may be preprocessed by, e.g., cleaning, formatting, sorting, or filtering the information, or modeling the information in some predetermined fashion so that, e.g., the data 112 is compatible or commonly formatted between the datasets. For example, in some embodiments, the first dataset or the second dataset may be processed by applying text translation, topic modeling, content tagging, social network analysis, or any number or combination of artificial intelligence methods such as machine learning applications.

Any of such data cleaning techniques can be used to filter content of the first dataset from other content commonly discussed in the D2web such as drug-related discussions or pornography.

[0063] Referring to blocks 1004 and 1006, utilizing any number of artificial intelligence methods such as natural language processing, the processor 104 scans the data 112 to identify components of the second dataset associated with CPE identifiers corresponding to CPEs of the first dataset. More specifically, by non-limiting example, the processor 102 conducts a character or keyword search of the second dataset defining the components/inventory of the IT system 130 in view of CPE identifiers and corresponding CPEs from the first dataset. In this manner, the processor 102 identifies possible components of the IT system 130 that are affiliated with at least one CPE (and possible CVE).

[0064] In addition, the processor 102 maps (or leverages pre-defined mappings between CPEs and CVEs) least one of the components of the IT system 130 to a CVE based on an identified CPE associated with the IT system 130. This step identifies at least one vulnerability to the IT system 130. For example, an exemplary technology configuration of the IT system 130 may define a computing environment running Windows Server 2008 on an IBM computing device, and it may be discovered via intelligence from the first dataset that such an exemplary technology configuration is susceptible or vulnerable to an Attack Vector V (which may include, for example, malware, exploits, the known use of common system misconfigurations, or other attack methodology), based on e.g., historical cyber-attacks. In either case, this functionality outputs at least one CVE/attack vector that poses at least some threat to the IT system 130 [0065] Referring to block 1008, the processor 104 may further execute functionality based on any of the embodiments of the system 100 described herein to generate an overall problem or mathematical model, and variants thereof as desired for different applications. As indicated herein, the overall problem may generally define variables such as a population of organizations, a single organization or select organizations of an industry vertical, a vulnerability, a payout threshold or cost, and the like. The overall problem may, using the variables, define an expression for calculating a probability of an attack costing a certain amount in terms of damage, and may consider vulnerabilities of a specific or single IT system and/or a vulnerability known generally to be problematic to an industry vertical comprising a plurality of IT systems (e.g., where it is desired to weigh the risk to an organization but it is further desired to keep the specifics of the technology configuration associated with the organization confidential - such that the overall problem is modeled to assess the probability of an attack to any IT system associated with an industry vertical where IT systems associate with the industry vertical generally implement at least generic versions of the same or similar technology).

[0066] Referring to block 1010, the processor 104 computes a solution to the overall problem to at least calculate a probability of an attack. As indicated in the descriptions of the embodiment 150 and the embodiment 200 of the system 100, computations executed by the processor 104 to solve the overall problem may include exponential-time algorithms, a dynamic programming algorithm, sampling, application of a subset problem, and the like. As further described, algorithms applied and

processed/computed to solve the problem may include variations; e.g., sampling may be biased, cost may be set to“1 ,” and the like. A related model may further be defined and solved to identify organizations to be incentivized to reduce aggregation risk, as set forth in the description of the embodiment 300.

[0067] Computations for defining and solving the expressions herein and processing related algorithms may

Exemplary Computing Device [0068] Referring to FIG. 7, a computing device 1200 is illustrated which may take the place of the computing device 102 be configured, via one or more of an application 1211 or computer-executable instructions, to execute functionality described herein. More particularly, in some embodiments, aspects of the predictive methods herein may be translated to software or machine-level code, which may be installed to and/or executed by the computing device 1200 such that the computing device 1200 is configured to execute functionality described herein. It is contemplated that the computing device 1200 may include any number of devices, such as personal computers, server computers, hand-held or laptop devices, tablet devices,

multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, digital signal processors, state machines, logic circuitries, distributed computing environments, and the like.

[0069] The computing device 1200 may include various hardware components, such as a processor 1202, a main memory 1204 (e.g., a system memory), and a system bus 1201 that couples various components of the computing device 1200 to the processor 1202. The system bus 1201 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.

[0070] The computing device 1200 may further include a variety of memory devices and computer-readable media 1207 that includes removable/non removable media and volatile/nonvolatile media and/or tangible media, but excludes transitory propagated signals. Computer-readable media 1207 may also include computer storage media and communication media. Computer storage media includes removable/non-removable media and volatile/nonvolatile media implemented in any method or technology for storage of information, such as computer-readable

instructions, data structures, program modules or other data, such as RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store the desired information/data and which may be accessed by the computing device 1200. Communication media includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. For example, communication media may include wired media such as a wired network or direct-wired connection and wireless media such as acoustic, RF, infrared, and/or other wireless media, or some combination thereof. Computer-readable media may be embodied as a computer program product, such as software stored on computer storage media.

[0071] The main memory 1204 includes computer storage media in the form of volatile/nonvolatile memory such as read only memory (ROM) and random access memory (RAM). A basic input/output system (BIOS), containing the basic routines that help to transfer information between elements within the computing device 1200 (e.g., during start-up) is typically stored in ROM. RAM typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processor 1202. Further, data storage 1206 in the form of Read-Only Memory (ROM) or otherwise may store an operating system, application programs, and other program modules and program data.

[0072] The data storage 1206 may also include other removable/non removable, volatile/nonvolatile computer storage media. For example, the data storage 1206 may be: a hard disk drive that reads from or writes to non-removable, nonvolatile magnetic media; a magnetic disk drive that reads from or writes to a removable, nonvolatile magnetic disk; a solid state drive; and/or an optical disk drive that reads from or writes to a removable, nonvolatile optical disk such as a CD-ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media may include magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The drives and their associated computer storage media provide storage of computer-readable instructions, data structures, program modules, and other data for the computing device 1200.

[0073] A user may enter commands and information through a user interface 1240 (displayed via a monitor 1260) by engaging input devices 1245 such as a tablet, electronic digitizer, a microphone, keyboard, and/or pointing device, commonly referred to as mouse, trackball or touch pad. Other input devices 1245 may include a joystick, game pad, satellite dish, scanner, or the like. Additionally, voice inputs, gesture inputs (e.g., via hands or fingers), or other natural user input methods may also be used with the appropriate input devices, such as a microphone, camera, tablet, touch pad, glove, or other sensor. These and other input devices 1245 are in operative connection to the processor 1202 and may be coupled to the system bus 1201 , but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). The monitor 1260 or other type of display device may also be connected to the system bus 1201. The monitor 1260 may also be integrated with a touch-screen panel or the like.

[0074] The computing device 1200 may be implemented in a networked or cloud-computing environment using logical connections of a network interface 1203 to one or more remote devices, such as a remote computer. The remote computer may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computing device 1200. The logical connection may include one or more local area networks (LAN) and one or more wide area networks (WAN), but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

[0075] When used in a networked or cloud-computing environment, the computing device 1200 may be connected to a public and/or private network through the network interface 1203. In such embodiments, a modem or other means for establishing communications over the network is connected to the system bus 1201 via the network interface 1203 or other appropriate mechanism. A wireless networking component including an interface and antenna may be coupled through a suitable device such as an access point or peer computer to a network. In a networked environment, program modules depicted relative to the computing device 1200, or portions thereof, may be stored in the remote memory storage device.

[0076] The computing device 1200 is just one example of a physical device that may be implemented to perform the computations for defining and solving the expressions and processing related algorithms set forth herein. Many variations and related computing approaches are contemplated. For example, multiple processors may be clustered and balanced to reduce computational overhead to one machine and leverage the computational resources of a cluster. Cluster parallel machines and hybrid cluster parallel machines may be implemented. Scalable multithreaded shared memory supercomputer architectures may further be leveraged such as CRAY MTA to parallelize algorithms described herein. Quantum or photonic computing devices may further be leveraged to enhance processing of the functionality described herein.

[0077] Certain embodiments are described herein as including one or more modules. Such modules are hardware-implemented, and thus include at least one tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. For example, a hardware-implemented module may comprise dedicated circuitry that is permanently configured (e.g., as a special-purpose processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) to perform certain operations. A hardware-implemented module may also comprise programmable circuitry (e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software or firmware to perform certain operations. In some example embodiments, one or more computer systems (e.g., a standalone system, a client and/or server computer system, or a peer-to-peer computer system) or one or more processors may be configured by software (e.g., an application or application portion) as a hardware-implemented module that operates to perform certain operations as described herein.

[0078] Accordingly, the term“hardware-implemented module”

encompasses a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner and/or to perform certain operations described herein. Considering embodiments in which hardware-implemented modules are temporarily configured (e.g., programmed), each of the hardware-implemented modules need not be configured or instantiated at any one instance in time. For example, where the hardware-implemented modules comprise a general-purpose processor configured using software, the general-purpose processor may be configured as respective different hardware-implemented modules at different times. Software may accordingly configure the processor 1202, for example, to constitute a particular hardware- implemented module at one instance of time and to constitute a different hardware- implemented module at a different instance of time.

[0079] Hardware-implemented modules may provide information to, and/or receive information from, other hardware-implemented modules. Accordingly, the described hardware-implemented modules may be regarded as being communicatively coupled. Where multiple of such hardware-implemented modules exist

contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the hardware-implemented modules. In embodiments in which multiple hardware-implemented modules are configured or instantiated at different times, communications between such hardware- implemented modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware-implemented modules have access. For example, one hardware-implemented module may perform an operation, and may store the output of that operation in a memory device to which it is communicatively coupled. A further hardware-implemented module may then, at a later time, access the memory device to retrieve and process the stored output.

Hardware-implemented modules may also initiate communications with input or output devices.

[0080] Computing systems or devices referenced herein may include desktop computers, laptops, tablets e-readers, personal digital assistants, smartphones, gaming devices, servers, and the like. The computing devices may access computer- readable media that include computer-readable storage media and data transmission media. In some embodiments, the computer-readable storage media are tangible storage devices that do not include a transitory propagating signal. Examples include memory such as primary memory, cache memory, and secondary memory (e.g., DVD) and other storage devices. The computer-readable storage media may have instructions recorded on them or may be encoded with computer-executable instructions or logic that implements aspects of the functionality described herein. The data transmission media may be used for transmitting data via transitory, propagating signals or carrier waves (e.g., electromagnetism) via a wired or wireless connection.

[0081] It should be understood from the foregoing that, while particular embodiments have been illustrated and described, various modifications can be made thereto without departing from the spirit and scope of the invention as will be apparent to those skilled in the art. Such changes and modifications are within the scope and teachings of this invention as defined in the claims appended hereto.