TECHNICAL FIELD

[0001] The disclosure generally relates to the field of machine learning, and more particularly relates to co-optimization of materials development and product development based a capability map derived using machine learning.

BACKGROUND

[0002] Original equipment manufacturers (OEMs) design products and parts that achieve desired outcomes in a range of dimensions, such as performance specifications, test qualifications, cost of production, manufacturability targets, and so on. As part of the design process, these OEMs simulate performance, test outcomes, and manufacturability of a product or part while varying geometry and material. OEMs may select materials from a catalogue of available materials, or may write a specification for a new material and ask their vendors to develop the desired new material. However, where a new material is desired, it may be unclear what performance specifications are achievable by a materials supplier, or which suppliers might be most capable of achieving a given performance specification. Moreover, some materials specifications might be extremely difficult for a materials developer to achieve, while others may be relatively easy for the same materials developer to achieve, though it may be unclear at the outset what the level of difficulty is. Because the OEM lacks the data on how difficult various materials specifications are for a given materials vendor to achieve through new materials development, the OEM cannot efficiently co optimize the part performance with the materials specifications.

BRIEF DESCRIPTION OF DRAWINGS

[0003] The disclosed embodiments have other advantages and features which will be more readily apparent from the detailed description, the appended claims, and the accompanying figures (or drawings). A brief introduction of the figures is below.

[0004] Figure (FIG.) 1 illustrates one embodiment of an environment for interfacing a user and a materials supplier with design space evaluation service 130.

[0005] FIG. 2 illustrates one embodiment of exemplary modules and databases of design space evaluation service 130.

[0006] FIG. 3 illustrates an exemplary data flow for generating a capability map.

[0007] FIG. 4 illustrates an exemplary data flow for identifying top materials candidates to meet materials performance specifications.

[0008] FIG. 5 depicts exemplary activity by both a materials supplier and an OEM to achieve development of new material to meet materials specifications.

[0009] FIG. 6 depicts an exemplary representation of a capability map using user-selected dimensions.

[0010] FIG. 7 depicts an exemplary user interface for configuring a representation of a capability map.

[0011] FIG. 8 depicts an exemplary user interface facing a materials supplier based on experiments corresponding to selections of material properties by the OEM.

[0012] FIG. 9 is a block diagram illustrating components of an example machine able to read instructions from a machine-readable medium and execute them in a processor (or controller).

[0013] FIG. 10 is an exemplary flowchart of a process for generating a capability map.

DETAILED DESCRIPTION

[0014] The Figures (FIGS.) and the following description relate to preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of what is claimed.

[0015] Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying figures. It is noted that wherever practicable similar or like reference numbers may be used in the figures and may indicate similar or like functionality. The figures depict embodiments of the disclosed system (or method) for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.

CONFIGURATION OVERVIEW

[0016] One embodiment of a disclosed system, method and computer readable storage medium includes generating a capability map. In an embodiment, a system receives training data from a materials supplier, the training data including a plurality of components, the components including materials and/or chemicals, and the components possibly including a plurality of processing steps and/or parameters. The training data, for each respective component, may also include a plurality of respective performance properties. The system receives one or more design spaces from the materials supplier, the design spaces comprising components available to the materials supplier and tool capabilities of tools available to the materials supplier for manipulating the materials and chemical ingredients.

[0017] The system trains the machine learning model using the training data to take as input a design space for the materials supplier, and to output materials performance predictions which can be statistically aggregated into capability map of the materials supplier. The capability map may be a user interface driven by a data structure storing a difficulty of developing materials and/or chemicals having a given set of performance properties.

DESIGN SPACE EVALUATION SERVICE

[0018] Figure (FIG.) 1 illustrates one embodiment of an environment for interfacing a user and a materials supplier with design space evaluation service 130. FIG. 1 includes client device 110, network 120, design space evaluation service 130, and materials supplier 140. Client device 110 may be operated by any user wishing to evaluate capabilities of materials supplier 140 with respect to a desired specification. In an embodiment, client device 110 is operated by an original equipment manufacturer (OEM) that is exploring a new design specification. Client device 110 inputs desired specifications into a user interface (the user interface and design space particulars are discussed below with respect to FIGS. 2-7), and those parameters are transmitted over network 120 (which may be any network, such as the Internet) to design space evaluation service 130. Client device 110 may select materials supplier 140 from several candidate materials suppliers, and may select more than one materials supplier.

[0019] Separately, materials supplier 140 transmits training data to design space evaluation service 130. The training data includes a plurality of components that include input materials and/or chemicals as well as processing setpoints and/or conditions, and, for respective combinations of the plurality of components, a plurality of respective performance properties. Design space evaluation service 130 uses the training data to train one or more machine learning models to take new design spaces as input, and to output predicted performance properties which can be statistically aggregated into a capability map. Thus, where materials supplier 140 provides to design space evaluation service 130 a new design space, design space evaluation service 130 is able to input the new design space into the trained model, and output capabilities of the design space for consumption by the user of client device 110. Design spaces each include a subset of relevant candidate components (e.g., materials and/or chemicals) available to materials supplier 140, as well as tool capabilities of tools available to materials supplier 140 for manipulating the candidate components.

[0020] Design space evaluation service 130 provides the capability map to client device 110. Client device 110 may then input the desired specifications into the capability map, and may view likelihoods of success of materials supplier 140 being able to manufacture a material that satisfies the desired specifications. This helps, e.g., inform an OEM as to what materials suppliers are likely to succeed in hitting desired specifications.

[0021] FIG. 2 illustrates one embodiment of exemplary modules and databases of design space evaluation service 130. As depicted in FIG. 2, design space evaluation service 130 includes design space module 231, training module 232, capability map module 233, and visualization module 234, as well as design space database 240 and model database 241. The modules and databases depicted in FIG. 2 are merely exemplary; fewer or more modules may be used to achieve the functionality described herein.

[0022] Design space module 231 receives design space information from materials suppliers (e.g., material supplier 140). The term design space, as used herein, may refer to a collection of candidate materials including one or more candidate components available to a materials supplier and tool capabilities of tools available to the materials supplier for manipulating the candidate components. The components may include chemicals, materials, or some combination thereof. The tools may include any industrial equipment, such as mixing equipment, heating equipment, smelting equipment, and the like. These design spaces may encompass some subset of the parameter space of materials the supplier could manufacture given specified constraints. A design space may include variation of material synthesis, structure, composition and processing. An example design space for a paint formulations company could include all of the various combinations of available binder, solvent, pigment, and additives in the ratios allowable by their formulation processes.

Another example design space for a paint formulations company could include the various combinations of available binder, solvent, pigment, and additives in the ratios allowable by their formulation processes for which the total recipe cost is below a specified threshold. Another example design space could be a subset of the previous design space with a limited number of candidate paint formulations sampled from the larger set of possibilities.

[0023] Design space module 231 may receive design space information from multiple material suppliers and may store this information in design space database 240. Design space module 231 may periodically, at various time intervals receive, or when design space information changes, either through a push or pull mechanism, updated design space information from the materials suppliers, thus ensuring design space module 231 maintains current design space information in design space database 240.

[0024] Training module 232 trains one or more machine learning models to take a design space as input, and to output a material performance for each candidate within the design space. In order to train the machine learning model(s), training module 232 receives training data from each materials supplier. The training data includes components, such as input materials and/or chemicals. The training data also includes, for respective combinations of the plurality of components, a plurality of respective performance properties. The training data may include additional information, such as indicia of how the combinations were made (e.g., indicia of the tools used and parameters of how the tools were used (such as amount of heat applied for how long, etc.)).

[0025] After training a given model, training module 232 stores the model in model database 241. Training module 232 may receive updated training data and may augment training of a given model periodically to improve the accuracy of that model. Design space evaluation service 130 may train different machine learning models for different materials suppliers. The trained model is trained to predict materials performance, i.e. the materials properties of interest (including, but not limited to: electrical, mechanical, thermal, and optical properties), as a function of, e.g., the material composition, structure, synthesis, and processing. The model is trained to be fit to existing data that include varying material composition and processing, and the associated material performance. The model is also trained to estimate the uncertainty associated with each prediction. In other words, for each prediction the model is able to express the distribution over which the model predicts the true material property for the given composition and processing. The model may include some combination of one or more machine learning models and / or analytical formulae. The machine learning algorithms may include, for example, random forest regressors, Gaussian process regressors, support vector machines, and/or neural networks.

[0026] Capability map module 233 receives a design space from a materials supplier (e.g., materials supplier 140), and inputs the design space into the machine learning model. Capability map module 233 receives, as output from the machine learning model, the predicted performances of the candidates in the design space and the estimated uncertainty associated with each prediction. The predicted material properties and uncertainties of each candidate in the design space are then statistically aggregated into a capability map. As used herein, the term capability map may refer to the aggregated performance of a catalog of materials not yet created, but which potentially could be created if demand existed. The capability map indicates a likelihood that a given design space contains materials whose properties extend into any given point in an n-dimensional material property space (interchangeably referred to herein as an “output space”). Likelihood here refers to a measure of confidence that a given specification can be achieved using the given design space, and/or it may represent a measure of difficulty of achieving a given specification using the given design space. During statistical aggregation, the predicted material properties and uncertainties of all candidates in the design space are combined to provide the likelihood for every point in output space through the use of design space metrics. In an embodiment, statistical aggregation may be done using a design space metric such as the Maximum Joint Probability Density (MJPD), which indicates a probability of reaching a given region in output space given the best candidate in the design space. In an embodiment, a statistical aggregation may be done using a design space metric such as Summed Probability Density (SPD), which gives a predicted density of candidates in the output space. Other design space metrics also exist. In general, these metrics serve to aggregate the predictions over a design space and use that aggregation to estimate the likelihood for each point in output space. Further detail on MPJD, SPD and other metrics are described below.

[0027] Capability maps enable OEMs to co-optimize product performance with material specifications while taking into account the achievability of various materials specifications. This is done without giving access to OEMs to materials supplier data on composition and processing, which is often highly proprietary. In order to safeguard the proprietary data, training module 232 may discard training data after it is used to train a machine learning model for a given materials supplier. Capability maps enable OEMs to determine achievability of a given materials performance target before developing that material, which reduces cost, time, and materials required to develop a material with the desired performance. Thus, the capability map enables the OEM to take control of the tradeoff between material performance and cost/lead-time for developing that material.

[0028] Visualization module 234 generates for display (e.g., on client device 110) a visual representation of the capability map. The visualization may be generated using any of a number of candidate metrics. In an embodiment, a metric may be the Visualization module 234 may visualize these metrics using two-dimensional contour plots, where materials of interest are on the x and y axes, and the SPD or MJPD metrics are used to color the contours. Examples of this are described in further detail below with respect to FIGS. 6-8.

[0029] The metrics incorporate a similar treatment of candidates and their predicted properties. Each output is described as a random variable T _k with probability density f _!i. In an embodiment, the random variable will be distributed according to a normal distribution:

T _k ~N ( i _ί , a _k ). In another embodiment, the random variable will be distributed according to other distributions, including multi-modal distributions. Because multiple objectives are of concern, a candidate with d > 1 objectives may be defined as a set of random variables with a joint distribution p\ C = {T _k } _k=1 ~p. In an embodiment, an assumption in these approaches is that the objectives are independent of one another, such that the joint probability density can be calculated from: p _c = W _{=i (} p _k. This, however, can be a poor assumption in some cases where outputs are co-variant. Despite this simplification, the resulting capability map is nevertheless useful for understanding which regions of output space are achievable with a given design space. In another embodiment, objectives are not assumed to be independent of one another, such that the joint probably density cannot be factored into a product of marginal probability densities.

[0030] In an embodiment, visualization module 234 treats a design space of n candidates as a set of candidates, each being a set of random variables, each with its own distribution described by the objectives’ means and uncertainties D = {Ci~Pi} _=1. The MJPD takes the maximum value of the joint probability density for each gridded point in output space, t°, over all n candidates in the design space, D: MJPDo(t ⁰ )=max p _c. (t°). The MJPD metric

1 £i£n ¹ thus provides the value of the joint probability density for the candidate most likely to achieve the property values at a given point in output space.

[0031] Visualization module 234 may use SPD to sum the joint probability density over all n candidates at a given point in output space: SPD/ (t°) = The resulting capability map thus enables visualization module 234 to indicate density of design space predictions in the output space, factoring in the uncertainty of these predictions.

[0032] When outputting an MJPD plot, visualization module 234 may indicate within the plot whether the current data and model suggests that a region of performance space is attainable by any single candidate in the design space. When outputting a SPD plot, visualization module 234 may indicate how easy it is to find a candidate in any given region of performance space. Other metrics are also possible. For example, visualization module 234 may multiply the SPD by the number of candidates in the design space to yield a candidate number density, which provides information about how many candidates are expected to have performance in a given region. Other metrics used by visualization module 234 may include the Number of Likely Candidates (NLC) which at each point in the performance space is the number of candidates within one standard deviation with respect to their individual predictive distributions, and the Fraction of Likely Candidates (FLC) which is the fraction of candidates within one standard deviation with respect to their individual predictive distributions.

[0033] Other possible metrics, such as the Cumulative Likelihood of Improvement (CLI), evaluate the probability that the design space contains at least N materials with specified performance. Similarly, the Predicted Fraction of Improved Candidates (PFIC) is the fraction of candidates whose performance is predicted to exceed each point in materials property space. Any of these metrics may be used, individually or in combination, to build a visual representation of a capability map by visualization module 234.

[0034] In an embodiment, rather than generate a visualization of the capability map, design space evaluation service 130 may train the machine learning model to directly to predict the performance of specific materials or chemicals given their synthesis, composition, structure, and processing. Based on these ML predictions, the OEM can determine which candidate material(s) in the design space is/are predicted to be closest to hitting a desirable material performance specification. This is further described with respect to FIG. 4.

[0035] FIG. 3 illustrates an exemplary data flow for generating a capability map. Workflow 300 begins at 310, where training module 232 receives training data and uses the training data to fit machine learning model 320. Capability map module 233 receives design space 330 from a materials supplier 140 (e.g., by retrieving design space 330 from design space database 240, or by receiving design space 330 directly from materials supplier 140), and inputs design space 330 into machine learning model 320 to yield predicted performance of the design space 340. The predicted performance of the design space 340 (that is, raw data forming a capability map) is applied to design space visualization metrics 350 to yield capability map 360 (e.g., using visualization module 234).

[0036] FIG. 4 illustrates an exemplary data flow for identifying top materials candidates to meet materials performance specifications. Data flow 400 begins with training data 410 being used to train and fit machine learning model 420. Design space 430 (e.g., as received from materials supplier 140) is input into machine learning 420. Capability map module 233 obtains a predicted performance of the design space 440. Separately, design space evaluation service 130 may have received input of desired performance specifications from client device 110. Rather than, or in addition to, having the user of client device 110 navigate a visual representation of a capability map, design space evaluation service 130 may compare the predicted performance of the design space 440 to the desired performance specifications, and may, based on the comparison, identity top materials candidates 450. The identified top materials candidates 450 may be shown using visualization module 234, and may include indicia of likelihood of success of achieving each of, and/or all of, the desired performance specifications.

[0037] FIG. 5 depicts exemplary activity by both a materials supplier and an OEM to achieve development of new material to meet materials specifications. As depicted in scenario 500, design space 130 generates a capability map based on information (e.g., training data and design space information) received from materials supplier 540 (which carries the same weight of description of materials supplier 140). Using design space evaluation service 130, OEM 510 is able to simulate the viability of a given product (that is, a product having desired specifications that OEM 510 provides as input to design space evaluation service). For example, OEM 510 is able to simulate product performance 512 (that is, the likelihood that, if a product is generated, it would meet each specification input by OEM 510), simulate product manufacturability 513 (that is, the likelihood that a given combination of components is manufacturable), and simulate product test outcomes 514. [0038] Based on the results of the simulations and the capability map, OEM 510 is able to co-optimize its simulations with the capabilities of materials suppler 540 without materials supplier 540 having to hand over to OEM 510 proprietary details. That is, OEM 510 applies the results of the simulations to the capability map, optimizing 515 for product performance, manufacturability, and test outcomes over varying materials performance, taking into account achievability of materials performance. OEM 510 is then able to select a desired materials specification 516 (e.g., by selecting the materials specification based on a visual representation provided by visualization module 234). In an embodiment, design space evaluation service 130 transmits a command 547 to materials supplier 540 to develop a new material to meet the selected materials specifications.

[0039] FIG. 6 depicts an exemplary representation of a capability map using user-selected dimensions. User interface 600 depicts a visual representation of a capability map generated to represent capabilities of a design space of a materials supplier 140. User interface 600 may be generated for display by client device 110 (e.g., to an OEM). User interface 600 may be rendered by a dedicated application installed to client device 110, or may be rendered by a browser operated by client device 110 that is retrieving information from design space evaluation service 130. User interface 600 may show a two-dimensional graphical depiction of any metric, with y-axis 610 and x-axis 620 corresponding to any parameter selected by the user. The manner in which the user may select parameters for the axes is described with respect to FIG. 7 below. The metric guiding the graphical representation in FIG. 6 may also be configured by the user.

[0040] FIG. 7 depicts an exemplary user interface for configuring a representation of a capability map. User interface 700 includes specification options 710, axes options 720, and visual representation 730. Specification options 710 include selectable options to designate parameters (e.g., yield strength, hardness, fatigue strength, and so on) by a user of client device 110. Specification option 710 also include selectable options to designate requirements with respect to those parameters (e.g., minimum and maximum values, types, and so on). Axes options 720 include selectable options for the user of client device 110 to select which parameters will dictate the X and Y axes of visual representation 730. Visual representation 730 acts in a manner similar to that described with respect to user interface 600 of FIG. 6.

[0041] FIG. 8 depicts an exemplary user interface facing a materials supplier based on experiments corresponding to selections of material properties by the OEM. FIG. 8 depicts user interface 800, having target properties 810 and suggested experiments 820. In an embodiment, the user of client device 110 manually populates target properties 810 and/or suggested experiments 820. In an embodiment, design space evaluation service 130 automatically populates target properties 810 and/or suggested experiments 820 based on input from the user and/or top candidates identified by design space evaluation service 130. Target properties 810 may indicate the values input by the user into specification option 710 (e.g., ranges and/or types of values). Design space evaluation service 130 may populate suggested experiments 820 by identifying combinations of components that are most likely to yield a material that satisfies target properties 810 from the capability map. Design space evaluation 130 identifies the most likely components based on, e.g., the uncertainty scores indicated in the capability map.

[0042] FIG. 9 is a block diagram illustrating components of an example machine able to read instructions from a machine-readable medium and execute them in a processor (or controller). [0043] FIG. 10 is an exemplary flowchart of a process for generating a capability map. Process 1000 begins with design space evaluation service 130 receiving 1002 one or more design spaces from a materials supplier, the one or more design spaces comprising candidate components available to the materials supplier, and tool capabilities of tools available to the materials supplier for manipulating the candidate components. This receiving may be performed by design space module 231.

[0044] Design space evaluation service inputs 1004 a design space of the one or more design spaces into a machine learning model, the machine learning model trained using training data received from the materials supplier, the training data comprising a plurality of components, the components including input materials and/or chemicals, and, for respective combinations of the plurality of components, a plurality of respective performance properties. The model may be retrieved from model database 241. The model may be trained using training module 232. The training data may include at least one of a plurality of processing elements (e.g., properties, tools used to manufacture, etc.) and a plurality of processing parameters (e.g., minimum and maximum values, types, capabilities of tools (e.g., heat achievable by given tool), etc.).

[0045] Design space evaluation service 130 receives 1006 as output from the machine learning model respective predicted values for each material property as well as the uncertainty measure associated with each predicted value, of each of a plurality of design space candidates. Design space evaluation service 130 statistically aggregates 1008 the respective predicted performances of each candidate in the design space to generate a capability map of the materials supplier, the capability map being a data structure storing possible combinations of performance properties and a respective difficulty of developing a composition with that combination of performance properties, the composition being a combination of the candidate components. The capability map may be generated and/or stored using capability map module 233.

[0046] Design space evaluation service 130 outputs 1010 a user interface for display to a user indicating data of the capability map. Design space evaluation service 130 may generate the user interface using visualization module 234. In an embodiment, the capability map is in a space comprising three or more dimensions, wherein the user interface comprises a selectable option to select two properties of interest, and wherein responsive to receiving a selection of the two properties of interest, outputting a graph showing levels of achievability in two dimensions corresponding to the two properties of interest. In another embodiment, the capability map is in a space comprising any number of dimensions where the user can query the likelihood by entering the target range in each of those dimensions and receive back a numerical metric corresponding to the achievability in those n-dimensional target ranges. It is then possible to conduct mathematical operations on these capability maps, such as, but not limited to, integrating over these metrics to filter out one of the dimensions, or subtracting one capability map from another to highlight areas where there are large differences between the design spaces, or applying maximum or minimum functions across multiple capability maps to combine multiple capability maps into one.

[0047] In an embodiment, an application programming interface enables a program to query the capability map automatically, returning the likelihood of achieving output properties to the program. An application for this is for OEMs to optimize part design using modeling and optimization software while constraining the optimization within a particular level of likelihood of achieving specific material properties.

EXEMPLARY USE CASE - STEEL ALLOY

[0048] To set the stage for an exemplary use case, we introduce a fictional character Rosa the Researcher, who works at a materials company AlloysX. Rosa is a materials researcher and helps AlloysX develop new materials. We also introduce the fictional character Derek the Designer, who works at an OEM called FlightParts. Derek is responsible for the design of a new widget at FlightParts.

[0049] Derek the Designer has been told to design a widget that will go into a new design of an aircraft engine that his company, FlightParts, is working on. This widget will be near a hot part of the engine, be load-bearing and has another part that slides over it constantly during engine operation. The part therefore needs to be high performance, and requires (1) high thermal conductivity to remove away heat generated by the engine, (2) great fatigue strength to bear with cycling loads found in the engine, and (3) be of high hardness to be able to withstand another part sliding over it continuously, and (4) be as light-weight as possible. Derek knows that he can improve conductivity and fatigue strength by making the part thicker, but this then greatly increases the weight of the part and is ultimately undesirable. If he could find strong and light materials that are also high hardness and have good thermal properties, the part could be designed much better.

[0050] Without the benefit of design space evaluation service 130, Derek would go through a laborious and inefficient process. Derek would see what materials are currently available to him. He will go to his Procurement department and ask them for the catalogues he has on file from all their suppliers. Procurement hands over the AlloysX catalogue of materials. Derek goes through this list of hundreds of materials, but does not find the perfect material, although he does find one that might work if he adjusts the design for it. Derek now has two options: 1. He uses this adequate material and designs around its limitations by adjusting the geometry of the part. 2. He works with Paul to put out a Request for Proposal (RFP) to make a new material that is ideally suited for Derek's application. However, this process is long, indirect, and Derek has no idea what material specs are reasonable enough to put in the proposal. Plus, when Rosa as part of AlloysX comes across this RFP there is no framework by which she can quantify how achievable the request is, other than her using her expert judgement. Not having a quantitative sense of achievability, it is very difficult to accurately price the materials development process. There are weeks of back-and-forth communication, with tedious analysis by both parties, until a set of target material properties and contract cost are even decided upon. Only then does progress towards a new material begin, and waste in resources occurs in undue consumption of raw materials, electricity, and so on toward failed experiments.

[0051] Using the systems and methods disclosed herein, design space evaluation service 130 trains one or more machine learning models for AlloysX based on training data provided by AlloysX. Design space evaluation service 130 receives a design space from Rosa and generates a capability map for AlloysX that Derek can explore to figure out what properties he wants to aim for. He can quantify how hard it might be to achieve certain properties and can therefore make an informed decision on how much of a moonshot he is willing to make the project. Derek will then communicate his ideal properties (e.g., via design space evaluation service 130) to Rosa and the rest of the AlloysX team.

[0052] The advantages are not only prevalent to an OEM, but are also prevalent to materials supplier 140. Where AlloysX is the materials supplier 140, Rosa has experimental data on about 450 steel alloys, including their composition, process parameters and output properties (hardness, yield strength, thermal conductivity, density, fatigue strength, etc.). In general, these output properties can include mechanical properties, thermal properties, optical properties, electrical properties, chemical, or other properties. Rosa takes the data that she has from her experiments and simulations for steel alloys and creates a predictive machine learning model using design space evaluation service 130 as discussed above. Rosa then takes other candidates in her design space (i.e. other steel that she could make with various alloy compositions and processing parameters) and using design space evaluation service 130, makes predictions on them on all, for each of the six material properties. These predictions are probability distributions; predicting the correct value with 100% accuracy may not be feasible, but design space evaluation service 130, using machine learning models with quantified predictive uncertainty, estimates the range where the true value is most likely to fall given the available data.

[0053] From these predictions with uncertainty estimates, the design space evaluation service 130 generates a capability map, which can be shared with AlloysX's customers (e.g., client device 110) via the design space evaluation service 130. Rosa might repeat this entire process for different design spaces. For example, she might create a capability map for aluminum alloys, or for steel alloys that do not use any chromium, or any other set of experiments that she thinks might make sense.

[0054] Following this example, an example point in a capability map might show that, at point: (hardness: 800 HB, yield strength: 580 Mpa, thermal conductivity: 90 W/m*K, density: 8023 kg/m3, fatigue strength: 230 MPa, manufacturability: good}, the MPJD is .08. The example might show values for any other metric discussed herein. Importantly, at each point in the output space, no information on how these material properties is achieved is included, thus protecting the proprietary data of the materials supplier by anonymizing the recipe but providing material performance information to client device 110.

[0055] Mathematical operations can be done on the capability map. For instance, Derek does not care about yield strength, so Derek can command design space evaluation service 130 to integrate the MPJD over all possible yield strength values to factor the yield strength out. Any other operation might also be applied. One may integrate over some dimensions until only two dimensions are left to be able to visualize the remaining dimensions and analyze trade-offs; in some embodiments, the two-dimensional visualizations disclosed herein are derived using such operations.

[0056] To further exemplify the advantages of the systems and methods disclosed herein, we step through an end-to-end workflow for both Derek and Rosa, beginning with Derek. Derek logs in to a portal of Alloy sX that connects to design space evaluation service 130, and goes to the capability map for steels (e.g., a capability map corresponding to the visualization shown in FIG. 7). Derek starts investigating and fills out the hardness and the density requirements (e.g., in specification options 710. He notices that as he puts in more restrictive requirements, the achievability metric begins to drop. Also, Derek enters the X and Y axis so that he can easily explore the trade-offs in risk that his target specifications might take a significant time to achieve, or may not even be achievable at all. Derek enjoyed an improved user interface that enabled him to browse through not just existing AlloysX materials, but also new potential materials that AlloysX could develop. Derek is also happy that he didn't have to get on a phone with AlloysX's sales team and to slowly get information about what new materials they could develop, and how achievable those targets would be. That process usually takes weeks, with sales having to check with their technical teams to get an answer. And even then, the technical teams were rarely able to give a clear answer about how feasible a set of properties are.

[0057] With respect to Rosa’s workflow, Rosa receives a notification saying that a new material request has been entered from design space evaluation service 130. Rosa logs into the design space evaluation service 130 and goes to a page corresponding to the request (e.g., the page depicted in FIG. 8). Rosa is confers with Derek about the project and negotiates terms for proceeding with experimentation. Rosa’s use of a quantitative, data-driven framework of design space evaluation service 130 reduces experimentation, preserving resources that would otherwise go to wasteful and unfruitful experimentation. Rosa may use design space evaluation service’s machine learning models to suggest which experiments should be run next to most quickly hit Derek’s materials performance targets (e.g., top materials candidate recommendations described with respect to FIG. 4).

COMPUTING MACHINE ARCHITECTURE

[0058] FIG. (Figure) 9 is a block diagram illustrating components of an example machine able to read instructions from a machine-readable medium and execute them in a processor (or controller). Specifically, FIG. 9 shows a diagrammatic representation of a machine in the example form of a computer system 900 within which program code (e.g., software) for causing the machine to perform any one or more of the methodologies discussed herein may be executed. The program code may be comprised of instructions 924 executable by one or more processors 902. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server machine or a client machine in a server- client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

[0059] The machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a smartphone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions 924 (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute instructions 924 to perform any one or more of the methodologies discussed herein.

[0060] The example computer system 900 includes a processor 902 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application specific integrated circuits (ASICs), one or more radio-frequency integrated circuits (RFICs), or any combination of these), a main memory 904, and a static memory 906, which are configured to communicate with each other via a bus 908. The computer system 900 may further include visual display interface 910. The visual interface may include a software driver that enables displaying user interfaces on a screen (or display). The visual interface may display user interfaces directly (e.g., on the screen) or indirectly on a surface, window, or the like (e.g., via a visual projection unit). For ease of discussion the visual interface may be described as a screen. The visual interface 910 may include or may interface with a touch enabled screen. The computer system 900 may also include alphanumeric input device 912 (e.g., a keyboard or touch screen keyboard), a cursor control device 914 (e.g., a mouse, a trackball, a joystick, a motion sensor, or other pointing instrument), a storage unit 916, a signal generation device 918 (e.g., a speaker), and a network interface device 920, which also are configured to communicate via the bus 908. [0061] The storage unit 916 includes a machine-readable medium 922 on which is stored instructions 924 (e.g., software) embodying any one or more of the methodologies or functions described herein. The instructions 924 (e.g., software) may also reside, completely or at least partially, within the main memory 904 or within the processor 902 (e.g., within a processor’s cache memory) during execution thereof by the computer system 900, the main memory 904 and the processor 902 also constituting machine-readable media. The instructions 924 (e.g., software) may be transmitted or received over a network 926 via the network interface device 920.

[0062] While machine-readable medium 922 is shown in an example embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions (e.g., instructions 924). The term “machine-readable medium” shall also be taken to include any medium that is capable of storing instructions (e.g., instructions 924) for execution by the machine and that cause the machine to perform any one or more of the methodologies disclosed herein. The term “machine-readable medium” includes, but not be limited to, data repositories in the form of solid-state memories, optical media, and magnetic media.

ADDITIONAL CONFIGURATION CONSIDERATIONS

[0063] Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0064] Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute either software modules (e.g., code embodied on a machine-readable medium or in a transmission signal) or hardware modules. A hardware module is tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein.

[0065] In various embodiments, a hardware module may be implemented mechanically or electronically. For example, a hardware module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) to perform certain operations. A hardware module may also comprise programmable logic or circuitry (e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations. [0066] Accordingly, the term “hardware module” should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein. As used herein, “hardware-implemented module” refers to a hardware module. Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where the hardware modules comprise a general-purpose processor configured using software, the general- purpose processor may be configured as respective different hardware modules at different times. Software may accordingly configure a processor, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.

[0067] Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple of such hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).

[0068] The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor- implemented modules that operate to perform one or more operations or functions. The modules referred to herein may, in some example embodiments, comprise processor- implemented modules.

[0069] Similarly, the methods described herein may be at least partially processor- implemented. For example, at least some of the operations of a method may be performed by one or processors or processor-implemented hardware modules. The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processor or processors may be located in a single location (e.g., within a home environment, an office environment or as a server farm), while in other embodiments the processors may be distributed across a number of locations.

[0070] The one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application program interfaces (APIs).)

[0071] The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the one or more processors or processor- implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the one or more processors or processor-implemented modules may be distributed across a number of geographic locations.

[0072] Some portions of this specification are presented in terms of algorithms or symbolic representations of operations on data stored as bits or binary digital signals within a machine memory (e.g., a computer memory). These algorithms or symbolic representations are examples of techniques used by those of ordinary skill in the data processing arts to convey the substance of their work to others skilled in the art. As used herein, an “algorithm” is a self-consistent sequence of operations or similar processing leading to a desired result. In this context, algorithms and operations involve physical manipulation of physical quantities. Typically, but not necessarily, such quantities may take the form of electrical, magnetic, or optical signals capable of being stored, accessed, transferred, combined, compared, or otherwise manipulated by a machine. It is convenient at times, principally for reasons of common usage, to refer to such signals using words such as “data,” “content,” “bits,” “values,” “elements,” “symbols,” “characters,” “terms,” “numbers,” “numerals,” or the like. These words, however, are merely convenient labels and are to be associated with appropriate physical quantities.

[0073] Unless specifically stated otherwise, discussions herein using words such as “processing,” “computing,” “calculating,” “determining,” “presenting,” “displaying,” or the like may refer to actions or processes of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

[0074] As used herein any reference to “one embodiment” or “an embodiment” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

[0075] Some embodiments may be described using the expression “coupled” and “connected” along with their derivatives. It should be understood that these terms are not intended as synonyms for each other. For example, some embodiments may be described using the term “connected” to indicate that two or more elements are in direct physical or electrical contact with each other. In another example, some embodiments may be described using the term “coupled” to indicate that two or more elements are in direct physical or electrical contact. The term “coupled,” however, may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other. The embodiments are not limited in this context.

[0076] As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0077] In addition, use of the “a” or “an” are employed to describe elements and components of the embodiments herein. This is done merely for convenience and to give a general sense of the invention. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise.

[0078] Upon reading this disclosure, those of skill in the art will appreciate still additional alternative structural and functional designs for a system and a process for using machine learning to generate a capability map through the disclosed principles herein. Thus, while particular embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise construction and components disclosed herein. Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims.