CROSS-REFERENCE TO RELATED APPLICATIONS AND PRIORITY

[001] The present application claims priority to India Patent Application No. 202021004663, filed before Indian Patent Office on February 3, 2020. Entire contents of the aforementioned application are incorporated herein by reference.

TECHNICAL FIELD

[002] The disclosure herein generally relates to job processing, and, more particularly, to a method and system for distributed job execution.

BACKGROUND

[003] Every data processing system has specific capabilities. Traditionally when a single system was used to handle simultaneous execution and associated data processing of several jobs requiring same or different capabilities, it is possible that the system gets overloaded and this in turn may affect performance of the system. In distributed data processing/job processing architectures, multiple processing systems/units are used, and overall load as well as the different capabilities required for executing the jobs are distributed among the systems. As execution of each job/task may require the system to have specific capabilities, randomly distributing the jobs among the processing units may not provide intended results.

[004] In order to overcome this, load balancing approach is widely used. Load balancing systems consider specific parameters associated with the processing systems available, and accordingly assigns the jobs. However, a disadvantage of the state of the art systems is that while determining one or more processors for assigning a particular job, they rely on static information pertaining to capabilities and specifications of each of a plurality of processors. SUMMARY

[005] Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, in one embodiment, a processor implemented method for distributed job execution is provided. Initially, an execution specification for at least one job is collected as input, via one or more hardware processors, wherein the execution specification comprises a plurality of steps involved in execution of the at least one job. Further, properties of one or more plugins associated with the at least one job are determined, via the one or more hardware processors. Further, at least one job execution agent for executing the at least one job is determined, via the one or more hardware processors. The step of determining the at least one job execution agent for executing the at least one job involves initially determining whether at least one external tool license is required to perform at least a part of data processing associated with execution of the at least one job. If the at least one external tool license is required, then one or more job execution agents having at least one external tool license to execute the at least one job are identified as one or more candidate job execution agents, from a plurality of job execution agents. If no external tool license is required, then all of the plurality of job execution agents are identified as the candidate job execution agents. Further, a subset of job execution agents from among the candidate job execution agents is shortlisted, wherein a plurality of properties of each of the shortlisted job execution agents match the determined properties of the one or more plugins. Further, real-time value of each of a plurality of performance parameters of each of the job execution agents in the subset of job execution agents is determined. Further, the at least one job execution agent for executing the at least one job is determined from among the job execution agents in the subset of job execution agents, based on value of each of the plurality of performance parameters and number of external tool licenses available with of each of the job execution agents in the subset of job execution agents. Further the at least one job is assigned to the determined at least one job execution agent, which further executes the at least one job. [006] In another aspect, a system for distributed job execution is provided. The system includes one or more hardware processors, one or more communication interfaces, and one or more memory storing a plurality of instructions. The plurality of instructions when executed cause the one or more hardware processors to collect an execution specification for at least one job as input, wherein the execution specification includes a plurality of steps involved in execution of the at least one job. Further, properties of one or more plugins associated with the at least one job are determined, via the one or more hardware processors. Further, at least one job execution agent for executing the at least one job is determined, via the one or more hardware processors. The step of determining the at least one job execution agent for executing the at least one job involves initially determining whether at least one external tool license is required to perform at least a part of data processing associated with execution of the at least one job. If the at least one external tool license is required, then one or more job execution agents having at least one external tool license to execute the at least one job are identified as one or more candidate job execution agents, from a plurality of job execution agents. If no external tool license is required, then all of the plurality of job execution agents are identified as the candidate job execution agents. Further, a subset of job execution agents from among the candidate job execution agents is shortlisted, wherein a plurality of properties of each of the shortlisted job execution agents match the determined properties of the one or more plugins. Further, real-time value of each of a plurality of performance parameters of each of the job execution agents in the subset of job execution agents is determined. Further, the at least one job execution agent for executing the at least one job is determined from among the job execution agents in the subset of job execution agents, based on value of each of the plurality of performance parameters and number of external tool licenses available with of each of the job execution agents in the subset of job execution agents. Further the at least one job is assigned to the determined at least one job execution agent, which further executes the at least one job.

[007] In yet another aspect, a non-transitory computer readable medium for distributed job execution is provided. The non-transitory computer readable medium comprises a program code to execute the following steps using one or more hardware processors to perform the distributed job execution. Initially, an execution specification for at least one job is collected as input, via one or more hardware processors, wherein the execution specification comprises a plurality of steps involved in execution of the at least one job. Further, properties of one or more plugins associated with the at least one job are determined, via the one or more hardware processors. Further, at least one job execution agent for executing the at least one job is determined, via the one or more hardware processors. The step of determining the at least one job execution agent for executing the at least one job involves initially determining whether at least one external tool license is required to perform at least a part of data processing associated with execution of the at least one job. If the at least one external tool license is required, then one or more job execution agents having at least one external tool license to execute the at least one job are identified as one or more candidate job execution agents, from a plurality of job execution agents. If no external tool license is required, then all of the plurality of job execution agents are identified as the candidate job execution agents. Further, a subset of job execution agents from among the candidate job execution agents is shortlisted, wherein a plurality of properties of each of the shortlisted job execution agents match the determined properties of the one or more plugins. Further, real time value of each of a plurality of performance parameters of each of the job execution agents in the subset of job execution agents is determined. Further, the at least one job execution agent for executing the at least one job is determined from among the job execution agents in the subset of job execution agents, based on value of each of the plurality of performance parameters and number of external tool licenses available with of each of the job execution agents in the subset of job execution agents. Further the at least one job is assigned to the determined at least one job execution agent, which further executes the at least one job.

[008] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[009] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:

[010] FIG. 1 illustrates an exemplary system for performing distributed job execution, according to some embodiments of the present disclosure.

[Oil] FIG. 2 is a functional block diagram of a job execution agent, according to some embodiments of the present disclosure.

[012] FIGS. 3A and 3B (collectively referred to as FIG. 3) are flow diagrams illustrating steps involved in the process of performing the job execution, using the system of FIG. 1, in accordance with some embodiments of the present disclosure.

[013] FIG. 4 is an example job specification model used by the system of FIG. 1, according to some embodiments of the present disclosure.

[014] FIGS. 5A through 5C are different implementations of the job specification model, in accordance with some embodiments of the present disclosure.

[015] FIG. 6 is an example tool registry model used by the system of FIG. 1, according to some embodiments of the present disclosure.

DETAIFED DESCRIPTION OF EMBODIMENTS

[016] Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments. It is intended that the following detailed description be considered as exemplary only, with the true scope being indicated by the following claims. [017] Referring now to the drawings, and more particularly to FIG. 1 through FIG. 6, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.

[018] FIG. 1 illustrates an exemplary system for performing distributed job execution, according to some embodiments of the present disclosure. In an embodiment, the system 100 includes a processor (s) 104, communication interface device(s), alternatively referred as input/output (I/O) interface(s) 106, and one or more data storage devices or a memory 102 operatively coupled to the processor (s) 104. In an embodiment, the processor (s) 104, can be one or more hardware processors (104). In an embodiment, the one or more hardware processors (104) can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) 104 is configured to fetch and execute computer-readable instructions stored in the memory 102. In an embodiment, the system 100 can be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices, workstations, mainframe computers, servers, a network cloud and the like.

[019] The I/O interface(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a Graphical User Interface (GUI), and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface (s) 106 can include one or more ports for connecting a number of devices to one another or to another server. For example, the I/O interface 106 enables the authorized user to access the system disclosed herein through the GUI and communicate with other similar systems 100.

[020] The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. Thus, the memory 102 may comprise information pertaining to input(s)/output(s) of each step performed by the processor(s) 104 of the system 100 and methods of the present disclosure.

[021] FIG. 2 is a functional block diagram of an execution agent, according to some embodiments of the present disclosure. In an embodiment, the execution agent is an example implementation of the system of FIG. 1. The execution agent 200 includes an execution controller 201, a tool registry manager 202, an agent communicator 203, a user access component 204, a job controller 205, a plugin interface 206, and one or more external tool(s) 207. Terms ‘execution agent’ and ‘job execution agent’ and ‘ToolServer’ are used interchangeably.

[022] The agent communicator 203 facilitates communication with one or more other execution agents 200 i.e. execution agents , for communication and data transfer related to at least the following three steps in the distributed job execution process: (1) communication and data transfer related to an ongoing execution, (2) data exchange between transfer of tool registry of different job execution agents 200, and (3) communication related to availability of a specific tool. The tool registry manager 202 is configured to store and maintain the tool registry at the job execution agent 200, wherein maintaining the tool registry involves updating the tool registry using data collected from other job execution agents .200. The execution controller 201 is configured to execute each step in the execution specification as depicted in FIGS. 3 A and 3B. If the job execution involves one or more external tools available with a job execution agent 200, the execution controller 201 delegates the job execution to the job controller 205 locally available and waits for completion. Once the job execution is complete, the job controller 205 then transfers the execution control and relevant data to another execution agent 200 for executing next job in queue as per the execution specification. The job controller 205 can be configured to support different types of jobs. The job controller 205 may also provide a provision for a user to add one or more custom jobs. A few examples of jobs being referred to here are “data transformation”, “tool invocation”, “nested job execution”, and so on. The job controller 205 manages all these jobs on a specific agent. For example, the job controller 205 manages tool handlers available locally, for the purpose of execution of the one or more jobs assigned. Once an execution request is received, the job controller 205 invokes specific tool handlers to execute the job if the job is a tool invocation job. The job controller 205 manages a job queue for every tool handler registered with it. By inspecting this queue, load on a specific job execution agent 200 for a specific tool can be estimated. The job controller 205 can be further configured to query one or more other job execution agents 200 about the availability and load of a specific tool.

The user access component 204 is configured to provide one or more appropriate interfaces for an authorized user to interact with the job execution agent 200 when required. The user access component 204 takes a new execution specification from the user and sends it to the execution controller 201 for execution. Once the job execution is over, the user access component 204 returns the result back to the user. The plugin interface 206 interfaces one or more plugins responsible for executing one specific job type such as invocation of a specific external tool or a specific data transformation, with the job execution agent 200. The term ‘plugins’ in this context refer to software add-ons which help in connecting required external tools as well as any internal tool required for executing one or more jobs, with the system 100. New plugins may be created to support a new job types such as a new external tool invocation, and the newly created plugins can be connected to the job execution agent 200 via the plugin interface 206. Such a plugin-based design of the job specification is depicted in FIG. 4. A few examples of the ‘external tools’ that may be used by the job execution agents are Abaqus, and Ansys. As depicted in FIG. 4, the job specification includes a job specification entity, a job entity, a plugin entity, a property entity, an external tool interface, a tool server entity. The job specification entity includes a plurality of jobs, information about an execution sequence in which the plurality of jobs are to be executed, and input/output files for the whole specification (i.e. for the whole set of jobs). The job entity includes information such as plugins and input/output for individual jobs among the plurality of jobs. The data entity includes one or more data which is one of an input or output of at least one job among the plurality of jobs. The plugin entity includes properties of the at least one plugin and data on how the plugin is to be implemented. The property entity includes at least one property of one or more plugin in the form of a name-value pair representing at least one environmental property. The external tool interface provides at least one interface to connect the at least one external tool with the system 100. The tool server entity includes information on at least one attribute of a system in which the at least one execution agent is installed.

[023] FIGS. 3 A and 3B (collectively referred to as FIG. 3) are flow diagrams illustrating steps involved in the process of performing the job execution, using the system of FIG. 1, in accordance with some embodiments of the present disclosure. The system 100 initially collects (302) an execution specification as input. The execution specification comprises one or more jobs to be executed, and steps for execution of each of the jobs. The execution specification may further indicate whether one or more external tools are required for the execution of the one or more jobs. In an embodiment, all jobs in the execution specification are part of same task. For example, multiple jobs may have to be executed for completion of a task. If there is any specific order in which the jobs are to be executed, the same is indicated in the execution specification, and the system 100 may execute the jobs in the specified order. An example of the execution specification (alternately referred to as “job specification model”) is given in FIG. 4, and simplified views of the execution specification are given in FIGS. 5A through 5C.

[024] As shown in FIG. 5A, the execution specification is composed of three Jobs, Jobl, Job2 & Job3. Every specification also takes in few input files and produces some output files, and the input files and the output files may contain mapping to an ontological context. Here the input files are data file A, B & C and the output files are D and E. Every Specification also has an association to an execution agent 200/ ToolServer (Here ABC), which represents the controller for that Job-spec. The execution specification in FIG. 5B shows JOB A having input and output files specified. It is also associated to a plugin (plugin XYZ), which knows how to execute JOB A, for every job a plugin association is necessary. Every Job is also associated to its previous and next jobs in the job spec, here the previous job is JOB Z and next job is JOB B. Every Job also has an association to a tool- server which indicates where (which tool-server) the job got executed. Information pertaining to the association of tool-server for each job gets populated with execution of each job. As can be seen in FIG. 5C, a plugin (XYZ), is associated with a Data-File A, which contains the implementation details of the plugin. It has few properties (A, B & C here) as shown. The plugin is associated to a Job Specification D if the plugin is of type Nested Job Plugin. The plugin is associated to an external Tool E if the plugin is of type Tool Plugin.

[025] Each job may have one or more associated plugins. In an embodiment, as the execution specification contains a list of a plurality of jobs to be executed for completion of a task, the system processes one job at a time, and upon completion of execution of each job, next job is picked and executed. At step 304, the system 100 determines properties of all the plugins associated with the job being processed at the current instance of time. In an embodiment, only one plugin is associated with a job. In an alternate embodiment, more than one plugin are associated with the job. For example, when 2 different plugins are associated with a particular job, one of the two plugins may specify to the system 100 that the job can be executed on a Windows system, whereas the other plugin may specify to the system 100 that the job can be executed on a Linux system as well. The determined properties of the one or more associated plugins indicates to the system 100 requirements (in terms of capabilities an execution agent 200 need to have) to execute the job. For example, if an Abaqus tool is required for execution of the job, and the same is specified in the execution specification, then the system 100 understands that an Abaqus plugin is required to interface an Abaqus tool with the system 100 (if not internally available).

[026] Execution of certain jobs may require one or more external tools. For a job execution agent 200 to use an external tool, at least one license (referred to as “external tool license”) of the external tool is required. The system 100 determines (306), based on the execution specification, whether one or more of the external tool licenses are required for execution of the at least one job. In an embodiment, the system 100 determines whether the one or more external tool licenses are required or not, based on data in the collected execution specification. If no external tool license is required, then all the execution agents 200 in the subset of execution agents are identified (310) as candidate job execution agents. If at least one external tool license is required, then the system 100 identifies (308) all the execution agents 200 having the at least one external tool license as the candidate job execution agents, from among the plurality of job execution agents available.

[027] The system 100 then identifies (312) a subset of job execution agents 200 from the candidate job execution agents, based on match between of properties of each of the job execution agents 200 and properties of the one or more plugins. In the aforementioned example of one job having two different plugins, the system 100 identifies the subset of job execution agents 200 by considering all the job execution agents 200 that support Windows and/or Linux platforms, and the job execution agents 200 which do not support neither of the Windows/Linux platforms are discarded. In this example, all the job execution agents 200 which support the Windows and/or Linux platforms are together identified as the subset of job execution agents. In a scenario in which only one plugin is associated with the job to be executed, identifying the subset of job execution agents involves identifying all the job execution agents having properties matching the only one plugin associated with the job.

[028] Further, the system 100 obtains/collects (314) real-time value of each of a plurality of performance parameters of each of the available job execution agents. Value of the ‘performance parameters’ at any given point of time represents real-time capability of the job execution agent 200. A few examples of the ‘performance parameters’ are, but not limited to ‘CPU utilization’, available memory space, and so on. Each job execution agent 200 may be configured to share such information in response to certain defined triggers. For example, the trigger is a periodic trigger or an event trigger. If the periodic trigger is set by defining a time interval, the one or more execution agents share ‘current value’ of each of the parameters, when the set time interval is expired. The event trigger for example may prompt the one or more execution agents 200 to share the values when one or more of a pre-defined set of event triggers are detected. An example of such an event trigger is completion of an ongoing job execution i.e. as soon as the execution agent completes the execution of a job, values of the one or more performance parameters are broadcasted, and the values are updated in one or more databases in the tool registry stored in the memory 102. In an embodiment, the parameters associated with each of the execution agents 200 and corresponding values are updated in the tool registry (depicted in FIG. 6), which is stored in the memory 102 as time series data, i.e. the values are dynamically updated from time to time or when specific triggers are triggered. As the values of the parameters in the tool registry are updated based on the one or more triggers, the values of the parameters of an execution agent 200 at any instance indicates/represents real-time capability of the execution agent 200.

[029] From among the job execution agents in the subset of job execution agents, the system 100 further determines (316) at least one job execution agent 200 as a job execution agent for executing the at least one job, and then the at least one job is assigned (318) to the determined at least one job execution agent 200. In an embodiment, the system 100 determines the at least one job execution agent 200 as the job execution agent for executing the at least one job, based on values of the one or more performance parameters and a number of the external tool licenses available with the job execution agents 200 in the subset of job execution agents. For example, consider two job execution agents A and B. Both A and B have one external tool license each (of the required external tool). CPU utilization of A is 50% whereas that of B is 30%, and CPU utilization is not permitted to be exceeding 60% for execution of a job, and number of external tool license is required is at least one. Here, both A and B satisfy criteria in terms of number of external tool licenses required and the CPU utilization. However, as the CPU utilization of B (30%) is less than that of A (50%), B is less loaded, and is expected to perform faster (given hardware configurations of A and B are same). As a result, in this scenario, the job execution agent B is determined as the one matching the determined properties. In another embodiment, a user may define weightage of each of the performance parameters, and at least when all the candidate job execution agents satisfy the requirements, then the weightage is used for determining the job execution agent matching the determined properties of the plugin(s). Once the job is assigned, then the job execution agent(s) 200 executes (320) the assigned job(s).

[030] The system 100 may be using a proprietary data transfer protocol for data communication to and from an execution agent 200. As per the protocol used, the data is divided into two parts, the header and the payload. The header has a max length of 512 bytes and has the following format, tokenld: command: identifier. Here the token ID is a unique ID for every specification and is mandatory for every communication. Command is a list of allowed operations on any execution agent 200, and the following are valid commands: PUSH (push files to an execution agent), PULL (get files from and execution agent), START (Start execution of a given spec). The identifier has following structure JobSpecId-input/output file name, and is ignored for START command. The payload can be of any length for PUSH command and it is empty for PULL and START command.

[031] In an embodiment, one or more steps in method 300 may be omitted. In another embodiment, the steps in method 300 may be executed in the same order depicted in FIG. 3 or in at least one other alternate order which is technically feasible.

Example:

[032] In this example, an execution agent 200 receives the job specification with id jbspec_l, and also receives all input data associated with the specifications. In an example mode of implementation, the execution agent 200 that received the job specification (termed as ‘root specification’ \) and the input data acts as a controller for the job specification. In this example, the execution agent 200 that is the controller for the job specification is termed as primaryServer. By processing the job specification, the primaryServer decides that the first job to be executed is job_l. The primaryServer inspects the plugin associated with the job job_l and finds that a 64-bit Linux system is suitable for the plugin execution and in turn the job execution. The job controller 205 of the primaryServer also finds that the current execution agent 200 is a Linux 64 bit system and also meets all the criteria for the execution of the job (in terms of performance parameters such as CPU Load, RAM and so on) and hence starts the job execution and updates the job specification post successful execution of job_l. The job controller 205 also collects all input data for the job, either from the job controller 205 of a parent job specification or from any other job execution agent(s) 205 where any previous job had executed.

[033] Once job, job_l gets executed, the execution controller 205 finds out job_2 as next job to be executed. It again inspects the plugin for job_2 and finds out that it needs a tool, “Abaqus-4.2”, which is not available on the current execution agent 200 (tool server). Therefore, the execution controller refers the tool registry and finds out a suitable execution agent 200 for execution for job_2, which in this example is “secondaryServer”.

[034] Once job_2 gets executed the execution controller 205 on the secondaryServer inspects plugin of the next job, job_3 and finds out that the plugin is a NestedJobPlugin, and also finds out that the current job execution agent 200 is capable of executing it and hence becomes the controller for the jobSpec associated with it. Specifications inside the NestedJobPlugin need not have any input or output Data, they can be obtained as input from the parent job and can send outputs to the parent job. This job specification gets handled in a way similar to the root job specification and each of the jobs get executed. Once all sub jobs complete execution, the parent job i.e. job_3 of the nested plugin collects required outputs from execution agents where certain sub-jobs got executed. Once all the jobs in the root specification are executed, the controller of the root job specification i.e. the primaryServer collects all outputs that need to be sent back to caller of the job and thereby transfers all the required output data, thereby marking end of execution. These steps are explained with the help of a sample use-case, below:

[035] The Problem: Consider that there are only two Systems i.e. System A and System B in a network where jobs can be executed. Now, there are two jobs that are to be executed in sequence, Job 1 and Job2. Consider that Job 1 is the job in which a program is to be written, which pre-processes given inputs in a format that would be acceptable by a tool like Abaqus, and the second job is actually calling the tool Abaqus, carrying out some simulation, and collecting the outputs. Also consider that the systems System A and B are both Linux based systems and that only the system B has Abaqus. Various steps are:

1. MODEL POPULATION:

• SUB -STEP 1- Plugin Creation: Every job is associated with at least one plugin, which defines how the Job needs to be executed. For the two jobs here, two plug-ins are created. Job 1 as mentioned is a preprocessing activity, and hence a self-executable plugin, plugin 1 is needed for this kind of Job. A script/program for the preprocessing that is needed is the data-file that is associated with every plugin. Similarly, a plugin plugin2 is created for the second Job. The second job is required to invoke Abaqus and hence a tool plugin is created and is associated with its external tool, which is Abaqus. While defining the plugins, properties of the plugins also are specified, which for example may specify that both plugins can only run on systems where the operating system is Linux.

• SUB-STEP 2- Job Creation: In this sub-step, a JOB model Jobl is created. The JOB model also specifies inputs for Jobl, which for example, are 3 files, param.txt, Image.jpg and sv.csv. Outputs that are expected from this job also are specified, which are outl.txt and trf.txt. Then job2 is created and corresponding inputs are specified as trf.txt and output as final_out.txt. While specifying the inputs and outputs, source of these files also are specified. The source for param.txt, image, .jpg, sv.csv, outl.txt, trf.txt for job 1 is root (root means no transfer needed) in this example. The source for trf.txt in job2 is jobl, because it expects the job to output of previous job as input, the source for final_out.txt is root. After specifying the inputs, the respective plugins that are created in SUB-STEP- 1 are interfaced with respective jobs, which is pluginl for Jobl and plugin2 for job2.

• SUB-STEP 3- Job SPEC Creation: At this step, the Job specification ‘Job spec V is created, and this is associated with the job specification created in SUB-STEP 2. All the input files across all the jobs that have root as source automatically become the input for job spec 1 and all outputs that have root as source become outputs for the spec. This forms SPEC for further processing.

2. INPUT DATA TRANSFER AND EXECUTION INITIATION

• SUB -STEP 1- Once the SPEC is ready, the XML version of the SPEC is created and pushed to any execution agent (say System A) 200, using a PUSH command.

• SUB -STEP 2- Apart from the SPEC, all input files are also pushed to the same execution agent 200, using the same PUSH command as in SUB STEP 1.

• SUB-STEP 3- After sending all required inputs, a START command is passed to the same execution agent 200, to start the execution on the execution agent.

3. EXECUTION

[036] The execution agent 200 receives the SPEC and all input files. The execution agent 200 (Execution controller in execution agent) finds out that there are two jobs (Job 1 and Job 2) to be executed. The execution agent 200 then selects Jobl and finds out the plugin associated with it, which is plugin 1, it also checks the properties associated with plugin, which in this case is that OS needs to be Linux. The execution controller 205 of the execution agent 200 checks with Tool registry and finds out that both System A and System B are Linux systems, hence both are suitable for execution of Jobl. So, both system A and system B are candidate systems, however the execution controller 205 further checks and finds out that based on CPU utilization and available memory (i.e. performance parameters), System A is more suitable and hence it delegates the Job to a local job controller. The Job is executed on system A and the outputs are generated.

[037] Once Jobl is complete, the execution controller 205 on system A checks the plugin for Job2 and finds out that the job has a tool plugin and that the tool associated with it is Abaqus. The execution controller 205 refers to the tool registry to find out the systems where Abaqus tool is available and finds out that system B has Abaqus and licenses are available to execute jobs. The execution controller 205 then transfers the updated spec along with all the required inputs to the execution agent on system B and requests it to execute Job2. System B executes job2 and sends the produced outputs back to system A along with updated SPEC (controller of the job spec), once the execution of job 2 is complete, the execution agent on system2 passes the control back to system 1. The system A checks the SPEC, finds out that all Jobs in the SPEC are now executed, and sends an execution completion message to the sender of the specification.

4. OUPUT DATA COLLECTION

[038] The sender of the SPEC upon receiving execution completion message, starts collecting the required output files using PULL commands.

[039] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.

[040] An example of Multi-plugin scenario (a job having multiple plugins) is given below:

[041] Consider a Job Specification comprising of Just one job, say Job A. Let the job A be an external tool invocation job, which has two plugins (two implementations), say Plugin X and Plugin Y. Consider that the external tool associated with the Job is Abaqus. Plugin X has a property with name value pair as “OSName” and “Linux”. Plugin Y also has the same named property (i.e. OSName), however with value as “WindowslO”. Consider that 5 Execution Agents (EA1 to EA5) are available and only 3 of them have Abaqus Licenses, say EA1, EA2 and EA3. In this scenario, the system 100 picks EA1, EA2, and EA3 as the candidate execution agents. EA1 is installed on a Linux based host, EA2 on a windows 10 host, whereas EA3 is on a MaC host. Now the system 100 checks if EA1 matches with all the properties of any of the plugins. Here there is a match between the properties of EA1 and Plugin X. Hence EA1 is considered for further processing. Same process is repeated for EA2 and EA3 and EA3 is omitted as properties of EA3 do not match the properties of any of the plugins. EA1 and EA2 are identified as the subset of job execution agents. Further the system 100 checks value of one or more of the performance parameters of each of the job execution agents in the subset of job execution agents. In this example, the system 100 determines EA2 as the job execution agent for executing the job, as EA2 is less loaded in comparison with load of EA1.

[042] The embodiments of present disclosure herein addresses unresolved problem of selection of execution agent for executing a job, in a distributed job execution scenario. The embodiment, thus provides a method and system for selection of one or more job execution agents for execution of a job, based on match with plugin properties, real-time values of certain performance parameters, and number of external tool licenses available when needed.

[043] It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs.

[044] The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various components described herein may be implemented in other components or combinations of other components. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[045] The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.

[046] Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer- readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer- readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.

[047] It is intended that the disclosure and examples be considered as exemplary only, with a true scope of disclosed embodiments being indicated by the following claims.