A GRAPHIC USER INTERFACE BASED NETWORK MANAGEMENT SYSTEM TO DEFINE AND EXECUTE TROUBLESHOOTING PROCEDURE

CROSS-REFERENCE

[0010] Priority is claimed from the US Nonprovisional Application No. 13/841735 filed on March 15, 2013; the US Nonprovisional Application No. 13/842024 filed on March 15, 2013; he US Nonprovisional Application No. 13/842222 filed on March 15, 2013; the content of which are hereby incorporated by reference.

BACKGROUND

[0011] The present application relates to network management, and more particularly to graphic user interface based automated procedures in network management.

[0012] Note that the points discussed below may reflect the hindsight gained from the disclosed inventions, and are not necessarily admitted to be prior art.

[0013] No doubt we are living in a time that almost every one of us and every single entity is connected by devices and computers via the Internet, proprietary intra- electronic networks through cable or wireless. Data and communications are being inter- exchanged constantly through the vast and complex network connections. A single interruption in network communication could mean hundreds of thousands of dollars in losses and damages. According to some current conservative estimates, network outages could cost $1,400 per minute on average. Reducing the down time is critical to the success of business.

[0014] Like the transportation highways in the real world, the communication highways in the virtual world are becoming ever more tangled and more complicated each single minute. Management of these networks is becoming more challenging at the most basic levels. Identifying a problematic device from the vast sea of network devices is literally like finding a needle in a hay stack.

[0015] The conventional way for network troubleshooting requires a network professional to manually run a set of standard commands and processes for each of the devices. However, to become familiar with those commands, along with each of its parameters takes years of practice. Also complicated troubleshooting methodology is often hard to share and transfer. Therefore even though a similar network problem happens again and again, each instance of troubleshooting may still have to start from scratch. However, networks are getting more and more complex and it is increasingly difficult to manage it efficiently with traditional methods and tools. The following are the key challenges using conventional ways to troubleshoot network problems:

[0016] Firstly, with text-based Command-Line Interface (CLI) as the primary method for troubleshooting a network problem, a network professional usually needs to repetitively execute the same CLI commands and decode key data from the command output many times for many network devices. This process is error-prone, strenuous and time consuming.

[0017] Secondly, currently there is no efficient mechanism or method to record a troubleshooting process for future reference. Consequently network professionals cannot share their troubleshooting knowledge with other network professionals. Within the same enterprise the same network professional may need to spend the same amount of time and effort to troubleshoot the same problem which had occurred before.

[0018] A generic network troubleshooting process consists of the following tasks:

• Define the problem

• Gather the data

• Analyze the data

• Eliminate the possible problem causes

• Find the root cause of the problem

[0019] Many books and papers have been written to analyze the typical actions and decisions that are taken during each of these processes and how these could be planned and implemented via the standard procedures. However these procedures are static, and the process to gather and analyze data (usually via CLI commands) is still a very manual and meticulous process.

[0020] The invention of a computer-aided network engineering system, NETBRAIN® Workstation (as described in US Patent No. 8386593 by the inventors of this application) provides a graphic user interface (GUI) that renders network troubleshooting automation possible. In a GUI -based system, a network structure may be represented with graphic features (icons, lines and menus) that represent corresponding features in a physical network. Such visual representation liberates a network engineer from memorizing the standard or proprietary protocols and the tedious manual tasks of typing.

[0021] The inventions provide GUIs for users to write Executable Procedures without having any programming background. After a Procedure is created, it can be run in NETBPvAIN™ Workstation in connection with any network system. From start to finish, troubleshooting with a proposed solution may just take a few minutes instead of hours or days traditionally.

[0022] To troubleshoot a network problem or to simply verify if a network functions, a network engineer still needs to manually log in to each of the network devices and issue a CLI command to gather the data, manually parse and analyze each of the output for key data, and manually eliminate each of the possible problem causes.

[0023] For each of the enterprises across the vast network world, this process may be repeated again and again, without any benefit from learning past lessons or from other people's experiences. This process is a waste of time and energy for an engineer, and certainly a waste of a company's sparse resources.

[0024] To further complicate this already tangled process, many vendors and models of network hardware devices that exist in today's network, are providing different sets of CLI commands which output many different formats of data information. It is difficult, if not impossible, for a network engineer to simplify this process by writing a simple executable program to retrieve, parse and analyze the output data of each of these different devices. It is even more challenging to require a network engineer to master a programming language in a short time, and apply such skills in a reliable manner.

[0025] There is an urgent need to find a universal solution to be able to parse and analyze data outputs of different devices from various vendors. There is an urgent need for a network management system that guides a network engineer through visual user interfaces and samples, and enables him to customize their data analysis without having to master a programming language.

[0026] The network system gets more and more complex. Today's businesses rely heavily on a stable network system. Initiating, planning, constructing and conducting a network change can incur serious consequences. Unintended interruption of a network system may collapse an entire working environment, causing hundreds of thousands of dollars in losses and delays. However, vast network systems have been in constant dynamics of change, re-configuration and upgrading, in numerous nonstop endeavors to provide new services and satisfactions to the ever-changing waves of customer needs.

[0027] Generally, a task of conducting a network change often includes four major steps: plan, design, implementation and verification. So far, these steps have been mainly performed manually. There is no efficient way to assure that a network change does not cause network problems. In fact, the majority of network problems are caused by network changes.

[0028] There is an urgent need for a system and method that conducts reliable and efficient network changes, for network management.

SUMMARY

[0029] The present application discloses new approaches to troubleshooting a network problem. A system is invented to define a Procedure which can be automatically executed. This type of Procedures is called an Executable Procedure. An Executable Procedure utilizes a visual programming method to enable a CLI-based troubleshooting processes executable and re-useable. It emulates the thinking process of human troubleshooters when they use CLI commands. A network professional without any programming background can also effectively program his know-how and the end result of this programming can be applied to any other type of network by anyone to troubleshoot a similar type of network problems.

[0030] In one embodiment, GUIs are provided to define an Executable Procedure. The definitions of an Executable Procedure are divided into a set of visual blocks and each block can be defined with a visual interface.

[0031] In one embodiment, by using a GUI, a user defines how to collect data from network devices, how to parse the key information from the data, and the methods to analyze the data and messages to be output when a certain condition occurs. After a Procedure is defined, the system automatically creates an executable application. [0032] In one embodiment, the executable application is enabled to run from within a network map, on one or multiple network devices or through any other input from a user. A Procedure can be re-used to troubleshoot another network problem, create a map, verify the network health and create a report.

[0033] In one embodiment, functions that group together a set of processes for gathering data from execution results of network devices and connections are made accessible through a set of corresponding GUIs represented as a Parser.

[0034] In one embodiment, functions that group together a set of processes for analyzing data collected from network devices and connections are made accessible through a set of corresponding GUIs represented as a Trigger.

[0035] In one embodiment, a set of GUIs are provided to visually display an execution of a set of processes and commands in real time.

[0036] In one embodiment, a set of GUIs are provided to visually display identified possible errors and warning messages.

[0037] In one embodiment, a set of GUIs are provided to visually display a possible solution to a network problem.

[0038] In one embodiment, a set of troubleshooting processes and strategies are saved as a Procedure and are made accessible through a set of user interfaces.

[0039] The disclosed innovations, in various embodiments, provide one or more of at least the following advantages. However, not all of these advantages result from every one of the innovations disclosed, and this list of advantages does not limit the various claimed features.

[0040] The advantages of a system with a GUI for providing user control and access are obvious— dramatically shortening the learning curves and maximizing efficiency, and therefore enabling a junior network professional to consistently perform complicated network management tasks.

[0041] Further any time saved in troubleshooting may mean real money for an enterprise that relies on network stability and network performance. With a visual system running in real time, any network trouble may be identified instantly and therefore be fixed in a shorter period of time. [0042] A well-built Procedure can automatically gather data, analyze data and eliminate possible causes. Besides troubleshooting the network problems, the Executable Procedure can also be used to:

• Create a map, for example, mapping an application's path Procedure.

• Provide network compliance or health checks.

• Create a customized report.

[0043] The present application discloses a new approach and system for network output data retrieval and data analyses. The system uses a sample-driven model which enables a user to visually define a Parser utility with any sample output for any device from any vendor. The data control and flow are visible from the user interface and most of the logic loop controls are automatically implemented. Such Parser Utility in combination with a Trigger Utility to take the result of the Parser Utility as input Variables provides a uniform and ubiquitous way of parsing and analyzing various data output in any format.

[0044] In one embodiment, a Graphic User Interface (GUI) based network system is provided, where a sample output of a device is collected and saved. A sample data format from a device may be collected in real time and be used for defining a Parser and subsequent automation of a process.

[0045] In one embodiment, a collected sample data output is used for defining a Keyword Parser, wherein the selected Keyword from the sample data output is used as a marker in collecting data adjacent to such marker into variables to be used for data analysis in subsequent Trigger functions.

[0046] In one embodiment, a collected sample data output is used for defining a Paragraph Parser, wherein the selected text pattern is used as a marker for identifying the beginning and the end of a paragraph of a similar data output, and each of the paragraphs can be further parsed with Child Parsers to collect data into a Table of Variables (two dimensional array), to be used in subsequent Trigger functions.

[0047] In one embodiment, a collected sample data output is used for defining a Table Parser which uses the text of the headers of the table in the sample output as marker to identify values in similar tables and to collect such data into a Table of Variables, to be used in subsequent Trigger functions.

[0048] In one embodiment, a collected sample data output is used for defining a Filter Parser, wherein the text pattern is used for identifying the beginning and end of the partial data, which is saved as a Filter Variable for subsequent Trigger functions.

[0049] In one embodiment, the parsed variables are visually available in defining Trigger Utilities, a user need only specify simple functions to provide data analyses for these data values and generate data reports.

[0050] In one embodiment, visual hints and suggestions are provided in the Trigger Utilities where a network engineer has the convenience of simply deciding which variable to analyze and what to see, without worrying about any underlying logic loop controls.

[0051] The inventions provide GUI to write executable applications so that a user without any programming background can write a Procedure. After a network application is created, it can be run in NETBRAINTM Workstation in connection with any network system. From start to finish, troubleshooting with the proposed solution may just take a few minutes instead of hours or days traditionally.

[0052] The present application discloses new approach and system for conducting a network change. A system is invented to manage a network change task from beginning to end. The system first designs a work flow to greatly reduce possible problems that may be caused by a network change. By providing visual user interfaces and methods, the system streamlines a network change into various stages with safeguards of benchmarked data verification.

[0053] In one embodiment, the status of a network before a proposed change is benchmarked. After executing a planned change, the status of the network after the change is benchmarked. Then the status of the network before and after the change is compared in detail and documentation on the change is created. With the safeguards of data verification, any hazardous change can be instantly rolled back without causing any serious harm. [0054] In one aspect of an embodiment, a time line of events is provided, and all actions or subtasks are recorded. A network change task can also be saved as a standalone file and exported to a Word document for review.

[0055] The inventions provide GUIs to guide a user through and perform a reliable network change. Using NETBRAIN™ Workstation in connection with any network system, a user can conduct the entire process from initiating to finalizing a network change in an orderly manner.

BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The disclosed inventions will be described with reference to the accompanying drawings, which show important sample embodiments of the invention and which are incorporated in the specification hereof by reference, wherein:

[0057] FIG. 1 diagramly shows an example functional interaction flow of an Executable Procedure of a GUI based network management system in accordance with this application.

[0058] FIG. 2 diagramly shows an example execution flow of an Executable Procedure of a GUI based network management system in accordance with this application.

[0059] FIG. 3 diagramly illustrates an example building process for construction of an Executable Procedure in a GUI based network management system in accordance with this application.

[0060] FIG. 4 shows an example GUI of a Procedure center for managing various Procedures in a GUI based network management system in accordance with this application.

[0061] FIG. 5 shows an example method to run a Procedure within a network device map in a GUI based network management system in accordance with this application.

[0062] FIG. 6 shows an example user interface for selecting Procedures to run in a GUI based network management system in accordance with this application. [0063] FIG. 7 shows an example GUI for displaying execution results of a Procedure in a GUI based network management system in accordance with this application.

[0064] FIG. 8 shows an example Procedure that has three Process Nodes in a GUI based network management system in accordance with this application

[0065] FIG. 9 shows an example GUI for defining a Process Node in a GUI based network management system in accordance with this application.

[0066] FIG. 10 shows an example GUI for defining a CLI command Probe in a GUI based network management system in accordance with this application.

[0067] FIG. 11 shows example GUI for defining a Trigger in a GUI based network management system in accordance with this application.

[0068] FIG. 12 shows an example GUI for defining a set of running settings to execute a Procedure in a GUI based network management system in accordance with this application.

[0069] FIG. 13 shows an example GUI for defining a set of input variables to execute a Procedure in a GUI based network management system in accordance with this application.

[0070] FIG. 14 shows an example GUI for defining a set of output variables to execute a Procedure in a GUI based network management system in accordance with this application.

[0071] FIG. 15 shows an example GUI for defining a Ping Probe in a GUI based network management system in accordance with this application.

[0072] FIG. 16 shows an example GUI for defining a Traceroute Probe in a GUI based network management system in accordance with this application.

[0073] FIG. 17 shows an example GUI for configuration of a Probe Parser in a GUI based network management system in accordance with this application.

[0074] FIG. 18 shows an example a network map created by using a Procedure in a GUI based network management system in accordance with this application.

[0075] FIG. 19 illustrates an example set of relevant components of a network management system on which Executable Procedures are defined and run, in accordance to this application. [0076] FIG. 20 shows an example control and data flows of an Executable Procedure in accordance to this application.

[0077] FIG. 21 shows an example User Interface for defining a Keyword Parser in accordance to this application.

[0078] FIG. 22 is an example sample output containing paragraphs in accordance to this application.

[0079] FIG. 23 shows an example User Interface for defining a Paragraph Parser in accordance to this application.

[0080] FIG. 24 shows a sample output with a table format in accordance with this application.

[0081] FIG. 25 shows an example User Interface for defining a Table Parser in accordance to this application.

[0082] FIG. 26 shows an example User Interface for advanced setting in defining a Table Parser in accordance to this application.

[0083] FIG. 27 shows an example User Interface for defining a Filter Parser in accordance to this application.

[0084] FIG. 28 shows an example User Interface for defining a Threshold Trigger in accordance to this application.

[0085] FIG. 29 shows an example User Interface for the action blocks of an Advanced Trigger in accordance with this application.

[0086] FIG. 30 shows an example logic loop that a Procedure automatically executes in accordance with this application.

[0087] FIG. 31 shows an example output of a Procedure in debugging mode in accordance with this application.

[0088] FIG. 32 illustrates an example management system for conducting network changes in accordance with this application.

[0089] FIG. 33 shows an example work flow for conducting a network change in accordance with this application.

[0090] FIG. 34 shows an example user interface for implementing and conducting a network change in accordance with this application. [0091] FIG. 35 shows an example of timeline interface in accordance with this application.

[0092] FIG. 36 shows an example method and user interface for defining a configuration change template in accordance with this application.

[0093] FIG. 37 shows an example method and user interface for defining a configuration change for a network device in accordance with this application.

[0094] FIG. 38 shows an example method and user interface for benchmarking subtasks in accordance with this application.

[0095] FIG. 39 shows an example user interface for adding benchmarked commands via a template in accordance with this application.

[0096] FIG. 40 shows an example method and user interface for executing a configuration change in accordance with this application.

[0097] FIG. 41 shows an example user interface for comparing network status before and after a change in accordance with this application.

[0098] FIG. 42 shows the contents to be exported in a Word document

DETAILED DESCRIPTION OF SAMPLE EMBODIMENTS

[0099] The numerous innovative teachings of the present application will be described with particular reference to presently preferred embodiments (by way of example, and not of limitation). The present application describes several inventions, and none of the statements below should be taken as limiting the claims generally.

[00100] For simplicity and clarity of illustration, the drawing figures illustrate the general manner of construction, and description and details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the invention. Additionally, elements in the drawing figures are not necessarily drawn to scale, some areas or elements may be expanded to help improve understanding of embodiments of the invention.

[00101] The terms "first," "second," "third," "fourth," and the like in the description and the claims, if any, may be used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable. Furthermore, the terms "comprise," "include," "have," and any variations thereof, are intended to cover nonexclusive inclusions, such that a process, method, article, apparatus, or composition that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, apparatus, or composition.

[00102] The present application may be described herein in terms of functional block components and various processing steps. It should be appreciated that such functional blocks may be realized by any number of hardware and/or software components configured to perform the specified functions. For example, the present invention may employ various integrated circuit components, e.g., memory elements, processing elements, logic elements, look-up tables, and the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices.

[00103] Similarly, the software elements of the present invention may be implemented with any programming or scripting languages such as C, C++, Java, COBOL, assembler, PERL, Python, or the like, with the various algorithms being implemented with any combination of data structures, objects, processes, routines, or other programming elements. Further, it should be noted that the present invention may employ any number of conventional techniques for data transmission, signaling, data processing, network control, and the like.

[00104] A particularly powerful tool for understanding network behavior is graphic visualization. A computer-aided network engineering system, NETBRAIN™ Workstation enables automating network troubleshooting possible. A network professional can follow three steps to troubleshoot a problem: map the problem area; probe from the map and compare the current network state with baseline data. With this invention of Executable Procedure, one can select and execute Procedures relevant to the network problem from within a network map. The output of the Procedures may help identify the root cause of the problem quickly.

[00105] Background technologies and terminologies are further described in US Patent No. 8386593, the content of which is incorporated by reference. [00106] For network troubleshooting, a network engineer depends on a set of commonly used commands, methods and tools, standard or proprietary of the manufacturers:

[00107] The Command Line Interface (CLI): almost all network devices provide CLI commands to check the network status or statistics. For example, in a Cisco IOS switch, the command "show interface" can be used to show the interface status such as input errors.

[00108] Ping: a simple tool used to check whether a device can be reachable from another device. For example, after a network change, it is the best practice to ping the main servers from the core network devices to prevent any major outage of the key applications.

[00109] Traceroute: a tool to check the route from a device to a destination device. This tool is useful to troubleshoot a connectivity problem.

[00110] Configuration management: a system used to find differences of configurations of network devices in a certain period. This is important since about half of the network problems are caused by configuration changes.

[00111] Troubleshooting procedures, usually provided by the hardware vendor or the expert in the field, generally comprises the following sequences of actions:

• Execute the CLI, ping, traceroute or other commands from the network devices;

• Find the key value from the command output;

• Compare the key value with a standard value;

^• Take different actions depending on the key value. For example, execute other commands to further troubleshooting, find the root cause or escalate the issue.

[00112] However, each of these steps is generally performed manually on one network device at a time. No tools are yet available to simplify the tedious manual and error prone steps of the various network commands.

[00113] With the present invention, GUIs are utilized to provide a visual presentation of network commands, network executable processes and network strategic procedures. These commands and processes are enabled to be visually represented, defined, and made accessible through GUIs and visual symbols as well.

[00114] The system includes a GUI to define an Executable Procedure. This user interface provides an easy way to define Procedures (used inter-exchangeably also as Executive Procedure, Executable Procedure) so that a user without any programming knowledge can create a Procedure. After a Procedure is saved, the system creates a standalone application containing executable codes. An exemplary implementation is done by Python Script. Any other suitable types of programming languages can also be used to convert a Procedure defined through the GUI to an executable code.

[00115] A "Probe" is a set of functions that retrieves and parses data from a device.

[00116] A "Trigger" is a set of functions that defines the logic to analyze data.

[00117] A "Process Node" is a visual representation of a block of executable codes that generally include zero to multiple "Probes" and "Triggers".

[00118] There are four types of Probes: CLI command Probe runs CLI commands, and parses and analyzes the result; Configuration Probe analyzes the configurations; Ping Probe checks the connectivity between devices; Traceroute Probe runs the traceroute command between two devices.

[00119] An "Executable Procedure" (Sometimes called "Procedure") is a set of processes and strategies to achieve a result which can be presented visually through GUI. It may contain multiple Process Nodes and logic workflows from one Process Node to another.

[00120] A "Parser" is a set of functions that defines how to retrieve data from the output of an execution of a CLI, ping, traceroute and any other types of commands. Depending on the output formats, for example, four types of Parsers are provided: Keyword, Paragraph, Table and Filter Parsers.

[00121] The configured and saved Executable Procedures automate conventional troubleshooting steps. Using the GUI based network management system, NETBRAI Workstation, an Executable Procedure can perform the following tasks automatically:

• Issue the command (CLI command/ping/traceroute/SNMP) in network devices and collect the output via a Probe;

• Parse the command output to retrieve key data via a Parser;

• Analyze the key data via a Trigger;

• Output possible errors or warnings and expert advices via a GUI.

• Create maps and/or documents for an underlying network system or the troubleshooting process.

[00122] A "Sample Output" is an output of a command executed in a target device. A Sample Output can be generated in real time on real devices at the time of troubleshooting. In order to provide correct data analysis, a network professional is guided to collect a Sample Output on a device and use this Sample Output to define a set of Parsers for subsequent data analysis.

[00123] In reference to FIG. 1, a GUI based Procedure system 100 for a network management is shown. The system includes a GUI 105 to define an Executable Procedure 107. The Executable Procedure is defined by a set of visual block-based programming interfaces so that a user without any programming background can still effectively program his know-how. After a Procedure is saved, the system creates a standalone application containing the executable codes. An exemplary implementation is done by Python Script though any other type of programming language can be used to convert the Procedure defined through the GUI to an executable code.

[00124] Executable Procedure 107 can be executed within a network map

101. A common use case is: a user creates a map 101 to include the network devices and/or network interfaces relevant to a network task, and then selects the relevant Procedures to run. Executable Procedures can also take the user input 103 through a user interface. While a Procedure is performed, it collects the data from various types of network devices in the live network 111 via a Live Access Mechanism 109. The output of an Executable Procedure includes warning or error messages 113, customized reports 115 and network maps 117 with the problem area being highlighted or noted. [00125] In reference to FIG. 2, the process of Troubleshooting via an Executable Procedure is illustrated. At step 201, a group of built-in functions are called and executed on a network or a device to collect data; the data is parsed at step 203 to extract key information; then a Trigger is used to analyze the obtained key information at step 205, the results are thus displayed at step 207; a network map or document may be created to record the troubleshooting results or process at step 209, and possible solutions are provided at step 211. The knowledge or logic to troubleshoot a network problem is included in the Procedure and so a network professional does not need to memorize manuals or the steps for a common network problem.

[00126] In reference to FIG. 3, an example Executable Procedure 300 includes various numbers of Process Nodes 301 which further includes various number of Probes (Probel 303, Probe2 302, ...etc). Probel 303 could include a number of combinations of commands, standard functions and/or proprietary functions, such as CLI Command 305, Configuration (DR) 307, Ping 309 and/or Traceroute 311. Process Node 301 also includes a various number of combinations of Parsers 313 which may include Keyword Parser 315, Table Parser 317, Paragraph Parser 319 and/or a Filter 321. Process Node 301 also includes a various number of combinations of Triggers 325 that defines a various set of "If and "Then" analysis logic loops 327 and 329. Trigger 325 may include a various number of settings, for example, the settings for Threshold, Compare, Delta and/or Advanced settings. Variable output 323 from Parser 313 is thus analyzed automatically with the pre-set conditions of normality or abnormalities.

[00127] Node 331 is an Overview Node that may be provided to include the description, as to what the Procedure does, the author and the sample map.

[00128] A Process Node may be configured to finish a single task and is the programming unit of an Executable Procedure. Each Node is conceptually executed on one device at a time, although a built-in logic loop allows the same logic to be executed across a dynamic set of devices. A Process Node may contain zero to multiple Probes and Triggers. A Probe retrieves and parses the data from a device. A Trigger defines the logic to analyze the data. There are four built-in Probes corresponding to the common tools for the network management: [00129] CLI command Probe is to run CLI command, parse and analyze the result. Configuration Probe is to analyze the configurations. Ping Probe is to check the connectivity between devices. Traceroute Probe is to run a traceroute between two devices.

[00130] Besides these Probes, the system can expand to other Probes such as SNMP Probes, retrieving the data via SNMP and analyzing the data.

[00131] A Parser defines how to parse the data from the sample output. Depending on the formats of the sample output, Parsers may parse the data using a Keyword, Paragraph, Table and Filter parser.

[00132] Keyword Parser is a Parser to retrieve a single instance of the data; for example, to retrieve the IOS version from the output of "show version" command.

[00133] A Paragraph Parser is for parsing the data if the original data (configurations or CLI command output) includes multiple repeating instances of the data; for example, to retrieve the CDP neighbor entries from the output of the "show cdp neighbors" command.

[00134] A Table Parser is for parsing the data if the CLI command output is formatted as a table; for example, to retrieve EIGRP neighbor details from the command "show ip eigrp neighbor".

[00135] A Filter Parser is for parsing the data if you want to filter a partial data from the original data.

[00136] The data retrieved from the parser are stored in various output variables.

[00137] A Trigger defines the control flow to analyze the output variables retrieved by the Parser. For example, a Threshold Trigger can run a Parser once and compare a variable with a threshold value. For example, a Threshold Trigger can compare the CPU usage of network devices with a threshold value, such as 90%. If the CPU usage is higher than this threshold value, a warning message is created.

[00138] A Compare Trigger can run a Parser against two data sources (live data and baseline data) and check whether a variable changes. For example, Compare Trigger can compare configurations retrieved from a live network with benchmarked configurations and output any difference.

[00139] A Delta Trigger can run a Parser twice with a certain time interval and checks whether a variable changes. For example, a Delta Trigger can retrieve CRC errors of a network interface within a certain interval such as 5 seconds, and if the CRC errors increase, an error message is created indicating that the cable connected to this interface does not work properly

[00140] If the other Triggers do not find the problem, an Advanced Trigger which provides advanced options may be used.

[00141] The general logic for a Trigger is as follows: if (condition 1)

action block 1

else if (condition 2)

action block 2

else

action block 3

[00142] Under the conditions is an action block that the system conducts under each condition. Each action block can include multiple messages, one block of expert advice, one statement block, one export variable block, and one control action probe.

[00143] The message will be shown in the Message fields in a Procedure Task (the GUI to show Procedure results after the Procedure is run). There may be three types of messages: the error message indicating an error requiring an immediate action, the warning message indicating something abnormal occurred, which requires attention, and the information message.

[00144] The Expert Advice field is pure text for the Procedure user to give advice if a specified condition occurs. It will be displayed in the Procedure Task window when a user views the detail of a message.

[00145] The Statement fields can be any executable code such as making function calls to draw a map or creating customized fields for device properties. [00146] The Executable Procedures can be organized by category. In one example implementation, in reference to FIG. 4, a Procedure Center 400 is provided to manage the Procedures. Built-in Procedures for common use cases are provided under the built-in category 403, but a user created Procedure can also be placed and managed here and shared through a common server. By sharing the Executable Procedures inside an enterprise or across network professionals around the world, some common types of network problems can be quickly solved by running shared Executable Procedures. There are other categories of Procedures, such as Path Procedure 405, Shared Procedure 407 and Local Procedure 409.

[00147] At the top of the Procedure Center is search box 401, where a keyword (for example, "eigrp") can be entered and the Procedures matching the keyword will be found.

[00148] For Built-in Procedures, they may be categorized by the following use cases: Compliance, Device Level Check, Draw Map, Interface Level Check, Inventory, Multicasting, QoS, Routing, Switching and Verification. A category can also have subcategories. For example, the routing Procedure may have five subcategories: BGP, EIGRP, ISIS, OSPF and RIP.

[00149] For Path Procedures, they are a special type of Procedures used to discover the path between two end points. There are built-in path Procedures and customized Procedures.

[00150] For Shared Procedures, they are saved in a common database of the network management system and can be accessed by any client.

[00151] For Local Procedures, they are only saved on the local disk and not shared with others.

[00152] The Procedures are often run from within a network topology map. A common use case is: the user creates a map for the network devices relevant to a network (for example, the problem area for a troubleshooting task). Then he runs the Procedures from within the map to gather data, analyze data and eliminate possible causes.

[00153] FIG. 5 shows an example implementation to run a Procedure within a map 500. A run procedure menu 501 is added in the float menu 503 of the map. After a user clicks Run Procedure in Menu 501, a window shown in FIG. 6 is displayed for the user to select Procedures from the Procedure Center. The user can click the + sign in front of any category and select Procedures in the Procedure Center to run that Procedure.

[00154] FIG. 7 is a Procedure Task window 700 to display Procedure results. The selected Procedures are listed in Pane 701 and all messages relevant to this Procedure are displayed in Pane 703. If selecting a Procedure in Pane 701, then only the messages of this Procedure are displayed in Pane 703. A user can also select the type of messages to be displayed. For example, by only checking the Error checkbox and unchecking other checkboxes a user can only view error messages. Details of a selected message are displayed in Pane 705. The command output related to this message is also shown in Pane 705. The expert advice is shown in Pane 707 and the trigger to print out this message is shown in Pane 709. The execution log for the whole Procedure Task can also be displayed in Pane 705 when the tab Execution Log 720 is selected. The execution log displays the details on how the Procedures are executed.

[00155] The network devices on which the Procedures are run are listed in Pane 713. You can use the Select Seed Devices link to add more devices. Or, you can remove one or multiple devices by right clicking and selecting "Remove" from the menu.

[00156] The Procedure tasks can be saved as a file by clicking the Save button 715. The saved Procedure Task can be opened for future examination or be sent to a peer for review. Also the Run Procedure button 717 allows a user to rerun a Procedure Task.

[00157] FIG. 8 shows an Executable Procedure example window 800. This example Procedure is used to check whether the speed or duplex of the neighbor interfaces are mismatched. The Buttons 810 and 820 are used to define the global input and output variables of a Procedure, which will be discovered in detail later. The flow chart in the upper pane 830 describes the overall flow of the Procedure. A Procedure has the summary Node 832 and one or more Process Nodes. In this example there are three Process Nodes 834, 836 and 838. The lower pane 850 shows the details of the current Node (the Node with the arrow 860 under it). Click any node to set it as the current node. [00158] In the summary node 832, a user can enter a description 852 to describe what the Procedure is for, its author 854, and its contact 856. The link Import Sample Qmap 858 can be used to import a map to illustrate the problems this Procedure tries to solve.

[00159] In this example, the description 852 gives the summary of the

Procedure and steps to solve the problems:

This procedure checks whether speed and duplex values are consistent across connected interfaces. Discrepancies are highlighted in the map.

Step 1

Get CDP neighbor details on local device to identify adjacent interfaces Related command: show cdp neighbors detail

Step 2

Check local interface speed and duplex

Related command: show interface

Step 3

Compare speed/duplex on local interface with speed/duplex on neighbor interface

Note: This procedure requires CDP to be enabled on each device.

[00160] Without any automation it may take a few days to perform these steps. With the Executable Procedure Interface, three process nodes 834, 836 and 838 are created to execute corresponding steps 1, 2 and 3 in minutes.

[00161] After the Procedure is defined, click save button 870 to save the Procedure. The Procedure will be saved as a file with the specific file name, for example, .qapp (meaning the quick application).

[00162] FIG. 9 shows how to define a Process Node. For example, two options may control how the Process Node is run by Loop 920 and Devices 930. The Loop option defines the loop for the block of codes corresponding to the Process Node. The Devices option defines what network devices the Node should be run on.

[00163] There may be two options for Loop 920: Run Once, which means that the Node will only run once for each seed device, and Loop by Variable, meaning that the Node will run for each element of the variable.

[00164] There may be three options for Devices Option 930: Seed Device, By Variable and Dynamic Device. The default option Seed Device means that the Node will run on the seed device. The seed devices are selected by the user while running the Procedure. The option By Variable means that the node will run on the devices defined by the variable. The option Dynamic Device is used to run the Procedure recursively until a certain condition is satisfied. The Dynamic Device option can be used to map out the topology from a seed device.

[00165] The user can select one of the four types of Probes. For example, by clicking "add a CLI command Probe" 930 to define the CLI command probe, a window 1000 is shown (FIG. 10).

[00166] A user may first enter the CLI command in field 1010. In the example here, the CLI command, "show cdp neighbors detail", is used to retrieve the neighbor device and connected interfaces. Second a user may retrieve a sample output to define a Parser. Click the Retrieve Sample button 1020 and select a device. The sample output is shown in field 1030. The following is an example Sample output: labIosSwitch3>show cdp neighbors detail

Device ID: 2900XL-1

Entry address(es):

IP address: 192.168.1.210

Platform: cisco WS-C2924C-XL, Capabilities: Trans-Bridge Switch

Interface: FastEthernetO/3, Port ID (outgoing port): FastEthernetO/5

Holdtime : 150 sec

Version :

Cisco Internetwork Operating System Software

IOS (tm) C2900xl Software (C2900xl-C3H2S-M), Version 12.0(5) WC5, RELEASE SOFTWARE (fcl)

Copyright (c) 1986-2002 by cisco Systems, Inc.

Compiled Tue 28-May-02 11:11 by devgoyal advertisement version: 2

Protocol Hello: OUI=0x00000Q Protocol ID=0x0112; payload len=27, value=00000000FFFFFFFF010121FF000000000000005080703CC0FF0001 VTP Management Domain: "

Native VLAN: 1

Duplex: full

Management addres (es):

Device ID: NY POP

Entry address(es): IP address: 172.22.20.2

Platform: cisco 2500, Capabilities: Router

Interface: FastEthernetO/7 , Port ID (outgoing port): EthernetO

Holdtime : 160 sec

Version :

Cisco Internetwork Operating System Software

IOS (tm) 3000 Software (IGS-IN-L), Version 11.1(10), RELEASE SOFTWARE (Id)

Copyright (c) 1986-1997 by cisco Systems, Inc.

Compiled Mon 10-Mar-97 15:53 by dschwart advertisement version: 1

Management address (es):

[00167] By using the provided Sample output, a user can define a set of Parsers with window 1040 for the Procedure to retrieve the data from a running output. Depending on the sample formats, you can select four types of Parsers: Keyword, Paragraph, Table and Filter.

[00168] The sample output includes multiple neighbors. The output of each neighbor has identical formatting. For this type of output, select the Paragraph parser 1042 to parse the data. The Paragraph Identifier 1044 is the keyword to identify the start of a new paragraph, ' 'in this sample. For each paragraph you can define the keyword/variable pair 1046 (keyword parser). The keyword is the string which is always the same and the variable value is the value which can change. In this example, we define three keyword variable pairs:

IP Address: $nbr_ip

Interface: nbrjntf,

(outgoing port): $local_intf

[00169] The matched values are highlighted in the sample output and also shown in the pane 1050.

[00170] FIG. 11 shows an example window 1100 to define a Trigger. This threshold trigger 1110 checks whether one of the variables defined in the parser Is "Not None". If so, it executes the statements to assign variables and then export these variables so that the process nodes after this process node can use them. [00171] FIG. 12 shows an example GUI 1200 with the settings to run a Procedure. There are three types of settings: 1) Data Source 1210. By default, a standard Procedure retrieves the data from the live network. However, you can set the option to use cached data stored in a data folder. In the Trigger, the current data is compared with the baseline data. By default, the current baseline serves as the baseline data. You can also select another data folder for the baseline data. 2) Default interval for Delta trigger 1220. For the Delta trigger, the data will be retrieved twice, with the interval value defined here. 3) Export global output variable results 1230. Check this option to export the global output variables and select a file directory for the export.

[00172] A Procedure can have input variables and output variables like an application. The input variables allow a Procedure to be run in different environments without any modification.

[00173] FIG. 13 shows an example implementation on how to define input variables for an Executable Procedure. To define a global input variable, click the Define Input Variable button 1310 at the top of the Procedure window. In the Define Global Input Variable window 1320, click the Add button 1330 to add the input variables. In the Add Global Input Variable window, enter the variable name and select the type. In our example implementation, the global variables always starts with $$ to differentiate with the local variables of a process node. Other options of symbols may also be used or implemented. The description is optional, but a meaningful description can make the Procedure easy to read and use. The initial value is also optional and should be set to the most frequently used values if possible. Click the Multiple Value link 1340 to set more than one value and the system will run the Procedure with each value. This can be convenient in some cases, for example, if we create a Procedure to map a multicasting source tree, we can run this Procedure with the input variable set to multiple sources.

[00174] FIG. 14 shows an example implementation on how to define output variables. One of the primary purposes of the global output variable is to create a report. For example, you may want to create a report to include all devices and neighbor interfaces having the duplex or speed mismatched.

[00175] To define output variables, click the Define Output Variables button 1410 at the top of the Procedure window 1400. In the Define Global Output Variable window 1420, click the Add Table button 1430 to add a variable table or the Add Single button 1440 to add a basic variable. Like the global input variable, the global output variable should start with $$. A table can have many columns and each column can have different types of variables.

[00176] Besides the CLI command probe, the system also supports the Ping, Traceroute and Configuration Probes.

[00177] FIG. 15 shows how to define a Ping Probe. Mainly you need define the source 1510 (the device to ping from) and the destination 1520 (the IP to ping to). For the source 1510, you have three options: local PC 1512; the network server 1514, which is a specified server used to work as a proxy to the live network; or the selected devices 1516, where you can define a list of core devices as the input variables and ask the system to ping from these devices.

[00178] For the destinations 1520, you can either enter the IP address 1522 to ping from or select a device 1524 and then an interface on the device. In the example shown here, we check the IP Host option and enter the input variable which defines the IP address to ping to.

[00179] FIG. 16 shows how to define the Traceroute Probe. The definition of the Traceroute Probe is similar to the Ping Probe. Ping and Traceroute Probes can be defined to run in a list of core network devices to a list of main servers after a network change. This automation can be much quicker and more reliable compared to the manual process.

[00180] The Configuration Probe enables one to parse and highlight configurations. For example, the Configuration Probes can be used in the following use cases: 1) to create a report for the devices containing a particular configuration line, for example, finding the devices with the "no service password-encryption" configuration, which violates basic security policies; 2) to highlight or draw a particular configuration in the Q-map; or 3) to do a preliminary check before applying an additional Procedure. This can improve the Procedure performance since the configuration probe uses the baseline configurations without retrieving data from the devices. For example, we can check whether OSPF is conFIG.d to run on a router before applying any Procedure to troubleshoot OSPF routing issues. [00181] FIG. 17 shows how to define a configuration probe. The Parser and Trigger of a Configuration Probe are the same as those of the CLI command Probe. The differences may be that the Configuration Probe only works on configurations, and you do not need to define a CLI command to retrieve the data.

[00182] FIG. 18 shows an example a network map created by using a Procedure.

[00183] In reference to FIG. 19, a GUI based network management system 1900 is shown. The system includes a GUI 1905 to define an Executable Procedure 1907. The Executable Procedure is defined by a set of visual block-based programming interfaces so that a user without programming background can still effectively program his know-how. After a Procedure is saved, the system creates a standalone application containing the executable codes. An exemplary implementation is done by Python Script, though any other type of programming language can be used to transfer the procedure defined through the GUI to executable codes.

[00184] Executable Procedure 1907 can be executed within a network map 1901. A common use case is: a user creates a map 1901 to include the network devices and/or network interfaces relevant to a network task, and then selects the relevant Procedures to run. Executable Procedures can also take the user input 1903 through a user interface. While a Procedure is performed, it collects data from the various types of network devices in the live network 1911 via the Live Access Mechanism 1909. The output of an Executable Procedure includes warning or error messages (1913), customized reports (1915) and maps (1917) with the problem area being highlighted or noted.

[00185] FIG. 20 shows an example control and data flow of an Executable Procedure. An Executable Procedure takes a network map, global input variables and/or seed devices (2020) as the input. The seed devices are network devices on which a user selects to run. When the procedure is run within a map, all devices in the map are automatically selected as the seed devices.

[00186] An Executable Procedure has one or multiple Process Nodes 2030. A Process Node performs three basic tasks: collect data from the network devices using commands defined by the user (step 2032); parse the command output to retrieve the data (step 2034); and analyze the data (step 2036). The data retrieved from a Parser are stored in computer variables.

[00187] The utility component that defines how to parse a command output is called a Parser. The system provides a sample guided method to configure a Parser. The parsed data are stored in variables and passed to another utility component, Trigger for analyses. A Trigger loops through all the value- loaded variables to explain and understand the data saved in the variables, as specified by a user.

[00188] A Trigger allows a user to easily define data control flow loops as follows:

if ($varl > threshold 1)

action block 1

else

action block 2

[00189] Inside each of the action blocks are actions to perform under each condition, which creates the desired output for this Process Node. The action blocks can include multiple messages, expert advice, statement blocks, export variable blocks and control action Probes, etc. The outputs of an Executable Procedure 2040 are created by the action blocks.

[00190] FIG. 21 is an example user interface for defining a Parser Utility. The CLI command is used as an example. The same idea can be applied to any other type of commands such as SNMP, Traceroute and Ping.

[00191] A user performs these steps to define a Parser: 1) enter the CLI command in field 2110, for example "show version" shown in this figure; 2) retrieve a sample output from the device of interest by clicking the Retrieve Sample button and select a device. The sample output is shown in field 2130; 3) define a set of Parsers using Parser User Interface 2135.

[00192] The Retrieve Sample function allows a user to see a real time device output in field 2130. The sample output is used to help define the parser.

[00193] Parser user interface 2135 defines how to retrieve data by the aid of a sample output, for example, four types of Parsers may be defined: Keyword, Paragraph, Table and Filter Parsers. [00194] The Keyword parser is used to retrieve a single instance of data with the selected keyword as a marker for the start of the data. The Keyword Parser uses the unchanged keyword to identify the value and the value is saved into a variable. For example, to get the version number from the following sample output of "show version" command:

BST-CORE>show version

Cisco Internetwork Operating System Software

IOS (tm) 2500 Software (C2500-IK8OS-L), Version 12.2(2%), RELEASE SOFTWARE (fcl)

Copyright (c) 1986-2007 by cisco Systems, Inc.

Compiled Mon 30-M-07 16:46 by kellythw

Image text-base: 0x03070A14, data-base: 0x00001000

[00195] To guarantee accuracy, a user can copy and paste the exact text "Version 12.2(29b)," into the keyword variable field 2140, leaving "Version" as is, to be the marker and replacing "12.2(29b)" with a variable "$version" to indicate to the Parser Utility that the text string following the text "Version" is to be collected as value and to be saved as a variable. After clicking the Apply Parser button 2150, the matched value is immediately highlighted in the output sample in field 2130 and the value is displayed in the pane 2160 under the sample output, which verifies that the Keyword Parser is correctly configured.

[00196] A Keyword parser contains one or more keyword/variable pairs. A variable may start with $, or some other symbol as defined in the system. The example system prefers the use of $.

[00197] The variable can be of different types other than a string, for example, an integer or a floating number. In order to define other types of variables, you need to explicitly set its type using the syntax "$<type>:<name>", where "<name>" is the variable name and the "$<type>" can be "int" for an integer, or "double" for a floating number. For example, to define an integer variable named memory for the memory usage, you may need to define it as "$int:memory".

[00198] Besides the plain text keywords, the system also supports the Regular Expression for retrieving data value more precisely. For example, in the previous sample output, "regex[$double:major_version]:Version (.+?)\(" can be used to retrieve the major version (12.2). The reserve word "regex" indicates that the keyword/variable pair definition is a Regular Expression, not a normal text matching. The variable(s) defined here are placed inside the brackets, "[ ]". Multiple variables are separated by a comma, for example, "regex[$double:major_version, $minor_version] : Version (.+?)\((.+?)\)". The regular expression is defined after the colon, The values to be captured are inside the parentheses, "( )", and assigned to the variables in order.

[00199] The Paragraph Parser is used to retrieve multiple instances of data if similarly-formatted paragraphs are included in a data output, such as a sample output of the CLI command "show cdp neighbors" shown in FIG. 22.

[00200] A sample output 2200 is collected. The sample output includes multiple paragraphs such as paragraph 2210 and 2220 separated by the line 2230. The output of each paragraph has identical formatting. For example, all paragraphs includes the line 2240 to show the IP addresses of the neighbor devices and line 450 to show the local interface and remote interface.

[00201] FIG. 23 shows how to define a Paragraph Parser to collect the IP addresses in each paragraph shown in FIG. 22. The Paragraph Identifier field 2310 is filled with the paragraph keyword to identify the start of a paragraph, string value of line

2230 " " for the sample in FIG. 22 that indicates a start of a new paragraph. A

Paragraph Parser can have multiple Keyword Parsers as child Parsers. The keyword variable pairs of these child Parsers are applied to each paragraph. In this example, we define three keyword variable pairs 2320, 2322 and 2324:

IP Address: $nbr_iplnterface: $nbr_intf, and ( outgoing port): $local_intf.

[00202] After clicking the Apply button 2330 to save this Parser, the matched values are immediately highlighted in the sample output 2340 and parsed results are shown as a table in pane 2350. The immediate visualization of the parsing of the sample output enables the confidence that the Parser is functionally and correctly defined.

[00203] The system automatically creates a data container, a table for the parsed variables. It does not require the user to define the variables as the table data type. A table or array variables can be used like the standard singular variables inside an Executable Procedure. The system will automatically create the logic loops in the executable code whenever necessary.

[00204] A Table parser is used to retrieve data from a table-formatted CLI command output such as the one sample output shown in FIG. 24.

[00205] The data output 2400 is formatted as a table. It includes header rows 2410 followed by data rows 2420 and 2430.

[00206] FIG. 25 shows how to define a Table Parser. First simply copy and paste the text string of Header from the sample output 2510 to the Header field 2520. Remove the space between fields and add a semicolon (;) to separate each field. Then define the variables by simply selecting the header field and assign it to a field name, such as "Interface => $nbr_int" 2530, and "Address => $nbr_addr" 2540. The variable definition follows the same rules as the Keyword Parser, for example, set the variable type as "Hold => $int:hold" 2550.

[00207] By clicking Apply 2560, the matched values are highlighted in sample output 760 and the parsed results are shown in pane 2570.

[00208] Among the parsed results shown in pane 2570, the first row shows "sec" also collected as data. This is because the header of this table is split into two lines. The advanced settings of the Table Parser can be used to skip the first line from the header.

[00209] FIG. 26 shows an advanced setting of the Table Parser. Split column method 2610 specifies table alignment: left alignment, right alignment or by number of characters. Set key variables field 2620 specifies the key variables for a table, which is used for the table comparison. End of table parse field 2630 defines the keyword to match for the end of a table. This can be helpful since a table often has a summary at the end. Skip the defined number (n) of lines after the header 2640 and it will not parse the first n lines after the header. Finally, the "If the column is empty, use the previous parsed value" field 2650 takes care of this situation when a value is empty and asks the system to use the last parsed value. [00210] Filter parser is used to filter partial data from the original output. For example, a user can filter EIGRP routing protocol configurations from the "show run" results.

[00211] FIG. 27 shows how to define a Filter Parser. There are two options: use a built-in Design Reader (DR) filter to filter the configuration or create a standard filter. A DR is a built-in standard filter for common use cases.

[00212] A standard filter filters the data between a beginning line keyword specified in field 2730 and an end line keyword specified in field 2740. Within a Filter Parser you can add new Parsers (child parser), which can be of any type and is applied to the filtered data. The filtered data is immediately highlighted in window 2750.

[00213] A Trigger analyzes the data retrieved by the Parsers. There are four types of Triggers:

[00214] Threshold Trigger: runs the Parser once and compares the variables with the threshold value. For example, compare the CPU of network devices with the threshold value such as 90%. If the CPU is larger than this value, a warning message is output.

[00215] Compare Trigger: runs the Parser against two data sources (live data and baseline data) and checks whether the variables change. For example, compare the configurations retrieved from the live network with the benchmarked configurations and output any difference.

[00216] Delta Trigger: runs the Parser twice with a certain time interval and checks whether the variables change. For example, retrieve CRC error of the network interfaces within a certain interval such as 5 seconds and if the CRC errors increase, output an error message indicating that the cable connected to this interface does not work properly.

[00217] Advanced Trigger: provides advanced options. Use this Trigger if the other triggers cannot complete the desired function.

[00218] The Threshold trigger is used to compare a variable defined in the parser with a threshold value. It supports the following control flow: if ($varl is None) action block 1

else if ( varl >= threshold value 1)

action block 2

else if ($varl >= threshold value 2)

action block 3

else

action block 4

[00219] Here action blocks can include one or more messages, one expert advice block, a statement block, one block to export variables, and one control action block. .

[00220] If the "$varl" is an array, the system automatically adds logic loops in this Threshold Trigger. Inside the logic loops, the system automatically loops through each element of the array. This same rule applies to all types of Triggers, except when using an advanced Trigger.

[00221] The Threshold Trigger can only compare one variable against the threshold value. For complex conditions involving two variables or involving "and" or "or" Boolean operations, use the Advanced trigger.

[00222] FIG. 28 shows a user interface of a Threshold Trigger. A threshold always includes one "if condition 2810 and can have one or multiple "else if conditions 2820 and "one else" conditions 2830. Twelve operations 2840 to compare the variables with the threshold value are supported by the threshold Trigger in the current implementation. The operations "==", "!=", "Is None" and "Is not None" can be applied to any type of variables. The operations ">", "<", ">=", "<=", "Belongs to" and "Does not belong to" can be applied to the integer and double type and the threshold value must be a number. The operations "Contains" and "Does not contain" can only be applied to the string variable.

[00223] The operators "Belongs to" and "Does not belong to" defines a range of numerical values such as "1-10; 50-80".

[00224] The Compare Trigger is used to compare whether a variable changes between two data sources (live data and baseline data). It supports the following control flow: if ($varl. Change () Changed)\

Action block 1

else

Action block 2

[00225] The Delta Trigger is used to compare whether a variable changes within a certain time interval. The Parser is run twice within the time interval. The Delta Trigger supports the following control flow which is similar to the compare parser: if$varl.delta() > valuel

Action block 1

else if$var2.delta() < value2

Action block 2

else

Action block 3

[00226] The Advanced Trigger can be used to implement a complex control flow which is too difficult to be implemented by the Threshold Trigger, for example:

If varl >= threshold '_value and $var2 < = threshold '_value2)

Action block 1

else $varl == value3 or $var2 < value4

Action block 2

[00227] Under the "if, "else if and "else" conditions is the action block, where a user can tell the system what to do under each condition. FIG. 29 shows all possible actions in an action block.

[00228] The message 2910 will be shown in the Message fields in the Procedure result window after the procedure is run. The message can include variables, and typically does.

[00229] The Expert Advice field 2920 is the pure text for the Procedure author to give advice if the condition occurs. It will be displayed in the Procedure Task window when the user views the detail of a message.

[00230] The Statement fields 2930 can be any code such as making function calls to draw in the map and creating customized fields for device properties. [00231] The Export Variables field 2940 exports the variables to be used by the next probe.

[00232] Finally the Control Action 2950 asks the system to exit the probe and procedure.

[00233] All programming codes require certain loop controls to solve a non-trial problem. Executable Procedure is no exception. However, the loops are hidden from the user and automatically implemented while the system creates the executable codes from the user-defined Procedure.

[00234] FIG. 30 illustrates the hidden loops implemented by the system. For loop 3010, the procedure is run once for each of the seed devices. If the global input variables 3020 of the procedure have multiple values, the procedure will be run for each of the input variables. For example, an Executable Procedure to check whether the route to multiple destinations exists in core routers can take the multiple destinations as the global input variables and loop through each core device.

[00235] Inside the procedure, the user can select a process node to loop through a variable 3030, meaning that the Node will be run for each element of the variable. For example, to check whether the duplex of the neighbor interfaces are matched, the first Process Node of the procedure may get all CDP neighbors of a seed device and pass the CDP neighbor (which is a table) to the next Process Nodes. The second Process Node of the Procedure will loop through each local interface to retrieve the duplex. And the third Process Node of the Procedure loops through each remote interface to retrieve the duplex and do the comparison. Basically we have a loop such as:

For each local interface having the neighbor

Run Process node 2

[00236] Inside the Process Node, the system also provides a recursive control 3040 to recursively loop through the Process Node until a certain condition is satisfied. This option can be used to map out the topology from a seed device and discover the path.

[00237] Finally inside a Trigger, if the variable to be compared is an array or a table, the system automatically adds a loop. [00238] The Procedure can be run in two modes: the debugging mode and the live mode. In the debugging mode, the system will show the flow, including all loops, and the values of loop variables. In the live mode, only the final results are displayed.

[00239] FIG. 31 shows the output of a Procedure run in the debugging mode. The loop variables are displayed with the value in the current cycle. For example, the Procedure is looping through the input variable 3110 which is a table and has the value ("10.10.10.10", "2.2.2.2") in the current loop. The second loop is the seed device 3220. For the Process node 1, the Trigger 1 loops through the variable "$cdp.nbr_ip 3330".

[00240] FIG. 32 is a schematic block diagram showing the relevant components of a network change management system. The system includes a user interface 3201 to define a network change task 3203 from beginning to end. A network change task is divided into clearly defined subtasks or action items. The history of each action or subtask is recorded in a time line (or timeline, both are used inter- exchangeably). A network change task can be saved at any stage as a self-contained file, which can be sent to anyone for the purpose of peer review or proof of a change.

[00241] A network change task can be subsequently instructed to send CLI commands 3205 to network devices to execute the network change or collect the benchmarked data. Documents can be created for a network management task. In an example implementation, a Word document 3204 may be created, though other formatting options, such as a PDF file can be also created.

[00242] FIG. 33 outlines a work flow for conducting a change management. Following this best practiced workflow can greatly reduce the problems or errors caused by a network change. The flow has the following steps: the first step 3310 is to define the planned network change. A user defines what is going to be changed, for example, what configurations are to be modified. In this step the user may also define what devices are to be impacted by this planned change. Since all network devices are connected, the planned network change may not only affect the target devices but also affect other related devices. For example, the configuration change of a dynamic routing protocol such as EIGRP in a router will affect the routings of its EIGRP neighbor routers. [00243] The next step 3320 is to benchmark the network states for the modified and potentially impacted devices before the change. This benchmarked data will be used to compare with the data benchmarked after the change.

[00244] Next, the user can execute the change at step 3330. He can instruct the system to automatically log into the devices and issue CLI commands to execute the change. The user can view the execution process, stop and/or continue the process anytime. Or, he can rollback the change and make modifications on the plan.

[00245] After the change is executed, another benchmarking process (step 3340) is run to collect the network state. The benchmarked data before and after the change are now compared in step 3350. Visual user interface is provided in which the differences are highlighted and displayed. After analyzing the difference, the user can decide whether the results are acceptable. If the results indicate a problem was introduced by the change, the user can go back to step 3330 and roll back the configuration changes.

[00246] In step 3360 the benchmarking task can be imported to a document. The user can select what data is to be included in the document. In a preferred implementation, a Word document is created. Other formats may be selected at the user's preference.

[00247] FIG. 34 is a user interface to implement the workflow illustrated in FIG. 33. The flow chart 3410 shows the subtasks or action items in the order of time. The first subtask, the summary subtask 3412, is for a general description of the task and can include a topology map 3434 to show the network devices to be changed or to be affected by the change. Other subtasks, Define Network Change 3413, Benchmark Before 3414, Execute 3415, Benchmark After 3416, Compare 3417, and Document 3418 comprise a full circle for managing a network change, as illustrated in FIG. 33.

[00248] The order of each subtask in the flow chart is recommended but not mandatory. The user can skip a subtask or jump to any subtask. A subtask can also be repeated many times. In fact, for a complex network change task, a user often needs to go through the flow a few times to get satisfactory results. [00249] A timeline 3420 shows the history of the network change task. Each node of the timeline corresponds to an action or an executed subtask. The timeline is automatically updated after a subtask or an action is finished.

[00250] The details of the current subtask are displayed in the work window 3430. For example, for the summary subtask 3412, the description and map 3434 are displayed.

[00251] FIG. 35 is an example of the timeline. The nodes in the time line are the executed subtasks or actions of a network task in the order of time. Any reviewing action, benchmarking, or execution tasks are displayed in the timeline.

[00252] The first node 3510 shows the time the task is created. The second node 3520 shows a completed benchmarking task. Moving mouse over node 3520 brings up a tip window 3530 showing the details of the benchmarking task: the executed time of the benchmarking, the number of devices on which the benchmarking task was executed, and the number of CLI commands that were issued. Right click the node 3520 and the operations which can be performed on a node are displayed in the menu 3540. The operations may be different for different types of subtasks. The "History Review ..." menu 3542 will display the benchmarked result status in the work window. The "Set as First DataFolder" menu 3544 and "Set as Second DataFolder" 3546 will set the benchmarked results as the data source for the comparison subtask. A DataFolder is an alias of the file folder while the benchmarked data are stored. The delete menu 3548 removes this node as well as the historical record from the time line.

[00253] The time line is a good way to understand the history related to a network change task. Particularly, the time line can be used to show the proving process of a task. Click the button "History ..." 3550 and the user can add a review task 3555. In this task, he can prove or reject a network change. The review task is added as a node 3560 in the time line.

[00254] FIG. 36 shows how a user defines a network change. The devices to be changed, added, removed, upgraded and impacted are listed in pane 3610. The right click menu is provided for the user to select or unselect the devices.

[00255] The majority of network changes are configuration changes that are executed through the CLI interfaces. The config template window 3620 provides a way to create a template of configuration changes which can be applied to all devices to be changed. The template 3630 supports the normal configuration commands such as: conflg t

ntp sewer 10.10.10.100

[00256] The first command is used to enter the configuration command and the second command configures the NTP server. The template also supports a special syntax to auto-create the same configurations for a list of interfaces. For example:

<interfacef*

duplex auto>

[00257] This template is used to create the configuration "duplex auto" for all interfaces with the name starting with the letter "f." After the template is applied to a particular device, it may create the following configurations for example: interface FastEthernetO/0

duplex auto

interface FastEthernetO/1

duplex auto

[00258] The auto-created configurations are displayed in pane 3640.

[00259] FIG. 37 is a configuration change for an individual device. Pane 3710 shows the configurations to be entered in device 3701. Configurations are edited in this space. Pane 3720 shows the rollback configurations. Rollback configurations are useful in the case where a specific change causes a serious problem on the network. In one implementation, rollback configurations can be automatically created. However, a user can always edit the rollback configurations.

[00260] It is critical to provide the rollback configurations for a network management task. Due to the complexity of the modern network, a change can often lead to unexpected results and the rollback configurations can be applied in case the problem cannot be resolved quickly. [00261] FIG. 38 shows an example interface for performing a Benchmarking subtask. Benchmarking means screenshot of a network state. Generally it is about the states of the target device and its neighbor devices. A user may add a list commands and devices to represent a network state. Network professionals usually use CLI commands to check the network state. It is not efficient and error prone to manually collect the data before and after a network change. This invention provides an automatic and systematic way to collect system state data before and after a change.

[00262] The benchmarked CLI commands are listed in pane 3810, as well as the execution results for each device. Two types of data, configuration file 3820 and route table 3830 are always retrieved since they will often change. Other CLI commands can also be retrieved. In this example, the CLI commands 3840 related to the EIGRP routing protocols, "show ip eigrp neighbors", "show ip eigrp summary" and "show ip eigrp interfaces" are also benchmarked.

[00263] The commands to be benchmarked can be added individually (3860) or via a template (3850). The template defines a list of CLI commands which should be checked for a type of network change task according to best practices.

[00264] FIG. 39 shows a user interface to edit and apply the CLI command template. The examples shown here are the CLI commands which should be benchmarked for a network change related to BGP routing protocol: "show ip bgp summary", "show ip route summary", "show ip bgp neighbors" and "show ip bgp peer- group".

[00265] FIG. 40 is a user interface to execute CLI commands. The system provides three options to execute commands. With the option "All at Once" 4010, the system will execute all CLI commands on all devices without pause. With the option "Device by Device" 4012, the system will pause after executing the CLI commands on one device and will only continue on the next device after the user clicks the execution button 4020. With the option "Line by Line" 4014, the system will pause after executing one line and will continue the next line after the user clicks the execution button.

[00266] The stop button 4022 is provided to allow a user, at any time, to prevent the system from executing the commands further. The execution button 4020 is also used to instruct the system to continue the execution. [00267] The window 4030 is the telnet/SSH window showing how the system logs into the device and issues CLI commands. All commands that the user defines in step 2 will be automatically issued on the device. The execution log 4040 shows how the system logs into the device and all CLI commands issued to the device.

[00268] The pane 4050 shows the execution status for each CLI command and pane 960 shows execution status for each device. The status is completed if all CLI commands are executed successfully. If any error occurs, such as where a device cannot be accessed, an error will be reported in window 4060.

[00269] FIG. 41 shows how to run the Comparison subtask. The comparison subtask compares the benchmarked data before and after the change. The differences will be highlighted so that the user can easily see what differences the network change makes.

[00270] If the user runs the benchmarking task before and after the change, the folder to store these two benchmarked data are automatically selected as the data source to be compared in the first DataFolder 4110 and the second DataFolder 4120. The user can also select other data sources available in the system to be compared.

[00271] The comparison results of the benchmarked data including the configuration files, route tables, and CLI command results are summarized in the pane 4130 under the different tags. For the configuration files, the summary shows whether or not the configuration file for each device has been changed. Double click an entry to show the configuration difference.

[00272] The configuration differences are highlighted in window 4140. In this example, the system provides two buttons, "Next" 4150 and "Previous" 4160 for a user to quickly find the next or last configuration difference.

[00273] The comparison of route tables and other CLI commands are displayed in a similar fashion. By looking through these differences, the user can quickly find out whether the network change leads to the expected results or any unexpected side effect. If he is not satisfied with the result, he can either execute the rollback configurations to roll back his change or define another network change.

[00274] The network management task can be saved as a self contained document. This document can be sent to another user for review. The other user can open the document by the change management system. If the other user does not have the change management system installed in his PC, the task can be imported to a popular document format, such as a Word document.

[00275] FIG. 42 shows the contents the user can select to export. By checking the "Review History" option 4210, the system will export a review history table like this:

Review History

Issue Date Author Comments

Created 2012/8/20 John create change management task Reviewed 2012/8/22 Smith Approved

[00276] By checking the "Implementation History" option 1120, the system will export an implementation history such as:

Implementation History

Issue Date Author Comments Zipped Data

Benchmark 2012/8/23 Nick 50 Devices, 300 Show commands Execute 2012/8/23 Nick Configlet in Changel

[00277] If the user checks the "Attach Zipped File of DataFolder and Log" option 4230, the data and the execution log will be zipped and attached into the zipped data column.

[00278] Similar data will be exported under other categories: Summary 4240, Export Map 4250, and Change and Config Change and Result 4260. Most data are exported as table formats for easy reading.

[00279] As will be recognized by those skilled in the art, the innovative concepts described in the present application can be modified and varied over a tremendous range of applications, and accordingly the scope of patented subject matter is not limited by any of the specific exemplary teachings given. It is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

[00280] None of the description in the present application should be read as implying that any particular element, step, or function is an essential element which must be included in the claim scope: THE SCOPE OF PATENTED SUBJECT MATTER IS DEFINED ONLY BY THE ALLOWED CLAIMS. Moreover, none of these claims are intended to invoke paragraph six of 35 USC section 112 unless the exact words "means for" are followed by a participle.

[00281] The claims as filed are intended to be as comprehensive as possible, and NO subject matter is intentionally relinquished, dedicated, or abandoned.