This invention relates to testing computer networks.

Extensive testing of all possible configurations for a system under test (SUT) is unrealistic under time and resource constraints. Testing for complex systems is a challenging task. This is especially true when the system under test has a large number of test configurations and a series of tests must be executed for each configuration. For example, ethernet-based deterministic networking (DTN) technologies have been developed to cope with the high-speed data communication requirements of modern avionics applications. Deterministic networks operate using a global sense of time and a shared schedule among network components. ARINC 664, a widely used DTN technology, provides limited latency and jitter when safety-critical applications communicate with each other over avionic networks within an air vehicle. Testing ARINC 664 switches is a challenge due to their safety-critical nature, availability of numerous configuration options, and high-speed operation using thousands of virtual links. The most important components of an ARINC 664 DTN are: Switch, End Systems and Virtual Links. Applications running on different end systems communicate with each other using virtual links directed by the switch. Limited latency and jitter requirements for Virtual Links are met by ARINC 664's offline scheduling: Number of virtual links, source and destination end systems, minimum and maximum frame sizes, and bandwidth allocation gap (BAG) for all virtual links. The main idea behind providing deterministic communication is to limit the amount of traffic injected into the network by each virtual link by adjusting the allowed frame size and BAG values to achieve the desired latency and jitter targets. BAG specifies the minimum transmission time between the first bits of two consecutive frames. A trafficshaping mechanism running on each virtual link would control the amount of traffic added by each node and shape traffic that exceeds the average and peak rates. The switch would enforce filtering and monitoring requirements, limit error propagation, and ensure determinism.

Orthogonal Arrays (OAs) are among the mathematical tools for designing an optimal mixture of multiple variables in a series of experiments. They are effective in reducing the number of experiments where a large number of combinations would otherwise be required. They are classified as same-level and mix- level orthogonal arrays. In general, an orthogonal array is defined as a tuple of 4 variables: OA (N, F, L, S). N is the number of experiments (also called runs), representing the number of rows in an orthogonal array, F, called Factor, is the number of columns in an orthogonal array, where each column corresponds to the variable of one experiment. L is the number of levels representing the number of different values a factor can take. S is called the power of an OA (0 < S < F). An orthogonal array is designed such that the number of rows required to contain S-wise combinations of variables is given by L ^s . When S = F, all possible combinations of variables are included in the experiment. By setting smaller values for S, a user can choose the degree of combinations of variables to be considered. When all factors F have the same number of levels L, orthogonal arrays are called same-level orthogonal arrays. Orthogonal arrays allow reducing the number of runs to L ^s by choosing a value of S to include S-method variable combinations. For example, when S is set to 2, all combinations of binary variables can be covered. Due to time constraints where A is a real number between 0 and 1, the number of runs can be further reduced to N = A ■ L ^s . The choice of A is determined by testers depending on the complexity and number of parameters of the particular system under test, so smaller values are preferred for complex systems. However, in real-life situations, some factors may have different numbers of levels to test. For example, on an ARINC 664 switch, mapping incoming traffic flows to outgoing ports may have a large number of possible assignments to test (e.g., up to 24 ports), while the number of different priorities for a flow may require much fewer options (for example, having two priorities, high and low). Orthogonal arrays with factors at different levels are called mixed-level orthogonal arrays. The variable groups of a mixed-level orthogonal array are defined as: OA (N, , S). iF is the number of different factors with the same number of Lt levels for i=(1 ,2, ... v) and = F .

JP2019096029A, which is included in the known-state of the art, discloses a register optimization method for optimizing the set values of registers in a microprocessor. It comprises the steps of defining a first orthogonal array in which all level combinations are defined, defining a first factor that describes the factors affecting the operation of the system, determining a factor as a result of the test, and assigning the bits in the predetermined unit bits to the factor by a second orthogonal array. US5621665A, which is included in the known-state of the art, discloses the optimization of industrial processes. For each experiment, the levels of the second-factor group are determined according to the levels of the first-factor group.

CN112949094A, which is included in the known-state of the art, discloses an electromagnetic performance margin analysis and reliability evaluation method for an avionic product. In the method for electromagnetic performance margin analysis and reliability evaluation for the avionic product, Taguchi simulation test design is considered to create an m-level n-factor simulation test scheme for use in extracting and recording key performance parameters in each simulation instance.

Thanks to an electronic test system according to the present invention, optimum selections can be made for multiple variables in experimental sets by using chained orthogonal arrays.

Another object of the invention is to enable all configurations to be tested by using an output of the first orthogonal array as the input of the second orthogonal array.

The electronic test system realized to achieve the object of the invention, which is defined in the first claim and other claims dependent thereon, comprises a plurality of electronic equipment communicating with each other. There is at least one computer network that realizes the Ethernet data network between electronic equipment. There is at least one switch that manages data communication between electronic equipment in the computer network. At least one port provides the input and/or output of data between electronic equipment. The processor enables performance testing of the computer network.

In the electronic test system according to the invention, a plurality of parameters is determined by a user to test the computer network. Among the detected parameters, a plurality of first factors through each port is determined by the user. There is a plurality of second factors that have been determined to affect the operating performance of the first factors determined by the user. The test is carried out by combining and/or matching combinations of the first factors so that at least one output parameter is obtained. The output parameter is used as the input parameter for the second factor. The second factor is tested by combining and/or matching their combinations as much as the value of the output parameter. In this way, useful configurations are determined from all configurations obtained.

In an embodiment of the invention, in the electronic test system, the processor applies an orthogonal array to the first factors. An orthogonal array is applied to the second factors depending on the output parameter obtained by the processor.

In an embodiment of the invention, in the electronic test system, the first factor is selected by the user as the number of virtual links.

In an embodiment of the invention, in the electronic test system, the second factors are selected by the user as the number of ports, bandwidth allocation gap, minimum frame size, maximum frame size and priority.

In an embodiment of the invention, in the electronic test system, the first power number is the number of combinations of the first factors created among themselves during a test, which is determined by the user. The second power number is the number of combinations of the second factors created among themselves during a test, which is determined by the user. The processor allows the first power number and the second power number to be different from each other.

In an embodiment of the invention, in the electronic test system, the processor allows the user to select the first power number to be larger than the second power number. This allows testing of more combinations of the first factors determined by the user to be more effective for the electronic test system.

In an embodiment of the invention, in the electronic test system, the processor enables all binary combinations between the first factors to be tested.

In an embodiment of the invention, in the electronic test system, the processor uses the Taguchi method to create orthogonal arrays. In an embodiment of the invention, in the electronic test system, in order to test the switch in deterministic avionics networks, the number of virtual links passing through each port is determined by the user as the first factor. The number of ports, bandwidth allocation gap, minimum frame size, maximum frame size and priority parameters for testing the switch, which depend on the operating performance of the virtual link, are determined as the second factor. An orthogonal array is applied to the first factors, so that a value of each output parameter obtained from all the combinations thereof is determined. The orthogonal array method is applied to the second factors as much as the value of each output parameter obtained from the first factors, so that almost all configurations are tested by the processor.

The electronic test system realized to achieve the object of the invention is illustrated in the attached drawings, in which:

Figure 1 is a block diagram of the computer network.

Figure 2 is a flow diagram of the electronic test system.

All the parts illustrated in figures are individually assigned a reference numeral and the corresponding terms of these numbers are listed below:

1. Electronic test system

2. Electronic equipment

3. Computer network

4. Switch

5. Port

6. Processor

7. Parameter

701. First factor

702. Second factor

8. Output parameter

9. First power number

10. Second power number It comprises a plurality of electronic equipment (2) that can communicate with each other; at least one computer network (3) defining Ethernet data network for the electronic equipment (2); at least one switch (4) that manages data communication in the computer network (3); at least one port (5) that provides data input and/or output between electronic equipment (2); at least one processor (6) that enables performance testing of the computer network (3).

The subject of the invention comprises a plurality of parameters (7) determined by the user for testing the computer network (3); a plurality of first factors (701) selected by the user among the parameters (7) and belonging to the port (5); a plurality of second factors (702) selected by the user among the parameters (7) and affecting the operating performance of the first factors (701); at least one output parameter (8) created by matching and/or combining combinations of the first factors (701); the processor (6) executing the steps of performing tests by matching and/or combining combinations of second factors (702) as much as the value of the output parameter (8), thus detecting useful configurations from the obtained configurations.

The computer network (3) defines the Ethernet data network for a plurality of electronic equipment (2) that communicate with each other. Data communication in the computer network (3) is provided by the switch (4). Data input and/or output between electronic equipment is provided by port (5). The performance test of the computer network (3) is carried out by the processor (6).

In order to perform the testing of the computer network (3), a plurality of parameters (7) is first determined by the user. A plurality of first factors of the port (5) is selected among the parameters (7) selected by the user. In the next step, a plurality of second factors (702) that affects the operating performance of the first factors (701) is selected by the user. After the parameters (7) are determined, the processor (6) performs a minimum number of tests by matching and/or combining the combinations of the first factors (702). Output parameters (8) are obtained from the first factors. The second factors (702) are tested as much as the value of the output parameters (8) created. The processor (6) performs a minimum number of tests by matching and/or combining the combinations of the second factors (702). In fact, the output parameter (8) created from the first factors (701) is used as the input parameter for testing the second factors. In the chained orthogonal array method, where multiple and consecutive orthogonal array applications are used, the first factors (701) that are considered to be most effective in the system performance are selected first, and the first power number (9) is selected larger than the second power number (10). Therefore, useful configurations are detected from all configurations obtained, and a debugging process is carried out.

In an embodiment of the invention, the electronic test system (1) comprises the processor (6) that enables the application of an orthogonal array to the second factors (702) depending on the output parameter (8) after the application of orthogonal array to the first factors (701). The output parameter (8) created after applying the orthogonal array method to the first factors (701) is used as the input parameter for the second factors (702). In this way, the orthogonal array method is applied in a chained manner.

In an embodiment of the invention, the electronic test system (1) comprises the first factor (701) which is several virtual links. The number of virtual links is an important element that shows the traffic usage rate of ports (5), and ultimately the features of a switch (4). While testing a deterministic network switch (4), the number of virtual links is the most critical parameter, as it directly affects the determination of traffic flows through the use of port (5).

In an embodiment of the invention, the electronic test system (1) comprises the second factor (702) which is a parameter including the number of ports (5), bandwidth allocation gap, minimum frame size, maximum frame size and priority. Bandwidth allocation gap, minimum frame size and maximum frame size determine the traffic transmission speed of the virtual link. The priority parameter directly affects the latency of the virtual link. The number of ports (5) determines the number of virtual links provided by a port (5) when all virtual links are considered, which is an important parameter that directly affects the latency in the switch (4). Therefore, these parameters are required to be tested simultaneously. In an embodiment of the invention, the electronic test system (1) comprises at least a first power number (9) determined by the user, which is the number of combinations between the first factors (701); at least a second power number (10) determined by the user, which is the number of combinations between the second factors (702); the processor (6) that allows the first power number (9) and the second power number (10) to be different from each other. Thanks to the chained orthogonal array method, in which multiple and consecutive orthogonal array applications are used, the first power number (9) and the second power number (10) can be different from each other. The chained orthogonal array method provides flexibility.

In an embodiment of the invention, the electronic test system (1) comprises the processor

(6) that enables the user to select the first power number (9) to be larger than the second power number (10), thus allowing more combinations to be tested for the first parameter

(7). When the first power number (9) is selected high by the user, different combinations using the number of virtual link parameter that is considered to be the most important, are produced more with the orthogonal array method. Such a number of virtual link parameters, which is considered to be important, will be tested more comprehensively.

In an embodiment of the invention, the electronic test system (1) comprises the processor (6) that enables testing of all possibilities in binary combinations between the first factors (701). The first power number (9) for the first factors (701) is selected by the user as 2 (pair-wise), so that all binary combinations of the first factors are tested.

In an embodiment of the invention, the electronic test system (1) comprises the processor (6) that enables the creation of orthogonal arrays using the Taguchi method. The Taguchi method reduces costs by reducing the number of tests using orthogonal arrays, as well as minimizing the effects of uncontrollable factors by means of the best combination of controllable factors.

In an embodiment of the invention, the electronic test system (1) comprises the processor (6) which enables: determining, by the user, the number of virtual links passing through each port (5) of the switch (4) in deterministic avionics networks, as the first factors (701); determining the parameters including the number of ports (5), bandwidth allocation gap, minimum frame size, maximum frame size and priority, as the second factors (702), wherein these parameters are required by the user to test the switch (4) and affect the operating performance of the virtual link; obtaining all combinations by applying the orthogonal array method to the first factors (701), and determining a value of each output parameter (8); enabling the second factors (702) to be tested as much as the value of each output parameter (8) obtained from the first factors (701), thus allowing almost all configurations of the switch (4) to be tested.

Modern avionics applications require high-speed data communications. Accordingly, ethernet-based deterministic network technologies are used. Testing ARINC 664 switches is a difficult task due to their safety-critical nature, availability of numerous configuration options, and high-speed operation using thousands of virtual links. The most important components of an ARINC 664 deterministic network include the switch (4), end systems, and virtual links. Applications running on different end systems communicate with each other using virtual links directed by the switch (4). First, different numbers of virtual link combinations that can be assigned to each switch (4) port (5) are created by the orthogonal array method. Then, in order to determine the remaining switch (4) parameters such as parameters including orthogonal array output (8), number of ports (5), bandwidth allocation gap, minimum frame size, maximum frame size and priority parameters, a number of second stages is used as an input to orthogonal array.