BACKGROUND

[0001] The present invention relates to the field of digital computer systems, and more specifically, to a processor unit for performing a set of one or more multiply and accumulate (“MAC”) operations.

[0002] Computation of MAC operations is commonly used in a number of algorithms such as training of deep neural networks, image processing, and other machine learning techniques. For example, in case of a deep neural network, MAC operations may involve activations and gradients. However, conventional electronic systems for the computation of MAC operations may be resource intensive. In particular, there is a need of energy efficient and resource saving architectures for performing such operations.

SUMMARY

[0003] Various embodiments provide a processor unit for performing a set of one or more MAC operations, method, computer program product and MAC unit as described by the subject matter of the independent claims. Advantageous embodiments are described in the dependent claims. Embodiments of the present invention can be freely combined with each other if they are not mutually exclusive.

[0004] In one aspect, embodiments of the present invention discloses a processor unit for multiply and accumulate (“MAC”) operations. In one embodiment, the processor unit may include a plurality of MAC units for performing a respective subset of MAC operations of a set of MAC operations. In one embodiment, each MAC unit of the plurality of MAC units may include a respective execution unit and a respective one-write one-read (“ 1 W/1R”) register file. In one embodiment, the respective 1W/1R register file may have at least one accumulator. In one embodiment, the processor unit may include another register file. In one embodiment, the respective execution unit of each MAC unit may be configured to perform the respective subset of MAC operations of the set of MAC operations by computing a product of a set of values received from the another register file and adding the computed product to a content of the at least one accumulator of the MAC unit. In one embodiment, each MAC unit may be configured to perform the respective subset of MAC operations in a single clock cycle.

[0005] In advantageous embodiments, each MAC unit of the plurality of MAC units may include an associated index. Accordingly, each MAC unit may be configured to perform the respective subset of MAC operations by executing a processor instruction referencing the associated index.

[0006] In advantageous embodiments, the associated index may include an architected accumulator register index of the at least one accumulator of each MAC unit.

[0007] In advantageous embodiments, the at least one accumulator may include a respective accumulator element, where the computed product may be added to a content of the respective accumulator element. In advantageous embodiments, each MAC unit may be configured to perform the respective subset of MAC operations of a set of MAC operations using the respective accumulator element of the at least one accumulator.

[0008] In advantageous embodiments, the processor unit may include a dispatch/issue unit. In advantageous embodiments, the dispatch/issue unit may be configured to process a plurality of processor instructions, select a MAC unit using the associated index, and send a respective set of processor instructions to the selected MAC unit for performing the set of MAC operations. In advantageous embodiment, the dispatch/issue unit may include the MAC unit therein.

[0009] In advantageous embodiments, the respective set of processor instructions may include at least one operand. The at least one operand may indicate the at least one accumulator as a source and target register of the respective set of processor instructions. The at least one operand may also indicate at least one further register, of another register file, as including a set of numbers.

[0010] In advantageous embodiments, the dispatch/issue unit may be configured to dispatch the plurality of processor instructions in accordance with a single threaded (“ST”) mode such that the selected MAC unit may receive the respective set of processor instructions from a single thread. [0011] In advantageous embodiments, the dispatch/issue unit may be configured to dispatch the plurality of processor instructions in accordance with a two-way simultaneous multithreading (“SMT2”) mode such that the selected MAC unit may receive the respective set of processor instructions from any one of two threads.

[0012] In advantageous embodiments, the dispatch/issue unit may be configured to dispatch the plurality of processor instructions in accordance with a four-way simultaneous multithreading (“SMT4”) mode such that each MAC unit of the plurality of MAC units may receive the respective set of processor instructions from respective two threads.

[0013] In advantageous embodiments, each MAC unit may include at least one multiplier for computing the product and at least one adder for performing the addition of the computed product.

[0014] In advantageous embodiments, the processor unit may be configured to perform further sets of MAC operations, where all the sets of MAC operations provide all elements of an output matrix. In advantageous embodiments, the output matrix may include a result of a matrix convolution on an input matrix.

[0015] In one aspect, embodiments of the present invention disclose a method and a computer program product for performing MAC. Operations. The present invention may include, receiving, at a multiplier of a MAC unit, a set of values from a register file. The present invention may include calculating, using the multiplier, a product of the received set of values. The present invention may include, reading a current content of an accumulator of the MAC unit. The present invention may include, calculating, using an adder of the MAC unit, a sum of the read current content of the accumulator and the calculated product of the received set of value. The present invention may include, writing the calculated sum to the accumulator of the MAC unit.

[0016] In one aspect, embodiments of the present invention discloses a MAC unit. In one embodiment, the MAC unit may include an execution unit and a one-write one-read (“1W/1R”) ported register file. In one embodiment, the 1W/1R ported register file may include at least one accumulator. In one embodiment, the execution unit of the MAC unit may be configured to perform a MAC operation by computing a product and adding the product to a content of the at least one accumulator of the MAC unit. [0017] In advantageous embodiments, the execution unit of the MAC unit may be configured to consecutively perform a plurality of MAC operations using a same accumulator for accumulating the product of each MAC operation of the plurality of MAC operations.

[0018] In advantageous embodiments, the MAC unit may include a plurality of accumulators. In advantageous embodiment, the MAC unit may be configured to perform a plurality of MAC operations in parallel using a respective accumulator of the plurality of accumulators.

[0019] In advantageous embodiments, the at least one accumulator of the MAC unit may include a plurality of accumulator elements. In advantageous embodiments, the MAC unit may include a plurality of multipliers and an adder. In advantageous embodiments, the MAC unit may be configured to perform a plurality of MAC operations in parallel using respective accumulator elements of the plurality of accumulator elements and respective multipliers of the plurality of multipliers.

[0020] The present disclosure may provide an energy-efficient and application-specific processor for performing MAC operations. The processor unit has a small number of register file ports which may outperform architectures with a larger number of ports. In particular, the use of a register file having a single read port and single write port may enable a simplified hardware control and reduce the energy consumption. The implementation of the 1 write-port/ 1 read-port register file (“1W/1R”) may require less wiring, a smaller occupied area, and less power demand compared, for example, to a 2W/2R register file that provides two read and two write ports to connect two MAC execution units. The connectivity reduction may maintain the same application speed, reduce the energy consumption, and allow for an increase in processor frequency. In addition, the present subject matter may prevent a sharing of one register file between the MAC units. It is contemplated that providing each MAC unit with its own register file may speed up the MAC operations as each execution unit may independently and optimally be connected to its respective register file.

[0021] The present subject matter may enable to perform MAC operations as basic arithmetic operations using single instruction multiple data (“SIMD”) operands. This may improve the efficiency of a processor in that the SIMD operands can be efficiently stored in separate register files so that they can be loaded from the register files into the MAC unit in a single clock cycle.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0022] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. The various features of the drawings are not to scale as the illustrations are for clarity in facilitating one skilled in the art in understanding the invention in conjunction with the detailed description. In the drawings:

[0023] FIG. 1 is a block diagram illustrating an exemplary structure of a processor system according to at least one embodiment;

[0024] FIG. 2A is a block diagram illustrating an exemplary matrix operation of a MAC operation according to at least one embodiment;

[0025] FIG. 2B is a block diagram illustrating an accumulator for the computation of the elements of the matrix operation according to at least one embodiment;

[0026] FIG. 2C is a block diagram illustrating an exemplary MAC unit according to at least one embodiment;

[0027] FIG. 3 is a block diagram illustrating an exemplary MAC unit depicted in FIG. 1 according to at least one embodiment;

[0028] FIG. 4 is a block diagram illustrating an exemplary processor unit according to at least one embodiment;

[0029] FIG. 5 A is a block diagram illustrating an exemplary processor unit according to at least one embodiment;

[0030] FIG. 5B is a block diagram illustrating an exemplary processor unit according to at least one embodiment; [0031] FIG. 6 A is an operational flowchart illustrating an exemplary process for performing a set of one or more MAC operations according to at least one embodiment;

[0032] FIG. 6B is a block diagram illustrating an exemplary MAC unit associated with the MAC operations of FIG. 6 A according to at least one embodiment; and

[0033] FIG. 6C is a block diagram illustrating the exemplary MAC unit associated with the MAC operations of FIG. 6 A according to at least one embodiment.

DETAILED DESCRIPTION

[0034] The descriptions of the various embodiments of the present invention will be presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand.

[0035] According to one embodiment, each MAC unit of the MAC units is associated with an index, wherein the MAC unit is configured to perform the MAC operation by executing a processor instruction referencing the index. This may enable identifying a specific MAC unit to process the instruction in an optimal way by reading the index of the instruction. This may be optimal as such reading is anyhow performed in processor systems in order to identify the operands of the instruction (e.g., there may be no need for extra operations to read the index).

[0036] Each MAC unit of the MAC units may be configured to perform a respective subset of one or more MAC operations of the set of MAC operations. The subset of MAC operations, to be performed by the MAC unit, may for example compute one or more elements of a matrix C. For example, the matrix C may be defined by the following formula [1]: [0038] In the above formula [1], the subset of MAC operations may comprise the sum of products å _k -A _ki B _jk , for a given value of k. For example, if A, B and C are 4x4 matrices, a subset of MAC operations may comprise 4 MAC operations A11*B11, A21*B12, A31*B13 and A41*B14 for obtaining element Cl l. Each group of one or more MAC operations that can be performed, by the MAC unit, in one clock cycle, may be performed by executing a processor instruction. The processor instructions may for example be executed in parallel by the MAC units. For example, the set of MAC operations comprises N MAC operations to be performed by one MAC unit and other N MAC operations to be performed by the other MAC unit. The two MAC units may be configured to consecutively perform each pair of MAC operations in parallel (e.g., if N=3, 3 pairs of MAC operations may consecutively be performed in parallel). This may particularly be advantageous if each MAC unit comprises a single accumulator that is used to store one element. The accumulator may be a register. In another example, the accumulator may comprise multiple fields or accumulator elements (e.g., m elements), wherein each accumulator element may be used to accumulate results of respective MAC operations. In this case, the MAC unit may be configured to perform m MAC operations in parallel in one clock cycle.

[0039] If more than one accumulator exists in each MAC unit, a pipelined execution of the instructions may be advantageously used. According to one embodiment, each MAC unit of the MAC units is configured to perform the respective MAC operation in at least one clock cycle (e.g., in multiple clock cycles). The MAC operation execution may for example comprise four clock cycles to generate the result of the processor instruction. The processor instruction may comprise four steps, wherein each step contributes to obtain the overall result of the processor instruction being processed. In every clock cycle, a new processor instruction may be started using a different accumulator register. As soon as the result of the instruction is available, after 4 cycles, the accumulator used by the first instruction can be used again by a subsequent instruction. According to one embodiment, the processor instructions are decoded in one clock cycle.

[0040] According to one embodiment, the index is an architected accumulator register index of the accumulator of the MAC unit. This may save processing resources because the same index is used to index both the accumulator and the MAC unit that comprises the accumulator. This is by contrast to another implementation that would require separate indexing of the MAC unit and the accumulators. [0041] According to one embodiment, the processor unit further comprises a dispatch and/or issue unit. The dispatch and/or issue unit is configured to process multiple processor instructions, select MAC units using the index and send the processor instructions to the selected MAC units for performing the set of MAC operations. This may enable a seamless integration of the present subject matter in existing systems, by making minimal changes to the existing dispatch and/or issue units.

[0042] According to one embodiment, each processor instruction comprises operands indicating the accumulator as a source and target register of the instruction and indicating further registers, of the other register file, comprising source data to be multiplied and accumulated. The source data comprises the numbers. Using the accumulator as a source and target register enables a resource saving access to data. This is by contrast to a MAC unit using two separate accumulators, one for writing and one for reading data, which may introduce extra latencies and additional source-target dependency tracking logic. Due to the reduced connectivity in a processor unit using a single accumulator (e.g., by adequately placing a single accumulator near the execution unit), the single accumulator can be accessed multiple times faster than accessing separate accumulators.

[0043] According to one embodiment, the MAC units are part of the dispatch and/or issue unit. This may save area on a chip and power compared to a separate implementation of the MAC units. For example, in case the MAC units are implemented as separate components of the dispatch and/or issue unit, they may occupy more space and may use extra power sources.

[0044] According to one embodiment, the dispatch and/or issue unit is configured to dispatch the multiple processor instructions in accordance with a single threaded (“ST”) mode so that the MAC units receive processor instructions of a single thread.

[0045] For example, the processor unit may operate in a ST mode, so that only MAC operations of one thread may be performed or handled at a time.

[0046] According to one embodiment, the dispatch and/or issue unit being configured to dispatch the multiple processor instructions in accordance with a two-way simultaneous multithreading (“SMT2”) mode so that the MAC units receive processor instructions of any one of two threads. [0047] According to one embodiment, the dispatch and/or issue unit is configured to dispatch the multiple processor instructions in accordance with a four-way simultaneous multithreading (“SMT4”) mode so that each MAC unit of the MAC units receives processor instructions from respective two threads.

[0048] These embodiments may enable a seamless integration of the present subject matter with existing thread enabling systems.

[0049] According to one embodiment, the processor unit is configured to perform further sets of MAC operations, wherein all the sets of MAC operations provide all elements of an output matrix. The output matrix is a result of a matrix convolution on an input matrix. For example, the input matrix may represent pixels of an image. A kernel may be used to perform a convolution with the pixel data stored in the input matrix. The kernel does, for example, edge detection, and operates on a sub-matrix of the input matrix of the same size as the kernel. The output matrix is the result of the convolution of the sub-matrix matching the kernel size at an input matrix position.

[0050] Referring to FIG. 1, a block diagram illustrating a structure of a processor system 101 in accordance with one embodiment is depicted.

[0051] The processor system 101 may enable storing data for performing operations on the stored data. According to one embodiment, one or more levels of a cache 103 of the processor system 101 may be employed to buffer memory data in order to improve processor performance. The cache 103 may include a high speed buffer holding cache lines of memory data that are likely to be used. For example, typical cache lines may include 64, 128, or 256 bytes of memory data. According to one embodiment, the cache 103 may be configured to cache data of higher hierarchical storage such as a main memory storage 105.

[0052] According to one embodiment, the processor system 101 may further include a register file 119 comprising registers 120a-n. The registers 120a-n may, for example, include general-purpose registers that each includes a certain number of bits to store data items processed by instructions executed in the processor system 101.

[0053] According to one embodiment, the instructions may be provided by a compiler. For example, a source code of a program may be compiled into a series of machine- executable instructions defined in an instruction set architecture (“ISA”) associated with the processor system 101. The ISA may include at least one processor instruction defined in accordance with at least one embodiment of the present disclosure. The instructions of the ISA may be provided to process data stored in memory 105 and/or in the registers 120a-n of the register file 119 of the processor system 101. According to one embodiment, when the processor system 101 starts to execute the executable instructions, these machine-executable instructions may be placed in order to be executed sequentially.

[0054] According to one embodiment, instruction fetch/decode unit 109 may be employed to fetch placed instructions. For example, the instruction fetch/decode unit 109 may fetch the next sequential instructions, the target instructions of branch taken instructions, or the first instructions of a program following a context switch. The fetched instructions may be decoded by the instruction fetch/decode unit 109.

[0055] According to one embodiment, the decoded instruction(s) may be passed to a dispatch/issue unit 111. The dispatch/issue unit 111 may deposit the instructions in one or more issue queues 112, where the instructions may wait for their source operands and an appropriate MAC unit 113 of the processor system 101 to become available. For example, each MAC unit 113 of the MAC units 113 may be associated with an issue queue 112 such that instructions to be executed by the MAC unit 113 may be queued in the issue queue 112 associated with the MAC unit 113. The dispatch/issue unit 111 may deposit the instructions in an issue queue 112 based on an index that is assigned to the MAC unit 113 of the issue queue. This index may be part of the instructions. That is, by reading the index from an instruction, the dispatch/issue unit 111 may know (e.g., determine) which MAC unit 113 can execute the instruction. Each MAC unit 113 of the MAC units 113 may be configured to execute one respective type of instruction which may not be executed by other MAC units 113. In one embodiment, a type of an instruction may be defined by the registers associated with the instruction.

[0056] In conventional methods, every execution unit may be implemented to execute every instruction (e.g., any instruction can go to any execution unit). However, in contrast to the operation of the present disclosure, the conventional methods may be more expensive in term of processing resources.

[0057] According to one embodiment, each issue queue 112 may include a window logic which may be responsible for monitoring dependencies between instructions in the queue and issuing instructions to the respective MAC unit 113. The window logic may include a wakeup logic and a select logic. The wakeup logic may be responsible for invoking instructions waiting in the issue queue 112 for the source operands to become available. Once all the source operands of an instruction are available, the instruction may be flagged ready for execution. The select logic may be responsible for selecting instructions for execution from the pool of ready instructions. An instruction may be ready if all of its source operands are available. Then the operand values of the instruction may be loaded from the memory 105 and/or registers using a load/store unit 117.

[0058] A MAC unit 113 may typically receive information about instructions from the dispatch/issue unit 111 and may perform operations on operands according to the operation code of the instruction. Operands are provided to the MAC unit 113 from registers in accordance with the present disclosure. Results of the execution, when stored, may be stored either in memory 105 and/or registers 119. According to one embodiment, each MAC unit 113 of the MAC units 113 may include at least one multiplier and at least one adder. If the MAC unit 113 includes more than one multiplier, each multiplier may be associated with a respective accumulator element of the accumulator of the MAC unit 113. Each multiplier of the MAC unit 113 may be configured to perform the product of two numbers and the adder may be configured to add the product to the content of an accumulator associated with the multiplier or to a content of an accumulator element associated with the multiplier. The MAC unit 113 may be configured to perform a MAC operation in at least one clock cycle. In one example, the MAC unit 113 may include an X number of multipliers that may enable the MAC unit 113 to perform an X number of MAC operations in at least one clock cycle.

[0059] According to one embodiment, a program counter (instruction counter) 107 may keep track of the address of the current instruction to be executed. For example, a program counter 107 in processor, such as, a z/ Architecture® (z/ Architecture and all IBM — based trademarks and logos are trademarks or registered trademarks of International Business Machines Corporation and/or its affiliates) processor may include 64 bits and may be truncated to 31 or 24 bits to support prior addressing limits. A program counter 107 may typically be embodied in a program status word (“PSW”) of a computer such that it persists during context switching. In one embodiment, the program counter may be incremented by an amount equal to the number of bytes of the current instruction. [0060] The processor system 101 may be configured to perform a pipelined execution of the instructions. For example, a four-cycle latency pipelined execution may be used by each MAC unit 113 of the processor system 101 such that each instruction may take the MAC unit 113 four clock cycles to produce the result of the MAC operation. For that, the execution of the instruction at the MAC unit 113 may be broken down into four processes, where one of the four processes may include the multiplication process and another of the four processes may include the accumulation process by the MAC unit.

[0061] According to one embodiment, the processor system 101 may be configured to operate in an ST mode, an SMT2 mode, or an SMT4 mode. In ST mode, MAC operations from one thread may be performed or handled at a time by the MAC units 113. For example, in the ST mode, each of the MAC units 113 may be configured so that the MAC units 113 receive instructions from one specific thread (e.g., TO). In SMT2 mode, the MAC units 113 may receive instructions from any one of two threads. For example, in the SMT2 mode, each of the MAC units 113 may be configured so that the MAC units 113 receive instructions from thread TO and/or thread T1. In SMT4 mode, each MAC unit 113 of the MAC units 113 may receive processor instructions from all four threads (e.g., TO, Tl, T2 and T3) or the MAC unit 113 may operate in a split mode to receive two threads. For example, in split mode, if the processor system 101 includes two MAC units 113, one MAC unit 113 may be configured to process instructions from thread TO and/or thread T2 and the other MAC 113 unit may be configured to process instructions from thread Tl and/or thread T3.

[0062] Thus, the processor system 101 may enable processing instructions from one or more programs using MAC units 113. For example, the source code of a program may implement one or more MAC operations. A MAC operation may include multiplying two numbers and adding the result of the multiplication to an accumulated value. At least one MAC operation may be performed by the MAC unit 113 by executing a respective processor instruction of the ISA.

[0063] Referring now to FIG. 2 A, a block diagram illustrating an exemplary matrix operation of a MAC operation according to at least one embodiment is depicted.

[0064] According to one embodiment, the matrix operation of FIG. 2 A may be referred to as a matrix multiply and accumulate (“MMA”) operation. The matrix operation may multiply an input matrix 201 by an input matrix 203, and the accumulated result may include a resulting matrix 205. If matrix 201 includes an M-by-K matrix and matrix 203 includes a K-by-N matrix, then the resulting matrix 205 may include an M-by-N matrix. Thus, each element of the resulting matrix 205 may be obtained by K MAC operations. As shown in FIG. 2 A, the input matrix 201 may include a 4x4 matrix and the input matrix 203 may include a 4x4 matrix. According to one embodiment, the matrix operations may not be limited to the dimensions illustrated in FIG. 2A, which are provided as examples.

[0065] One or more accumulators may be used for computing the MAC operations and storing elements of the resulting matrix 205. An accumulator may have an accumulator size, e.g., 512 bits, indicating the size of the accumulator. An accumulator may include multiple accumulator elements or fields. An accumulator element may have an accumulator element size which may be smaller than the accumulator size, e.g., an accumulator element size may be 32 bits. Each accumulator element of the accumulator may have or occupy respective number of bits of the accumulator, e.g., a first accumulator element may occupy bits 0 to 31 of the accumulator, the second accumulator element may occupy bits 32 to 63 and the sixteenth accumulator element may occupy the bits 480 to 511. In one embodiment, the accumulator element size may be defined as the size of an element of the resulting matrix 205. This may be advantageous as the accumulator may be configured to include as many accumulator elements as the number of elements in the resulting matrix 205. This may enable storing all elements of the resulting matrix 205 in one accumulator.

[0066] Referring now to FIG. 2B, a block diagram illustrating an accumulator for the computation of the elements of the matrix operation according to at least one embodiment is depicted. Referring additional to FIG. 2C, a block diagram illustrating an exemplary MAC unit according to at least one embodiment is depicted.

[0067] According to one embodiment, FIGS. 2B-2C indicate that the elements of the resulting matrix 205 may be computed using one MAC unit having one accumulator 223, one adder 222, and 16 multipliers 221. As shown in FIG. 2B, each element of the resulting matrix 205 may be stored in a respective accumulator element of a single accumulator ACC(0). An accumulator element of the accumulator ACC(0) may be referred to as ACC'(O) in FIG. 2B, where j=l, ...,16. Element Cll of the resulting matrix 205 may be stored in an accumulator element ACC'(O) having bits 0 to 31 of the accumulator ACC(0), C21 may be stored in an accumulator element ACC ² (0) having bits 32 to 63 of the accumulator ACC(0), C31 may be stored in an accumulator element ACC ³ (0) having bits 64 to 95 of the accumulator ACC(O), C41 may be stored in an accumulator element ACC ⁴ (0) having bits 96 to 127 of the accumulator ACC(O) and so on. Accordingly, the storage of 16 times a 32 bit element may result in using the full width of the 512 bit of the accumulator 223.

[0068] Before computing the elements of the resulting matrix 205, each accumulator element ACC'(O) of the accumulator ACC(O) may be initialized with (or store) an initial value. The initial value may, for example, be zero. In another example, the element ACC'(O) may be initialized by other values such as a pre-computed portion of a matrix element (e.g., for obtaining Cl l, the accumulator element ACC ¹ (0) may be initialized with the value of A11 *B 11). In another example, the initialization may be performed by a first instruction that may differ from the MAC operation instruction, such that the multiplication result may be stored in the accumulator element of the accumulator ACC(0) without using the accumulator ACC(0) as an input. The example in FIG. 2B uses the K dimension of matrix 201 as an outmost loop of programing loops used for computing an outer product. The MMA operation may load a column of matrix 201 and a row of matrix 203 once as operands from the other register file, in order to compute its outer product, and accumulate the result of this outer product as an element in the result matrix 205. Afterward, this column of matrix 201 and row of matrix 203 may not be used again.

[0069] For example, for obtaining each element of the 16 elements Cll to C44 of the result matrix 205, four MAC operations may be performed using a respective accumulator element. As indicated in FIG. 2B, in order to obtain the element Cll, the MAC operations 210.1, 211.1, 212.1, and 213.1 may be performed using the content of the accumulator element ACC^O).

[0070] Knowing the structure of the resulting matrix and the nature of the MMA operation, the MAC units may be configured to perform the MMA operation in an optimal way. For example, the MMA operation may be split into sets of MAC operations so that each set of MAC operations may be performed in one clock cycle by a MAC unit. Following the example of FIG. 2A and as shown in FIG. 2B, the matrix operation may involve four sets of MAC operations 210.1-16, 211.1-16, 212-.1-16, and 213.1-16 that each can be processed in one clock cycle by the MAC unit of FIG. 2C. Each MAC operation involves the multiplication and the addition of the result of the multiplication to the content of a respective accumulator element. The execution of each set of the four sets of MAC operations may be performed by issuing a respective instruction, such as, for example, an integer-based GER instruction. The accumulator element used by each MAC operation of a set of MAC operations may be indicated in the GER instruction. For example, accumulator element used by each MAC operation of a set of MAC operations may be determined by the GER instruction type (e.g., the GER instruction may have different types, wherein each type may be used for different numbers of MAC operations). The GER instruction name determines the number of accumulator elements and data types in the operands as well as the 512 bit accumulator.

[0071] In the first MAC operation of the set of 16 MAC operations, the product A11*B11 may be performed and the result of the product may be added to the content of the first accumulator element ACC^O) having bits 0 to 31 of the accumulator ACC(O) which may be zero. Thus, after the first MAC operation, the first accumulator element ACC'(O) may contain the product All *B11. In the second MAC operation of the set of 16 MAC operations, the product A11*B21 may be performed and the result of the product may be added to the content of the second accumulator element ACC ² (0) of the accumulator ACC(O) which may be zero. Thus, after performing the set of 16 MAC operations 210.1-210.16, each element ACC'(O) of the accumulator ACC(0) may include an intermediate value of a respective element of the resulting matrix as indicated in FIG. 2B. After completing the set of 16 MAC operations 210.1-210.16, another set of 16 MAC operations 211.1-211.16 may be performed by the MAC unit. This repetition may be performed until all sets of MAC operations are performed. FIG. 2B shows the evolution of the content of each accumulator element ACC'(O) of the accumulator ACC(0) that results from a non-pipelined execution of the sets of MAC operations.

[0072] However, for performing multiple matrix operations (e.g., similar to the matrix operation of FIG. 2A), a pipelined execution of the MAC operations may be advantageous. For example, in case of four matrix operations, where each matrix operation includes four sets of MAC operations as described with the matrix operation of FIG. 2 A, the execution of the 16 sets of MAC operations may be performed as follows.

[0073] In the case of a four cycle pipelined execution, each MAC operation may be performed in four steps. Thus, each set of the 16 sets of MAC operations may be performed in four clock cycles. For example, the first step of each MAC operation of the set of MAC operations may be performed by a MAC unit in one, same, clock cycle (e.g., all first steps of the set of MAC operations may be performed in one, same, clock cycle). The second step of each MAC operation of the set of MAC operations may be performed by the MAC unit in another, same, clock cycle. The third step of each MAC operation of the set of MAC operations may be performed by the MAC unit in another, same, clock cycle. The fourth step of each MAC operation of the set of MAC operations may be performed by the MAC unit in another, same, clock cycle. The pipelined execution enables performing different steps of different sets of MAC operations during a same clock cycle.

[0074] The elements of each resulting matrix of the four resulting matrices of the four matrix operations may be stored in respective accumulators ACC(O), ACC(l), ACC(2), and ACC(3).

[0075] The example provided in FIGS. 2A-C illustrates the MAC computation for 4 x 32bit elements as input sources: SRC1 and SRC2. However, it is contemplated that other element widths, such as, for example, 8 x 16bit or 16 x 8bit, may be handled in a similar way by multiplying and adding two pairs of 16bit in the 32bit wide accumulator elements or doing the MAC operation on four pairs of 8bit to calculate the 32bit wide accumulator fields during the four cycles execution. In a similar manner, the accumulator element size may be less or more as 32bit as well as the number of elements stored into the ACC registers. FIG. 2C illustrates the details of the 4x4 implementation with a 4x32bit operand A (e.g., representing one column) and a 4x32bit operand B (e.g., representing one row) as inputs and the 512bit accumulator illustrated at the output storing the 16 32bit results. FIG. 2B illustrates the operation by an outer product computation algorithm. However, the matrix operation may not be limited to these outer product computation algorithms. Other algorithms including the inner product or dot product algorithms may be used.

[0076] Referring now to FIG. 3, a block diagram illustrating a MAC unit according to at least one embodiment is depicted. According to one embodiment, the MAC unit of FIG. 3 may depict the MAC unit 113.

[0077] According to one embodiment, the MAC unit 113 may include an execution unit 301 and a register file 303. The execution unit 301 may read accumulator data (ACC(i) data) from the register file 303 and may write accumulator data into the register file 303. The register file 303 may include one or more accumulators. In one embodiment, each accumulator of the register file 303 may include an architected accumulator register index i so that each accumulator may be referred to as ACC(i). The index may be an integer number which may include an even or odd number. The architected accumulator ACC(i) may be renamed, such that the register file entry storing the accumulator ACC(i) differs from the register file entry number. The register file entry number containing the state of the accumulator ACC(i) may be tracked by the logic of the dispatch/issue unit 111.

[0078] The register file 303 may include a 1W/1R ported register file. The register file 303 may be advantageous as it may require less wiring, less area, and less power demand compared for example to 2W/2R register files.

[0079] The MAC unit 113 may be configured to receive a processor instruction from a respective issue queue of the dispatch/issue unit 111 in order to perform a MAC operation. The processor instruction may, for example, be named “GER” and may be defined as follows: GER ACC(i) G ACC(i), vector scalar register (“VSR”)(j), VSR(k). The processor instruction GER may include operands indicating the accumulator ACC(i) as a source and target register of the processor instruction and indicating further source registers VSR(j), VSR(k) of the register file 119. The source registers VSR(j), VSR(k) may include the numbers to be multiplied. As shown in FIG. 3, the execution unit 301 may read the values/numbers to be multiplied 304 and 305 from the register file 119 and read the content (referred to as SRC0) of the accumulator ACC(i) from the register file 303 in order to perform a MAC operation. The result (referred to as TGT) of the MAC operation may be stored in the accumulator ACC(i). Thus, each accumulator ACC(i) of the register file 303 may enable the MAC unit 113 to perform a respective set of one or more MAC operations so that resulting values of the set of MAC operations may be accumulated in the accumulator ACC(i).

[0080] Following the example of FIG. 2A, the MAC unit 113 may be configured to perform the four sets of 16 MAC operations for obtaining elements C11...C44 of the resulting matrix 205 using the accumulator ACC(0). For that, the execution unit 301 may include multiple multipliers and the accumulator ACC(0) may include multiple elements as described with reference to FIG. 2C.

[0081] In order to compute the overall resulting matrix 205, a stream of four processor instructions each performing a set of 16 MAC operations for obtaining C11...C44 may be received at the MAC unit. Each processor instruction of the stream references accumulator ACC(0) of the MAC unit, and which accumulator element of the accumulator ACC(0) to receive which result of a multiplication. For each processor instruction of the stream, the MAC unit may perform 16 MAC operations and accumulate the results to the respective content of the accumulator element referenced in the processor instruction. For example, a processor instruction may compute among other products, the product A11 *B 11 and add the product to the respective accumulator element ACC'(O) followed by another instruction to compute at least A21 *B 12 and add the product to a respective accumulator etc.

[0082] The execution step of each MAC operation may be performed by the MAC unit 113 in one clock cycle or in multiple clock cycles by a pipelined implementation.

[0083] Referring now to FIG. 4, a block diagram illustrating a processor unit 400 according to at least one embodiment is depicted.

[0084] The processor unit 400 may include two MAC units 413A and 413B. The MAC unit 413Amay include an execution unit 401 A and a register file 403 A. The execution unit 401A may read data from the register file 403A and may write data into the register file 403 A. The register file 403 A may include one or more accumulators. Each accumulator of the register file 403 A may include an architected accumulator register index odd so that each accumulator may be referred to as ACC(odd). The index odd may be an odd number. The MAC unit 413B may include an execution unit 401B and a register file 403B. The execution unit 40 IB may read data from the register file 403B and may write data into the register file 403B. The register file 403B may comprise one or more accumulators. Each accumulator of the register file 403B may include an architected accumulator register index even so that each accumulator may be referred to as ACC(even). The index even may be an even number. Each register file of the register files 403A and 403B may be a 1W/1R ported register file. Each of the MAC units 413 A-B may be configured to operate as describe with reference to the MAC unit 113 of FIG. 3.

[0085] In one embodiment, the processor unit 400 may further include a register file 419 which may provide the numbers to be multiplied by each of the MAC units 413 A and 413B.

[0086] According to one embodiment, each of the MAC units may be associated with a respective index. For example, the MAC unit 413 A may be associated with an odd index and MAC unit 413B may be associated with an even index. Each MAC unit may receive its own instructions from the respective issue queue (e.g., the dispatch and/or issue unit that issues instructions to the processor unit 400 may comprise two issue queues, one issue queue for odd indexed instructions and another issue queue for even indexed instructions). In another embodiment, the dispatch and/or issue unit may comprise a single issue queue with at least two issue ports. In this embodiment, one issue port may be configured to issue even indexed instructions to the even MAC unit 413B and the other issue port may be configured to issue the odd indexed instructions to the odd MAC unit 413 A. However, the present disclosure is not limited to such implementations.

[0087] For example, the MAC unit 413A may comprise four accumulators ACC(l), ACC(3), ACC(5), and ACC(7) and the MAC unit 413B may comprise four accumulators ACC(0), ACC(2), ACC(4), and ACC(6). Each of the MAC units 413 A-B may receive a respective processor instruction as described with reference to FIG. 3. The processor instruction may reference an index that corresponds to the MAC unit which receives instruction. Accordingly, the processor unit 400 may enable performing two processor instructions in parallel by the MAC units 413A-B.

[0088] The processor unit 400 may be configured to perform a pipelined execution of the instructions. Assuming for example, that eight processor instructions are received at the processor unit 400 (e.g., as a stream of instructions, GER(0), GER(2), GER(3), GER(l), GER(5), GER(6), GER(7), and GER(4) whereby GER(i) with reference to FIG. 4 uses accumulator ACC(i) as a source SRC0 and a target TGT and the latency to execute a GER(i) instruction is four cycles). Each instruction GER has an even or odd index. In this case, the instructions GER(0), GER(2), GER(4) and GER(6) may be in the issue queue associated with MAC unit 413B and instructions GER(l), GER(3), GER(5) and GER(7) may be in the issue queue associated with MAC unit 413 A. If there is one single accumulator that can comprise only one element, GER(i) may enable performing one step of the four steps of one MAC operation. If there are m accumulator elements of one accumulator that can comprise m elements, GER(i) may enable performing one same step of each MAC operation of m MAC operations. Following the latency of four pipelined execution example, the stream of instructions may be performed cycle by cycle as follows:

[0089] Cycle c: GER(0), GER(3)

[0090] c+1: GER(2), GER(1)

[0091] c+2: GER(4), GER(5) [0092] c+3: GER(6), GER(7)

[0093] c+4: GER(O), GER(3)...

[0094] During the clock cycle c, the instructions GER(O) and GER(3) are issued to and respectively next executed by the MAC units 413B and 413 A. In the following cycle c+1, the instructions GER(2) and GER(l) are issued to and respectively next executed by the MAC units 413B and 413 A. In the following cycle c+2, the instructions GER(4) and GER(5) are issued to and respectively next executed by the MAC units 413B and 413 A. In the following cycle c+3, the instructions GER(6) and GER(7) are issued to respectively next executed by the MAC units 413B and 413 A. The processor unit 400 may enable a throughput comparable or better than existing systems. In cycle c+4 the GER(0) and GER(3) instructions execution have finished hence a new GER(0) and GER(3) instruction may be issued in the c+4 cycle to proceed the outer product calculation for the accumulators ACC(0) and ACC(3). GER(0) in cycle c may perform a first step of the MAC operation and GER(0) in cycle c+4 may perform a second step of that same MAC operation.

[0095] In another example, the stream of instructions may comprise only even indexed instructions GER(0), GER(2), GER(4) and GER(6). That is, only one issue queue comprises instructions to be executed. In this case, only one MAC unit may be used to perform the stream of instructions as follows:

[0096] Cycle cc: GER(0), none

[0097] cc+1: GER(2), none

[0098] cc+2: GER(4), none

[0099] cc+3: GER(6), none

[00100] During the clock cycle cc, the instruction GER(0) is issued to and next executed by the MAC unit 413B, while the MAC unit 413 A is not executing any instruction. In the following cycle cc+1, the instruction GER(2) is issued to and next executed by the MAC units 413B while the MAC unit 413Ais not executing any instruction. In the following cycle cc+2, the instruction GER(6) is issued to and next executed by the MAC unit 413B while the MAC unit 413A is not executing any instruction. In the following cycle cc+3, the instruction GER(6) is issued to and next executed by the MAC units 413B while the MAC unit 413 A is not executing any instruction. The overall throughput may not be impacted by not using the 410A execution unit as with 4 ACCs in use the overall throughput may not be higher as 4 instructions given the 4 cycle latency that an instruction takes to finish the ACC result calculation.

[00101] Referring now to FIG. 5 A, a block diagram illustrating a dispatch logic of a processor unit 500 according to at least one embodiment is depicted. Referring also to FIG. 5B, a block diagram illustrating an issue logic of the processor unit 500 according to at least one embodiment is depicted. The processor unit 500 may include a dispatch/issue unit 511 and MAC units 413 A and 413B as described with reference to FIG. 4.

[00102] As indicated in FIG. 5 A, the dispatch/issue unit 511 may be configured to receive a fetched and decoded instruction. The received instruction may reference an accumulator ACC(i) with a given index i. The dispatch and/or issue unit 511 comprises a logic circuitry 514. The logic circuitry 514 may be configured to determine, based on the index i, which of issue queues (“ISQs”) 412A and 412B of the dispatch/issue unit 511 would receive the instruction. Alternatively, as shown in FIG. 5B, the processor unit 500 may include a common issue queue port of the logic circuitry 514 and the combined issue queues 412A-B issue logic may be configured to issue respective instructions into an even or odd port of the MAC units 413 A-B based on the index i of the accumulator ACC(i) in use.

[00103] Referring now to FIG. 6A, an operational flowchart illustrating an exemplary process for performing a set of one or more MAC operations according to at least one embodiment is depicted. Referring additionally to FIGS. 6B-6C, a block diagram illustrating the MAC unit associated with the MAC operations of FIG. 6A according to at least one embodiment is depicted.

[00104] For simplicity of the following description, the set of MAC operations in FIGS. 6A-6C may comprise (but is not limited to) the MAC operations for determining a matrix element equal to A11*B11+A21*B12.

[00105] For enabling the execution of the set of MAC operations, a stream of two instructions may be generated, where each of the instructions may enable performing a MAC operation of the set of MAC operations. [00106] At 601, an accumulator is initialized. According to one embodiment, an accumulator of a MAC unit 613 may be initialized to zero. The accumulator may be part of a 1W/1R register file.

[00107] Then at 602, an instruction is received. According to one embodiment, a first instruction of the stream may be received at 602. The first instruction may reference two VSR registers as source registers. The first instruction further may reference the accumulator as a source and target register of the first instruction. The first instruction may for example be defined as follows using the GER instruction: GER ACC(i) G ACC(i), VSR(j), VSR(k). As indicated in FIG. 6B, the VSR registers VSR(j) and VSR(k) comprise the values to be multiplied All and Bll and the accumulator stores the initial value. The values All and B11 may, for example, be stored in the VSR registers of the register file 119 by the load/store unit 117 for enabling the execution of the first instruction.

[00108] Then at 603, the values are multiplied. According to one embodiment, the values Al l and Bl l may be received at a multiplier of the MAC unit 613. The multiplier may multiply the values A11 and B 11 at 603 and provide the result of the multiplication A11 *B 11 to an adder of the MAC unit 613.

[00109] Then at 604, the result of the multiplication is added to an accumulator. According to one embodiment, the adder may read a current content of the accumulator ACC which may be zero as indicated in FIG. 6B. At 604, the adder may add the content of the accumulator ACC to the result of the multiplication Al l and Bll.

[00110] Then at 605, the result is written. According to one embodiment, the result of the addition (e.g., A11*B11) may be written to the accumulator at 605. This may result in the first instruction being executed and completed.

[00111] A next instruction of the stream may be processed by repeating 602 to 605, as described above, for the second instruction of the stream. FIG. 6C illustrates the status of the MAC unit 613 for the execution of the second instruction. The VSR registers VSR(j) and VSR(k) may include the values to be multiplied A21 and B 12 and the accumulator may store the value A11*B11. The second instruction may for example have a same syntax as the first instruction. 602-605 may be repeated until all instructions of the stream are processed. [00112] Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

[00113] The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

[00114] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non- exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[00115] Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.

[00116] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the "C" programming language, Python programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

[00117] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

[00118] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

[00119] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be accomplished as one step, executed concurrently, substantially concurrently, in a partially or wholly temporally overlapping manner, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.