Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
SERDES LINK MANAGEMENT
Document Type and Number:
WIPO Patent Application WO/2017/027028
Kind Code:
A1
Abstract:
A memory device includes a serializer-deserializer (SERDES) link controller that receives a flit of data associated with a transaction across a SERDES link. The SERDES link controller forwards the flit of data to a next SERDES hop before completing a cyclical redundancy check (CRC) on a superflit. The superflit includes the flit of data. The SERDES link controller forwards the flit of data to a memory interface before completing the CRC, if a memory address of the transaction matches a memory address associated with the SERDES link controller. Additionally, the SERDES link forwards a result of the CRC to the memory interface.

Inventors:
AKKERMAN RYAN (US)
LESARTRE GREGG B (US)
REGAN JAMES DONALD (US)
Application Number:
PCT/US2015/044849
Publication Date:
February 16, 2017
Filing Date:
August 12, 2015
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
HEWLETT PACKARD ENTPR DEV LP (US)
International Classes:
G06F11/10; G06F13/16
Domestic Patent References:
WO2014158130A12014-10-02
Foreign References:
US20080168323A12008-07-10
US20110191647A12011-08-04
US7801121B12010-09-21
US20100185919A12010-07-22
Attorney, Agent or Firm:
FEBBO, Michael A. et al. (US)
Download PDF:
Claims:
CLAIMS

What is claimed is:

1 . A memory system comprising:

a serializer-deserializer (SERDES) link controller that:

receives a flit of data associated with a transaction across a SERDES link; forwards the flit of data to a next SERDES hop before completing a cyclical redundancy check (CRC) on a superflit comprising the flit of data; forwards the flit of data to a memory interface before completing the CRC, if a memory address of the transaction matches a memory address associated with the SERDES link controller; and

forwards a result of the CRC to the memory interface.

2. The memory system of claim 1 , comprising the memory interface, wherein the memory interface:

buffers a plurality of flits of data comprising the flit of data if the transaction comprises a write transaction; and

executes the write transaction if the result of the CRC is good.

3. The memory system of claim 1 , comprising a media controller that: receives a plurality of transactions from a host; and

transmits the transactions to the link controller.

4. The memory system of claim 3, wherein the media controller:

receives a response to the transmitted transaction;

determines the memory cluster is in panic mode;

clears all transactions in flight; and

re-transmits all the dropped transactions.

5. The memory system of claim 4, wherein the link controller:

generates a CRC code for the superflit; and

incorporates the CRC code into the superflit.

6. The memory system of claim 1 , comprising a second SERDES link controller that:

generates a CRC code for a response superflit based on one or more

transactions of a tail memory component;

generates a response superflit comprising the one or more transactions and the CRC code; and

transmits the response superflit to another SERDES link controller.

7. The memory system of claim 6, wherein the SERDES link controller: receives the response superflit;

determines whether the memory interface comprises transactions for adding to the response superflit;

generates a new superflit by merging the transactions with the transactions of the tail memory component based on the determination; and transmits the new superflit to a SERDES link controller of a media controller.

8. A computing system comprising:

a processor; and

a memory device comprising a media controller chip that:

receives a plurality of transactions;

generates a cyclical redundancy check (CRC) code for a superflit comprising the plurality of transactions;

incorporates the CRC code into the superflit;

transmits the superflit to memory components associated with the transactions;

receives a response to the transmitted superflit;

determines that the memory device is in panic mode;

clears all the transactions; and

re-transmits the superflit.

9. The computing system of claim 8, comprising a serializer-deserializer (SERDES) link controller that:

performs cyclical redundancy checking (CRC ) on the superflit; and forwards one of the flits before the CRC is completed.

10. The computing system of claim 9, wherein the SERDES link controller: forwards the flits to a next hop before the CRC is completed; and

forwards some of the flits based on an address to a memory interface before the CRC is completed.

1 1 . The computing system of claim 9, wherein the SERDES control block forwards a good CRC signal to the media controller when the CRC is complete and valid.

12. The computing system of claim 9, wherein the SERDES link controller sends a panic response to the media controller when the CRC check is complete and invalid.

13. The computing system of claim 8, wherein the link controller regenerates the superflit from a transaction buffer of the memory device.

14. A method for communicating across serializer-deserializer (SERDES) links, the method comprising:

generating a CRC code for a response superflit based on one or more

transactions of a memory component ;

generating a response superflit comprising the one or more transactions and the CRC code; and

transmitting the response superflit to a SERDES link controller of a second memory component.

15. The method of claim 14, comprising:

receiving the response superflit at the SERDES link controller of the second memory component;

determining whether the second memory component has transactions to add to the response superflit;

generating a new superflit by merging the transactions to add to the response superflit with the transactions of the memory component based on the determination; and

transmitting the new superflit to a link controller of a media controller.

Description:
SERDES LINK MANAGEMENT

BACKGROUND

[ΘΘ01] When a computer program is operating, the program typically performs writes to, and reads from, memory. Each of the reads and writes is a transaction that is performed in computer hardware. When requested, the transactions are directed to the correct memory locations by a media controller. In this way, reads and writes take place in the actual physical memory where they are specified.

DESCRIPTION OF THE DRAWINGS

[ΘΘ02] The advantages of the present techniques are better understood by referring to the following detailed description and the attached drawings, in which:

[0Θ03] Fig. 1 is an example media device for serializer-deserializer (SERDES) link management;

[0ΘΘ4] Fig. 2 is an example memory cluster for SERDES link management;

[0005] Fig. 3 is a process flow diagram of a method for SERDES link

management; and

[ΘΘΘ8] Fig. 4 is a process flow diagram of a method for SERDES link

management.

DETAILED DESCRIPTION

[0007] The media controller transmits the data across serializer-deserializer (SERDES) links, which are high-speed communication links between a computer processing chip and a system, such as a memory cluster. The SERDES link takes a parallel input signal, and sends one bit at a time over a differential pair, to a receiver at the other end. At the other end, the bits are turned back into a parallel signal. SERDES links are typically used because they are a more power efficient, and wire- count efficient, way to transfer data between components in a computer system. Further, SERDES links provide a useful communication mechanism for accessing a large number of memory components efficiently from a memory controller, both directly and forwarded from memory component to memory component. SERDES links, however, have a non-zero transmission failure rate. As such, in order to use these links for a memory system, transmission errors are detected and corrected.

[0008] Each packet transmitted over SERDES links undergoes cyclical redundancy checking (CRC). CRC is a validation that is run to detect transmission errors. In current systems, if there are no errors, the SERDES controller sends an acknowledgement (ACK) packet to the transmitter. If there are errors, the SERDES controller sends a no ACK (NACK) packet to the transmitter, which retries the transmission with either link-level retry or end-to-end retry. A link-level retry means the transmission is retried just from the last transmitter. End-to-end retry means the transmission is retried from the original transmitter. In both methods, a retry buffer holds data in case the data is to be re-transmitted. However, these buffers occupy valuable storage. Additionally, memory systems typically keep latency low. This makes retry mechanisms challenging to implement because both methods increase latency. Further, because each packet is stored by the receiver until the entire transaction is received, a store and forward penalty is incurred, which slows the flow of data. The additional transmissions, and the exchange of ACK-NACK packets, increase overhead and consume valuable bandwidth.

[0009] In an example memory device, instead of performing CRC for every packet before forwarding, CRC is performed in parallel with forwarding transactions to memory. If there is a CRC error, the transactions are canceled, and retransmitted. In this way, read and write transactions can be performed without a store and forward penalty. While these techniques may incur a larger latency when CRC errors occur, the frequency with which the errors occur present a long-term gain in terms of computational efficiency.

[0010] Fig. 1 is an example media device 100 for SERDES link management. The media device 100 may be a random access memory device that includes a media controller chip 1 02, and memory components 104. The media controller chip 102 includes a media controller 106 and a SERDES link controller 108-1 . The media controller 106 may be a memory controller that receives read and write transactions from a computer processor (not shown). The media controller 106 transmits the transactions to memory component 104-1 via the SERDES link controller 108-1 across SERDES links 1 1 0. [0011] The memory components 104-1 , 104-2 include link controllers 108-2 - 108-5 and interfaces to memory 1 1 2-1 , 1 12-2. The interfaces 1 12-1 , 1 12-2 are each associated with a pool of memory being read from, or written to. Actual reads and writes are forwarded to the interface to memory 1 1 2-1 , 1 12-2 to perform the transaction.

[0012] The SERDES link controller 108-1 receives transactions from the media controller 106, and combines them into a superflit. A superflit is a set of bits, or flits, of a specific size. A flit is a group of bits. The number of bits in a flit is a ratio of the frequency of the SERDES bit rate to the internal core frequency. For example, if the frequency of the SERDES bit rate is 2.66 Gigahertz (Ghz) and the internal core frequency is 0.0833 Ghz, a flit would be 32 bits. In one example, a superflit is 288 bits, or 9 flits. In such an example, 256 bits of the superflit are payload, which may be used for multiple transactions. The other 32 bits of the superflit are reserved for a CRC code used to validate the transmission.

[0013] The SERDES link controller 108-1 serializes and transmits the superflit to a SERDES link controller 108-2 at the other end of link 1 10. The controller 108-2 turns the transmission back into a parallel signal.

[0014] The first received flit of a transaction contains the address of the transaction, and the transaction type. Thus, if the address of one of the transactions in the superflit matches the address of the memory component 104-1 , the transaction is forwarded to an interface to memory 1 12-1 . It may take several flits to constitute an entire transaction, but the link controller 108-2 does not wait until the entire transaction is received to start forwarding. Rather, the link controller 108-2 automatically forwards flits of data as they arrive at the link controller 108-2. The link controller 108-2 knows, based on the transaction type in the first flit, how many subsequent flits are to be forwarded. TABLE 1 shows the number of flits in each transaction type in an example of the claimed subject matter: TABLE 1

[0Θ15] The Link Layer transaction type is used in the event of a transmission error. Transmission errors are discussed in greater detail below.

[0018] If the transaction is a write transaction, the interface to memory 1 1 2-1 places the transaction in a write buffer (not shown) until the CRC is complete. Write transactions are placed in the write buffer to avoid writing potentially bad data to memory. Accordingly, the write transactions are held in the write buffer until a good transmission is confirmed by the CRC. No buffer is used for read transactions because read transactions do not overwrite existing data.

[0017] The received flits are also automatically forwarded to the link controller 108-3 for transmission to memory component 104-2. The link controller 108-3 combines each forwarded flit into a superflit, and transmits the superflit to link controller 108-4. Link controller 108-4 performs similar techniques to those described for link controller 108-2. The flits for the transaction to be performed on memory component 104-2 are automatically forwarded to the interface to memory 1 12-2. All flits are automatically forwarded to link controller 108-5, which ignores the flits as link controller 108-5 is the last link controller and has no further memory components 104 to forward to.

[0018] The link controllers 108 also send responses back to the media controller 106. For example, a link controller 1 08-4 generates a superflit with a read response from the interface to memory 1 12-2. The read response includes the data read from memory. Write responses include an indicator whether the write was performed successfully. The superflit is padded with blank flits if the read response does not take up the full complement of bits in the superflit, and there is not another read or write response available to pack into the superflit. The link controller 108-4 also computes the CRC code, and incorporates the CRC code into the superflit. The link controller 108-4 then transmits the superflit to link controller 108-3. The link controller 108-3 automatically forwards the received flits to link controller 108-2 while performing CRC on the received superflit. If there are no transactions in the interface to memory 1 1 2-1 to add to the superflit, the superflit is forwarded to the link controller 108-1 unchanged. However, if there are transactions in the interface to memory 1 1 2-1 to add to the superflit, the link controller 108-2 buffers the

transactions in the received superflit, and merges its own transactions with the buffered transactions to create a new superflit. Specifically, the link controller 108-2 may remove blank flits from the received superflit to make room for the transactions of the interface to memory 1 12-1 . If there is not enough room in the superflit for the transactions of the interface to memory 1 12-1 , the link controller 108-2 may continue the transactions in a new superflit. The link controller 108-2 then transmits the superflit to the link controller 108-1 .

[001 ] Advantageously, read and write transactions are forwarded on to the interface to memory 1 1 2 or the next SERDES link hop, flit-by-flit immediately, instead of waiting for the complete transaction to accumulate. A hop is going over the SERDES link 1 10 to the next memory component 104. In one example, the transaction is always forwarded to the next hop since it is slightly faster to not have to decode it first. In this way, all transactions go all the way down the chain of memory components 1 04, but are only consumed at one hop.

[ΘΘ2Θ] In parallel with forwarding the transaction, CRC is performed on the data transmitted from the link controller 108-1 to link controller 108-2. As the flits come across, CRC codes are built, and compared to a CRC code stored in the superflit. If the CRC is good, the link controller 108-1 sends a good CRC signal to the interface to memory 1 12-1 . If the current transaction is a write transaction, the transaction is moved from the buffer to actual memory. If the CRC is not good, the link controller 108-2 signals a panic mode to the interface to memory 1 12-1 , the memory component 104-2, and the media controller 106. Additionally, a packet declaring the panic mode is spread along the SERDES links 1 10 so that all the memory components 104, and the media controller 1 06 are informed. In one example of the claimed subject matter, the read or write response includes a panic indicator.

[0021] In panic mode, the media device 100 clears all outstanding transactions in the memory components 104. Additionally, the interfaces to memory 1 1 2 drop all outstanding transactions. Further, the media controller 106 waits until all memory components 104 are done panicking, and then re-transmits the transaction.

[0022] CRC generation happens in the transmit portion of the link controller 108. CRC checking happens in the receive portion of the link controller 1 08. Specifically, Read and Write transactions go outbound from the media controller towards the end of the chain and are checked in the even 108 blocks, e.g., link controllers 108-2, 108-4. Further, read responses and write responses come from memory towards the media controller and are checked in the odd 108 blocks. Thus, when a superflit, with the response transactions, is transmitted back to the link controller 108-1 from link controller 108-2, the CRC checking is performed on that transmission in link controller 108-1 .

[ΘΘ23] It is noted that Figure 1 is described with respect to the execution of a single transaction. However, examples of the claimed subject matter are capable of executing multiple transactions at a time, described with reference to Fig. 2.

[ΘΘ24] Fig. 2 is an example memory cluster 200 for SERDES link management. The memory cluster 200 includes a media controller chip 202, memory components 204, a SERDES interface 208, and SERDES links 210. As discussed with respect to Figure 1 , each memory component 204 includes two SERDES interfaces (not shown), which perform CRC, and forward the superflits through the memory components 204.

[0025] The media controller 206 receives requests from a host (not shown), to perform read or write transactions to the memory components 204. The media controller 206 forwards transactions so the SERDES interface 208, which may group transactions into superflits and transmit them to the memory components 204.

Additionally, the media controller 206 includes a scheduler 214 and a transaction buffer 216. The scheduler 214 determines the sequence of execution for the transactions. The transaction buffer 21 6 is used by the media controller 206, which writes transactions to the transaction buffer 216 before transmission to the memory components. If there is an error in a given transmission, the media controller 206 panics, dropping all transactions active in the memory components 204, and retransmitting the transactions from the transaction buffer 216. In this way, dedicated retry buffers at the SERDES interface 208 can be avoided. [0028] The SERDES interface 208 may communicate with 4 chained ranks 21 2 of SERDES control blocks through 9 SERDES links 210 for a 4x9 array of memory components 204. Each chained rank 212 is considered a hop. A link controller in the SERDES interface 208 communicates with the first chained rank, which talks to the second chained rank, etc. As shown, there are 9 separate SERDES interfaces, all in parallel. In this way, a single cacheline is spread across all 9 interfaces in order to provide needed bandwidth and redundancy. The SERDES control blocks are responsible for converting slower parallel data into high speed serial bits. The chained ranks are additional SERDES hops behind the first set of memory components 204. The media controller 206 transmits multiple transactions in a superflit of data to the memory components 204 through the SERDES interface 208. [ΘΘ27] As stated previously, a superflit is a predetermined number of bits, encompassing multiple flits. In one example, a superflit is 288 bits, 256 bits of payload plus 32 bits for CRC. Advantageously, transmitting in superflits with CRC helps keep the bandwidth overhead of the CRC low while still having a large enough CRC to give good error protection. Further, the memory device 200 includes multiple SERDES links 21 0 going to different memory components 204. Each 288 bit superflit is entirely transmitted across one SERDES link 210. Thus, each of the 9 SERDES links will have its own 288 bit superflit. In the example device 200, individual READ and WRITE transactions are split across the 9 links so each 64 byte cacheline gets 8 bytes from each of 8 links 21 0. Overhead, such as error correcting code, is provided on the 9th link 210. However, in panic mode, all links 210 are panicked in order to allow the media controller 206 to be able to keep them all synchronized. In one example, a 64 byte write is transmitted as 8 bytes (64 bits) across each of 8 SERDES links 210. Each of these 8 byte chunks are part of the 256 bits of payload in 8 separate superflits, each on a separate SERDES link. Thus, if any link in a given transmission has an error, all the memory components 204 are panicked.

[0028] Fig. 3 is a process flow diagram of a method 300 for serializer-deserializer (SERDES) link management. The method 300 is performed by a media controller, and begins at block 302, where the media controller receives transactions from a computer processor. The media controller stores the received transactions in a transaction buffer. At block 304, the media controller combines the received transactions into a superflit of predetermined size. Additionally, the media controller uses a specific portion of the superflit for the CRC code. For a 256-bit payload, a CRC code may be up to 32 bits, depending on the desired redundancy. Thus, the superflit for a 256-bit payload may be 288 bits.

[0029] At block 306, the media controller transmits the superflit to the memory components over SERDES links. In an example of the claimed subject matter, the link controller at the media controller chip, e.g., link controller 108-1 , performs the transmission. At block 308, a response to the transmission is received. The response may include the data read from a READ transaction, or an indicator that a WRITE transaction was performed. Additionally, the response may indicate a panic mode. At block 310, it is determined if the response indicates a panic mode. If so, at block 312, the media controller clears all transactions in flight. The transactions in flight are all the outstanding transactions in memory when the CRC error occurs. All the transactions in flight are cleared from their respective interfaces to memory. This includes all transactions connected to the memory component with the CRC error, and memory components that are not connected with the CRC error, along the chained rank. At block 314, the media controller re-transmits all transactions that were in flight when the panic was received. The media controller then returns to block 302. In some embodiments, the order of these steps may be modified. For instance, media controller 206 may receive additional new transactions even as a CRC error is processed, and retransmission of cleared transactions may be mixed with transactions that were not in flight when the CRC panic error was encountered.

[0030] Fig. 4 is a process flow diagram of a method 400 for SERDES link management. The method 400 is performed by the link controller, and begins at block 402 where the link controller receives a flit of data over a SERDES link. At block 404, the flit is forwarded to the next SERDES hop. At block 406, the link controller determines if the memory address of the associated transaction matches the address of the memory component. If so, at block 408, the flit is forwarded to the interface to memory.

[0031] In parallel with forwarding, at block 41 0, the link controller performs CRC on the superflit. At block 41 2, it is determined if the CRC is good. If so, at block 414, the link controller sends a good CRC signal to the interface to memory. At block 416, the transaction is released for execution. If the transaction is a WRITE transaction, the transaction is released to modify memory. If the CRC is not good, at block 41 8, the link controller signals a panic mode to the interface to memory, all connected memory components, and the media controller. The link controller returns to block 402.

[0Θ32] While the present techniques may be susceptible to various modifications and alternative forms, the embodiments discussed above have been shown only by way of example. However, it should again be understood that the techniques are not intended to be limited to the particular embodiments disclosed herein. Indeed, the present techniques include all alternatives, modifications, and equivalents falling within the true spirit and scope of the appended claims.