Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
LOW DELAY, ERROR RESILIENT VIDEO TRANSPORT PROTOCOL OVER PUBLIC IP TRANSIT
Document Type and Number:
WIPO Patent Application WO/2018/109500
Kind Code:
A1
Abstract:
The invention is directed to data transport between a transmitter and a receiver, particularly for the transport of high-quality video. The methods and systems of the invention encompasses the novel application of forward error correction (FEC) methods to improve the effectiveness of automatic repeat request (ARQ) protocols. ARQ relies on communication from the receiver back to the transmitter in order to fulfill missing or damaged packets. In the novel methods of the invention, these communications from the receiver back to the transmitter are encoded with FEC methods, greatly improving the efficiency of the ARQ protocol and reducing the number of re-transmission requests and re-transmissions, significantly reducing latency. When combined with FEC coding of the video data in the forward direction, the novel Two-Way FEC is highly effective, enabling the transit of high-quality video over lossy public networks.

Inventors:
CARPENÈ ALBERTO BORATTO (GB)
KOLODZIEJ PAWEL (PL)
Application Number:
PCT/GB2017/053772
Publication Date:
June 21, 2018
Filing Date:
December 15, 2017
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
IN ARIA! LTD (GB)
International Classes:
H04L1/18; H04L1/00; H04L29/06; H04L1/12
Foreign References:
US6104712A2000-08-15
US20110243066A12011-10-06
US20030206524A12003-11-06
US20150223237A12015-08-06
Other References:
None
Attorney, Agent or Firm:
APPLEYARD LEES IP LLP (GB)
Download PDF:
Claims:
CLAIMS

What is claimed is:

Claim 1. A method for the transmission of data, the data being transmitted by a transmitter to a receiver over a network: wherein the transmitter and receiver comprise devices capable of sending and receiving data packets over and from the network; wherein, at the transmitter, the data is formatted as a series of packets, wherein the packets comprise data from the source file, packet sequence identifiers, and addressing data; wherein the transmitter and receiver are configured to perform an ARQ protocol wherein ACK and/or NACK messages are transmitted from the receiver to the transmitter to facilitate the fulfillment of missing packets and wherein missing packets are re-transmitted by the transmitter to the receiver; and wherein the ACK and/or NACK messages transmitted from the receiver to the transmitter are encoded by a forward error correction method.

Claim 2. The method of Claim 1, wherein the data comprises voice, audio, or video data.

Claim 3. The method of Claim 2, wherein the data comprises video data selected from the group consisting of MPEG2-TS video data, HD video data, SD video data, and 4K-UHD resolution video data.

Claim 4. The method of Claim 1, wherein the data transmitted from the transmitter to the receiver is encrypted.

Claim 5. The method of Claim 4, wherein wherein the encryption is by an encryption standard selected from the group consisting of AES 128, AES 192, and AES 256.

Claim 6. The method of Claim 1, wherein wherein the transmitter and/or receiver comprises a streaming media server, video broadcast router, video encoder, or IP surveillance camera.

Claim 7. The method of Claim 1, wherein wherein the receiver comprises a personal computer, mobile device, or an IP consumer Set Top Box.

Claim 8. The method of Claim 1, wherein the network comprises a network selected from the group consisting of the following: the public internet, a wired network, a wireless network, a hybrid network comprising both wired and wireless links, a public switched telephone network, a wireless data network, a proprietary network, and a cable network.

Claim 9. The method of Claim 1, wherein the network comprises an overlay network.

Claim 10. The method of Claim 9, wherein the overlay network is selected from the group consisting of a reflector network, a peer-to-peer network, a virtual private network or a content delivery network.

Claim 11. The method of Claim 1, wherein the data is transmitted within a tunneling protocol for the transmission of proprietary data over a public network.

Claim 12. The method of Claim 1, wherein the ARQ protocol is selected from the group consisting of: a Stop-and-wait ARQ protocol, a Go-Back-N ARQ protocol, Selective Repeat ARQ protocol, and Delay Constrained Selective Repeat ARQ.

Claim 13. The method of Claim 1, wherein the ARQ protocol comprises a protocol wherein only NACKs are utilized to fulfill missing packets.

Claim 14. The method of Claim 1, wherein the data transmitted from the transmitter to the receiver is encoded by a forward error correction method.

Claim 15. The method of Claim 1, wherein the forward error correction method used to encode transmissions from the receiver to the transmitter, and/or from the transmitter to the receiver is selected from the group consisting of the following: a systemic coding method, a non-systemic method, an interleaving coding method, a block coding method, a convolutional coding method, a selective rate FEC coding method, an RF5109 (ULP) protocol, an RFC6015 (1-d interleaved) protocol, a parity method, a 1- or 2-d interleaved XOR method, a Reed- Solomon method, and a Bose-Chaudhuri-Hocquenghem method.

Claim 16. The method of Claim 1, wherein the forward error correction method used to encode transmissions from the receiver to the transmitter applies a coding rate of 1/2.

Claim 17. The method of Claim 14, wherein the forward error correction method used to encode transmission from the transmitter to the receiver is applied at a coding rate of in the range of 3/4 to 15/16.

Claim 18. The method of Claim 1, wherein additional diagnostic transmissions between the transmitter and the receiver are performed at periodic intervals to assess current network conditions.

Claim 19. The method of Claim 18, wherein forward error correction coding rates are dynamically adjusted in response to the current network conditions.

Claim 20. The method of Claim 18, wherein the receiver comprises a buffer and the buffer size is dynamically adjusted in response to the current network conditions; Claim 21. The method of Claim 18, wherein one or more ARQ parameters are adjusted in response to the current network conditions.

Claim 22. The method of Claim 21, wherein the one or more ARQ parameters are selected from the group consisting of the packet request timeout value, the packet request repetition timeout value, and the packet give up timeout value.

Claim 23. The method of any of Claims 1-22, wherein the method is automatically applied to the transmission of data between the transmitter and receiver in response to slow network conditions.

Claim 24. The method of any of Claims 1-23, wherein the receiver is one of a plurality of receivers receiving data from the transmitter.

Claim 25. The method of any of Claims 1-24, wherein the ARQ protocol comprises a protocol wherein only NACKs are utilized and wherein the method comprises the steps of:

(a) at the transmitter, the series of packets is buffered and is sequentially transmitted to the network; (b) at the receiver, any received packets are decoded, damaged packets are identified and discarded, non-damaged packets are buffered in a packet buffer, and the identification of missing packets is carried out continuously or at periodic intervals;

(c) at the receiver, if one or more packets is determined to be damaged or missing, a packet retransmission request identifying the missing packet is generated, addressed to the transmitter, and transmitted to the network, wherein such packet retransmission request is encoded by a forward error correction coding method;

(d) wherein, at the transmitter, if a packet re-transmission request is received from the receiver, the packet re-transmission request is decoded and the one or more requested packets are retrieved from the packet buffer and re-transmitted to the network;

(e) wherein steps (b), (c), and (d) are repeated until all packets within the series have been successfully received at the receiver, a timeout limit is reached, or a maximum number of packet re-transmission requests has been reached for all missing packets; and

(f) outputting the received packets of the window from the receiver buffer to an output destination.

Claim 26. A system, the system comprising a transmitter and a receiver, wherein the transmitter and receiver are configured to perform the methods of any of Claims 1-25.

Claim 27. A transmitter, wherein the transmitter is configured to perform any of the transmitter functions of any of Claims 1-25.

Claim 28. The transmitter of Claim 27, wherein the transmitter comprises a device selected from the group consisting of: a streaming media server, a video broadcast router, a video encoder, and an IP surveillance camera; Claim 29. A receiver, the receiver being configured to perform any of the receiver functions of any of Claims 1-25.

Claim 30. The receiver of Claim 29, wherein the transmitter comprises a device selected from the group consisting of a video broadcast routers, a personal computer, a mobile device, and an IP consumer Set Top Box.

Description:
UNITED STATES PROVISIONAL PATENT APPLICATION

Title: Low Delay, Error Resilient Video Transport Protocol over Public IP Transit

CROSS-RELATED APPLICATIONS: This application claims the benefit of priority to United States Provisional Application Serial Number 62/435,306, filed December 16, 2016, entitled "Low Delay, Error Resilient Video Transport Protocol over Public IP Transit," the contents of which are hereby incorporated by reference.

Background of the Invention.

[0001] The delivery of high quality video from its source to its destination typically requires dedicated fiber circuit networks or satellite links in order to ensure adequate bandwidth and minimal latencies. These modalities, while reliable, are very expensive. Therefore, delivery of high quality video files over public IP networks would be highly advantageous, as public IP networks are ubiquitous and much less expensive to use. However, currently, the delivery of high quality video over public IP networks is very problematic. Public IP networks are lossy, wherein datagram packets are frequently lost, damaged, or delivered with unacceptably high latency, such that video quality is degraded below acceptable standards.

[0002] Various methods are known in the art to compensate for data loss or delay in network transit. A first error control technique for dealing with congested networks is Forward Error Correction (FEC). In forward error correction, data packets are formatted in a redundant coding scheme, such that if a packet is received at the receiving node in a damaged or incomplete state, its data can be successfully reconstructed from the redundant data coded therein. FEC advantageously allows for fulfillment of damaged or incomplete packets without any back communication to the sending node, which decreases latency in error correction. However, the redundant data included in each packet means that larger packets, or a greater number of packets, are sent. In a congested network setting, the increased bandwidth requirements of FEC transmission, especially for high overhead FEC coding schemes, may be problematic, as bandwidth is limited.

[0003] A second error control method is known as Automatic Repeat reQuest (ARQ), which relies on feedback sent from the receiving node to the transmitting node to identify missing or damaged packets and to request their re-transmission. In some implementations of ARQ, the receiving node sends acknowledgement messages to the sending node indicating which packets in the data window have been correctly received and decoded, for example an acknowledgement message (ACK). In other implementations, the feedback comprises a negative acknowledgement (NACK) for those packets that are missing or damaged. By means of these feedback messages from the receiving node, the identity of missing or damaged packets is determined at the sending node, and the missing packets are

retransmitted. This process may be continued or repeated as necessary, until each packet in the series is fulfilled, or until a timeout limit is reached.

[0004] Selective Repeat is part of the family of automatic repeat-request (ARQ) error control methodologies. With selective repeat, the sending node and receiving node are configured such that corrupted or missing packets are identified by the receiving node. Feedback messages sent from the receiver to the sending node result in the re-transmission of corrupted or missing packets, and the process is performed until each packet in the series is fulfilled, or until a timeout limit is reached. Despite the added complexity and required computing power, Selective Repeat ARQ is widely considered to be the best match for correcting video streaming packets among all the other ARQ schemes. While ARQ provides great improvement and error control to network transmissions, it is not without shortcomings. Mainly, the requirement for a feedback loop to fulfill missing data increases latency. In congested networks, the conditions which create data loss in the forward direction will affect the transit time of the feedback messages, and re-transmissions as well, and latencies may be unacceptable, especially for packets that are retransmitted a second, third, or more times.

[0005] The use of what is called "hybrid ARQ" is also known. In hybrid ARQ, packets sent from the transmitter to the receiving node may comprise FEC encoding to correct errors, within an ARQ error detection scheme for requesting unfulfilled packets. In some implementations of hybrid ARQ, FEC encoded blocks for the packets are generated at the sending node, but are not transmitted unless the overlaying ARQ mechanism determines that there are errors in a packet, in which case the FEC parity or other correction data is requested by the receiving node and sent by the transmitting node. Hybrid ARQ can be used to increase transmission quality, yet may not be fast or efficient enough to enable high quality video transmission over lossy public networks. In general in Hybrid ARQ is very inefficient to recover the packet packet from the FEC than just from the original packet itself. [0006] With FEC, ARQ, and hybrid schemes each having inherent shortcomings, there is accordingly a need in the art for improved methods of transferring high quality video and other data over lossy public networks wherein bandwidth is used efficiently and the latency of packet delivery is minimized.

[0007] SUMMARY OF THE INVENTION

[0008] Provided herein are data transfer protocols which employ a novel combination of error control methods, wherein the advantages of FEC and ARQ are exploited while the shortcomings of each methodology are ameliorated by the use of the complementary methods.

[0009] In one aspect, the scope of the invention encompasses a method of transmitting data between a transmitting node and a receiving node, such as video data, over a network, such as the internet, wherein an ARQ is used to identify missing data and request retransmission of such missing data, and FEC is used in the coding of the feedback messages to increase the probability of successful transmission and speed the fulfillment of missing data.

[0010] The method may be carried out in a system comprising a transmission node, a receiving node, and intermediate routers. The scope of the invention encompasses transmission and receiving routers configured to perform the methods of the invention as well. The scope of the invention further includes hardware elements capable of carrying out the operations of the combined SR-ARQ-FEC methods.

[0011 ] Brief Description of the Drawings

[0012] Fig. 1 is a diagram depicting an overview of the general method of the invention. [0013] Fig. 2 is a diagram depicting the operations of the general method of the invention. [0014] Detailed Description of the Invention. [0015] The various elements of the invention are described next.

[0016] Setting. The various embodiments of the invention are directed to the transmission of data, especially video (and associated audio data) over a public network such as the internet. [0017] The data transmitted in the methods of the invention may be derived from any source. For example, the data may be supplied from a storage device, a network such as the internet or a proprietary network, or a camera. For example, the source may comprise high quality video in a format such as MPEG-TS supplied over UDP/IP from a unicast or multicast LAN input. The data being transmitted may comprise any data type, including data coding for voice, audio, video, and other data. In one embodiment, the data is video data from a video file. For example, the video data may comprise MPEG2-TS video data, HD video data, SD video data, and 4K-UHD resolution video data.

[0018] The data will be transmitted in the form of a packet. Each packet within the data window is formatted according to the network protocol being used for transfer of the data. The packet will comprise and addressing data specifying the destination of the packet. The packet may comprise packet sequence identifiers, comprising packet sequence information indicating the packet's order in the data window. The packet may further comprise transmission timing information, indicating the time of transmission, so that the receiving node can determine the age of the packet. The packet may further comprise information indicating the type of encryption and/or encoding that was used to format the packet, so that it can be properly decoded at the receiver. The packet will also comprise a payload comprising the data from the data source. The packet may also comprise a constant length header. Additional control data may be added, for example as used by an overlay network to perform error correction and data reconstruction. Packets may be of any length, for example, a data packet in the range of 56 to 1372 bytes. In some embodiments, the packets comprise the data to be transmitted, e.g. video data and will be referred to herein as data packets.

Additionally, information sent from a receiving node back to the sending node, such as NACK, as described below, will be formatted as a packet as well. Packets can be created from the source data using any packetization methodology known in the art.

[0019] The data may be encrypted for transmission and de-crypted at the destination node. Any encryption methodology known in the art may be used, including, for example encryption standards such as AES 128, AES 192, or AES 256.

[0020] The methods of the invention may be employed on any data network, and may be used for the transmission of any type of data. For example, the methods of the invention may be employed in wired networks, wireless networks, and hybrid networks comprising both wired and wireless links. The network in which the invention is applied may, for example, be the internet, a public switched telephone network, a wireless data network, or a proprietary network such as a cable network.

[0021] In one embodiment, the error control methods of the invention are applied in an overlay network setting. The overlay network may comprise any overlay network, for example a peer-to-peer (P2P) network, virtual private networks (VPN) or a content delivery network (CDN). In one implementation of the methods of the invention, the network is an overlay network comprising a set of reflector (proxy) network elements (e.g. routers). The overlay network may overlay any known network, for example, the public internet.

[0022] In the overlay network implementations of the invention, the protocols described herein are applicable as a tunneling protocol, allowing tunneling of proprietary data over the public internet or other network.

[0023] The transmissions described herein may be accomplished by any appropriate protocol known in the art. Any unidirectional video protocol can be used for the transmission of the packets, for example, protocols based on User Datagram Protocol (UDP) can be used, for example, real-time protocol (RTP), for the transmission of real-time data. In some embodiments, the data transmissions may be divided among multiple UDP sessions and transmitted over multiple UDP ports, as known in the art, to further improve the multipath reliability of the transmission.

[0024] Transmitters and Receivers. The network protocols of the invention are carried out by a transmitter and one or more receivers. For clarity, the various implementations disclosed herein will be described with respect to a unicast transmission having one transmitter and one receiver, for example, "a transmitter transmitting to the receiver." However, it will be understood that the various inventions described herein may be applied in the context of a multicast or broadcast system, wherein one (or more) transmitter sends data to a plurality of receivers, for example tens, hundreds, thousands, or even millions of receivers. As used herein, "transmitter" will refer to a source device or system from which data is transferred to the receiver, i.e. the device or system that is providing data, such as video data, to the receiver. Likewise, "receiver" will refer to the recipient device or system which receives the data. The terms are fixed with respect to the identity of the devices and are not context sensitive, i.e. the device referred to as the transmitter will be referred to as such even as it acts as a receiver, receiving packets (such as NACKs) transmitted from the device referred to as the receiver.

[0025] The transmitter, also referred to as the transmitting node, as used herein, is a device (or combination of devices) which transmits the data stream to its destination, and the receiver, also referred to as the receiving node, is a device or combination of devices which receives the transmitted data. The transmitter and receiver may comprise servers or network devices of any kind, including streaming media servers, video broadcast routers, video encoders or high end IP surveillance cameras. In some embodiments, the receiver comprises a consumer receiving device such as a personal computer, mobile device, or an IP consumer Set Top Boxes, for example, with HDMI or other interface for TVs.

[0026] Transmission from the transmitter to the receiver will flow through a network. For example, the network may comprise the public internet, having a plurality of nodes intermediate between the transmitter and the receiver. Generally, the intermediate nodes do not perform any storage, processing, or logical operations on the data packets beyond ascertaining the packet' s destination and re-transmitting the packet to the next node in the network, as dictated by the selected networking protocol.

[0027] In some implementations, the network comprises an overlay network, for example a reflector (proxy) network. If the packets are being routed between nodes of an overlay network, for example a reflector network, control signals can be included in the packet. In this way, the data can be tunneled via the underlying public network.

[0028] ARQ Error Correction Protocol. In the various implementations of the invention, an error correction protocol is implemented, wherein the error correction protocol comprises an Automatic Repeat reQuest ("ARQ") error control protocol. In the broadest term, ARQ comprises any protocol implemented on the transmitter and the receiver by which a feedback loop between the receiver and transmitter is used to identify and direct the re-transmission of missing packets.

[0029] The scope of the invention encompasses any ARQ protocol, wherein the receiving node transmits acknowledgement messages ("ACKs") identifying properly received packets, and/or negative acknowledgement messages ("NACKs") identifying missing or damaged packets. In some embodiments, a single NACK specifies multiple missing packets to be retransmitted. Various ARQ protocols include Stop-and-wait ARQ, Go-Back-N ARQ, Selective Repeat ARQ, and Delay Constrained ARQ.

[0030] In preferred implementations of the invention, the ARQ protocol implemented by the transmitter and receiver will comprise a Selective Repeat ARQ protocol. For example, delay constrained Selective Repeat ARQ, and variations thereof may be used. In a preferred implementation of selective repeat ARQ, only NACKs are generated.

[0031] In the transmission of data under an ARQ error correction protocol, any number of communications will be made by the transmitter to the receiver. Such transmissions from the transmitter to the receiver will be referred to herein as "forward transmissions" or transmissions in the "forward direction." Examples of forward transmissions include the transmission of packets from the transmitter to the receiver, including primary transmissions (packets being sent for the first time) and secondary transmissions (subsequent retransmission of packets), for example second, third, fourth, etc. retransmissions.

[0032] In the transmission of data under an ARQ error correction protocol, any number of communications will be made by the receiver to the transmitter. Such transmissions between the receiver and the transmitter will be referred to herein as "backward transmissions," or transmissions in the "backward direction."

[0033] Examples of backward transmissions include ACKs, which are messages that identify packets or series of packets which have been successfully received at the receiver. Likewise, backward transmissions include NACKs, which comprise messages that identify packets or series of packets which have not been successfully received. The ACKs and NACKs comprise packets which comprise address data for the destination (the transmitter) and a payload, encoded in FEC as described below, which comprises data that identifies packets (received in the case of ACKs and not received in the case of NACKs. For example, if the ARQ protocol is capable of identifying out-of-order packets, one or more NACKs identifying the unfulfilled packets in the series will be sent to the transmitter. Likewise, if a packet is received and is unreadable or otherwise determined to be corrupted, the receiver will reject the packet and a NACK identifying the missing packet is sent to the transmitter.

Accordingly, NACKs act as packet re-transmission requests. For a given packet or series of packets identified as missing by the system at the receiving end, NACKs will be sent at regular intervals until the packets are received, or a timeout or limit on the number of NACKs is reached.

[0034] Forward Error Correction. Various forward and backward transmissions are made in the transmission of data under an ARQ scheme. As will be discussed below, certain of these forward and backwards transmissions may be encoded using a Forward Error Correction scheme.

[0035] In FEC, data is formatted in a redundant manner that allows the receiver to detect errors in the data and to correct these errors without requiring a retransmission of the packet. Any number or algorithms and data formatting methods known in the art may be utilized in the selected FEC scheme. FEC schemes may include systemic and non-systemic methods, interleaving methods, block codes, convolutional codes. Likewise decoding of the data at the receiving node may employ any number of methods, including hard decision and soft decision algorithms. The FEC schemes utilized in the practice of the invention may encompass any FEC coding and decoding schemes, including Selective Rate FEC schemes, RF5109 (ULP), RFC6015 (1-d interleaved), parity, 1- or 2-d interleaved XOR, Reed- Solomon, techniques, Bose-Chaudhuri-Hocquenghem codes, and others known in the art.

[0036] FEC may be applied at various coding rates, wherein the coding rate (R Q is defined as a ratio of the number of original information bits k to the total number of transmitted bits n, the latter number including both the information bits and redundant bits, i.e., Rc=k/n. "Stronger" FEC coding rates comprise higher proportions of redundant data. Strong FEC coding improves the odds of salvaging partially corrupted packets because the larger amounts of redundant data provides a means of reconstructing the original data. However, the transmission of larger amounts of redundant data will use up bandwidth and reduce throughput. Accordingly, "lighter," lower coding rate FEC can be employed, which reduces bandwidth and increases throughput, although having less error control capability. Coding rate can be expressed as a proportion, for example 7/8, which indicates that 7 of 8 bits in the packet are the raw data and 1 of 8 bits in the packet comprise redundant data. Exemplary overhead values include 1/2, 2/3, 3/4, 5/6, 7/8, and 15/16.

[0037] Upon receipt of an FEC encoded packet, the receiving device will decode the packet and perform processing steps. One such processing step in the decoding process is the detection of errors. Data is checked, by methods known in the art appropriate to the selected FEC coding scheme, to determine if the data is damaged, and if errors are detected, processing steps are performed to correct the errors using the redundant data within the FEC encoded packet.

[0038] Combined ARQ-FEC Protocol of the Invention

[0039] The various embodiments of the invention encompass the combined use of FEC and ARQ for error control and data re-fulfillment. The invention advantageously takes advantage of ARQ's error monitoring capabilities to efficiently identify missing data. FEC is used to efficiently request and/or re-transmit missing packets, enabling the rapid completion of missing data. The novelty of the invention lies in the use of FEC in combination with ARQ. FEC may be applied to certain forward and/or backward transmissions in an ARQ error control protocol.

[0040] In a first aspect, the scope of the invention encompasses the use of FEC in backwards transmissions from the receiver to the transmitter. In one embodiment, the invention encompasses the use of an ARQ error correction protocol wherein all transmissions from the receiver to the transmitter comprise FEC-encoded packets. In one embodiment, NACK messages are encoded using FEC encoding.

[0041] The use of FEC in the encoding of transmissions between the receiver and transmitter presents certain advantages. Efficient ARQ depends on communication between the transmitter and receiver If the network is noisy, as is typical for public networks such as the internet, and if a typical ARQ protocol is applied, there is a probability that the messages from the receiver to the transmitter may be corrupted in transit, such that the transmitter is not be able to make use of the message, causing further delays in packet fulfillment.

[0042] In the case of ARQ protocols that utilize ACKs, when the ACKs are corrupted in transit to the transmitter, the transmitter will not ascertain that the packets specified by the ACK have been received, and will unnecessarily retransmit the packets.

[0043] In an ARQ protocol such as selective repeat ARQ, wherein NACKS are sent, the loss of NACKs will cause delays. If a NACK is not properly received by the transmitter, delay is introduced because the system must wait until the NACK is re-transmitted by the receiver and successfully received by the transmitter. Accordingly, the use of FEC in encoding the NACK packets will decrease latency by ensuring that a higher percentage of NACKs are properly received, speeding the re-transmission and fulfillment of missing packets.

[0044] In one aspect, the invention encompasses an improvement to the prior art ARQ protocols, wherein the "backwards" communications made between the receiver and transmitter that underlie the operations of such ARQ protocols are FEC encoded. When communications back to the transmitter are corrupted, this necessitates an additional retransmission request, adding to latency. The use of FEC encoding in these backwards transmissions greatly increases the probability of their being effectively received and acted upon. The effect of this improvement is to reduce the number of necessary re-transmission requests, speeding the fulfillment of missing packets and improving the efficiency of the ARQ protocol to which the methods of the invention are applied.

[0045] In the practice of the invention, FEC may further be applied to the transmission of packets in the forward direction. In one embodiment, FEC is applied to the primary transmission of packets to increase the percentage of successfully received packets. In one embodiment, FEC is applied to the secondary and higher order (e.g. tertiary, quaternary) retransmissions. In one embodiment, FEC is applied to both the primary and any secondary or higher order re-transmissions. The use of FEC in encoding data packets sent by the transmitter increases the likelihood of successful packet delivery and reduces the number of necessary re-transmission requests and re-transmissions.

[0046] When FEC is applied to data transmissions and/or re-transmissions in the forward direction and is also applied to transmissions in the backwards direction from the receiver to the transmitter, the protocol can be described as a novel "two-way FEC" protocol, in contrast to the typical use of FEC schemes in only the forward direction. In such an implementation, the advantages of ARQ error detection are enhanced by the power of FEC to improve the success rate for the communications back and forth between the transmitter and receiver. The result is faster packet fulfillment and decreased latency.

[0047] FEC may be applied at any desired coding rate, for example at a rate from 1/2 to 15/16, for example, at 1/2, 2/3, 3/4, 5/6, 7/8, or 15/16, or intermediate values thereto.

[0048] When applied to the forward transmissions from the transmitter to the receiver, the FEC coding rate may comprise a relatively lighter FEC, for example, at a coding rate in the range of 1/2 to 15/16. For example, in a preferred implementation, FEC with a high coding rate, e.g. 7/8-15/16 is applied in the primary transmission. This lighter FEC implementation provides an optimized balance between error rate and bandwidth, with sufficient

improvement in error rate that minimizes the need for larger number of SR-ARQ

retransmissions, i.e. to reduce the latency of the primary transmission stream, because high number of ARQ retransmission cycles increases latency and can create additional increases of error rate over the transmission of such data.

[0049] When applied to transmissions in the backwards direction from the receiver to the transmitter, the FEC coding rate may comprise a stronger FEC, for example, with coding rates in the range of 1/2. This stronger application of FEC provides optimal insurance that the communications made from the receiver to the transmitter will arrive in a usable form.

[0050] An overview of one implementation of the general method of the invention is presented in Fig. 1. In Fig. 1, data from a source is input (101) to a transmitter (102), such as a streaming media server. The streaming media server packetizes the data and transmits the packets (103) over the internet or other network (104) to a receiver. The packets are also sent to a buffer (109, connection with server not shown), and these freshly transmitted packets (110) are stored. The receiver (hardware not shown) comprises a buffer (105) wherein received packets are stored (106). Missing packets (107) are identified by the receiver and a request for retransmission (108) of the missing packets is generated and sent over the network (104) to the transmitter. This request is encoded in a forward error correction format, which greatly improves the probability of successful transmission to the transmitter. The transmitter retrieves the requested, previously transmitted packets (111) from a buffer (109) and re-transmits (112) them over the network (104) to the receiver. When the receiver's buffer (105) reaches timeout for the window in question, the packets are output (113).

[0051] Dynamic Tuning. An additional novel feature of the invention is the use of automatic tuning of protocol parameters in response to the changing conditions of the underlying IP transit network. Optionally, the various methods described above can be supplemented with automatic tuning steps, wherein the status of the network is assessed and parameters of the protocol are adjusted in response to maintain efficient data transfer to the receiving node. [0052] The assessment of current network conditions is performed using tools known in the art for determining network status. Test packets, or pings, as known in the art, may be sent between the transmitter and receiver (or by other relevant devices) in both directions, including round-trip transmissions. For clarity, these network diagnostic pings are not subject to the FEC encoding utilized in the data transfer methods of the invention. For example, packet travel time may be assessed at regular intervals, for example 10 to 100 times a second. Processors at the sending and/or receiving nodes will then analyze sampled results, for example, over a timespan ranging from one minute to ten minutes or more. Current network conditions, as used herein, may encompass any measure of network bandwidth, latency, or other factors, including packet travel time between the nodes, packet drop error rates, and bit error rates, or any other parameter of network flow. Following such

assessments, the parameters of the protocol of the invention may be adjusted.

[0053] A first parameter that may be dynamically adjusted is the receiver's binary packet buffering time (latency). For example, the lowest buffering time available at the receiver within a defined timeframe may be determined, to maintain the buffer within a selected range of fulfillment. For example, in one implementation, the buffer size may be adjusted to maintain the buffer in at least 50% fulfillment at all times.

[0054] A second parameter that may be dynamically adjusted in response to network conditions is the weight of FEC coding, as applied to either/or forward and backward transmissions. For example, based on available latency, the maximum number of packet retransmission attempts before timeout may be calculated, for example as the receiver endpoint binary packet buffering time divided by the average packet trip time. The FEC coding should be adjusted to give a constant protection level, keeping retransmissions as low as possible, thus the lower the latency / buffer, the higher the degree of FEC coding.

[0055] A third parameter that may be dynamically adjusted in response to network conditions includes the ARQ timing parameters. These may be adjusted to respond to values such as the average packet transit time, for example, one-way trip time from the transmitter to the receiver (as in packet transmission) or round-trip time from the receiver to the transmitter and back (as in re-transmission request and re-transmission). [0056] For example, based on the average trip time between the transmitter and the receiver, the minimum packet request timeout value may be set, i.e. the minimum time that the receiver will wait for a packet before sending a NACK retransmission request. In this case, the receiver "knows" not to request packets before they have time to arrive. Likewise, based on the observed maximum trip time, the maximum packet request timeout value may be set, i.e. the maximum time that the receiver will wait for a packet before sending a NACK retransmission request. In this case, the receiver "knows" not to wait for packets that exceed this time as they are unlikely to arrive. Accordingly, packet timeout value can be set at a value between the minimum and maximum times, by selected preferences.

[0057] Likewise, based on the average round-trip time between the receiver, transmitter and back, the minimum packet request repetition timeout value may be set, i.e. the minimum time that the receiver will wait for a packet following a NACK retransmission request before sending another NACK retransmission request. In this case, the receiver "knows" not to re-request packets before the previous re-transmission request has had time to be fulfilled. Similarly, based on the observed maximum round-trip time between the receiver, transmitter and back, the maximum packet request repetition timeout value may be set, i.e. the maximum amount of time that the receiver will wait for a packet following a NACK retransmission request before sending another NACK retransmission request. In this case, the receiver "knows" not to further wait for re-requested packets that exceed this time value as they are unlikely to arrive. Accordingly, packet request repetition timeout value can be set at a value between the minimum and maximum times, by selected preferences.

[0058] Another ARQ timing factor that may be adjusted is the packet request give up timeout. For example, based on round-trip time from the receiver to the transmitter and back and the buffer size (in time), the maximum number of re-transmission requests that can be made within the receiver buffer time window can be calculated. Packet requests give up timeout can be set as the time at which the last such re-transmission request is timed out.

[0059] Hybrid Transmission Systems. The methods of the invention may be applied to improve hybrid FEC-ARQ protocols known in the art. In some embodiments, the methods of the invention are applied only when the current network conditions are sub-optimal. For example, the systems of the invention may be configured to operate without ARQ when there are no network traffic issues. Upon the detection of increased latencies, packet loss errors, bitrate errors, or other sub-optimal network conditions, the system may automatically switch to operating in an ARQ mode that employs the improved methods of the invention.

[0060] Implementations of the Invention. The inventions described herein comprise systems, methods, computer programs for the implementation of such methods, hardware, and other elements of a data transit protocol. It will be understood by one of skill in the art that the inventions described herein encompass overlapping subject matter comprising systems, methods, computer programs, computer program products, and hardware configured to perform the operations described herein. It will be understood by one of skill in the art that the invention is practiced in any suitable network environment, broadly encompassing any type of computer, processor, router, broadcast server, reflector router, computer network, or combinations thereof and that practice of the invention is not limited to any single configuration of hardware, processors, operating systems, or programming languages.

[0061] In one aspect, the scope of the invention encompasses a method for the transmission of data, the data being transmitted by the transmitter to the receiver over a network: wherein, at the transmitter, the data is formatted as a series of packets, wherein the packets comprise data from the data source, packet sequence identifiers, and addressing data; wherein the transmitter and receiver comprise devices capable of sending and receiving data packets over and from the network; wherein the transmitter and receiver are configured to perform an ARQ protocol wherein ACK and/or NACK messages are transmitted from the receiver to the transmitter to facilitate the fulfillment of missing packets and wherein missing packets are re-transmitted by the transmitter to the receiver; and wherein the ACK and/or NACK messages transmitted from the receiver to the transmitter are encoded in a forward error correction coding scheme.

The general method of the invention may be implemented in any number of ways, as described herein, including, for example: wherein the data may be voice, audio, or video data; wherein if the data is video data it may comprise MPEG2-TS video data, HD video data, SD video data, or 4K-UHD resolution video data; wherein the data transmitted from the transmitter to the receiver may be encrypted, including by encryption standards AES 128, AES 192, or AES 256; wherein the transmitter and/or receiver may comprise a streaming media server, video broadcast router, video encoder, or IP surveillance camera; wherein the receiver may comprise a video broadcast router, a personal computer, a mobile device, or an IP consumer Set Top Box; wherein the network may comprise the public internet, a wired network, a wireless network, a hybrid network comprising both wired and wireless links, a public switched telephone network, a wireless data network, or a proprietary network, including a such a cable network; wherein the network may comprise an overlay network, including a reflector network, peer-to-peer network, virtual private networks or a content delivery network; wherein the data is transmitted within a tunneling protocol for the transmission of proprietary data over a public network such as the internet; wherein the ARQ protocol may be a Stop-and-wait ARQ, Go-Back-N ARQ, a Selective Repeat ARQ, or Delay Constrained Selective Repeat ARQ protocol; wherein the ARQ protocol may comprise a protocol wherein only NACKs are utilized to fulfill missing packets; wherein the data transmitted from the transmitter to the receiver may be encoded by a forward error correction coding method; wherein the forward error coding method used to encode transmissions from the receiver to the transmitter, and/or from the transmitter to the receiver may be systemic coding method, a non-systemic method, an interleaving coding method, a block coding method, a convolutional coding method, a selective rate FEC coding method, an RF5109 (ULP) protocol, an RFC6015 (1-d interleaved) protocol, a parity method, a 1- or 2-d interleaved XOR method, a Reed-Solomon method, or a Bose-Chaudhuri- Hocquenghem method; wherein the forward error coding used to encode transmissions from the receiver to the transmitter, and/or from the transmitter to the receiver may be applied at a coding rate of 1/2, 2/3, 3/4, 5/6, 7/8, or 15/16, or intermediate values thereto; wherein the forward error coding applied to transmissions from the receiver to the transmitter may be applied at a coding rate of 1/2 to 3/4, for example, 1/2; wherein, if forward error coding is used to encode transmissions from the transmitter to the receiver, it may be applied at a coding rate of in the range of 1/2 to 15/16; wherein additional transmissions between the transmitter and the receiver may be performed at periodic intervals to assess current network conditions; wherein the coding rate of any forward error correction applied to transmitted data may be dynamically adjusted in response to the current network conditions; wherein the receiver comprises a buffer and the buffer size is dynamically adjusted in response to the current network conditions; wherein ARQ parameters may be adjusted in response to network conditions, including the packet request timeout value, the packet request repetition timeout, and the packet give up timeout; wherein the method of the invention is selectively applied to the transmission of data between the transmitter and receiver in response to network conditions; wherein the receiver in the method of the invention may be one of a plurality of receivers receiving data from the transmitter; and wherein the method is implemented as a protocol installed in system components.

[0062] For example, within the various implementations set forth above, in one

embodiment, the method is carried out as follows:

A method for the transmission of data, the data being transmitted by a transmitter to a receiver over a network, wherein the ARQ protocol comprises a protocol wherein only NACKs are utilized and wherein the method comprises the steps of:

(a) at the transmitter, the series of packets is buffered and is sequentially transmitted to the network;

(b) at the receiver, any received packets are decoded, damaged packets are identified and discarded, non-damaged packets are buffered in a packet buffer, and the identification of missing packets is carried out continuously or at periodic intervals;

(c) wherein, at the receiver, if one or more packets is determined to be damaged or

missing, a packet retransmission request identifying the missing packet is generated, addressed to the transmitter, and transmitted to the network, wherein such packet retransmission request is encoded by a forward error correction coding method;

(d) wherein, at the transmitter, if a packet re-transmission request is received from the receiver, the packet re-transmission request is decoded and the one or more requested packets are retrieved from the packet buffer and re-transmitted to the network;

(e) wherein steps (b), (c), and (d) are repeated until all packets within the series have been successfully received at the receiver, a timeout limit is reached, and

(f) outputting the received packets of the window from the receiver buffer to an output destination.

[0063] Output of buffered packets from the receiver is made to the next (e.g. higher order) layer of the system, where further processing may take place, for example, error correction, jitter control, and other corrective processes to account for any missing packets in the window. The output data may be rendered, concatenated, displayed, played, buffered, or used to reconstruct the original source data , depending on the desired application.

[0064] The general method described above is depicted in the diagrams of Fig. 1 and Fig. 2. With reference to Fig. 1, this diagram provides an overview of the general method of the invention. [0065] With reference to Fig. 2, the operations of an exemplary implementation of the invention are depicted. In the depiction of this exemplary method, operations at the transmitter (201) and receiver (202) sides are depicted. At the transmitter side, the input of data from a source is made (203) to the transmitter (201), for example, video data from a video data source. The source data is packetized (204) as a series of sequential packets, preferably FEC encoded packets. These packets are serially transmitted (206) to the network and at the same time, are buffered (205) and stored if needed for retransmission. The network transmission (208) may be unsuccessful (packet loss) or may be successfully received and decoded (209) at the receiver . If packets are determined to be corrupted, as may be achieved by any error detection mechanisms known in the art, the damaged packets are rejected (210) and not buffered. Undamaged packets are buffered (211), for example in a dynamic circular buffer. The receiver assesses which packets are missing (212), using ARQ methodologies known in the art. Missing packets assessment is repeated continuously or at regular intervals. If no missing packets of the window are detected (213), the buffered series of packets is output to the next layer (217). If missing packets are detected (214), but a timeout (or retransmission limit) has been reached (216), the buffered series of packets is output to the next layer (218). If missing packets are detected (214) and a timeout (or retransmission limit) has not been reached (215), the receiver will generate a NACK specifying packets to be retransmitted (218), and will transmit such NACK to the network to be delivered to the transmitter. Such NACK will be encoded using FEC to improve the probability of successful transmission. If the retransmission request is received by the transmitter, it is decoded and the specified packets are retrieved from the buffer (205) and retransmitted (220) to the receiver, where the process is repeated until successful fulfillment of all packets in the window or timeout/retransmission limits are reached.

[0066] The process depicted in Fig. 2 may be auto-tuned according to current network conditions. For example, network status may be determined and appropriate adjustments may be made to the packetization scheme at the transmitter (222), the buffer size of the receiver (223), the determination of timeouts or the formatting of NACKs (224).

[0067] In one aspect, the inventions described herein comprise systems, wherein specific operations are carried out using a combination of hardware elements and non-transitory computer-readable storage medium having computer-readable program instructions stored therein, i.e. software, which directs he operations of the hardware elements. The system of the invention comprises: a transmitter and a receiver, wherein the transmitter and receiver are devices capable of sending and receiving data packets over and from a network and are configured to perform an ARQ protocol wherein ACK and/or NACK messages are transmitted from the receiver to the transmitter to identify fulfilled and/or missing packets and wherein the transmitter will re-transmit packets determined to be unfulfilled or missing; and wherein the transmitter and receiver are configured such that ACK and/or NACK messages transmitted from the receiver to the transmitter are formatted in an FEC coding scheme that can be decoded by the transmitter.

[0068] In one aspect, the invention may comprise a system element. For example, in one embodiment, the invention comprises a routing device configured to act as a transmitter in the system of the invention. In another embodiment, the invention comprises a routing device configured to act as the receiver in the system of the invention.

[0069] It will be understood that the logical operations of the methods of the invention, for example, packetizing, FEC encoding and decoding, assessing missing packets, formulating packet re-transmission requests, etc., are performed by one or more processors of, within, or in connection with the transmitter and by one or more computer processors of, within, or in connection with the receiver.

[0070] In another aspect, the inventions described herein comprise non-transitory memory or data storage devices which store the specific computer-readable program instructions, i.e. software, which enables practice of the operations described herein by suitable hardware devices.

[0071] In one aspect, the invention encompasses a protocol, the protocol being embedded into and carried out by various network elements and hardware devices. For example, the protocol may be embedded into physical IP routing devices, for example devices with Ethernet interfaces. The protocol may be embedded into "virtualized" or "software defined" (SDN) cloud based routing environments (private or public Cloud), such as VMware or Amazon AWS. The protocol may be embedded into "baseband" video routing devices like SDI Interface Routers or ASI Interface Routers. The protocol may be embedded into digital video encoders or professional integrated Digital Receivers Decoders (IRDs). The protocol may further be embedded into hardware elements of a video camera, for example, surveillance IP cameras of any kind. The protocol may further be embedded into consumer receiving devices like personal computers, mobile devices, IP consumer Set Top Boxes, for example, with HDMI or other interface for TVs.

[0072] EXAMPLES.

[0073] Example 1. Description of an Exemplary Embodiment. Reference is made to Fig. 1, which depicts the general process of the exemplary embodiment, described in greater detail here. In this example, MPEG-TS video data transmitted from a source over UDP/IP from a unicast or multicast LAN input is received at the transmitter. The data is packetized and formatted with FEC error correction coding. The level of FEC coding can be dynamically adjusted, depending on network conditions. The packets are stored in a dynamically adjusted size circular buffer. The packets are also transmitted over a global IP transit network, such as the internet. The transmission may be single or may be divided into multiple sessions.

[0074] In this primary transmission, packets may be lost, arrive at the destination after delay, or may arrive out of order, which would disrupt the MPEG-TS container integrity and video quality, if not corrected. At the receiver, error detection is applied to determine packet integrity. Damaged packets are rejected. Good packets are stored in a circular buffer. A Selective Repeat Delay Constrained ARQ protocol running at the receiver recognizes missing packets.

[0075] NACKs which identify the missing packets are generated by the receiver, and these messages are encoded using FEC. The FEC coding may be dynamically determined, with higher coding rates (more redundancy) applied if network conditions are poor. The NACKs are transmitted over the network to the transmitter. The use of FEC increases the probability of successful transmission of functional NACKs to the transmitter. [0076] Upon receiving the NACKs, the transmitter retrieves the requested packets from the buffer and re-transmits them over the network to the receiver. If received and if intact, the packets are stored in the buffer. Re-transmitted packets that were not successfully fulfilled will be identified as missing packets by ARQ protocols monitoring for missing packets, and additional re-transmission requests will be generated for such packets.

[0077] When the buffer window has reached its limit, the received packets within the window are output. With successful fulfillment of missing packets, the output will comprise a high quality fully reconstructed MPEG-TS stream which may be output to a LAN for multicast or unicast.

[0078] The data may be protected with strong encryption such as AES 256. The data transmission may be relayed over a proprietary overlay network of reflector proxy routers, imparting multipath capabilities.

[0079] This method ensures the most efficient video feed reconstruction mechanism with both with very low overhead (SR-ARQ + Selectable Rate FEC over UDP/IP on the main data feed) and very low latency. The novel use of FEC in requesting re-transmission of packets minimizes the occurrence of lost NACKs, avoiding additional Re-Requests and Re- Transmissions that would increase the buffer size and thus the latency.

[0080] EXAMPLE 2. Performance Metrics. Using standard professional video transmission protocols and buffering technologies (i.e. one way standard FEC with non intelligent "two way-checked" buffer on public networks), average latency can be about 30X of ping travel time (nearly 5 sees minimum buffer at 150 msec of ping travel time) and bandwidth overhead of about 35% is typical, with no video container integrity guaranteed stability at the receiving node ("one way" buffer does not guarantee packet integrity).

[0081] In contrast, using the methods of the invention, stable, very low latency video transmission over public IP with a high guarantee of video container integrity (SR-ARQ based two-way buffer reconstruction) at the receiving node can be as low as 2X the ping travel time with an overhead of just 7% (FEC 15/16 + SR-ARQ process with retransmitted packets if needed), under optimal underlying network conditions.

[0082] For example, at ping travel times of 50 msec, latencies of 0.120 seconds may be attained on long haul IP networks links. Using the methods of the invention, there are less retransmission, and the retransmissions have a higher rate of success, with lower latency overall.

[0083] Additionally, any given latency, the advantageous combined use of FEC and ARQ employed in the methods of the invention creates a video link that is much more robust and stable than that attained using standard methods.

[0084] Example 4. Dynamic FEC and ARQ Parameters. The following illustrates implementation of dynamic adjustment of FEC and ARQ parameters in response to current network conditions. In this example, under good conditions, the buffer size at the receiving node is set at 200 ms and average packet round trip travel time is 40 ms. In this case, five retransmissions may be achieved during the buffer window. Under such conditions, the system may be configured to implement no FEC for forward transmissions, or a very light level of FEC, for example, 15/16 (increasing overhead by about 7%).

[0085] When network conditions become sub-optimal, for example, if travel times increase to 80 ms, only two re-transmissions become possible in the 200 ms buffer windows. In such case, system may be configured to increase the FEC level to 3/4 or even 2/3, resulting in a 30-50% increase in overhead, but improving the probability of successful transmission.

[0086] Likewise, ARQ parameters may be adjusted in response to the increased travel time. For example, the packet request timeout value, the packet request repetition timeout value, and the packet give up timeout value may be increased to a value somewhat higher than the average travel time, in order to reduce or eliminate packet duplication, a situation that occurs when a previously received packet is received a second time, the result of requesting a retransmission of the packet before the previous retransmission attempt has had sufficient time to succeed.

[0087] Likewise, if the packet loss rate increases, the proportion of the receiver's buffer that is filled will decrease. In response, the buffer time can be increased, making the ARQ protocol more resilient, at the cost of a higher transmission delay of packets from the buffer.

[0088] When network conditions improve, the adjustments may be reversed, for example with a hysteresis delay applied. [0089] All patents, patent applications, and publications cited in this specification are herein incorporated by reference to the same extent as if each independent patent application, or publication was specifically and individually indicated to be incorporated by reference. The disclosed embodiments are presented for purposes of illustration and not limitation. While the invention has been described with reference to the described embodiments thereof, it will be appreciated by those of skill in the art that modifications can be made to the structure and elements of the invention without departing from the spirit and scope of the invention as a whole.