BACKGROUND

[0001] Data Over Cable Service Interface Specification (DOCSIS) 3.0 technology uses multiple bonded channels to enable services such as Internet Protocol (IP) video delivery. The video streams are typically delivered as multicast packets to a cable modem termination system (CMTS) and "switched" by the CMTS so that only the video streams that a customer is watching are actually sent down a DOCSIS downstream channel. Since the bandwidth required may exceed the capacity of the channel and drop packets, the prior art describes a variety of improvements to channel bonding to improve the quality of service.

[0002] The prior art also describes an innovation that overlaps bonding groups that span multiple channels. In this case, the CMTS scheduler implements instantaneous load balancing to schedule excess peak Variable Bit Rate (VBR) video on DOCSIS channels with extra capacity, or conversely to allow best effort High Speed Data (HSD) to utilize DOCSIS channels when the VBR video is in a "valley". The prior art describes load balancing across a bonding group that is within the control of a single CMTS entity; however, as IP video delivery over DOCSIS expands and scales, there are scenarios where the DOCSIS channels that are available to leverage are spread across multiple devices. In a first scenario, the deployed system uses a DOCSIS IP Television (IPTV) Bypass Architecture (DIBA) to send the IP video directly to an edge Quadrature Amplitude Modulator (EQAM) where it is

encapsulated as a DOCSIS packet and sent directly to the user devices, thereby bypassing the CMTS. In the first scenario, the EQAM controls the IP video channels, while the CMTS controls the HSD channels. In a second scenario, multiple devices generate DOCSIS channels and broadcast those channels to user devices. For IP video broadcast delivery, a CMTS may use static multicast to deliver the IP video over DOCSIS channels to a large group of users that span many different service groups; while a different CMTS may provide the HSD and narrowcast video channels to a single service group.

[0003] There is a demand for an IP video delivery method and system to allocate DOCSIS flexibly sized bonding groups across multiple devices. The presently disclosed invention satisfies this demand.

SUMMARY

[0004] Aspects of the present invention provide a system and method receives Internet Protocol (IP) packets on a first device for delivery over a DOCSIS interface, each DOCSIS packet encapsulating IP data, and including a sequence number that the first device generates for a bonding group. The method delivers a first number of the DOCSIS packets to a DOCSIS device using first downstream channels on the first device that are associated with the bonding group. When a capacity of the first downstream channels is exceeded, the method determines an available capacity of second downstream channels on a second device that are associated with the bonding group, identifies a second number of the DOCSIS packets that do not exceed the available capacity on the second downstream channels, and forwards the second number of the DOCSIS packets to the second device, where the second device delivers the second number of the DOCSIS packets to the DOCSIS device using the second downstream channels.

[0005] Aspects of the present invention also describe the use of DOCSIS 3.0 bonding groups with an "instantaneous load balancing" algorithm to provide optimum utilization that is extensible to a distributed environment that spans multiple CMTS and/or DIBA devices. Specifically, the present invention implements multiple logical bonding groups that span the shared DOCSIS channels and that each CMTS and DIBA entity controls at least one of the bonding groups, thereby controlling the bonding sequence numbers for their respective bonding groups.

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] FIG. 1 is a network diagram that illustrates one embodiment of the hardware components of a system that performs the present invention.

[0007] FIG. 2 is a block diagram that illustrates, in detail, one embodiment of the hardware components shown in FIG. 1.

[0008] FIG. 3 is a message flow diagram that illustrates a method according to one embodiment of the present invention.

[0009] FIG. 4 is a message flow diagram that illustrates a method according to one embodiment of the present invention.

DETAILED DESCRIPTION

[0010] FIG. 1 is a network diagram that illustrates one embodiment of the hardware components of a system that performs the present invention. FIG. 1 depicts an IP video delivery system 100 that includes an IPTV server 110, IP network 120, and hybrid fiber-coaxial (HFC) network 150. The IPTV server 110 is a streaming server that uses the IP protocol to deliver Video-on-Demand, Audio-on-Demand, and Pay- per-View streams to the IP network 120. The HFC network 150 is a data and video content network that connects a cable modem termination system (CMTS) 130 and edge device 140 to a customer location 160. The IP video delivery system 100 shown in FIG. 1 may include any number of interconnected IPTV server 110, IP network 120, CMTS 130, edge device 140, HFC network 150, and customer location 160.

[0011] The IP network 120, in one embodiment, is a public communication network or wide area network (WAN). The present invention also contemplates the use of comparable network architectures. Comparable network architectures include the Public Switched Telephone Network (PSTN), a public packet-switched network carrying data and voice packets, a wireless network, and a private network. A wireless network includes a cellular network (e.g., a Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), or Orthogonal Frequency Division Multiplexing (OFDM) network), a satellite network, and a wireless Local Area Network (LAN) (e.g., a wireless fidelity (Wi-Fi) network). A private network includes a LAN, a Personal Area Network (PAN) such as a Bluetooth network, a wireless LAN, a Virtual Private Network (VPN), an intranet, or an extranet. An intranet is a private communication network that provides an organization such as a corporation, with a secure means for trusted members of the organization to access the resources on the organization's network. In contrast, an extranet is a private communication network that provides an organization, such as a corporation, with a secure means for the organization to authorize non-members of the organization to access certain resources on the organization's network. The system also contemplates network architectures and protocols such as Ethernet, Internet Protocol, and

Transmission Control Protocol. In various embodiments, the IP network 120 will support a variety of network interfaces, including 802.3ab/u/etc, Multimedia over Coax Alliance (MoCA), and 801.11.

[0012] The HFC network 150 is a broadband network that combines optical fiber and coaxial cable, technology commonly employed globally by cable television operators since the early 1990s. The fiber optic network extends from the cable operators master head end, sometimes to regional head ends, and out to a neighborhood hubsite, and finally to a fiber optic node that serves anywhere from 25 to 2000 homes. The master head end will usually have satellite dishes for reception of distant video signals as well as IP aggregation routers. Some master head ends also house telephony equipment for providing telecommunications services to the community. The regional head ends receive the video signal from the master head end and add to it the Public, Educational and/or Governmental (PEG) channels as required by local franchising authorities or insert targeted advertising that would appeal to the region. The various services are encoded, modulated and up-converted onto RF carriers, combined onto a single electrical signal and inserted into a broadband optical transmitter. This optical transmitter converts the electrical signal to a downstream optically modulated signal that is sent to the nodes. Fiber optic cables connect the head end to optical nodes in a point-to-point or star topology, or in some cases, in a protected ring topology. [0013] In the IP video delivery system 100 shown in FIG. 1, the CMTS 130 and edge device 140 both support DOCSIS 3.0 bonding groups and include downstream channels (e.g., QAM channels as defined in DOCSIS), and the edge device 140 is a DIBA-capable EQAM. In one embodiment, the CMTS 130 provides downstream channels for the HSD (e.g., channels 1-4), and the edge device 140 provides downstream channels for the IP video (e.g., channels 5-8). In this embodiment, the CMTS 130 controls an HSD bonding group that spans the downstream channels on the CMTS 130 and the downstream channels on the edge device 140, and the edge device 140 controls an IP video bonding group that spans the downstream channels on the edge device 140 and the downstream channels on the CMTS 130. As the edge device 140 receives VBR IP video traffic, it primarily schedules delivery of the IP video traffic on its IP video bonding group using downstream channels on the edge device 140. When the IP video traffic peaks during certain times and exceeds the capacity of the IP video downstream channels on the edge device 140, the edge device 140 can forward the excess IP video traffic to the CMTS 130, but since the edge device 140 controls the IP video traffic, including the excess IP video traffic, the edge device 140 assigns the bonding group sequence number for the IP video bonding group. Upon receiving the excess IP video traffic, the CMTS 130 schedules its transmission in a queue for downstream channels appropriate for the IP video bonding group and does not need to generate the bonding group sequence number. Any DOCSIS 3.0 compliant cable modem can successfully reassemble the IP video stream using the bonding group sequence numbers. Similarly, the CMTS 130 primarily uses its downstream channels for sending best effort HSD traffic on its HSD bonding group. However, the CMTS 130 can forward the excess best effort HSD traffic to the edge device 140 when the edge device 140 downstream channels associated with the HSD bonding group are underutilized. In essence, since the IP Video bonding group and the HSD bonding group are overlapping, this allows the CMTS 130 to fill in the VBR IP video 'Valleys". Since this excess best effort HSD traffic use the CMTS 130 controlled HSD bonding group, the CMTS 130 assigns the bonding group sequence numbers. Upon receiving the excess best effort HSD traffic, the edge device 140 can use simple priority queuing of the excess best effort HSD traffic to fill in any unused capacity on its transmission queue for downstream channels appropriate for the HSD bonding group.

[0014] The Modular CMTS (M-CMTS) specification describes the use of a DOCSIS external physical interface (DEPI) tunnel for forwarding packets from a CMTS core to an EQAM for delivery over an HFC network. In the embodiment described above, the present invention uses a DEPI tunnel as described in the M-CMTS specification to forward the excess traffic between the CMTS 130 and edge device 140 for delivery over the HFC network 150. One skilled in the art may employ similar tunnel mechanisms to forward the excess traffic between the CMTS 130 and the edge device 140.

[0015] Since the forwarding of excess traffic can potentially overburden the edge device 140 and the CMTS 130, in the embodiment described above, the IP video delivery system 100 also includes a flow control mechanism. In one embodiment, for IP video traffic the flow control mechanism may be a simple admission control that identifies the amount of excess video traffic that can be forwarded. Thus, in the above example, the CMTS 130 may be able to statically or dynamically determine how much VBR IP video traffic that the edge device 140 can forward to it. The best effort HSD traffic is more problematic because it may sit for a while in a lower priority queue; however, a proactive flow control mechanism, such as issuing credits, can be used. Thus, in the above example, the edge device 140 issues credits to the CMTS 130 to indicate how much best effort HSD traffic it can forward on its QAM channels appropriate for HSD delivery. In various other embodiments, the flow control mechanisms to solve this problem include "credit-based" mechanisms in which the receiver issues credits for forwarding traffic to a receiver, "rate-based" mechanisms in which the receiver sends information for the transmitter to adjust the rate at which it sends to the receiver, and the like.

[0016] The customer location 160 shown in FIG. 1, in one embodiment, is the premises, such as a home, of a customer such as a cable subscriber. The customer location 160 includes a cable modem 162, IPTV 164, and customer premises equipment (CPE) 166. In various embodiments, the IPTV 164 is a device that is capable of receiving an IP video stream, and the CPE 166 is a set-top box or Digital Television (DTV) Converter (DTC).

[0017] FIG. 2 is a block diagram that illustrates, in detail, one embodiment of the hardware components shown in FIG. 1. In particular, FIG. 2 illustrates the hardware components and software comprising the CMTS 130, and edge device 140 shown in FIG. 1.

[0018] The CMTS 130, in one embodiment, is a general-purpose computing device that performs the present invention. A bus 205 is a communication medium that connects a processor 210, data storage device 215, communication interface 220, DOCSIS external physical interface (DEPI) 225, and memory 230 (such as Random Access Memory (RAM), Dynamic RAM (DRAM), non-volatile computer memory, flash memory, or the like). The communication interface 220 connects the CMTS 130 to the IP network 120, and HFC network 150. In one embodiment, the communication interface 220 utilizes downstream channels (e.g., channels 1-4) to communicate with the HFC network 150. The DEPI 225 connects the CMTS 130 to the edge device 140. In one embodiment, the implementation of the present invention, or portions thereof, on the CMTS 130 is an application-specific integrated circuit (ASIC).

[0019] The processor 210 performs the disclosed methods by executing the sequences of operational instructions that comprise each computer program resident in, or operative on, the memory 230. The reader should understand that the memory 230 may include operating system, administrative, and database programs that support the programs disclosed in this application. In one embodiment, the configuration of the memory 230 of the CMTS 130 includes a DOCSIS program 232, transmission channel queue 234, distributed flexible channel bonding program 236, and instantaneous load balancing process 238. The DOCSIS program 232 implements the DOCSIS 3.0 specification. The transmission channel queue 234 is a memory location that the CMTS 130 uses to schedule and prioritize the delivery of data on the CMTS 130 downstream channels. The distributed flexible channel bonding program 236 allocates DOCSIS flexibly sized bonding groups across multiple devices. The instantaneous load balancing process 238 balances the load across the downstream channels in a bonding group across multiple devices. The DOCSIS program 232, transmission channel queue 234, distributed flexible channel bonding program 236, and instantaneous load balancing process 238 perform the methods of the present invention disclosed in detail in FIG. 3 and FIG. 4. When the processor 210 performs the disclosed methods, it stores intermediate results in the memory 230 or data storage device 215. In another embodiment, the memory 230 may swap these programs, or portions thereof, in and out of the memory 230 as needed, and thus may include fewer than all of these programs at any one time.

[0020] The edge device 140, in one embodiment, is a general-purpose computing device that performs the present invention. A bus 255 is a communication medium that connects a processor 260, data storage device 265, communication interface 270, DOCSIS external physical interface (DEPI) 275, and memory 280 (such as Random Access Memory (RAM), Dynamic RAM (DRAM), non-volatile computer memory, flash memory, or the like). The communication interface 270 connects the edge device 140 to the IP network 120, and HFC network 150. In one embodiment, the communication interface 270 utilizes downstream channels (e.g., channels 5-8) to communicate with the HFC network 150. The DEPI 275 connects the edge device 140 to the CMTS 130. In one embodiment, the implementation of the present invention, or portions thereof, on the edge device 140 is an application-specific integrated circuit (ASIC).

[0021] The processor 260 performs the disclosed methods by executing the sequences of operational instructions that comprise each computer program resident in, or operative on, the memory 280. The reader should understand that the memory 280 may include operating system, administrative, and database programs that support the programs disclosed in this application. In one embodiment, the configuration of the memory 280 of the edge device 140 includes a DOCSIS program 282, transmission channel queue 284, distributed flexible channel bonding program 286, and instantaneous load balancing process 288. The DOCSIS program 282 implements the DOCSIS 3.0 specification. The transmission channel queue 284 is a memory location that the edge device 140 uses to schedule and prioritize the delivery of data on the edge device 140 downstream channels. The distributed flexible channel bonding program 286 allocate DOCSIS flexibly sized bonding groups across multiple devices. The instantaneous load balancing process 288 balances the load across the downstream channels in a bonding group across multiple devices. The DOCSIS program 282, transmission channel queue 284, distributed flexible channel bonding program 286, and instantaneous load balancing process 288 perform the methods of the present invention disclosed in detail in FIG. 3 and FIG. 4. When the processor 260 performs the disclosed methods, it stores intermediate results in the memory 280 or data storage device 265. In another embodiment, the memory 280 may swap these programs, or portions thereof, in and out of the memory 280 as needed, and thus may include fewer than all of these programs at any one time.

[0022] FIG. 3 is a message flow diagram that illustrates a method according to one embodiment of the present invention. In particular, FIG. 3 illustrates distributed flexible channel bonding for an edge device 140 receiving an IP video stream.

[0023] The process 300 shown in FIG. 3, with reference to FIG. 1 and FIG. 2, begins when the edge device 140 receives a stream of IP packets for its DOCSIS interface, where each DOCSIS packet encapsulates an IP video stream, and includes a sequence number that the edge device 140 generates for an IP video bonding group that the edge device 140 controls (step 305). In one embodiment, the sequence number is a bonding group sequence number that the edge device 140 assigns to each DOCSIS packet, and that a downstream DOCSIS device (e.g., a cable modem 162) can use to reconstruct the DOCSIS stream. The process 300 delivers a first number of the DOCSIS packets to a downstream DOCSIS device (e.g., a cable modem 162) using downstream channels on the edge device 140 that are associated with the IP video bonding group (e.g., channels 5-8) (step 310). The edge device 140 continually monitors the downstream channels on the edge device 140 that are associated with the IP video bonding group (e.g., channels 5-8) to detect that a capacity of those downstream channels exceed a threshold value (step 315). In one embodiment, the threshold value is a percentage of available bandwidth for the downstream channels on the edge device 140 that are associated with the IP video bonding group (e.g., channels 5-8). The edge device 140 then determines an available capacity of downstream channels on the CMTS 130 that are associated with the IP video bonding group (e.g., channels 1-4) to schedule excess IP video traffic (i.e., flow control capacity) (step 320). In one embodiment, the edge device 140 sends a request to the CMTS 130 to provide the available capacity of the downstream channels on the CMTS 130 that are associated with the IP video bonding group, and receives the available capacity in response. The process 300 then identifies a second number of the DOCSIS packets that do not exceed the available capacity of the downstream channels on the CMTS 130 that are associated with the IP video bonding group (e.g., channels 1-4) (step 325), and forwards the second number of the DOCSIS packets to the CMTS 130 (step 330). In one embodiment, a DEPI tunnel transfers the second number of the DOCSIS packet to the CMTS 130. The CMTS 130 receives the second number of the DOCSIS packets (step 335), and delivers the second number of the DOCSIS packets to the DOCSIS device using the downstream channels on the CMTS 130 that are associated with the IP video bonding group (e.g., channels 1-4) (step 340).

[0024] FIG. 4 is a message flow diagram that illustrates a method according to one embodiment of the present invention. In particular, FIG. 4 illustrates distributed flexible channel bonding for a CMTS 130 receiving HSD.

[0025] The process 400 shown in FIG. 4, with reference to FIG. 1 and FIG. 2, begins when the CMTS 130 receives a stream of IP packets for delivery for its DOCSIS interface, where each DOCSIS packet encapsulates an HSD stream, and includes a sequence number that the CMTS 130 generates for an HSD bonding group that the CMTS 130 controls (step 405). In one embodiment, the sequence number is a bonding group sequence number that the CMTS 130 assigns to each DOCSIS packet, and that a downstream DOCSIS device (e.g., a cable modem 162) can use to reconstruct the DOCSIS stream. The process 400 delivers a first number of the DOCSIS packets to a downstream DOCSIS device (e.g., a cable modem 162) using downstream channels on the CMTS 130 that are associated with the HSD bonding group (e.g., channels 1-4) (step 410). The CMTS 130 continually monitors the downstream channels on the CMTS 130 that are associated with the HSD bonding group (e.g., channels 1-4) to detect that a capacity of those downstream channels exceed a threshold value (step 415). In one embodiment, the threshold value is a percentage of available bandwidth for the downstream channels on the CMTS 130 that are associated with the HSD bonding group (e.g., channels 1-4). The CMTS 130 then determines an available capacity of downstream channels on the edge device 140 that are associated with the HSD bonding group (e.g., channels 5-8) to schedule excess HSD traffic (i.e., flow control capacity) (step 420). In one embodiment, the CMTS 130 sends a request to the edge device 140 to provide the available capacity of the downstream channels on the edge device 140 that are associated with the HSD bonding group, and receives the available capacity in response. The process 400 then identifies a second number of the DOCSIS packets that do not exceed the available capacity of the downstream channels on the edge device 140 that are associated with the HSD bonding group (e.g., channels 5-8) (step 425), and forwards the second number of the DOCSIS packets to the edge device 140 (step 430). In one

embodiment, a DEPI tunnel transfers the second number of the DOCSIS packet to the edge device 140. The edge device 140 receives the second number of the DOCSIS packets (step 435), and delivers the second number of the DOCSIS packets to the DOCSIS device using the downstream channels on the edge device 140 that are associated with the HSD bonding group (e.g., channels 5-8) (step 440).

[0026] The prior art flexible channel bonding scheme with instantaneous load balancing enables an IP video delivery system to fully utilize VBR video streams for IP video delivery over cable over a single CMTS. The present invention describes the use of multiple logical bonding groups, each controlled by a separate device, to enable the flexible channel bonding scheme to be implement in a distributed manner across multiple CMTS and/or DIBA-capable EQAM devices. Thus, the present invention permits separate transmitting entities of DOCSIS bonding groups (e.g., the CMTS 130 and edge device 140) to share bandwidth on the same downstream channels. In other embodiments, the present invention may support other types of data and services other than VBR video streams and HSD, and may support a multitude of devices, not just the two devices disclosed in the embodiments. Furthermore, the logical bonding groups need not cover identical channels, but may also be non-overlapping bonding groups as described in the prior art

[0027] Although the disclosed embodiments describe a fully functioning method and system for IP video delivery, the reader should understand that other equivalent embodiments exist. Since numerous modifications and variations will occur to those reviewing this disclosure, the method and system for IP video delivery is not limited to the exact construction and operation illustrated and disclosed. Accordingly, this disclosure intends all suitable modifications and equivalents to fall within the scope of the claims.