CONNECTION

Technical field

The invention relates to a network node for scheduling packets for transport over a Multipath Transmission Control Protocol (MPTCP) connection, a method of scheduling packets for transport over an MPTCP connection, a corresponding computer program, and a corresponding computer program product.

Background

Communication via the Transmission Control Protocol (TCP) and the Internet Protocol (IP) is restricted to a single network path per connection, even though multiple network paths may exist between the peers of the connection. For instance, a mobile terminal may simultaneously be connected to a cellular radio access network, such as Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), or Long Term Evolution (LTE), network, and a Wireless Local Area Network (WLAN) or WiFi access network. Likewise, modern residential gateways, or home gateways, for deployment at customer premises oftentimes provide hybrid access to the Internet by means of cellular connectivity in addition to a wired access, such as a Digital

Subscriber Line (DSL) service.

Multipath TCP (MPTCP) is an ongoing effort of the Internet

Engineering Task Force (IETF) and aims at allowing an end-to-end TCP connection to use multiple paths to maximize resource usage and increase redundancy, thereby improving user experience (see, e.g., "TCP Extensions for Multipath Operation with Multiple Addresses", IETF RFC 6824). MPTCP extends TCP so that it presents a standard TCP interface to applications while in fact spreading data across several MPTCP subflows, i.e., separate TCP connections, typically using disjoint network paths. MPTCP is designed to be backwards compatible with plain TCP.

MPTCP is particularly useful in the scenarios described above. For instance, a mobile terminal using both a WLAN/WiFi access network and a cellular radio access network may experience a gain in throughput and connection reliability as the mobile terminal moves in or out of coverage without disrupting the end-to-end TCP connection. The problem of handover between MPTCP subflows is solved by abstraction in the transport layer, without any special mechanisms at the network or link level. Advantageously, when used in connection with hybrid-access solutions for residential gateways, the Internet Service Provider (ISP) may control the behavior of the residential gateway and thereby steer the traffic so as to optimize user experience and reduce cost. For instance, residential gateways may be configured to predominantly use DSL or any other wired access, whereas the cellular communications network is only used for excess traffic.

While MPTCP is designed as an extension to standard TCP, there are efforts to extend its use to the User Datagram Protocol (UDP) with the aim of offering multi-path support for also for UDP (see, e.g., "An MPTCP Option for Network-Assisted MPTCP Deployments: Plain Transport Mode", IETF

Internet-Draft, draft-boucadair-mptcp-plain-mode-07).

A problem which is inherent to hybrid access solutions is the difference in forward delay which is experienced by packets which are transported over distinct access networks. For instance, the RTT is typically larger for a DSL access network than for LTE (of the order of 50 ms for DSL, as compared to 20 ms for LTE), and the reverse scenario may occur if the LTE access network is congested. Such a considerable difference in forward delay increases the variation in packet delay, also known as jitter, which is experienced by the end-to-end connection. This is a result of an increased amount of packets which need to be buffered at the receiving endpoint of the MPTCP connection in order to provide in-order delivery. The increase in jitter has a negative impact on the service or application utilizing the connection. In addition, the increased amount of packets which need to be buffered at the receiving endpoint also requires an increase in buffer size.

An overview of known MPTCP schedulers is given in "Experimental Evaluation of Multipath TCP Schedulers", by C. Paasch, S. Ferlin, O. Alay, and O. Bonaventure, in CSWS '14 Proceedings of the 2014 ACM SIGCOMM workshop on Capacity sharing, pages 27-32, 2014.

A scheduling algorithm based on forward delay is described in

"Forward Delay-based Packet Scheduling Algorithm for Multipath TCP", by T.-A. Le and L. X. Bui, in OALib Journal Computer Science,

http://arxiv.Org/abs/1501 .03196v1 , 2015. Summary

It is an object of the invention to provide an improved alternative to the above techniques and prior art.

More specifically, it is an object of the invention to provide an improved solution for scheduling packets for transport over an MPTCP connection comprising at least two TCP connections as MPTCP subflows. In particular, it is an object of the invention to provide an improved solution for hybrid-access scenarios in which the at least two TCP connections exhibit considerably different forward delays.

These and other objects of the invention are achieved by means of different aspects of the invention, as defined by the independent claims. Embodiments of the invention are characterized by the dependent claims.

According to a first aspect of the invention, a network node for scheduling packets for transport over an MPTCP connection is provided. The MPTCP connection comprises at least two TCP connections as MPTCP subflows. The network node is operative to acquire a difference in forward delay between a first TCP connection and a second TCP connection of the at least two TCP connections, wherein the second TCP connection has a lower forward delay than the first TCP connection. The network node is further operative to delay packets scheduled for transport over the second TCP connection by a time interval which is substantially equal to the difference in forward delay.

According to a second aspect of the invention, a method of scheduling packets for transport over an MPTCP connection is provided. The MPTCP connection comprises at least two TCP connections as MPTCP subflows. The method is performed at a sending endpoint of the MPTCP connection and comprises acquiring a difference in forward delay between a first TCP connection and a second TCP connection of the at least two TCP

connections, wherein the second TCP connection has a lower forward delay than the first TCP connection. The method further comprises delaying packets for transport over the second TCP connection by a time interval which is substantially equal to the difference in forward delay.

According to a third aspect of the invention, a computer program is provided. The computer program comprises computer-executable

instructions for causing a device to perform the method according to an embodiment of the second aspect of the invention, when the computer- executable instructions are executed on a processing unit comprised in the device.

According to a fourth aspect of the invention, a computer program product is provided. The computer program product comprises a computer- readable storage medium which has the computer program according to the third aspect of the invention embodied therein.

In the present context, forward delay is defined as the delay in forward direction from a sending network node, a sender, of a packet to a receiving network node, a receiver, of the packet. Similarly, backward delay is the delay in backward direction, i.e., from the receiver back to the sender. The RTT is the total of forward delay and backward delay.

The invention makes use of an understanding that an improved transport of packets, such as TCP packets or UDP packets/datagrams, over an MPTCP connection comprising at least two TCP connections as MPTCP subflows may be achieved by taking a difference in forward delay between the distinct TCP connections into account in scheduling packets for transport over either of the at least two TCP connections. Embodiments of the invention are advantageous in that they result in improved transport characteristics, in particular a reduction in the amount of packets which need to be buffered at the receiving endpoint of the MPTCP connection for the purpose of in-order delivery to their destination. The need for buffering packets at the receiving endpoint stems from a difference in forward delay between the distinct TCP connections, as packets which are transported over the faster TCP connection, the TCP connection having the lower forward delay, need to be buffered at the receiving endpoint in order to be interleaved with packets which are transported over the slower TCP connection, the TCP connection having the larger forward delay, for in-order delivery to the destination of the connection. It will be appreciated that the issue of in-order delivery not only arises for packets which are transported by means of TCP, but may also occur if UDP packets are transported in plain transport mode over an MPTCP connection ("An MPTCP Option for Network-Assisted MPTCP Deployments: Plain Transport Mode", IETF Internet-Draft, draft- boucadair-mptcp-plain-mode-07). In plain transport mode, in-order delivery may be enforced at the receiving endpoint of the MPTCP connection so as to reduce jitter in the stream of UDP packets which is delivered to the

destination of the UDP flow. This problem which arises in transport of packets over distinct network paths having different forward delays is referred to as Head-of-Line (HOL) delivery. As a further advantage, the decrease in the number of packets which need to be buffered at the receiving endpoint of the MPTCP connection, the required buffer size at the receiving endpoint is reduced. This is particularly advantageous for consumer devices such as mobile terminals, mobile phones, smartphones, and residential gateways or home gateways, which implement an MPTCP proxy as receiving endpoint of the MPTCP connection.

Embodiments of the invention are advantageous in all kinds of scenarios in which there is a considerable and/or persistent difference in forward delay between two or more distinct TCP connections, which are combined into an MPTCP connection between two endpoints, a sending endpoint and a receiving endpoint. Oftentimes the two or more TCP connections are carried over distinct access networks and different types of access technology, giving rise to a difference in forward delay. For instance, one of the TCP connections may be carried over a wired access network, such as a DSL connection, whereas the other TCP connection is carried over a cellular access network, such as an LTE network. This is, e.g., the case for hybrid-access solutions which are frequently implemented in residential gateways for deployment at customer premises. Besides the residential gateway as one of the endpoints of the MPTCP connection, an MPTCP proxy in the ISP network acts as the second endpoint. In such scenario, the ISP oftentimes applies a policy for directing traffic over the "cheaper" access network, typically the DSL link incurring lower costs, whereas excess traffic is sent over the cellular link if the DSL access is congested or unavailable. Due to the lower forward delay of the cellular access network, there is a risk for HOL delivery and resulting jitter, unless the difference in forward delay is taken into account when scheduling packets for transport over the MPTCP connection. In the present context, scheduling packets for transport is to be understood as encompassing selecting, for each packet which is to be transmitted over the MPTCP connection, one of the TCP connections, i.e., MPTCP subflows, for transmitting the packet, in addition to queueing, or buffering, packets for transmission at a later point in time. Scheduling is typically performed at a network node acting as the sending endpoint of the MPTCP connection.

As a further example, embodiments of the invention may also be envisaged for solutions utilizing an MPTCP connection combining a TCP connection over a cellular access network and a TCP connection over a WiFi/WLAN network. In such case, one of the endpoints of the MPTCP connection is typically an MPTCP-capable mobile terminal, such as a mobile phone, a smartphone, a tablet, or a laptop, and the other endpoint is an MPTCP proxy deployed in the operator network.

According to an embodiment of the invention, the difference in forward delay is acquired by determining a difference between a sending time difference and a receiving time difference. The sending time difference is the time difference between sending a first packet over the first TCP connection and sending a second packet over the second TCP connection by the sending endpoint of the MPTCP connection. The receiving time difference is the time difference between receiving the first packet and receiving the second packet at the receiving endpoint of the MPTCP connection.

According to an embodiment of the invention, the difference in forward delay is acquired by determining the difference in forward delay based on a difference in RTT between the first TCP connection and the second TCP connection. For instance, the difference in forward delay may be determined by calculating a fraction of the difference in RTT, e.g., as half of the difference in RTT. Alternatively, a factor deviating from one-half may be used to take into account any knowledge about an asymmetry between the forward path and the backward path of a connection, i.e., a difference between forward and backward delay for the same network path. The difference in RTT may be determined as the difference between an RTT for the first TCP connection and an RTT for the second TCP connection. According to an embodiment of the invention, packets are scheduled for transport over the second TCP connection in response to determining that an amount of packets scheduled for transport over the first TCP connection is above an upper threshold. Advantageously, in a hybrid-access scenario the ISP may implement a traffic steering policy which has the effect to

predominantly transport traffic, i.e., packets, over the "cheaper link" (the first TCP connection), e.g., a wired connection such as DSL, and utilizing any additional TCP connection(s), e.g., over a cellular access network incurring a higher cost, only for excess traffic. For instance, packets may be scheduled for transport over the second TCP connection in response to detecting that a queue or buffer which is associated with the first TCP connection is full, or has reached a threshold for a maximum amount of data in the queue or buffer. Alternatively, packets may be scheduled for transmission over the second TCP connection in response to detecting that the first TCP

connection is congested, e.g., by determining that the forward delay or RTT for the first TCP connection, or a change thereof, exceeds a threshold for an absolute value for forward delay or RTT, or a change thereof. Optionally, packets may be scheduled for transmission over the second TCP connection in response to detecting that the first TCP connection has been congested for an extended period of time. Packets which are scheduled for transport over the second TCP connection are delayed by a time interval which is

substantially equal to the difference in forward delay.

According to an embodiment of the invention, packets scheduled for transport over the second TCP connection are re-scheduled for transport over the first TCP connection in response to determining that the amount of packets scheduled for transport over the first TCP connection is below a lower threshold. With reference to the hybrid-access scenario described above, if the status of a queue or buffer which is associated with the first TCP connection (the "cheaper link") has improved, e.g., because packets have been de-queued from the queue/buffer or congestion has been alleviated, packets which have been scheduled for transport over the second TCP connection and are still present in the queue/buffer associated with the second TCP connection may be re-scheduled for transport over the first TCP connection. This situation may occur since the second TCP connection has a lower forward delay than the first TCP connection, an d consequently delaying packets which are scheduled for transport over the second TCP connection by a time interval substantially equal to the difference in forward delay. Advantageously, this allows scheduling packets for transport over the second TCP connection (the "excess link") in anticipation of a delay in transport over the first TCP connection (the "cheap" link), but eventually transporting such packets over the first TCP connection, rather than over the second TCP connection, if the status of the first TCP connection, or its queue/buffer, has improved before all packets in the queue/buffer associated with the second TCP connection have been de-queued, i.e., transmitted over the second TCP connection.

Even though advantages of the invention have in some cases been described with reference to embodiments of the first aspect of the invention, corresponding reasoning applies to embodiments of other aspects of the invention.

Further objectives of, features of, and advantages with, the invention will become apparent when studying the following detailed disclosure, the drawings and the appended claims. Those skilled in the art realize that different features of the invention can be combined to create embodiments other than those described in the following.

Brief description of the drawings

The above, as well as additional objects, features and advantages of the invention, will be better understood through the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the appended drawings, in which:

Fig. 1 illustrates a hybrid-access scenario utilizing MPTCP, in accordance with embodiments of the invention.

Fig. 2 illustrates forward delay, backward delay, and RTT.

Fig. 3 shows a sequence diagram illustrating transporting packets over an MPTCP connection, in accordance with embodiments of the invention.

Fig. 4 illustrates determining a difference in forward delay, in accordance with embodiments of the invention.

Fig. 5 schematically illustrates scheduling and re-scheduling packets for transport over an MPTCP connection, in accordance with embodiments of the invention.

Fig. 6 shows an embodiment of an MPTCP-capable network node for scheduling packets for transport over an MPTCP connection.

Fig. 7 shows another embodiment of an MPTCP-capable network node for scheduling packets for transport over an MPTCP connection, in accordance with another embodiment of the invention.

Fig. 8 shows further embodiments of the MPTCP-capable network node for scheduling packets for transport over an MPTCP connection.

Fig. 9 shows a method of scheduling packets for transport over an MPTCP connection, in accordance with embodiments of the invention.

All the figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested.

Detailed description

The invention will now be described more fully herein after with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

In Fig. 1 , a typical scenario for utilizing MPTCP to increase bandwidth and improve reliability for a network connection between a client 101 and a server 104 is exemplified, which network connection is utilized for

transporting packets, in particular TCP or UDP packets, from client 101 to server 104, or vice versa. Throughout this disclosure, embodiments of the invention are exemplified for an end-to-end TCP connection between client 101 and server 104, i.e., it is assumed that client 101 and server 104 are capable of communicating via TCP. The terminology "client" and "server" as used herein refers to the distinct roles of the two peers in a connection, wherein the client requests data from the server, or initiates a service which is provided by the server.

In the present context, client 101 may be any device capable of effecting communications with server 104 providing content to client 101 , such as a web server, an email server, a media server, a node of a Content Distribution Network (CDN), or the like. For instance, client 101 may be a mobile terminal, a User Equipment (UE), a mobile phone, a smartphone, a tablet, a laptop, or a personal computer. Communications between client 101 and server 104 may be effected though one or more communications networks, wired or wireless, or a combination thereof. In particular, a communications network may be any one of a cellular radio access network, such as GSM, UMTS, LTE, or WiMAX, a Local Area Network (LAN), a WLAN/WiFi access network, an Ethernet network, a corporate network, and the Internet.

In the scenario illustrated in Fig. 1 , it is assumed that communications between client 101 and server 104, i.e., the transport of TCP packets, is effected over a sequence of links, e.g., a first link between client 101 and a first MPTCP proxy 102 over a LAN 1 1 1 , a second link between first MPTCP proxy 102 and a second MPTCP proxy 103 over an MPTCP connection encompassing two disjoint network paths 1 12 and 1 13, and a third link between second MPTCP proxy 103 and server 104 over a Wide Area Network (WAN) 1 14, e.g., the Internet.

In general, a proxy is a network node or an application which acts as an intermediary for requests from clients, such as client 101 , to a server, such as server 104. For the purpose of elucidating embodiments of the invention, MPTCP proxies 102 and 103 are herein described and illustrated as network nodes which are separate from client 101 and server 104, respectively. This is, e.g., the case in hybrid-access scenarios where (client- side) first MPTCP proxy 102 is implemented in a residential gateway deployed by an ISP on the premises of a customer, and second MPTCP proxy 103 is deployed in the ISP's network, acting as Concentrator for the distinct MPTCP subflows 1 12 and 1 13. It will be appreciated that either of first MPTCP proxy 102 and second MPTCP proxy 103, or both, may alternatively be combined with client 101 and server 104, respectively. As an example, a client 101 in the form of a mobile terminal, mobile phone, smartphone, tablet, laptop, or the like, may implement the MPTCP protocol stack and accordingly act as first MPTCP proxy 102, e.g., for combining two distinct TCP connections which are carried over a cellular access network and a WLAN/WiFi network, respectively. Further, server 104 may also implement the MPTCP protocol stack, acting as second MPTCP proxy 103. Accordingly, throughout this disclosure client 101 , first MPTCP proxy 102, second MPTCP proxy 103, and server 104, are described as functional units which assume certain roles in an end-to-end TCP connection between client 101 and server 104, but these functional units do not need to be implemented as separate network nodes. With reference to Fig. 1 , an MPTCP proxy is provided at both endpoints 102 and 103 of an MPTCP connection, as is illustrated in Fig. 1 . More specifically, on the client side, first MPTCP proxy 102 may be provided as an intermediate protocol layer for interfacing requests from applications at higher protocol layers, originating from client 101 , for setting up a TCP connection with server 104. Alternatively, first MPTCP proxy 102 may be deployed as a separate network node, e.g., implemented in a residential gateway. In response to such requests, MPTCP proxy 102 may, acting as an MPTCP client, establish an MPTCP connection with an MPTCP-capable server, or alternatively with second MPTCP proxy 103 deployed upstreams in the network and acting as MPTCP server. Correspondingly, second MPTCP proxy 103 which is deployed on the network side, i.e., upstream from client 101 , may act as an MPTCP server to communicate with an MPTCP- capable client, or alternatively with client-side first MPTCP proxy 102 acting as MPTCP client.

MPTCP aims at allowing an end-to-end TCP connection, e.g., between client 101 and server 104, to utilize multiple network paths via access networks 122 and123, respectively, to maximize resource usage and increase redundancy. This is achieved by extending the standard TCP stack such that it presents a standard TCP interface to applications requesting an end-to-end TCP connection, while in fact spreading data across several MPTCP subflows, i.e., separate TCP connections 1 12 and 1 13 which are associated with separate TCP interfaces using disjoint network paths 122 and 123, respectively. It will be appreciated that an MPTCP connection may comprise any number of MPTCP subflows other than two, including one.

In the scenario depicted in Fig. 1 , it is assumed that one of the subflows, first TCP connection 1 12, is carried over a wired access

network 122, e.g., a DSL network, whereas an additional subflow, second TCP connection 1 13, is carried over a cellular access network 123, such as a UMTS or LTE network. This situation is common in hybrid-access solutions which ISPs utilize for providing customers with Internet access having increased bandwidth and reliability. Typically, a home gateway, or residential gateway, which is deployed at premises of the customer, implements the client-side MPTCP proxy 102 for providing Internet access over two access networks simultaneously, e.g., DSL network 122 and a UMTS or LTE network 123. Oftentimes, IPSs configure their residential gateways to apply some form of policy to control the distribution of traffic over the access networks 122 and 123. For instance, a residential gateway acting as MPTCP proxy 102 may predominantly utilize the cheaper link over DSL access network 122 for most of the traffic between client 101 and server 104, in either direction, whereas the additional wireless access network 123 is utilized for surplus or excess traffic, or when DSL access network 122 suffers from congestion or outage.

Whilst MPTCP has been designed as an extension to TCP, it has been suggested (see, e.g., draft-boucadair-mptcp-plain-mode-07) to extend the multi-path capabilities offered by MPTCP also to UDP flows, i.e., to transport UDP packets, or UDP datagrams, between a UDP-capable client 101 and a UDP-capable sever 104 by means of an MPTCP connection. The MPTCP connection may either be set up statically, e.g., as configured by an ISP managing a residential gateway acting as MPTCP client 102, or in response to receiving a UDP packet as an indicator that client 101 has initiated or requested a UDP-based service. Typically, UDP is utilized by applications which do not require the same level of reliability and congestion control mechanisms as are provided by TCP. Numerous key Internet applications use UDP, including the Domain Name System (DNS), the Simple Network Management Protocol (SNMP), the Routing Information Protocol (RIP), and the Dynamic Host Configuration Protocol (DHCP). In addition, real-time traffic, such as voice and video, is generally transmitted using UDP. This is the case since real-time video and audio streaming services, as well as coders and decoders used for implementing such services, are designed to handle occasional lost packets, resulting in a slight degradation in quality which is preferred to the oftentimes large delays caused by packets retransmission. Further, some Virtual Private Network (VPN) solutions such as OpenVPN may use UDP while implementing reliable connections and error checking at the application level. Whereas transport control functions like ordered transfer, retransmission of lost packets, flow control, and congestion control, which are commonly used in TCP are absent in UDP, it has been realized that such functionality may advantageously be used for UDP transport. In particular, this is the case since the issue of HOL delivery, which may arise if UDP packets are transported over an MPTCP connection combining MPTCP subflows with different forward delays, may have a negative impact on certain UDP services and applications which are sensitive to jitter.

With reference to Fig. 2, forward delay is the time interval between sending 201 a packet from a sending network node ("Sender" in Fig. 2), such as first MPTCP proxy 102 (second MPTCP proxy 103), and receiving 202 the packet at a receiving network node ("Receiver" in Fig. 2), such as second MPTCP proxy 103 (first MPTCP proxy 102). Correspondingly, backward delay is the time interval between sending 202 a packet and receiving 203 the packet in the reverse direction. In the case of TCP, the packet

transported in reverse direction is oftentimes a TCP ACK packet which is transmitted by the Receiver in response to receiving a TCP packet, acknowledging that the packet has been successfully received. The RTT is the total of forward delay and backward delay.

Embodiments of the invention are advantageous in that they provide a solution for mitigating problems which arise when packets during transport from their source to a destination are spread over two or more distinct network paths exhibiting a difference in forward delay, such as HOL delivery and jitter. This is achieved by delaying packets which are to be transported over the TCP connection, or MPTCP subflow, having the lower forward delay by a time interval which is substantially equal to the difference in forward delay between the distinct TCP connections, or MPTCP subflows. Thereby, if a sequence of packets is transmitted in-order by the sending endpoint of the MPTCP connection, e.g., first MPTCP proxy 102 (second MPTCP

proxy 103), the fraction of packets which are received out-of-order at the receiving endpoint of the MPTCP connection, second MPTCP proxy 103 (first MPTCP proxy 102), is reduced as compared to solutions which do not take the difference in forward delay into account.

It will be appreciated that packets which are scheduled for transport over second TCP connection 1 13, having the lower forward delay, may be delayed by a time interval which deviates from the determined difference forward delay. In this respect, the term "substantially equal" is to be

understood such that the delay time interval may be equal to the determined difference in forward delay, or deviate from the determined difference in forward delay by a small amount, e.g., to take into account any corrections for removing the effect of systematic errors, or the like. Further, since the determined difference in forward delay is used as an estimate for a difference in forward delay which is experienced by packets which are scheduled for transport, but have not yet been transmitted, the delay time interval may be set to a value which takes into account any observed systematic change in the difference in forward delay.

In the following, embodiments of the invention are described with reference to the sequence diagram shown in Fig. 3, which illustrates establishing a TCP connection between client 101 and server 104,

encompassing an MPTCP connection between first MPTCP proxy 102 and second MPTCP proxy 103, and transporting TCP packets over the MPTCP connection.

First, a TCP connection between client 101 and first MPTCP

proxy 102 is established by means of a three-way handshake 301 , as is known in the art. Three-way handshake 301 involves a TCP SYN packet which is transmitted from client 101 to first MPTCP proxy 102, which responds with a TCP SYN-ACK packet. Subsequently, client 102 transmits a TCP ACK packet to first MPTCP proxy 102.

After the TCP connection between client 101 and first MPTCP proxy 102 has been established, or directly after the TCP SYN packet is received from client 101 as part of the three-way handshake 301 , first TCP proxy 102 initiates establishing the MPTCP connection by setting up a first TCP connection 1 12 as first MPTCP subflow between first MPTCP proxy 102 and second MPTCP proxy 103 over one of the available access

networks 122 and 123, in this case DSL network 122. Setting up first TCP connection 1 12 is similar to setting up a regular TCP connection and involves a three-way handshake 302, as is known in the art. More specifically, first MPTCP proxy 102 sends a TCP SYN packet to second MPTCP proxy 103, which responds with a TCP SYN-ACK packet. Subsequently, first MPTCP proxy 102 transmits a TCP ACK packet to second MPTCP proxy 103. In addition to the information which is carried in the IP header and the TCP header of each packet, in particular source IP address/port number and destination IP address/port number, the TCP SYN, SYN-ACK, and ACK, packets 302 which are exchanged between first MPTCP proxy 102 and second MPTCP proxy 103 during the three-way handshake 302 also carry additional information indicating that first MPCTCP proxy 102 is "MPTCP capable". This additional information is carried as an MPTCP Option

"MP_CAPABLE" in the Options" field of the TCP header, as is known in the art (see, e.g., "TCP Extensions for Multipath Operation with Multiple

Addresses", IETF RFC 6824).

More specifically, a number of MPTCP Options are defined in

RFC 6824 for the purpose of signaling MPTCP operations. All MPTCP operations are signaled as variable-length options using the optional Options field of the TCP header, herein referred to as MPTCP Options. The TCP Options field has a defined format which comprises a "Kind" information field, a "Length" information field, a "Subtype" information field, and a field for subtype-specific data. The Internet Assigned Numbers Authority (IANA) has reserved TCP Options Kind "30" for all MPTCP operations. Individual MPTCP messages are identified by a "Subtype", a four-bit field, the values of which are listed in a sub-registry entitled "MPTCP Option Subtypes" under the

"Transmission Control Protocol (TCP) Parameters" registry maintained by the IANA. The currently defined Subtypes are as follows:

Values 0x8 through Oxe are currently unassigned. Details about MPTCP operations and MPTCP Options can be found in RFC 6824.

Further with reference to Fig. 3, if additional network paths are available between first MPTCP proxy 102 and second MPTCP proxy 103, such as via cellular access network 123, additional TCP connections may be created as MPTCP subflows on these paths and combined with an existing subflow or subflows. The combined subflows appear as a single end-to-end TCP connection to the applications at both ends, i.e., client 101 and server 104. For instance, in the scenario depicted in Fig. 1 , a second TCP connection 1 13 may be established as a second MPTCP subflow, utilizing a path through cellular access network 123, and combined with the first subflow over TCP connection 1 12.

Establishing an additional TCP connection for use as additional

MPTCP subflow is similar to initiating a normal TCP connection, but requires a four-way handshake 304, as is described in RFC 6824. More specifically, in addition to the three-way handshake which is known from conventional TCP, i.e., sending a TCP SYN packet from first MPTCP proxy 102 to second MPTCP proxy 103, sending a TCP SYN-ACK packet from second MPTCP proxy 103 to first MPTCP proxy 102, and sending a TCP ACK packet from first MPTCP proxy 102 to second MPTCP proxy 103, an additional TCP ACK packet is sent from first MPTCP proxy 102 to second MPTCP proxy 103. All TCP packets which are sent during the four-way handshake 304 carry additional information in the TCP Options field of their TCP header for identifying TCP connection 1 13 as an additional MPTCP subflow which should be combined into an MPTCP connection at both first MPTCP proxy 102 and second MPTCP proxy 103. This is achieved by including

MPTCP Option "MP_JOIN" in the TCP Options. It is noted that setting up and combining additional TCP connections with already existing subflows may require keys which are exchanged during a handshake procedure, as is described in RFC 6824.

Simultaneously with setting up the MPTCP connection 302/304 between first MPTCP proxy 102 and second MPTCP proxy 103, or directly after the TCP SYN packet is received from first MPTCP proxy 102 as part of the three-way handshake 302, second TCP proxy 103 initiates

establishing 303 a TCP connection with server 104 by means of a three-way handshake 303. More specifically, second MPTCP proxy 103 sends a TCP SYN packet to server 104, which responds with a TCP SYN-ACK packet. Subsequently, second MPTCP proxy 103 transmits a TCP ACK packet to server 104.

After first TCP connection 1 12 has been established, in addition to the TCP connections between client 101 and first MPTCP proxy 102 as well as between second MPTCP proxy 103 and server 104, transport of TCP packets from client 101 to server 104 (upstreams) and vice versa, from server 104 to client 101 (downstreams), may commence via the MPTCP connection between first MPTCP proxy 102 and second MPTCP proxy 103. If second TCP connection 1 13 has been successfully established and combined with first TCP connection 1 12 at both endpoints 102 and 103 of the MPTCP connection, TCP packets may be transported between first MPTCP proxy 102 and second MPTCP proxy 103 via either one of TCP

connections 1 12 and 1 13. Further MPTCP subflows may be stablished, and existing subflows may be released or modified, as is described in RFC 6824.

Further with reference to Fig. 3, transporting TCP packets between client 101 and server 104, and vice versa, is described in the following. For the purpose of elucidating embodiments of the invention, it is assumed that client 101 requests data from server 104, which subsequently is delivered from server 104 to client 101 .

To this end, after an end-to-end TCP connection has been established between client 101 and server 104, client 101 transmits a request 31 1 to first MPTCP proxy 102, which forwards 312 the request over one of the available TCP connections 1 12 and 1 13 to second MPTCP proxy 103, which in turn forwards 313 the request to server 104. As an example, client 101 may request a web page from server 104 acting as web server, using the

HyperText Transfer Protocol (HTTP). In this case, requests 31 1-313 are TCP packets carrying HTTP GET requests identifying a resource which client 101 wants to retrieve, e.g., a web page "index. html" which is hosted by web server 104.

In response to receiving request 313, server 104 starts sending "index.html" to client 101 , via second MPTCP proxy 103 and first MPTCP proxy 102, as TCP packets 321 carrying HTTP 200 OK responses. If the size of "index.html" is too large to be accommodated in a single TCP packet, e.g., due to a limitation in Maximum Transmission Unit (MTU) size along the network path between server 104 and client 101 , server 104 fragments "index.html" into several TCP packets 321 .

As a further example, client 101 may request a media segment, e.g., a video segment, from server 104. Due to the size of video segments which typically exceed the maximum MTU size, video segments are sent by server 104 in multiple TCP packets 321 .

In response to receiving TCP packets 321 from server 104, second MPTCP proxy 103 starts transmitting 322 the TCP packets to first MPTCP proxy 102. As is illustrated in Fig. 3, TCP packets 322 are transmitted over first TCP connection 1 12, i.e., the DSL link. This may, e.g., be achieved by implementing a policy in second MPTCP proxy 103 for steering traffic which is received for transport over the MPTCP connection as configured by an operator of second MPTCP proxy 103, e.g., an ISP. For the scenario illustrated in Fig. 1 , first TCP connection 1 12 over a DSL access network 122 typically has a lower cost associated with it than a cellular network 123, over which the second TCP connection 1 13 is established. By means of a traffic policy implemented in one or both endpoints 1 12 and 1 13 of the MPTCP connection, packets may be predominantly transported over the cheaper link.

It will be appreciated that embodiments of the invention are not limited to transporting traffic predominantly over one of the links of an MPTCP connection, e.g., the cheaper link. Rather, embodiments of the invention may be envisaged which distribute traffic evenly over the available TCP

connections, e.g., in a round-robin fashion, or by transporting traffic predominantly over the link having higher bandwidth, lower latency, less congestion, or the like.

Accordingly, second MPTCP proxy 103 may start transmitting TCP packets received 321 from server 104 to first MPTYCP proxy 102 also over second TCP connection 1 13, i.e., over cellular access network 123. TCP packets 332 which are to be transported over second TCP connection 1 13 are delayed 331 by a time interval which is substantially equal to a difference in forward delay between first TCP connection 1 12 and second TCP connection 1 13. The difference in forward delay may be acquired in different ways, as is described in the following. As is illustrated in Fig. 4, the difference in forward delay may be acquired by determining the difference between a sending time difference As, which is the time difference between sending 41 1 , by a sending network node ("Sender" in Fig. 4), a first packet over first TCP connection 1 12 and sending 421 a second packet over second TCP connection 1 13, and a receiving time difference Ar, which is the time difference between

receiving 412, at a receiving network node ("Receiver" in Fig. 4), the first packet and receiving 422 the second packet at a receiving network node. Then, the time difference in forward delay can be calculated as the difference between the sending time difference As and the receiving time difference Ar, i.e., as As - Ar. Advantageously, any difference in clock time between the sending network node and the receiving network node cancels out. The time difference in forward delay may be calculated by the sending network node, utilizing receiving timestamps (time at 412 and 422) which are piggybacked with TCP ACK packets sent from the Receiver to the Sender. Alternatively, the time difference in forward delay may also be calculated by the receiving network node, utilizing sending timestamps (time at 41 1 and 421 ) which are piggybacked with TCP packets sent from the Sender to the Receiver, and subsequently reported to the sending network node, e.g., piggybacked in a TCP packet. For more details, reference is made to "Forward Delay-based Packet Scheduling Algorithm for Multipath TCP", by T.-A. Le and L. X. Bui, in OALib Journal Computer Science, http://arxiv.org/abs/1501 .03196v1 (2015).

To this end, first MPTCP proxy 102 and second MPTCP proxy 103 may determine the time difference in forward delay while sending TCP packets in either direction, i.e., either from first MPTCP proxy 102 to second MPTCP proxy 103, or vice versa, and sending TCP ACK packets in the reverse direction. For instance, the endpoints 102 and 103 may

determine 341/342 the time difference in forward delay during setup 302 of first TCP connection 1 12 and setup 304 of second TCP connection 1 13. More specifically, during setup 302 of first TCP connection 1 12, first MPTCP proxy 102 (which corresponds to the "Sender" in Fig. 4) may include the respective sending times of the TCP SYN packets which initiate three- way handshake 302 and four-way handshake 304, respectively, in the TCP Options field of the respective TCP SYN packet. Upon receipt, second MPTCP proxy 103 (which corresponds to the "Receiver" in Fig. 4) may calculate 342 the time difference in forward delay as the difference in sending time difference (time difference between sending the TCP SYN packet during setup 304 of second TCP connection 1 13 and sending the TCP SYN packet during setup 302 of first TCP connection 1 12) and receiving time difference (time difference between receiving the TCP SYN packet over second TCP connection 1 13 and receiving the TCP SYN packet over first TCP

connection 1 12). In this approach, second MPTCP proxy 103 receives the sending time stamps from first MPTCP proxy 102 piggybacked in the TCP SYN packets. After calculating 342 the time difference in forward delay, second MPTCP proxy 103 may signal the calculated time difference to first MPTCP proxy 102, e.g., piggybacked in one of the TCP SYN-ACK packets ("ACK" in Fig. 4) sent during three-way handshake 302 and four-way handshake 304, respectively. Alternatively, the calculated time difference in forward delay may be signaled to first MPTCP proxy 102 at a later stage. As a further alternative, the calculated time difference in forward delay may be signaled to a network node operated by the ISP, e.g., a network node for monitoring network conditions and/or implementing traffic policies.

An alternative solution is to determine 341 the time difference in forward delay at first MPTCP proxy 102 (which corresponds to the "Sender" in Fig. 4). This may be achieved by storing, during setup 302 of first TCP connection 1 12 and setup 304 of second TCP connection 1 13, the respective sending time stamps of the TCP SYN packets which initiate three-way handshake 302 and four-way handshake 304, respectively, at first MPTCP proxy 102. Upon receipt of the TCP SYN packets, second MPTCP proxy 103 responds by sending TCP SYN-ACK packets to first MPTCP proxy 103, over first TCP connection 1 12 and second TCP connection 1 13, respectively, with the respective receiving time stamps of the TCP SYN packets included in the TCP Options of the TCP SYN-ACK packets. Then, based on the stored sending time stamps and the receiving time stamps carried in the TCP SYN- ACK packages received from second MPTCP proxy 103, first MPTCP proxy 102 may calculate 341 the time difference in forward delay as the difference in sending time difference (time difference between sending the TCP SYN packet during setup 304 of second TCP connection 1 13 and sending the TCP SYN packet during setup 302 of first TCP connection 1 12) and receiving time (time difference between receiving the TCP SYN packet over second TCP connection 1 13 and receiving the TCP SYN packet over first TCP connection 1 12). Optionally, the calculated time difference in forward delay may be signaled to a network node operated by the ISP, e.g., a network node for monitoring network conditions and/or implementing traffic policies.

According to an alternative embodiment of the invention, the difference in forward delay may be determined based on a difference in RTT between first TCP connection 1 12 and second TCP connection 1 13.

Preferably, the difference in forward delay is determined by calculating a fraction of the difference in RTT. In particular, the difference in forward delay may be determined as half of the difference in RTT, i.e., by multiplying the difference in RTT by 0.5. Alternatively, knowledge about a difference in forward delay and backward delay may be taken into account by using a factor deviating from 0.5. More specifically, the difference in forward delay may be determined by measuring the RTT for first TCP connection 1 12 and the RTT for second TCP connection 1 13, and determining the difference in RTT by subtracting the two values. Subsequently, the difference in forward delay is determined by multiplying the difference in RTT by a factor. The factor may either be determined by first MPTCP proxy 102 and second MPTCP proxy 103, by measuring forward and backward delay as described hereinbefore and calculating a factor representing the relation between forward and backward delay, or forward delay and RTT. Alternatively, the factor may be configured by the ISP, e.g., based on a known, and preferably substantially constant, asymmetry between forward and backward delay. Advantageously, measuring the RTT for a TCP connection does not require piggybacking sending time stamps or receiving time stamps in TCP packets which are transmitted between first MPTC proxy 102 and second MPTCP proxy 103, or vice versa. Rather, with reference to Fig. 4, the RTT for first (second) TCP connection 1 12 (1 13) may simply be determined as the time difference between sending 41 1 (412) a TCP packet by first MPTCP proxy 102 ("Sender") and receiving 413 (423) a corresponding TCP ACK packet at first MPTCP proxy 102. Note that the RTT can be determined for a single TCP connection at a time, and is not affected by any difference in clock time between the sending endpoint and the receiving endpoint.

The respective RTTs for TCP connections 1 12 and 1 13 may alternatively be determined by an out-of-band approach similar to that used by the known "ping" utility. This may be achieved by sending ping packets from one of the endpoints 102 and 103 of the MPTCP connection to the other endpoint of the MPTCP connection, and receiving corresponding ping responses. Thereby, the RTT may be measured separately for first TCP connection 1 12 and second TCP connection 1 13, as the difference between the sending time of a ping packet and the receiving time of a ping response.

Even though determining the difference in forward delay or RTT has hereinbefore been described for packets which are transmitted upstreams, i.e., from first MPTCP proxy 102 to second MPTCP proxy 103, embodiments of the invention may easily be envisaged which determine the difference in forward delay or RTT in the reverse direction, i.e., downstreams. It will also be appreciated that the difference in forward delay or RTT may be

determined in connection with any TCP packets which are transmitted between the endpoints 102 and 103 of the MPTCP connection, and corresponding TCP ACK packets which are transmitted in the reverse direction, other than during establishing 302/304 TCP connections 1 12 and 1 13. For instance, the difference in downstream forward delay may be determined 343/344 when TCP packets received from server 104 are transported 322/332 over the MPTCP connection, via first TCP

connection 1 12 and second TCP connection 1 13.

To this end, differences in forward delay or RTT may be determined during setup of the MPTCP connection, i.e., during setup 302 of first TCP connection 1 12 and setup 304 of second TCP connection 1 13, continuously during the lifetime of the MPTCP connection, i.e., every time TCP packets are sent either upstreams or downstreams, at regular time intervals, or in response to detecting congestion, degradation in link capacity, or increase in latency, of the preferred link.

In an alternative embodiment of the invention, the difference in forward delay may also be configured at first MPTCP proxy 102 and/or second MPTCP proxy 103, e.g., by the ISP, or received from a network node for monitoring traffic conditions and/or implementing traffic policies. This is advantageous if the forward delay is known to be substantially constant, alleviating the endpoints 102 and 103 of the MPTCP connection to measure the difference in forward delay or RTT.

Depending on a traffic policy which is enforced by a sending endpoint of the MPTCP connection, such as second MPTCP proxy 103, TCP packets 321 received from server 104 may distributed over the available TCP connections 1 12 and 1 13 in different ways. For instance, second MPTCP proxy 103 may predominantly transport TCP packets 322 received from server 104 over first TCP connection 1 12, i.e., DSL access network 122, and start scheduling 330 TCP packets 332 received for transport over second TCP connection 1 13 in response to determining that an amount of packets scheduled for transport over first TCP connection 1 12 is above an upper threshold, as is described below, or in response to determining that first TCP connection 1 12 is congested, unavailable, suffers from an increased latency or low throughput, or the like.

Fig. 5 further elucidates scheduling and re-scheduling packets for transport over an MPTCP connection, in accordance with embodiments of the invention. It is assumed here that packets which are received at a sending endpoint 500 of the MPTCP connection, such as packets 321 received at second MPTCP proxy 103 from server 104, are scheduled for transport over either of two available MPTCP subflows, first TCP

connection 1 12 and second TCP connection 1 13. This may, e.g., be achieved by placing incoming packets 321 in either of two queues, or buffers, a first queue 501 which is associated with first TCP connection 1 12 (over DSL access network 122) and a second queue 502 which is associated with second TCP connection 1 13 (over cellular access network 123). In the example illustrated in Fig. 5 (upper part), incoming packets 321 are first placed in first queue 501 (packets 1-7). In response to determining that an amount of packets scheduled for transport over first TCP connection 1 12, packets 1-7 in first queue 501 , is above an upper threshold ("Maximum queue size" in Fig. 5), subsequent packets 321 (packets 8-10) are

scheduled 330 for transport over second TCP connection 1 13, i.e., placed in second queue 502. The packets in second queue 502 are delayed 331 ("At" in Fig. 5) by a time interval substantially equal to the difference in forward delay. The de-queueing of packets which are placed in queues 501 and 502, i.e., the transmission of packets over the respective TCP connection 1 12 and 1 13, respectively, is under control of a scheduler and is based on availability of resources on the respective TCP connection.

In the upper part of Fig. 5, it is illustrated that packets placed in first queue 501 are scheduled for transport 322 over first TCP connection 1 12. Correspondingly, packets placed in second queue 502 are scheduled for transport 332 over second TCP connection 1 13. More specifically, the packets in second queue 502 (packets 8-10) are scheduled for transmission at a point in time which is later than the time of transmission of the last packet in first queue 501 (packet 7), by a time interval which is substantially equal to the difference in forward delay. Thereby, in-order delivery at the receiving endpoint, in particular of packets 7 and 8, is achieved.

In the lower part of Fig. 5, it is illustrated that packets 1 and 2 have been de-queued 322 from queue 501 , i.e., transmitted over first TCP connection 1 12. As a result, sending endpoint 500 may re-schedule 334 packets which are scheduled 330 for transport over second TCP

connection 1 13, i.e., placed in queue 502, for transport over first TCP connection 1 12 in response to determining that the amount of packets scheduled for transport over first TCP connection 1 12 (packets 3-7 in queue 501 ) is below a lower threshold, e.g., below the maximum queue size. In the example illustrated in Fig. 5, two packets (packets 8 and 9) may be moved from second queue 502 to first queue 501 .

The embodiment described hereinbefore, with reference to Fig. 5, is particularly advantageous in hybrid-access scenarios in which packets are predominantly transported over the "slower" of two available TCP

connections (the TCP connection having the larger forward delay or RTT), in this case first TCP connection 1 12 over DSL access network 122, which oftentimes is also the connection incurring lower cost. In situations when first queue 501 associated with first TCP connection 1 12 exceeds a maximum queue size, which may occur as a result of congestion, an increase in latency, or a decrease in bandwidth, packets can be scheduled for transport over the second "faster" TCP connection 1 13 (having the lower forward delay or RTT), where they are delayed by a time interval substantially equal to the forward delay, to mitigate HOL delivery.

Now, if the status of first queue 501 has improved before the packets in second queue 502, which are scheduled for transmission over second TCP connection 1 13, have been transmitted, some or all of the remaining packets in second queue 502 can be moved to, i.e., re-scheduled, to first queue 501 . Advantageously, this allows scheduling packets for transport over an "excess" link which oftentimes incurs a higher cost, in this case second TCP connection 1 13, in anticipation of a delay in transport over first TCP connection 1 12, but eventually transporting such packets over the "cheaper" first TCP connection 1 12, rather than over second TCP connection 1 13.

In the following, embodiments of a network node for scheduling packets for transport over an MPTCP connection are described with reference to Figs. 6 to 8.

In Fig. 6, an embodiment 600 of a network node for scheduling packets for transport over an MPTCP connection is illustrated as comprising a first network interface 601 , a second network interface 602, an optional third network interface 603, a processor 604, and a memory 605. First network interface 601 and second network interface 602 are operative for communicating by means of TCP via a respective access network, e.g., with an MPTCP proxy. For instance, first network interface 601 may be operative for communicating over a wired access network, such as a DSL network 122, and second network interface 602 may be operative for communicating over a wireless access network, such as cellular access network 123. Optional third network interface 603 may be operative for communicating with client 101 over LAN 1 1 1 , e.g., a WLAN/WiFi access network, or with server 104 over a WAN 1 14, e.g., the Internet. Processor 604 may, e.g., be a general purpose processor which is operative to perform embodiments of the invention described herein with reference to first MPTCP proxy 102, second MPTCP proxy 103, or both, when executing instructions 606, i.e., a computer program, comprised in memory 605. Memory 605 may be a Random Access Memory (RAM), a Read-Only Memory (ROM), a Flash memory, a Hard-Disk Drive (HDD), or any other type of computer-readable data storage.

In particular, network node 600 is operative to acquire a difference in forward delay between a first TCP connection and a second TCP connection of the at least two TCP connections, wherein the second TCP connection has a lower forward delay than the first TCP connection, and delay packets scheduled for transport over the second TCP connection by a time interval substantially equal to the difference in forward delay. The difference in forward delay may, e.g., be acquired by determining a difference between a sending time difference, which is the time difference between sending a first packet over the first TCP connection and sending a second packet over the second TCP connection by the network node, and a receiving time

difference, which is the time difference between receiving the first packet and receiving the second packet at a receiving endpoint of the MPTCP

connection. Alternatively, the difference in forward delay may be acquired by determining the difference in forward delay based on a difference in RTT between the first TCP connection and the second TCP connection.

Optionally, the difference in forward delay may be determined by calculating a fraction of the difference in RTT. The difference in RTT may, e.g., be determined as the difference between an RTT for the first TCP connection and an RTT for the second TCP connection. Network node 600 may further be operative to schedule packets for transport over the second TCP connection in response to determining that an amount of packets scheduled for transport over the first TCP connection is above an upper threshold.

Network node 600 may even further be operative to re-schedule packets scheduled for transport over the second TCP connection for transport over the first TCP connection in response to determining that the amount of packets scheduled for transport over the first TCP connection is below a lower threshold.

In Fig. 7, another embodiment 700 of the network node for scheduling packets for transport over an MPTCP connection is illustrated as comprising a first network interface 701 , a second network interface 702, an optional third network interface 703, a delay acquisition module 704, and a scheduling module 705. Network interfaces 701-703 are similar to network interfaces 601-603, respectively, described with reference to Fig. 6. Delay acquisition module 704 and scheduling module 705 are operative to perform embodiments of the invention described hereinbefore with reference to first MPTCP proxy 102, second MPTCP proxy 103, or both.

In particular, delay acquisition module 704 is operative to acquire a difference in forward delay between a first TCP connection and a second TCP connection of the at least two TCP connections, wherein the second TCP connection has a lower forward delay than the first TCP connection. Scheduling module 705 is operative to delay packets scheduled for transport over the second TCP connection by a time interval substantially equal to the difference in forward delay. Delay acquisition module 704 may further be operative to acquire the difference in forward delay by determining a difference between a sending time difference, which is the time difference between sending a first packet over the first TCP connection and sending a second packet over the second TCP connection by the network node, and a receiving time difference, which is the time difference between receiving the first packet and receiving the second packet at a receiving endpoint of the MPTCP connection. Alternatively, delay acquisition module 704 may further be operative to acquire the difference in forward delay by determining the difference in forward delay based on a difference in RTT between the first TCP connection and the second TCP connection. Optionally, the difference in forward delay may be determined by calculating a fraction of the difference in RTT. The difference in RTT may, e.g., be determined as the difference between an RTT for the first TCP connection and an RTT for the second TCP connection. Scheduling module 705 may further be operative to schedule packets for transport over the second TCP connection in response to determining that an amount of packets scheduled for transport over the first TCP connection is above an upper threshold. Scheduling module 705 may even further be operative to re-schedule packets scheduled for transport over the second TCP connection for transport over the first TCP connection in response to determining that the amount of packets scheduled for transport over the first TCP connection is below a lower threshold.

Network node 600 or 700 may be any one of an MPTCP-capable network node, an MPTCP client, an MPTCP server, a residential gateway, and an MPTCP proxy.

For instance, an embodiment 81 1 of network node 600 or 700 may be comprised in a client device, such as a mobile terminal 810 shown in Fig. 8. In this case, an application communicating with server 104 by means of TCP or UDP may act as client 101 . Mobile terminal 810 may, e.g., be a UE, a mobile phone, a smartphone, a tablet, a laptop, a media player, or the like. The application acting as TCP or UDP client may either be implemented by means of instructions 606 to be executed by processor 604, i.e., as a computer program, or provided as a client module 706. First network interface 601/701 and second network interface 602/702 may, e.g., be a cellular radio access module and a WLAN/WiFi module comprised in mobile terminal 810 (not shown in Fig. 8).

Alternatively, an embodiment 821 of network node 600 or 700 may be provided separate from a client device (client 101 ) or a server (server 104), e.g., comprised in a client-side residential gateway 820 (at the location of first MPTCP proxy 102) or in a network-side proxy server (at the location of second MPTCP proxy 103). In the case of residential gateway 820, first network interface 601/701 may, e.g., be a DSL module which is operative for communicating over a DSL network, via DSL connector 822, and second network interface 602/702 may, e.g., be a cellular radio access module which is operative for communicating over a cellular radio access network, via antenna 823. Third network interface 603/703 may, e.g., be an Ethernet module which is operative for communicating over LAN 1 1 1 , via

connector 824, with client 101 . In case of a network-side proxy server, such as a TCP proxy, an HTTP proxy, a traffic shaping proxy, a Concentrator, a CDN edge node, or the like, third network interface 603/703 may be operative for communicating with server 104 over WAN 1 14, e.g., the Internet, via connector 824.

Network interfaces 601-603 and 701-703, modules 704-706, as well as any additional modules comprised in network node 600 or 700, may be implemented by any kind of electronic circuitry, e.g., any one, or a

combination of, analogue electronic circuitry, digital electronic circuitry, and processing means executing a suitable computer program.

It will be appreciated that embodiments 600 and 700 of the network node are not limited to communicating over the types of access networks described in relation to Figs. 6 to 8, i.e., DSL access networks, WLAN/WiFi access networks, and cellular access network. Rather, embodiments of the invention may be envisaged which utilize any type or types of access networks, including cellular access networks, WLAN/WiFi access networks, and optical networks, which are suitable for use in the context of MPTCP.

In the following, embodiments 900 of a method of scheduling packets for transport over an MPTCP connection are described with reference to Fig. 9. Method 900 may be performed at a sending endpoint of the MPTCP connection, such as an MPTCP-capable network node, an MPTCP client, an MPTCP server, a residential gateway, or an MPTCP proxy.

Method 900 comprises acquiring 901 a difference in forward delay between a first TCP connection and a second TCP connection of the at least two TCP connections, wherein the second TCP connection has a lower forward delay than the first TCP connection, and delaying 903 packets for transport over the second TCP connection by a time interval substantially equal to the difference in forward delay. The difference in forward delay may be acquired 901 by determining a difference between a sending time difference, which is the time difference between sending a first packet over the first TCP connection and sending a second packet over the second TCP connection by the sending endpoint, and a receiving time difference, which is the time difference between receiving the first packet and receiving the second packet at a receiving endpoint of the MPTCP connection.

Alternatively, the difference in forward delay may be acquired 901 by determining the difference in forward delay based on a difference in RTT between the first TCP connection and the second TCP connection.

Optionally, the difference in forward delay may be determined 901 by calculating a fraction of the difference in RTT. The difference in RTT may, e.g., be determined as the difference between an RTT for the first TCP connection and an RTT for the second TCP connection. Method 900 may further comprise scheduling 902 packets for transport over the second TCP connection in response to determining that an amount of packets scheduled for transport over the first TCP connection is above an upper threshold. Method 900 may even further comprise re-scheduling 904 packets scheduled for transport over the second TCP connection for transport over the first TCP connection in response to determining that the amount of packets scheduled for transport over the first TCP connection is below a lower threshold.

It will be appreciated that method 900 may comprise additional, or modified, steps in accordance with what is described throughout this disclosure. Embodiments of method 900 may be implemented as software, such as computer program 606, to be executed by a processing unit 604 comprised in the device, whereby the device is operative to perform in accordance with embodiments of the invention as described herein.

The person skilled in the art realizes that the invention by no means is limited to the embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.