MADDAH-ALI MOHAMMADALI (US)
| US20090222509A1 | 2009-09-03 | |||
| US20070255844A1 | 2007-11-01 | |||
| US6816872B1 | 2004-11-09 |
| I CLAIM: 1. A method for managing transmission of data files to users, comprising: dividing a first file into a plurality of first segments; dividing a second file into a plurality of second segments; sending a first segment to a first user and a different first segment to a second user; sending a second segment to the first user and a different second segment to the second user; combining at least part of a segment sent to the first user with at least part of a segment sent to the second user to produce a combined segment which is of smaller size than the total size of the at least parts of segments before combining; and transmitting the combined segment to the first user and to the second user for each user to recover a segment using the combined segment and at least part of a segment. 2. The method as claimed in claim 1 wherein the first and second segments sent to the first and second users are sent prior to the combined segment being sent to the first and second users. 3. The method as claimed in claim 1 and including dividing a segment into first and second parts; incorporating the first part into the combined segment; and transmitting the second part to the first and second users. 4. The method as claimed in claim 3 and including transmitting the second part with the combined segment. 5. The method as claimed in claim 1 and including choosing the sizes of the first segments and of the second segments to minimize the average rate. 6. The method as claimed in claim 1 and including using the probability that the first file and the second file will be required by the first user and/or the second user in optimizing the sizes of the first segments and of the second segments. 7. Data transmission manager for managing transmission of data files to users, comprising: a divider for dividing a first file into a plurality of first segments and for dividing a second file into a plurality of second segments; transmitter apparatus for sending a first segment to a first user and a different first segment to a second user, and for sending a second segment to the first user and a different second segment to the second user; a combiner for combining at least part of a segment sent to the first user with at least part of a segment sent to the second user to produce a combined segment which is of smaller size than the total size of the at least parts of segments before combining; and said transmitter apparatus being operative to send the combined segment to the first user and to the second user for each of the first and second users to recover a segment using the combined segment and at least part of a segment. 8. The data transmission manager as claimed in claim in claim 7 wherein the divider is operative to divide a segment into first and second parts; the combiner is operative to incorporate the first part into the combined segment; and said transmitter apparatus is operative to send the second part to the first and second users. 9. The data transmission manager as claimed in claim 7 and including a processor for choosing the sizes of the first segments and of the second segments to minimize the average rate. 10. The data transmission manager as claimed in claim 7 and including a processor for using the probability that the first file and the second file will be required by the first user and/or the second user to optimize the sizes of the first segments and of the second segments. |
FIELD OF THE INVENTION
The present invention relates to a method and apparatus for managing transmission of data files to users, and more particularly, but not exclusively, to transmission of data files to be cached locally to a user.
BACKGROUND
A server may hold data files to be sent to a user over a network, which may be, for example, the Internet or some other network. The data files may be media content files, or of some other type, and sent to the user when the user requests them, or when the content creator or provider wants to distribute them, for example. If a user does not require immediate access to a data file, or wishes to keep it available for later access, the data file may be stored in a cache memory local to the user. For example, the cache memory may be a hard drive included in the user equipment or the cache memory may be separately provided and arranged to be readily accessible by the user, for example, by being located nearer the edge of a network than the server. When the user needs a cached file, it may be obtained from the cache memory with reduced or no network resources.
Data transmission may be efficiently managed by sending data files to be cached at times when network resources are at high capacity and/or low cost. For example, a data file may be sent if a WiFi network is available, or when the user is in a low-traffic area or it is a low-traffic time period.
BRIEF SUMMARY
According to a first aspect of the invention, in a method for managing transmission of data files to users, a first file is divided into a plurality of first segments and a second file is divided into a plurality of second segments. A first segment is sent to a first user and a different first segment is sent to a second user. A second segment is sent to the first user and a different second segment is sent to the second user. At least part of a segment sent to the first user is combined with at least part of a segment sent to the second user to produce a combined segment which is of smaller size than the total size of the at least parts of segments before combining. The combined segment is transmitted to the first user and to the second user for each user to recover a segment using the combined segment and at least part of a segment.
The smaller size of the combined segment compared to the total size of the at least parts of segments before combining means that the combined segment requires less space if it is to be cached in a cache, or caches, local to the users. Additionally, as the combined segment is of smaller size, it requires fewer network resources for transmission than if the segments were to be sent individually in uncombined form. This is advantageous even for arrangements in which segments are not cached at a memory cache or caches local to a user.
The user or users may be any node in a network. For example, in a wireless network, a user could be a router or an end user or some other network node.
A method in accordance with the invention may be applied to wireless, fixed line or other types of network and is not limited to one particular technology type.
A method in accordance with the invention may be applied where there are f files and K users, and includes partitioning each file into 2 to the power of K segments, where for each subset of users, there is a segment which is saved on all of the users in that subset.
In one embodiment, the first and second segments sent to the first user may be cached a first cache memory local to the first user. Similarly, the first and second segments sent to the second user may be cached at a second cache memory local to the second user. Thus, a cache memory may be arranged to store only part of the first file and part of the second file, which may be advantageous when the cache memory is of insufficient size to store the complete files. When at a later time, one or both of the users want to access the first file or the second file, the entire file need not be sent to them as they already have access to some segments locally. Thus, even though remaining segments may in some cases need to be sent at a relatively expensive time, or during high network loads, as the entire file need not be sent then, overall costs for sending the file may be reduced. Additionally, the combined segment is smaller than the total size of the contributing segments, which may provide efficient file transmission management.
In an embodiment, the at least part of a segment sent to the first user is combined with at least part of a segment sent to the second user by using addition in a finite field. In one embodiment, the finite field is the binary field.
In one method, the first and second segments sent to the first and second users are sent prior to the combined segment being sent to the first and second users.
However, in another embodiment, the data is sent in a different order.
In one embodiment, a segment is divided into first and second parts, the first part is incorporated into the combined segment and the second part transmitted to the first and second users. The second part may be transmitted at the same time as the combined segment or as a separate transmission.
According to a second aspect of the invention, a data transmission manager for managing transmission of data files to users comprises: a divider for dividing a first file into a plurality of first segments and for dividing a second file into a plurality of second segments; transmitter apparatus for sending a first segment to a first user and a different first segment to a second user, and for sending a second segment to the first user and a different second segment to the second user; a combiner for combining at least part of a segment sent to the first user with at least part of a segment sent to the second user to produce a combined segment which is of smaller size than the total size of the at least parts of segments before combining; and said transmitter apparatus being operative to send the combined segment to the first user and to the second user for each of the first and second users to recover a segment using the combined segment and at least part of a segment.
BRIEF DESCRIPTION OF THE DRAWINGS
Some embodiments of the present invention will now be described by way of example only, and with reference to the accompanying drawings, in which:
Figure 1 schematically illustrates a flowchart of a method in accordance with the invention;
Figure 2 is a schematic graph illustrating transmission; and
Figure 3 schematically illustrates an apparatus for implementing the method of Figure 1.
DETAILED DESCRIPTION In a first embodiment, assume that there is a first file A and a second file B available from a server, each file being 1MB in size. There are first user Ul and second user U2, and each user has an associated local cache memory Ml and M2 respectively, Ml and M2 being 1MB each. It is assumed that the history of these two users Ul and U2 shows that each may require file A and file B with equal probability.
In this caching scheme, it is necessary to broadcast on average 0.5 MB in expensive networks. Moreover, the peak rate is 0.5 MB.
With reference to Figure 1, at 1, the first and second files are divided into several segments. File A is divided into two equal size segments Al and A2, where each segment is 0.5 MB, such that A=(A1, A2).
Similarly, file B is divided into two equal size segments Bl and B2, where each segment is 0.5 MB, and B=(B1, B2).
At 2, segments Al and Bl are transmitted to the first user Ul and, at 3, saved on the associated cache memory Ml . Also, A2 and B2 are transmitted to the second user U2 and saved on the associated cache memory M2. The caching strategy is summarized in Table 1 and the probability in Table 2.
Table 1
Table 2
Thus, each user has part of each file.
At a later time, for example, if the first user Ul wants to access file A and the second user U2 requires file B, neither has sufficient segments cached to provide a complete file. The users transmit their requirements to the server at 4.
At 5, the server arranges for the transmitter to broadcast a combined segment A2+B1, where + indicates summation in a finite field to generate the combined segment. In this example, combination is carried out in the binary field and therefore + is simply bit-wise XOR operation. The combined segment A2+B 1 has 0.5 MB size. This compares with a size of 0.5MB for segment A2 and 0.5 MB for segment Bl, that is, 1MB in total.
In the next stage, shown at 6, the first user Ul receives A2+B 1 and already has B l in its cache memory Ml . Thus, the first user Ul can recover A2 by the operation (A2+B 1)-B1. The first user Ul also has Al in it cache memory Ml . Therefore, the first user Ul has both Al and A2 and is able to reconstitute the required file A.
The second user U2 also receives A2+B1 broadcast by the transmitter. The second user U2 already has A2 in its cache memory, and therefore can recover B 1 by the operation (A2+B 1)-A2. The second user U2 already has B2 in it cache memory M2. Therefore, both B 1 and B2 are available to the second user U2 to reconstitute the file B it requires.
In another scenario, assuming that the starting point is that shown in Table 1, assume that the first and second users both require file A. In this case, the transmitter broadcasts A2+A1, where + again indicates combination is carried out in the binary field and is a bit-wise XOR operation. The combined segment A2+A1 has a size of 0.5 MB compared to a size of 0.5MB for segment A2 and 0.5 MB for segment Al, that is, 1MB in total
The first user Ul thus receives A2+A1 and already has Al in its cache memory. Therefore it can recover A2 by (A2+A1)-A1. Therefore, the first user Ul has both segments Al and A2 and can thus reconstitute file A.
The second user U2 also receives the broadcast A2+A1 and already has A2 in its cache memory M2. Therefore it can recover Al by (A2+A1)-A2. Therefore, it will have both Al and A2 and can thus reconstitute file A.
For other cases, the broadcasting strategy and also the recovery method are shown in Table 3 below. It can be seen that no matter which file each user requests, the transmitter needs to broadcast only 0.5 MB. Therefore, the average rate of data that the transmitter broadcasts is 0.5 MB.
User Two Wants B User Two Wants Al User Two: (Bl+B2)-B2
Table 3
In this embodiment, the caching scheme is optimized and the average load or maximum load of the network is minimized. In the simple example given above, the caching scheme can reduce the average load up to 50% and peak load of 100%.
Figure 2 illustrates the normalized average transmission rate against the normalized cache-memory size for two files, of size F, and two users, each with cache size M. In addition, the probability that each user may need one of the files is equally likely.
In another embodiment, it is assumed that there are first and second users User 1 and User 2 with available caching memory size of Mi and M 2 respectively. In addition, assume that the server has two files A and B with sizes F A and F B respectively, i.e.
|A|=F A
The users may need one of the files with some probabilities listed
following Table 4: User 1 wants A A B B
User 2 wants A B A B
Probability PAA PAB PBA PBB
Table 4
Each file is divided into 5 parts, which need not be equal, to g
B={Bo, Bi, B 2 , Bi 2 }
with sizes
|Ao|= xo, |Ai|= xi, |A 2 |= x 2 , |A 12 |= x 12 ,
|Bo|= yo, |Bi|= yi, |B 2 |= y 2 , |B 12 |= y 12 ,
Therefore,
x 0 + xi+ x 2 + x 12 = F A
yo+ yi+ y 2 + yi 2 = F B
Parts of the files denoted by Ai and Bi are cached on User 1 memory.
Parts of the files denoted by A 2 and B 2 are cached on User 2 memory.
Parts of the files denoted by A 12 and Bi 2 are cached on both users' memories.
Parts of the files denoted by A 0 and B 0 are cached on none of the memories,
Therefore,
xi+ yi+ xi 2 + yi 2 <= Mi
x 2 + y 2 + xi 2 + y 12 <= M 2
In a first scenario, User 1 asks for file A and User 2 asks for file B. Thus,
User 1 requires A 0 which is only available at the server
User 2 requires B 0 which is only available at the server
User 1 already has Ai and A 12 .
User 2 already has Bi and Bi 2.
User 1 wants A 2 while User 2 has it on its caching memory
User 2 wants Bi while User 1 has it on its caching memory Therefore the server must send both A 0 and B 0 to Users 1 and 2.
Assuming that |A 2 | > |Bi|, then the server partitions segment A 2 into two parts, denoted by A 2U and A 2L , i.e. A 2 ={ A 2U , A 2L }, where |A 2U |=|Bi|. Then the server sends combined segment A 2U + Bi ,where + is addition in binary field or any other finite field, and also sends A 2L . Note that:
|A 2U + Bi| plus |A 2L |=|A 2 | = max { |A 2 |, |Bi| } where max { |A 2 |, |Bi| } is the maximum of the size of A 2 and the size of Bi.
Then user 1 can use A 2U + Bi, A 2L , and Bi to recover A 2 ={ A 2U , A 2L } . Also user 2 can use A 2U + Bi and A 2 to recover Bi.
On other hand, if |A 2 | < |Bi|, then the server divides segment Bi into two parts, denoted by Biu and Bi L , i.e. Bi={ Biu , Bi L }, where |B 1U |=|A 2 |. Then the server sends combined segment Biu + A 2 and also sends Bi L and again + is addition in binary field or any other finite fields. Note that
|Biu + A 2 | plus |BIL |=|BI| = max { |A 2 |, |Bi| } .
Then user 2 can use Biu + Ai, Bi L , A 2 , to recover Bi={ Biu , Bi L } . Also User 1 can use Biu + A 2 , and Bi to recover A 2 .
Then each user has the necessary segments to reconstitute the file requested by it.
In a second scenario, User 1 and User 2 both request file A. Note:
Users 1 and 2 want A 0 which is only available at the server memory
Both Users 1 and 2 already have A 12 .
User 1 wants A 2 while User 2 has it on its caching memory
User 2 wants Ai while User 1 has it on its caching memory
Therefore the server has to sends A 0 to users 1 and 2.
Assume that |A 2 | > |Ai|, then the server partitions segment A 2 into two parts, denoted by A 2U and A 2L , i.e. A 2 ={ A 2U , A 2L }, where |A 2U |=|Ai|. Then the server sends combined segment A 2U + Ai and A 2L where + is addition in binary field or any other finite fields. Note that |A 2U + Ai| + |A 2L |=| A 2 | = max { |A 2 |, |Ai| }.
Then User 1 can use A 2U + Ai, A 2L , and Ai to recover A 2 ={ A 2U , A 2L }. Also User 2 can use A 2U + Ai and A 2 to recover Ai.
On other hand if |A 2 | < |Ai|, then the server partitions Ai into two parts, denoted by ½ and Ai L , i.e. Ai={ ½ , Ai L }, where |Aiu|=|A 2 |. Then the server sends A tu + A 2 and A 1L where + is addition in binary field or any other finite fields. Note that
|Aiu + A 2 | + |A IL |=|Bi| = max { |A 2 |, |Bi| }.
Then User 2 uses ½ + Ai, Ai L , and A 2 to recover Ai={ ½ , Ai L }. Also User 1 can use ½ + A 2 , and Ai to recover A 2 .
Following this strategy, the rate required for different cases is illustrated in Table 5 below:
Table 5
Therefore, the average rate is equal to:
PAA (xo+max{xi, x 2 }) + p A B (x 0 +yo+max{x 2 , yi}) + p BA (x 0 +yo+max{xi, y 2 })
Therefore, x 0 , xi, x 2 , x 12 , y 0 , yi, y 2 , and y 12 may be chosen to minimize the average rate:
Min PAA (x 0 +max{xi, x 2 }) + p A B (x 0 +yo+max{x 2 , yi}) + p B A (x 0 +yo+max{xi, y 2 }) +pBB (yo+max{yi, y 2 })
Subject to:
x 0 + xi+ x 2 + x 12 = F A yo+ yi+ y 2 + yi 2 = F B
xi+ yi+ xi 2 + yi 2 <= Mi
x 2 + y 2 + i 2 + yi 2 <= M 2
xo, xi, x 2 , x 12 , yo, yi, y 2 , and y 12 =>0
If the maximum rate is the main concern, the following optimization can be used:
Min Max [p A A (x 0 +max{xi, x 2 }) , p A B (x 0 +yo+max{x 2 , yi}) , p B A
(xo+yo+max{xi, y 2 }) , p B B(yo+max{yi, y 2 }) ]
Subject to x 0 + xi+ x 2 + x 12 = F A
y 0 + yi+ y 2 + y 12 = F B
xi+ yi+ xi 2 + yi 2 <= Mi
x 2 + y 2 + i 2 + yi 2 <= M 2
x 0 , xi, x 2 , x 12 , yo, yi, y 2 , and y u =>0 Other aspects may be used to refine the method. For example, if caching has some costs, the corresponding cost may be added to the objective function of the optimization of the data rate. For example, if sending data to both users has different costs than sending data to one user, the objective function may be correspondingly modified. Also, if one user already has some parts of the files, then this may be exploited in the optimization. Also, users may have some priorities on segments of a file which should be cached on their memory. Such priorities may be taken into account in the optimization. For transmission of data to the users, users priorities may be considered. For example, files may be transmitted in a manner that allows a user to recover its file in a specific order.
The method can be extended to any number of files and any number of users.
For example, assume that there are three users, user 1, user 2, and user 3, and three files A, B, and C. Each file is partitioned into 8 segments as follows:
A={Ao, Ai, A 2 , A 3 , Ai2 , A 13 , A 23 , Ai 23 }
B={Bo, Bi, B 2 , B 3; Bi 2 , Bi 3 , B 23 , Bi 23 }
C={Co, Ci, C 2 , C 3; Ci 2 , Ci 3 , C 23 , Ci 23 }
Then the segments are saved as follows:
Segments Xo at none of the users, for X=A,B,C
Segments Xi at user i, for i=l,2,3, and X=A,B,C
Segments Xi j at both users i and j for i,j= 1,2,3, and X=A,B,C
Segments Xi 23 at all users i and j for X=A,B,C
For simplicity, it is assumed in this example that Segments X; for i=l,2,3, and X=A,B,C have the same size, and also X y - for i,j=l,2,3, and X=A,B,C have the same size. This assumption is just for this example, and is not general
requirement.
Then, if for example, user 1 wants A, user 2 wants B, and user 3 wants C, then the transmitter sends
Ao
Bo
Co
B 3 +C 2
Then each user has received enough segments and combined segments to retrieve the desired file.
If for example all users want A, then the transmitter sends
Ao
qiA i2 + q 2 A i3 + q 3 A 23 where the operations are any large enough finite-field, and ¾ , g j , φ are from the same field. Then each user has enough equations to solve for entire A.
The size of the segments can be optimized as explained above. The size of the files does not need to be the same.
If there are f files and K users, then each file is partitioned into 2 to the power of K segments, where for each subset of users, there is a segment which is saved on all of the users in that subset. The size of the some of these segments may be zero.
With reference to Figure 3, a data transmission manager 7 for implementing the embodiment described with reference to Figure 1 includes a content store 8 which holds data files A and B. A divider 9 accesses the content store 8 to obtain the data files A and B and divides the files into segments. Some of the segments are to be transmitted to users 10 and 11 at a time when network capacity is large and/or resources required are not expensive. Those segments to be initially transmitted to the users 10 and 11 are selected by a control processor 12 which also maintains a record of which segments are transmitted. The control processor 12 instructs a server 13 as to which segment is to be transmitted to which user. The server 13 acquires the relevant segments and sends them via transmitter 14 to the users 10 and 11 over a network. The users 10 and 11 each have an associated memory cache 15 and 16 respectively in which to store the data segments sent to them from a server 11. Each user receives segments from file A and also from file B.
When the users 10 and 11 wish to have a complete file A or B, they send a message to the server 13 over the network. The server 13 and control processor 12 determine what combined segment is required to fulfill the users requests. The combined segment is produced from file segments by combiner 17 and delivered via server 13 to the users 10 and 11. The users 10 and 11 are then able to reconstitute the complete files using the previously transmitted segments and the combined segment.
The apparatus illustrated in Figure 3 may be adapted to perform more complex data file delivery to implement other methods as set out above.
The functions of the various elements shown in the figure, including any functional blocks labeled as "processors", may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term "processor" should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non volatile storage. Other hardware, conventional and/or custom, may also be included.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.
Next Patent: COMPACTED RUMEN PROTECTED NUTRIENT PELLETS AND METHOD OF MANUFACTURE