METHOD AND SYSTEMS TO UTILISE LOCAL PEERS IN CHUNK-LEVEL PEER-TO-PEER (P2P) FILE SHARING

DESCRIPTION

The invention described here, by way of example only, with reference to the accompanying drawings. In th regard, no attempt is made to show structural details of the invention in more detail than is necessary for fundamental understanding of the invention.

[Dl] FIELD AND BACKGROUND OF THE INVENTION

This invention is generally in the field of peer-to-peer (p2p) file sharing.

Inventions disclosed here extends the inventions disclosed under " METHOD AND SYSTEMS FOR CHUNK- LEVEL PEER-TO-PEER (P2P) FILE SHARING" (PCT/IB2012/056000), to maximise the utilisation of local peers in the chunk-level p2p overlay network.

According to innovations disclosed here, and once fully implemented, the whole chunk-level p2p overlay network will prefer download chunks from local peers including, a chunk cache within a house, one or more chunk caches maintaining for a building or for set of buildings, one or more chunk caches within a peer's Internet Services Provider (ISP), and from chunk caches of the ISP's customers, before a chunk-level p2p storage service truly download a chunk out of the ISP or out of the country over the Internet.

The utilisation of local peers in the extended chunk-level p2p overlay network, will leads to many economic and efficiency benefits starting from end users, to the ISPs and to content owners/distributors.

The chunk-level p2p overlay network can be extended to cover multiple or all ISPs in a country via an Internet Exchange, with many economic and efficiency benefits for that country.

[D2] SUMMARY OF THE INVENTION

Methods and systems disclosed here implements an extended version of the granular chunk-level peer-to- peer (p2p) file sharing system to maximise the utilisation of local peers in the chunk-level p2p overlay network.

Two crucial components of a chunk-level peer-to-peer (p2p) file sharing system are:

1. chunk-level p2p Tracker

2. chunk-level p2p storage service

Chunk-level p2p Tracker is a p2p Tracker but operates at the chunk-level rather than at file-level. Chunk-level p2p Trackers are not aware of what file a given chunk belongs to. Chunk-level p2p Trackers maintain peer lists at chunk-level.

Chunk-level p2p storage service is a p2p storage service or a p2p storage system, shares parts of files called chunks. Chunk-level p2p storage services do not reassemble the main file after downloading all of its chunks. Chunk-level p2p storage services are not aware of what file a given chunk belongs to, and may not necessarily holds all chunks of a main file. A chunk-level p2p storage service Tracker Announce at chunk- level, rather than at file-level, therefore, instead of one Tracker Announcement per main file, a chunk-level p2p storage service issues multiple Tracker Announcements per main file.

A chunk-level p2p storage service connected to a Local Area Network (LAN), can serve as a central chunk cache to a one or more computers within a household-level, therefore, such a central chunk-level p2p storage service is considered as a local peer to download chunks from.

A chunk-level p2p storage service connected to a Local Area Network (LAN), can serve as a central chunk cache to multiple households/apartments/hotel rooms, etc., therefore, such a central chunk-level p2p storage service is also considered as a local peer to download chunks from.

One or more chunk-level p2p storage services maintained at an Internet Services Provider (ISP), can serve to the ISP's customers to download chunks from. Therefore, such chunk-level p2p storage services are also considered local peers. Chunk-level p2p storage services function at an Internet Services Provider's (ISP's) customers-level can be used to download chunks among the ISP's customers. Therefore, such chunk-level p2p storage services are also considered local peers.

Therefore, a local peer is peer so that other peers can download chunks without transmitting via Internet.

Following methods and systems are disclosed to maximise the utilisation of local peers in a chunk-level p2p overlay network:

1. Provide a configuration method for a chunk-level p2p Tracker in its configuration file, to specify the host name or the Internet Protocol (IP) address of a paired chunk-level p2p storage service.

2. A method of load-balancing for chunk-level p2p Trackers and chunk-level p2p storage services

3. Download a chunk if not available in the chunk store of a chunk-level p2p storage service before serve for remote chunk download requests.

4. Set the Internet Protocol (IP) address or the host name of the Master chunk-level p2p storage service for a chunk-level p2p storage service.

5. Set the Internet Protocol (IP) address or the host name of an external chunk-level p2p storage service for a chunk-level p2p storage service.

6. P2P request message method for a chunk-level p2p storage service to upload a chunk from its chunk store to the Master chunk-level p2p storage service on demand.

7. P2P request message method for a chunk-level p2p storage service to upload all of its chunks from its chunk store to the Master chunk-level p2p storage service on demand.

8. P2P request message method for a chunk-level p2p storage service to upload a chunk from its chunk store to the specified chunk-level p2p storage service on demand.

9. P2P request message method for a chunk-level p2p storage service to upload all of its chunks from its chunk store to the specified chunk-level p2p storage service on demand.

10. Issue Tracker Announcements by a chunk-level p2p Tracker.

11. Block chunks referenced in a metadata file getting automatically removed to reclaim space at a

chunk-level p2p storage service by uploading the metadata file to the permanent metadata file store of the chunk-level p2p storage service.

12. Upload an image to a chunk-level p2p storage service, related to a previously uploaded metadata file.

13. Upload content title to a chunk-level p2p storage service, related to a previously uploaded metadata file, per language encoding.

14. Upload content description to a chunk-level p2p storage service, related to a previously uploaded metadata file, per language encoding.

15. Upload content bookmark to a chunk-level p2p storage service, related to a previously uploaded metadata file.

16. Block a chunk getting automatically removed to reclaim space at a chunk-level p2p storage service using a p2p request message method.

17. Upload a metadata file to the temporary metadata file store of a chunk-level p2p storage service using a p2p request message method, once at least one chunk referenced in that metadata file is fully downloaded.

18. Limited Tracker Announcements by a chunk-level p2p storage service.

19. A method of removal of chunks from a chunk-level p2p storage service. 20. A method of identifying peers by a chunk-level p2p storage service.

21. Disable all forms of Tracker Announcements of a chunk-level p2p storage service, and optionally, enable a one or more forms of Tracker Announcements.

22. Generating an accounting entry for billing purpose for a chunk downloaded, on behalf of a client, from Internet by a chunk-level p2p storage service.

23. Private chunks.

24. Intercept connections going to remote chunk-level p2p Trackers and forward such connections to Internet Services Provider's (ISP's) chunk-level p2p Tracker.

25. Intercept connections going to remote chunk-level p2p storage services and forward such

connections to Internet Services Provider's (ISP's) chunk-level p2p storage service.

[D3] BRIEF DESCRIPTION OF THE DRAWINGS

With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention, and are presented in the context of the most useful and readily understood description of the principles and conceptual aspects of the invention.

FIG. 1 is a diagrammatic representation of the extended chunk-level p2p system. 100 is the Master chunk- level p2p Tracker, 101 is the Master chunk-level p2p storage service, 102 is the router of an Internet Services Provider (ISP), 103 is the ISP's chunk-level p2p Tracker, 104 is the ISP's chunk-level p2p storage service, 105 is a peer local to the ISP, ie. ISP's customer, 106 is an external peer, 107 and 108 are the Tracker

Announcement from the ISP's local customer, 109 is the Tracker Announcement from the ISP's chunk-level p2p Tracker to the Master chunk-level p2p Tracker, 110 and 111 are the chunk upload from ISP's local customer, 112 is Tracker Announcement from the ISP's chunk-level p2p storage service for the chunk uploaded, 113 and 114 are the peer list request from the ISP's local customer for a chunk to be downloaded, 115 and 116 are the peer list returned for the peer list request, 117 is the chunk download request, 118 is the peer list request from the Master chunk-level p2p Tracker for the chunk required to be downloaded, 119 is the peer list return from the Master chunk-level p2p Tracker, 120 is the chunk download request from the external peer, 121 is streaming the requested chunk to the ISP's chunk-level p2p storage service from the external peer, 122 is streaming the requested chunk to the ISP's customer from the ISP's chunk-level p2p storage service, 123 are external chunk-level p2p storage services on a Local Area Network (LAN), and 124 is an external chunk-level p2p storage service on the public network (Internet).

[D4] DETAILED DESCRIPTION

In p2p request message methods mentioned below:

(a) The p2p service version number optionally may also be added.

E.g. Method Parameters p2pName/Version\r\n

(b) The method name and parameters may be separated by one or more spaces or any other delimiter such as a colon (':'), semi-colon (';'), etc.

(c) The "session-key" is optional but highly recommended as a very important security measure. The program issuing this p2p request message method, could pass a passphrase or userid/password pair, or other, to get a session key from the chunk-level p2p storage service and the provided session key with this p2p request message method must be validated by the chunk-level p2p storage service before carry out this p2p request message method.

(d) The trailing \r\n is optional, but at least a trailing \n is recommended. The \r is Carriage Return and \n is new line or Line Feed.

The Tracker Announcement refers to the announcement of the availability of a complete chunk ready to share by a peer to the Tracker, so that the Tracker could update its registry to reflect the Internet Protocol (IP) address of the peer regarding the chunk. [D4.1] Provide a configuration method for a chunk-level p2p Tracker in its configuration file, to specify the host name or the Internet Protocol (IP) address of a paired chunk-level p2p storage service

In the extended version of the granular chunk-level peer-to-peer (p2p) file sharing system to maximise the utilisation of local peers in the chunk-level p2p overlay network, one of the crucial element used is assignment of host or Internet Protocol (IP) address of a paired chunk-level p2p storage service 104 (FIG. 1) to a chunk-level p2p Tracker 103 (FIG. 1) hosted at an Internet Services Provider (ISP).

The meaning of paired 103a (FIG. 1) in this regard is, if a chunk-level p2p Tracker maintains entries only for chunk identifications starting with 'a' or 'Α', the paired chunk-level p2p storage service maintains chunks with chunk identifications starting with 'a' or 'A' only.

When returning the peer list 116 (FIG. 1) for a chunk, the chunk-level p2p Tracker 103 (FIG. 1) adds the IP address of its paired chunk-level p2p storage service 104 (FIG. 1) to the peer list, practically, to the end of the peer list.

If there is no entry for the requested chunk in the chunk-level p2p Tracker hosted at the ISP, the returned peer list is then only contain the IP address of the paired chunk-level p2p storage service. The chunk-level p2p storage service requires to download the chunk now requests the chunk from the chunk-level p2p storage service maintained at the ISP, and the chunk-level p2p storage service maintained at the ISP downloads the required chunk from the Internet, keeps a copy and send the requested chunk to the customer. Subsequent requests for the same chunk from other customers of the ISP now could be served from the chunk-level p2p storage service maintained at the ISP without downloading again from the Internet.

A chunk-level p2p Tracker could be designed to look for a configuration file at a known location in the file system to read configuration methods, if such a configuration file exists, before bring up the chunk-level p2p Tracker. The location of the configuration file also can be supplied to the chunk-level p2p Tracker as a command-line parameter.

Format of the configuration method to supply the paired chunk-level p2p storage service as follows:

PAIRED STORAGE SERVICE host | IP Address

A host name or IP address can be provided for this configuration method.

Note, hosting of a chunk-level p2p Tracker is not only limited to an Internet Services Provider, chunk-level p2p Trackers could be hosted at any large institutions such as universities, military, etc. at their point of network termination.

[D4.2] A method of load-balancing for chunk-level p2p Trackers and chunk-level p2p storage services

One of the crucial methods used to maximise the utilisation of local peers in a chunk-level p2p overlay network is intercept connections 102 (FIG. 1) going out to remote chunk-level p2p Trackers and forward such connections to Internet Services Provider's (ISP's) chunk-level p2p Tracker. For large ISPs this may create significant loads on their chunk-level p2p Tracker 103 (FIG. 1). The method disclosed here uses the ISP's chunk-level p2p Trackers 103 (FIG. 1) themselves to distribute the load among themselves by avoiding dedicated load-balancers.

This method, in the context of chunk-level p2p Trackers, comprising that every chunk-level p2p Tracker is configured with a map of information about all chunk-level p2p Trackers running at a location. This map comprising with Internet Protocol (IP) addresses of each chunk-level p2p Tracker at the location and the range of chunks processed by each chunk-level p2p Tracker. The range may comprises a single alphanumeric character or a string of alphanumeric characters or two set of strings of alphanumeric characters separated by a separator indicating a range. The map comprising an optional indicator to indicate to match with either chunk Identification or with the cryptographic digest of the chunk. The match will be done from the beginning of either chunk Identification or with the cryptographic digest of the chunk. The map comprising a further optional indicator to indicate to whether the comparison is case sensitive or not.

For an example, an ISP may hosts 36 chunk-level p2p Trackers covering one chunk-level p2p Tracker for each permutation of the first character of the cryptographic digest of a chunk. That is, 26 chunk-level p2p Trackers for 'a' to 'z' and another 10 chunk-level p2p Trackers for Ό' to '9'. The cryptographic digest of a chunk may of type SHA256.

The map about all chunk-level p2p Trackers are defined in the configuration file of each chunk-level p2p Trackers as follows:

HANDLES TYPE Digest

HANDLES CASE SENSITIVE No

HANDLES CHUNKS a IP1

HANDLES CHUNKS b IP2

HANDLES CHUNKS c IP3

HANDLES CHUNKS z IP26

HANDLES CHUNKS 0 IP27

HANDLES CHUNKS 9 IP36

HANDLES TYPE Digest | ChunkID

HANDLES_TYPE indicates whether the comparison is with either chunk Identification or with the

cryptographic digest of the chunk. Absence of HANDLES_TYPE in the configuration file may default to comparison with cryptographic digest of a chunk.

HANDLES CASE SENSITIVE Yes | No

HANDLES CASE SENSITIVE indicates whether the comparison is case sensitive or not. Absence of

HANDLES CASE SENSITIVE in the configuration file may default to the comparison is not case sensitive.

HANDLES CHUNKS indicates the range of chunks processed by the chunk-level p2p Tracker identified by the Internet Protocol (IP) address. In the example above, the HANDLES TYPE is Digest, the chunk-level p2p Tracker at IP1 processes only cryptographic digest of chunks starting with 'a'.

Format of the HANDLES CHUNKS is as follows:

HANDLES CHUNKS range IP address

This method further comprising, before a request is processed by a chunk-level p2p Tracker, the chunk-level p2p Tracker checks whether the request qualifies to be within the range of chunks processed by the chunk- level p2p Tracker. If the request qualifies, then continue to process the request. If the request does not qualify, find the IP address of the chunk-level p2p Tracker at the location which should process the request from the map of rules configured to the chunk-level p2p Tracker and send a respond message to the client indicating to redirect, followed by the IP address of the chunk-level p2p Tracker to be redirected to. The client in this case should close the existing connection to the current chunk-level p2p Tracker and reconnect to the given chunk-level p2p Tracker and resend the request.

As per the example above, the HANDLES_TYPE is Digest, and if the chunk-level p2p Tracker at IP1 receives a request regarding a chunk whose cryptographic digest starting with 'c', then the chunk-level p2p Tracker at IP1 does not process that request instead it finds the chunk-level p2p Tracker which should process this request, and sends a respond message to the client indicating to redirect, followed by the IP address of the chunk-level p2p Tracker to be redirected to, that is IP3.

E.g. Status-code Redirect\r\n

IP Address\r\n

Above described method of load-balancing for chunk-level p2p Trackers is applicable for chunk-level p2p storage services. When an ISP runs multiple chunk-level p2p storage services, the above described method of load-balancing for chunk-level p2p storage services is required to distribute ISP's customers' chunk uploads to correct chunk-level p2p storage services.

[D4.3] Download a chunk if not available in the chunk store of a chunk-level p2p storage service before serve for remote chunk download requests

A chunk-level p2p storage service, by default, serves a chunk if it is available in its chunk store for remote chunk requests, and if the requested chunk not available in its chunk store, the chunk-level p2p storage service returns a p2p respond message with a status code and description indicating the requested chunk not available in its chunk store.

A chunk-level p2p storage service could be designed to look for a configuration file at a known location in the file system to read configuration methods, if such a configuration file exists, before bring up the chunk-level p2p storage service. The location of the configuration file also can be supplied to the chunk-level p2p storage service as a command-line parameter.

In the configuration file for the chunk-level p2p storage service, a configuration method could be supplied to override the default behaviour and download a chunk before serve if the chunk does not exists in its chunk store for remote chunk requests as follows:

DO WNLOAD 4 RE MOTE REQU ESTS YES | N O

If this configuration method does not exists in a configuration file, it does not alter the default behaviour of chunks are not downloaded if does not exists in the chunk store for remote chunk requests.

If the "DOWNLOAD 4 REMOTE REQUESTS NO", is equivalent to the default behaviour of chunks are not downloaded if does not exists in the chunk store for remote chunk requests.

If the "DOWNLOAD 4 REMOTE REQUESTS YES", exists in a configuration file, it does alter the default behaviour and chunks are downloaded if does not exists in the chunk store for remote chunk requests before serve.

Internet Services Provider's (ISP's) paired chunk-level p2p storage services 104 (FIG. 1) should set to

"DOWN LOAD 4 RE M OTE RE Q U E STS YES" to download chunks if does not exists in the chunk store for remote chunk requests before serve to its customers.

Chunk download for remote requests could be restricted for one Internet Protocol (IP) address or to a range of IP addresses by specifying following configuration method in the configuration file of a chunk-level p2p storage service:

DOWNLOAD 4 REMOTE IP RANGE <IP address range>

The <IP address range> can be a single IP address, or a sub network prefix followed by an asterisk (eg. NN N.NNN.*) or an IP range (IPl-IPn), where NNN is an alphanumeric string and IP1 indicates a starting IP address and IPn indicates an ending IP address.

If the <IP address range> is a single IP address, the chunk-level p2p storage service restricts download a chunk from Internet if the requested chunk is not available in its chunk store only if the request originates from the specified IP address.

If the <IP address range> is indicates a sub network prefix followed by an asterisk (eg. NNN.NN N.*) or an IP range (IPl-IPn), the chunk-level p2p storage service restricts download a chunk from Internet if the requested chunk is not available in its chunk store only if the request originates from an IP address qualifying for the specified range.

Multiple IP addresses or multiple ranges could be specified by specifying the

DOWNLOAD_4_REMOTE_IP_RANGE in multiple times in the configuration file.

[D4.4] Set the Internet Protocol (IP) address or the host name of the Master chunk-level p2p storage service for a chunk-level p2p storage service

In the configuration file for the chunk-level p2p storage service, an optional configuration method could be specified to set the Master chunk-level p2p storage service 101 (FIG. 1), to chain upload or continue to further upload chunks that are uploaded to the chunk-level p2p storage service by other programs such as segmentation program, etc.

The Master chunk-level p2p storage service 101 (FIG. 1) is the root chunk-level p2p storage service control by the entity who control the root chunk-level p2p Tracker for the chunk-level p2p overlay network. Chunks that are downloaded by the chunk-level p2p storage service are not uploaded to the Master chunk-level p2p storage service.

Chunk uploads by the local chunk-level p2p storage service 105 (FIG. 1) to the Master chunk-level p2p storage service 101 (FIG. 1) may be processed delayed. A local chunk-level p2p storage service is the chunk- level p2p storage service that runs on a peer's computer or the device attached to a network. Whenever a chunk arrives to the local chunk-level p2p storage service as an upload from a segmentation program, etc., an entry by the chunk identification can be created under a temporary directory, and once a chunk is successfully uploaded to the Master chunk-level p2p storage service, this entry for the chunk can be removed, so that chunk uploads for new chunks to the Master chunk-level p2p storage service could be even carry out delayed, and between even reboots of the peer. Chunk uploads by the local chunk-level p2p storage service 105 (FIG. 1) to the Master chunk-level p2p storage service 101 (FIG. 1) may be intercepted by the Internet Services Provider's (ISP's) router 102 (FIG. 1) and forward to the ISP's chunk-level p2p storage service 104 (FIG. 1), in that case, chunks will be uploaded to the ISP's chunk-level p2p storage service 104 (FIG. 1) instead of to the Master chunk-level p2p storage service 101 (FIG. 1).

Format of the configuration method is as follows:

MASTER P2P STORAGE host | IP Address

A host name or IP address can be provided for this configuration method.

If this configuration method, MASTER_P2P_STORAGE, not available in the configuration file for the chunk- level p2p storage service, or the internal state of the chunk-level p2p storage service is not already set by other means to the Master chunk-level p2p storage service, that chunk-level p2p storage service does not make any attempt to chain upload or continue to further upload chunks that are uploaded to the chunk-level p2p storage service by other programs such as segmentation program, etc. to the Master chunk-level p2p storage service.

P2P request message method to set the Master chunk-level p2p storage service to a chunk-level p2p storage service

The P2P request message method format can be as follows:

SET MASTER P2P STORAGE host | IP Address [session-key][\r\n]

The effect is as same as for MASTER_P2P_STORAGE.

A host name or IP address can be provided for this method.

The "host" is the host name of the Master chunk-level p2p storage service, in text format.

The "IP Address" is the IP address of the Master chunk-level p2p storage service, in text format.

The SET MASTER P2P STORAGE method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

2. A response message with status code and description for failure.

This method further comprising, the chunk-level p2p storage service rejects this p2p request message method if the Master chunk-level p2p storage service is already defined in the configuration file for the chunk-level p2p storage service, that is, the configuration method MASTER_P2P_STORAGE is already set in the configuration file.

P2P request message method to get the Master chunk-level p2p storage service specified to a chunk-level p2p storage service

A p2p request message method for a chunk-level p2p storage service, could be sent to get the assigned host name or IP address of Master chunk-level p2p storage service.

The P2P request message method format can as follows:

GET MASTER P2P STORAGE [session-key][\r\n]

The GET MASTER P2P STORAGE method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

host I IP Address\r\n

The "host" is the host name of the Master chunk-level p2p storage service, in text format.

The "IP Address" is the IP address of the Master chunk-level p2p storage service, in text format.

2. A response message with status code and description for failure. P2P request message method to remove the Master chunk-level p2p storage service set to a chunk-level p2p storage service

A p2p request message method for a chunk-level p2p storage service, could be sent to unset or remove the Master chunk-level p2p storage service previously specified and set.

This method further comprising, immediate stop chain upload or continue to further upload chunks that are uploaded to the chunk-level p2p storage service by other programs such as segmentation program, etc. to the specified Master chunk-level p2p storage service.

This method further comprising, the chunk-level p2p storage service rejects this p2p request message method if the Master chunk-level p2p storage service is defined in the configuration file for the chunk-level p2p storage service.

The p2p request message method format can be as follows:

UNSET MASTER P2P STORAGE [session-key][\r\n]

The UNSET MASTER P2P STORAGE method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

2. A response message with status code and description for failure.

[D4.5] Set the Internet Protocol (IP) address or the host name of an external chunk-level p2p storage service for a chunk-level p2p storage service

In the configuration file for a chunk-level p2p storage service, an optional configuration method could be specified to set an external chunk-level p2p storage service 123 or 124 (FIG. 1) to a chunk-level p2p storage service, to upload chunks that comes to its chunk store, either by download or upload, to the specified external chunk-level p2p storage service.

The specified external chunk-level p2p storage service is referenced before a chunk is downloaded, to check the existence of a chunk, and if exists download from the external chunk-level p2p storage service.

An external chunk-level p2p storage service may be attached to a Local Area Network (LAN).

Format of the configuration method is as follows:

EXTERNAL P2P STORAGE host | IP Address

A host name or IP address can be provided for this configuration method. Multiple

EXTERNAL_P2P_STORAGE entries may be specified in the configuration file to specify multiple external chunk-level p2p storage services.

P2P request message method to specify an additional external chunk-level p2p storage service to a chunk-level p2p storage service

A p2p request message method for a chunk-level p2p storage service, could be sent to set an external chunk- level p2p storage service to the chunk-level p2p storage service. The effect is as same as for

EXTERNAL P2P STORAGE.

The P2P request message method format can be as follows:

SET EXTERNAL P2P STORAGE <host | IP Address> [session-key][\r\n]

A host name or IP address can be provided for this method. This p2p request message method can call multiple times to set multiple external chunk-level p2p storage services to a chunk-level p2p storage service.

The "host" is the host name of the external chunk-level p2p storage service, in text format.

The "IP Address" is the IP address of the external chunk-level p2p storage service, in text format.

The SET EXTERNAL P2P STORAGE method returns:

1. A response message with status code for success. E.g. Status-code Success\r\n

2. A response message with status code and description for failure.

P2P request message method to get a list of external chunk-level p2p storage services specified to a chunk-level p2p storage service

A p2p request message method for a chunk-level p2p storage service, could be sent to get the list of one or more specified external chunk-level p2p storage services. This method returns a list of either host or IP addresses set as the external chunk-level p2p storage services from the connected chunk-level p2p storage service.

The P2P request message method format can be as follows:

GET EXTERNAL P2P STORAGE LIST[\r\n]

The GET EXTERNAL P2P STORAGE LIST method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

host I IP Address host | IP Address ...\r\n

The "host" is the host name of the external chunk-level p2p storage service, in text format. The "IP Address" is the IP address of the external chunk-level p2p storage service, in text format.

2. A response message with status code and description for failure.

P2P request message method to remove an external chunk-level p2p storage service specified to a chunk-level p2p storage service

A p2p request message method for a chunk-level p2p storage service, could be sent to unset or remove an external chunk-level p2p storage service previously specified and set.

This method further comprising, immediate stop upload chunks that comes to its chunk store, either by download or upload, to the specified external chunk-level p2p storage service.

This method further comprising, immediate stop referring the specified external chunk-level p2p storage service before a chunk is downloaded.

This method further comprising, the chunk-level p2p storage service rejects this p2p request message method if the external chunk-level p2p storage service is defined in the configuration file for the chunk-level p2p storage service.

The p2p request message method format can be as follows:

UNSET EXTERNAL P2P STORAGE <host | IP Address> [session-key][\r\n]

A host name or IP address can be provided for this method.

The "host" is the host name of the external chunk-level p2p storage service, in text format.

The "IP Address" is the IP address of the external chunk-level p2p storage service, in text format.

The UNSET EXTERNAL P2P STORAGE method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

2. A response message with status code and description for failure.

[D4.6] P2P request message method for a chunk-level p2p storage service to upload a chunk from its chunk store to the Master chunk-level p2p storage service on demand

A p2p request message method for a chunk-level p2p storage service, could be sent to upload a chunk from its chunk store to the Master chunk-level p2p storage service 101 (FIG. 1) on demand. The Master chunk- level p2p storage service is the root chunk-level p2p storage service control by the entity who control the root chunk-level p2p Tracker for the chunk-level p2p overlay network.

The p2p request message method format can be as follows:

UPLOAD TO_MASTER P2P STORAGE ChunkID [session-key][\r\n]

The "ChunkID" is the identification of the required chunk.

This method expect the host name or the Internet Protocol (IP) address of the Master chunk-level p2p storage service is already preconfigured to the chunk-level p2p storage service either by the the configuration method MASTER P2P STORAGE in the configuration file or by the p2p request message method,

SET MASTER P2P STORAGE.

The UPLOAD TO_MASTER P2P STORAGE method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

2. A response message with status code and description for failure.

Upload to the Master chunk-level p2p storage service could be used to repopulate Internet Services Provider's (ISP's) chunk-level p2p storage services 104 (FIG. 1) by detecting 102 (FIG. 1) the connections to chunk-level p2p storage services going out of the ISP's IP address space and forward them to ISP's chunk-level p2p storage services 104 (FIG. 1).

[D4.7] P2P request message method for a chunk-level p2p storage service to upload all of its chunks from its chunk store to the Master chunk-level p2p storage service on demand

A p2p request message method for a chunk-level p2p storage service, could be sent to upload all of its chunks from its chunk store to the Master chunk-level p2p storage service 101 (FIG. 1) on demand. The Master chunk-level p2p storage service is the root chunk-level p2p storage service control by the entity who control the root chunk-level p2p Tracker for the chunk-level p2p overlay network.

The p2p request message method format is as follows:

UPLOAD ALL TO_MASTER P2P STORAGE [session-key][\r\n]

This method expect the host name or the Internet Protocol (IP) address of the Master chunk-level p2p storage service is already preconfigured to the chunk-level p2p storage service either by the the configuration method MASTER P2P STORAGE in the configuration file or by the p2p request message method,

SET MASTER P2P STORAGE.

The UPLOAD ALL TO_MASTER P2P STORAGE method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

2. A response message with status code and description for failure.

Upload to the Master chunk-level p2p storage service 101 (FIG. 1) could be used to repopulate Internet Services Provider's (ISP's) chunk-level p2p storage services 104 (FIG. 1) by detecting 102 (FIG. 1) the connections to chunk-level p2p storage services going out of the ISP's IP address space and forward them to ISP's chunk-level p2p storage services 104 (FIG. 1).

[D4.8] P2P request message method for a chunk-level p2p storage service to upload a chunk from its chunk store to the specified chunk-level p2p storage service on demand

A p2p request message method for a chunk-level p2p storage service, could be sent to upload a chunk from its chunk store to the specified chunk-level p2p storage service on demand. The specified chunk-level p2p storage service may include the Master chunk-level p2p storage service 101 (FIG. 1). The Master chunk-level p2p storage service is the root chunk-level p2p storage service control by the entity who control the root chunk-level p2p Tracker for the chunk-level p2p overlay network.

The p2p request message method format is as follows:

UPLOAD TO_P2P STORAGE ChunkID <host | IP Address> [session-key][\r\n] The "ChunkID" is the identification of the required chunk.

The "host" is the host name of the external chunk-level p2p storage service, in text format.

The "IP Address" is the IP address of the external chunk-level p2p storage service, in text format.

The UPLOAD TO_P2P STORAGE method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

2. A response message with status code and description for failure.

Upload chunks to an external chunk-level p2p storage service 123 (FIG. 1) accessed over a Local Area Network (LAN) is useful to populate the external chunk-level p2p storage service with new chunks so that others could download such chunks the over the LAN from the external chunk-level p2p storage service.

Upload chunks to the Master chunk-level p2p storage service 101 (FIG. 1) could be used to populate or repopulate Internet Services Provider's (ISP's) chunk-level p2p storage services 104 (FIG. 1) by detecting 102 (FIG. 1) the connections to chunk-level p2p storage services going out of the ISP's IP address space and forward them to ISP's chunk-level p2p storage services 104 (FIG. 1).

[D4.9] P2P request message method for a chunk-level p2p storage service to upload all of its chunks from its chunk store to the specified chunk-level p2p storage service on demand

A p2p request message method for a chunk-level p2p storage service, could be sent to upload all of its chunks from its chunk store to the specified chunk-level p2p storage service on demand. The specified chunk- level p2p storage service may include the Master chunk-level p2p storage service 101 (FIG. 1). The Master chunk-level p2p storage service is the root chunk-level p2p storage service control by the entity who control the root chunk-level p2p Tracker for the chunk-level p2p overlay network.

The p2p request message method format is as follows:

UPLOAD ALL TO_P2P STORAGE <host | IP Address> [session-key][\r\n]

The "host" is the host name of the external chunk-level p2p storage service, in text format.

The "IP Address" is the IP address of the external chunk-level p2p storage service, in text format.

The UPLOAD ALL TO_P2P STORAGE method returns:

1. A response message with status code for success.

E.g. Status-code Success\r\n

2. A response message with status code and description for failure.

Upload chunks to an external chunk-level p2p storage service 123 (FIG. 1) accessed over a Local Area Network (LAN) is useful to populate the external chunk-level p2p storage service with new chunks so that others could download such chunks the over the LAN from the external chunk-level p2p storage service.

Upload chunks to the Master chunk-level p2p storage service 101 (FIG. 1) could be used to populate or repopulate Internet Services Provider's (ISP's) chunk-level p2p storage services 104 (FIG. 1) by detecting 102 (FIG. 1) the connections to chunk-level p2p storage services going out of the ISP's IP address space and forward them to ISP's chunk-level p2p storage services 104 (FIG. 1).

[D4.10] Issue Tracker Announcements by a chunk-level p2p Tracker

The Tracker Announcement 107, 112 (FIG. 1) in this regard refers to the announcement to a chunk-level p2p Tracker of the availability of a complete chunk ready to share by a peer, so that the chunk-level p2p Tracker could update its registry to reflect the Internet Protocol (IP) address of the peer willing to share the chunk.

As per the extended chunk-level p2p ecosystem disclosed here, it is required to issue Tracker Announcements from a chunk-level p2p Tracker itself 109 (FIG. 1).

Once the connection is established with a chunk-level p2p Tracker, issue this method TRAC KE R_A N N O U N C E BY TRAC KE R to announce the availability of a chunk to share.

The P2P request message method format can be as follows:

TRACKER ANNOUNCE BY TRACKER Chunk-ID digest IP Address[\r\n]

The "Chunk-ID" is the unique identification of the chunk, in text format.

The "digest" is a cryptographic digest of the chunk, in text format. E.g. SHA256. The implementation of this method specifies which cryptographic hash function is accepted for the digest.

The "IP_Address" is the Internet Protocol (IP) address of the peer sharing the chunk, in text format.

Normal Tracker Announcements (TRACKER ANNOUNCE) 107 (FIG. 1) are expected to be intercepted by Internet Services Provider's (ISP's) routers 102 (FIG. 1) and forwarded to the ISP's chunk-level p2p Tracker 103 (FIG. 1) and ends there. That is, it only updates an ISP's chunk-level p2p Tracker and does not propagate beyond the ISP.

Whereas, Tracker Announcements by a chunk-level p2p Tracker (TRACKER ANNOUNCE BY TRACKER) 109 (FIG. 1) is required to be reissued to the Master chunk-level p2p Tracker 101 (FIG. 1) even if intercepted by ISP's routers.

Whenever a new entry is created in an ISP's chunk-level p2p Tracker's 103 (FIG. 1) registry for a chunk identification, it is required to issue a Tracker Announcement (TRACKER ANNOUNCE BY TRACKER) 109 (FIG. 1) to the Master chunk-level p2p Tracker 101 (FIG. 1) regarding the newly created entry. The ISP's chunk-level p2p Tracker 103 (FIG. 1), may skip N number of subsequent normal Tracker Announcements (TRACKER ANNOUNCE) 107 (FIG. 1) for an entry, but the ISP's chunk-level p2p Tracker 103 (FIG. 1) is required to issue a Tracker Announcement (TRACKER AN NOU NCE BY TRACKER) 109 (FIG. 1) to the Master chunk-level p2p Tracker 101 (FIG. 1) periodically. The N may be automatically computed by the ISP's chunk- level p2p Tracker 103 (FIG. 1) depending on the rate normal Tracker Announcements

(TRACKER ANNOUNCE) 107 (FIG. 1) arrive for a chunk.

Providing the cryptographic digest of the chunk in the TRAC KE R_A N N O U N C E BY TRAC KE R request further allows load-balancing for chunk-level p2p Trackers. The connected chunk-level p2p Tracker or a load-balancer could use the first one or more characters of the digest to decide which chunk-level p2p Tracker should serve the request.

The Tracker Announcement receiving chunk-level p2p Tracker (e.g. Master chunk-level p2p Tracker) may use the chunk existence check to connect back to the peer ("IP_Address") to verify the existence of the chunk and its ability to share. If the Tracker Announcement receiving chunk-level p2p Tracker finds the chunk does not exists or not possible to share, i.e. behind a firewall or on a private IP address, then the Tracker

Announcement receiving chunk-level p2p Tracker does not accept this TRAC KE R AN N O U N C E BY TRAC KE R request.

The TRACKER ANNOUNCE BY TRACKER method returns:

1. A response message with status code for success if the request accepted by the chunk-level p2p Tracker.

E.g. Status-code Success\r\n

2. A response message with status code to redirect.

E.g. Status-code Redirect\r\n

IP address\r\n

In this case, close the existing connection with the chunk-level p2p Tracker, make a new connection to the received IP address and resend the TRAC KE R_A N N O U N C E BY TRAC KE R method.

3. A response message with status code and description for failure.

[D4.11] Block chunks referenced in a metadata file getting automatically removed to reclaim space at a chunk-level p2p storage service by uploading the metadata file to the permanent metadata file store of the chunk-level p2p storage service using a p2p request message method

It is necessary to remove chunks to reclaim space at a chunk-level p2p storage service to accommodate space for incoming new chunks. In practice, not all chunks may not be removed to reclaim space. There can be movies or music purchased, or media one requires to be kept in long term, etc., therefore, such media cannot be removed to reclaim space. Chunks of such media requires to be locked such that they will not be removed to reclaim space.

Innovation disclosed here is a method of locking certain chunks such that they will not be removed to reclaim space at a chunk-level p2p storage service. To support the method, it is required to extend the functionality of the chunk-level p2p storage service to support a permanent metadata file store. The permanent metadata file store could be implemented as a file system directory. Metadata files in the permanent metadata file store will also not automatically be removed to reclaim space.

The method involves, upload a metadata file to the permanent metadata file store of a chunk-level p2p storage service, and the chunk-level p2p storage service will not remove chunks referenced in that metadata file to reclaim space.

This method makes available chunks of a peer (i.e. chunk-level p2p storage service) for long term to share over a p2p overlay network or make available to consume locally for long term without downloading over the Internet.

The p2p request message method to initiate upload a metadata file to a chunk-level p2p storage service requires to be provided with the metadata file name, the length of the metadata file in bytes, and a cryptographic digest of the metadata file, as parameters.

The p2p request message method format can be as follows:

UPLOAD METADATAFILE Metadata-file-name Metadata-file-length digest [session-key][\r\n]

The " Metadata-file-name" is the name of the metadata file, in text format.

The " Metadata-file-length" is the length of the metadata file in bytes, in text format.

The "digest" is a cryptographic digest of the metadata file, in text format. E.g. SHA256

The process of the UPLOAD METADATAFILE method:

1. Upon receive the initiate metadata file upload request, the chunk-level p2p storage service checks the existence of the specified metadata file in its permanent metadata file store by metadata file name.

The UPLOAD METADATAFILE method returns:

(a) A response message with status code for continue to send the metadata file, if the metadata file does not exists in the chunk-level p2p storage service's permanent metadata file store.

E.g. Status-code Continue to send\r\n

(b) A response message with status code for the metadata file already exists in the chunk-level p2p storage service's permanent metadata file store, therefore, do not send the metadata file.

(c) A response message with status code and description for failure, therefore, do not send the metadata file.

2. If the client receives the response message with status code for continue to send the metadata file, then send the metadata file to the chunk-level p2p storage service as a binary byte stream. The chunk-level p2p storage service saves the metadata file in the permanent metadata file store.

3. Finally, the chunk-level p2p storage service sends either a response message with status code for success or a response message with status code and description for failure.

This method further comprising, the chunk-level p2p storage service makes sure the chunks referenced in that metadata file will not be removed to reclaim space from the chunk-level p2p storage service either by (a) apply file system undeletable flags to chunks referenced in that metadata file if the file system of the chunk- level p2p storage service supports such file system flags or (b) adds references to chunks referenced in that metadata file to a list of chunks that should not be removed to reclaim space at the chunk-level p2p storage service, if the file system of the chunk-level p2p storage service does not support file system undeletable flags.

FreeBSD operating system supports chflags(2) system call to avoid a file being getting renamed or deleted once the UFJMOUNLINK or SFJMOUNLINK flags set. The said list may be implemented (a) as a hash-table. Should the chunk-level p2p storage service reboots, once up, the chunk-level p2p storage service rebuilds this hash-table from metadata files from its permanent metadata file store. Or (b) as empty files as same names as chunks in a directory.

Implementation may restrict this p2p request message method, UPLOAD_METADATAFILE, only for the local chunk-level p2p storage service. The local chunk-level p2p storage service runs on the same peer as the client which issue this p2p request message method, UPLOAD METADATAFILE.

Removal of a previously uploaded metadata file from the chunk-level p2p storage service

A p2p request message method for the purpose could be used to remove a previously uploaded metadata file from a chunk-level p2p storage service with Metadata-file-name as the parameter. This method should further comprising, the chunk-level p2p storage service removes either file system undeletable flags or references to chunks referenced in that metadata file from the list of chunks that should not be removed to reclaim space, depending on how the chunk-level p2p storage service implemented to block relevant chunks getting removed to reclaim space as referenced above in the upload section.

Example: REMOVE METADATAFILE Metadata-file-name [session-key][\r\n]

Retrieval of metadata files from the chunk-level p2p storage service

Retrieval of metadata files from the chunk-level p2p storage service can be of few types:

a) Get a list of all metadata file names from the permanent metadata file store of a chunk-level p2p storage service by a p2p request message method for the purpose.

Example: GET METADATAFILE LIST ALL[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

Metadata-file-namel Metadata-file-name2 ....\r\n

b) Get a list of metadata file names from the permanent metadata file store of a chunk-level p2p storage service by supplying a search criteria as a parameter to a p2p request message method for the purpose.

Example: GET METADATAFILE LIST SEARCH <Search criteria>[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

Metadata-file-namel Metadata-file-name2 ....\r\n

c) Get an metadata file from the permanent metadata file store of a chunk-level p2p storage service as a byte stream by supplying the Metadata-file-name as the parameter to a p2p request message method for the purpose.

Example: GET METADATAFILE DATA Metadata-file-name[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

Metadata-file-length Metadata-file-digest\r\n

byte stream

d) Get the full file system path of the requested metadata file from the permanent metadata file store of local chunk-level p2p storage service by supplying the Metadata-file-name as the parameter to a p2p request message method for the purpose.

Example: GET METADATAFILE LOCAL Metadata-file-name[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

metadata file full file-system path\r\n [D4.12] Upload an image to a chunk-level p2p storage service, related to a previously uploaded metadata file using a p2p request message method

P2P request message method to initiate upload an image file to a chunk-level p2p storage service related to a previously uploaded metadata file by the metadata file name, image file name, the length of the image file in bytes, and a cryptographic digest of the image file, as parameters.

The p2p request message method format can be as follows:

UPLOAD IMG Metadata-file-name img-file-name img-file-length img-file-digest [session-key] [\r\n]

The " Metadata-file-name" is the name of the metadata file, in text format.

The "img-file-name" is the name of the image file, in text format.

The "img-file-length" is the length of the image file in bytes, in text format.

The "img-file-digest" is a cryptographic digest of the image file, in text format. E.g. SHA256

The process of the UPLOAD IMG method:

1. Upon receive the initiate image file upload request, the chunk-level p2p storage service checks the existence of the specified metadata file by metadata file name in its permanent metadata file store and also check the existence of the specified image file by the image file name.

2. The UPLOAD IMG method returns:

(a) A response message with status code for continue to send the image file, if the image file does not exists in the chunk-level p2p storage service for the specified metadata file.

E.g. Status-code Continue to send\r\n

(b) A response message with status code for the image file already exists with the chunk-level p2p storage service for the specified metadata file, therefore, do not send the image file.

(c) A response message with status code and description for failure, therefore, do not send the image file.

3. If the client receives the response message with status code for continue to send the image file, then send the image file to the chunk-level p2p storage service as a binary byte stream.

4. Finally, the chunk-level p2p storage service sends either a response message with status code for success or a response message with status code and description for failure.

Implementation may restrict this p2p request message method, UPLOADJMG, only for the local chunk-level p2p storage service. The local chunk-level p2p storage service runs on the same peer as the client which issue this p2p request message method, UPLOADJMG.

Removal of a previously uploaded image from the chunk-level p2p storage service

A p2p request message method for the purpose could be used to remove a previously uploaded image from a chunk-level p2p storage service with Metadata-file-name and img-file-name as parameters.

Example: REMOVE IMG Metadata-file-name img-file-name [session-key][\r\n]

Retrieval of images from the chunk-level p2p storage service

Retrieval of images from the chunk-level p2p storage service can be of few types:

a) Get a list of image file names from the chunk-level p2p storage service by supplying the Metadata- file-name as a parameter to a p2p request message method for the purpose.

Example: GET IMG LIST Metadata-file-name[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code Success\r\n

img-file-namel img-file-name2 ....\r\n b) Get an image from the chunk-level p2p storage service as a byte stream by supplying the Metadata- file-name and img-file-name as parameters to a p2p request message method for the purpose. Example: GET IMG DATA Metadata-file-name img-file-name[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

img-file-length img-file-digest\r\n

byte stream

c) Get the full file system path of the requested image from the local chunk-level p2p storage service by supplying the Metadata-file-name and img-file-name as parameters to a p2p request message method for the purpose.

Example: GET IMG LOCAL Metadata-file-name img-file-name[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

full image file path\r\n

[D4.13] Upload content title to a chunk-level p2p storage service, related to a previously uploaded metadata file, per language encoding

The title is the title of the content. E.g. "Miss Universe Preliminary Competition 2012 Full 1080p"

P2P request message method to initiate upload content title to a chunk-level p2p storage service related to a previously uploaded metadata file by the metadata file name and Language code, as parameters.

The p2p request message method format can be as follows:

UPLOAD TITLE Metadata-file-name Language-code [session-key][\r\n]

The " Metadata-file-name" is the name of the metadata file, in text format.

The "Language-code" is the code of the language encoding, in text format. E.g. ISO 639 is recommended for Language codes.

The process of the UPLOAD TITLE method:

1. Upon receive the initiate content title upload request, the chunk-level p2p storage service checks the existence of the specified metadata file by metadata file name in its permanent metadata file store and also check the existence of the specified content title by the Language-code for the specified metadata file.

2. The UPLOAD TITLE method returns:

(a) A response message with status code for continue to send the content title, if the content title does not exists in the chunk-level p2p storage service for the specified metadata file under language encoding.

E.g. Status-code Continue to send\r\n

(b) A response message with status code for the content title already exists with the chunk-level p2p storage service, therefore, do not send the content title.

(c) A response message with status code and description for failure, therefore, do not send the content title.

3. If the client receives the response message with status code for continue to send the content title, then send the content title to the chunk-level p2p storage service, preferably, as an UTF-8 encoded text as "\r\n" as last two characters. It is possible to save the content title as a text file with same name as the language encoding code.

4. Finally, the chunk-level p2p storage service sends either a response message with status code for success or a response message with status code and description for failure.

Implementation may restrict this p2p request message method, UPLOAD_TITLE, only for the local chunk- level p2p storage service. The local chunk-level p2p storage service runs on the same peer as the client which issue this p2p request message method, UPLOAD_TITLE.

Removal of a previously uploaded content title from the chunk-level p2p storage service

A p2p request message method for the purpose could be used to remove a previously uploaded content title from a chunk-level p2p storage service with Metadata-file-name and Language-code as parameters.

Example: REMOVE TITLE Metadata-file-name Language-code [session-key][\r\n]

Retrieval of content titles from the chunk-level p2p storage service

Retrieval of content titles from the chunk-level p2p storage service can be of few types:

a) Get a list of content title language codes from the chunk-level p2p storage service by supplying the Metadata-file-name as a parameter to a p2p request message method for the purpose.

Example: GET TITLE LIST Metadata-file-name[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

Language-codel Language-code2 ....\r\n

b) Get an content title from the chunk-level p2p storage service as a text stream by supplying the

Metadata-file-name and Language-code as parameters to a p2p request message method for the purpose.

Example: GET TITLE DATA Metadata-file-name Language-code[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

Content Title\r\n c) Get the full file system path of the requested content title from the local chunk-level p2p storage service by supplying the Metadata-file-name and Language-code as parameters to a p2p request message method for the purpose.

Example: GET TITLE LOCAL Metadata-file-name Language-code[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

content title full file-system path\r\n

[D4.14] Upload content description to a chunk-level p2p storage service, related to a previously uploaded metadata file, per language encoding

P2P request message method to initiate upload content description to a chunk-level p2p storage service related to a previously uploaded metadata file by the metadata file name and Language code, as parameters.

The p2p request message method format can be as follows:

UPLOAD DESC Metadata-file-name Language-code [session-key][\r\n]

The " Metadata-file-name" is the name of the metadata file, in text format.

The "Language-code" is the code of the language encoding, in text format. E.g. ISO 639 is recommended for Language codes.

The process of the UPLOAD_DESC method:

1. Upon receive the initiate content description upload request, the chunk-level p2p storage service checks the existence of the specified metadata file by metadata file name in its permanent metadata file store and also check the existence of the specified content description by the Language-code under the specified metadata file.

2. The UPLOAD_DESC method returns:

(a) A response message with status code for continue to send the content description, if the content description does not exists in the chunk-level p2p storage service for the specified metadata file and under the language code.

E.g. Status-code Continue to send\r\n

(b) A response message with status code for the content description already exists with the chunk- level p2p storage service, therefore, do not send the content description.

(c) A response message with status code and description for failure, therefore, do not send the content description.

3. If the client receives the response message with status code for continue to send the content

description, then send the content description to the chunk-level p2p storage service, preferably, as an UTF-8 encoded text as "\r\n" as last two characters. It is possible to save the content description as a text file with same name as the language encoding code.

4. Finally, the chunk-level p2p storage service sends either a response message with status code for success or a response message with status code and description for failure.

Implementation may restrict this p2p request message method, UPLOAD_DESC, only for the local chunk- level p2p storage service. The local chunk-level p2p storage service runs on the same peer as the client which issue this p2p request message method, UPLOAD DESC.

Removal of a previously uploaded content description from the chunk-level p2p storage service

A p2p request message method for the purpose could be used to remove a previously uploaded content description from a chunk-level p2p storage service with Metadata-file-name and Language-code as parameters.

Example: REMOVE DESC Metadata-file-name Language-code [session-key][\r\n]

Retrieval of content descriptions from the chunk-level p2p storage service

Retrieval of content descriptions from the chunk-level p2p storage service can be of few types:

a) Get a list of content description language codes from the chunk-level p2p storage service by

supplying the Metadata-file-name as a parameter to a p2p request message method for the purpose.

Example: GET DESC LIST Metadata-file-name[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

Language-codel Language-code2 ....\r\n

b) Get an content description from the chunk-level p2p storage service as a text stream by supplying the Metadata-file-name and Language-code as parameters to a p2p request message method for the purpose.

Example: GET DESC DATA Metadata-file-name Language-code[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

content description\r\n c) Get the full file system path of the requested content description from the local chunk-level p2p storage service by supplying the Metadata-file-name and Language-code as parameters to a p2p request message method for the purpose.

Example: GET DESC LOCAL Metadata-file-name Language-code[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

content description full file-system path\r\n [D4.15] Upload content bookmark to a chunk-level p2p storage service, related to a previously uploaded metadata file

A content bookmark is an user defined point in the content time-line.

If the metadata file refers to an audio visual content, a content bookmark, once selected, could be used to directly start play from the defined point in the time-line. A content bookmark may be saved with an user entered textual description, and when the content bookmark is presented on a media player, if it is a video content, based on the point of time, the media player could display a video frame relevant to the point of time referenced in the content bookmark together with the textual description.

The p2p request message method format can be as follows:

UPLOAD BOOKMARK Metadata-file-name value [session-key][\r\n]

The " Metadata-file-name" is the name of the metadata file, in text format.

The "value" is an unsigned integer number without fractional part specified in textual form. Regarding audio visual content, the value is the Presentation Time Stamp (PTS) value of the bookmarked point in the content time-line. That is, point of time in seconds from the start of the play of the content, divided by the Time Base. The Time Base is, if there is a video stream in the content, then it is the video Time Base value taken from the metadata file. If there is a no video stream in the content, then it is the audio Time Base value taken from the metadata file.

The process of the UPLOAD BOOKMARK method:

1. Upon receive the initiate content bookmark upload request, the chunk-level p2p storage service checks the existence of the specified metadata file by metadata file name in its permanent metadata file store and also check the existence of the specified content bookmark by the value.

2. The UPLOAD BOOKMARK method returns:

(a) A response message with status code for continue to send the content bookmark, if the content bookmark does not exists in the chunk-level p2p storage service.

E.g. Status-code Continue to send\r\n

(b) A response message with status code for the content bookmark already exists with the chunk- level p2p storage service, therefore, do not send the content bookmark.

(c) A response message with status code and description for failure, therefore, do not send the content bookmark.

3. If the client receives the response message with status code for continue to send the content

bookmark, then send the textual description of the content bookmark to the chunk-level p2p storage service, preferably, as an UTF-8 encoded text as "\r\n" as last two characters. It is possible to save the textual description of the content bookmark as a text file with same name as the number provided in value .

4. Finally, the chunk-level p2p storage service sends either a response message with status code for success or a response message with status code and description for failure.

Implementation may restrict this p2p request message method, UPLOAD_BOOKMARK, only for the local chunk-level p2p storage service. The local chunk-level p2p storage service runs on the same peer as the client which issue this p2p request message method, UPLOAD BOOKMARK.

Removal of a previously uploaded content bookmark from the chunk-level p2p storage service

A p2p request message method for the purpose could be used to remove a previously uploaded content bookmark from a chunk-level p2p storage service with Metadata-file-name and value as parameters.

Example: REMOVE BOOKMARK Metadata-file-name value [session-key][\r\n]

Retrieval of content bookmarks from the chunk-level p2p storage service

Retrieval of content bookmarks from the chunk-level p2p storage service can be of few types: a) Get a list of content bookmarks from the chunk-level p2p storage service by supplying the Metadata-file-name as a parameter to a p2p request message method for the purpose.

Example: GET BOOKMARK LIST Metadata-file-name[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

valuel value2 ....\r\n

b) Get the textual description of a content bookmark from the chunk-level p2p storage service as a text stream by supplying the Metadata-file-name and value as parameters to a p2p request message method for the purpose.

Example: GET BOOKMARK DATA Metadata-file-name value[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

Content bookmark description\r\n c) Get the full file system path of the requested the textual description of a content bookmark from the local chunk-level p2p storage service by supplying the Metadata-file-name and value as parameters to a p2p request message method for the purpose.

Example: GET BOOKMARK LOCAL Metadata-file-name value[\r\n]

Response message from the chunk-level p2p storage service:

E.g. Status-code SuccessVAn

content bookmark full file-system path\r\n

[D4.16] Block a chunk getting automatically removed to reclaim space at a chunk-level p2p storage service using a p2p request message method

This method blocks a chunk getting automatically removed to reclaim space at a chunk-level p2p storage service using a p2p request message method without uploading a metadata file.

This method further comprising, once received the p2p request message method, the chunk-level p2p storage service either (a) apply a file system undeletable flag to the required chunk if the file system of the chunk-level p2p storage service supports such file system flags or (b) adds a reference to the required chunk to a list of chunks that should not be removed to reclaim space at the chunk-level p2p storage service, if the file system of the chunk-level p2p storage service does not support file system undeletable flags.

The p2p request message method format is as follows:

LOCK CHUNK ChunkID [session-key][\r\n]

A host name or IP address can be provided for this method.

The "ChunkID" is the identification of the required chunk.

The LOCK_CHUNK method returns:

1. A response message with status code for success.

E.g. Status-code SuccessVAn

2. A response message with status code to redirect.

E.g. Status-code Redirect\r\n

IP addressVAn

In this case, close the existing connection with the chunk-level storage service, make a new connection to the received IP address and resend the LOCK_CHUNK method.

3. A response message with status code and description for failure.

FreeBSD operating system supports chflags(2) system call to avoid a file being getting renamed or deleted once the UFJMOUNLINK or SFJMOUNLINK flags set.

The said list may be implemented as create an empty file as same name as the required chunk in a directory. Implementation may restrict this p2p request message method, LOCK_CHUNK, only for the local chunk-level p2p storage service. The local chunk-level p2p storage service runs on the same peer as the client which issue this p2p request message method, LOCK CHUNK.

Unblock a chunk from the chunk-level p2p storage service

A p2p request message method for the purpose could be used to unblock a previously blocked chunk using LOCK CHUNK p2p request message method from a chunk-level p2p storage service:

Example: UNLOCK CHUNK ChunkID [session-key][\r\n]

Upon receive this p2p request message method, chunk-level p2p storage service either (a) remove the file system undeletable flag from the specified chunk if the chunk was blocked using a file system undeletable flag or (b) remove the reference of the required chunk from the list of chunks that should not be removed to reclaim space at the chunk-level p2p storage service. It is only unblock the chunk, but does not remove the chunk itself.

[D4.17] Upload a metadata file to the temporary metadata file store of a chunk-level p2p storage service using a p2p request message method, once at least one chunk referenced in that metadata file is fully downloaded

Innovation disclosed here is always upload a metadata file to the temporary metadata file store of a chunk- level p2p storage service using a p2p request message method, once at least one chunk referenced in that metadata file is fully downloaded. To support the method, it is required to extend the functionality of the chunk-level p2p storage service to support a temporary metadata file store. The temporary metadata file store could be implemented as a file system directory. Metadata files in the temporary metadata file store are also subject to automatically be removed to reclaim space.

Innovation disclosed here provides the basis for (a) efficient Tracker Announcements of chunks ( [D4.18]), (b) a method of removal of all chunks of a metadata file as and when require to reclaim space ([D4.19]), and (c) efficiently identify peers to download chunks from ([D4.20]).

The p2p request message method to initiate upload a metadata file to the temporary metadata file store of a chunk-level p2p storage service requires to be provided with the metadata file name, the length of the metadata file in bytes, and a cryptographic digest of the metadata file, as parameters.

The p2p request message method format can be as follows:

UPLOAD METADATAF I LE TEMP Metadata-file-name Metadata-file-length digest [session-key] [\r\n]

The " Metadata-file-name" is the name of the metadata file, in text format.

The " Metadata-file-length" is the length of the metadata file in bytes, in text format.

The "digest" is a cryptographic digest of the metadata file, in text format. E.g. SHA256

The process of the UPLOAD METADATAFILE TEMP method:

1. Upon receive the initiate metadata file upload request, the chunk-level p2p storage service checks the existence of the specified metadata file in both of its permanent and temporary metadata file stores by metadata file name.

2. The UPLOAD METADATAFILE TEMP method returns:

(a) A response message with status code for continue to send the metadata file, if the metadata file does not exists in both permanent and temporary metadata file stores of the chunk-level p2p storage service.

E.g. Status-code Continue to send\r\n

(b) A response message with status code for the metadata file already exists in either permanent or temporary metadata file store of the chunk-level p2p storage service, therefore, do not send the metadata file. (c) A response message with status code and description for failure, therefore, do not send the metadata file.

3. If the client receives the response message with status code for continue to send the metadata file, then send the metadata file to the chunk-level p2p storage service as a binary byte stream. The chunk-level p2p storage service saves the metadata file in the temporary metadata file store.

4. Finally, the chunk-level p2p storage service sends either a response message with status code for success or a response message with status code and description for failure.

Implementation may restrict this p2p request message method, UPLOAD_METADATAFILE_TEMP, only for the local chunk-level p2p storage service. The local chunk-level p2p storage service runs on the same peer as the client which issue this p2p request message method, UPLOAD METADATAFILE TEMP.

Removal of a previously uploaded metadata file from the temporary metadata file store of a chunk-level p2p storage service

No explicit p2p request message method is required as removal of metadata files from the temporary metadata file store are supposed to be automatic as and when space run out or fell below a certain thre

Retrieval of metadata files from the temporary metadata file store of a chunk-level p2p storage service

No retrieval of metadata files from the temporary metadata file store of a chunk-level p2p storage service is desired to protect privacy of users. Therefore, no p2p request message method for this purpose is provided.

[D4.18] Limited Tracker Announcements by a chunk-level p2p storage service

Innovation disclosed here is an efficient method of limited Tracker Announcements 107 (FIG. 1) by a chunk- level p2p storage service 105 (FIG. 1) comprising only the first chunk of each stream of all metadata files from both permanent and temporary metadata file stores of a chunk-level p2p storage service are announced to the Master chunk-level p2p Tracker 100 (FIG. 1), provided such chunks exist in the chunk store of the chunk-level p2p storage service, and for those chunks do not exist, the Tracker Announcement is skipped. An Internet Services Provider (ISP) may intercept these Tracker Announcements and forward them to the ISP's chunk-level p2p tracker 103 (FIG. 1).

The p2p request message method for Tracker Announcement by the Chunk ID is as follows:

TRACKER ANNOUNCE Chunk-ID digest[\r\n]

The "Chunk-ID" is the unique identification of the chunk, in text format.

The "digest" is a cryptographic digest of the chunk, in text format. E.g. SHA256. The implementation of this method specifies which cryptographic hash function is accepted for the digest.

The steps of the method:

1. Read a metadata file.

2. Identify the first chunk of next stream. E.g. First chunk of video stream, first chunk of audio stream, etc.

3. Check the chunk identified exists in the chunks store of the chunk-level p2p storage service. If does not exists, goto step 2.

4. If the chunk identified exists in the chunks store of the chunk-level p2p storage service, issue a

Tracker Announcement (TRACKER ANNOUNCE). Goto step 2.

5. Goto step 1.

Perform step 1. to 5. for all metadata files of both permanent and temporary metadata file stores of the chunk-level p2p storage service. This limited Tracker Announcements by a chunk-level p2p storage service is expected to be performei immediately or after few minutes after the chunk-level p2p storage service boots up and periodically, every 24hrs, etc.

[D4.19] A method of removal of chunks from a chunk-level p2p storage service

When it triggered automatically to remove chunks from a chunk-level p2p storage service 105 (FIG. 1) to reclaim space, select a metadata file to be removed from the temporary metadata file store of a chunk-level p2p storage service and remove all chunks referenced in that metadata file and finally remove the metadata file itself.

The said automatic triggering may happens by the chunk-level p2p storage service 105 (FIG. 1) when the drive (hard disk or Solid State Drive) space fell below a certain threshold or insufficient space to save a chunk.

The method of selection of a metadata file to be removed leaves for the implementation, however, the chunk- level p2p storage service may select first the oldest metadata file of a partially downloaded content from the temporary metadata file store. A partially downloaded content is, if it is a video content, not all video chunks were downloaded, if it is an audio content without a video stream, then not all audio chunks were

downloaded, if it is a non audio visual content (eg. data file, program installation package, operating system ISO file, etc.), then not all of its chunks were downloaded.

The steps of the method:

1. Read the selected metadata file.

2. Remove all chunks referenced in that metadata file.

3. Finally, remove the metadata file itself.

This method of removing chunks, that is, remove all chunks of a metadata file or nothing, makes sure if one chunk of a metadata file available, then all the other downloaded chunks of that metadata file are available.

[D4.20] A method of identifying peers by a chunk-level p2p storage service

The objective of the method is to avoid query or getting peer list from the chunk-level p2p Tracker for each and every chunk require to download to lower the unnecessary load from a chunk-level p2p Tracker.

As per the innovation disclosed in "[D4.19] A method of removal of chunks from a chunk-level p2p storage service", it can be safely assumed, if one chunk of a metadata file available in a chunk-level p2p storage service, then all the other downloaded chunks of that metadata file are also available.

Therefore, this method comprising, first get the peer list from the chunk-level p2p Tracker 103 (FIG. 1) for the first chunk of the interested stream and caches the returned peer list in the chunk-level p2p storage service 105 (FIG. 1), and uses that cached peer list to download other chunks of the interested stream.

As per the "[D4.1] Provide a configuration method for a chunk-level p2p Tracker in its

configuration file, to specify the host name or the Internet Protocol (IP) address of a paired chunk-level p2p storage service", if an Internet Services Provider (ISP) hosts one or more chunk-level p2p Trackers 103 (FIG. 1) and one or more chunk-level p2p storage services 104 (FIG. 1), IP address of the paired chunk-level p2p storage service 104 (FIG. 1) of the ISP is always added to the peer list returned 116 (FIG. 1) from the chunk-level p2p Tracker 103 (FIG. 1) of the ISP. Practically, the IP address of the paired chunk-level p2p storage service 104 (FIG. 1) of the ISP is either at the beginning or at the end of the peer list returned. If a chunk does not exists with an ISP's local peers, the returned peer list 116 (FIG. 1) contain only the IP address of the paired chunk-level p2p storage service 104 (FIG. 1) of the ISP. The IP address of the paired chunk-level p2p storage service 104 (FIG. 1) of the ISP is required to contact the ISP to get the ISP to download the required chunk from the Internet, which is to the most probability chargeable. Therefore, it is required to exclude the IP address of the paired chunk-level p2p storage service 104 (FIG. 1) of the ISP before cache the returned peer list 116 (FIG. 1) or updates the cached peer list.

And to utilise the cached peer list to download other chunks, a new p2p request message method is used. The significant difference of the new p2p request message method to download a chunk is the identification of the first chunk is always provided along with the identification of the required chunk. The p2p request message method to get a chunk from the local chunk-level p2p storage service 105 (FIG. 1) comprises Required-ChunkID, Digest, and First-ChunklD, as parameters. The "Required-ChunklD" is the identification of the chunk required to be downloaded, the "Digest" is cryptographic digest of the Required Chunk, and the "First-ChunklD" is the identification of the first chunk of the required content stream.

The P2P request message method format is as follows:

GET CHUNK LOCAL2 <Required-ChunklD> <Digest> <First-ChunklD>[\r\n]

This method further comprising following steps:

(1) Once the chunk-level p2p storage service 105 (FIG. 1) receives this method, the chunk-level p2p storage service 105 (FIG. 1) accesses a table within itself with the provided "First-ChunklD" as the key to find the peers share that first chunk. This table comprising with, essentially, two columns, "First-ChunklD" and "IP-list". The "First-ChunklD" column contains identifications of first chunks of each stream being downloaded. The "IP-list" column contains a list of Internet Protocol (IP) addresses of peers share the respective first chunk.

(2) If the said table does not exists, the chunk-level p2p storage service 105 (FIG. 1) creates an empty table. If the table already exists, but there is no corresponding entry for the provided "First-ChunklD", the chunk-level p2p storage service 105 (FIG. 1) gets the peer list from the chunk-level p2p Tracker 100 or 103 (FIG. 1) for the given "First-ChunklD" and enters an entry in to the table excluding the IP address of the ISP's chunk-level p2p storage service 104 (FIG. 1), ie., drop the last or first IP address, respectively. If there is a corresponding entry in the table for the provided "First-ChunklD", the chunk- level p2p storage service 105 (FIG. 1) uses the list of IP addresses in the "IP-list" to download the chunk identified by "Required-ChunkID".

(3) If the "Required-ChunkID" not available in any one of the IP addresses in the relevant "IP-list" or the "IP-list" is empty, then the chunk-level p2p storage service 105 (FIG. 1) get the peer list from the chunk-level p2p Tracker 100 or 103 (FIG. 1) for the given "Required-ChunkID", updates the relevant "IP-list" if there are newer IP addresses excluding the IP address of the ISP's chunk-level p2p storage service 104 (FIG. 1). If there are newer IP addresses, use those newer IP addresses to download the chunk identified by "Required-ChunkID". If there are no newer IP addresses, use the excluded IP address of the ISP's chunk-level p2p storage service 104 (FIG. 1) to download the chunk identified by "Required-ChunkID".

This method returns: (a) A response message with status code for success followed by the full file system path of the chunk or (b) A response message with status code and description for failure.

Every N number of minutes, the chunk-level p2p storage service 105 (FIG. 1) goes through the said table and updates every "IP-list" with new peers for each "First-ChunklD" by getting a new peer list from the chunk- level p2p Tracker 100 or 103 (FIG. 1) for the "First-ChunklD". Table entries not accessed for more than N number of minutes, should be removed from the table.

[D4.21] Disable all forms of Tracker Announcements of a chunk-level p2p storage service, and optionally, enable a one or more forms of Tracker Announcements

Tracker Announcements 107 and 112 (FIG. 1) by a chunk-level p2p storage service 104 and 105 (FIG. 1) for its chunks is a fundamental requirement in p2p based file sharing. But not all chunk-level p2p storage services require to Tracker Announce for its chunks. According to the extended chunk-level p2p system disclosed here, there can be caching only chunk-level p2p storage services 123 (FIG. 1) not connected to Internet but offer services over Local Area Network. Such caching only chunk-level p2p storage services do not require to Tracker Announce for its chunks. Internet Services Providers (ISPs) may only offer a limited form of Tracker Announcements (eg. for chunk uploads only, etc) on their chunk-level p2p storage services 104 (FIG. 1).

Disclosed here are the configuration methods for a chunk-level p2p storage service to disable Tracker Announcements for its chunks.

The default behaviour of a chunk-level p2p storage service is to Tracker Announce for its chunks when a new chunk comes to its chunk store or periodically for its chunks. In the configuration file for the chunk-level p2p storage service, a configuration method could be supplied to override the default behaviour and disable Tracker Announcements for its chunks as follows:

TRACKER ANNOUNCE YES| NO

If this configuration method does not exists in a configuration file, it does not alter the default behaviour of Tracker Announce for its chunks.

If the "TRACKER AN NOU NCE YES", is equivalent to the default behaviour of Tracker Announce for its chunks.

If the "TRACKER_ANNOUNCE NO", exists in a configuration file, it does alter the default behaviour and disable Tracker Announcements for its chunks. This includes disabling: (1) First-time Tracker Announcements after boot up the chunk-level p2p storage service for all chunks in the chunk store, (2) Tracker Announcement when a new chunk arrive in the chunk store, (3) Periodic Tracker Announcements for all chunks in the chunk store.

After completely disable the Tracker Announcement of a chunk-level p2p storage service, depending on the usage, it may be required to enable a one or more forms of Tracker Announcements:

ENABLE TRACKER ANNOUNCE BOOTUP YES| NO

ENABLE TRACKER ANNOUNCE UPLOAD YES|NO

ENABLE TRACKER ANNOUNCE DOWNLOAD YES| NO

ENABLE TRACKER ANNOUNCE PERIODIC YES|NO

The ENABLE TRACKER ANNOUNCE BOOTUP, ENABLE TRACKER ANNOUNCE UPLOAD,

ENABLE TRACKER ANNOUNCE DOWNLOAD, and ENABLE TRACKER ANNOUNCE PERIODIC are only applicable if the Tracker Announcements by a chunk-level p2p storage service is completely disabled. That is, "TRACKER ANNOUNCE NO" is specified.

The ENABLE TRACKER ANNOUNCE BOOTUP, could be used to enable the first-time Tracker

Announcements after boot up the chunk-level p2p storage service for all chunks in the chunk store of the chunk-level p2p storage service.

The ENABLE TRACKER ANNOUNCE UPLOAD, could be used to issue Tracker Announcements for chunks that are uploaded from another chunk-level p2p storage service 105 (FIG. 1). If an Internet Services Provider (ISP) runs the ISP's chunk-level p2p storage service 104 (FIG. 1) with Tracker Announcement disabled, then it is required to enable Tracker Announcements for chunks uploaded by ISP's customers using this configuration method so that those chunks are notified to the Master chunk-level p2p Tracker 100 (FIG. 1).

The ENABLE TRACKER ANNOUNCE DOWNLOAD, could be used to issue Tracker Announcements for chunks that are downloaded from another chunk-level p2p storage service.

The ENABLE TRACKER ANNOUNCE PERIODIC, could be used to enable the periodic Tracker

Announcements for all chunks in the chunk store of the chunk-level p2p storage service.

Alternatively, a p2p request message method also could be used to disable Tracker Announcements by a chunk-level p2p storage service for its chunks.

The p2p request message method to disable Tracker Announcements as follows:

DISABLE TRACKER ANNOUNCE [session-key][\r\n]

The p2p request message method to enable Tracker Announcements as follows:

ENABLE TRACKER ANNOUNCE [session-key][\r\n]

The p2p request message method to enable the first-time Tracker Announcements after boot up the chunk- level p2p storage service as follows:

ENABLE TRACKER ANNOUNCE BOOTUP [session-key][\r\n]

The p2p request message method to issue Tracker Announcements for chunks that are uploaded as follows: ENABLE TRACKER ANNOUNCE UPLOAD [session-key][\r\n] The p2p request message method to issue Tracker Announcements for chunks that are downloaded as follows:

ENABLE TRACKER ANNOUNCE DOWNLOAD [session-key][\r\n]

The p2p request message method to enable the periodic Tracker Announcements as follows:

ENABLE TRACKER ANNOUNCE PERIODIC [session-key][\r\n]

The chunk-level p2p storage service sends either a response message with status code for success or a response message with status code and description for failure.

[D4.22] Generating an accounting entry for billing purpose for a chunk downloaded, on behalf of a client, from Internet by a chunk-level p2p storage service

As per the "[D4.1] Provide a configuration method for a chunk-level p2p Tracker in its

configuration file, to specify the host name or the Internet Protocol (IP) address of a paired chunk-level p2p storage service", if an Internet Services Provider (ISP) hosts one or more chunk-level p2p Trackers 103 (FIG. 1) and one or more chunk-level p2p storage services 104 (FIG. 1), the IP address of the paired chunk-level p2p storage service 104 (FIG. 1) of the ISP is always added to the peer list returned 116 (FIG. 1) from the chunk-level p2p Tracker 103 (FIG. 1) of the ISP.

If a chunk does not exists with an ISP's local peers, the returned peer list 116 (FIG. 1) contain only the IP address of the paired chunk-level p2p storage service 104 (FIG. 1) of the ISP. The returned only IP address is then used to connect to the paired chunk-level p2p storage service 104 (FIG. 1) of the ISP to get the ISP to download the required chunk from the Internet. The downloaded chunk is then cached in the paired chunk- level p2p storage service 104 (FIG. 1) of the ISP and a copy of the chunk is then streamed to the client.

After the chunk is downloaded by the chunk-level p2p storage service 104 (FIG. 1) of the ISP, it is possible to generate an accounting entry on to a log file, or issue a p2p request message method, or both, to an accounting server of the ISP for billing purposes. This allows chunks downloaded from the Internet for a customer by the ISP to be charged very differently than a chunk downloaded from another peer of the same ISP. With this ability to identify and charge for chunks downloaded from Internet (a.k.a. International traffic), for chunks downloaded from another peer of the same ISP (a.k.a. Local traffic), can be even offer free of charge or at a fixed rate.

In the configuration file of the ISP's chunk-level p2p storage service 104 (FIG. 1), following configuration methods could be specified:

PRE DOWNLOAD ACCT SERVER host | IP address

PRE DOWN LOAD ACCT pre_download_acct_spec

POST DOWNLOAD ACCT SERVER host | IP address

POST DOWN LOAD ACCT post _download_acct spec

The objective of the PRE_DOWNLOAD_ACCT method is to specify a p2p request message method to be sent to get the customer's account identification by providing the IP address of the customer, the method may return other session info, etc. The $CUSTOMER_IP, $YYYY-MM-DD and $HH-MM-SS, in the

pre _download_acct spec will be replaced with IP address of the client, current date, and current time, respectively.

The objective of the POST_DOWNLOAD_ACCT method is to specify a p2p request message method to be sent to provide information to the ISP's accounting server in full. The $CUSTOMER_IP, $CHUNK_SIZE, $YYYY- MM-DD, $HH-MM-SS, and $PRE_DOWNLOAD_ACCT_OUTPUT, in the post_download_acct_spec will be replaced with IP address of the customer, size of the chunk in bytes, current date, and current time, and output of the PRE DOWNLOAD ACCT method, respectively.

If the POST_DOWNLOAD_ACCT method does not exists in a configuration file of a chunk-level p2p storage service 104 (FIG. 1) of the ISP, skip the PRE DOWNLOAD ACCT method even if it is specified, and chunks will be downloaded without logging or issuing p2p request message methods for accounting purposes by the chunk-level p2p storage service 104 (FIG. 1) of the ISP. [D4.23] Private chunks

One may not share all of his or her contents over a p2p network. Some contents are private in nature, some contents may not be legal to share and only for personal consumption. Such contents intend only for private consumption, can be brought to a local chunk-level p2p storage service 105 (FIG. 1) as private chunks.

One of the fundamental features of private chunks is, private chunks are not possible to share on the p2p overlay network. Therefore, private chunks only resides on a local chunk-level p2p storage service 105 (FIG. 1).

Functionality of a chunk-level p2p storage service is required to be further extended to support private chunks.

Properties of private chunks:

(1) Chunk identifications (ChunklDs) are prefixed or suffixed with a tag to identify as private chunks.

(2) Segmentation process does not fetch an unique ChunkID from an external server.

(3) Private chunks are not shareable, therefore, not uploaded to any chunk-level p2p storage service, other than the local chunk-level p2p storage service 105 (FIG. 1).

(4) Private Chunks resides only on the local peer (a.k.a local chunk-level p2p storage service).

(5) No Tracker Announcements are issued for private chunks.

(6) Private chunks are not automatically removed to reclaim space from the local chunk-level p2p storage service.

(7) Private chunks are not downloaded if not available in the local chunk-level p2p storage service. Method of generating private chunks

Private chunks are generated by the content segmentation process. Type of the chunks to be generated, ie. private chunks or public chunks, can be specified to the segmentation process as an extra parameter. The identification difference of a private chunk and a public chunk is only in the chunk identification. The identification of a private chunk is either prefixed or suffixed with a consistent tag to indicate it's a private chunk. A single character such as an underscore (_) or a dot (.) is sufficient as either a prefix or as a suffix. A chunk is identified as a private chunk is only by this tag.

Steps of generating private chunks:

1. Generate the chunk as usual.

2. Instead of getting an unique chunk identification from an external server, use a cryptographic hash function (eg. SHA256) to get the digest to be used as the chunk identification by providing the chunk as the input. Any other method of generating a chunk identification can be used.

3. Add an underscore (_) as the prefix to the cryptographic digest obtain in step 2. Use the prefix+digest combination as the identification of the private chunk.

[D4.24] Intercept connections going to remote chunk-level p2p Trackers and forward such connections to Internet Services Provider's (ISP's) chunk-level p2p Tracker

Intercept connections going out of Internet Services Provider's (ISP's) Internet Protocol (IP) address space to remote chunk-level p2p Trackers, excluding such connections generated from the Internet Services Provider (ISP) itself, at a point of network termination by one or more routers belong to the ISP and forward such connections to a chunk-level p2p Tracker running at the ISP itself. If there are multiple chunk-level p2p Trackers running at the ISP, then router forwards the connection to one of the chunk-level p2p Trackers.

End user's chunk-level p2p storage service requires to announce availability of chunks via Tracker

Announcements to the relevant Master chunk-level p2p Tracker. End user's chunk-level p2p storage service requires get the peer list from its relevant Master chunk-level p2p Tracker.

For both Tracker Announcement and getting a peer list requires in first place establish a connection with its relevant Master chunk-level p2p Tracker.

Intercept connections going out of Internet Services Provider's (ISP's) Internet Protocol (IP) address space to remote chunk-level p2p Trackers, excluding such connections generated from the Internet Services Provider (ISP) itself, at a point of network termination by one or more routers belong to the ISP and forward such connections to the ISP's chunk-level p2p Trackers, so that Tracker Announcements populates ISP's chunk-level p2p Trackers with IP addresses belong to ISP's own IP address space, i.e., with local peers, and getting peer lists receive peers within ISP's IP address space, i.e., receive local peers.

[D4.25] Intercept connections going to remote chunk-level p2p storage services and forward such connections to Internet Services Provider's (ISP's) chunk-level p2p storage service

Intercept connections going out of Internet Services Provider's (ISP's) Internet Protocol (IP) address space to remote chunk-level p2p storage services, excluding such connections generated from the Internet Services Provider (ISP) itself, at a point of network termination by one or more routers belong to the ISP and forward such connections to a chunk-level p2p storage service running at the ISP itself. If there are multiple chunk- level p2p storage services running at the ISP, then router forwards a connection to one of the chunk-level p2p storage services.

Receiving connections going out of ISP's IP address space to remote chunk-level p2p storage services, gives an ISP an opportunity to download the interested chunk and keep a copy and serve the request to the client. Subsequent requests for the same chunk from different clients could be served from the saved copy of the chunk thereby saving Internet bandwidth.

[D5] DETAILS OF CARRY OUT THE INNOVATION

The chunk-level p2p Tracker and chunk-level p2p storage service are central to the operation of the extended chunk-level p2p system, therefore, get them implemented as described here. Any communication (i.e.

connections) from an user to an out of Internet Services Provider (ISP) chunk-level p2p Tracker or a chunk- level p2p storage service, must be forwarded to ISP's chunk-level p2p Tracker or chunk-level p2p storage service as per [D4.24] and [D4.25] respectively. Downloading and sharing chunks as per the extended chunk-level p2p system should be apparent to those skilled in the art.

Although the methods and systems has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent also to those skilled in the art.