FIELD

[001 ] The present disclosure relates generally to mobile device communications and, more particularly, to systems and methods that facilitate local multiplayer gaming.

BACKGROUND

[002] Smartphone ownership has seen tremendous rate of growth in recent years. One of the fastest growing sectors of smartphone applications is mobile games. In fact, mobile games consistently dominate the list of top applications downloaded in both Android and iOS markets. The popularity of mobile games has led to innovations in both hardware and software, e.g., from the introduction of mobile quad-core CPUs to 3D sensor-based games.

[003] Smartphones offer opportunities for mobile game players to engage in highly interactive, local multi-player game activities, also known as LAN parties. These activities offer high levels of satisfaction and entertainment to the participants, as they allow them to interact with each other on the spot while playing the games. With a traditional gaming platform, such as PCs, gamers have to make a significant effort to setup a LAN game party, e.g., planning the time and arranging a powered table space. In contrast, smartphones are usually carried by the owners, operate on battery, and have small footprints, making them an ideal gaming platform for spontaneous, local multiplayer games.

[004] The strategies used to synchronize game state in multi- player games can be divided in two groups: client-server and peer-to-peer. In the first approach, players synchronize their local game states with the authoritative state held by the server. In the peer-to-peer approach, no single peer is responsible for the game state. Instead, the state is synchronized using a distributed protocol run by all players. The advantages of the client-server model are that it is easier to implement and more resilient to cheating. The appeal of the second approach is that it obviates the need of expensive servers and no node in the network must be present for the duration of the game. There are also efforts to build peer-to-peer based games that are resilient to cheating.

[005] The recent popularity and powerful capabilities of smartphones and mobile devices (including tablets) have enabled development of multiplayer games over wireless networks. Current commercial engines essentially re-use the approaches designed for the wired setup over wireless. Highly interactive multiplayer games, such as FPS, have been implemented both over LAN and wide-area networks (WAN); the latter have much larger network latency. Commercial solutions employ mechanisms that can cope with this large latency over WANs, and simply reuse these mechanisms over the lower latency LANs as well. In the academic community, solutions have been proposed to deal with the high latency of the wireless WAN links and the performance of various broadcasting schemes, such as flooding and distance- based broadcast, has been studied. Multicast has also been proposed to distribute game commands and states, but has not been used in commercial systems. SUMMARY

[006] A system and method for local multiplayer gaming are disclosed. According to one embodiment, a system comprises a plurality of mobile client devices including a mobile client device designated as a server device. Each mobile client device comprises a display, at least one local network interface for communication with said server device and a processor. The processor is programmed for playing a multiplayer game with other mobile client devices and rendering on said display a game situation depending on commands sent by a plurality of mobile client devices to said server device over said at least local network interface. At least one of said mobile client devices is configured for overhearing commands sent by at least one other mobile client device to said server device and for rendering on its display a game situation depending on said overheard command.

[007] The systems, methods, features and advantages of the invention will be or will become apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the accompanying claims. It is also intended that the invention is not limited to require the details of the example embodiments.

BRIEF DESCRIPTION

[008] The accompanying drawings, which are included as part of the present specification, illustrate the presently preferred embodiment and, together with the general description given above and the detailed description of the preferred embodiment given below, serve to explain and teach the principles of the present invention.

[009] Figure 1 illustrates an exemplary networking model for use with the present system, according to one embodiment.

[010] Figure 2A illustrates game latency in a prior art multiplayer networking framework.

[01 1 ] Figure 2B illustrates game latency in accordance with the present system, according to one embodiment.

[012] Figure 3 illustrates an exemplary game architecture for use with the present system, according to one embodiment.

[013] Figure 4 illustrates an exemplary multi player game method for use with the present system, according to one embodiment.

[014] Figure 5 illustrates an exemplary interface to an exemplary game for use with the present system, according to one embodiment.

[015] Figure 6 illustrates the percentage of commands lost, averaged over all players when 4 players are sitting around a table.

[016] Figure 7 illustrates the time needed to send one state packet as the number of players increases, according to one embodiment of the present system.

[017] It should be noted that the figures are not necessarily drawn to scale and that elements of similar structures or functions are generally represented by like reference numerals for illustrative purposes throughout the figures. It also should be noted that the figures are only intended to facilitate the description of the various embodiments described herein. The figures do not necessarily describe every aspect of the teachings disclosed herein and do not limit the scope of the claims.

DETAILED DESCRIPTION

[018] A system and method for local multiplayer gaming are disclosed. According to one embodiment, a system comprises a plurality of mobile client devices including a mobile client device designated as a server device. Each mobile client device comprises a display, at least one local network interface for communication with said server device and a processor. The processor is programmed for playing a multiplayer game with other mobile client devices and rendering on said display a game situation depending on commands sent by a plurality of mobile client devices to said server device over said at least local network interface. At least one of said mobile client devices is configured for overhearing commands sent by at least one other mobile client device to said server device and for rendering on its display a game situation depending on said overheard command.

[019] Smartphones and other mobile devices (including tablets) are an ideal platform for local multiplayer games, thanks to their computational and networking capabilities as well as their popularity and portability. However, existing game engines do not exploit the locality of players to improve game latency.

[020] The present system is directed to a complete networking framework for local multiplayer mobile games. The present system is also directed to low delay multi party applications. The present system exploits local connections between mobile devices and the broadcast nature of the wireless medium to provide smooth, accurate rendering for all players with two desired properties. First, it performs direct-input rendering (i.e., without any inter- or extrapolation of game state) for all players. Second, it provides very low latency. Implementation of the present system in an exemplary prototype (e.g. on Android phones with an exemplary multiplayer racing game) indicates that cars can be rendered smoothly, without any prediction of state, and with only 20- 40 ms game latency (for example).

[021 ] One challenge of developing multi-player games is game latency, i.e., the delay between when one of the players inputs a command and when the results of the command are rendered on the screens of all players. Local multiplayer games on smartphones have two important characteristics: (i) the phones are connected via a common wireless network (e.g., WiFi) and (ii) the players are typically close to each other. The present system exploits these two characteristics to significantly reduce game latency and, at the same time, simplifies multi- player games development. In particular, the present system eliminates the need for game state prediction.

[022] The present system exploits local connections and overhearing, for

collaboration between devices within proximity of each other. The present system is directed to an application scenario including multiplayer games. An objective of the present system is to deliver game-related packets, which are much smaller than video packets, to all players with minimum latency.

[023] According to one embodiment, the present system explicitly uses local connections between smartphones and exploits overhearing over the shared wireless medium (WiFi). In particular, games are hosted locally, i.e., one of the phones acts as a server and the others act as clients. The packets sent to the server can be overheard by other players' devices. Using the overheard packets, the local game engine of each player can render the movements of the other players precisely, without the need to make predictions about the other players' movements.

[024] This design brings benefits both in terms of performance (low game latency, sufficient even for games with the most stringent delay requirements, such as first person shooter (FPS) and racing games) and simplified game development (unlike other engines, it does not rely on game state prediction). Furthermore, it enables spontaneous setup of local multiplayer games.

[025] According to one embodiment, the present system provides a user interface that enables phones to easily setup a local network and establish all the necessary connections to play a game on- the-fly, without the need of existing network infrastructure, such as local access point or Internet connectivity. This significantly reduces the overhead of setting up multiplayer games.

[026] The present system can be applied to any mobile device, including without limitation a mobile phone, a tablet, a music player device, a GPS mobile navigation device, a camera, a video camera, or a personal digital assistant (PDA). The term player is used interchangeably with phone and/or mobile device throughout this description.

[027] The expression "typically close to each other" related to mobile devices indicates that the distance between mobile devices is such that the mobile devices can communicate by exploiting, e.g., a Bluetooth, WiFi, ZigBee or NFC interface. For example this distance can be greater than 20 m, less than 5 m, or 2 m. The number of mobile devices in the group can be also greater than 6.

[028] Although the examples described herein are related to the mobile device Android-based, the present system can be applied to mobile devices using other operating system as for example Symbian, Apple iOS, RIM BlackBerry, MeeGo, Windows Phone, Boot2Gecko (B2G) and Bada.

[029] Although the examples described herein are related to a multiplayer gaming application, the present system can be applied to other multi party applications where low latency is desired.

[030] Other programming languages, different from the mentioned Java and C, can be used in the context of the present system, for example and not in a limiting way C++, Python, or Celyon.

[031 ] Although mention is made of multiple electronic modules, it will be appreciated that all of the electronic modules described herein can be the same electronic module.

[032] To illustrate the present system, an implementation in Java and C is described herein. An exemplary multiplayer racing game, referred to herein as Racer (based on AndEngine, an open-source Android game engine) is used for performance comparisons herein.

[033] The present application is related to U.S. Patent Application Serial No.

13/841 ,500, titled "System and Method for Cooperative Data Streaming," filed on March 15, 2013, the disclosure of which is incorporated herein in its entirety.

[034] Figure 1 illustrates an exemplary networking model for use with the present system, according to one embodiment. According to one embodiment, the present system adopts a client-server model similar to what is currently used by the majority of commercial game engines. In this model, one of the mobile devices in a group of mobile devices acts as the server 101 and the rest act as clients 102, 103, 104. The server 101 and the clients 102, 103, 104 participate in the game. The server 101 is responsible for updating and distributing the game state. To this end, the server 101 and clients 102, 103, 104 communicate with each other using a local WiFi network 105.

[035] According to one embodiment, when a client 102 inputs a command, the command is put into a command packet and sent to the server 101. The server processes the command and updates the game state. The server sends, periodically, the updated game state (i.e. , snapshot of the game) in a state packet to all of the clients 102, 103, 104. The clients 102, 103, 104 then render the new game state. According to one embodiment, command packets are sent by the clients very frequently, e.g., every 12-20 ms, and state packets are sent by the server less frequently, e.g. every 50-80 ms, as illustrated in Figure 2A.

[036] Figure 2A illustrates game latency 201 in a prior art multiplayer networking framework. If a client rendered the game based only on the discrete game states sent by the server, moving objects would look choppy and jittery. Also, missing state packets would cause noticeable glitches. To address these problems, the present system buffers the received states and renders states that are slightly in the past (illustrated in Figure 2A). The client has received up to snapshot 4; however, it has only rendered up to snapshot 2 and still buffers 3 and 4.

[037] With this buffering technique, first, animation of objects between game states can be smoothly rendered by interpolating the state of the objects between the latest rendered game state and the immediate next game state, e.g., snapshots 2 and 3 in Figure 2A. Second, loss of a single game state update can be tolerated. For instance, if snapshot 3 is lost, the client can still render the game by interpolating between snapshots 2 and 4. The interpolation time 203 is usually chosen large enough to include two snapshots. Furthermore, when no information about an existing object is available in the buffered snapshots, the client will try to extrapolate the object state, i.e., predicting its future state, using the currently known information (e.g., velocity and position).

[038] However, buffering also causes a constant game latency, as shown in Figure 2A. The delayed visual feedback due to game latency makes it hard to move or aim precisely. To overcome this, each player's local game engine adopts a technique called input prediction, where the client predicts the results of its own user commands and renders them immediately (referred to herein as direct-input rendering). This makes the game feel more interactive. For example, in Figure 2A, although the rendering time 202 only includes state 2, the player updates its own state using its own input commands up to the current time 204. With direct-input rendering, the client does not have to interpolate its own player since exact commands are executed. Thus, the rendering of the local player is accurate. Direct- input rendering, however, is not possible for other players, and the local game engine has to resort to inter- or extrapolation to render them.

[039] According to one embodiment, the present system exploits the nature broadcast of WiFi to enable direct-input rendering for all players. In particular, the present system allows a client to overhear command packets sent by all other clients to the server and use them to perform direct-input rendering.

[040] Figure 2B illustrates game latency 207 in accordance with the present system, according to one embodiment. Figure 2B illustrates how a client (client 2 205) performs direct-input rendering for objects of another client (client 1 206). There are two key benefits with this approach: precise rendering and low game latency. [041 ] When direct-input rendering is available for all players, the rendering of other players and their objects can be done similar to the local ones. Hence, there is no longer need for performing inter- or extrapolation. Note that direct-input rendering is strictly more accurate than rendering using these predictions. The elimination of state prediction code also brings many side benefits: it simplifies game development (simpler and more maintainable objects' update code), reduces processing needed at each phone, and improves scalability when the number of players or interactive objects increases.

[042] The interpolation time can be eliminated since (i) no interpolation is needed and (ii) the loss of state packets is not as critical as before, thanks to the overheard command packets. In the present system, the interpolation time is reduced to the average in-game frame rendering time, 20 ms (50 frames per sec), as illustrated in Figure 2B.

[043] Figure 3 illustrates an exemplary game architecture for use with the present system, according to one embodiment. Phones participating in the game are organized in an 802.1 1 basic service set (BSS). One of the phones, the server, acts as an access point (AP) while the other phones act as clients and connect to it. A connection module 306, as part of a multi player game engine 302, sets up this network and configures the IP connectivity when the game starts. The connection module 306 enables a software-based access point (soft AP) and a DHCP server. The server can broadcast frames to all clients; clients can only unicast frames to the server. Each client maintains a TCP connection to the server, used as a reliable control channel, and also listens for UDP packets. The server broadcasts UDP messages using IP broadcast and the clients unicast UDP packets to the server. According to one embodiment, the connection module 306 can be implemented using non-public APIs provided by, example, the Android WiFiManager. Once the local AP is enabled, the connection module 306 opens a UDP socket that is subsequently used to broadcast state packets and to receive command packets. It also binds a TCP socket to receive connections from clients. If the player wants to join an existing game (user input 313), the standard API of WiFiManager is used to discover the BSS created by the server, to connect to it and to obtain an IP address. After acquiring an IP address, the module establishes a TCP connection to the server. The TCP channel is used to transmit control packets that notify clients of connections and disconnections of players.

[044] An overhearing module 305 provides an API to let each client overhear UDP command packets sent by all other clients to the server. To enable overhearing of UDP packets, the overhearing module 305 opens a raw socket and sets the network interface 301 into promiscuous mode by running an external executable daemon called overheahng _d as root. Overhearing is necessary because when connected to an AP, the clients cannot broadcast frames; they have to first send them to the AP that will then broadcast them. Therefore, without overhearing, it is not possible to achieve a direct client-to-all communication.

[045] According to one embodiment, the overhearingd daemon is implemented in C. It filters packets received from the raw socket and delivers the relevant packets to the overhearing module using a named pipe. The overhearing module 305 contains a JNI library that wraps the system call select to perform non-blocking reads from the named pipe, which is not possible using only the Android Java API. [046] A server module 303 runs on the server and periodically takes snapshots of the game and sends them in state packets to all clients using IP broadcast. The server module 303 detects client disconnection as well as accepts new client connections.

[047] A client module 304 runs on the server (as the server is also a player) and all clients. The client module 304 periodically samples input commands 313 of the local player and sends these commands in command packets to the server. It parses overheard packets to obtain command packets.

[048] According to one embodiment, the client module 304 and server module 303 are implemented as update handlers of a game engine (referred to in Figure 3 as Engine). Update handlers are called once per screen redraw. This single thread approach was chosen to tightly control the scheduling of network operations. Since the handler code is run in the main game rendering thread, it cannot block. For this reason, all I/O operations are performed using non-blocking calls provided by the Android NIO API. Note that both client and server modules can be ported to other game engines as independent threads.

[049] A command module 311 serializes commands performed by the local user (user input 312 and engine input 312) to a binary representation that is then put in command packets and sent by the client module 304. The command module 311 also parses the serialized commands contained in the received command packets. Finally, the command module 311 applies the commands generated by the local player and overheard from the other players to the local game scene 308 that is rendered 309 on screen 310. Note that command packets contain a generation time and are applied to the local scene only when the rendering time reaches it. [050] On the server, a game state module 307 serializes the current local scene to a binary representation that is then sent in state packets. On a client, the game state module 307 updates the local scene using the content of received state packets. Every state packet is accompanied with a timestamp that indicates when it has been created. Similar to a command, a game state is only processed when the rendering time reaches it.

[051 ] Furthermore, the game state module 307 only applies a game state if the current scene, which is the result of applying the overheard command packets, is significantly different from the received game state. This situation is referred to as a sync error. This error may come from a variety of sources, such as missing command packets or diverging results of physics simulation on the server and the clients due to difference of system clocks. The game developer decides when to correct the local scene based on the type of games since in many games, minor errors can be tolerated.

[052] The authoritative game time is the server time. Clients synchronize their game time based on the time indicated in the periodic state packets sent by the server. This approach does not take into account the one-way trip time between the server and the clients, which is negligible (a few ms). With this approach, sporadic time synchronization errors due to transient network conditions and systematic errors due to differences in frequency of the client and server clocks are quickly fixed.

[053] Figure 4 illustrates a multi-player game method for use with the present system, according to one embodiment. A command is sent from a first mobile client device to a server device over a wireless network 401. A second mobile device can overhear the command 402. A game situation in accordance with the overheard command is rendered 405 on a display of the second mobile device.

[054] The server, also in receipt of the command sent from the first mobile device, computes a game situation (e.g. game state) in accordance with the command 403 and periodically sends a command to update the game situation 404. The server can send a command in multicast to update the game situation to a plurality of mobile client devices, or the server can send a command in unicast to one client device and the other client devices can overhear the sent command. A game situation in accordance with the command (received from the server or overheard) is rendered on a second mobile device 405. The server sends update commands periodically, and less frequently than user commands are sent, according to one embodiment.

[055] In order to evaluate an embodiment of the present system, an exemplary multiplayer racing game called Racer is implemented using a game engine. Racer serves as an example application to demonstrate key properties of the underlying engine according to one embodiment of the present system. It will be appreciated that the present method and system is intended to support many different games that can benefit from its capabilities.

[056] Figure 5 illustrates an exemplary interface to an exemplary game for use with the present system, according to one embodiment. In a game session, several players race their cars around a terrain 502. Each player controls her own car using an on-screen analog joystick 501 .

[057] In the exemplary implementation, the command module 31 1 samples an input command by recording the current position of the joystick 501 , which comprises x and y positions. It applies a command by adjusting the velocity and angle of the car based on the pair (x, y). The game state module 307 takes a game snapshot by recording the positions, velocities, and angles of all cars. It applies a snapshot by updating all the players cars from the recorded values.

[058] In Racer, all of the received state and command packets are tracked and plotted. Whenever a car state is updated as a result of applying a command, a blue dot (or in Figure 5 a black circle 503) appears on the screen, at the current position of the car. On the other hand, whenever a car state is updated as a result of applying a game state, a white dot appears on the screen (or in Figure 5 a black star 504). This helps keep track of how the cars are rendered.

[059] The game engine provides a multiplayer extension to support development of multiplayer games. This extension facilitates client-server connection establishment, including automatic server discovery. All players must join an existing WiFi network, and use unicast TCP connections to send all packets from clients to the server and vice versa. As a baseline for comparison, Racer is adapted in one embodiment so that it works with the multiplayer extension. In other words, when using the multiplayer extension, all packets of Racer are sent using unicast sessions over TCP.

[060] Figures 6 and 7 illustrate exemplary experimental results, according to one embodiment of the present system. For evaluation purposes, multiple sessions of Racer are run with up to 6 players. For example, the command sampling interval is set equal to the redrawing interval (frame rate) of the game, at 20-30 ms, and the state sending interval is set at 80 ms. Upper and lower bounds for the box plots in Figures 6 and 7 are the first and third quartiles, and lines in the boxes are the medians.

[061 ] According to one embodiment, the present system relies on the assumption that every client can overhear command packets sent by the others. In order to verify this assumption, the percentage of command packets is measured that each node cannot receive. To detect packet loss, a sequence number is attached to every command packet. During a run of Racer for 20 minutes, packet loss is measured every minute. The experiments were conducted in a residential area with a high number of co-located WiFi networks.

[062] Figure 6 illustrates the percentage of commands lost, averaged over all players when 4 players are sitting around a table. Phones are positioned in a square whose side is 50 cm. The node at position (0,0) is the server. In this experiment, the server does not lose any command packet since all players send command packets to the server as UDP unicast packets and therefore they are re- transmitted multiple times if the server does not acknowledge (ACK) them. Other phones lose some of the updates but no more than 0.25% of them.

[063] Since missing command packets can cause sync error (as discussed above), the number of times that state packets were used to resynchronize game state was recorded. A sync error in Racer is when a car position is 10 pixels off of its correct position. Results indicate, on average, less than 3% of state packets were used for resynchronizing. This shows that majority of the time, it is sufficient to render all players using just the (overheard) command packets.

[064] In another experiment, the effect of distance between the clients and the server is evaluated. The phones were placed on a line with distance between two consecutive phones of 8 cm. The phone at the beginning of the line is the server. The results indicate that the loss rate increases with the distance from the server. Although the loss rate was higher than in the previous experiment, the upper quartile was still below 1 %.

[065] Figure 7 illustrates the time needed to send one state packet as the number of players increases, according to one embodiment of the present system. A significant advantage of the present system is the use of broadcasting to efficiently disseminate state packets to the clients. The median time increases with the number of players when a multiplayer extension is used, but stays flat when the present system is used (less than 2 ms to broadcast the state packet, as an example).

[066] In experiments of up to 6 phones implementing the present system,

observations include that players are rendered at the rate at which their inputs are sampled, which equals the game's frame rate (at 20-40 ms for 25-50 frame per sec, as an example). This latency is much smaller than the latency typically needed for highly interactive multiplayer games (-100 ms, as an example). Smooth rendering is possible without the need of any prediction (interpolation or extrapolation) code.

[067] The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and/or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims.

[068] The functions described may be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the functions may be stored as one or more instructions on a computer-readable medium. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[069] Thus, certain aspects may comprise a computer program product for performing the operations presented herein. For example, such a computer program product may comprise a computer readable medium having instructions stored (and/or encoded) thereon, the instructions being executable by one or more processors to perform the operations described herein. For certain aspects, the computer program product may include packaging material.

[070] Software or instructions may also be transmitted over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of transmission medium.

[071 ] Further, it should be appreciated that modules and/or other appropriate means for performing the methods and techniques described herein can be downloaded and/or otherwise obtained by a user terminal and/or base station as applicable. For example, such a device can be coupled to a server to facilitate the transfer of means for performing the methods described herein. Alternatively, various methods described herein can be provided via storage means (e.g. , RAM, ROM, a physical storage medium such as a compact disc (CD) or floppy disk, etc.), such that a user terminal and/or base station can obtain the various methods upon coupling or providing the storage means to the device. Moreover, any other suitable technique for providing the methods and techniques described herein to a device can be utilized.

[072] A system and method for local multiplayer gaming have been disclosed. It is understood that the embodiments described herein are for the purpose of elucidation and should not be considered limiting the subject matter of the disclosure. Various modifications, uses, substitutions, combinations, improvements, methods of productions without departing from the scope or spirit of the present invention would be evident to a person skilled in the art.