METHODS

CROSS-REFERENCE TO RELATED APPLICATION

[0001 ] The present application claims priority under 35 U.S.C. § 1 19(e) to U.S.

Provisional Application No. 62/517,009, filed on June 8, 2017, and entitled: "Multi- Group Collaboration System and Associated Methods," which is incorporated herein by reference in its entirety.

BACKGROUND

[0002] Collaboration is becoming increasingly critical in numerous scenarios, e.g., active learning scenarios. As discussed in detail below, embodiments are directed to a bring your own device (BYOD) collaboration system that is simple yet supports many different platforms, e.g., is hardware agnostic. Thus, a myriad of collaboration options may be realized with one seamless user experience. In particular, rather than relying on hardware solutions and physical connections, embodiments are directed to software solutions that work seamlessly across a myriad of devices.

SUMMARY

[0003] One or more embodiments is directed to a collaboration system, including a first group display and a first group display computer, the first group display computer to receive mobile device data streams from at least two mobile devices over a wireless access protocol and to output a first group display data stream over a network, a second group display and a second group display computer, the second group display computer to receive mobile device data streams from at least two mobile devices, different from the at least two mobile devices streaming to the first group display computer, over the wireless access protocol and to output a second group display data stream over the network, a common display computer to receive the first and second group display data streams from the network, and a common display to display data streamed from the first and second group display computers in first and second group display windows.

[0004] The common display may broadcast data in the first group display window to the second group display computer or data in the second group display window to the first group display computer. [0005] The common display computer may broadcast a first window on the common display to the first and second group displays.

[0006] The first window on the common display may be one of the first and second group display windows.

[0007] The common display computer may select which of the first and second group display window dominate the common display.

[0008] Ethernet cables may connect the first and second group display computers and the common display computer to the network.

[0009] One or more embodiments is directed to a collaboration system that includesN group displays and N group display computers, where N is a natural number greater than two, each of the group display computers capable of receiving mobile device data streams from at least M mobile devices, different from mobile devices of other group display computers, where M is a natural number greater than two, over a wireless access protocol and to output a group display data stream over a network; a network to receive N group display data streams, a common display computer to receive the N group display data streams from the network, and a common display to display data streamed from any one of the N group display computers in a group display window or full screen.

[0010] The common display to display data may be streamed from multiple group

display computers in multiple group display windows simultaneously.

[001 1] The common display computer may broadcast data in a first group display

window of the N group display windows to N-1 group display computers other than the first group display window.

[0012] One of the group display computers and common display computer may

broadcast data in a first group display window of the N group display windows to N-1 group display computers other than the first group display window.

[0013] The common display to display data may be streamed from the N group display computers in N group display windows simultaneously.

[0014] The common display computer may broadcast a first window on the common display to any one or more of the N group display computers. [0015] The common display computer may select which of N group display windows dominate the common display.

[0016] Ethernet cables may connect the N group display computers and the common display computer to the network.

[0017] The common display computer may broadcast a second window on the common display to P group display computers, P is a natural number between 1 and N - 1. The P group display computers to receive the second window may be selected through touch mechanisms on the common display, may be controlled by the common display computer, and/or controlled by the common display computer and the group display computers.

[0018] One or more embodiments is directed to a method of connecting a mobile device to a common display associated with a common display computer controlling the common display, via a group display associated with a common display computer having presentation software. For each mobile device, the method may include downloading a display application onto the mobile device, selecting a group display computer in the display application, connecting the mobile device to the group display computer, selecting, on the mobile device, what is to be displayed on the group display, sending a data stream of the selection to the group display computer to be displayed in a window on the group display corresponding to the mobile device. The group display computer may determine a first resolution (Rl) of the data stream sent by the mobile device to the group display computer, determine a region in which the data stream is to be presented and a second resolution (R2) associated therewith, and when R2 is less than Rl , then sending a third resolution (R3) to the mobile device, where Rl > R3 >= R2. The display application on the mobile device may down sample the selection in accordance with the resolution, R3, and only transmitting the down sampled resolution (R3) to the group display computer; and sending a data stream of the group display computer to the common display computer to be displayed in a window on the common display corresponding to the group display computer. The common display computer may determine a first resolution (Rl ') of the data stream displayed by the group display computer on the group display, determine a region in which the data stream is to be presented and a second resolution (R2') associated therewith, and when R2' is less than Rl ', then sending a third resolution (R3') to the group display computer, where Rl ' > R3' >= R2', and the presentation software on the group display computer is to down sample the data stream in accordance with the resolution, R3', and only transmitting the data stream with the down sampled resolution (R3') to the common display computer.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Features will become apparent to those of skill in the art by describing in detail exemplary embodiments with reference to the attached drawings in which:

[0020] FIG. 1 illustrates a block diagram of a collaboration system in accordance with embodiments;

[0021] FIG. 2 illustrates a block diagram of a display system in accordance with

embodiments;

[0022] FIGS. 3A and 3B illustrate flow chart of connecting a mobile device to a group display in accordance with embodiments;

[0023] FIG. 4 illustrates a screen for connecting a mobile device in accordance with embodiments;

[0024] FIGS. 5 to 8 illustrate block diagrams of a display system in accordance with embodiments having a different number of mobile devices connected thereto;

[0025] FIG. 9 illustrates a block diagram of a collaboration system in accordance with embodiments;

[0026] FIG. 10 illustrates a flowchart of using a collaboration system in accordance with embodiments; and

[0027] FIGS. 1 1 to 13 illustrate block diagrams of a collaboration system operating in different modes in accordance with embodiments.

DETAILED DESCRIPTION

[0028] There are three primary components of a collaboration system in accordance with an embodiment. First is a sharing application, e.g., AirConnect ^{1 M} application by Tl V, that enables wireless device sharing from a user's mobile device, e.g., laptop, tablet, smartphone, etc., that may be used in any common operating system, e.g., Mac® OS, iOS © , Windows® , Android® , Linux® , etc. Second is presentation software, e.g., ViewHub™ application by Tl V, that allows a number of devices to be connected to a single display using the sharing application, herein a Group Display. Third, is a collaboration software, e.g., ThinkHub™ application by Tl V, that receives the image from one or more Group Displays that are running the presentation software, expands the physical area by using a digital canvas, and includes built-in applications to interact with the output from computers associated with one or more Group Displays on another, larger display, herein a Common Display. The Common Display may also be connected to an unlimited number of mobile devices running the sharing application.

[0029] FIG. 1 illustrates a block diagram of a collaboration system 1000 in accordance with an embodiment. As shown therein, the collaboration system 1000 includes a plurality of Group Display system 100a to l OOh, a wireless access point 130, a network switch 350, and two Common Displays 310a and 310b all on a network.

[0030] Each Group Display System 100 includes a Group Display 1 10 and Group

Display Computer 120, described in detail below in connection with FIG. 3. Each Group Display Computer 120 runs the presentation software, described in detail below. One or more of the Group Display Systems 100a to lOOh is connected, e.g., through the WAP 130, to one or more mobile devices (see FIG. 2) running the mobile application. Each Group Display 1 10 may display data from the one or more mobile devices connected thereto.

[0031 ] The Common Displays 310a and 310b may share a Common Display Computer

320, as described below in connection with FIG. 9. There may be one or more Common Displays connected to the Common Display Computer 320. The Common Display Computer 320 runs the collaboration software, described in detail below.

[0032] As shown in FIG. 1 , each Group Display System is connected, e.g., through wires 140, e.g., Ethernet cables, or through wireless technology to a Computer Network. Each Group Display Computer 120 and each Common Display Computer 320 may form a node on the Computer Network. A network switch 350, may be used to facilitate the connections from each node to the Computer Network. For example, the Ethernet cables from multiple Group Display Computers 120 and from one or more Common Display Computer 320 may all be connected, e.g., through wires 340, e.g., Ethernet cables, to the network switch 350. One or more mobile devices may be connected through the WAP 130 to the Common Display Computer 320. Both Common Displays 310a and 310b may display any one or all of information connected thereto, e.g., from one or more of the Group Display Computers 120 or one or more mobile devices.

[0033] Typically, when sending video data from multiple sources to a display device, a matrix switch, e.g., a HDMI (High-Definition Multimedia Interface) matrix switch, or a video wall processor is used to control the flow of data, including the switching of the various video streams coming from each Group Display Computer and each Common Display Computer to each display However, this requires more hardware and is more expensive than controlling the flow of data (including performing the switching functions) with the Group Display Computers and Common Display Computers the via a Computer Network . Using the Group Display Computers and Common Display Computers via a Computer Network is more demanding on the Common Display Computers and Group Display Computers. For example, a matrix switcher may be able to switch any of the video sources to any output display. The processing power of the Common Display Computer can be reduced if only a limited number of switch modes are employed. For example, in a classroom setting, typical modes including all Group Display Computers to be sent to the Common display Computer and any one input source to be Broadcast to all displays or certain groups of displays. Further, the processing burden on the Common Display Computer 320 may need to be further reduced using down sampling, which would not be needed when using the matrix switch or the video wall processor. However, for many applications, these limitations are not an issue.

[0034] By having mobile devices connected to Common Display Computer 320

indirectly, i.e., through the Group Display Computers 120a to 120h, rather than directly to the Common Display Computer 320, the Group Display Computers 120a to 120h may output a single group display data stream, e.g., a 1 080p data stream, containing the information from each of the mobile devices connected thereto, as discussed in detail below. Additionally, each Group Display Computer may receive up to M data streams, where M is a natural number greater than one. There may be N Group Display Computers and the Common Display Computer may receive N data streams, one from each Group Display Computers, where N is a natural number greater than 1. Thus, N group display data streams may have data streams from at least M mobile devices, e.g., unique mobile devices among the group displays, output as a single signal, to the Computer Network, which, in turn, outputs a common display data stream having at least two group display data streams as a single signal. If all of these mobile devices were to connect directly to the Common Display Computer 320, huge bandwidth issues would arise, as the Common Display Computer 320, would need to be capable of receiving NxM data streams. With the approach of this embodiment the Common Display

Computer 320, only needs to be capable of receiving N data streams, rather than NxM data streams.

[0035] FIG. 2 illustrates a block diagram of a Group Display system 100 interacting with one or more mobile devices 200a, 200b, and so forth. The Group Display 1 10 may be an LCD display, LED display, or other monitor that is capable of having an electronic video signal as an input and converting the input to a visual image. Additionally, the Group Display 1 10 may include a touch screen or other gesture input mechanism.

[0036] Information regarding a Machine Identifier 122 (see FIG. 7) of the Group

Display Computer 120 and the digital information to be displayed on the Group Display 1 10 may be sent from the Group Display Computer 120 to the Group Display 1 10. The digital information from the mobile devices may be displayed within windows, e.g., Mobile Device Windows (MDWs).

[0037] Digital information from Mobile Device 1 (200a) to Mobile Device5 (200e) is streamed from these Mobile Devices to the Group Display Computer 120 through the network. In FIG. 2, digital information is streamed from the mobile devices through the WAP 130 to the Display Computer 120. This streaming data may then be displayed in the Mobile Device Windows (MDW) on the Group Display 1 10.

[0038] Typically, the data streamed from the Mobile Device (200a to 200e) is a video signal that is the same as the signal displayed on the Mobile Device. This streaming data will be referred to as a Mobile Device Data Stream (MDDS). A MDDS is typically displayed within a MDW in the Group Display 1 10. Within or adjacent this MDW, there may also be displayed an identifier associated with the Mobile Device, e.g., the name of the MDDS, a name associated with the corresponding Mobile Device, the Mobile Device's owner, and so forth.

[0039] For use across all platforms, a user of a mobile device may download a sharing application 210a to 201 e thereon to assist in connecting the to the display computer 120. The sharing application may be written for each common operating system and is herein referred to as an AirConnect™ App. Instructions for downloading the AirConnect™ App may be readily viewable, e.g., on or adjacent the common display 1 10, or a region to be scanned, e.g., a bar code, quick response (QR) code, and so forth, using a mobile device QR, so that once scanned, the AirConnect ^{1 M} App would be downloaded. Once the AirConnect™ App is downloaded, then a user can launch the AirConnect™ App and then enter the Machine Identifier 122 associated common display 1 10. The Machine Identifier 122 may by an IP address or other alphanumeric code associated with the Display Computer 120. The Machine Identifier 122 may be simply displayed on the Common Display 1 10, in which case the user of the AirConnect™ App may simply enter the Machine Identifier 122 when prompted by the AirConnect™ App on their Mobile Device.

[0040] Details of this are further disclosed in U.S. Patent Application Serial No.

15/056,787, entitled "SYSTEM FOR CONNECTING A MOBILE DEVICE AND A COMMON DISPLAY," and U.S. Patent Application Serial No. 15/184,814, entitled " REMOTE GESTURE CONTROL, INPUT MONITOR, SYSTEMS INCLUDING THE SAME, AND ASSOCIATED METHOD," both of which is incorporated herein in its entirety for all purposes.

[0041 ] Once the Machine Identifier 122 is entered, the AirConnect™ App may then use the Machine Identifier 122 to connect the Mobile Device directly to the display computer 120. Details of this are illustrated in FIGS. 3 A and 3B, which is a flowchart of an operation of connecting Mobile Device(s) to the common display 1 10 in accordance with an embodiment.

[0042] In operation 10, a user walks within range of the display system 100 with a

mobile device (MD) connected to a wireless network. If the AirConnect ^{I M} App is on the MD in operation 20, the user launches the AirConnect ^{I M} App in operation 30 on the MD. If not, then the user may be instructed, e.g., by a notice posted on or near the common display 1 10, to download the AirConnect ^rM App on the MD in operation 25, and then to proceed to launch the AirConnect ^{I M} App in operation 30.

[0043] Once launched, the AirConnect ^{I M} App may display a list of locally stored

names and/or recently used connections on the mobile device. The Group Display 1 10 may also display the name or Machine Identifier 122 of the Group Display Computer 120 that it is connected to and/or the name of the Group Display 1 10 or Group Display Computer 120 may be posted near the Group Display 1 10. If the name associated with the Group Display 1 10 is not listed in the AirConnect™ App, the user may type a Machine Identifier 122 that may be continuously displayed on or near the Group Display 1 10 to add the Group Display Computer 120 to the list of FIG. 4 in operation 45.

[0044] An example of a screen 250 that may appear when using the AirConnect™ App is shown in FIG. 4. Here, a user would be prompted to select which display to be connected with. These options may include a nickname for the Group Display that is visually apparent, the machine identifier of the Group display that is visually apparent, and so forth. The screen 250 for selection may look the same regardless of the operating system of the mobile device running the AirConnect ^{1 M} App.

[0045] Once the Group Display computer 120 associated with the Group Display 1 10 is in the list displayed by the AirConnect ¹ App as shown in FIG. 4, the user of the MD may select a computer in operation 50, and then the AirConnect App can establish a connection between the MD device and the Display Computer, using the Machine Identifier associated with the name of the Group Display computer in the list. Once the connection is established, the Group Display computer 120 may tell the MD what resolution to stream in operation 60, up to a maximum resolution of the MD, and the desired content of the MD may be streamed to the selected computer as discussed below in detail with reference to FIG. 3B.

[0046] The Machine Information 122 regarding the selected computer may be saved on the mobile device 200a, 200b in operation 55. For example, if the user then leaves the room and disconnects and then returns, all of the initialization noted above may not need to be repeated.

[0047] Once a mobile device (MD) is connected to the Group Display Computer 120, the MDDS may occupy a MDW to match the aspect ratio of the MDDS. This MDW may occupy as much of the Group Display 1 10 as possible (See FIG. 5). The MDW may take up the entire screen if the MDDS has the same aspect ratio as the Group Display 1 10. When a second MDDS is connected, each MDW may have a same size and may maintain an aspect ratio (see FIG. 7) or may each occupy approximately half of the screen (see FIG. 8). In FIG. 8, as the MDWs have a different aspect ratios than that of the mobile device, only a portion may be viewed. Similarly, when there are four MDWs, each MDW may occupy one quarter of the display region 1 12 (see FIG. 6). When more than a specific number (Maximum Viewable Streams (MVS)) of MDDS windows are displayed, then when subsequent new MDDS connections are made, the oldest MDDS connection may be dropped.

[0048] When the MDW covers the entirety of the common display 120, a full

resolution, e.g., 1920x1080 resolution, may be used. When the MDW covers only ¼ of the screen, as shown in FIG. 6, the stream resolution may be reduced, e.g., 960x540 pixels. If the Group Display 1 10 includes a touch screen, a user may pinch/zoom the window smaller the stream resolution may be further reduced, e.g., 480x270 pixels. Thus, in accordance with a size of the MDW or reduced resolution on the Group Display 1 10 determined in operation 70, the Group Display Computer 120 may instruct the mobile device 200a, 200b to down sample the information to be transmitted to the Group display 1 10, in operation 75. This down sampling (Dynamic Resolution Scaling (DRS)) may increase the speed of the system and reduce network congestion.

[0049] If operation 90 determines that a mobile device is moved further than a

predetermined distance from the Group Display 1 10 and is still connected thereto, a warning may be sent to the mobile device alerting the mobile device that it is about to be disconnected in operation 92. Such a warning may be audible, visible, and/or tactile. If a user selects to stay connected in operation 94, the connection may be maintained. If the user selects to disconnect in operation 94, the session may end in operation 96 and the mobile device is disconnected and disappears from the Group Display 1 10.

[0050] FIG. 9 illustrates a block diagram of the collaboration system 1000. The

Common Display 310 may be an LCD display, LED display, or other monitor that is capable of having an electronic video signal as an input and converting the input to a visual image.

[0051 ] The Common Display 310 may include a display region 312 and a tray region

314, e.g., below the display region 312. The Common Display 310 may also include a touch sensor 308, e.g., overlaying an entirety of the Common Display 310, that it is sensitive to touch inputs including taps and gestures.

[0052] This digital information can be within windows, e.g., Group Device Windows

(GDWs) or Mobile Device Windows (MDWs), e.g., editable windows, or on the entire screen of display region 312 of the Common Display 310. In addition, there may be windows displaying contents from group displays (GD) or mobile devices (MD) or other appropriate mobile device icons (MDI) 220a, 220b, or group display icons (GDI) 122a, 122b, e.g., a thumbnail of what is displayed on the mobile device or the group display, in the tray region 314 on the Common Display 310, e.g., at a lower region thereof. The tray region 314 may be a region on which the MDWs or GDWs cannot be zoomed and pinched, annotated, and so forth, but may be dragged, tapped or tossed onto the display region 312, e.g., to open an GDW or MDW corresponding to the GDI or MDI, and/or to received MDWs from the display region 312 to transmit that MDW to the mobile device corresponding to the MDI. The tray region 314 may include buttons, e.g., a broadcast button 316 to allow broadcasting of the common display or a restore button 318 to return to an original configuration.

[0053] FIG. 10 illustrates a flowchart of an operation of using the collaboration system

1000 in which mobile device have already been connected to and displayed on various Group Displays. Here, only two mobile devices and two Group Displays are illustrated for simplicity. Further, details of connecting the mobile device to the Common Display 310 are the same as for the Group Display 1 10 already discussed above in connection with FIGS. 3A and 3B, and will not be repeated. [0054] An icon of a mobile device (Mobile Device Icon or MDI) 220a. 220b, etc., or an icon of the Group Display (GDI) 122a, 122b, may be shown on the Common Display 310, e.g., in the tray region 314, and the MD or GD may stream data to the Common Display Computer 320, in operation 365, after receiving a resolution request from the Common Display Computer 320 in operation 360. The MDI or GDI could be either a graphic image or a thumbnail of a small live video stream of the MDDS of GDSS. The Common Display Computer 320 may tell the mobile device MD the resolution of the data stream needed for the MDI, e.g., 160x90, in operation 360.

[0055] In one operational mode (Standard Mode), to move Mobile Device data streams

(MDDS) or Group Display data streams (GDDS) from the tray region 314 to the display region 312 of the Common Display 310 the MDIs or GDIs can be dragged by a user by a touch gesture on the Common Display touch sensor 308, e.g., a drag from the tray region 314 to the display region 312. Alternative methods of dragging the MDIs 220a, 220b and/or the GDIs 122a, 122b may include gesture recognition, voice recognition, or other user interface.

[0056] In another operational mode (AutoCanvas Mode) there is a virtual canvas

(Canvas) that exists within the software on the Common Display Computer 320. The Canvas may be much larger than the physical display region 312 on the Common Display 310. Any section of the Canvas can be viewed on the display region 312.

Suppose that, at first, the entire Canvas is viewed on the display region 312. Then by zooming into regions, a portion of the Canvas can be viewed. By pinching, zooming and panning with gestures on the touch sensor 316, a user can zoom into various regions of the Canvas to be shown on the display region 312.

[0057] The Canvas is a virtual region that expands to greater than the physical area of the display region 312 of the Common Display 310, e.g., any number of times the physical area up to infinite. The use of the Canvas allows additional files to be accessible and can be saved, but off the display region. Further discussion of the canvas may be found in U.S. Patent No. 9,596,329 entitled "Simultaneous Input System for Web Browsers and Other Applications", which is hereby incorporated by reference in its entirety for all purposes. [0058] Gestures, e.g., pan, zoom and pinch gestures, can be made to move and resize the scale of the Canvas, allowing the full canvas to be displayed or only a small section thereof. The system can be configured so that the MDW or GDW can be moved and manipulated. For example, if the Common Display 310 includes the touch sensor 308, then touch gestures such as pan, zoom and pinch may be used to move and/or resize the MDWs or CDWs. Snapshots of the MDWs or GDWs can be taken from the Common Display 310 with the use of touch gestures. Annotations could be performed on the MDWs or GDWs. Further details may be found, e.g., in U.S. Patent No. 9,596,329.

[0059] In the Standard Mode, if a Canvas is used, and a user drags an MDI or GDI in to the display region 312 of the Common Display 310, then at this point, the user can zoom the MDDS window or can zoom the Canvas to make the MDDS window to fill most of the display region 312. Then, if the user pans the Canvas, so that the MDDS window is off-screen, then the MDDS will not be visible on the display region 312.

[0060] In AutoCanvasMode, when the first MD or GD is connected, the MDW or

GDW may be automatically placed onto the center of the Canvas and the Canvas may be automatically zoomed in such that the MDW or GDW fills most of the display region 312. When a second connection is made, the second window may be automatically placed onto an empty section of the canvas (typically next to the first MDW or GDW) and the Canvas may be automatically panned and zoomed to view only the second window. To see both windows side by side, the user can pinch, zoom and pan the Canvas and/or the window. The window may be sized to the same aspect ratio of the MDDS or GDDS (or the MD or GD screen). Alternatively, all GDIs displaying something thereon may be displayed in the display region automatically, e.g., in a grid.

[0061 ] The MDDS or GDDS from mobile devices or Group Display Computers 120 may be displayed in individual window on the common display region 312. When they are editable windows, each window may be moved, expanded and or resized, for example through the use of touch or other gestures. As the size of a window on the Common Display 310 changes, different resolutions may be used within the window. When the window covers the entirety of the Common Display 312, a full resolution, e.g., 1920x 1080 resolution may be used. When the window covers only ¼ of the screen, the stream resolution may be reduced, e.g., 960x540 pixels, and if the user zooms the window smaller the stream resolution may be further reduced, e.g., 480x270 pixels.

[0062] Thus, in accordance with a size of the window or reduced resolution on the

Common Display 310 determined in operation 370, the Common Display Computer 320 may instruct the mobile device 200a, 200b or Group Display 100a, 100b, to down sample the information to be transmitted to the Common Display 310, in operation 375. This down sampling (Dynamic Resolution Scaling (DRS)) may increase the speed of the system and reduce network congestion. Also when a window is moved off of the viewable Canvas, the Common Display Computer 320 may instruct the MD or GD to only send the resolution and frame rate (e.g., 160x90 pixels @ 1 frame per 2 seconds) needed to render the MDI or GDI in the tray 314, since this will be the only visible representation of the MDDS or GDDS.

[0063] For example, the Common Display Computer 320 may determine, in operation

370, a first resolution (Rl) of the data stream sent to the Common Display Computer 320 and whether a region, e.g., the tray region 314 or an window in the display region 312, in which the data stream is to be presented and a second resolution (R2) associated with that region is different than the first resolution Rl . When R2 is less than Rl, the Common Display Computer 320 may request, in operation 375, a third resolution (R3) to the device, where Rl > R3 >= R2, and the AirConnect™ App on the mobile device or the presentations software on the Group Display Computer 1 10 is to down sample the selection in accordance with the resolution, R3, and only transmitting the down sampled resolution (R3) to the Common Display Computer 320. Alternatively, the Common Display Computer 320 may automatically tell the Group Display Computers 120 what resolution to transmit.

[0064] When the window on Common Display is a GDW from Group Display

Computer 120, there may be sub-windows, e.g., each associated with a different mobile device. These sub-windows may have been down sampled, as discussed above in connection with FIG. 3B. Thus, the sub-windows may be down sampled twice: once by the mobile device and then a second time by the Group Display Computer 120.

Alternatively, the Group Display Computer 120 could receive the full MDDS, down sample the MDDS, and display the down sampled images in MDWs on the Group Display 1 10. Then, when the Group Display Computer 120 sends the display data to the Common Display Computer 320, the Group Display Computer 120 will down sample the image data again.

[0065] Additionally, in DRS mode when the GDW or MDW is expanded on the

touchscreen 316, the Common Display Computer 320 may at first expand the content to fill the GDW or MDW, even though sufficient resolution cannot be displayed. At the same time, the Common Display Computer 320 may request, in operation 375, the MD through the sharing application or the Group Display Computer 120 through the presentation software to transmit the MDDS or GDDS at a higher resolution. The net result will be that as the GDW or MDW is expanded it will first appear slightly blurry as it expands and then, as the higher resolution is received, it will fill in the MDW or GDW to show the MDDS or GDDS at the resolution needed to match the window.

Alternatively, the Common Display Computer 320 may request a resolution larger than needed for the window from the mobile device or the Group Display, but not as large as the full resolution of the display region 312, e.g., to smooth the transition to a larger image.

[0066] In contrast, in Full Resolution Mode (FRM), when a MD or GD first connects to the display computer, it sends its MDDS or GDDS at the full resolution of that data stream displayed thereon. FRM is described, e.g., in U.S. Patent No. 9,596,329, in which the full resolution of the MDDS or GDDS is stored on the display computer and down sampled to the size needed for the window. Then, as the window is expanded the down sampling is reduced. While the FRM may allow for the resolution to be quickly changed as the window is expanded, FRM may slow down the system, when multiple MDDS or GDDS are simultaneously being transmitted.

[0067] In a typical mode of operation there may be eight Group Display systems in a classroom, as shown in FIG. 1. The Common Display may be displaying GDWs of all eight Group Display systems. Students may be seated by each Group Display and a teacher may operate the Common Display. In FIG. 1 , there are seats for six students near each Group Display. Consider a first Group Display System 100a. Each of the six students seated at the table near Group Display 100a may have a mobile device running the sharing application. If the MVS on Group Display 100a is set to four, then the content from up to four of these mobile devices will be displayed on Group Display 100a, e.g. in a quad mode format, as shown in FIG. 6. All eight Group Displays may be set to this same MVS and all eight of these Group Displays may all be in quad mode, all displaying the content from the mobile devices from four students.

[0068] The Group Display computers 120a-120h and the Common Display computer

320 may be configured so that on start-up, all of the Group Display computers auto- connect to the Common Display Computer or only those Group Display computers actually displaying something may be automatically connected, e.g., the GDIs may be displayed in the tray region 314. The Common Display 310 may include inputs, e.g., buttons, that allow different modes of operation, e.g., in the tray region 314. One mode of operation could be an Initial Classroom mode. In this mode, the Common Display Computer 320 could place the contents from each Group Display Computer 120 connected to the Computer Display Computer 320 into its own individual GDW. In this manner all eight GDWs would be displayed on the Common Display 310, i.e., in the display region 312, not the tray region 314. The user could then zoom in to the canvas to display only a subset of these eight GDWs, e.g., two of them. In this case, the other six GDWs would still be on the virtual canvas, but would be located off-screen, i.e., not visible. Scrolling through the canvas, would allow the teacher to quickly look at all 8 GDWs. The user could then arrange the GDWs by pinching, zooming, and dragging the GDWs.

[0069] Data displayed on the Common Display 310 may be broadcast to the Group

Displays Computers 120 and/or mobile devices. One option for this is to establish a separate peer-to-peer connection from each Group Display Computer 120a-120h to the Common Display Computer 320. However, this will require a great deal of processing power on the Common Display Computer 320 and could cause the responsiveness of the collaboration system to slow down. So while any computer on the network may serve to broadcast, one manner to speed up the response of the system is to use a separate Broadcast server 400, as shown in FIG. 13. This Broadcast server 400 may be connected on either side of the network switch 350 in FIG. 1. The Common Display Computer 320 may send one stream to the Broadcast server 400. Then the Common Display Computer 320 then contacts each Group Display Computers 120 (or mobile device) and directs it to get a video data stream from the Broadcast server 400. Then, each Group Display computer 120 can capture the same video data stream from the Broadcast Server 400 and display it on the associated Group Display 1 10.

[0070] In this manner, either the entire content displayed on the Common Display 310 can be broadcast or a single item, e.g., window, on the Common Display can be broadcast.

[0071] Alternatively to broadcasting an entirety of the Common Display 310, any single

GDW or MDW could be broadcast. For example, each GDW or MDW may include a band, e.g., a Group Display band GDB1 and GDB2 of FIGS. 1 1 and 12, that may include a broadcast button, an annotation button, a snapshot button and restore button discussed above in connection with the tray region 314 of the Common Display 320.

[0072] Further, each GDW may have multiple modes of operation. As disclosed in the patent and patent applications noted above, each GDW may have at least two modes of operation: one mode allows dragging, pinching and zooming of the GDW and a second mode of operation that allow for operation within the GDW (e.g. annotation, taking a snapshot of the GDW, etc.). When the GDW is in the first mode of operation, tapping on the GDW may convert it to the second mode. Typically a frame or bar GDB will be displayed around the GDW to indicate that it is in the second mode of operation, as illustrated in FIG. 1 1 and 12. The frame may contain icons to indicate allowable actions on the GDW.

[0073] In addition to the allowable actions discussed previously for an MDW, as

disclosed in detail in the patents and patent applications noted above, additional allowable actions for the GDW may include a Broadcast function, a "Switch Group Display Mode" function and a Particular Group Display Transmit function. The

Broadcast function may be enabled by tapping a Broadcast icon within the frame around a GDW. Once the Broadcast function is enabled, the Common Display Computer may then send the contents of the particular associated GDW, in the form of a Specific Data Stream (SDS) to all Group Display Computers connected in the Common Display Computer 320 and instruct each Group Display Computer 120a-120h to display the SDS on the associated Group Display. In this way, the user is able to select a particular GDW and broadcast its contents to all Group Displays in the room. If the user annotates on a GDW and then broadcasts the GDW, then the annotation may also be included in the SDS and therefore also be displayed on all of the Group Displays.

[0074] Broadcasting of a window may be performed on a live window or a snapshot of a window. In either case, if the user of the Common Display Broadcasts a window to the Group Displays, they can then work (e.g. annotate) on another item on the canvas or another part of the canvas without affecting the Group Displays. If the teacher (the user at the Common Display) broadcasts a live GDW, then the users at the Group Displays (the students) will see any annotations made by the teacher in real time, which is sometimes desirable. On the other hand, sometimes it is desirable for the teacher to draw on an item and not send it to the Group Displays until ready. In this case, the teacher can broadcast a snapshot of a GDW and then annotate on the same GDW and broadcast when annotation is completed.

[0075] If a particular Group Display function is enabled, this may work similarly to the

Broadcast function, except that with this function, the teach may be able to send the SDS to only a subset of the Group Displays or to a particular Group Display or to only the Group Display corresponding to the particular GDW. For example, in the frame around the GDW for Group Display 100a there may be an icon, e.g., in the Group Display band GB 1 , indicating to "Transmit to Sender". The teacher may annotate on the GDW1 for Group Display 100a and then tap on the "Transmit to Sender" icon and then the contents of the GDW1 may be sent via an SDS to Group Display Computer 120a and then the SDS may only be displayed on Group Display 1 10a.

[0076] The Switch Group Display mode, may allow a user (e.g. the teacher) to switch the mode of a particular Group Display 100a, for example from quad mode (FIG. 6) to single user mode (FIG. 5) from the Common Display 310. It may allow the user to change the MVS of a particular Group Display 100a. One option to change this mode would be to add an icon to the band of the GDW once tapped. Another option is to auto-invoke this mode (Auto Switch Group Display mode). For example, once a particular GDW is tapped, the band GDB could appear around the GDW. At this point, suppose the corresponding Group Display is in quad mode (MVS of 4) and four different windows are displayed on the Group Display, each displaying a different MDW on the Group Display. Then within the GDW on the Common Display 310, the GDW will be further subdivided into four different sub-windows, each sub-window corresponding to one of the MDWs on the Group Display.

[0077] If the Auto Switch Group Display mode is in operation, then just tapping within the GDW may switch the mode of the corresponding Group Display from quad mode to single mode. The Common Display Computer 320 may send a signal in this case to the corresponding Group Display Computer 120 causing the Group Display Computer 120 to display only a single MDW and expand it to fill the entire Group display. At the same time, the Common Display Computer 320 may also change the contents of the GDW to fill the GDW with the contents of one of its sub- windows - the one corresponding to the same single MDW on the Group Display, so that the GDW is still mirroring the content on the Group Display 120.

[0078] Further if the user taps on a particular sub-window within the GDW, this can be the particular sub-window that is expanded within the GDW and the corresponding MDW on the Group Display 120 that is expanded on the Group Display 120. If the teacher than taps a second time anywhere within the GDW on the Common Display 310, this may return the Group Display 120 and the GDW back to its previous state (quad mode).

[0079] When connected to a Group Display system 100, each window for the Group

Display system 100 on the Common Display Region 312 could show the content of a number, e.g., up to four, mobile devices, as shown in FIGS. 1 1 and 12. The Group Display 1 10 may be controlled by the Common Display 310, e.g., by selecting a window within the GDW, e.g., a sub-window, both the GDW and the Group Display 1 10 may go from a quad mode (see FIGS. 6 and 1 1 ) to single mode (see FIGS. 5 and 12). By tapping in that GDW again, the original configuration, e.g., quad mode, will be restored on both displays. Changes made to a window on the Common Display 310, e.g. annotations, may be broadcast to all Group Displays automatically, may be broadcast automatically only an associated Group Display or to a mobile device associated with a MDW or sub- window in the GDW, or MDW or GDW may be tossed to a broadcast area or off screen to send the changes to all stations, e.g., all directly connected mobile devices and Group Displays in the session.

[0080] As shown in FIG. 13, the system 1000 may include a remote control device 500, e.g., a table, that allows a user to do everything that can be done on the Common Display 310. In particular, the remote control device 500 would be in communication with the Common Display Computer 320 over the WAP 130 and would mirror what is on the Common Display 310 and may just forward touch events thereon to the Common Display Computer 320, e.g., may allow to zoom in or pinch out to remotely control the Common Display 310.

[0081 ] The methods and processes described herein may be performed by code or

instructions to be executed by a computer, processor, manager, or controller. Because the algorithms that form the basis of the methods (or operations of the computer, processor, or controller) are described in detail, the code or instructions for implementing the operations of the method embodiments may transform the computer, processor, or controller into a special-purpose processor for performing the methods described herein.

[0082] Also, another embodiment may include a computer-readable medium, e.g., a non-transitory computer-readable medium, for storing the code or instructions described above. The computer-readable medium may be a volatile or non-volatile memory or other storage device, which may be removably or fixedly coupled to the computer, processor, or controller which is to execute the code or instructions for performing the method embodiments described herein.

[0083] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and are to be interpreted in a generic and descriptive sense only and not for purpose of limitation. In some instances, as would be apparent to one of ordinary skill in the art as of the filing of the present application, features, characteristics, and/or elements described in connection with a particular embodiment may be used singly or in combination with features, characteristics, and/or elements described in connection with other embodiments unless otherwise specifically indicated.

Accordingly, it will be understood by those of skill in the art that various changes in form and details may be made without departing from the spirit and scope of the present invention as set forth in the following claims.