PROJECTOR SYSTEM FOR SPHERICAL PLATFORM

Field of the invention

The invention relates to the field of computer systems engineering with relations to a projector system for spherical platform of a virtual reality (VR), augmented reality (AR), or mixed reality (AR) gameplay or environment for the purpose of gaming, exercising, training, or other leisure and business activities wherein the projector system is useful for projecting the images and related sensory cues from the environment, which is being seen or experienced by a player through controllers, interactive objects, and or technologies, to the sphere seamlessly throughout the entire 360° surface of the spherical platform.

Background of the invention

Spherical platform is a platform that enables a simulation, duplication, or interaction with a real world experience that can be presented in various forms and which normally includes a set of complementary technologies such as a simulator headset or goggle, input controller, and/or a motion simulator so that the first-person player would feel further engaged with his/her virtual environment through visual, auditory, and tactile or physical interactions or stimulations from the computing and various components inside the platform that receive inputs from the players input controller and respond to the player in real-time. Further, there has been a development of projection technology so that the images being seen by the player in the virtual environment, possibly through the headset, is partially projected to the spherical portion of the sphere for a third-person viewpoint from outside the sphere. In such way, the third-party external observers shall know what the first-person player is seeing or experiencing in the virtual environment and heshe may feel involved and want to play or use the sphere for similar activity. Presently, a conventional motion simulator which utilizes a motion simulation system can simultaneously duplicate the movements of a car, motorcycle, or other vehicle so the player can feel the full-functioning, real world experience of racing and wherein the images that the first-person player sees can be partially projected on the spherical portion of the sphere. However, such conventional motion simulator cannot duplicate the full movements in all directions and so cannot be effectively used together with a VR technology in which the player shall immerse in a freely open virtual environment. In addition, the conventional motion simulator may also utilize one part of the sphere wherein the part is often opaque and cannot be easily viewed by the external party locating outside the sphere. In such case, the external party may not feel as engaged as the first-person player; unlike the spherical platform according to this invention which is designed for projection of images onto the translucent, transparent, or semi-transparent spherical surface so that the external party may feel more involved with the player and will be inclined to queue up for the platform for gaming or other activities. Additionally, other conventional VR simulators are available for use with a VR headset or goggle and can simulate the head movements of player in many directions, but still cannot rotate in 360° like the spherical platform. Further, the VR simulator does not have a third-person display or projection portion like the spherical platform.

Thus, the spherical platform has become a popular platform for VR, AR, and MR technology integrations and the inventors according to this invention have provided an important improvement over the conventional projector system, which can merely project images seen by first-person player on one part or portion of the sphere, by means of integrating two computing systems with embedded image correction and blending technologies to a set of projectors for seamlessly projection of images seen by player onto the sphere, either in part or in whole 360° surface of the sphere, for improved attractiveness and engagement by the external parties.

Summary of the invention

A projector system for spherical platform according to this invention uniquely presents a process for projecting and displaying image or series of images being seen by a first-person player inside the VR, AR, or MR environment to the outer surface of the sphere, in part or in whole (360°), in order for the audience outside to see the same images that the player is seeing or experience the same thing in real-time. The projector system according to this invention mainly comprising: 1) A first computer for receiving inputs from a controller, a position tracking sensor, an image information and head-tracking information from the a headset, and other input devices, and further processing the inputs received for feeding back or interacting with the player via a motion simulator, the controller, or display panel in the headset; and 2) A second computer for receiving the image information from the first computer wherein the information shall preferably be corrected, integrated and optimized by the set of cameras and image correction and optimization technologies presented in the second computer before and during transmission of said images for projection on the sphere by a set of projectors.

The image correction, integration and optimization process for projection of images onto the whole sphere, in one embodiment, requires the steps of receiving the images from the virtual environment by the second computer, using the cameras with fisheye distortion effect to correct each of the image distortions, integrating the images from the cameras and transmitting to the projectors, and using the geometric correction and edge blending technology for correcting or finetuning the projected images on a round surface of the sphere and adjusting the edges of the images accordingly for optimal display or projection of players environment on the whole 360° of the sphere. The invention is disclosed in fuller detail with examples and illustrations in the later section of the detailed description of the invention.

Brief description of the drawings

FIG. 1 illustrates a flow diagram of one embodiment of the projector system for spherical platform according to this invention wherein the environment is a VR environment.

FIG.2 illustrates a flow diagram of one embodiment of image correction and optimization steps before projection of image onto at least one part or whole surface of the sphere.

FIG. 3 illustrates a set of examples of software set-ups for the four cameras in the second computer wherein FIG. 3A shows one example of a texture or view being seen by a front camera, FIG.3B shows one example of a texture or view being seen by a back camera, FIG.3C shows one example of a texture or view being seen by a left camera, and FIG.3D shows one example of a texture or view being seen by a right camera

FIG.4 illustrates one set of examples of the image before applying a camera fish eye distortion effect in FIG.4A and after applying the camera fish eye distortion effect in FIG.4B. FIG.5 illustrates one set of examples of the image before applying a geometric correction and edge blending technology as in FIG. SA and FIG. SC, and after applying the geometric correction and edge blending technology as in FIG. SB.

FIG.6 illustrates one example of the possible set-up arrangement of four cameras in the second computer wherein each camera exhibits at least 120° FOV.

FIG. 7 illustrates one example of the possible set-up arrangement of four projectors facing inward for projection of images to the surface of the sphere.

FIG. 8 illustrates one example of the sphere system with an internal motion simulator set-up wherein the motion simulator provides a seat for a player to sit on and is movable on certain axes on a rail.

Detailed description of the invention

A projector system (1) for spherical platform for at least one of VR, AR, or MR gameplay or environment as shown in one embodiment can fundamentally comprising at least two main components: 1. First computer (3) for:

- Receiving inputs from at least one controller (4), a controllers set of position and movement tracking sensors (5), and a series of images (30) being seen by a player (10) in a VR, AR, or MR environment and a head-tracking sensor or device (7) through a headset (9), and, optionally, a web, high-speed, or standard camera (20);

- Processing the information received from the inputs above and responding to or interacting with the player (10)-either visually, tactilely, mechanically, and/or auditorily— through at least one of a motion simulator (6), the controller (4), and a display (8) in the headset (9); and

- Further transmitting the information wirelessly or wired via network connection to a real-time image projection module (11);

2. Second computer (12) in the real-time image projection module (11)for

Receiving the transmitted information from the first computer (3); and Transmitting the information to a set of projectors in said module ( 11 ) for displaying the images being seen by the player (10) onto a sphere (2).

The motion simulator (6) as depicted in FIG.8 further comprises a seat (40) for the first- person player (10) to sit on and is possibly driven, on pitch and yaw axes on a rail (41), by an electro-mechanical system. Further, the input controller (4) used by the player ( 10) can be chosen from any conventional controller, such as a two-handed controller, a one-handed controller, a gamepad, a smartphone, a keyboard, a mouse, an input glove, a joystick, a remote control, a footpad, an armband, a foot band, an ankle band, a full bodysuit controller, an aim controller, or the combination of at least two of these controllers. Further, the controller (4) is preferably comprising a set of position tracking sensors for tracking the players position, action, and movement in the VR, AR, or MR environment Similarly, the headset (9) may be chosen from a conventional headset that shall comprise the head-tracking sensors or device (7) for tracking players head position and movement in the virtual environment

The structure of the sphere (2) according to this invention may be chosen from any conventional spheres, but is preferably made up of transparent, semi-transparent, or translucent round-shaped materials with a set of projection films being placed or adhered on the surface of said sphere as shown in FIG. 8. The material used for the sphere may be chosen from glass, plastic, or the combination of the two. Further, the plastic material may be chosen from acrylic or polycarbonate plastic or ABS. The projectors used in this invention may comprise at least 2 projectors, or most preferably 4 projectors in accordance with FIG.7 for optimal coverage on the whole 360° of the sphere, and are preferably projecting images (30) being seen by the player (10) in VR, AR, or MR environment onto at least one part or the whole sphere (2) so that the same images (10) seen in the environment by the first-person player (10) can also be seen by the audience from outside of the sphere.

The second computer (12) which receives information from the first computer (3) comprises at least a set of two cameras for at least for producing, processing, and projecting images as depicted in FIG.6 and wherein each camera provides at least 30° field of view (FOV).

These set of cameras, preferably in a camera viewing software or application, in the second computer may further connect to and transmit image information to a set of corresponding projectors for projecting images (30) onto the sphere, accordingly. In the first preferred camera arrangement of this invention, the second computer (12) comprises a set of four cameras (25-28) which connect and transmit image information to a set of corresponding four projectors (13-16) for projecting or displaying the images (30) onto the sphere (2).

Further, the second computer (12) shall comprise the following 3 technologies for seamless projection of images via at least two projectors onto the sphere (2)·.

1. A multi-display technology for aiding the projection of images to cover large surface or the whole sphere (2) wherein the possible examples of said technology are AMD eyefinity technology and Graphic expansion module, which can be utilized individually or interchangeably;

2. A camera fisheye distortion effect for adjusting or finetuning distort images to their normal or corrected states before projection onto the sphere (2); and

3. A geometric correction and edge-blending technology for correcting or finetuning the projected images on a rounded surface of the sphere and adjusting the edges of the images accordingly so that no gaps between images being projected onto the sphere (2) by different projectors are seen and mat the projected images are fused to one another.

Considering the presence of the 3 technologies in the second computer (12) as described above, the second preferred camera arrangement may comprise a set of four cameras (25-28) connecting to a fifth camera (29) with a setting for orthographic projection wherein said fifth camera (29) receives and integrates the image information or textures and transmits them to a set of corresponding four projectors (13-16) for further projecting or displaying images (30) onto the sphere (2). The fifth camera (29) may receive the corrected or fine-tuned images or textures from the four cameras (25-28) having the camera fisheye distortion effect before transmitting the images to the projectors (13-16), accordingly. FIG.2 illustrates the flow diagram of one embodiment of image correction, integration, and optimization process being carried out by the second computer (12) and its set of cameras

(25-29) before and during projection of images via the projectors (13-16) onto at least one part or the whole surface of the sphere (2) according to this invention. The process begins by: 1) Receiving the images from the environment by the second computer (12) wherein the FIG.3 A shows example of a view from a front camera (25), FIG.3B shows example of a view from a back camera (26), FIG.3C shows example of a view from left camera (27), and FIG.3D shows example of a view from right camera (28); 2) Using the cameras (25-28) with fisheye distortion effect to normalize or correct each of the image distortions as illustrated in FIG.4A to FIG.4B; 3) Having the fifth camera (29) integrating the images from the four cameras (25-28) and transmitting to the projectors (13-16); and 4) Using the geometric correction and edge blending technology for correcting or finetuning the projected images on a 3-D rounded surface of the sphere and adjusting the edges of the images accordingly for optimal display or projection of players environment to the sphere (2) wherein the FIG.5C shows the projected image before geometric correction and edge blending and FIG.5B shows the projected image after geometric correction and edge blending, accordingly.

Although this invention has been disclosed in the context of certain embodiments and examples, it will be understood by those skilled in the art that the present invention extends beyond the specifically disclosed embodiments to other alternative embodiments and/or uses of the invention and obvious modifications and equivalents thereof. In addition, while several variations of the invention have been shown and described in detail, other modifications, which are within the scope of this invention, will be readily apparent to those of skill in the art based upon this disclosure. It is also contemplated that various combinations or sub-combinations of the specific features and aspects of the embodiments may be made and still fall within the scope of the invention. It should be understood that various features and aspects of the disclosed embodiments can be combined with, or substituted for, one another in order to form varying modes of the disclosed invention. Thus, it is intended that the scope of the present invention herein disclosed should not be limited by the particular disclosed embodiments described above, but should be determined only by a fair reading of the claims that follow.