Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
HEAD-MOUNTED AUDIOVISUAL CAPTURE DEVICE
Document Type and Number:
WIPO Patent Application WO/2017/156622
Kind Code:
A1
Abstract:
Described is a head-mounted device for capturing audiovisual content in a three-dimensional space. The device is a head-worn item having at least one camera lens and at least two microphone elements. The camera lenses are positioned on the head-worn item to capture at least 180 degrees of visual content away from the head-mounted device. The microphone elements capture 360 degrees of audio content about the head-mounted device. A first microphone element is positioned proximate to a left ear of a user when wearing the head-mounted device and a second microphone element is positioned proximate to a right ear of the user when wearing the head-mounted device. Additional lenses and microphone elements may be added to the head-worn item. An electronic processor electrically receives the visual content and the audio content received from the camera lenses and the microphone elements and thereafter stores, processes or transmits the visual and audio content.

Inventors:
GODFREY MICHAEL (CA)
Application Number:
PCT/CA2017/050326
Publication Date:
September 21, 2017
Filing Date:
March 13, 2017
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
RISING SUN PRODUCTIONS LTD (CA)
International Classes:
G03B37/04; A42B1/00; A42B3/30; G02B27/01; G02C11/00; H04N13/243
Foreign References:
CA2953335A12015-12-17
GB2516758A2015-02-04
US20110157327A12011-06-30
US20140180684A12014-06-26
US20140125579A12014-05-08
US5778083A1998-07-07
US6980661B22005-12-27
Attorney, Agent or Firm:
HEER, Christopher (CA)
Download PDF:
Claims:
What is claimed is:

1. A method for capturing audiovisual content in a three-dimensional space from a head-mounted device, the method comprising: receiving visual input from at least one camera lens positioned on the head-mounted device, the at least one camera lens configured to capture at least 180 degrees of visual content away from the head-mounted device; simultaneously receiving audio input from at least two microphone elements positioned on the head-mounted device, wherein of the at least two microphone elements, a first microphone element is proximate to a right ear of a user wearing the head-mounted device and wherein a second microphone element is proximate to a left ear of the user wearing the head-mounted device, wherein the at least two microphone elements capture audio content about the head-mounted device; and electrically receiving the visual content received and the audio content received at at least one electronic processor for storage, processing or transmission.

2. The method of claim 1, wherein the visual and audio content simultaneously captured by the at least one camera lens and the at least two microphone elements, respectively, create at least a 180- degree visual image or moving visual image, with 360-degree audio when combined for playback.

3. The method of claim 1, wherein the at least one camera lens comprises at least two camera lenses and the visual and audio content captured by the at least two camera lenses and the at least two microphone elements, respectively, create 360-degree audio content and a at least 270-degree visual image or moving image when combined.

4. The method of claim 1, wherein the at least one camera lens comprises at least two camera lenses and the visual and audio content captured by the at least two camera lenses and the at least two microphone elements, respectively, create 360-degree audio content and a 360-degree visual image or moving image when combined.

5. The method of claim 1, further comprising the at least one electronic processor storing the visual content received and the audio content received separately for future processing.

6. The method of claim 1, further comprising the at least one electronic processor processing the visual content received and the audio content received into a single audiovisual content data type.

7. The method of claim 6, further comprising transmitting the combined audiovisual content in realtime or near-real time.

8. The method of claim 1, further comprising transmitting the visual content received and audio content received in real-time or near-real time.

9. The method of claim 8, wherein the visual content received and audio content received is transmitted for use in a virtual reality or augmented reality environment.

10. The method of claim 8, further comprising transmitting the visual content received and audio content received to a second head-mounted device via a network or linked telephone.

11. The method of claim 1, wherein the individually captured components of the audiovisual content are processed and transmitted simultaneously and received by an electronic viewfinder for virtual reality or augmented reality playback, wherein the electronic viewfinder is used by a user in conjunction with stereo headphones or a loudspeaker arrangement around the user suitable for delivering stereo sound comprising at least two loudspeakers.

12. A system for capturing audiovisual content in a three-dimensional space from a head-mounted device, the system comprising: at least one camera lens positioned on the head-mounted device, wherein the at least one camera lens receives visual input and is configured to capture at least 180 degrees of visual content in a direction outward from the head-mounted device; at least two microphone elements positioned on the head-mounted device, wherein of the at least two microphone elements, a first microphone element is proximate to a left ear of a user wearing the head-mounted device and a second microphone element is proximate to a right ear of the user wearing the head-mounted device, wherein the at least two microphone elements capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer-readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the at least two microphone elements and thereafter store, process or transmit the visual and audio content.

13. The system of claim 13, wherein the at least one camera lens and the at least two microphone elements may be removed and reattached to the head-mounted device at different positions.

14. The system of claim 13 , wherein the at least two microphone elements includes a third microphone element positioned on the head-mounted device.

15. The system of claim 14, wherein the at least two microphone elements includes a fourth microphone element positioned on the head-mounted device.

16. The system of claim 15, wherein the third and fourth microphone elements are spaced away from each of the first and second microphone elements, such that the first, second, third, and fourth microphone elements are at different coordinates in a three-dimensional Cartesian coordinate system.

17. The system of claim 12, wherein the microphone elements are omnidirectional, cardioid, hypercardioid, or bidirectional microphone elements.

18. The system of claim 12, wherein the at least one electronic processor includes a first electronic processor for the video content received and a second electronic processor for the audio content received.

19. The system of claim 12, wherein the at least one electronic processor stores the visual content received and the audio content received separately for future processing.

20. The system of claim 12, wherein the at least one electronic processor combines the visual content received and the audio content received into a single audiovisual content data type.

21. The system of claim 12, wherein the at least one camera lens comprises at least two camera lenses which are configured to capture at least 270 degrees of visual content away from the head-mounted device.

22. The system of claim 12, wherein the at least one camera lens comprises at least two camera lenses which are configured to capture at least 360 degrees of visual content away from the head-mounted device.

23. The system of claim 12, wherein the at least one electronic processor is in a separate unit communicatively connected to the head-mounted device.

24. A head-mounted device for capturing audiovisual content in a three-dimensional space, the device comprising: a head-worn item; at least one camera lens for receiving visual input, the at least one camera lens positioned on the head-worn item to capture at least 180 degrees of visual content away from the head- mounted device; at least two microphone sensors positioned on the head-worn item, wherein of the at least two microphone sensors, a first microphone sensor is mounted on the left side of the head- worn item and a second microphone sensor is mounted on the right side of the head-worn item and the first microphone sensor and the second microphone are positioned at least two inches apart, wherein the at least two microphone sensors capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer-readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the at least two microphone sensors and thereafter store, process or transmit the visual and audio content.

25. The head-mounted device of claim 24, wherein the at least one camera lens and the at least two microphone sensors may be removed and reattached to the head-worn item at different positions.

26. The head-mounted device of claim 24, wherein the at least two microphone sensors includes a third microphone sensor positioned on the head-worn item.

27. The head-mounted device of claim 26, wherein the at least two microphone sensors includes a fourth microphone sensor positioned on the head-worn item.

28. The head-mounted device of claim 27, wherein the third and fourth microphone sensors are spaced away from each and the first and second microphone sensor, such that the first, second, third, and fourth microphone sensors are at different coordinates in a three-dimensional Cartesian coordinate system.

29. The head-mounted device of claim 24, wherein the microphone sensors are omnidirectional, cardioid, hypercardioid, or bidirectional microphone sensors.

30. The head-mounted device of claim 24, wherein the at least one electronic processor includes a first electronic processor for the video content received and a second electronic processor for the audio content received.

31. The head-mounted device of claim 24, wherein the head-mounted device is configured to be suitable for use by military, security or law enforcement personnel.

32. The head-mounted device of claim 24, wherein the at least one camera lens comprises at least two camera lenses which are configured to capture at least 270 degrees of visual content away from the head-mounted device.

33. The head-mounted device of claim 24, wherein the at least one camera lens comprises at least two camera lenses which are configured to capture at least 360 degrees of visual content away from the head-mounted device.

34. The head-mounted device of claim 24, wherein the head-worn item is worn in front of the wearer's face.

35. The head-mounted device of claim 24, wherein the head-worn item is worn on top of the wearer's head.

36. The head-mounted device of claim 24, wherein the head-worn item is glasses.

37. The head-mounted device of claim 24, wherein the head-worn item is a hat.

38. The head-mounted device of claim 24, wherein the head-worn item is a helmet.

39. A head-mounted device for capturing audiovisual content in a three-dimensional space, the device comprising: a head-worn item; at least one camera lens for receiving visual input, the at least one camera lens positioned on the head-worn item to capture at least 180 degrees of visual content away from the head- mounted device; at least one microphone element group positioned on the head-worn item, the at least one microphone element group comprising four microphone elements arranged in a tetrahedral array, wherein the microphone elements capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer-readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the microphone elements and thereafter store, process or transmit the visual and audio content.

Description:
HEAD-MOUNTED AUDIOVISUAL CAPTURE DEVICE

FIELD OF THE INVENTION

[0001] The present invention relates to a head-mounted audiovisual capture device and related methods and systems for capturing audiovisual content about the device. More particularly, the present invention relates to a head-mounted audiovisual capture device for capturing 360 degrees of immersive audiovisual content for use with virtual and augmented reality applications.

BACKGROUND OF THE INVENTION

[0002] Virtual reality and its inevitable successor, augmented reality, provide humans with a highly immersive sensory experience. Virtual reality is a highly interactive form of tactile audiovisual entertainment, broadcast and communication that involves the three-dimensional capture, creation and delivery of an environment, either live or in a simulated format. Most often, virtual reality video is experienced by a user via electronic visual devices such as a head-mounted virtual reality viewer or virtual reality glasses. Virtual reality audio is generally experienced by a user simultaneously via head- mounted headphones or multichannel loudspeaker systems. A virtual reality video viewer essentially blocks out the typical way humans experience naturally occurring stimuli and substitutes real-life sight and sound with electronic content information. In other words, a user now exists in a naturally sensory- deprived state where they are relying on electronics for sensory stimulation and communication. The virtual reality video and sound artificially re-create reality and stimulate brain sensors to perceive the audiovisual material as reality. Virtual reality allows a user to feel fully immersed in a created three- dimensional world where the sense of realism experienced by the user is immense.

[0003] Virtual reality cameras may contain camera lenses that point outwardly from a central point in space on a single device, otherwise known as a single-point source. A single-point source captures visuals in a first-person perspective of approximately 180 to 360 degrees. Similarly, virtual reality microphones capture 360 degrees of audio through microphones that point outwardly from a central point in space on a single device. These audiovisual elements can be used and combined together in a single-point source spaced array to create a virtual reality 360-degree audiovisual camcorder. Although this multi-lens, multi- microphone camera array will continue to be designed to be smaller and smaller in size, due to the way it is designed to capture in an outwardly direction, it will nevertheless continue to capture the actual camera-operator or the person holding the camera from a secondary perspective. The intention of this camera/microphone array is for the content to be shot in a first-person perspective. This, however, will not be possible due to the angle of the camera. Instead, it will be shot from a secondary perspective. Further, the shadow of the camera-operator or other production equipment may be visible or there may be a blocked view in at least one direction, or there may be an acoustic shadow of the total 360 degrees sound. Overall, the sound and visual captured will be compromised and imperfect.

[0004] Further, virtual reality production techniques teach that first-person perspective capture, content creation and final viewer perspective are the most effective methods for humans to electronically sense the total three-dimensional experience of virtual reality. However, there is no practical way for single way and two-way audiovisual content capture for both personal communications and general three- dimensional content creation. The benefits of live broadcast or recorded 360-degree audiovisual communication in a first-person perspective would be numerous.

[0005] Accordingly, there remains a need for improvements in the art.

SUMMARY OF THE INVENTION

[0006] In accordance with an aspect of the invention, there is provided a method for capturing audiovisual content in a three-dimensional space from a head-mounted device, the method comprising: receiving visual input from at least one camera lens positioned on the head-mounted device, the at least one camera lens configured to capture at least 180 degrees of visual content away from the head-mounted device; simultaneously receiving audio input from at least two microphone elements positioned on the head-mounted device, wherein of the at least two microphone elements, a first microphone element is proximate to a right ear of a user wearing the head-mounted device and wherein a second microphone element is proximate to a left ear of the user wearing the head-mounted device, wherein the at least two microphone elements capture audio content about the head-mounted device; and electrically receiving the visual content received and the audio content received at at least one electronic processor for storage, processing or transmission.

[0007] According to a further embodiment, the present invention provides a system for capturing audiovisual content in a three-dimensional space from a head-mounted device, the system comprising: at least one camera lens positioned on the head-mounted device, wherein the at least one camera lens receives visual input and is configured to capture at least 180 degrees of visual content away from the head-mounted device; at least two microphone elements positioned on the head-mounted device, wherein of the at least two microphone elements, a first microphone element is proximate to a left ear of a user wearing the head-mounted device and a second microphone element is proximate to a right ear of the user wearing the head-mounted device, wherein the at least two microphone elements capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer- readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the at least two microphone elements and thereafter store, process or transmit the visual and audio content.

[0008] According to a further embodiment, the present invention provides a head-mounted device for capturing audiovisual content in a three-dimensional space, the device comprising: a head-worn item; at least one camera lens for receiving visual input, the at least one camera lens positioned on the head-worn item to capture at least 180 degrees of visual content away from the head-mounted device; at least two microphone elements positioned on the head-worn item, wherein of the at least two microphone elements, a first microphone element is proximate to a left ear of a user wearing the head-mounted device and a second microphone element is proximate to a right ear of the user wearing the head-mounted device, wherein the at least two microphone elements capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer-readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the at least two microphone elements and thereafter store, process or transmit the visual and audio content.

[0009] A head-mounted device for capturing audiovisual content in a three-dimensional space, the device comprising: a head-worn item; at least one camera lens for receiving visual input, the at least one camera lens positioned on the head-worn item to capture at least 180 degrees of visual content away from the head-mounted device; at least one microphone element group positioned on the head-worn item, the at least one microphone element group comprising four microphone elements arranged in a tetrahedral array, wherein the microphone elements capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer-readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the microphone elements and thereafter store, process or transmit the visual and audio content.

[0010] Other aspects and features according to the present application will become apparent to those ordinarily skilled in the art upon review of the following description of embodiments of the invention in conjunction with the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

[001 1] Reference will now be made to the accompanying drawings which show, by way of example only, embodiments of the invention, and how they may be carried into effect, and in which:

[0012] Figure 1A is a flow diagram of a method for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device according to an embodiment of the present invention;

[0013] Figure IB is a flow diagram of a method for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device and storing the audiovisual content according to an embodiment of the present invention;

[0014] Figure 2 is a diagram of a system for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device in accordance with an embodiment of the present invention;

[0015] Figure 3A is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0016] Figure 3B is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0017] Figure 3C is a top view of three configurations of camera lenses and microphone elements on a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0018] Figure 3D is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0019] Figure 4 is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0020] Figure 5 is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0021] Figure 6 is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0022] Figure 7 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention;

[0023] Figure 8 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention;

[0024] Figure 9 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention;

[0025] Figure 10 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention;

[0026] Figure 11 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention;

[0027] Figure 12A is a back perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0028] Figure 12B is a side perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention;

[0029] Figure 12C is a top view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention;

[0030] Figure 13 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention; [0031] Figure 14 is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention; and

[0032] Figure 15 is a front perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention.

[0033] Like reference numerals indicate like or corresponding elements in the drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The detailed embodiments of the present invention are disclosed herein. It should be understood, however, that the disclosed embodiments are merely exemplary of the invention, which may be embodied in various forms. Therefore, the details disclosed herein are not to be interpreted as limiting, but merely as a basis for teaching one skilled in the art how to make and use the invention.

[0035] Referring to Figures 1 to 15, a method, system and head-mounted device for simultaneously capturing as much as 360 degrees of visual and audio content in a three-dimensional space defined by an X-axis, a Y-axis, and a Z-axis for full three-dimensional virtual reality production is disclosed.

[0036] According to embodiments of the invention as shown in Figures 1A and IB, a method 100 may simultaneously capture as much as 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device 201. The method 100 may comprise receiving visual input from at least one camera lens 110 positioned on the head-mounted device 201, wherein the at least one camera lens 110 may be configured to capture at least 180 degrees of visual content. The method 100 may further comprise receiving audio input from at least two microphone elements 105 positioned on the head- mounted device 201. According to an embodiment, at least one microphone element 105 is proximate to a right ear of a user wearing the head-mounted device and at least one microphone element 105 is proximate to a left ear of the user wearing the head-mounted device. The at least two microphone elements 105 may capture 360 degrees of audio content when the device is in use. The method 100 may also comprise electrically receiving the visual content and the audio content at at least one electronic processor 135 electrically connected to the at least one camera lens 110 and the at least two microphone elements 105. According to an embodiment, there may be separate electronic processors 135 for visual content received and audio content received. According to an embodiment as shown in Figure IB, the visual content and the audio content received at the at least one electronic processor 135 may be stored integrally or transmitted with a wired or wireless connection to a storage medium, such as a computer- readable and writeable memory 150, and may then be retrieved from the memory 150 for further processing and playback by another device, such as remote computer 160.

[0037] The head-mounted device 201 may comprise a head-worn item such as a cap, a headband, a helmet, a hat or other head-worn item such as head-worn straps contained or made part of a virtual reality content viewer. Camera lens 110 may capture immersive video according to techniques known in the art. One or more camera lenses 110 may be used according to embodiments of the invention. These camera lenses 110 may be of a chosen size appropriate for the head-worn item and the number of camera lenses on the head-worn item. One or more microphone elements 105, which may also of a chosen size appropriate for the head-worn item and the number of microphone elements or sensors on the head-worn item, may capture immersive audio according to techniques known in the art. According to an embodiment, the microphone elements 105 or microphone sensors may be omnidirectional, cardioid, hypercardioid, or bidirectional microphone elements or sensors. According to an embodiment, the visual and audio content received may then transmitted and processed as desired for viewing in a virtual reality or augmented reality environment. According to further embodiments, the head-mounted device 201 may include a head-worn item appropriate for use by military, security or law enforcement personnel such as a protective or reinforced helmet.

[0038] The at least one camera lens 110 and the at least two microphone elements 105 may be positioned on the head-mounted device 201 facing outwards and the positions of the at least one camera lens 110 and the at least two microphone elements 105 may be positioned in various locations as shown in Figures 1A to 15. According to an embodiment, at least one microphone element 105 will be proximate to the left ear of the user and at least one microphone element will be proximate to the right ear of the user. Additional microphone elements 105 may be placed on the head-mounted device 201 which may help better determine the points of origination of sounds in capturing 360 degrees of audio content and may supply more complete coverage of all points in space in order to create a more complete soundscape for post- production workflows and processes that may utilize the extra information for immersive content delivery. The additional microphone elements 105 are spaced away from each of the first and second microphone elements, such that the first, second, and any additional microphone elements are at different coordinates in a three-dimensional Cartesian coordinate system.

[0039] According to an embodiment, the visual and audio content that may be captured by the at least one camera lens 110 and the at least two microphone elements 105, respectively, may create a 360-degree audio and at least a 180-degree visual image or moving image when combined. According to an embodiment, the visual content captured from the at least one camera lens 110 and the audio content captured from the at least two microphone elements 105 may be directly or discretely output for external processing or storage. According to an embodiment, the audio content received and the visual content received may be connected to outboard electronic equipment such as amplifiers and processors.

According to an embodiment, the electronic processor 135 may store the visual content and the content output separately for future processing. Alternatively, the electronic processor 135 may merge the visual content and the audio content received into a single data type such as through the use of a live video stitching application that may combine the video and audio cameras signals together in a VR viewable format such as an MP4 or VRML file encoded with MPEG4 or H.264 encoding. This may be used for live retransmission through a network connection or another wired or wireless connection, or for storage on the head-mounted device 201 such as in memory included as part of the device or in remote memory such as on a linked phone, or on a network-accessible device having its own memory.

[0040] According to an embodiment as shown in Figure 2, a system 200 for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device 201 may comprise at least one camera lens 110 positioned on the head-mounted device 201, wherein the at least one camera lens 110 receives visual input. The system 200 may further comprise at least two microphone elements 105 positioned on the head-mounted device 201, wherein one microphone element 105 is proximate to the left ear of the user wearing the head-mounted device 201 and one microphone element 105 is proximate to the right ear of the user wearing the head-mounted device 201. Each microphone element 105 may receive audio input. The system 200 may also comprise a processor 135 for executing stored computer-readable instructions which when executed, may cause the processor 135 to receive and process and/or store received audio content and visual content from any of the at least one camera lens 110 and the at least two microphone elements 110.

[0041] The at least one camera lens 110 and the at least two microphone elements 105 may be positioned on the head-mounted device 201 facing outwards. According to an embodiment, the positions of the at least one camera lens 110 and the at least two microphone elements 105 may be varied and reconfigured by removing and reattaching the elements at a different location, for example, by attachment using adhesives or hook and loop fasteners, on the head-mounted device 201. Additional microphone elements 105 may also be placed on the head-mounted device 201 which may help better determine the points of origination of sounds in capturing 360 degrees of audio content. [0042] According to one embodiment as shown in Figure 4, the at least one camera lens 110 may capture at least 180 degrees of visual content in a three-dimensional space and the at least two microphone elements 105 may capture at least 360 degrees of audio content in a three-dimensional space. The visual and audio content captured by the at least one camera lens 110 and the at least two microphone elements 110, respectively, may create a 360-degree audio and at least a 180-degree visual image or moving image when combined.

[0043] According to a further embodiment as shown in Figure 5, at least two camera lenses 110 may capture at least 270 degrees of visual content in a three-dimensional space and the at least two microphone elements 105 may capture at least 360 degrees of audio content in a three-dimensional space. The visual and audio content captured by the at least two camera lenses 110 and the at least two microphone elements 105, respectively, may create a 360-degree audio and at least a 270-degree visual image or moving image when combined.

[0044] According to a further embodiment as shown in Figure 7, at least two camera lenses 110 may capture at least 360 degrees of visual content in a three-dimensional space and the at least two microphone elements 105 may capture at least 360 degrees of audio content in a three-dimensional space. The visual and audio content captured by the at least two camera lenses 110 and the at least two microphone elements 105, respectively, may create a 360-degree audio and at least a 360-degree visual image or moving image when combined.

[0045] According to an embodiment, the visual input and the audio input may be combined to output a 360-degree sphere of audiovisual content. Individually derived components of the audiovisual content may be stitched, merged, processed or broadcast simultaneously and may be received and played, with or without further processing, by an electronic viewfinder designed for virtual reality or augmented reality. The electronic viewfinder may be used in conjunction with stereo headphones or another loudspeaker arrangement comprising of at least two speakers. The audiovisual content received may be broadcast live over 360-degree one-way communication or two-way communication. According to an embodiment, the audiovisual content may be recorded by a recording device (which may be part of electronic processor 135 or remote computer-readable and writeable memory) on the head-mounted device 201. The various electronics used in the system may be contained in the head-mounted device 201, or in one or more separate units on the user, such as a smartphone, or remotely such as in an electronic cloud.

[0046] According to an embodiment, the head-mounted device 201 may communicate with a second head-mounted device via a network or linked telephone or other communication device. The link between the head-mounted device 201 and linked telephone, such as a smartphone, may be by short-range wireless communication such as Bluetoothâ„¢, ZigBeeâ„¢, infrared or wi-fi.

[0047] According to an embodiment, merged visual and audio output may contain spatial information and orientation from where original audiovisual signals physically originated and from where original audiovisual signals were physically captured in the three-dimensional space. Further processing or program applications may stitch a total of the individually derived components of the audiovisual content together to create a viable and believable 360-degree audiovisual sphere for playback to a viewer.

Processors may include stereoscopic visual processors and other means to enhance the visual signal, as well as "de-fish" or fisheye to rectilinear image software to straighten visual artifacts of ultra-wide angle lenses. Processing may also include low light processing and color and image enhancement processing. Spatialization or HRTF (head related transfer function) processors, additional audio delays, and reverberation processors may also be used to achieve a desired three-dimensional effect for sound. A storage medium may be used in the signal chain before the processors, wherein the storage medium may be used live or subsequent to initially capturing the audiovisual content. Audio signals and visual signals that may be output at the end of the signal chain to the viewer may be 360 degrees or 360x360 video, defined by an X-axis, a Y-axis, and a Z-axis, or stereoscopic three-dimensional video or a combination of both. The audio signals may be stereophonic, stereo binaural, multichannel loudspeaker or other immersive audio signals. The audio signals and the visual signals may be discrete or combined for delivery.

[0048] According to an embodiment, the audio content may be captured by the methods disclosed in U.S Patent No. 5,778,083A, entitled "GLOBAL SOUND MICROPHONE SYSTEM," published on July 7, 1998. Alternatively, the audio content may be captured by the methods disclosed in U.S. Patent No. 6,980,661B2, entitled "METHOD OF AND APPARATUS FOR PRODUCING APPARENT MULTIDIMENSIONAL SOUND," filed on November 14, 2001 as U.S. Patent Application No. 09/987,217 with priority claim to U.S. Provisional Application No. 60/248,225 filed on November 15, 2000.

[0049] According to an embodiment as shown in Figures 3 A to 15, a head-mounted device 201 for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space may comprise a head-worn item, at least one camera lens 110 mounted or positioned on the head-worn item, wherein the at least one camera lens 110 may receive visual input. The head-mounted device 201 may further comprise at least two microphone elements 105 mounted or positioned on the head-mounted device 201, wherein one microphone element 105 is proximate to the left ear of the user wearing the head-mounted device 201 and one microphone element 105 is proximate to the right ear of the user wearing the head-mounted device 201. Each microphone element 105 may receive audio input. The at least one camera lens 110 and the at least two microphone elements 105 may be positioned on the head- mounted device 201 facing outwards and the positions of the at least one camera lens 110 and the at least two microphone elements 105 may be mounted or positioned according to various configurations as suggested throughout the figures. Additional microphone elements 105 may also be mounted or positioned on the head-mounted device 201 to better determine points of origination of sound to capture 360 degrees of audio content.

[0050] According to an embodiment as shown in Figure 3 A, the head-mounted device 201 may comprise a head-worn item such as a virtual reality viewer. The virtual reality viewer may comprise camera lenses 110 and microphone elements 105 around a perimeter of the virtual reality viewer in alternating positions, on a strap across the head of the user in alternating positions and on a strap extending from the right ear of the user to the left ear of the user in alternating positions.

[0051] According to an embodiment as shown in Figure 3B, the head-mounted device 201 may comprise a head-worn item such as a helmet. The helmet may comprise camera lenses 110 and microphone elements 105 positioned linearly in alternating positions across a top of the helmet.

[0052] According to embodiments as shown in Figure 3C, head-mounted device 201 may, for example, comprise three camera lenses 110 and three microphone elements 105, or four camera lenses 110 and four microphone elements 105, or six camera lenses 110 and six microphone elements 105. According to an embodiment, the camera lenses 110 and the microphone elements 105 may be positioned around the perimeter of the head-mounted device 201 in alternating positions.

[0053] According to an embodiment as shown in Figure 3D, the head-mounted device 201 may be a head-worn item such as a cap. The cap may comprise camera lenses 110 and microphone elements 105 positioned around the perimeter of the cap in a tiered fashion in alternating positions.

[0054] According to the embodiments as shown in Figures 4 to 6, the head-mounted device 201 may comprise a head-worn item such as a cap, for example cap 300. According to the embodiment as shown in Figure 4, cap 300 may comprise a first camera lens 110 mounted or positioned on the cap 300 proximate to the center of a forehead of the user wearing the head-mounted device 201. The cap 300 may further comprise two microphone elements 105 mounted thereon or therein, wherein a first microphone element 105 may be mounted or positioned on the cap 300 proximate to the right ear of the user and a second microphone element 105 may be mounted or positioned on the cap 300 proximate to the left ear of the user. According to an embodiment, the user may also be wearing an electronic viewer 115.

[0055] According to an embodiment, the camera lenses 110 may be mounted in holes made in a head- worn item for the particular shape and size of the camera lens 110 such that the camera lenses may be aligned and mounted firmly in position. According to an embodiment, microphone elements 105 may be either be embedded in pockets designed in the manufacturing process for a head-worn item or can be mounted in rubber grommets to reduce vibration transmitting noise and then mounted in holes similar to camera lens holes described above. In a fabric cap or on bands or straps of a viewfinder, sensor mounts may be sewn or clipped onto fabric that will hold camera lenses 110 and microphone elements 105 or they may be sewn in during manufacturing with wiring and electronics hidden in channels of fabric and foam on the garment itself, or may be placed in channels designed directly into the shell of a helmet.

[0056] According to an embodiment as shown in Figure 5, a cap 300 may comprise three camera lenses 110 and three microphone elements 105 mounted thereon or therein, wherein a first camera lens 110 and a first microphone element 105 may be mounted or positioned on the cap 300 proximate to the right ear of the user, a second camera lens 110 and a second microphone element 105 may be mounted or positioned on the cap 300 proximate to the left ear of the user, and a third camera lens 110 and a third microphone element 105 may be mounted or positioned on the cap 300 proximate to the center of the forehead of the user. According to an embodiment, the user may also be wearing one microphone element 105, such as a wireless microphone, in one or both ears.

[0057] According to an embodiment as shown in Figure 6, a cap 300 may comprise three camera lenses 110 mounted thereon or therein, wherein a first camera lens 110 may be mounted or positioned on the cap 300 proximate to the right ear of the user, a second camera lens 110 may be mounted or positioned on the cap 300 proximate to the left ear of the user, and a third camera lens 110 may be mounted or positioned on the cap 300 proximate to the center of the forehead of the user. The cap 300 may further comprise a tetrahedral microphone array 130 composed of a microphone element group of four microphone elements 105 mounted or positioned on top of the cap 300. According to an embodiment, the user may also be wearing one microphone element 105, such as a wireless microphone, in one or both ears.

[0058] According to an embodiment as shown in Figures 7 to 11, the head-mounted device 201 may be a head-worn item such as a helmet, for example helmet 700. According to an embodiment as shown in Figure 7, a helmet 700 may comprise two camera lenses 110 mounted thereon or therein, wherein a first camera lens 110 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user and a second camera lens 110 may be mounted or positioned on the helmet 700 proximate to the back of a head of the user. The helmet 700 may further comprise two microphone elements 105 mounted thereon or therein, wherein a first microphone element 150 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user and a second microphone element 105 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user. According to an embodiment, the helmet 700 may further comprise a microphone 120 positioned proximate to the mouth of the user for receiving voice communication.

[0059] According to an embodiment as shown in Figure 8, a helmet 700 may comprise three camera lenses 110 and three microphone elements 105 mounted around the perimeter of the helmet 700 in alternating positions, wherein a first camera lens 110 and a first microphone element 105 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user, a second camera lens 110 and a second microphone element 105 may be mounted or positioned on the helmet 700 proximate to the left ear of the user, and a third camera lens 110 and a third microphone element 105 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user. According to an embodiment, the helmet 700 may further comprise a microphone 120 positioned proximate to the mouth of the user for receiving voice communication. According to an embodiment as shown in Figure 9, the user may also be wearing an electronic viewer 115.

[0060] According to an embodiment as shown in Figure 10, a helmet 700 may comprise three camera lenses 110 and three microphone elements 105 mounted around the perimeter of the helmet 700 in alternating positions, wherein a first camera lens 110 and a first microphone element 105 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user, a second camera lens 110 and a second microphone element 105 may be mounted or positioned on the helmet 700 proximate to the left ear of the user, and a third camera lens 110 and a third microphone element 105 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user. According to the embodiment shown, the helmet 700 may further comprise three camera lenses 110 and three microphone elements 105 mounted around the perimeter of a top half of the helmet 700 in alternating positions, wherein a fourth camera lens 105 and a fourth microphone element 105 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user, a fifth camera lens 110 and a fifth microphone element 105 may be mounted or positioned on the helmet 700 proximate to a left side of the head of the user, and a sixth camera lens 110 and a sixth microphone element 105 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user. The helmet 700 may further comprise a seventh camera lens 110 and a seventh microphone element 105 on top of the helmet 700. According to an embodiment, the helmet 700 may also comprise a microphone 120 proximate to the mouth of the user for voice communication and the user may be wearing an electronic viewer 115.

[0061] According to an embodiment as shown in Figure 11, the helmet 700 may comprise three camera lenses 110 mounted thereon or therein, wherein a first camera lens 110 may be mounted or positioned on top of the helmet 700, a second camera lens 110 may be mounted or positioned on the helmet 700 proximate to a front of the head of the user, and a third camera lens 110 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user. The helmet 700 may further comprise three microphone elements 105 mounted thereon or therein, wherein a first microphone element 105 may be mounted or positioned proximate to the center of the forehead of the user, a second microphone element 105 may be mounted or positioned on the helmet 700 proximate to a front of the left ear of the user, and a third microphone element 105 may be mounted or positioned on the helmet 700 proximate to a back of the left ear of the user. According to an embodiment, the helmet 700 may also comprise a microphone 120 positioned proximate to the mouth of the user for voice communication.

[0062] According to an embodiment as shown in Figures 12A to 12C, the head-mounted device 201 may be a head-worn item such as a virtual reality viewer, for example virtual reality viewer 1200. According to an embodiment as show in Figure 12A, a virtual reality viewer 1200 may comprise three camera lenses 110 mounted thereon, wherein a first camera lens 110 may be mounted or positioned on top of a strap over the head of the user, a second camera lens 110 may be mounted or positioned on a back-left side of a strap across the back of the head of the user, and a third camera lens 110 may be mounted or positioned on a back-right side of the strap across the back of the head of the user. The virtual reality viewer 1200 may further comprise three microphone elements 105 mounted thereon or therein, wherein a first microphone element 105 may be mounted or positioned on the strap across the back of the head of the user proximate to the right ear of the user, a second microphone element 105 may be mounted or positioned on the strap across the back of the head of the user proximate to the left ear of the user, and a third microphone element 105 may be mounted or positioned on a middle of the strap across the back of the head of the user.

[0063] According to an embodiment as shown in Figure 12B, a virtual reality viewer 1200 may comprise three camera lenses 110 and three microphone elements 105 mounted or positioned on a front side of the viewfinder of virtual reality viewer 1200. The virtual reality viewer 1200 may further comprise a further camera lens 110 and further microphone element 105 mounted or positioned on top of the strap over the head of the user. According to an embodiment, the virtual reality viewer may also comprise two microphone elements 105 mounted or positioned on the strap across the back of the head of user, wherein one microphone element 105 may be mounted or positioned proximate to the right ear of the user and the other microphone element 105 may be mounted or positioned proximate to the left ear of the user.

[0064] According to a further embodiment as shown in Figure 12C, a virtual reality viewer 1200 may comprise two camera lenses 110 and three microphone elements 105 mounted or positioned in alternating positions across a top of a viewfinder of the virtual reality viewer 1200. According to an embodiment, the virtual reality viewer 1200 may further comprise two microphone elements 105 mounted or positioned on the strap across the back of the head of user, wherein one microphone element 105 may be mounted or positioned proximate to the right ear of the user and one microphone element 105 may be mounted or positioned proximate to the left ear of the user.

[0065] According to an embodiment as shown in Figure 13, the head-mounted device 201 may be a head-worn item such as a headset, for example headset 1300. The headset 1300 may comprise a camera lens 110 mounted or positioned on the headset 1300 proximate to the left ear of the user. The headset 1300 may further comprise microphone elements 105, such as microphone sensors. A microphone element 105 may be mounted or positioned on the headset 1300 proximate to the left ear of the user and a microphone element 105 may be mounted or positioned on the headset 1300 proximate to the right ear of the user. According to an embodiment, microphone elements 105 may also be mounted or positioned along a strap across the back of the head of the user.

[0066] According to embodiments as shown in Figures 14 and 15, the head-mounted device 201 may be a head-worn item such as glasses, for example glasses 1400 or glasses 1500. According to an embodiment as shown in Figure 14, glasses 1400 may comprise a camera lens 110 mounted or positioned proximate to a left-side lens of the glasses 1400. Glasses 1400 may further comprise two microphone elements 105, or microphone sensors, wherein a first microphone element 105 or sensor may be mounted or positioned on a left arm of the glasses 1400 and a second microphone element 105 or sensor may be mounted or positioned on a right arm of the glasses 1400 such that the first microphone element or sensor and the second microphone element or sensor may be positioned at least two inches apart.

[0067] According to an embodiment as shown in Figure 15, glasses 1500 may comprise a camera lens 110 between the left-side lens and the right-side lens of the glasses 1500. The glasses 1500 may further comprise four microphone elements 105, for example microphone sensors, wherein a first microphone element 105 or sensor may be mounted or positioned proximate to the left-side lens of the glasses 1500, a second microphone element 105 or sensor may be mounted or positioned proximate to the right-side lens of the glasses 1500 such that the first microphone element or sensor and the second microphone element or sensor may be positioned at least two inches apart. A third microphone element 105 or sensor may be mounted or positioned on the right arm of the glasses 1500, and a fourth microphone element 105 or sensor may be mounted or positioned on the left arm of the glasses 1500.

[0068] The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Certain adaptations and modifications of the invention will be obvious to those skilled in the art. Therefore, the presently discussed embodiments are considered to be illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than the foregoing description and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.