CROSS-REFERENCE

[0001] This application claims priority toU.S. Provisional Patent Application No. 63/032,171, filed on May 29, 2020, the contents of which are incorporated herein by reference in their entirety.

TECHNICAL FIELD

[0002] This application generally relates to transducer steering and configuration systems and methods using a local positioning system. In particular, this application relates to systems and methods that utilize the position and/or orientation of transducers, devices, and/or objects within a physical environment to enable steering of lobes and nulls of the transducers, to create self assembling arrays of the transducers, and to enable configuration of the transducers and devices through an augmented reality interface.

BACKGROUND

[0003] Conferencing environments, such as conference rooms, boardrooms, video conferencing settings, and the like, can involve the use of transducers, such as microphones for capturing sound from various audio sources active in such environments, and loudspeakers for sound reproduction in the environment. Similarly, such transducers are often utilized in live sound environments, such as for stage productions, concerts, and the like, to capture sound from various audio sources. Audio sources for capture may include humans speaking or singing, for example. The captured sound may be disseminated to a local audience in the environment through the loudspeakers (for sound reinforcement), and/or to others remote from the environment (such as via a telecast and/or a webcast).

[0004] The types of transducers and their placement in a particular environment may depend on the locations of the audio sources, listeners, physical space requirements, aesthetics, room layout, stage layout, and/or other considerations. For example, microphones may be placed on a table or lectern near the audio sources, or attached to the audio sources, e.g., a performer. Microphones may also be mounted overhead to capture the sound from a larger area, such as an entire room. Similarly, loudspeakers may be placed on a wall or ceiling in order to emit sound to listeners in an environment. Accordingly, microphones and loudspeakers are available in a variety of sizes, form factors, mounting options, and wiring options to suit the needs of particular environments.

[0005] Traditional microphones typically have fixed polar patterns and few manually selectable settings. To capture sound in an environment, many traditional microphones can be used at once to capture the audio sources within the environment. However, traditional microphones tend to capture unwanted audio as well, such as room noise, echoes, and other undesirable audio elements. The capturing of these unwanted noises is exacerbated by the use of many microphones.

[0006] Array microphones having multiple microphone elements can provide benefits such as steerable coverage or pick up patterns (having one or more lobes and/or nulls), which allow the microphones to focus on the desired audio sources and reject unwanted sounds such as room noise. The ability to steer audio pick up patterns provides the benefit of being able to be less precise in microphone placement, and in this way, array microphones are more forgiving. Moreover, array microphones provide the ability to pick up multiple audio sources with one array microphone or unit, again due to the ability to steer the pickup patterns.

[0007] Similarly, loudspeakers may include individual drivers with fixed sound lobes, and/or may be array loudspeakers having multiple drivers with steerable sound lobes and nulls. For example, the lobes of array loudspeakers may be steered towards the location of desired listeners. As another example, the nulls of array loudspeakers may be steered towards the locations of microphones in an environment so that the microphones do not sense and capture sound emitted from the loudspeakers.

[0008] However, the initial and ongoing configuration and control of the lobes and nulls of transducer systems in some physical environments can be complex and time consuming. In addition, even after the initial configuration is completed, the environment the transducer system is in may change. For example, audio sources (e.g., human speakers), transducers, and/or objects in the environment may move or have been moved since the initial configuration was completed. In this scenario, the microphones and loudspeakers of the transducer system may not optimally capture and/or reproduce sound in the environment, respectively. For example, a portable microphone held by a person may be moved towards a loudspeaker during a teleconference, which can cause undesirable capture of the sound emitted by the loudspeaker. The non-optimal capture and/or reproduction of sound in an environment may result in reduced system performance and decreased user satisfaction.

[0009] Accordingly, there is an opportunity for transducer systems and methods that address these concerns. More particular, there is an opportunity for transducer steering and configuration systems and methods that can use the position and/or orientation of transducers, devices, and/or objects within an environment to assist in steering lobes and nulls of the transducers, to create self- assembling arrays of the transducers, and to configure the transducers and devices through an augmented reality interface.

SUMMARY

[0010] The invention is intended to solve the above-noted problems by providing transducer systems and methods that are designed to, among other things: (1) utilize the position and/or orientation of transducers and other devices and objects within a physical environment (as provided by a local positioning system) to determine steering vectors for lobes and/or nulls of the transducers; (2) determine such steering vectors based additionally on the position and orientation of a target source; (3) utilize the microphones, microphone arrays, loudspeakers, and/or loudspeaker arrays in the environment to generate self-assembling arrays having steerable lobes and/or nulls; and (4) utilize the position and/or the orientation of transducers and other devices and objects to generate augmented images of the physical environment to assist with monitoring, configuration, and control of the transducer system.

[0011] In an embodiment, a system may include a plurality of transducers, a local positioning system configured to determine and provide one or more of a position or an orientation of each of the plurality of transducers within a physical environment, and a processor in communication with the plurality of transducers and the local positioning system. The processor may be configured to receive the one or more of the position or the orientation of each of the plurality of transducers from the local positioning system; determine a steering vector of one or more of a lobe or a null of at least one of the plurality of transducers, based on the one or more of the position or the orientation of each of the plurality of transducers; and transmit the steering vector to a beamformer to cause the beamformer to update the location of the one or more of the lobe or the null of the at least one of the plurality of transducers.

[0012] These and other embodiments, and various permutations and aspects, will become apparent and be more fully understood from the following detailed description and accompanying drawings, which set forth illustrative embodiments that are indicative of the various ways in which the principles of the invention may be employed.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG. 1 is an exemplary depiction of a physical environment including a transducer system and a local positioning system, in accordance with some embodiments.

[0014] FIG. 2 is a block diagram of a system including a transducer system and a local positioning system, in accordance with some embodiments.

[0015] FIG. 3 is a flowchart illustrating operations for steering of lobes and/or nulls of a transducer system with the system of FIG. 2, in accordance with some embodiments.

[0016] FIG. 4 is an schematic diagram of an exemplary environment including a microphone and a loudspeaker, in accordance with some embodiments.

[0017] FIG. 5 is an exemplary block diagram showing null steering of the microphone with respect to the loudspeaker in the environment shown in FIG. 4, in accordance with some embodiments.

[0018] FIG. 6 is a flowchart illustrating operations for configuration and control of a transducer system using an augmented reality interface with the system of FIG. 2, in accordance with some embodiments. [0019] FIG. 7 is an exemplary depiction of a camera for use with the system of FIG. 2, in accordance with some embodiments.

DETAILED DESCRIPTION

[0020] The description that follows describes, illustrates and exemplifies one or more particular embodiments of the invention in accordance with its principles. This description is not provided to limit the invention to the embodiments described herein, but rather to explain and teach the principles of the invention in such a way to enable one of ordinary skill in the art to understand these principles and, with that understanding, be able to apply them to practice not only the embodiments described herein, but also other embodiments that may come to mind in accordance with these principles. The scope of the invention is intended to cover all such embodiments that may fall within the scope of the appended claims, either literally or under the doctrine of equivalents.

[0021] It should be noted that in the description and drawings, like or substantially similar elements may be labeled with the same reference numerals. However, sometimes these elements may be labeled with differing numbers, such as, for example, in cases where such labeling facilitates a more clear description. Additionally, the drawings set forth herein are not necessarily drawn to scale, and in some instances proportions may have been exaggerated to more clearly depict certain features. Such labeling and drawing practices do not necessarily implicate an underlying substantive purpose. As stated above, the specification is intended to be taken as a whole and interpreted in accordance with the principles of the invention as taught herein and understood to one of ordinary skill in the art. [0022] The transducer systems and methods described herein can enable improved and optimal configuration and control of transducers, such as microphones, microphone arrays, loudspeakers, and/or loudspeaker arrays. To attain this functionality, the systems and methods can leverage positional information (i.e., the position and/or orientation) of transducers and other devices and objects within a physical environment, as detected and provided in real-time by a local positioning system. For example, when the positional information of transducers and target sources within an environment are obtained from a local positioning system, the lobes and/or nulls of the transducers can be steered to focus on the target sources and/or reject the target sources. As another example, the positional information of transducers within an environment can be utilized to create self assembling transducer arrays that may consist of single element microphones, single element loudspeakers, microphone arrays, and/or loudspeaker arrays. As a further example, an augmented reality interface can be generated based on the positional information of transducers, devices, and/or objects within an environment in order to enable improved monitoring, configuration, and control of the transducers and devices. Through the use of the systems and methods, the transducers can be more optimally configured to attain better capture of sound and/or reproduction of sound in an environment. The more optimal capture and/or reproduction of sound in the environment may result in improved system performance and increased user satisfaction.

[0023] FIG. 1 is an exemplary depiction of a physical environment 100 in which the systems and methods disclosed herein may be used. In particular, FIG. 1 shows a perspective view of an exemplary conference room including various transducers and devices of a transducer system and a local positioning system, as well as other objects. It should be noted that while FIG. 1 illustrates one potential environment, it should be understood that the systems and methods disclosed herein may be utilized in any applicable environment, including but not limited to offices, huddle rooms, theaters, arenas, music venues, etc.

[0024] The transducer system in the environment 100 shown in FIG. 1 may include, for example, loudspeakers 102, a microphone array 104, a portable microphone 106, and a tabletop microphone 108. These transducers may be wired or wireless. The local positioning system in the environment 100 shown in FIG. 1 may include, for example, anchors 110 and tags (not shown), which may be utilized to provide positional information (i.e., position and/or orientation) of devices and/or objects within the environment 100. The tags may be physically attached to the components of the transducer system and/or to other devices in the environment 100, such as a display 112, rack mount equipment 114, a camera 116, a user interface 118, and a transducer controller 122. In embodiments, the tags of the local positioning system may also be attached to other objects in the environment, such as one or more persons 120, musical instruments, phones, tablets, computers, etc., in order to obtain the positional information of these other objects. The local positioning system may be adaptive in some embodiments so that tags (and their associated objects) may be dynamically added as and/or subtracted from being tracked as the tags enter and/or leave the environment 100. The anchors 110 may be placed appropriately throughout the environment 100 so that the positional information of the tags can be correctly determined, as is known in the art. In embodiments, the transducers in the environment 100 may communicate with components of the rack mount equipment, e.g., wireless receivers, digital signal processors, etc. It should be understood that the components shown in FIG. 1 are merely exemplary, and that any number, type, and placement of the various components in the environment 100 are contemplated and possible. The operation and connectivity of the transducer system and the local positioning system is described in more detail below. [0025] Typically, the conference room of the environment 100 may be used for meetings where local participants communicate with each other and/or with remote participants. As such, the microphone array 104, the portable microphone 106, and/or the tabletop microphone 108 can detect and capture sounds from audio sources within the environment 100. The audio sources may be one or more human speakers 120, for example. In a common situation, human speakers may be seated in chairs at a table, although other configurations and placements of the audio sources are contemplated and possible. Other sounds may be present in the environment 100 which may be undesirable, such as noise from ventilation, other persons, electronic devices, shuffling papers, etc. Other undesirable sounds in the environment 100 may include noise from the rack mount equipment 114, and sound from the remote meeting participants (i.e., the far end) that is reproduced on the loudspeakers 102. When the locations of such undesirable sounds are known (e.g., a vent in the environment 100 is static and fixed), tags can be attached to the sources of the undesirable sounds, and/or the positional information of the sources of the undesirable sounds can be directly entered into the local positioning system.

[0026] The microphone array 104 and/or the microphone 108 may be placed on a ceiling, wall, table, lectern, desktop, etc. so that the sound from the audio sources can be detected and captured, such as speech spoken by human speakers. The portable microphone 106 may be held by a person, or mounted on a stand, for example. The microphone array 104, the portable microphone 106, and/or the microphone 108 may include any number of microphone elements, and be able to form multiple pickup patterns so that the sound from the audio sources can be detected and captured. Any appropriate number of microphone elements are possible and contemplated in the microphone array 104, portable microphone 106, and microphone 108. In embodiments, the portable microphone 106 and/or the microphone 108 may consist of a single element. [0027] Each of the microphone elements in the array microphone 104, the portable microphone 106, and/or the microphone 108 may detect sound and convert the sound to an analog audio signal. Components in the array microphone 104, the portable microphone 106, and/or the microphone 108, such as analog to digital converters, processors, and/or other components, may process the analog audio signals and ultimately generate one or more digital audio output signals. The digital audio output signals may conform to the Dante standard for transmitting audio over Ethernet, in some embodiments, or may conform to another standard and/or transmission protocol. In embodiments, each of the microphone elements in the array microphone 104, the portable microphone 106, and/or the microphone 108 may detect sound and convert the sound to a digital audio signal.

[0028] One or more pickup patterns may be formed by the array microphone 104, the portable microphone 106, and/or the microphone 108 from the audio signals of the microphone elements, and a digital audio output signal may be generated corresponding to each of the pickup patterns. The pickup patterns may be composed of one or more lobes, e.g., main, side, and back lobes, and/or one or more nulls. In other embodiments, the microphone elements in the array microphone 104, the portable microphone 106, and/or the microphone 108 may output analog audio signals so that other components and devices (e.g., processors, mixers, recorders, amplifiers, etc.) external to the array microphone 104, the portable microphone 106, and/or the microphone 108 may process the analog audio signals. In embodiments, higher order lobes can be synthesized from the aggregate of some or all available microphones in the system in order to increase overall signal to noise. In other embodiments, the selection of particular microphones in the system can gate (i.e., shut off) the sound from unwanted audio sources to increase signal to noise. [0029] The pickup patterns that can be formed by the array microphone 104, the portable microphone 106, and/or the microphone 108 may be dependent on the type of beamformer used with the microphone elements. For example, a delay and sum beamformer may form a frequency- dependent pickup pattern based on its filter structure and the layout geometry of the microphone elements. As another example, a differential beamformer may form a cardioid, subcardioid, supercardioid, hypercardioid, or bidirectional pickup pattern. The microphone elements may each be a MEMS (micro-electrical mechanical system) microphone with an omnidirectional pickup pattern, in some embodiments. In other embodiments, the microphone elements may have other pickup patterns and/or may be electret condenser microphones, dynamic microphones, ribbon microphones, piezoelectric microphones, and/or other types of microphones. In embodiments, the microphone elements may be arrayed in one dimension or multiple dimensions.

[0030] In embodiments, sound in an environment can be sensed by aggregating the audio signals from microphone elements in the system, including microphone elements that are clustered (e.g., in the array microphone 104) and/or single microphone elements (e.g., in the portable microphone 106 or the microphone 108), in order to create a self-assembling microphone array. The signal to noise ratio of a desired audio source can be improved by leveraging the positional information of the microphones in the system to weight and sum individual microphone elements and/or clusters of microphone elements using a beamformer (such as beamformer 204 in FIG. 2 described below), and/or by gating (i.e., muting) microphone elements and/or clusters of microphone elements that are only contributing undesired sound (e.g., noise).

[0031] Each weighting of the microphone elements and/or clusters of microphone elements may have a complex weight (or coefficient) Cxthat is determined based on the positional information of the microphone elements and clusters. For example, if the microphone array 104 has a weight ci, the portable microphone 106 has a weight C2, and the microphone 108 has a weight C3, then an audio output signal from the system using these microphones may be generated based on weighting the audio signals Px from the microphones (e.g., the audio output signal may be based on CIPKM + C2P106 + C3P108). The weight c _x for a particular microphone may be determined based on the difference in distance between each microphone (r _x ) and a reference distance ro (which may be the distance between the audio source and the furthest microphone). Accordingly, the weight c _x for a particular microphone may be determined by the following equation c _x = e ^~ > ^k£x , where e _c = l *l ^— I Pol, which results in delaying the signals from the microphone that are closer than the reference distance ro. In embodiments, the contributions from each microphone element or clusters of microphone elements may be nested in order to optimize directionality over audio bandwidth (e.g., using a larger separation between microphone elements for lower frequency signals).

[0032] The loudspeakers 102 may be placed on a ceiling, wall, table, etc. so that sound may be reproduced to listeners in the environment 100, such as sound from the far end of a conference, pre-recorded audio, streaming audio, etc. The loudspeakers 102 may include one or more drivers configured to convert an audio signal into a corresponding sound. The drivers may be electroacoustic, dynamic, piezoelectric, planar magnetic, electrostatic, MEMS, compression, etc. The audio signal can be a digital audio signal, such signals that conform to the Dante standard for transmitting audio over Ethernet or another standard. In embodiments, the audio signal may be an analog audio signal, and the loudspeakers 102 may be coupled to components, such as analog to digital converters, processors, and/or other components, to process the analog audio signals and ultimately generate one or more digital audio signals.

[0033] In embodiments, the loudspeakers 102 may be loudspeaker arrays that consist of multiple drivers. The drivers may be arrayed in one dimension or multiple dimensions. Such loudspeaker arrays can generate steerable lobes of sound that can be directed towards particular locations, as well as steerable nulls where sound is not directed towards other particular locations. In embodiments, loudspeaker arrays may be configured to simultaneously produce multiple lobes each with different sounds that are directed to different locations. The loudspeaker array may be in communication with a beamformer. In particular, the beamformer may receive and process an audio signal and generate corresponding audio signals for each driver of the loudspeaker array. [0034] In embodiments, acoustic fields generated by the loudspeakers in the system can be generated by aggregating the loudspeakers in the system, including loudspeakers that are clustered or single element loudspeakers, in order to create a self-assembling loudspeaker array. The synthesis of acoustic fields at a desired position in the environment 100 can be improved by leveraging the positional information of the loudspeakers in the system, similar to the self- assembling microphones described above. For example, individual loudspeaker elements and/or clusters of loudspeaker elements may be weighted and summed by a beamformer (e.g., beamformer 204) to create the desired synthesized acoustic field.

[0035] Turning to FIG. 2, a block diagram including a system 200 is depicted that includes a transducer system and a local positioning system. The system 200 may enable improved and optimal configuration and control of the transducer system by utilizing positional information (i.e., the position and/or the orientation) of the transducers, devices, and/or objects within a physical environment, as detected and provided in real-time by the local positioning system. In an embodiment, the system 200 may be utilized within the environment 100 of FIG. 1 described above. The components of the system 200 may be in wired and/or wireless communication with the other components of the system 200, as depicted in FIG. 2 and described in more detail below. [0036] The transducer system of the system 200 in FIG. 2 may include a processor 202, a beamformer 204, equipment 206 (e.g., the rack mounted equipment 114 and transducer controller 122 of FIG. 1), a microphone 208 (e.g., the portable microphone 106 or tabletop microphone 108 of FIG. 1), a microphone array 210 (e.g., the microphone array 104 of FIG. 1), and a loudspeaker 212 (e.g., the loudspeakers 102 of FIG. 1). The microphone 208 and the microphone array 210 may detect and capture sounds from audio sources within an environment. The microphone 208 and the microphone array 210 may form various pickup patterns that each have one or more steerable lobes and/or nulls. The beamformer 204 may utilize the audio signals from the microphone 208 and the microphone array 210 to form different pickup patterns, resulting in a beamformed signal. The loudspeaker 212 may convert an audio signal to reproduce sound, and may also have one or more steerable lobes and/or nulls. The beamformer 204 may receive an input audio signal and convert the input audio signal into the appropriate audio signals for each driver of the loudspeaker 212.

[0037] The local positioning system of the system 200 may include a local positioning system processor 220, one or more anchors 222, and one or more tags 224. The local positioning system may determine and provide positional information (i.e., position and/or orientation) of devices in the system 200 and other objects in an environment, e.g., persons, that have tags attached. In particular, the local positioning system processor 220 may utilize information from the anchors 222 and the tags 224 to determine the positional information of the devices and/or objects within an environment. The anchors 222 may be fixed in known positions within the environment in order to define a local coordinate system, e.g., as shown by the anchors 110 in FIG. 1. In embodiments, the anchors 222 may be attached to objects that are non-permanently fixed within an environment, in order to create a local positioning reference origin. For example, in a live music venue, anchors 222 may be attached to objects that are fixed for a particular performance, such as microphone stands. When anchors 222 are attached to multiple objects in this fashion, a nested positioning system or a master/slave-type system may result where the anchors 222 may provide improve performance by over-constraining the system.

[0038] The tags 224 may be physically attached to devices of the system 200 and/or to objects in the environment, and be in communication with the anchors 222, such that the positional information of the devices and/or objects in the environment can be determined based on the distances between the tags 224 and the anchors 222 (e.g., via trilateration, as is known in the art). In embodiments, some or all of the devices and/or objects in the system 200 and in the environment may have integrated tags 224 and/or anchors 222, and/or include components that perform the same functions as the tags 224 and/or anchors 222. For example, the devices in the system 200 may have integrated tags 224 and anchors 222 (e.g., microphones, speakers, displays, etc.), while other objects in the environment have tags 224 attached to them (e.g., asset tags, badges, etc.). In embodiments, a user may establish the locations of devices serving as the anchors 222 within an environment, such as by graphically placing such devices in setup software (e.g., Shure Designer system configuration software).

[0039] The local positioning system processor 200 may determine and provide the positional information of the devices and/or objects within the environment to the processor 202. The local positioning system processor 200 may also detect when tags 224 enter and/or leave the environment where the system 200 is by using, for example, a proximity threshold that determines when a tag 224 is within a certain distance of the environment. For example, as tags 224 enter the environment that the system 200 is in, the positional information of such tags 224 can be determined. [0040] For example, a tag 224 may be attached to a device or object in the environment and may transmit ultra-wideband radio frequency (UWB RF) pulses that are received by the anchors 222. The tag 224 and the anchors 222 may be synchronized to a master clock. Accordingly, the distance between a tag 224 and an anchor 222 may be computed based on the time of flight of the emitted pulses. For determining the position of a tag 224 (attached to a device or object) in three dimensional space, at least four fixed anchors 222 are needed, each having a known position within the environment. In other embodiments, technologies such as radio frequency identification (RFID), infrared, Wi-Fi, etc. can be utilized to determine the distance between the tags 224 and anchors 222, in order to determine the positional information of devices and/or objects within an environment. In embodiments, the local positioning system processor 220 may determine and provide the position of a device or object within an environment in Cartesian coordinates (i.e., x, y, z), or in spherical coordinates (i.e., radial distance r, polar angle Q (theta), azimuthal angle f (phi)), as is known in the art.

[0041] In embodiments, the position of a tag 224 (attached to a device or object) may be determined in two dimensional space through the use of three fixed anchors 222 (each having a known a position within the environment). The local positioning system processor 220 may determine and provide the position of a device or object in these embodiments in Cartesian coordinates (i.e., x, y), or in spherical coordinates (i.e., radial distance r, polar angle Q (theta)). For example, the x-y position of a speaker with a tag 224 attached may be determined by the local positioning system processor 220, and the system 200 may determine the three-dimensional position of such a speaker by combining the determined x-y position with an assumption that such a speaker is typically at a particular height. [0042] In embodiments, positional information may be obtained from devices in the environment that are not native to the system 200 but that have compatible technologies. For example, a smartphone or tablet may have hardware and software that enables UWB RF transmission. In this case, the system 200 may utilize positional information from such non-native devices in a similar fashion as the positional information obtained from tags 224 in the system 200. [0043] The orientation of the devices and objects within the environment may also be determined and provided by the local positioning system processor 220. The orientation of a particular device or object may be defined by the rotation of a tag 224 attached to a device or object, relative to the local coordinate system. In embodiments, the tag 224 may include an inertial measurement unit that includes a magnetometer, a gyroscope, and an accelerometer that can be utilized to determine the orientation of the tag 224, and therefore the orientation of the device or object the tag 224 is attached to. The orientation may be expressed in Euler angles or quaternions, as is known in the art.

[0044] Other devices in the system 200 may include a user interface 214 (e.g., user interface 118 of FIG. 1), a camera 216 (e.g., camera 116 of FIG. 1), and a display 218 (e.g., display 112 of FIG. 1). As described in more detail below, the user interface 214 may allow a user to interact with and configure the system 200, such as by viewing and/or setting parameters and/or characteristics of the devices of the system 200. For example, the user interface 214 may be used to view and/or adjust parameters and/or characteristics of the equipment 206, microphone 208, microphone array 210, and/or loudspeaker 212, such as directionality, steering, gain, noise suppression, pattern forming, muting, frequency response, RF status, battery status, etc. The user interface 214 may facilitate interaction with users, be in communication with the processor 202, and may be a dedicated electronic device (e.g., touchscreen, keypad, etc.) or a standalone electronic device (e.g., smartphone, tablet, computer, virtual reality goggles, etc.). The user interface 214 may include a screen and/or be touch-sensitive, in embodiments.

[0045] The camera 216 may capture still images and/or video of the environment where the system 200 is located, and may be in communication with the processor 202. In some embodiments, the camera 216 may be a standalone camera, and in other embodiments, the camera 216 may be a component of an electronic device, e.g., smartphone, tablet, etc. The images and/or video captured by the camera 216 may be utilized for augmented reality configuration of the system 200, as described in more detail below. The display 218 may be a television or computer monitor, for example, and may show other images and/or video, such as the remote participants of a conference or other image or video content. In embodiments, the display 218 may include microphones and/or loudspeakers.

[0046] It should be understood that the components shown in FIG. 2 are merely exemplary, and that any number, type, and placement of the various components of the system 200 are contemplated and possible. For example, there may be multiple portable microphones 208, a loudspeaker 212 with a single driver, a loudspeaker array 212, etc. Various components of the system 200 may be implemented using software executable by one or more computers, such as a computing device with a processor and memory, and/or by hardware (e.g., discrete logic circuits, application specific integrated circuits (ASIC), programmable gate arrays (PGA), field programmable gate arrays (FPGA), digital signal processors (DSP), microprocessor, etc.). For example, some or all components of the system 200 may be implemented using discrete circuitry devices and/or using one or more processors (e.g., audio processor and/or digital signal processor) executing program code stored in a memory (not shown), the program code being configured to carry out one or more processes or operations described herein, such as, for example, the methods shown in FIGs. 3 and 6. Thus, in embodiments, the system 200 may include one or more processors, memory devices, computing devices, and/or other hardware components not shown in FIG. 2. In one embodiment, the system 200 includes separate processors for performing various functionality, and in other embodiments, the system 200 may perform all functionality using a single processor.

[0047] In embodiments, position-related patterns that vary as a function of time may be detected and stored by the system 200. For example, a processor may execute a learning algorithm and/or perform statistical analysis on collected positional information to detect such patterns. The patterns may be utilized to adaptively optimize future usage of the system 200. For example, the intermittent cycling of an HVAC system, positional information of vents in an environment, and/or temperatures in the environment can be tracked over time, and compensated for during sound reinforcement. As another example, the positional information for a portable microphone may be tracked and mapped with instances of feedback in order to create an adaptive, positional mapping of equalization for the microphone to eliminate future feedback events.

[0048] An embodiment of a process 300 for steering lobes and/or nulls of the transducers in the transducer system of the system 200 is shown in FIG. 3. The process 300 may be utilized to steer the lobes and/or nulls of microphones and loudspeakers in the transducer system, based on positional information (i.e., the position and/or the orientation) of the microphones, loudspeakers, and other devices and objects within a physical environment. The positional information may be detected and provided in real-time by a local positioning system. The result of the process 300 may be the generation of a beamformed output signal that corresponds to a pickup pattern of a microphone or microphone array, where the pickup pattern has steered lobes and/or nulls that take into account the positional information of transducers and other devices and objects in the environment. The process 300 may also result in the generation of audio output signals for drivers of a loudspeaker or loudspeaker array, where the loudspeaker or loudspeaker array has steered lobes and/or nulls that take into account the positional information of transducers and other devices and objects in the environment.

[0049] The system 200 and the process 300 may be utilized with various configurations and combinations of transducers in a particular environment. For example, the lobes and nulls of a microphone, microphone array, loudspeaker, and/or loudspeaker array may be steered based on their positional information and also the positional information of other devices, objects, and target sources within an environment. As another example, a self-assembling microphone array with steerable lobes and nulls may be created from the audio signals of single element microphones and/or microphone arrays, based on their positional information within an environment. As a further example, a self-assembling loudspeaker array with steerable lobes and nulls may be created from individual loudspeakers and/or loudspeaker arrays, based on their positional information within an environment.

[0050] At step 302, the positions and orientations of the transducers, devices, and objects within an environment may be received at the processor 202 from the local positioning system processor 220. The transducers, devices, and objects being tracked within the environment may each be attached to a tag 224 of the local positioning system, as described previously. The transducers, devices, and objects may include microphones (with single or multiple elements), microphone arrays, loudspeakers, loudspeaker arrays, equipment, persons, etc. in the environment.

[0051] In embodiments, the position and/or orientation of some of the transducers, devices, and objects within the environment may be manually set and/or be determined without use of the local positioning system processor 220 (i.e., without having tags 224 attached). In these embodiments, transducers that do not utilize the local positioning system (such as a microphone or loudspeaker) may still be steered, as described in more detail below. In particular, the pointing of a lobe or null towards or away from the location of a particular target source can be based on the positional information of target sources from the local positioning system processor 220 and the positional information of the non-local positioning system transducers.

[0052] In embodiments, a transducer controller 122 (attached to a tag 224) may be pointed by a user to cause steering of a microphone (e.g., microphone array 104) or loudspeaker (e.g., loudspeakers 102) in the system 200. In particular, the position and orientation of the transducer controller 122 may be received at step 302 and utilized later in the process 300 for steering of a microphone or loudspeaker. For example, a user pointing the transducer controller 122 at themselves can cause a microphone to be steered to sense sound from the user. As another example, a user pointing the transducer controller 122 at an audience can cause a loudspeaker to generate sound towards the audience. In embodiments, the transducer controller 122 may appear to be a typical wireless microphone or similar audio device. In embodiments, gesturing of the transducer controller 122 may be interpreted for controlling aspects of the system 200, such as volume control.

[0053] At step 304, the positional information (i.e., position and/or orientation) of a target source within the environment may be received at the processor 202. A target source may include an audio source to be focused on (e.g., a human speaker), or an audio source to be rejected or avoided (e.g., a loudspeaker, unwanted noise, etc.). In embodiments, a position of the target source is sufficient for the process 300, and in some embodiments, orientation of the target source may be utilized to optimize the process 300. For example, knowing the orientation of a target source (i.e., which way it is pointing) that is between two microphones can be helpful in determining which microphone to utilize for sensing sound from that target source.

[0054] In embodiments, the position and/or orientation of the target source may be received from the local positioning system processor 220, such as when a tag 224 is attached to the target source. In other embodiments, the position and orientation of the target source may be manually set at step 304. For example, the location of a permanently installed ventilation system may be manually set since it is static and does not move within the environment.

[0055] It may be determined at step 306 whether a microphone or a loudspeaker is being steered. If a microphone is being steered, then the process 300 may continue to step 308. At step 308, audio signals from one, some, or all of the microphones in the environment may be received at the beamformer 204. As described previously, each microphone may sense and capture sound and convert the sound into an audio signal. The audio signals from each microphone may be utilized later in the process 300 to generate a beamformed signal that corresponds to a pickup pattern having steered lobes and/or nulls. Due to the local positioning system of the system 200 knowing the positional information of each microphone element, directionality can be synthesized from some or all of the microphone elements in the system 200 (i.e., self-assembling microphone arrays), as described previously.

[0056] At step 310, the processor 202 may determine the steering vector of a lobe or null of the microphone, based on the positional information of the transducers, devices, and/or objects in the environment, as received at step 302. The steering vector of the lobe or null of the microphone may also be based on the positional information of the target source, as received at step 304. The steering vector may cause the pointing of a lobe or null of the microphone towards or away from the location of a particular target source. For example, it may be desired to point a lobe of the microphone towards a target source that is a human speaker participating in a conference so that the voice of the human speaker is detected and captured. Similarly, it may be desired to point a null of the microphone away from a target source to ensure that the sound of the target source is not purposely rejected. As another example, it may be desired to point a null of the microphone towards a target source that is unwanted noise, such as a fan or a loudspeaker, so that the unwanted noise from that target source is not detected and captured. The detection and capture of unwanted noise may also be avoided by pointing a lobe of the microphone away from such a target source. In an embodiment using the transducer controller 122 described previously, the processor 202 may determine a steering vector for a microphone based on the positional information of the transducer controller 122.

[0057] In the scenario of pointing a lobe or null of a microphone towards or away from a target source, the steering vector may be determined at step 310 by taking into account the positional information of the microphone in the environment as well as the positional information of the target source in the environment. In other words, the steering vector of the lobe or null can point to a particular three dimensional coordinate in the environment relative to the location of the microphone, which can be towards or away from the location of the target source. In embodiments, the position vectors of the microphone and the target source can be subtracted to obtain the steering vector of the lobe or null.

[0058] The steering vector determined at step 310 may be transmitted at step 312 from the processor 202 to the beamformer 204. At step 314, the beamformer 204 may form the lobes and nulls of a pickup pattern of the microphone by combining the audio signals received at step 308, and then generating a beamformed signal corresponding to the pickup pattern. The lobes and nulls may be formed using any suitable beamforming algorithm. The lobes may be formed to correspond to the steering vector determined at step 310, for example.

[0059] Returning to step 306, if a loudspeaker is being steered, then the process 300 may continue to step 316. At step 316, an input audio signal may be received at the beamformer 204 that is to be reproduced on the loudspeaker. The input audio signal may be received from any suitable audio source, and may be utilized later in the process 300 to generate audio output signals for the loudspeaker such that the loudspeaker has steered lobes and/or nulls. Due to the local positioning system of the system 200 knowing the positional information of each loudspeaker element, directionality can be synthesized from some or all of the loudspeaker elements in the system 200 (i.e., self-assembling loudspeaker arrays), as described previously.

[0060] At step 318, the processor 202 may determine the steering vector of the lobe or null of the loudspeaker, based on the positional information of the devices and/or objects in the environment, as received at step 302. The steering vector of the lobe or null of the loudspeaker may also be based on the positional information of the target source, as received at step 304. The steering vector may cause the pointing of the lobe or null of the loudspeaker towards or away from the location of a particular target source. For example, it may be desired to point a lobe of the loudspeaker towards a target source that is a listener in an audience so that the listener can hear the sound emitted from the loudspeaker. Similarly, it may be desired to point a null of the loudspeaker away from a target source to ensure that a particular location is not purposely avoided so that the location may still be able to hear the sound emitted from the loudspeaker. As another example, it may be desired to point a null of the loudspeaker towards a target source so that a particular location does not hear the sound emitted from the loudspeaker. A particular location may also be avoided from hearing the sound emitted from the loudspeaker by pointing a lobe of the loudspeaker away from such a target source.

[0061] In the scenario of pointing a lobe or null of a loudspeaker towards or away from a target source, the steering vector may be determined at step 318 by taking into account the positional information of the loudspeaker in the environment as well as the positional information of the target source in the environment. In other words, the steering vector of the lobe or null can be a particular three dimensional coordinate in the environment relative to the location of the loudspeaker, which can be towards or away from the location of the target source.

[0062] The steering vector determined at step 318 may be transmitted at step 320 from the processor 202 to the beamformer 204. At step 322, the beamformer 204 may form the lobes and nulls of the loudspeaker by generating a separate audio output signal for each loudspeaker (or driver in a loudspeaker array) based on the input audio signal received at step 316. The lobes and nulls may be formed using any suitable beamforming algorithm. The lobes may be formed to correspond to the steering vector determined at step 318, for example.

[0063] An example of null steering of a microphone will now be described with respect to the schematic diagram of an exemplary environment as shown in FIG. 4 and the block diagram of FIG. 5. In FIG. 4, a portable microphone 402 and a loudspeaker 404 (e.g., a stage monitor) are depicted in an environment 400. It may be desirable that the microphone 402 does not detect and capture sound from the loudspeaker 404, in order to reduce feedback. The system 200 and the process 300 may be utilized to steer a null of the microphone 402 towards the loudspeaker 404 such that the microphone 402 does not detect and capture the sound emitted from the loudspeaker

404 [0064] The microphone 402 may include multiple elements so that lobes and nulls can be formed by the microphone 402. For example, the microphone 402 may include two microphone elements Cf and Cb, each with a cardioid pickup pattern, that face in opposite directions. As seen in FIG. 5, the output from the microphone elements Cf and Cb may be scaled by coefficients a and b, respectively. The coefficients may be calculated based on the positional information (i.e., position and orientation) of the microphone 402 and the positional information of the unwanted target source, i.e., the loudspeaker 404.

[0065] The positional information of the microphone 402 and the loudspeaker 404 can be defined with respect to the same origin of a local coordinate system. As seen in FIG. 4, the local coordinate system may be defined by three orthogonal axes. A unit vector A of the loudspeaker 404 and a unit vector B of the microphone 402 may be defined for use in calculating a steering angle Onuii and a steering vector C for the null of the microphone 402. In particular, the steering angle Onuiiof the null of the microphone 402 (i.e., towards the loudspeaker 404) can be calculated through the dot product of the unit vectors A and B, which is subtracted from 180 degrees, based on the following set of equations. In the following equations, the outputs of the elements are defined as Cf(t) and Cb(t) and the output of the microphone 402 is defined as Y(t).

[0066] The unit vector A (from the origin to the loudspeaker 404) may be calculated based on the positional information of the loudspeaker 404 using the equation:

The unit vector B (from the origin to the microphone 402) may be calculated based on the positional information of the microphone 402 using the equation: b = b _x x, b _y y, b _z z ( from rotation matrix ) The dot product of the unit vectors A and B may be calculated using the equation:

<jp = co5 ^_1 (a b )

Finally, the steering angle Onuii of the microphone 402 can be calculated as: null Tΐ y

[0067] Depending on the magnitude of the steering angle Onuii, the coefficients a and b for scaling the output of the microphone elements Cf and Cb, respectively, may be determined based on the following equations:

1. q > 90°, 1 - COS(TT - q _hii1i )

The output Y(t) of the microphone 402 may therefore include a pickup pattern having a null from the microphone 402 towards the loudspeaker 404. As the positional information of the microphone 402 and/or the loudspeaker 404 changes, the null of the microphone 402 can be dynamically steered sot that it always points towards the loudspeaker 404.

[0068] An embodiment of a process 600 for configuration and control of the system 200 using an augmented reality interface is shown in FIG. 6. The process 600 may be utilized to enable users to more optimally monitor, configure, and control microphones, microphone arrays, loudspeakers, loudspeaker arrays, equipment, and other devices and objects within an environment, based on the positional information of the devices and/or objects within the environment and based on images and/or video captured by a camera or other image sensor. The positional information may be detected and provided in real-time by a local positioning system. The result of the process 600 may be the generation of an augmented image for user monitoring, configuration, and control, as well as the ability for the user to interact with the augmented image to view and cause changes to parameters and characteristics of the devices in the environment.

[0069] The system 200 and the process 600 may be utilized with various configurations and combinations of transducers, devices, and/or objects in an environment. For example, using the process 600, the transducers and devices in the environment 100 may be labeled and identified in an augmented image, and a user may control and configure the transducers and devices on the augmented image. In embodiments, various parameters and/or characteristics of the transducers, devices, and/or objects can be displayed, monitored, and/or changed on the augmented image. In particular, the augmented image can include the parameters and/or characteristics for transducers, devices, and/or objects overlaid on the image and/or video captured by the camera. The configuration and control of the system 200 in the environment may be especially useful in situations where the user is not physically near the environment. For example, the user’s vantage point may be far away from a stage in a music venue, such as at a mixer board, where the user cannot easily see the transducers, devices, and objects in the environment. Furthermore, it may be convenient and beneficial for a user to use the augmented image to monitor, configure, and/or control multiple transducers and devices in the environment simultaneously, as well as to allow the user to see the transducers and devices and their parameters and/or characteristics in real-time. [0070] At step 602, the positional information (i.e., positions and/or orientations) of the transducers, devices, and/or objects within an environment may be received at the processor 202 from the local positioning system processor 220. The transducers, devices, and/or objects being tracked within the environment may each be attached to a tag 224 of the local positioning system, as described previously. The transducers, devices, and objects may include microphones (with single or multiple elements), microphone arrays, loudspeakers, loudspeaker arrays, persons, and other devices and objects in the environment.

[0071] In embodiments, the position and orientation of some of the transducers, devices, and objects within the environment may be manually set and/or be determined without use of the local positioning system processor 220 (i.e., without having tags 224 attached). For example, the display 212 may be fixed and non-movable within the environment, so its positional information may be known and set without needing to use the local positioning system. In embodiments, while a position of a camera 216 may be fixed within an environment, the orientation of the camera 216 may be received at the processor 202 to be used for computing and displaying a two dimensional projection of the transducers, devices, and objects on the augmented image.

[0072] At step 604, parameters and/or characteristics of the transducers and devices within the environment may be received at the processor 202. Such parameters and/or characteristics may include, for example, directionality, steering, gain, noise suppression, pattern forming, muting, frequency response, RF status, battery status, etc. The parameters and/or characteristics may be displayed on an augmented image for viewing by a user, as described later in the process 600. At step 606, an image of the environment may be received at the processor from the camera 216 or other image sensor. In embodiments, still photos and/or real-time videos of the environment may be captured by the camera 216 and sent to the processor 202. The camera 216 may be fixed within an environment in some embodiments, or may be moveable in other embodiments, such as if the camera 216 is included in a portable electronic device. [0073] The locations of the transducers, devices, and/or objects in the environment on the captured image may be determined at step 608, based on the positional information for the transducers, devices, and/or objects received at step 602. In particular, the locations of the transducers, devices, and/or objects in the environment can be determined since the position and orientation of the camera 216 (that provided the captured image) is known, as are the positions and orientations of the transducers, devices, and objects. In embodiments, the position vector r _c of the camera 216 can be subtracted from a position vector r _n of a transducer, device, or object to obtain the relative position r of the transducer, device, or object in the environment, such as in the equation:

[0074] The position of the transducer, device, or object can be projected onto the two- dimensional augmented image by computing the dot product of the relative position vector r with the unit vectors associated with the orientation of the camera 216. For example, a two-dimensional image may be aligned with the X-Y plane of the camera orientation, and the unit normal vector e _z may be aligned with the Z-axis of the camera orientation, where the unit normal vectors e _x , e _y , e _z are fixed to the camera 216, as shown in FIG. 7. The X and Y location on the augmented image can be computed by computing the dot product of the relative position vector r with the unit vectors e _x , ey, and scaled for pixel conversion, such as in the equation: (C, U,Z) = (f · ¾, G ej/,f ¾). Computing the dot product of the relative position vector r with the unit normal vector e _z can determine whether the relative position of the transducer, device, or object is in front of the camera (e.g., sgn(Z ) > 0) or behind the camera 216 (e.g., sgn(Z ) < 0). In some embodiments, an image recognition algorithm may be utilized at step 608 to assist or supplement the positional information from the local positioning system, in order to improve the accuracy and preciseness of the locations of the transducers, devices, and objects on the image. [0075] At step 610, an augmented image may be generated by the processor 202, based on the locations of the transducers, devices, and/or objects as determined at step 608. The augmented image may include various information overlaid on the transducers, devices, and/or objects as shown in the captured image of the environment. Such information may include a name, label, position, orientation, parameters, characteristics, and/or other information related to or associated with the transducers, devices, and objects. After being generated, the augmented image may be displayed on the user interface 214 and/or on the display 218, for example.

[0076] It may be determined at step 612 whether user input has been received at the processor 202, such as through the user interface 214. User input may be received when the user desires to monitor, configure, and/or control a transducer or device in the environment. For example, if the user wishes to mute the microphone 208, the user may select and touch where the microphone 208 is located on the augmented image displayed on the user interface 214. In this example, an interactive menu can appear having an option to allow the user to mute the microphone 208. As another example, a user may select and touch where the equipment 206 is located on the augmented image displayed on the user interface 214 to view the current parameters of the equipment 206. [0077] If user input is received at step 612, then at step 614, the augmented image of the environment may be modified by the processor 202 to reflect the user input, e.g., showing that the microphone 208 is muted. The modified augmented image may be shown on the user interface 214 and/or the display 218 at step 614. At step 616, a signal may be transmitted from the processor 202 to the transducer or device being configured and/or controlled. The transmitted signal may be based on the user input, e.g., a command to the microphone 208 to mute. The process 600 may return to step 602 to continue to receive the positional information of the transducers, devices, and/or objects within the environment. The process 600 may also return to step 602 if no user input is received at step 612.

[0078] Any process descriptions or blocks in figures should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the embodiments of the invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those having ordinary skill in the art.

[0079] This disclosure is intended to explain how to fashion and use various embodiments in accordance with the technology rather than to limit the true, intended, and fair scope and spirit thereof. The foregoing description is not intended to be exhaustive or to be limited to the precise forms disclosed. Modifications or variations are possible in light of the above teachings. The embodiment s) were chosen and described to provide the best illustration of the principle of the described technology and its practical application, and to enable one of ordinary skill in the art to utilize the technology in various embodiments and with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the embodiments as determined by the appended claims, as may be amended during the pendency of this application for patent, and all equivalents thereof, when interpreted in accordance with the breadth to which they are fairly, legally and equitably entitled.