TECHNICAL FIELD

[0001] The subject matter described herein relates to audio processing, and more particularly to dynamic range control of an audio signal in a spatially-aware context.

BACKGROUND

[0002] Range control refers to limiting of an audio signal below a threshold amount. For a stereo audio signal in left-right space including a left channel and right channel, range control can be achieved in the left-right space by applying gains to the left and right channels as needed so that the peak of each channel is below the threshold. However, it is desirable to shift artifacts of range control to different spatial locations.

SUMMARY

[0003] Embodiments relate to providing range control of an audio signal in a spatially- aware context. The audio signal is limited in an audio coordinate system (e.g., left-right space) using gain factors applied in another audio coordinate system (e.g., mid-side space) to shift artifacts of hard limiting to different spatial locations. A first (e.g., mid) component and a second (e.g., side) component is generated from a third component (e.g., a left channel) and a fourth component (e.g., a right channel) of the audio signal. An amplitude threshold in the second audio coordinate system defining a maximum level for each of the third component in and the fourth component is determined. One or more gain factors are applied to each of the first component and the second component to generate an adjusted first component and an adjusted second component in the first audio coordinate system. A first (e.g., left) output channel and a second (e.g., right) output channel in the second audio coordinate system are generated from the adjusted first component and adjusted second component. The first and second output channels are each limited below the amplitude threshold from the one or more gain factors applied to each of the first component and the second component.

[0004] In some embodiments, a non-transitory computer readable medium storing program code that when executed by a processor configures the processor to: generate a first component and a second component in a first audio coordinate system from a third component and a fourth component of an audio signal in a second audio coordinate system; determine an amplitude threshold in the second audio coordinate system defining a maximum level for each of the third component and the fourth component; apply one or more gain factors to each of the first component and the second component to generate an adjusted first component and an adjusted second component in the first audio coordinate system; and generate a first output channel and a second output channel in the second audio coordinate system from the adjusted first component and the adjusted second component. The first and second output channels are each limited below the amplitude threshold from the one or more gain factors applied to each of the first component and the second component.

[0005] In some embodiments, a system for processing an audio signal includes processing circuitry configured to: generate a first component and a second component in a first audio coordinate system from a third component and a fourth component of an audio signal in the second audio coordinate system; determine an amplitude threshold in the second audio coordinate system defining a maximum level for each of the third and the fourth component; apply one or more gain factors to each of the first component and the second component to generate an adjusted first component and an adjusted second component in the first audio coordinate system; and generate a first output channel and a second output channel in the second audio coordinate system from the adjusted first component and the adjusted second component. The first and second output channels in combination are each limited below the amplitude threshold from the one or more gain factors applied to each of the first component and the second component.

[0006] Other aspects include components, devices, systems, improvements, methods, processes, applications, computer readable mediums, and other technologies related to any of the above.

BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure (FIG.) 1 is a schematic block diagram of an audio processing system, in accordance with some embodiments.

[0008] FIG. 2 is a schematic block diagram of a spatial limiter, in accordance with some embodiments.

[0009] FIG. 3 A is a block diagram of a side channel gain application followed by a L/R gain application, in accordance with some embodiments.

[0010] FIG. 3B is a block diagram of a mid channel gain application followed by a L/R gain application, in accordance with some embodiments.

[0011] FIG. 4 is a block diagram of a mid channel gain application and a side channel gain application in parallel, followed by an L/R gain application, in accordance with some embodiments.

[0012] FIG. 5A is a block diagram of a side channel gain application, followed by a mid channel gain application, followed by a L/R gain application, in accordance with some embodiments.

[0013] FIG. 5B is a block diagram of a mid channel gain application, followed by a side channel gain application, followed by an L/R gain application, in accordance with some embodiments.

[0014] FIG. 6 is a flow chart of a method for spatially limiting an audio signal, in accordance with some embodiments.

[0015] FIG. 7 is a flow chart of a process for spatially limiting an audio signal using side (or mid) limiting followed by left-right limiting, in accordance with some embodiments.

[0016] FIG. 8 is a flow chart of a process for spatially limiting an audio signal using parallel side and mid limiting followed by left-right limiting, in accordance with some embodiments.

[0017] FIG. 9 is a flow chart of a process for spatially limiting an audio signal using serial side, mid, and left-right limiting, in accordance with some embodiments.

[0018] FIG. 10 is a flow chart of a process for controlling audio signal components in one audio coordinate system to achieve a constraint defined in another audio coordinate system, in accordance with some embodiments.

[0019] FIG. 11 is a schematic block diagram of a spatial limiter for side chain processing, in accordance with some embodiments.

[0020] FIG. 12 is a schematic block diagram of a computer system, in accordance with some embodiments.

[0021] The figures depict, and the detail description describes, various non-limiting embodiments for purposes of illustration only.

DETAILED DESCRIPTION

[0022] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, the described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0023] Embodiments of the present disclosure relate to range control of an audio signal in left-right space using gain factors applied in mid-side space. The audio signal including a left channel and a right channel are converted to a mid component and a side component. Gain factors are applied to each of the mid component and the side component to generate an adjusted mid component and an adjusted side component. The adjusted components are converted back to left-right space into a left output channel and a right output channel that each satisfies a left-right threshold in left-right space.

[0024] The gain factors may be defined according to a priority of spatial limiting between the mid and side components. The priority of spatial limiting may be adjustable, and defines a desired shifting of artifacts into different spatial locations to satisfy the left-right threshold. A gain factor for a lower priority component may be defined using a gain factor for the higher priority component so that the lower priority component the gain factor for the lower priority component is applied only when a gain reduction budget of the higher priority component has been reached without satisfaction of the left-right threshold. As such, the left- right threshold is satisfied for each of the left and right output channels according to the priority of spatial limiting between the mid and side components.

EXAMPLE AUDIO PROCESSING SYSTEM

[0025] Figure (FIG.) 1 is a schematic block diagram of an audio processing system 100, in accordance with some embodiments. The audio processing system 100 receives an input audio signal including a left input channel 112 and a right input channel 114, and gain adjusts a mid component 116 or a side component 118 of the channels 112, 114 to generate an output audio signal including a left output channel 132 and a right output channel 134 each having peaks that are limited below a left-right threshold 0 _LR. The audio processing system 100 provides for dynamic range control of the input audio signal in a spatially-aware context.

The audio processing system 100 can shift the artifacts of hard limiting into different spatial locations (e.g., mid or side components of the input audio signal) depending on where the input energy is focused, and settings that configure the operation of the audio processing system 100. The settings may be determined programmatically, or may be specified by a user.

[0026] The audio processing system 100 includes an L/R to M/S converter 102, a spatial limiter 104, an L/R limiter 106, an M/S to L/R converter 108, and a controller 110. The L/R to M/S converter 102 receives the left input channel 112 and the right input channel 114, and generates the mid component 116 and the side component 118 from the input channels 112,

114. In some embodiments, the mid component 116 may be generated based on a sum of the left input channel 112 and the right input channel 114. The side component 118 may be generated based on a difference between the left input channel 112 and the right input channel 114. The mid and side components may be generated in other ways, such as using various L/R to M/S transformations. In some embodiments, mid and side components are generated from a multichannel (e.g., surround sound) audio signal.

[0027] The spatial limiter 104 includes a mid gain processor 152 and a side gain processor 154. The mid gain processor 152 receives the mid component 116 and the side component 118, and determines a mid gain factor a _m for the mid component 116. The mid gain processor 152 applies the mid gain factor a _m to the mid component 116 to generate an adjusted mid component 120. The side gain processor 154 receives the mid component 116 and the side component 118, and determines a side gain factor a _s for the side component 118. The side gain processor 154 applies the side gain factor a _s to the side component 118 to generate an adjusted side component 122.

[0028] In some embodiments, there is a priority of spatial limiting between the mid component 116 and the side component 118. For example, a primary gain factor a ₇ is applied to the higher priority component down to a gain reduction budget for the higher priority component. If the gain reduction budget is exhausted, but the left-right threshold 0 _LR fails to be satisfied, then a secondary gain factor a _u is applied to the lower priority component down to a gain reduction budget for the lower priority component. If the primary component is the mid component and the secondary component is the side component, then the mid gain factor a _m is defined by a ₇ and the side gain factor a _s is defined by a _{u .} If the primary component is the side component and the secondary component is the mid component, then the mid gain factor a _m is defined by a _u and the side gain factor a _s is defined by a _7. The gain factor for the lower priority component is defined recursively using the gain factor for the higher priority component to specify the priority. If the left-right threshold 0 _LR remains unsatisfied after application of the gain factor for the lower priority component, then a left-right gain factor air is applied to each of the mid and side components as needed to satisfy the left-right threshold 0 _LR.

[0029] The L/R limiter 106 includes an L/R gain processor 156. The L/R gain processor 156 receives the adjusted mid component 120 and the adjusted side component 122 as adjusted by the spatial limiter 104, applies the left-right gain factor air to the adjusted mid component 120 to generate the adjusted mid component 124, and applies the left-right gain factor air to the side component 122 to generate the adjusted side component 126.

[0030] As discussed in greater detail below in connection with FIGS. 3A through 9, the gain factors am, a _s , and air may vary depending on the priority of spatial limiting of the audio processing system 100. The priority for spatial limiting defines a priority between the mid and side limiting stages. If either the mid or side gain reduction calculations are constrained, e.g. with gain budgets or side chain processing, they may be followed by a L/R limiting stage that is applied to both the mid and side components. This L/R limiting stage affects both components equally and therefore may be determined and applied in mid-side space or in the left-right space. Lower prioritized limiting stages may apply a gain factor that is defined using one or more gain factors applied in higher prioritized limiting stages. Each higher priority limiting stage may include a gain reduction budget Q that defines a minimum value for the gain factor, with lower priority limiting stages being used when the gain reduction budget is exhausted for the higher priority limiting stage and the left-right threshold 0 _LR fails to be satisfied. In each configuration, there is application of at least one gain factor to the mid component and at least one gain factor to the side component to ensure that peaks of the left channel 132 and right channel 134 of the output audio signal in left-right space are below the left-right threshold 0 _LR. It should be noted that these stages may not actually separate, nor are they necessarily recursive in nature. The separate stages may merely represent logical priority. The embodiments discussed herein are not limited to use or omission of recursive gain stage calculations, or operation via separate dynamic range controllers for each stage.

[0031] The M/S to L/R converter 108 receives the adjusted mid component 124 and the adjusted side component 126, and generates the left output channel 132 and the right output channel 134 from the adjusted mid component 124 and the adjusted side component 126. In some embodiments, the left output channel 132 may be generated based on a sum of the adjusted mid component 124 and the adjusted side component 126. The right output channel 134 may be generated based on a difference between the adjusted mid component 124 and the adjusted side component 126. Other types of transformations may be used to generate left and right channels from mid and side components. The M/S to L/R converter 108 outputs the left output channel 132 to a left speaker and the right output channel 134 to a right speaker.

As a result of the processing applied by the spatial limiter 104 and the L/R limiter 106, the peaks of the left channel 132 and right channel 134 of the output audio signal are below the left-right threshold i) _LR.

[0032] In some embodiments, the controller 110 controls the operations of the audio processing system 100. The controller 110 may be coupled to the spatial limiter 104 and the L/R to configure operation of the spatial limiter 104, such as definition of thresholds (e.g., 0 _LR , gain reduction budgets, etc.), determination of priority of processing stages, and determination of gain factors in accordance with the determined priority and thresholds. The various parameters used by the spatial limiter may be defined by user input,

programmatically, or combinations thereof as discussed in greater detail herein.

EXAMPLE SPATIAL LIMITER

[0033] FIG. 2 is a schematic block diagram of a spatial limiter 200, in accordance with some embodiments. The spatial limiter 200 includes a mid peak extractor 202, a side peak extractor 204, a mid gain processor 206, a side gain processor 208, a mid mixer 210, and a side mixer 212.

[0034] The mid peak extractor 202 receives the mid component 116, and determines a mid peak 214 representing a peak value of the mid component 116. The mid peak extractor 202 provides the mid peak 214 to the mid gain processor 206 and the side gain processor 208. The side peak extractor 204 receives the side component 118, and determines a side peak 216 representing a peak value of the side component 118. The side peak extractor 204 provides the side peak 216 to the mid gain processor 206 and the side gain processor 208.

[0035] The mid gain processor 206 determines a mid gain factor 218 (a _m ) based on the mid peak 214, the side peak 216, the threshold $LR in left-right space. The side gain processor 208 determines a side gain factor 220 (a _s ) based on the mid peak 214, the side peak 216, the threshold $LR in left-right space.

[0036] The mid mixer 210 receives the mid component 116 and the mid gain factor 218 (a _m ), and multiplies these values to generate the adjusted mid component 120. The side mixer 212 receives the side component 118 and the side gain factor 220 (a _s ), and multiplies these values to generate the adjusted side component 122.

[0037] In some embodiments, the L/R limiting stage is integrated with the spatial limiter 200. The mid gain processor 206 combines the left-right gain factor air with the mid gain factor 218, and mid mixer 210 multiplies the result with the mid component 116 to generate the adjusted mid component 124. The side gain processor 208 combines the left-right gain factor air with the side gain factor 220, and side mixer 212 multiplies the result with the side component 118 to generate the adjusted side component 126.

LEFT-RIGHT SPACE TO MID-SIDE SPACE COORDINATE TRANSFORMATION

[0038] Gain application may be applied to one of the mid component 116 or the side component 118 of the input audio signal. To create the mid component 116 and side component 116, a transformation M for converting a signal from left-right space to mid-side space may be defined by Equation 1 :

[0040] In mid-side space, various processing may be performed including subband spatial processing, crosstalk processing (e.g., crosstalk cancellation or crosstalk simulation), crosstalk compensation (e.g., adjusting for spectral artifacts caused by crosstalk processing), and gain application in the mid or side components. Processed mid and side components are converted to the left-right space as a left output channel for a left speaker and a right output channel for a right speaker.

[0041] The inverse transformation M ¹ for converting a signal from mid-side space to left-right space may be defined by Equation 2:

[0042] ^M_1 = [i _ ] Eq. (2)

[0043] The left-right space and mid-side space are examples of orthogonal audio coordinate systems. In practice, equations 1 and 2 may be preferred to the true orthogonal form, where both forward and inverse transformations are scaled by square root of 2 for reduction in computational complexity.

SIDE (OR MID) TO LEFT-RIGHT PRIORITY LIMITING

[0044] An audio processing system 100 may perform spatial priority limiting of an audio signal by prioritizing a gain applied to one of the side component m2 or mid component mi, followed by an L/R gain. For example, a left-right threshold 0 _LR in left-right space is determined that defines a threshold level for the audio signal in left-right space. The left output channel 132 and the right output channel 134 each should not exceed the threshold 0 _LR. To satisfy the threshold 0 _LR , the audio processing system may prioritize gain in the mid-side space between the mid or side component.

[0045] To minimize the reduction of the side component, a gain reduction budget 0 _S for the side component may be used. The gain reduction budget 0 _S defines a maximum amount of gain reduction that can be applied to the side component, and is used to determine a gain factor a _s for the side component. A left-right gain factor air is determined based on the gain factor a _s. Application of the side gain factor a _s to the side component, and a left-right gain factor air to both the mid and side component results in satisfaction of the threshold i) _LR for the audio signal.

[0046] After the determination of the gain factor a _s , a _s is applied to the side component. The left-right gain factor air is applied to each of the mid component and the side component (or each of the left and right channel after conversion to left-right space, since the same global scale factor could be applied to any orthogonal rotation of coordinates with the identical result). Here, there is a prioritization of gain control for the side component to satisfy the threshold i) _LR down to the side gain reduction budget 9 _S. If the gain reduction budget 9s is applied to the side component but the threshold 9 _LR fails to be satisfied, then the suitable air is applied to both the mid component and side component. This results in satisfaction of the threshold i) _LR using gain factors determined in the mid-side space.

[0047] To prioritize gain control for the side component m ₂ down to a side gain reduction budget 9s, the gain factor a _s applied to the side component m2 can be defined by Equation 3 :

[0049] where |mi| is the peak of the mid component mi, and | m2 1 is the peak of the side component m2.

[0050] The left-right gain factor air may be defined recursively using the side gain factor a _s. The left-right gain factor air is defined by Equation 4:

[0052] where PLR is a worst-case peak in left-right space after application of the side gain factor as to the side component m2.

[0053] PLR may be defined by Equation 5:

[0055] Once the gain coefficients a _s and air are determined, they are applied to the mid component miand the side component m2 as shown by Equation 6:

a _lr 0

[0056] m'= [m _j m ₂ ]

0 ¹ a Ί G [ 0 a Eq. (6)

lr

[0057] The inverse transformation as defined by Equation 2 may then be applied to the result of Equation 6 to produce the left output channel 132 and right output channel 134, each satisfying the left-right threshold 0 _LR.

[0058] Note that in Equation 3, if a _s = 0 for a given peak, this will completely collapse the soundstage to mono. However, we can mitigate this effect by specifying a non-zero value for Os. There will be some component of peak limiting applied to both side and mid components via air if the budget 0 _S is exhausted in the side component, or there is clipping in the mid channel.

[0059] Equations 3 through 6 imply a priority of gain reduction stages as shown in FIG. 3A, where first there is a side limiter stage 302, and then a left-right limiter stage 304. An analogous form of processing could be applied to the mid component mias shown in FIG. 3B, where first there is a mid limiter stage 306, and then a left-right limiter stage 304. Here, algorithmic control of the mid component is achieved using a gain reduction budget Om to define a mid gain factor a _m , along with use of a left-right gain factor air as needed to satisfy the threshold 0 _LR after application of the mid gain factor a _m.

[0060] To prioritize gain control for the mid component mi down to a mid gain reduction budget Om, the mid gain factor a _m applied to the mid component mi can be defined by

Equation 7:

[0062] where |mi| is the peak of the mid component mi, and |m ₂ | is the peak of the side component m2.

[0063] The left-right gain factor air may be defined recursively using the mid gain factor a _m. The left-right gain factor air is defined by Equation 8:

[0065] where PLR is a worst-case peak in left-right space after application of the mid gain factor a _m to the side component mi.

[0066] PLR may be defined by Equation 9:

[0068] Once the gain coefficients a _m and air are determined, they are applied to the mid component mi and the side component m2 as shown by Equation 10:

[0070] The inverse transformation as defined by Equation 2 may then be applied to the result of Equation 10 to produce the left output channel and right output channel, each satisfying the left-right threshold 0 _LR. PARALLEL MID AND SIDE TO LEFT-RIGHT PRIORITY LIMITING

[0071] The audio processing system 100 may use a parallel (flat) priority between the mid and side component, while still prioritizing the gain coefficients a _m and a _s above the left- right gain factor air. The priority of gain reduction stages is shown in FIG. 4, where first there is a side limiter stage 402 in parallel with a mid limiter stage 404, and an L/R limiter stage 406 following the parallel stages 402 and 404.

[0072] In this case, a side gam reduction budget Os and a mid gam reduction budget Om are both used. The side gain factor a _s can be determined using Equation 3, and the mid gain factor a _m can be determined using Equation 7. Here, the definition for the side and mid gain factors are independent because of the parallel priority for the mid and side components.

[0073] The left-right gain factor air is defined using the mid gain factor a _m and the side gain factor a _s. The left-right gain factor air is defined by the Equation 8 shown above, and where the worst-case peak PLR is defined Equation 1 1 :

[0075] Once the gain coefficients a _m , a _s , and air are determined, they are applied to the mid component mi and the side component im as shown by Equation 12:

[0077] Note that a _m might be permitted in this configuration to vanish to 0 to keep the output |m'| < 9 _lr, assuming Om = 0. This means that although this additional stage has prevented the mid channel information from clipping, the resulting soundstage still can reduce perceptually to mono. Providing a lower bound on a _m , would solve this issue, but would result in an incomplete gain reduction strategy, as our gain reduction budget is now finite, consisting of 20 loglO (0 _S ) + 20 loglO (Om). As such, the calculation and application of air is used to ensure satisfaction of the left-right threshold 0 _LR.

SERIAL SIDE, MID, AND LEFT-RIGHT PRIORITY LIMITING

[0078] The audio processing system may use a series priority relationship between mid and side stages, followed by the L/R limiter stage. As shown in FIG. 5A, first there is a side limiter stage 502 so that the side component m2 is the primary component for limiting, then a mid limiter stage 504 so that the mid component mi is the secondary component for limiting, then a L/R limiter stage 506. In another example shown in FIG. 5B, first there is a mid limiter stage 504 so that the mid component mi is the primary component for limiting, then a side limiter stage 502 such that the side component m2 is the secondary component for limiting, then a L/R limiter stage 506. It is noted that the stages 502, 504, and 506 are not necessarily separate processing stages and may represent only logical priority. Put another way, all gain reductions may be calculated simultaneously with different gain factors being used based on the logical priority. Recursive gain stage calculations may be used when gain factors are applied in separate stages.

[0079] In either case, let mi designate a primary component (either mid component mi or side component m2) and mu designate a secondary component (the other one of the mid component mi or side component m ₂ ). The audio processing system may determine the priority order between the mid and side component, either programmatically or based on user input, with the higher priority component being designated as the primary component mi and the lower priority component being designated as the secondary component mu.

[0080] A primary gain factor ai is applied to the primary component mi and a secondary gain factor an is applied to the secondary component mu. The secondary gain factor an is defined recursively to the primary gain factor ai to specify the priority. This is shown by Equation 13 for the primary gain factor ai and Equation 14 for the secondary gain factor an:

[0083] where |mi| is the peak of the primary component mi, | mu | is the peak of the secondary component mu, qi is the gain reduction budget of the primary component mi, and On is the gain reduction budget of the secondary component mu.

[0084] The left-right gain factor air may be defined recursively using the mid gain factor a _m and the side gain factor a _s. If the primary component is the mid component and the secondary component is the side component, then the mid gain factor a _m is defined by a ₇ and the side gain factor a _s is defined by a _{n .} If the primary component is the side component and the secondary component is the mid component, then the mid gain factor a _m is defined by a _u and the side gain factor a _s is defined by a _7. The left-right gain factor air is defined by Equation 15:

[0086] where PLR is defined by Equation 16:

[0088] Once the gain coefficients am, a _s , and air are determined, they are applied to the mid component mi and the side component m2 as shown by Equation 17:

[0090] In some embodiments, the L/R limiting stage 506 is integrated with each of the side limiting stage 502 and the mid limiting stage 504. The L/R limiter could use either a nontrivial gain reduction budget or side chain processing, with the associated risk of possible overshoot. To control these cases where overshoot might become problematic, such an embodiment would require an additional L/R limiter stage at the end of the signal path.

CONTROL SIGNAL SMOOTHING

[0091] The gain control equations described above pertain to instantaneous gain values.

If these values are applied sample-by-sample without smoothing, the result will effectively be controlled hard-clipping in the appropriate subspace. The resulting artifacts are essentially high frequency modulation of the gain-control function. To reduce these artifacts, a nonlinear low-pass filter can limit the slope of the gain-control function. In cases where a totally causal gain control response is desired, the downward clamping could occur immediately, but upward movement is restricted to some maximum slope. In cases where it is possible to look ahead in a control buffer, a maximally negative downward slope limit (determined by the lookahead length) may be applied and still hit the target control gain at the appropriate peak value. Either variant shifts the artifacts to the transient stage of musical sounds, where they are perceptually masked, and simultaneously reduces their bandwidth.

EXAMPLE PROCESSES

[0092] FIG. 6 is a flow chart of a process 600 for spatially limiting an audio signal, in accordance with some embodiments. The process 600 provides for limiting the audio signal below a threshold $LR in left-right space by gain adjusting mid and side components of the audio signal. The process 600 may have fewer or additional steps, and steps may be performed in different orders.

[0093] An audio processing system (e.g., L/R to M/S converter 102) generates 605 a mid component and a side component from an audio signal including a left channel and a right channel. The mid component and side component may be determined as defined in Equation 1. The mid component and side component represent the audio signal in mid-side space, and left channel and the right channel represent the audio signal in left-right space. The mid component may include a sum of the left channel and the right channel. The side component may include a difference between the left channel and the right channel.

[0094] The audio processing system (e.g., spatial limiter 104 or controller 110) determines 610 a left-right threshold. The left-right threshold 0 _LR defines a maximum level that is allowed for each of the left right channels. For example, neither the absolute value of the left channel nor the absolute value of the right channel should exceed the left-right threshold 0 _LR. The threshold 0 _LR may be defined by user input, or programmatically. As discussed in greater detail below, gain reduction is applied to the audio signal in mid-side space to ensure that the peaks of the left channel and the right channel are below the threshold 0 _LR.

[0095] The audio processing system (e.g., spatial limiter 104 and/or L/R limiter 106) applies 615 one or more gain factors to each of the mid component and the side component to generate an adjusted mid component and an adjusted side component. In one example, the one or more gain factors for the mid component may include a mid gain factor that is applied to the mid component, and an L/R gain factor that is applied to each of the mid component and the side component. The one or more gain factors for the side component may include a side gain factor that is applied to the side component, and the L/R gain factor. The one or more gain factors that are used for each component may depend on a priority of spatial limiting between the mid component and the side component, with the gain factor for the lower priority component being defined recursively using the higher priority component. Similarly, the L/R gain factor may be defined recursively using the gain factors for the mid and/or side components. Additional details regarding gain factors for different priorities for spatial limiting are discussed below in connection with FIGS. 7, 8 and 9.

[0096] The audio processing system (e.g., M/S to L/R converter 108) generates 620 a left output channel and a right output channel from the adjusted mid component and the adjusted side component. The left and right output channels are each limited below the left-right threshold from the one or more gain factors applied to each of the mid component and the side component.

[0097] FIG. 7 is a flow chart of a process 700 for spatially limiting an audio signal using side (or mid) limiting followed by left-right limiting, in accordance with some embodiments. The process 700 may be performed at step 615 for the process 600 to implement the priority of gain reduction stages shown in FIG. 3 A or FIG. 3B. In FIG. 3 A, the side component is a selected component for limiting, and the mid component is a non-selected component for limiting. In FIG. 3B, the mid component is the selected component, and the side component is the non-selected component. The process 700 may have fewer or additional steps, and steps may be performed in different orders.

[0098] The audio processing system (e.g., spatial limiter 104 or controller 110) determines 705 a priority for spatial limiting between the mid component and the side component defining one of the mid component or the side component as a selected component, and another one of the mid component or the side component as a secondary component. The priority for the spatial limiting may be determined programmatically, or based on user input.

[0099] The audio processing system (e.g., spatial limiter 104 or controller 110) determines 710 a gain factor for the selected component. For example, the gain factor may be defined by Equation 3 when the side component is the selected component, or the gain factor may be defined by Equation 7 when the mid component is the selected component. In either case, the gain factor is defined down to a gain reduction budget Q.

[0100] The audio processing system (e.g., spatial limiter 104) applies 715 the gain factor for the selected component to the selected component. The selected component may be multiplied by the gain factor.

[0101] The audio processing system (e.g., L/R limiter 106 or controller 110) determines 720 a left-right gain factor for the selected component and the non-selected component using the gain factor for the selected component. If the selected component is the side component, then the left-right gain factor air may be determined using Equations 4 and 5. If the selected component is the mid component, then the left-right gain factor air may be determined using Equations 8 and 9.

[0102] The audio processing system (e.g., L/R limiter 106) applies 725 the left-right gain factor to the selected component to generate an adjusted selected component and to the non- selected component to generate an adjusted non-selected component. The adjusted selected and non-selected components, one being the adjusted mid component and the other being the adjusted side component, may be used to generate the left and right output channels.

[0103] FIG. 8 is a flow chart of a process 800 for spatially limiting an audio signal using parallel side and mid limiting followed by left-right limiting, in accordance with some embodiments. The process 800 may be performed at step 615 for the process 600 to implement the priority of gain reduction stages shown in FIG. 4. The process 800 may have fewer or additional steps, and steps may be performed in different orders.

[0104] The audio processing system (e.g., spatial limiter 104 or controller 110) determines 805 a priority for spatial limiting between the mid component and the side component defining the mid and side components as equal priority components. The priority for the spatial limiting may be determined programmatically, or based on user input.

[0105] The audio processing system (e.g., spatial limiter 104 or controller 110) determines 810 a mid gain factor for the mid component. For example, mid gain factor may be defined by Equation 7, where the mid gain factor is defined down to a gain reduction budget Om.

[0106] The audio processing system (e.g., spatial limiter 104) applies 815 the mid gain factor to the mid component. The mid component may be multiplied by the mid gain factor.

[0107] The audio processing system (e.g., spatial limiter 104) determines 820 a side gain factor for the side component. For example, the side gain factor may be defined by Equation 7, where the side gain factor is defined down to a gain reduction budget 0 _S .

[0108] The audio processing system (e.g., spatial limiter 104) applies 825 the side gain factor to the side component. The side component may be multiplied by the side gain factor.

[0109] The audio processing system (e.g., L/R limiter 106 or controller 110) determines 830 a left-right gain factor for the mid component and the side component recursively using the mid gain factor and the side gain factor. The left-right gain factor air may be determined using the mid gain factor a _m and the side gain factor a _s as defined by Equations 8 and 11.

[0110] The audio processing system (e.g., L/R limiter 106) applies 835 the left-right gain factor to the mid component to generate the adjusted mid component and to the side component to generate the adjusted side component. The adjusted mid and side components may be used to generate the left and right output channels.

[0111] FIG. 9 is a flow chart of a process 900 for spatially limiting an audio signal using serial side, mid, and left-right limiting, in accordance with some embodiments. The process 900 may be performed at step 615 for the process 600 to implement the priority of gain reduction stages shown in FIG. 5 A or FIG. 5B. In FIG. 5 A, the side component is a primary component for limiting having higher priority, and the mid component is a secondary component for limiting having lower priority than the primary component. In FIG. 5B, the mid component is the primary component, and the side component is the secondary component. The process 900 may have fewer or additional steps, and steps may be performed in different orders.

[0112] The audio processing system (e.g., spatial limiter 104 or controller 110) determines 905 a priority for spatial limiting between the mid component and the side component defining one of the mid component or the side component as a primary component, and another one of the mid component or the side component as a secondary component. The priority for the spatial limiting may be determined programmatically, or based on user input.

[0113] The audio processing system may further determine one or more parameters for the spatial limiting. For example, the primary gain reduction budget qi and the secondary gain reduction budget qp may be determined either programmatically or from user input.

[0114] The audio processing system (e.g., mid gain processor 152 or side gain processor 154) determines 910 a primary gain factor for the primary component. The primary gain factor al may be defined by Equation 13. Here, the peak of the primary component |mi| is determined from mi, and the peak of the secondary component | mu | is determined from mu. The peak of the secondary component | mu | is subtracted from the threshold 9 _LR, and divided by the peak of the primary component | mi | to determine a result. The minimum value between the result and 1 is determined. The primary gain factor aiis determined from a maximum value between the result and the primary gain reduction budget qi.

[0115] The audio processing system (e.g., the mid gain processor 152 or side gain processor 154) applies 915 the primary gain factor to the primary component. For example, the primary gain factor ai may be multiplied with the primary component mi.

[0116] The audio processing system (e.g., the other one of the mid gain processor 152 or side gain processor 154) determines 920 a secondary gain factor for the secondary component recursively using the primary gain factor. The secondary gain factor is defined using the primary gain factor to implement the serial priority for spatial limiting between the mid and side components of the audio signal.

[0117] The secondary gain factor an may be defined by Equation 14. The peak of the primary component | mi | is multiplied by the primary gain factor ai, and the result of the multiplication is subtracted from the threshold 0 _LR. The result of the subtraction is divided by the peak of the primary component | mi | to determine a result. The minimum value between the result and 1 is determined. The secondary gain factor an is determined from a maximum value between the result and the secondary gain reduction budget On.

[0118] The audio processing system (e.g., the other one of the mid gain processor 152 or side gain processor 154) applies 925 the secondary gain factor to the secondary component. For example, the secondary gain factor an may be multiplied with the secondary component mu.

[0119] The audio processing system (e.g., the L/R limiter 106 or controller 110) determines 930 a left-right gain factor for the primary component and the secondary component using the primary gain factor and the secondary gain factor. The left-right gain factor air may be defined using the Equations 15 and 16. In the Equation 15, the left-right threshold i) _LR is divided by the worst-case peak in left-right space PLR, and the left-right gain factor air is determined as a minimum value between the result of the division and 1. In the Equation 16, the PLR is determined by multiplying the peak of the mid component |mi| by the mid gain factor am, multiplying the peak of the side component m2 by the side gain factor a _s , and adding the results of the multiplications. One of the gain factors a _m or a _s may be the primary gain factor ai, and the other one of the gain factors a _m or a _s may be the secondary gain factor an, depending on order of priority between the mid and side components.

[0120] The audio processing system (e.g., the L/R limiter 106) applies 935 the left-right gain factor to the primary component to generate an adjusted primary component, and the left-right gain factor to the secondary component to generate an adjusted secondary component. For example, the left-right gain factor air is multiplied with the primary component (as may be modified by the spatial limiter 140) to generate the adjusted primary component. The left-right gain factor air is multiplied with the secondary component (as may be modified by the spatial limiter 104) to generate the adjusted secondary component. The adjusted primary and secondary components may be used to generate the left and right output channels.

[0121] FIG. 10 is a flow chart of a process 1000 for controlling audio signal components in one audio coordinate system to achieve a constraint defined in another audio coordinate system, in accordance with some embodiments. The process 1000 is a generalization of the processes 600 through 900 shown in FIGS. 6 through 9, respectively, where an amplitude threshold can be defined in either the mid-side audio coordinate system or the left-right audio coordinate system, and gain factors are applied to components in the other audio coordinate system to satisfy the amplitude threshold. The process 1000 may have fewer or additional steps, and steps may be performed in different orders.

[0122] An audio processing system generates 1005 a first component and a second component in a first audio coordinate system from a third component and a fourth component of the audio signal in a second audio coordinate system. In one example, the first audio coordinate system is a mid-side audio coordinate system, the first component is a mid component, and the second component is a side component. The second audio coordinate system is the left-right audio coordinate system, the third component is a left component, and the fourth component is a right component. In another example, the first audio coordinate system is left-right audio coordinate system, the first component is a left component, and the second component is a right component. The second audio coordinate system is the mid-side audio coordinate system, the third component is a mid component and the fourth component is a side component. Conversion between audio coordinate systems may be performed by an L/R to M/S converter 102 or an M/S to L/R converter 108, as shown in FIG. 1.

[0123] The audio processing system determines 1010 determines an amplitude threshold in the second audio coordinate system defining a maximum level for each of the third component and the fourth component. If the second audio coordinate system is the left-right audio coordinate system, then the amplitude threshold may be the L-R threshold 0 _LR , defining a maximum level for the left and right components of the audio signal. If the second audio coordinate system is the mid-side audio coordinate system, then the amplitude threshold may be a M-S threshold 0 _MS, defining a maximum level for the mid and side components of the audio signal. As discussed in greater detail below, gain reduction may be applied to the audio signal in second audio coordinate system to ensure that the peaks of the third and fourth components are below the amplitude threshold.

[0124] The audio processing system applies 1015 one or more gain factors to each of the first component and the second component to generate an adjusted first component and an adjusted second component in the first audio coordinate system. The one or more gain factors may include one or more first gain factors that are applied to the first component, and one or more second gain factors that are applied to the second component. In some embodiments, a side chain processing may be performed as discussed in greater detail below in connection with FIG. 1. Here, a single gain factor is determined and applied for each of the first and second components.

[0125] In some embodiments, the one or more gain factors that are used for each component may depend on a priority of spatial limiting between the first component and the second component, with the gain factor for the lower priority component being defined recursively using the higher priority component. One or more of the gain factors may be constrained by a gain reduction budget. A smoothing function may be applied to reduce artifacts. In the example where the first audio coordinate system is the left-right audio coordinate system, the gain factors may be generated using similar techniques as discussed herein for the gain factors in the mid-side audio coordinate system.

[0126] In some embodiments, the audio processing system determines whether the amplitude threshold is satisfied from applying the one or more gain factors to each of the first component and the second component. For example, some or all of the gain factors may include a gain reduction budget. If the gain reduction budget is exhausted but the amplitude threshold fails to be satisfied, a global gain factor (e.g., left-right gain factor air) may be applied to the audio signal to satisfy the amplitude threshold. The global gain factor may be applied to the first and second components in the first coordinate system, or to the third and fourth components in the second coordinate system.

[0127] The audio processing system generates 1020 a first output channel and a second output channel in the second audio coordinate system from the adjusted first component and the adjusted second component. The first and second output channels are each limited below the amplitude threshold from the one or more gain factors applied to each of the first component and the second component. In the example where the second audio coordinate system is the left-right audio coordinate system, the first and second output channels are left and right output channels.

SIDE CHAIN PROCESSING

[0128] FIG. 11 is a schematic block diagram of a spatial limiter 1100 for side chain processing, in accordance with some embodiments. The spatial limiter 1100 integrates processing performed on the mid and side components of by the various limiter stages shown in FIGS. 3A, 3B, 4, 5A, or 5B into a single limiting stage. In that sense, the spatial limiter 1100 may be a component of the audio processing system 100 that operates with L/R to M/S converter 102 and the M/S to L/R converter 108, replacing the two-stage spatial limiter 104 and L/R limiter 106 shown in FIG. 1.

[0129] Side chain processing is particularly useful in cases where pumping artifacts caused by low frequencies are present in the cross stages. As popular conventions in audio mixing may include centering the low (e.g., bass) frequencies, the low frequencies of the mid component may need more gain reduction than the low frequencies of the side component.

[0130] The spatial limiter 1100 includes a mix peak extractor 1102, a side peak extractor 1104, a mid gain processor 1106, a side gain processor 1108, a mid mixer 1110, and a side mixer 1112.

[0131] The mid peak extractor 1102 receives the mid component 116, and determines the mid peak 214 representing a peak value of the mid component 116. The mid peak extractor 1102 provides the mid peak 214 to the mid gain processor 1106 and the side gain processor 1108. The side peak extractor 1104 receives the side component 118, and determines a side peak 216 representing a peak value of the side component 118. The side peak extractor 1104 provides the side peak 216 to the mid gain processor 1106 and the side gain processor 1108.

[0132] The mid gain processor 1106 determines a mid gain factor 1118 (a _m ) based on the mid peak 214, the side peak 216, the threshold $LR in left-right space. The side gain processor 1108 determines the side gain factor 1120 (a _s ) based on the mid peak 214, the side peak 216, the threshold $LR in left-right space.

[0133] The side chain processing may incorporate different priorities for limiting the mid or side components based on the calculations used for the mid gain factor a _m and the side gain factor a _s. By applying additional side chain processing to the control signals, we may derive the following operator matrix:

G MM MSl

ISM SS

[0134] where each entry is an independent operator. The operator matrix provides the ability to prioritize gain control not only based on broadband spatial characteristics, but a vast number of other characteristics, such as frequency content, etc. The entry MM is an operator which defines the control of the mid gain factor a _m by the mid component 214. MS is an operator which defines the control of of the side gain factor a _s by the mid component 214.

SM is an operator which defines control of the mid gain factor a _m by the side component 216. Finally, SS is an operator which defines control of the side gain factor a _s by the side component 216.

[0135] In an example where priority is implemented with side chain processing, the mid gain processor 1106 determines the mid gain factor and the side gain processor 1108 determines the side gain factor a _s using Equations 13 or 14, depending on the desired priority between the mid and side limiting stages, determines the left-right gain factor is using the Equations 15 and 16. The mid gain processor 1106 combines the mid gain factor with the left-right gain factor to generate the mid gain factor 1118. The side gain processor 1106 combines the side gain factor with the left-right gain factor to generate the final mid gain factor 1118.

[0136] The mid mixer 1110 receives the mid component 116 and the mid gain factor 1118 (a _m ), and multiplies these values to generate the adjusted mid component 124. The side mixer 212 receives the side component 118 and the side gain factor 1120 (a _s ), and multiplies these values to generate the adjusted side component 126. The adjusted mid component 124 and adjusted side component 126 may be used to generate a left output channel 132 and the right output channel 134, such as by the M/S to L/R converter 108 as shown in FIG. 1.

EXAMPLE COMPUTER

[0137] FIG. 12 is a schematic block diagram of a computer 1200, according to one embodiment. The computer 1200 is an example of circuitry that implements an audio processing system. Illustrated are at least one processor 1202 coupled to a chipset 1204. The chipset 1204 includes a memory controller hub 1220 and an input/output (I/O) controller hub 1222. A memory 1206 and a graphics adapter 1212 are coupled to the memory controller hub 1220, and a display device 1218 is coupled to the graphics adapter 1212. A storage device 1208, keyboard 1210, pointing device 1214, and network adapter 1216 are coupled to the I/O controller hub 1222. The computer 1200 may include various types of input or output devices. Other embodiments of the computer 1200 have different architectures. For example, the memory 1206 is directly coupled to the processor 1202 in some embodiments.

[0138] The storage device 1208 includes one or more non-transitory computer-readable storage media such as a hard drive, compact disk read-only memory (CD-ROM), DVD, or a solid-state memory device. The memory 1206 holds program code (comprised of one or more instructions) and data used by the processor 1202. The program code may correspond to the processing aspects described with FIGS. 1 through 11.

[0139] The pointing device 1214 is used in combination with the keyboard 1210 to input data into the computer system 1200. The graphics adapter 1212 displays images and other information on the display device 1218. In some embodiments, the display device 1218 includes a touch screen capability for receiving user input and selections. The network adapter 1216 couples the computer system 1200 to a network. Some embodiments of the computer 1200 have different and/or other components than those shown in FIG. 12.

ADDITIONAL CONSIDERATIONS

[0140] Some example benefits and advantages of the disclosed configuration include limiting an audio signal in left-right space using gain factors applied in mid-side space to shift artifacts of hard limiting to different spatial locations, and the preferences specified by the user. Processing of mid or side components of audio signals is used in various types of audio processing, and spatial priority limiting as discussed herein provides for more computationally efficient integration with such processing techniques in mid/side space.

These preferences are specified, at the lowest level, as thresholds between which the limiter enters different regimes of operation. At a higher level, this can be understood as a trade-off between the artifacts of various soundstage distortions and the artifacts of traditional peak limiting.

[0141] While particular embodiments and applications have been illustrated and described, it is to be understood that the invention is not limited to the precise construction and components disclosed herein and that various modifications, changes and variations which will be apparent to those skilled in the art may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope of the present disclosure.