FIELD

The present description relates to audio processing and in particular to determining statistical values for cepstral coefficient feature vectors.

BACKGROUND

Voice or speech recognition is increasingly being used as a part of the user interface in computing devices of many different types. Many cellular telephones allow users to push a button and speak into the microphone to perform queries and execute a variety of different commands. Portable and desktop computers perform similar functions. These systems may also convert the speech to text and use this as text input for appointments, messages, or stored documents. Some of these systems process the user speech locally in the device, but many send a recording of the speech to a remote server. Automobiles may also receive voice commands and queries to operate a navigation system or other functions in the vehicle including making calls and sending messages. Voice or speech recognition is also used for identification, logon, and other purposes.

Users prefer a quick response to voice inputs; however, it may require significant processing resources to complete an accurate analysis of the speech or any other audio. In addition many speech analysis techniques are designed to receive a complete utterance and then analyze the complete utterance as a whole. This requires that the system wait for the user to stop speaking and then begin processing using the entire utterance. This inherent latency may be annoying especially when compared to the immediacy of keyboard and mouse inputs. For some systems, there may not be enough memory available to buffer the utterance so that speech analysis is not possible.

One technique used for speech recognition is to analyze the speech for Mel-Frequency Cepstral Coefficients (MFCCs). The MFCC's are compared to a reference for automatic speech recognition (ASR) and speaker recognition. In order to improve the accuracy and reliability of the MFCC, Cepstral Mean Subtraction (CMS) is used in combination with Cepstral Variance Normalization (CVN). CVN can be regarded as noise normalization in that the mean of the signal is subtracted which leads to the removal of stationary noise. These techniques allow the system to be used to good effect not only for user input purposes but also for user authentication and login passwords. BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements.

Figure 1 is a diagram of feature vector calculation for a stream of audio samples in an audio pipeline according to an embodiment.

Figure 2 is a process flow diagram of feature vector determination for a steam of audio samples according to an embodiment.

Figure 3 is a block diagram of a speech recognition system according to an embodiment. Figure 4 is a block diagram of a computing device incorporating audio processing according to an embodiment.

DETAILED DESCRIPTION

A technique for cepstral variance normalization is described that may be performed in real time locally in a small, portable, or fixed device for verifying the identity of the speaker, understanding a command, and for other applications.

To perform CMS (Cepstral Mean Subtraction) and CVN (Cepstral Variance

Normalization) in MFCCs (Mel-Frequency Cepstral Coefficients) the mean and variance of the MFCCs is estimated on a complete audio file. The estimates are then used to apply the normalizations frame-wise on the audio file. As described herein, CVN may be performed as the audio is received sample-by-sample. This may be done without any buffering. Using the described technique, the accuracy of recognition is greatly improved over real-time feature extraction without normalization and is similar to that of offline feature extraction performed in forensics analysis.

A typical technique for determining CMS and CVN offline using the entire audio file may be stated in the form of the equation below.

where x _i denotes the speech signal at time step i,f(x _i ) denotes the feature extraction function, μ denotes the estimated mean of all features, σ denotes the estimated variance, μ and σ are required prior to normalization. This, however, requires a preprocessing of all features of an audio file. As shown the features are normalized by taking the feature for each frame, subtracting the mean and diving by the variance. These are summed for an overall value for the utterance. The determination of the mean μ over all frames may be done as follows:

wherein N denotes the number of frames.

Similarly having determined μ, the variance σ of the mean may be determined as follows:

N d enotes the number of frames.

In order to determine useful values of the features without first determining a mean and a variance, Equation 1 may be changed as follows:

In this case, the mean and variance values are replaced with an interim mean and variance that is determined up to and including the most recent frame. These values μ _{i- ι} and σ _{i- ι} are updated for each new frame x _i . The mean and variance after each sample are determined as follows:

As shown, each new mean value μ _i · is determined by using the previous value μ,·. _/ factored by a parameter α and adjusted by a factored current value of the featuref(x _i ). The same approach may be used with the variance, where the current variance σi ² is determined by using the previous value factored by a parameter a and adjusted by a factored current value of the feature The higher the relevance parameter a is, the more emphasis is put on the actual frame currently being processed, i.e. the i ^th frame. This technique for in line real time normalization does not introduce any latency or delay as the estimated mean μ and the estimated variance σ are updated with each subsequent frame. The estimated mean and variance rely only on frames that the system has already seen as it processes the audio as the audio is received. There is no look ahead required. This is depicted in Figure 1. In addition, the memory requirements are very small because only the two values for the last frame are used and these are updated with each frame.

Figure 1 is a diagram of feature vector calculation to indicate how feature vectors may be determined as audio is being received. Feature vectors are arriving from right to left with the most recently received feature vectors on the right. The current mean 1 12 and the current variance 1 14 are used to determine the incoming feature vector 116. As the feature vectors move to the left, the incoming feature vector 116 becomes the current feature vector 104. The current feature vector 104 is then used to update the prior mean 110 to determine the current mean 112. Similarly, the prior variance 114 is used together with the new current mean 1 12 to determine the current variance 114. The current mean and variance are then plugged into determining or processing the upcoming unused feature vector 116.

The already seen feature vectors 102 increase as the current feature vector 104 moves to the left and unseen feature vectors 106 become the current feature vector 104 and then the already seen feature vectors 102. The feature vectors may be summed or analyzed as they are produced so that the analysis may be finished almost immediately after the end of the utterance.

The same principles set forth in Equations 4, 5, and 6 may also be expressed in pseudo code. In the example below "feat" denotes a feature vector,

As shown, the whole audio file need not be read and buffered. The memory requirements latency from buffering the whole audio stream is avoided. This is particularly valuable for using speaker recognition for logins and for voice authentication.

The described CVN may be used to improve an online feature extraction system. The described online CMS and CVN may be performed without introducing any latency. The CVN does not require any offline processing and estimates the variance based on already seen data. During runtime, the variance is updated with every new feature frame.

The online CVN provides a performance in voice recognition that is similar to conventional offline techniques without requiring any form of feature buffering and without producing latency because the system does not require any knowledge of future feature frames.

The online techniques herein may be used with only CMS or CMS may be combined with CVN. A further modification may be made by initializing the mean μ and the variance σ to some value before the processing is started. Initial values for mean and variance can be specified by estimating appropriate values empirically using similar hardware. This may provide a significant improvement for determining early features. However, the advantages of the initial values are diminished as more features vectors are received.

Figure 2 is a process flow diagram of determining a CMS and a CVN online or in realtime as audio frames arrive. The system starts based on receiving audio frames from a microphone or other source. The audio may be processed or formatted in any of a variety of different ways. Typically the audio is in a digital sample format. At 202, the CMS (μ) value is optionally initialized and at 204, the CVN (σ) values are optionally initialized. These may be initialized to zero, to a middle value, or to some accurate estimate of a typical value for the particular type of audio and the hardware system that is receiving the audio.

At 206, a feature vector is determined for the first frame in the received audio. This may be done as described in the context of Equation 4. The initialization of the mean and variance of 202 is optional. If there is no initialization then the values from the first frame's feature vector may be used. In other words the mean μ is set equal to the feature value and the variance will be zero because there is only one mean value. For the next frame the same values from the first frame's feature vector may be used again as the values for the previous frame. As a further alternative, if there is no initialization then 0 may be used for μ and 1 may be used for σ. The second frame will then use the values from the first frame. The determination of the first frame's feature vector may be performed before the last frame has been received. The feature vector processing may start as soon as the first frame is received.

At 208 μ is updated using the determined feature vector. If there is no initialization and this is the first frame then updating μ corresponds to establishing μ in the first instance using the feature value for the first frame. At 210 σ is updated using the updated μ and the determined feature vector. If μ and σ were initialized then the initial values may be included in this updating process. An example updating process is set forth in Equations 5 and 6. If there is no initialized value, then the values of the first frame may be used as initial values, e.g. μ=(feature vector) and σ=(feature vector)* (feature vector). So equations 5 and 6 will be processed with these initial values for the first frame. If instead there are no initial values at all, that is if there is no current value of μ and σ to be updated, then the first terms in Equations 5 and 6 drop out for the first frame.

At 212, the feature vector for the next frame is determined using the updated μ and σ values. This process of updating μ and σ and determining feature vectors continues until the entire utterance is processed. At 214, it is determined whether there is a next frame. The operations continue until all of the frames are processed. At that time the feature vectors are sent to the recognition stage (216) and the process ends.

Figure 3 is a block diagram of a speech recognition system 300 which may use feature vectors and CMS and CVN values determined as described herein. While embodiments are described in the context of a speech recognition system, the approach described herein and in the context of Figure 3 is not restricted to speech recognition, but may also be used with speaker recognition or authentication, language recognition, speaker diarization, etc. An input signal, such as speech from a microphone, is received at an analog to digital converter (A/D) 310. A feature extractor or spectral analysis unit 320 receives digitized speech frames from the A/D. A decoder 330 compares the extracted features received from the feature extractor to a connected acoustic model 340, and a connected language model 350 and produces recognized speech 360 or commands as an output.

The input signal 305 represents the input speech or some other audio. It is first digitized using the A/D 310. The digital signal is then sliced up into frames typically of 10, 15, or 20 ms in the A/D or the feature extractor. Each frame of the signal is then converted into a corresponding feature vector which is used to characterize the spectral properties of the input signal.

The feature vector may have one or more dimensions and may include one or more different components. These components may include, for example MFCC components, delta MFCCs, and delta-delta MFCCs. The feature vectors generated by the feature extractor unit 320 are then submitted into the decoder 330 which determines whether the speech matches an authenticated user with enough accuracy. Alternatively, it may determine a sentence or a sequence of words that has the highest probability given the acoustic events characterized by the feature vectors. This is done using any of a variety of different models 340, 350 or analysis. The recognized speech 360 may then be provided for storage, execution, or communication to others.

Figure 4 illustrates a computing device 100 in accordance with one implementation. The computing device 100 houses a system board 2. The board 2 may include a number of components, including but not limited to a processor 4 and at least one communication package 6. The communication package is coupled to one or more antennas 16. The processor 4 is physically and electrically coupled to the board 2.

Depending on its applications, computing device 100 may include other components that may or may not be physically and electrically coupled to the board 2. These other components include, but are not limited to, volatile memory (e.g., DRAM) 8, non-volatile memory (e.g., ROM) 9, flash memory (not shown), a graphics processor 12, a digital signal processor (not shown), a crypto processor (not shown), a chipset 14, an antenna 16, a display 18 such as a touchscreen display, a touchscreen controller 20, a battery 22, an audio codec (not shown), a video codec (not shown), a power amplifier 24, a global positioning system (GPS) device 26, a compass 28, an accelerometer (not shown), a gyroscope (not shown), a speaker 30, a camera 32, a microphone array 34, and a mass storage device (such as hard disk drive) 10, compact disk (CD) (not shown), digital versatile disk (DVD) (not shown), and so forth). These components may be connected to the system board 2, mounted to the system board, or combined with any of the other components.

The communication package 6 enables wireless and/or wired communications for the transfer of data to and from the computing device 100. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communications channels, etc., that may communicate data through the use of modulated electromagnetic radiation through a non-solid medium. The term does not imply that the associated devices do not contain any wires, although in some embodiments they might not. The communication package 6 may implement any of a number of wireless or wired standards or protocols, including but not limited to Wi-Fi (IEEE 802.1 1 family), WiMAX (IEEE 802.16 family), IEEE 802.20, long term evolution (LTE), Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT, Bluetooth, Ethernet derivatives thereof, as well as any other wireless and wired protocols that are designated as 3G, 4G, 5G, and beyond. The computing device 100 may include a plurality of communication packages 6. For instance, a first communication package 6 may be dedicated to shorter range wireless communications such as Wi-Fi and Bluetooth and a second communication package 6 may be dedicated to longer range wireless communications such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev-DO, and others. The microphones 34 and the speaker 30 are coupled to one or more audio chips 36 to perform digital conversion, feature extraction, encoding and decoding, and frame based processing as described herein. The processor 4 is coupled to the audio chip, through an audio front end, for example, to drive the process with interrupts, set parameters, and control operations of the audio chip. Frame-based processing may be performed in the audio chip or in the communication package 6. Power management functions may be performed by the processor, coupled to the battery 22 or a separate power management chip may be used.

In various implementations, the computing device 100 may be a laptop, a netbook, a notebook, an ultrabook, a smartphone, a wearable device, a tablet, a personal digital assistant (PDA), an ultra mobile PC, a mobile phone, a desktop computer, a server, a printer, a scanner, a monitor, a set-top box, an entertainment control unit, a digital camera, a portable music player, or a digital video recorder. The computing device may be fixed, portable, or wearable. In further implementations, the computing device 100 may be any other electronic device that processes data.

Embodiments may be implemented as a part of one or more memory chips, controllers,

CPUs (Central Processing Unit), microchips or integrated circuits interconnected using a motherboard, an application specific integrated circuit (ASIC), and/or a field programmable gate array (FPGA).

References to "one embodiment", "an embodiment", "example embodiment", "various embodiments", etc., indicate that the embodiment(s) so described may include particular features, structures, or characteristics, but not every embodiment necessarily includes the particular features, structures, or characteristics. Further, some embodiments may have some, all, or none of the features described for other embodiments.

In the following description and claims, the term "coupled" along with its derivatives, may be used. "Coupled" is used to indicate that two or more elements co-operate or interact with each other, but they may or may not have intervening physical or electrical components between them.

As used in the claims, unless otherwise specified, the use of the ordinal adjectives "first", "second", "third", etc., to describe a common element, merely indicate that different instances of like elements are being referred to, and are not intended to imply that the elements so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

The drawings and the forgoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, orders of processes described herein may be changed and are not limited to the manner described herein. Moreover, the actions of any flow diagram need not be implemented in the order shown; nor do all of the acts necessarily need to be performed. Also, those acts that are not dependent on other acts may be performed in parallel with the other acts. The scope of embodiments is by no means limited by these specific examples. Numerous variations, whether explicitly given in the specification or not, such as differences in structure, dimension, and use of material, are possible. The scope of embodiments is at least as broad as given by the following claims.

The following examples pertain to further embodiments. The various features of the different embodiments may be variously combined with some features included and others excluded to suit a variety of different applications. Some embodiments pertain to a method that includes receiving a sequence of frames of digitized audio from a microphone, determining a feature vector for a first frame of the sequence of frames, the feature vector being determined using an initial mean and an initial variance, updating the initial mean to a current mean using the determined feature vector for the first frame, updating the variance to a current variance using the current mean and the determined feature vector for the first frame, determining a next feature vector for each of subsequent frames of the sequence of frames, after determining a next feature vector for each subsequent frame, updating the current mean to a next current mean and updating the current variance to a next current variance and wherein determining a feature vector for a subsequent frame comprises using the next current mean and the next current variance, and sending the determined feature vectors to a recognition stage of an audio processing pipeline.

Further embodiments include setting the initial mean and variance to estimated values for the digitized audio.

In further embodiments the initial mean and variance are set based on the feature vector determined for the first frame.

In further embodiments determining a feature vector comprises determining a mel- frequency cepstral coefficient.

In further embodiments the current mean comprises a cepstral mean and the current variance comprises a cepstral variance.

In further embodiments determining a next feature vector comprises determining a next feature vector using cepstral mean substraction and cepstral variance normalization.

In further embodiments the digitized audio is speech.

In further embodiments the recognition stage comprises a decoder, the method further comprising receiving the determined feature vectors at the decoder and determining at the decoder whether the digitized speech matches an authenticated user. In further embodiments the next current mean is determined after each next feature vector using only the corresponding next feature vector and feature vectors of earlier frames of the sequence of frames.

In further embodiments the next current variance is determined after each next feature vector using only the corresponding next current mean.

Some embodiments pertain to a machine-readable medium having instructions thereon that, when executed by the machine, cause the machine to perform operations that include receiving a sequence of frames of digitized audio from a microphone, determining a feature vector for a first frame of the sequence of frames, the feature vector being determined using an initial mean and an initial variance, updating the initial mean to a current mean using the determined feature vector for the first frame, updating the variance to a current variance using the current mean and the determined feature vector for the first frame, determining a next feature vector for each of subsequent frames of the sequence of frames, after determining a next feature vector for each subsequent frame, updating the current mean to a next current mean and updating the current variance to a next current variance and wherein determining a feature vector for a subsequent frame comprises using the next current mean and the next current variance, and sending the determined feature vectors to a recognition stage of an audio processing pipeline.

In further embodiments determining a feature vector comprises determining a mel- frequency cepstral coefficient.

In further embodiments the current mean comprises a cepstral mean and the current variance comprises a cepstral variance.

In further embodiments determining a next feature vector comprises determining a next feature vector using cepstral mean substraction and cepstral variance normalization.

In further embodiments the next current mean is determined after each next feature vector using only the corresponding next feature vector and feature vectors of earlier frames of the sequence of frames.

In further embodiments the next current variance is determined after each next feature vector using only the corresponding next current mean.

Some embodiments pertain to an apparatus that includes a feature extractor to receive a sequence of frames of digitized audio from a microphone through an analog to digital converter, to determine a feature vector for a first frame of the sequence of frames, the feature vector being determined using an initial mean and an initial variance, to update the initial mean to a current mean using the determined feature vector for the first frame, to update the variance to a current variance using the current mean and the determined feature vector for the first frame, to determine a next feature vector for each of subsequent frames of the sequence of frames, to update the current mean to a next current mean and update the current variance to a next current variance after determining a next feature vector for each subsequent frame, wherein determining a feature vector for a subsequent frame comprises using the next current mean and the next current variance, and a recognition stage to recognize the extracted feature vectors.

In further embodiments the digitized audio is speech.

In further embodiments the recognition stage comprises a decoder, the decoder receiving the determined feature vectors and determining whether the digitized speech matches an authenticated user.

In further embodiments the next current mean is determined after each next feature vector using only the corresponding next feature vector and feature vectors of earlier frames of the sequence of frames.

Some embodiments pertain to an apparatus that includes means for receiving a sequence of frames of digitized audio from a microphone, means for determining a feature vector for a first frame of the sequence of frames, the feature vector being determined using an initial mean and an initial variance, means for updating the initial mean to a current mean using the determined feature vector for the first frame, means for updating the variance to a current variance using the current mean and the determined feature vector for the first frame, means for determining a next feature vector for each of subsequent frames of the sequence of frames, for updating the current mean to a next current mean and updating the current variance to a next current variance after determining a next feature vector for each subsequent frame and wherein determining a feature vector for a subsequent frame comprises using the next current mean and the next current variance, and means for sending the determined feature vectors to a recognition stage of an audio processing pipeline.

In further embodiments the initial mean and variance are set based on the feature vector determined for the first frame.

In further embodiments determining a feature vector comprises determining a mel- frequency cepstral coefficient.

In further embodiments the current mean comprises a cepstral mean and the current variance comprises a cepstral variance.

In further embodiments determining a next feature vector comprises determining a next feature vector using cepstral mean substraction and cepstral variance normalization.

In further embodiments the recognition stage comprises means for decoding, the means for decoding determining whether the digitized speech matches an authenticated user.

Some embodiments pertain to a computing system that includes a plurality of microphones to receive speech, an analog to digital converter to convert the speech to a sequence of frames of digitized audio, a feature extractor to receive the sequence of frames of digitized audio from the analog to digital converter, to determine a feature vector for a first frame of the sequence of frames, the feature vector being determined using an initial mean and an initial variance, to update the initial mean to a current mean using the determined feature vector for the first frame, to update the variance to a current variance using the current mean and the determined feature vector for the first frame, to determine a next feature vector for each of subsequent frames of the sequence of frames, to update the current mean to a next current mean and update the current variance to a next current variance after determining a next feature vector for each subsequent frame, wherein determining a feature vector for a subsequent frame comprises using the next current mean and the next current variance, and a recognition stage to recognize the extracted feature vectors.

In further embodiments decoder, the decoder receiving the determined feature vectors and determining whether the digitized speech matches an authenticated user.