CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application is a non-provisional of and claims priority to U.S. Provisional Patent Application No. 62/339,782, filed on May 20, 2016, the entire contents of which are hereby incorporated by reference.

BACKGROUND

[0002] This specification relates to classifying data using neural networks.

[0003] Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters.

[0004] Some neural networks are recurrent neural networks. A recurrent neural network is a neural network that receives an input sequence and generates an output sequence from the input sequence. In particular, a recurrent neural network can use some or all of the internal state of the network from a previous time step in computing an output at a current time step.

SUMMARY

[0005] This specification describes a system implemented as computer programs on one or more computers in one or more locations. The system is a system for classifying an input example using a comparison set of comparison examples.

[0006] One example aspect of the present disclosure is directed to a method performed by one or more computers for classifying a new example using a comparison set of comparison examples, the method comprising: maintaining a comparison set, the comparison set comprising a plurality of comparison examples and a respective label vector for each of the plurality of comparison examples, each label vector including a respective score for each label in a predetermined set of labels; receiving a new example; determining a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples; and generating a respective label score for each label in the predetermined set of labels from, for each of the comparison examples, the respective attention weight for the comparison example and the respective label vector for the comparison example, wherein the respective label score for each of the labels represents a likelihood that the label is a correct label for the new example.

[0007] In an implementation of this aspect, the method may further comprise classifying the new example based on the respective label scores for the labels in the predetermined set of labels. For example, the new example may be classified using a classification derived from the label scores. As further examples, the new example may be classified using/based on the label with the highest likelihood of being correct, or it may be classified using/based on the n (n > 1) labels with the highest likelihood of being correct, or it may be classified using/based on all labels with a likelihood of being correct that exceeds a threshold, etc.

[0008] Another example aspect of the present disclosure is directed to a method performed by one or more computers for controlling an agent in an environment. The method comprises generating one or more control inputs for controlling the agent based on an output of a method of the first aspect (for example based on generated label scores, or based on a classification derived from generated label scores).

[0009] The subject matter described in this specification can be implemented in particular embodiments so as to realize one or more of the following advantages. A data classification system including a neural network can effectively determine labels for unobserved classes without requiring any changes to the neural network or re-training the neural network from scratch. In particular, the neural network can take a new set of labeled examples as its input, as well as a new input example that needs to be classified, and can effectively classify the new input example using the new set of labeled examples even though the new set of labeled examples are different from a set of labeled examples that was previously used to train the neural network. Embodiments of the subject matter may therefore provide neural networks that may be trained to process input data (for example image data) to generate output data indicating a property associated with the image data in a more effective manner.

[0010] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG. 1 shows an example data classification system.

[0012] FIG. 2 is a flow diagram of an example process for classifying a new example using a comparison set of comparison examples.

[0013] FIG. 3 is a flow diagram of an example process for determining a respective attention weight for each comparison example.

[0014] FIG. 4 is a flow diagram of an example process for training a neural network attention mechanism.

[0015] Like reference numbers and designations in the various drawings indicate like elements.

DETAIL DESCRIPTION

[0016] This specification generally describes a data classification system that is configured to classify any of a variety of data, i.e., any kind of data object that can be classified as belonging to one or more categories.

[0017] For example, if the input data is images, the data classification system may be a visual recognition system that determines whether an input image includes images of objects that belong to object categories from a predetermined set of object categories. As another example, if the input data is videos or portions of videos, the data classification system may be a video classification system that determines what topic or topics an input video or video portion relates to. As another example, if the input data is audio data, the data classification system may be a speech recognition system that determines, for a given spoken utterance, the term or terms that the utterance represents. As another example, if the input data is text data, the data classification system may be a text classification system that determines what topic or topics an input text segment relates to.

[0018] FIG. 1 shows an example data classification system 100.

[0019] The data classification system 100 is an example of a system implemented as computer programs on one or more computers in one or more locations in which the systems, components, and techniques described below are implemented.

[0020] The data classification system 100 is configured to receive an input example 102 and to generate an output 1 16 that classifies the input example 102, e.g., determines a category which the input example 102 belongs to or a topic that the input example 102 relates to. [0021] The data classification system 100 stores comparison example data 104 that includes a comparison set. The comparison set includes a number of comparison examples and a respective label vector for each of the comparison examples. Each label vector includes a respective score for each label in a predetermined set of labels. Generally, the label scores in a given label vector identify the known label or labels for the corresponding comparison example.

[0022] The data classification system 100 includes a classifier 112 that takes the comparison example data 104 and the input example 102 as inputs. The classifier 112 generates the output 116 as a function of the comparison example data and the input example 102 using a neural network attention mechanism 114. The process for generating the output 116 using the neural network attention mechanism 114 is described in more detail below with reference to FIG. 2 and FIG. 3.

[0023] An output 116 from the data classification system 100 may be used in many applications. As one example application, an output 116 from the data classification system 100 may be used in selection of one or more actions for control of an agent in an

environment. For example, the output 116 of the data classification system 100 may be used as, or used to generate, one or more control inputs that provide control of the agent in the environment. The environment may be a real -world environment in which an object in the real -world environment is controlled. For example, the object/agent may be a vehicle, such as an autonomous ("self-driving") or partially-autonomous vehicle, and the output 116 of the data classification system 100 may be used as, or may be used to generate, control inputs to control the vehicle, for example to control navigation of the vehicle. Some aspects may therefore address problems associated with efficient and effective selection, based on input data, of actions for control of an agent.

[0024] FIG. 2 is a flow diagram of an example process 200 for classifying a new example using a comparison set of comparison examples. For convenience, the process 200 will be described as being performed by a system of one or more computers located in one or more locations. For example, a data classification system, e.g., the data classification system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 200.

[0025] The system maintains a comparison set (step 202). As described above, the comparison set can include k comparison examples and can be denoted as: where ^α: *· is an comparison example and ^yi is a respective label vector for the comparison example ^;i * . Each label vector ^¾ includes a respective score for each label in a

predetermined set of labels. The label scores in a given label vector generally identify the known label or labels for the corresponding comparison example.

[0026] In some cases, the comparison set can be different from a comparison set that was used to train the neural network. That is, the neural network can be used to classify a new example using a new set of comparison examples different from the set used to train the neural network without re-training the neural network. In some of these cases, the new set of comparison examples can include some comparison examples that were used in the training of the neural network and some additional or "unseen" comparison examples. In others of these cases, the new set of comparison examples can include only "unseen" comparison examples and no comparison examples that were used in the training.

[0027] The system receives a new example (step 204). The new example can be denoted as

[0028] After receiving the new example, the system determines a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples (step 206). The process for determining the respective attention weight for each comparison example is described in more detail below with reference to FIG. 3.

[0029] The system then generates, for the new example, a respective label score for each label in the predetermined set of labels so that the respective label score for each of the labels represents a likelihood that the label is the correct label for the new example (step 208). In particular, the system generates the label scores from, for each of the comparison examples, the respective attention weight for the comparison example and the scores in the respective label vector for the comparison example.

[0030] In some implementations, the label scores can be computed as follows:

fc i ^=l , (1) wherein ^Xi is a comparison example and ^th is the label vector for the comparison example from the comparison set ^" ^ ^; *> #*¾i=t , and ' ^{l " ' '{} ' ' is an attention weight for the comparison example given the new example ^x . .

[0031] Eq. 1 describes the output for the new example as a linear combination of the label vectors in the comparison set. For each comparison example ^Xi , the system multiplies the label vector ^¾ for the comparison example ^r « by the attention weight " - ' ^! ' ^{! >: ;} for the comparison example ^j: Ho generate a weighted label vector 'H^ Wit* for the comparison example ^Xi . The system then sums all the weighted label vectors to generate a combined label vector ^;/ that includes a respective label score for each label in the predetermined set of labels. Each label score for each of the labels represents a likelihood that the label is a correct label for the new example.

[0032] In some implementations, the system may further classify the new example based on the respective label scores for the labels in the predetermined set of labels. For example, the new example may be classified using a classification derived from the label scores. As further examples, the new example may be classified using/based on the label with the highest likelihood of being correct, or it may be classified using/based on the n (n > 1) labels with the highest likelihood of being correct, or it may be classified using/based on all labels with a likelihood of being correct that exceeds a threshold.

[0033] After the new example has been classified, the system can provide the respective label scores for the labels in the predetermined set of labels, one or more labels with the highest likelihood of being correct, or both to another system for further processing. For example, the new example can be sensor data (e.g., an image) captured by sensor of an agent, and the other system can be a control system that makes decisions about how to control the agent in an environment using the received label scores, the one or more labels with the highest likelihood of being correct, or both.

[0034] FIG. 3 is a flow diagram of an example process 300 for determining the respective attention weight for each comparison example using a neural network attention mechanism. For convenience, the process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a data classification system, e.g., the data classification system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 300.

[0035] The system processes the comparison example ^Xi using a comparison example neural network ^' to determine a numeric embedding of the comparison example (step 302). Generally, a numeric embedding is an ordered collection of numeric values, e.g., a vector of floating point values or of quantized floating point values.

[0036] In some implementations, when the examples are images, the comparison example neural network ' ' ^{ν' ' * '} is a convolutional neural network configured to receive an input comparison example and to generate the embedding of the comparison example. In some other implementations, when the examples are words, phrases, or other text segments, the comparison example neural network is a text embedding neural network that embeds input text into a vector.

[0037] In some other implementations, the comparison example neural network ^{" " 5} ' is a bidirectional Long-Short Term Memory (LSTM) neural network configured to process the comparison example in the context of a sequence of comparison examples from the comparison examples in the comparison set to generate the numeric embedding of the comparison example. That is, the system can arrange the comparison examples as a sequence, and for each comparison example, process the comparison example or a feature representation of the comparison example, e.g., as generated by a convolutional neural network or a text embedding neural network, using a bidirectional LSTM neural network according to the sequence to generate the numeric embedding of the comparison example.

f ( ί

[0038] The system processes the new example using a new example neural network ^■ to determine a numeric embedding of the new example (step 304). In some implementations, when the examples are images, the new example neural network ' °* is a convolutional neural network configured to receive an input new example and to generate the embedding of the new example. In some other implementations, when the examples are words, phrases, or other text segments, the comparison example neural network is a text embedding neural network that embeds input text into a vector.

[0039] In some other implementations, the new example neural network " " ^{"* "} " ^' is an LSTM neural network with read attention over the numeric embeddings for the plurality of comparison examples that is configured to process a sequence including K instances of the new example or of a feature representation of the new example to generate the numeric embedding of the comparison example. The new example neural network ^* can be expressed as follows:

f( _s S) = attLSTM(/'{ ) _r £?(.S}, K)

ft ( q where ·' (x) is the new example or the feature representation of the new example and - ^; is the set of numeric embeddings of the comparison examples. That it is, at each time step of the K steps other than the last time step, the system processes the feature representation of the new example using an LSTM neural network in accordance with a current internal state of the LSTM neural network to generate an initial update to the internal state. The system then combines, e.g., sums, the initial update and the feature representation to generate an embedding key and applies a content-based attention mechanism over the numeric embeddings for the comparison examples using the embedding key to generate an attention vector. The system then combines, e.g., concatenates, the initial update and the attention vector to generate the updated internal state, i.e., the internal state that will be used as the current internal state for the next time step. The system can then use the initial update or the embedding key for the last time step, i.e., the K-th step, as the final numeric embedding of the new example.

[0040] In some cases, when the new example neural network and the comparison example neural network both are or include a convolutional neural network, the new example neural network and the comparison example neural network share at least some parameters.

[0041] The system determines the respective attention weight by determining a similarity metric between the numeric embedding of the comparison example and the numeric embedding of the new example (step 306). In some implementations, the similarity metric is a cosine distance. In some implementations, the system computes the respective attention weight ^{' '} · - ^{Λ '■ 1 ! ;} for each comparison example ^Xi given the new example · ^ε based on the following model: ' *·'' ' *-tf=* , (2) where ·' * is the new example neural network that embeds new example ¹ is the comparison example neural network that embeds comparison example ^;/ * , and c( ) is a cosine distance function.

[0042] FIG. 4 is a flow diagram of an example process 400 for training a neural network attention mechanism.

[0043] For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a data classification system, e.g., the data classification system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 400 to train the neural network attention mechanism.

[0044] The system samples a label set from a distribution over possible label sets (step 402). Each label set is a different combination of labels from a complete set of possible labels. In some cases, each label set includes the same number of labels, e.g., two or three. In other cases, the different label sets include different numbers of labels, e.g., ranging from one or two to five labels. For example, the distribution can uniformly weight all of the possible label sets and the system can randomly sample the label set using the distribution

[0045] The system then samples a training comparison set and a batch of input examples using the sampled label set (step 404). In particular, for each label in the sampled label set, the system samples a set of comparison examples having that label. The set of sampled comparison examples and their corresponding labels for all of the labels in the sampled label set form the training comparison set. The batch includes a random sequence of comparison examples that have one of the labels in the sampled labeled set.

[0046] The system trains a neural network attention mechanism to minimize an error predicting the labels for the examples in the batch conditioned on the training comparison set (step 406). In particular, the system trains the attention mechanism to adjust current values of the parameters of the attention mechanism to reduce the error using a conventional neural network training technique, e.g., a gradient descent-based training technique.

[0047] The system can repeatedly perform the process 400, i.e., by repeatedly sampling label sets and then training the neural network attention mechanism based on the sampled label set, to determine trained values of the parameters of the neural network attention mechanism. By training the neural network attention mechanism in this manner, the system can rapidly determine trained values of the parameters of the neural network attention mechanism that satisfy the training objective of reducing the error and the trained neural network can perform well even when the comparison set being used to classify an input example is different from any of the comparison sets used in training the neural network.

[0048] For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0049] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. The computer storage medium is not, however, a propagated signal.

[0050] The term "data processing apparatus" encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0051] A computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0052] As used in this specification, an "engine," or "software engine," refers to a software implemented input/output system that provides an output that is different from the input. An engine can be an encoded block of functionality, such as a library, a platform, a software development kit ("SDK"), or an object. Each engine can be implemented on any appropriate type of computing device, e.g., servers, mobile phones, tablet computers, notebook computers, music players, e-book readers, laptop or desktop computers, PDAs, smart phones, or other stationary or portable devices, that includes one or more processors and computer readable media. Additionally, two or more of the engines may be implemented on the same computing device, or on different computing devices.

[0053] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). For example, the processes and logic flows can be performed by and apparatus can also be implemented as a graphics processing unit (GPU).

[0054] Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0055] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0056] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.

[0057] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network ("LAN") and a wide area network ("WAN"), e.g., the Internet.

[0058] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0059] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0060] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0061] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.