Title:
METHOD AND DEVICE FOR NATURAL LANGUAGE PROCESSING
Document Type and Number:
WIPO Patent Application WO/2022/156561
Kind Code:
A1
Abstract:
A method and device for natural language processing, for use in the field of artificial intelligence. The method comprises: acquiring an input sequence (1201), which comprises an initial vector expression corresponding to at least one word in a first corpus; and producing an output sequence with the input sequence serving as an input for a self-attention model (1202), the output sequence expressing semantic information of each word in the first corpus. The self-attention model comprises multiple layers of networks, each layer of network comprises multiple self-attention modules, used for calculating degrees of relevance on the basis of an input vector, that is, the degrees of relevance between each word and neighboring words in the first corpus, and merging the degrees of relevance with the input vector to produce first sequences; the first sequences outputted by the multiple self-attention modules and outputs of the multiple self-attention modules in the preceding layer of network are merged to produce outputs of the current layer of network. The method allows, during natural language processing, the efficient interpretation of improved sematic information for each word in the corpus thereof.
Inventors:
ZHANG PENG (CN)
ZHANG JING (CN)
WEI JUNQIU (CN)
ZHANG JING (CN)
WEI JUNQIU (CN)
Application Number:
PCT/CN2022/071285
Publication Date:
July 28, 2022
Filing Date:
January 11, 2022
Export Citation:
Assignee:
HUAWEI TECH CO LTD (CN)
International Classes:
G06F40/30; G06F16/33; G06N3/04; G06N3/08
Domestic Patent References:
| WO2020237188A1 | 2020-11-26 | |||
| WO2020257812A2 | 2020-12-24 | |||
| WO2020228376A1 | 2020-11-19 |
Foreign References:
| CN112883149A | 2021-06-01 | |||
| CN111382584A | 2020-07-07 | |||
| CN110442720A | 2019-11-12 | |||
| CN110991190A | 2020-04-10 |
Attorney, Agent or Firm:
SHENPAT INTELLECTUAL PROPERTY AGENCY (CN)
Download PDF: