Title:
METHOD FOR BUILDING TRANSFORMER MODEL FOR VIDEO STORY QUESTION ANSWERING, AND COMPUTING DEVICE FOR PERFORMING SAME
Document Type and Number:
WIPO Patent Application WO/2023/286914
Kind Code:
A1
Abstract:
A method for building a transformer model for video story question answering, according to one embodiment, comprises the steps of: extracting a feature vector related to a character in a video, from video data comprising vision data and subtitle data, and question data for video question answering, and using the feature vector related to the character, in order to generate input embedding; and learning a transformer model by using the input embedding.
More Like This:
Inventors:
ZHANG BYOUNG-TAK (KR)
CHOI SEONGHO (KR)
CHOI SEONGHO (KR)
Application Number:
PCT/KR2021/013257
Publication Date:
January 19, 2023
Filing Date:
September 28, 2021
Export Citation:
Assignee:
SEOUL NAT UNIV R&DB FOUNDATION (KR)
International Classes:
G06F16/783; G06F16/9032; G06F16/906; G06N20/00; G06V10/46; H04N21/488
Domestic Patent References:
| WO2016163565A1 | 2016-10-13 |
Foreign References:
| KR20200144417A | 2020-12-29 | |||
| US20200279556A1 | 2020-09-03 |
Other References:
LINJIE LI; YEN-CHUN CHEN; YU CHENG; ZHE GAN; LICHENG YU; JINGJING LIU: "HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training", ARXIV.ORG, 29 September 2020 (2020-09-29), pages 1 - 21, XP081774033
VASWANI ASHISH, SHAZEER NOAM, PARMAR NIKI, USZKOREIT JAKOB, JONES LLION, GOMEZ AIDAN N, KAISER LUKASZ, POLOSUKHIN ILLIA: "Attention Is All You Need", ARXIV:1706.03762V1, 12 June 2017 (2017-06-12), pages 1 - 15, XP055888207
VASWANI ASHISH, SHAZEER NOAM, PARMAR NIKI, USZKOREIT JAKOB, JONES LLION, GOMEZ AIDAN N, KAISER LUKASZ, POLOSUKHIN ILLIA: "Attention Is All You Need", ARXIV:1706.03762V1, 12 June 2017 (2017-06-12), pages 1 - 15, XP055888207
Attorney, Agent or Firm:
ISQUARE PATENT & LAW FIRM (KR)
Download PDF: