Title:
SUPER-RESOLUTION USING TIME-SPACE-FREQUENCY TOKENS
Document Type and Number:
WIPO Patent Application WO/2023/240609
Kind Code:
A1
Abstract:
A computing device including a processor configured to receive input video data including a plurality of input images. Each of the input images may include a plurality of input pixels. For each input image, the processor may be further configured to perform upsampling on the input image and divide the upsampled input image into a respective plurality of patches. For each patch, the processor may be further configured to generate a plurality of time-space-frequency tokens. The plurality of time-space-frequency tokens generated for the patch may be indexed by timestep, spatial location, and frequency. At least in part at a trained machine learning model, the processor may be further configured to generate a plurality of super-resolved output images based at least in part on the time-space-frequency tokens. The processor may be further configured to output the super-resolved output images.
More Like This:
Inventors:
YANG HUAN (US)
FU JIANLONG (US)
FU JIANLONG (US)
Application Number:
PCT/CN2022/099502
Publication Date:
December 21, 2023
Filing Date:
June 17, 2022
Export Citation:
Assignee:
MICROSOFT TECHNOLOGY LICENSING LLC (US)
YANG HUAN (CN)
YANG HUAN (CN)
International Classes:
H04N19/85; G06N3/04; G06N3/08; G06T3/40; G06T5/00; H04N19/48
Foreign References:
| CN113139898A | 2021-07-20 |
Other References:
RUNYUAN CAI ET AL: "FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 21 November 2021 (2021-11-21), XP091101952
CHENGXU LIU ET AL: "Learning Trajectory-Aware Transformer for Video Super-Resolution", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 8 April 2022 (2022-04-08), XP091202389
CHARLIE NASH ET AL: "Generating Images with Sparse Representations", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 5 March 2021 (2021-03-05), XP081907026
MENG-HAO GUO ET AL: "Attention Mechanisms in Computer Vision: A Survey", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 15 November 2021 (2021-11-15), XP091099501
CHENGXU LIU ET AL: "Learning Trajectory-Aware Transformer for Video Super-Resolution", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 8 April 2022 (2022-04-08), XP091202389
CHARLIE NASH ET AL: "Generating Images with Sparse Representations", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 5 March 2021 (2021-03-05), XP081907026
MENG-HAO GUO ET AL: "Attention Mechanisms in Computer Vision: A Survey", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 15 November 2021 (2021-11-15), XP091099501
Attorney, Agent or Firm:
SHIHUI PARTNERS (CN)
Download PDF: