Title:
WEBSITE DUPLICATE REMOVING METHOD, ELECTRONIC DEVICE AND COMPUTER READABLE STORAGE MEDIUM
Document Type and Number:
WIPO Patent Application WO/2019/071896
Kind Code:
A1
Abstract:
A website duplicate removing method. The method comprises steps of sequentially reading a website to be processed (e.g., a URL address), and searching an improved generalized list for the website to be processed (S31); if the website to be processed is not found in the improved generalized list, interposing the website to be processed into the improved generalized list, and storing the website to be processed in a queue to be captured (S32); and if the website to be processed is found in the improved generalized list, stopping storing the website to be processed in the queue to be captured (S33). By means of the method, the website duplicate removing efficiency can be improved.
Inventors:
LI FANG (CN)
WANG JIANMING (CN)
XIAO JING (CN)
WANG JIANMING (CN)
XIAO JING (CN)
Application Number:
PCT/CN2018/076170
Publication Date:
April 18, 2019
Filing Date:
February 10, 2018
Export Citation:
Assignee:
PING AN TECH SHENZHEN CO LTD (CN)
International Classes:
G06F17/30
Foreign References:
| CN101620608A | 2010-01-06 | |||
| CN101458740A | 2009-06-17 | |||
| CN102184227A | 2011-09-14 | |||
| US20120023112A1 | 2012-01-26 |
Other References:
WU, XIAOHUI: "Improvement on Unrepeated Tactics of URL of Distributed Spider", JOURNAL OF PINGDINGSHAN UNIVERSITY, vol. 24, no. 5, 31 October 2009 (2009-10-31), pages 116, XP055591924, ISSN: 1673-1670
Attorney, Agent or Firm:
SHENZHEN WORLD INTELLECTUAL PROPERTY AGENCY (GENERAL PARTNERSHIP) (CN)
Download PDF: