Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
DATA COLLECTION METHOD AND APPARATUS, DEVICE AND STORAGE MEDIUM
Document Type and Number:
WIPO Patent Application WO/2023/041970
Kind Code:
A1
Abstract:
Provided are a data collection method and apparatus, a device and a storage medium. The method includes that: image data of a picture including a preset scenario is acquired; in the image data, an object in the preset scenario is identified to obtain an identification result; and the image data and the identification result are collected in response to the identification result and/or the image data meeting a preset collection rule.

Inventors:
WU JIACHENG (SG)
LIU ZHIHENG (SG)
ZHANG SHUAI (SG)
Application Number:
PCT/IB2021/058763
Publication Date:
March 23, 2023
Filing Date:
September 26, 2021
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
SENSETIME INT PTE LTD (SG)
International Classes:
G06V20/00; G06V10/00
Foreign References:
CN112818885A2021-05-18
CN112949459A2021-06-11
CN112927258A2021-06-08
CN107590212A2018-01-16
US20170004369A12017-01-05
Download PDF:
Claims:
CLAIMS

1. A data collection method, comprising: acquiring image data of a picture comprising a preset scenario; identifying an object in the preset scenario in the image data to obtain an identification result; and collecting the image data and the identification result in response to at least one of the identification result or the image data meeting a preset collection rule.

2. The method of claim 1, after identifying the object in the preset scenario in the image data to obtain the identification result, further comprising: determining a task requirement corresponding to the image data; and determining the preset collection rule based on the task requirement.

3. The method of claim 2, wherein in a case where the image data is a single frame image, determining the preset collection rule based on the task requirement comprises: determining parameter information associated with the identification result; and determining a first collection rule based on the task requirement and the parameter information, the preset collection rule comprising the first collection rule.

4. The method of claim 3, wherein the parameter information at least comprises one of: confidence, an object type, or a data status; wherein determining the first collection rule based on the task requirement and the parameter information comprises: determining a target parameter in at least one of the confidence, the object type, or the data status based on the task requirement; and determining the preset collection rule based on the target parameter.

5. The method of claim 2 or 3, wherein collecting the image data and the identification result in response to at least one of the identification result or the image data meeting the preset collection rule comprises: determining a value of the target parameter of the identification result; and storing the image data and the identification result in response to the value of the target parameter of the identification result meeting the first collection rule.

6. The method of claim 2, wherein determining the preset collection rule based on the task requirement comprises: determining business information associated with the object in the preset scenario; and determining a second collection rule based on the business information and the task requirement, the preset collection rule comprising the second collection rule.

7. The method of claim 6, wherein in a case where the image data is video data, acquiring the image data of the picture comprising the preset scenario comprises: determining a business stage included in an operation process of the object in the preset scenario; and determining the video data generated for the object in the preset scenario from an initial business stage to a business ending stage.

8. The method of claim 6 or 7, wherein collecting the image data and the identification result in response to at least one of the identification result or the image data meeting the preset collection rule comprises: determining business information of the video data; and storing the video data and the identification result in response to the business information of the video data meeting the second collection rule.

9. The method of claim 8, wherein in a case where the preset scenario is a game scenario, the object in the preset scenario is a game object, the image data is the video data of the game object in any round of game; wherein determining the business information of the video data comprises: determining at least one of the following as the business information: video duration of the video data, types of game objects included in the video data, or alarm information in the video data.

10. The method of any one of claims 1-9, after collecting the image data and the identification result in response to at least one of the identification result or the image data meeting the preset collection rule, further comprising: determining a to-be-trained network for identifying the object in the image data; updating a training data set of the to-be-trained network based on the collected image data and the identification result to obtain production environment data; and adopting the production environment data to train the to-be-trained network to obtain a trained network capable of identifying the object in the image data.

11. A computer device, comprising a memory and a processor, wherein the memory stores a

18 computer executable instruction thereon, and when running the computer executable instruction on the memory, the processor is configured to: acquire image data of a picture comprising a preset scenario; identify an object in the preset scenario in the image data to obtain an identification result; and collect the image data and the identification result in response to at least one of the identification result or the image data meeting a preset collection rule.

12. The computer device of claim 11, wherein after identifying the object in the preset scenario in the image data to obtain the identification result, the processor is further configured to: determine a task requirement corresponding to the image data; and determine the preset collection rule based on the task requirement.

13. The computer device of claim 12, wherein in a case where the image data is a single frame image, when determining the preset collection rule based on the task requirement, the processor is configured to: determine parameter information associated with the identification result; and determine a first collection rule based on the task requirement and the parameter information, the preset collection rule comprising the first collection rule.

14. The computer device of claim 13, wherein the parameter information at least comprises one of: confidence, an object type, or a data status; wherein when determining the first collection rule based on the task requirement and the parameter information, the processor is

19 configured to: determine a target parameter in at least one of the confidence, the object type, or the data status based on the task requirement; and determine the preset collection rule based on the target parameter.

15. The computer device of claim 12 or 13, wherein when collecting the image data and the identification result in response to at least one of the identification result or the image data meeting the preset collection rule, the processor is configured to: determine a value of the target parameter of the identification result; and store the image data and the identification result in response to the value of the target parameter of the identification result meeting the first collection rule.

16. The computer device of claim 12, wherein when determining the preset collection rule based on the task requirement, the processor is configured to: determine business information associated with the object in the preset scenario; and determine a second collection rule based on the business information and the task requirement, the preset collection rule comprising the second collection rule.

17. The computer device of claim 16, wherein in a case where the image data is video data, when acquiring the image data of the picture comprising the preset scenario, the processor is configured to: determine a business stage included in an operation process of the object in the preset scenario; and determine the video data generated for the object in the preset scenario from an initial

20 business stage to a business ending stage.

18. The computer device of claim 16 or 17, wherein when collecting the image data and the identification result in response to at least one of the identification result or the image data meeting the preset collection rule, the processor is configured to: determine business information of the video data; and store the video data and the identification result in response to the business information of the video data meeting the second collection rule.

19. A computer storage medium, storing a computer executable instruction thereon, wherein when executed, the computer executable instruction are configured to: acquire image data of a picture comprising a preset scenario; identify an object in the preset scenario in the image data to obtain an identification result; and collect the image data and the identification result in response to at least one of the identification result or the image data meeting a preset collection rule.

20. A computer program, comprising computer instructions executable by an electronic device, wherein when executed by a processor in the electronic device, the computer instructions are configured to: acquire image data of a picture comprising a preset scenario; identify an object in the preset scenario in the image data to obtain an identification result; and collect the image data and the identification result in response to at least one of the

21 identification result or the image data meeting a preset collection rule.

22

Description:
DATA COLLECTION METHOD AND APPARATUS, DEVICE

AND STORAGE MEDIUM

CROSS-REFERENCE TO RELATED APPLICATION(S)

[ 0001] The application claims priority to Singapore patent application No. 10202110226V filed with IPOS on 16 September 2021, the content of which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

[ 0002] Embodiments of the present application relate to the technical field of image processing, and relate to, but are not limited to, a data collection method and apparatus, a device and a storage medium.

BACKGROUND

[ 0003] In the process of image identification, due to the fact that the data distribution in a production environment is different from that in a test environment, the accuracy of image identification is reduced. In the related art, it is time-consuming to collect supplementary information manually to improve the effect of image identification.

SUMMARY

[ 0004] The embodiments of the present application provide technical solutions for data collection.

[ 0005] The technical solutions of the embodiments of the application are implemented as follows.

[ 0006] The embodiments of the application provide a data collection method, which may include the following operations.

[ 0007] Image data of a picture comprising a preset scenario is acquired.

[ 0008] An object in the preset scenario is identified in the image data to obtain an identification result.

[ 0009] The image data and the identification result are collected in response to the identification result and/or the image data meeting a preset collection rule.

[ 0010] In some embodiments, after the object in the preset scenario is identified in the image data to obtain the identification result, the method may further include: a task requirement corresponding to the image data is determined; and the preset collection rule is determined based on the task requirement. Therefore, by analyzing the task requirement and determining the collection rule of the image data, the user requirement can be better met.

[ 0011] In some embodiments, the operation that in the case where the image data is a single frame image, the preset collection rule is determined based on the task requirement may include: parameter information associated with the identification result is determined; and a first collection rule is determined based on the task requirement and the parameter information. The preset collection rule includes the first collection rule. Therefore, for the single frame image, by combining the parameter information of the image identification result and an application requirement of the image, the collection rule is determined, and the image data that meets the user requirement can be collected selectively and automatically.

[ 0012] In some embodiments, the parameter information may at least include one of: confidence, an object type, or a data status. The operation that the first collection rule is determined based on the task requirement and the parameter information may include: a target parameter is determined in at least one of the confidence, the object type, or the data status based on the task requirement; and the first collection rule is determined based on the target parameter. Therefore, the first collection rule is determined according to the target parameter matched with the task requirement, so that the collected image data can meet the task requirement.

[ 0013] In some embodiments, the operation that the image data and the identification result are collected in response to the identification result and/or the image data meeting a preset collection rule may include: the value of the target parameter of the identification result is determined; and the image data and the identification result are stored in response to the value of the target parameter of the identification result meeting the first collection rule. Therefore, by judging whether the data of the target parameter of the identification result meets the first collection rule, the image meeting the task requirement can be automatically and selectively collected.

[ 0014] In some embodiments, the operation that the preset collection rule is determined based on the task requirement may include: business information associated with the object in the preset scenario is determined; and a second collection rule is determined based on the business information and the task requirement, the preset collection rule comprising the second collection rule. Therefore, according to the task requirement, target business information is determined from the business information, and the second collection rule related to the business information of the object is determined, so that the video data meeting the task requirement can be collected.

[ 0015] In some embodiments, the operation that in the case where the image data is the video data, the image data of the picture including the preset scenario is acquired may include: a business stage included in the operation process of the object in the preset scenario is determined; and the video data generated for the object in the preset scenario from an initial business stage to a business ending stage is determined. Therefore, the operation process of the object is divided into a plurality of stages, and the whole video data from a beginning stage to an ending stage is determined, thus facilitating subsequent more logical selection of video data needing to be collected from the collected video data.

[ 0016] In some embodiments, the operation that the image data and the identification result are collected in response to the identification result and/or the image data meeting the preset collection rule may include: the business information of the video data is determined; and the video data and the identification result are stored in response to the business information of the video data meeting the second collection rule. Therefore, by judging whether the business information in the video data meets the second collection rule, video stream data meeting the task requirement can be automatically and selectively collected.

[ 0017] In some embodiments, in the case where the preset scenario is a game scenario, the object in the preset scenario is a game object, and the image data is the video data of the game object in any round of game. The operation that the business information of the video data is determined may include: at least one of the following is determined as the business information: video duration of the video data, types of game objects included in the video data, or alarm information in the video data. Therefore, in the game scenario, the alarm information or game duration appearing in the game is taken as the business information, so that the determined collection rule can automatically select the video data meeting the business information.

[ 0018] In some embodiments, after the image data and the recognition result are collected in response to the recognition result and/or the image data meeting the preset collection rule, the method may further include: a to-be-trained network for identifying the object in the image data is determined; a training data set of the to-be-trained network is updated based on the collected image data and the identification result to obtain production environment data; and the production environment data is adopted to train the to-be-trained network to obtain a trained network capable of identifying the object in the image data. Therefore, the problem of data inconsistency between the production environment and the test environment can be solved, and a network model is retrained based on the collected image data to achieve a better identification effect.

[ 0019] The embodiments of the application provide a data collection apparatus, which may include a first acquisition module, a first identification module, and a first collection module.

[ 0020] The first acquisition module is configured to acquire image data of a picture including a preset scenario.

[ 0021] The first identification module is configured to identify an object in the preset scenario in the image data to obtain an identification result.

[ 0022] The first collection module is configured to collect the image data and the identification result in response to the identification result and/or the image data meeting a preset collection rule.

[ 0023] Correspondingly, the embodiments of the application provide a computer storage medium. A computer executable instruction is stored thereon. When executed, the computer executable instruction implements steps of the abovementioned method.

[ 0024] The embodiments of the application provide a computer device. The computer device may include a memory and a processor. The memory stores a computer executable instruction thereon. When running the computer executable instruction on the memory, the processor implements steps of the abovementioned method.

[ 0025] The embodiments of the application provide a data collection method and apparatus, a device and a storage medium. An object in image data including a preset scenario is identified; it is judged whether an identification result and the image data meet a preset collection rule; and if the identification result and the image data meet the preset collection rule, the image data and the identification result are automatically stored. Therefore, the image data is selectively collected into a storage device in an automatic collection mode, thereby facilitating subsequent processing after collection.

BRIEF DESCRIPTION OF THE DRAWINGS

[ 0026] FIG. 1 is an implementation flowchart of a data collection method provided by an embodiment of the present application.

[ 0027] FIG. 2 is another implementation flowchart of a data collection method provided by an embodiment of the present application.

[ 0028] FIG. 3 is a structural schematic diagram of a data collection apparatus provided by an embodiment of the present application.

[ 0029] FIG. 4 is a structural schematic diagram of a computer device provided by an embodiment of the present application.

DETAILED DESCRIPTION

[ 0030] To make the objectives, technical solutions, and advantages of the present application clearer, the specific technical solutions of the present application are described below in detail with reference to the accompanying drawings in the embodiments of the present application. The following embodiments are used for illustrating the present application rather than limiting the scope of the present application.

[ 0031] "Some embodiments" involved in the following descriptions describes a subset of all possible embodiments. However, it can be understood that "some embodiments" may be the same subset or different subsets of all the possible embodiments, and may be combined without conflicts.

[ 0032] Terms "first/second/third" involved in the following descriptions are only for distinguishing similar objects and do not represent a specific sequence of the objects. It can be understood that "first/second/third" may be interchanged to specific sequences or orders if allowed to implement the embodiments of the application described herein in sequences except the illustrated or described ones.

[ 0033] Unless otherwise defined, all technological and scientific terms used in the present application have meanings the same as those usually understood by those skilled in the art of the application. The terms used in the present application are only adopted to describe the embodiments of the application and not intended to limit the application.

[ 0034] Before the embodiments of the application are further described in detail, nouns and terms involved in the embodiments of the application will be described. The nouns and terms involved in the embodiments of the application are applied to the following explanations.

[ 0035] 1) Computer Vision is a science for researching how to use a machine for

"looking", and refers to that a camera and a computer are used to replace human eyes to recognize, track and measure an object, and image processing is further performed.

[ 0036] 2) Image identification refers to the technology of processing, analyzing and understanding images by a computer to identify various targets and objects in different modes, which is a practical application of deep learning algorithm. The identification process is divided into four steps: image collection — image preprocessing — feature extraction — image identification.

[ 0037] The following descriptions are made to exemplary applications of a data collection device provided by the embodiments of the disclosure. The device provided by the embodiments of the disclosure may be implemented as various types of user terminals having an image collection function, such as a notebook, a tablet, a desktop computer, a camera and a mobile device (for example, the Personal Data Assistant (PDA), a special messaging device and a portable game device), and may also be implemented as a server. The descriptions are made below to the exemplary application in which the device is implemented as the terminal or the server. [ 0038] The method may be applied to a computer device. Functions implemented by the method may be implemented by enabling a processor in the computer device to call a program code. Certainly, the program code may be stored in a computer storage medium. Hence, the computer device at least includes the processor and the storage medium.

[ 0039] The embodiments of the application provide a data collection method, which may be as shown in FIG. 1, and described in combination with steps shown in FIG. 1.

[ 0040] At S 101 , image data of a picture including a preset scenario is acquired.

[ 0041] In some embodiments, the preset scenario may be a collection scenario specified by a user, such as a scenario where a game place is located, an outdoor scenario (for example, a road with pedestrians) or an indoor scenario (for example, indoor public places such as a shopping mall and a hospital). The image data is a single frame image or video data collected in this scenario. Taking the preset scenario as the game place as an example, the image data may be the image collected in the game place during this process, and may also be the video data collected during the game.

[ 0042] At S102, an object in the preset scenario is identified in the image data to obtain an identification result.

[ 0043] In some embodiments, the object in the image data is identified by using a convolutional neural network to obtain the recognition result. Taking the preset scenario as the game place as an example, the object is an object involved in the game, such as a token, a player, a game manager and a game table. A game object is identified in the image data to obtain the identification result.

[ 0044] At S103, the image data and the identification result are collected in response to the identification result and/or the image data meeting a preset collection rule.

[ 0045] In some embodiments, the preset collection rule may be a rule related to the identification result, a rule related to the image data, and a rule related to both the identification result and the image data. If the collection rule is only related to the identification result, the identification result meets the collection rule. That is, it is determined that the identification result and/or the image data meets the preset collection rule. Similarly, if the collection rule is related to both the identification result and the image data, in a case where the identification result and the image data are both related and meet the collection rule, the identification result and the image data are collected. The game place is taken as the preset scenario, a single frame image as the image data, and that the image includes a preset object (for example, a preset type of token) as the preset collection rule. Then, the identification result of the image data is judged to determine whether the identification result includes the preset type of token. If the identification result includes the preset type of token, the image data and the identification result are stored.

[ 0046] In the embodiments of the application, the object in the image data including the preset scenario is identified; it is judged whether the identification result and the image data meet the preset collection rule; and if the identification result and the image data meet the preset collection rule, the image data and the identification result are automatically stored. Therefore, the image data is selectively collected into a storage device in an automatic collection mode, thereby facilitating subsequent processing after collection.

[ 0047] In some embodiments, the preset collection rule is determined by analyzing the application requirement of the image data. That is, before S102, the following steps are further included, which are as shown in FIG. 2 and described as follows in combination with steps shown in FIG. 2.

[ 0048] At S201, a task requirement corresponding to the image data is determined.

[ 0049] In some embodiments, the task requirement is a requirement for applying the image data. For example, the task requirement includes: an image identification model is trained by using the image data; a certain type of object is obtained by identifying the image data; an abnormal image is found, and so on.

[ 0050] At S202, the preset collection rule is determined based on the task requirement.

[ 0051] In some embodiments, a rule of collecting the image data is determined for meeting the task requirement. For example, if the task requirement is to use the image data to train the image identification model, the determined preset collection rule may be to collect the same image data as the test environment. If the task requirement is to obtain a certain type of object by identifying the image data, the determined preset collection rule may be to collect the image data including the type of object in the identification result. If the task requirement is to find the abnormal image, the determined preset collection rule may be to collect abnormal image data in the identification process. Therefore, by analyzing the task requirement and determining the collection rule of the image data, the user requirement can be better met.

[ 0052] In the embodiments of the application, different collection rules are adopted for collection of the single frame image and the video data respectively. The process of collection of the single frame image is shown in Mode 1, and the process of collection of the video data is shown in Mode 2.

[ 0053] Mode 1: In some embodiments, in a case where the image data is the single frame image, the collection rule is determined to be a rule related to the identification result. That is, S202 may be implemented through the following steps S221 and S222 (not shown in the figure).

[ 0054] In S221, parameter information associated with the identification result is determined.

[ 0055] In a possible implementation mode, in a case where the image data is the single frame image, the object in the single frame image is identified to obtain an identification result. The parameter information associated with the identification result is a parameter involved in the identification result and at least includes one of: confidence, object number, object type, data status, or the like. The confidence is the confidence of the identification result. The object type is the category of the objects. The data status is whether the identification result is a normal identification result instance, for instance, whether there is abnormality in the process of identifying the object of the single frame image.

[ 0056] At S222, a first collection rule is determined based on the task requirement and the parameter information.

[ 0057] In a possible implementation mode, the preset collection rule includes the first collection rule. The task requirement corresponding to the single frame image is combined with the parameter information to determine the first collection rule related to the identification result. Therefore, for the single frame image, by combining the parameter information of the image identification result and an application requirement of the image, the collection rule is determined, and the image data that meets the user requirement can be collected selectively and automatically. [ 0058] In some possible implementation modes, the first collection rule is determined based on the task requirement and at least one of the confidence, the object type or the data status. That is, S222 may be implemented through the following steps.

[ 0059] In a first step, a target parameter is determined in at least one of the confidence, the object type, or the data status based on the task requirement.

[ 0060] In a possible implementation mode, if the task requirement is to obtain certain types of objects by identifying the image data, in the confidence, the object type, and the data status, the target parameter is determined as the object type in the parameter information. Alternatively, the task requirement is to find an abnormal image, the target parameter is the data status.

[ 0061] In a second step, the first collection rule is determined based on the target parameter.

[ 0062] In a possible implementation mode, the first collection rule is determined according to the target parameter. For instance, if the target parameter is the object type, according to a specific object type required in the task requirement, the first collection rule is determined to collect the single frame image including the specific object type in the identification result. If the target parameter is the confidence, according to the requirement for a confidence threshold in the task requirement, the first collection rule is determined to collect the single frame image whose confidence of identification result meets the confidence threshold. Therefore, the first collection rule is determined according to the target parameter matched with the task requirement so that the collected image data can meet the task requirement.

[ 0063] In some embodiments, in the case where the image data is the single frame image, after the first collection rule is determined according to the parameter information of the identification result in combination with the task requirement, it is judged whether the identification result of the single frame image meets the first collection rule so as to determine whether to collect the single frame image. That is, S103 may be implemented through the following steps S131 and S132 (not shown in the figure).

[ 0064] At S131, a value of the target parameter of the identification result is determined.

[ 0065] In a possible implementation mode, the data of each parameter in the identification result is determined. For example, it is determined what the confidence of the identification result is, whether the data status is abnormal, and what the specific type and number of the objects included in the identification result are.

[ 0066] At S132, the image data and the identification result are stored in response to the value of the target parameter of the identification result meeting the first collection rule.

[ 0067] In a possible implementation mode, if the target parameter in the identification result meets the first collection rule, it indicates that the frame image is the image needing to be collected in the task requirement. Hence, the image data and the identification result are stored in a storage system to facilitate subsequent implementation of the task. Therefore, by judging whether the data of the target parameter of the identification result meets the first collection rule, the image meeting the task requirement can be automatically and selectively collected.

[ 0068] Mode 2: In the case where the image data is the video data, a business stage is divided for the object implementation process, and a video in a preset task stage is collected. That is, S101 may be implemented through the following steps Si l l and SI 12 (not shown in the figure).

[ 0069] At Si l l, the business stage included in the operation process of the object in the preset scenario is determined.

[ 0070] In some embodiments, the business stage included in the object in the preset scenario is the business stage included in the whole implementation process of the object in the preset scenario. Taking the preset scenario as the game scenario and the object as the game as an example, the business stage includes each gaming stage from the beginning to the end of a game, such as a game preparation stage, a stage in which a manager joins the game, a stage in which a player joins the game, a gaming stage, a game ending and result outputting stage, etc.

[ 0071] At SI 12, the video data generated for the object in the preset scenario from an initial business stage to a business ending stage is determined.

[ 0072] In some embodiments, the video data includes a video stream of the object in the process from the initial business stage to the business ending stage and other business information generated for the object in the process, such as alarm information in the whole process. Taking the preset scenario as the game scenario and the object as the game as an example, the video data includes the video stream of a game from the game preparation stage to the game ending and result outputting stage and the alarm times and alarm types of the game in a round. Therefore, the operation process of the object is divided into a plurality of stages, and the whole video data from a beginning stage to an ending stage is determined, thus facilitating subsequent more logical selection of video data needing to be collected from the collected video data.

[ 0073] In some embodiments, in the case where the image data is the video, a second collection rule is determined by analyzing the object business information in the preset scenario in combination with the task requirement. That is, the above step S202 may also be implemented through the following steps S21 and S22.

[ 0074] At S21, the business information associated with the object in the preset scenario is determined.

[ 0075] In a possible implementation mode, the business information associated with the object is the business information of the object that can exist in the preset scenario, and the business information is the information describing the business where the object is located. For example, the business information may include whether the alarm information exists in the object. Taking the object as the game as an example, the business information includes the alarm times or alarm type of the game appearing in one round as well as the duration of the game in one round. That is, at least one of the video duration of the video data, the types of game objects included in the video data, or the alarm information in the video data is determined as the business information. Therefore, in the game scenario, the alarm information or game duration appearing in the game is taken as the business information, so that the determined collection rule can automatically select the video data meeting the business information.

[ 0076] At S22, the second collection rule is determined based on the business information and the task requirement.

[ 0077] In a possible implementation mode, the preset collection rule includes the second collection rule. The business information and the task requirement are combined to determine the collection rule meeting the task requirement. In a case where the image data is the video data, target business information matched with the task requirement is determined in the business information according to the task requirement; and then, the second collection rule is determined based on the target business information. For example, if the task requirement is to detect the alarm information of a preset type appearing in the video, the target business information is the alarm information of the preset type, and the second collection rule is to collect the video data including the alarm information of the preset type. Therefore, according to the task requirement, the target business information is determined from the business information, and the second collection rule related to the business information of the object is determined, so that the video data meeting the task requirement can be collected.

[ 0078] In some embodiments, in the case where the image data is the video data, after the second collection rule is determined according to the business information of the object in combination with the task requirement, it is judged whether the video data meets the second collection rule so as to determine whether to collect the video data. That is, S103 may be implemented through the following steps S141 and S142 (not shown in the figure).

[ 0079] At S 141, the business information of the video data is determined.

[ 0080] In a possible implementation mode, the target business information of the video data is determined according to the target business information when the second collection rule is determined. For example, if the target business information when the second collection rule is determined is the alarm information of the preset type, the target business information of the video data is the type of the alarm information.

[ 0081] At S142, the video data and the identification result are stored in response to the business information of the video data meeting the second collection rule.

[ 0082] In a possible implementation mode, if the target business information in the business information of the video data meets the second collection rule, it indicates that the video data is the video data needing to be collected in the task requirement. Hence, the image data and the identification result are stored in the storage system to facilitate subsequent implementation of the task. For example, taking the preset scenario as the game scenario as an example, the video data is the video stream collected for a round of game (for example, the video stream from the beginning to the end of the game) and the alarm information generated during the game. If the second collection rule is to collect the video data including the alarm information of the preset type, it is determined whether to collect the video data by judging whether the type of alarm information in the video data is the preset type.

[ 0083] In the embodiments of the application, by judging whether the business information in the video data meets the second collection rule, the video stream data meeting the task requirement can be automatically and selectively collected.

[ 0084] In some embodiments, after the image data is collected, in order to improve the identification effect of an image identification network, the image data is taken as the training data of a to-be-trained network model, which may be implemented through the following process.

[ 0085] First, a to-be-trained network for identifying the object in the image data is determined.

[ 0086] In a possible implementation mode, the to-be-trained network may be any to- be-trained network for executing image identification, such as a to-be-trained convolutional neural network.

[ 0087] Second, a training data set of the to-be-trained network is updated based on the collected image data and the identification result to obtain production environment data.

[ 0088] In a possible implementation mode, the collected image data and identification result are taken as supplementary training data of the to-be-trained network to add into the training data set to obtain generation environment data.

[ 0089] Finally, the production environment data is adopted to train the to-be-trained network to obtain a trained network capable of identifying the object in the image data.

[ 0090] In a possible implementation mode, the production environment data is taken as the updated training data set to train the to-be-trained network so as to obtain the trained network with better image identification effect. Therefore, the problem of data inconsistency between the production environment and the test environment can be solved, and a network model is retrained based on the collected image data to achieve a better identification effect.

[ 0091] Hereinafter, an exemplary application of the embodiment of the present application in an actual application scenario will be described below. Taking the game place as an example, the descriptions are made to automatic data recovery of an Artificial Intelligence (Al) system of the game place.

[ 0092] Usually, the accuracy of image identification will decrease in real production environment. The reasons are as follows: first, the data distribution of the production environment is different from that of the test environment (for example, a large number of faces wearing masks appear after the outbreak of the epidemic, and the face information accounts for a small proportion of the test data); second, abnormal data in the production environment may cause system errors.

[ 0093] In order to solve the problem of data inconsistency between the production environment and the test environment, in related art, personnel are required to collect sufficient supplementary information (such as images) in the production environment, and retraining is conducted based on the supplementary information to achieve a better identification effect. Therefore, it is time-consuming and laborious to collect data again after the production environment changes.

[ 0094] The embodiments of the application provide a data collection method. Production environment data is selectively collected into a certain storage device in an automatic collection mode, thereby facilitating subsequent processing after collection. In some embodiments, a data collection module is added to an edge Al device (this module integrates the processing result, time, device configuration parameters, service warnings and other related information of the Al system), and it is judged whether this data should be recycled according to the determined collection rule, and the data needing to be recycled is automatically written to a designated storage device. In the embodiments of the application, data collection may be executed based on an image or a video.

[ 0095] There are image-based data collection process steps.

[ 0096] In a first step, the identification result of each image is collected.

[ 0097] In a second step, a determined data collection rule in the configuration is acquired.

[ 0098] In a third step, it is judged whether the images and a processing result are collected into a storage system according to the data collection rule. [ 0099] In a possible implementation mode, the data collection rule includes: less than how many the number of tokens is in the processing result, whether there are errors in the confidence of the processing result or in the result, etc.

[ 00100] There are video-based data collection process steps.

[ 00101] In a first step, according to the gaming stage fed back by a business layer, the relevant data in the game is divided and the game data is recovered by taking the game division as a unit. Different games are stored in their own folders.

[ 00102] In a second step, the recovered data includes the video stream, the identification result and the alarm information generated during program execution.

[ 00103] In a third step, the collection rules of various types of data are determined through a configuration file distributed by the cloud. For example, all game videos with game time less than 30 seconds are collected, or the game videos with more than 3 warnings in the game are collected.

[ 00104] In a fourth step, it is judged whether the game videos are automatically collected into the storage system according to the collection rules.

[ 00105] After the data is collected in the above two modes, the collected data may be applied in the following business service directions, including the following operations.

[ 00106] First, the stored data is used by a test team to reproduce the problems in the production environment.

[ 00107] Second, the stored data is used for model training.

[ 00108] Third, the stored data serves as a test data set of another new service.

[ 00109] In the embodiments of the application, an edge device acquires the data collection rule, automatically and selectively collects the data according to the rule, groups videos in accordance with the game division according to the information fed back by a business layer, and stores the game round information. Thus, the production environment data is automatically collected, effective information is selectively collected according to the data collection rule, and the game videos are stored in groups according to the feedback information of the business layer, so that the data can be managed more conveniently.

[ 00110] The embodiments of the application provide a data collection apparatus. FIG. 3 is a structural schematic diagram of a data collection apparatus provided by an embodiment of the present application. As shown in FIG. 3, the data collection apparatus 300 may include a first acquisition module 301, a first identification module 302, and a first collection module 303.

[ 00111] The first acquisition module 301 is configured to acquire image data of a picture including a preset scenario.

[ 00112] The first identification module 302 is configured to identify an object in the preset scenario in the image data to obtain an identification result.

[ 00113] The first collection module 303 is configured to collect the image data and the identification result in response to the identification result and/or the image data meeting a preset collection rule.

[ 00114] In some embodiments, the apparatus may further include a first determination module and a second determination module.

[ 00115] The first determination module is configured to determine a task requirement corresponding to the image data. [ 00116] The second determination module is configured to determine the preset collection rule based on the task requirement.

[ 00117] In some embodiments, in a case where the image data is a single frame image, the second determination module may include a first determination submodule and a second determination submodule.

[ 00118] The first determination submodule is configured to determine parameter information associated with the identification result.

[ 00119] The second determination submodule is configured to determine a first collection rule based on the task requirement and the parameter information. The preset collection rule includes the first collection rule.

[ 00120] In some embodiments, the parameter information may at least include one of: confidence, an object type, or a data status. The second determination submodule includes a first determination unit and a second determination unit.

[ 00121] The first determination unit is configured to determine a target parameter in at least one of the confidence, the object type, or the data status based on the task requirement.

[ 00122] The second determination unit is configured to determine the first collection rule based on the target parameter.

[ 00123] In some embodiments, the first collection module 303 may include a third determination submodule and a first storage submodule.

[ 00124] The third determination submodule is configured to determine a value of the target parameter of the identification result.

[ 00125] The first storage submodule is configured to store the image data and the identification result in response to the value of the target parameter of the identification result meeting the first collection rule.

[ 00126] In some embodiments, the second determination module may include a fourth determination submodule and a fifth determination submodule.

[ 00127] The fourth determination submodule is configured to determine business information associated with the object in the preset scenario.

[ 00128] The fifth determination submodule is configured to determine a second collection rule based on the business information and the task requirement. The preset collection rule includes the second collection rule.

[ 00129] In some embodiments, in a case where the image data is video data, the first acquisition module 301 may include a sixth determination submodule and a seventh determination submodule.

[ 00130] The sixth determination submodule is configured to determine a business stage included in the operation process of the object in the preset scenario.

[ 00131] The seventh determination module is configured to determine the video data generated for the object in the preset scenario from an initial business stage to a business ending stage.

[ 00132] In some embodiments, the first collection module 303 may include an eighth determination submodule and a second storage submodule.

[ 00133] The eighth determination submodule is configured to determine the business information of the video data.

[ 00134] The second storage submodule is configured to store the video data and the identification result in response to the business information of the video data meeting the second collection rule.

[ 00135] In some embodiments, in a case where the preset scenario is a game scenario, the object in the preset scenario is a game object, and the image data is the video data of the game object in any round of game. The eighth determination submodule may include a third determination unit.

[ 00136] The third determination unit is configured to determine at least one of the following as the business information: video duration of the video data, types of game objects included in the video data, or alarm information in the video data.

[ 00137] In some embodiments, the apparatus may further include a third determination module, a first updating module, and a first training module.

[ 00138] The third determination module is configured to determine a to-be-trained network for identifying the object in the image data.

[ 00139] The first updating module is configured to update a training data set of the to- be-trained network based on the collected image data and the identification result to obtain production environment data.

[ 00140] The first training module is configured to adopt the production environment data to train the to-be-trained network to obtain a trained network capable of identifying the object in the image data.

[ 00141] It is to be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. Technical details undisclosed in the device embodiment of the application are understood with reference to the descriptions about the method embodiment of the application.

[ 00142] It is to be noted that, in the embodiment of the present application, when being implemented in the form of a software function module and sold or used as an independent product, the data collection method may also be stored in a computer- readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present application substantially or parts making contributions to the conventional art may be embodied in the form of software product, and the computer software product is stored in a storage medium, including a plurality of instructions configured to enable a piece of computer equipment (which may be a terminal, a sever and the like) to execute all or part of the method in each embodiment of the present application. The storage medium includes: various media capable of storing program codes such as a U disk, a mobile hard disk, a Read Only Memory (ROM), a magnetic disk or an optical disk. By doing so, the embodiments of the present application are not limited to any specific combination of hardware and software.

[ 00143] Correspondingly, the embodiments of the present application provide a computer program product, which may include a computer executable instruction. When executed, the computer executable instruction can implement steps of the data collection method provided by the embodiments of the present application.

[ 00144] Correspondingly, the embodiments of the present application further provide a computer storage medium, which stores a computer executable instruction thereon; and when executed by a processor, the computer executable instruction implements steps of the data collection method provided by the embodiments of the present application.

[ 00145] Correspondingly, the embodiments of the present application provide a computer device. FIG. 4 is a structural schematic diagram of a computer device according to an embodiment of the present application. As shown in FIG. 4, the computer device 400 may include: a processor 401, at least one communication bus, a communication interface 402, at least one external communication interface and a memory 403. The communication bus 402 is configured to implement connection and communication among these components. The communication bus 402 may include a display screen, and the external communication interface may include a standard wired interface and a wireless interface. The processor 401 is configured to execute an image processing program in the memory, to implement steps of the data collection method provided by the above embodiment.

[ 00146] The descriptions on the embodiments of the data collection device, computer device and storage medium are similar to those of the above method embodiment, so the technical descriptions and beneficial effects are the same to the corresponding method embodiment, may refer to the disclosures of the method embodiment for the ease of simplicity and will not be repeated herein. A technical detail not disclosed in the embodiments of the data collection apparatus, computer device and storage medium may be understood with reference to the descriptions on the method embodiment of the present application.

[ 00147] It is to be understood that reference throughout this specification to “one embodiment” or “an embodiment” means that particular features, structures, or characteristics described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It is further to be understood that the sequence numbers of the foregoing processes do not mean execution sequences in various embodiments of the present application. The execution sequences of the processes should be determined according to functions and internal logics of the processes, and should not be construed as any limitation to the implementation processes of the embodiments of the present application. The serial numbers of the embodiments of the application are merely for description and do not represent a preference of the embodiments. It is to be noted that the terms "include", "contain" or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or equipment including a series of elements not only includes those elements, but also includes those elements that are not explicitly listed, or includes elements inherent to such a process, method, article or device. Under the condition of no more limitations, it is not excluded that additional identical elements further exist in the process, method, article or device including elements defined by a sentence "including a . . .

[ 00148] In the several embodiments provided in the application, it should be understood that the disclosed device and method may be implemented in other manners. The device embodiment described above is only schematic, and for example, division of the units is only logic function division, and other division manners may be adopted during practical implementation. For example, multiple units or components may be combined or integrated into another system, or some characteristics may be neglected or not executed. In addition, coupling or direct coupling or communication connection between each displayed or discussed component may be indirect coupling or communication connection, implemented through some interfaces, of the device or the units, and may be electrical and mechanical or adopt other forms.

[ 00149] The units described as separate parts may or may not be physically separated, and parts displayed as units may or may not be physical units, and namely may be located in the same place, or may also be distributed to multiple network units. Part or all of the units may be selected to achieve the purpose of the solutions of the embodiments according to a practical requirement.

[ 00150] In addition, each function unit in each embodiment of the application may be integrated into a processing unit, each unit may also exist independently, and two or more than two unit may also be integrated into a unit. The integrated unit may be implemented in a hardware form, and may also be implemented in form of hardware and software function unit. Those of ordinary skill in the art should know that: all or part of the steps of the abovementioned method embodiment may be implemented by instructing related hardware through a program, the abovementioned program may be stored in a computer-readable storage medium, and the program is executed to execute the steps of the abovementioned method embodiment; and the storage medium includes: various media capable of storing program codes such as mobile storage equipment, an ROM, a Random Access Memory (RAM), a magnetic disk or an optical disc.

[ 00151] Or, when being implemented in form of software function module and sold or used as an independent product, the integrated unit of the present application may also be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the application substantially or parts making contributions to the conventional art may be embodied in form of software product, and the computer software product is stored in a storage medium, including a plurality of instructions configured to enable a piece of computer equipment (which may be a personal computer, a server, network equipment or the like) to execute all or part of the method in each embodiment of the present application. The abovementioned storage medium includes: various media capable of storing program codes such as mobile storage equipment, a ROM, a RAM, a magnetic disk or an optical disc. The above is only the specific implementation mode of the present application and not intended to limit the scope of protection of the present application. Any variations or replacements apparent to those skilled in the art within the technical scope disclosed by the application shall fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subjected to the scope of protection of the claims.