TECHNICAL FIELD

[0001] The present invention generally relates to a method of tracking an object and a system thereof.

BACKGROUND

[0002] Object tracking with multiple cameras is a challenging task because one cannot know exactly in which camera the target object will be captured. One naive approach is to always check all cameras to determine whether the target object is captured by one or more of the cameras. This strategy can prevent losing tracking of the target object. However, it also introduces huge computation cost because the system needs to continually check every frame of all cameras. In order to reduce computation cost, one conventional technique utilizes camera link information, i.e., the camera region appearance probability matrix, as described in U.S. Patent No. 8,284,255 entitled “Inter-camera ink relation information generating apparatus” filed on 29 Feb. 2008, which can be learned automatically by conducting experiments. In another conventional technique, U.S. Patent No. 7,006,950 entitled “Statistical modeling and performance characterization of a real-time dual camera surveillance system” filed on 12 June 2000 describes statistical models being used to estimate the location of target object and send PTZ control command to adjust the camera for ease of tracking. As described, conventional methods utilize models to predict tracking cameras. In order to build the model, typically experiments are conducted to train the model parameters (training stage). Then, the built model is used to perform the prediction task (test stage). However, the trained model may not give accurate predictions when the object is moving fast or the target area is quite large. Furthermore, these approaches have the drawback that if the movement of target object during the training stage is different from that of test stage, the trained model cannot give good predictions. This may happen if the monitored area of cameras is large and there are enormous possible paths in the area. In such case, one may lose the tracking of the object because of inaccurate predictions.

[0003] A need therefore exists to provide a method and system of object tracking that seek to overcome, or at least ameliorate, one or more of the deficiencies in conventional object tracking method s/systems. It is against this background that the present invention has been developed.

SUMMARY

[0004] According to a first aspect of the present invention, there is provided a method of tracking an object using at least one processor, the method comprising; selecting one or more tracking cameras from a plurality of cameras based on a tracking camera prediction model to form an initial set of one or more tracking cameras; performing reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras, comprising selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras; obtaining, from each tracking camera of the new set of tracking cameras, video data; and detecting, for said each tracking camera of the new set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object. [0005] According to a second aspect of the present invention, there is provided a method of tracking an object, the method comprising: obtaining, from each tracking camera of an initial set of one or more tracking cameras, video data; determining, for said each tracking camera of the initial set of one or more tracking cameras, a status of the tracking camera based on tracking data of the object obtained from the video data obtained from the tracking camera; determining whether a predefined condition is met based on the status of said each tracking camera of the initial set of one or more tracking cameras; in response to the predefined condition being met, performing reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras, comprising selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras; obtaining, from each tracking camera of the new set of tracking cameras, updated video data; and detecting, for said each tracking camera of the new set of tracking cameras, the object in the updated video data obtained from the tracking camera so as to track the object.

[0006] According to a third aspect of the present invention, there is provided a system for tracking an object, the system comprising: a memory; and at least one processor communicatively coupled to the memory and configured to: select one or more tracking cameras from a plurality of cameras based on a tracking camera prediction model to form an initial set of one or more tracking cameras; perform reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras, comprising selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras; obtain, from each tracking camera of the new set of tracking cameras, video data; and detect, for said each tracking camera of the new set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object.

[0007] According to a fourth aspect of the present invention, there is provided a computer program product, embodied in one or more non-transitory computer-readable storage mediums, comprising instructions executable by at least one processor to perform a method of tracking an object, the method comprising: selecting one or more tracking cameras from a plurality of cameras based on a tracking camera prediction model to form an initial set of one or more tracking cameras; performing reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras, comprising selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras; obtaining, from each tracking camera of the new set of tracking cameras, video data; and detecting, for said each tracking camera of the new set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object. a BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments of the present invention will be beter understood and readily apparent to one of ordinary skill in the art from the following written description, by way of example only, and in conjunction with the drawings, in which:

FIG. 1 depicts a diagram illustrating an example scenario of a trajectory of the car during training stage (dashed line) and testing stage;

FIGS. 2A-2B depict schematic flow diagrams of a method (computer- implemented method) of tracking an object using at least one processor according to various embodiments of the present invention:

FIGS. 3A-3B depicts a schematic block diagram of a system for tracking an object according to various embodiments of the present invention, such as corresponding to the method shown in FIGS. 2A-2B, respectively;

FIG. 4 depicts an example computer system which the system according to various embodiments of the present invention may be embodied in;

FIG. 5 illustrates an exemplary process flow of tracking the object according to various example embodiments of the present invention;

FIGS. 6A-6B show two examples of the camera selection model (or tracking camera prediction model), respectively, according to various example embodiments of the present invention;

FIG. 7 depicts an exemplary schematic diagram of a method for selecting one or more tracking cameras from a plurality of cameras based on a tracking camera prediction model according to various example embodiments of the present invention;

FIG. 8 shows another example method for combining the candidate cameras selected based on multiple camera selection models according to various example embodiments of the present invention;

FIG. 9 depicts an exemplary schematic diagram of a method for selecting one or more tracking cameras from candidate cameras by image feature matching, according to various example embodiments of the present invention;

FIG. 10 shows an example method of determining or updating the status of each tracking camera, and determining or evaluating the conditions for camera reselection so as to trigger camera reselection, according to various example embodiments of the present invention;

FIG. 11 shows an example of camera group information according to various example embodiments of the present invention; FIG. 12 shows a process flow of reselecting tracking cameras based on camera group information according to various example embodiments of the present invention;

FIGS. 13A-13B illustrate schematics for updating the camera region pair probability table and the position-camera pair probability table, respectively, according to various example embodiments of the present invention;

FIG. 14 illustrates another exemplary process flow of hacking the object according to various example embodiments of the present invention;

FIG. 15 illustrate another exemplary process flow' of tracking the object according to various example embodiments of the present invention; and

FIG. 16 depicts a simplified system according to various example embodiments of the present invention.

DETAILED DESCRIPTION

[0009] FIG. 1 depicts a diagram 100 illustrating an example scenario of a trajectory of an object, such as a car in a non-limiting example, during training stage (dashed line) and testing stage (solid line). During the training stage, the car moves from location LI to L2, L3, then turns back to a location which is not shown in the figure and subsequently moves to L4 and L5. In the test stage, the car moves as LI ---->L2---->L3— ->L4— ->L5. This indicates that the movement of the car is dynamic and difficult to predict accurately. This is especially so when the object is moving fast, and therefore it is difficult to predict its trajectory. Various embodiments of the present invention provide a method (computer-implemented method) of tracking an object and a system (including a memory and at least one processor communicatively coupled to the memory) thereof. The method of tracking the object according to various embodiments may facilitate a more accurate prediction and/or tracking of the object.

[0010] FIG. 2 A depicts a schematic flow diagram of a method 200a (computer- implemented method) of tracking an object (or target object) using at least one processor according to various embodiments of the present invention. The method 200a comprises selecting (at 202) one or more tracking cameras from a plurality of cameras based on a tracking camera prediction (or selection) model to form an initial set (or current set) of one or more tracking cameras; performing (at 204) reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras, comprising selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras; obtaining (at 206). from each tracking camera of the new set of tracking cameras, video data; and detecting (at 208), for said each tracking camera of the new set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object.

[0011] In relation to 202, for example, the plurality of cameras may be disposed at different geographical locations of an area or region desired to be monitored or covered. In various embodiments, each camera of the plurality of cameras may produce video data (e.g., video frames, image frames) of the object and at least one processor may then obtain (or receive) the video data for processing to detect (including identify) the object (e.g., including identify whether a detected object corresponds to a target object). In other embodiments, the video data may be stored in a database and at least one processor may then obtain (or receive) the video data for processing to track the object. [0012] In various embodiments, each tracking camera may relate to a camera in which the object is visible (e.g., showing) or captured.

[0013] Each tracking camera of the initial set of one or more tracking cameras may be a current tracking camera. In relation to 204, the above-mentioned performing reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras may relate to reselecting tracking cameras based on camera group information associated with each of the current tracking camera.

[0014] The camera group information describes the grouping details of each camera. In various embodiments, the plurality of cameras may be categorized into different groups according to properties of each camera to obtain the camera group information. For example, cameras which are near or in close proximity to each other may be categorized as one group. Accordingly, the cameras corresponding to a same group may be co -related to one another, for example, according to the properties of the cameras such as a geographical location of each of the cameras. In various embodiments, one camera may belong to multiple groups.

[0015] In relation to 208, the above-mentioned detecting, for said each tracking camera of the new set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object may relate to detecting a location of the object or movement of the object based on the location of the object.

[0016] As described, various embodiments provide a method of tracking an object which includes reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set (or current set) of one or more tracking cameras. The reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set (or current set) of one or more tracking cameras prevents or minimizes the problem of losing tracking of the object by the initial set of one or more tracking cameras when the object moves in a monitored area, especially in the case where the object is moving fast and it is challenging to predict the trajectory of the object. Accordingly, various embodiments of the invention improve the prediction and accuracy of the tracking process of the object.

[0017] In various embodiments, the above-mentioned performing reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras comprises determining, for said each tracking camera of the initial set of one or more tracking cameras, each camera corresponding to a group associated to the tracking camera as a candidate group-related camera; and said selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras comprises filtering the new tracking cameras from the candidate group -related cameras.

[0018] The tracking camera prediction model may define a mapping relationship from a detected or determined location of the object to candidate cameras in which the object is likely to be visible or captured, or region of camera views corresponding to one or more of the candidate cameras. The location of the object may be a point or region in a camera view (e.g., two-dimensional (2D) space) of a corresponding camera, or a position such as place or area (i.e., geographical location) in the real world (e.g., three-dimensional (3D) space). For example, the location of the object may first be detected using an object detection algorithm run on the camera view of a corresponding camera to automatically detect the object’s location, manually select the location on the camera view of the corresponding camera, or parse sensor signals sent from the target object. For example, when the object location (either in camera view or real world) is provided, the tracking camera prediction model may be used to obtain a set of candidate cameras which may capture the object. Video data from the candidate cameras needs to be processed to determine whether the object is captured, and the one or more tracking cameras may then be selected from the candidate cameras. The object may be detected in the video data may, for example, by image feature matching. [0019] In various embodiments, the tracking camera prediction model comprises a dataset in relation to a probability of the object being captured in a pair of region of camera views (with or without some time difference) corresponding to one or more of the plurality of cameras. For example, the dataset in relation to the probability of the object being captured in a pair of region of camera views corresponding to one or more of the plurality of cameras shows the likelihood that the target object will show (or be captured) in a pair of region of camera views corresponding to one or more of the plurality of cameras. Each region of a camera view in the pair of region of camera views may correspond to the same camera or different cameras. For example, the dataset in relation to a probability of the object being captured in a pair of region of camera views corresponding to one or more of the plurality of cameras may be arranged in the form of a table, which may be referred to herein as a camera region pair probability table.

[0020] In various embodiments, the tracking camera prediction model comprises a dataset in relation to a probability of the object being captured in a region of a camera view corresponding to each of the plurality of cameras (or camera, in the case an entire camera view is taken as one region) based on a geographical location of the object. For example, the dataset in relation to the probability of the object being captured in a region of a camera view corresponding to each of the plurality of cameras based on a geographical location of the object shows the likelihood that the target object will show (or be captured) in each camera if it is found in a predefined position in the real world (geographical location). For example, the dataset in relation to a probability of the object being captured in a region of a camera view corresponding to each of the plurality of cameras based on a geographical location of the object may be arranged in the form of a table, which may be referred to herein as a position-camera pair probability table. [0021] For example, the tracking camera prediction model may be trained by conducting experiments.

[0022] In various embodiments, the above-mentioned selecting one or more tracking cameras from a plurality of cameras based on a tracking camera prediction model comprises selecting one or more candidate cameras based on said dataset in relation to a probability of the object being captured in a pair of region of camera views corresponding to one or more of the plurality of cameras to obtain a first set of candidate cameras: selecting one or more candidate cameras based on said dataset in relation to a probability of the object being captured in a region of a camera view corresponding to each of the plurality of cameras based on a geographical location of the object to obtain a second set of candidate cameras; combining the first set of candidate cameras and the second set of candidate cameras to form a combined set of candidate cameras; and filtering the one or more tracking cameras from the combined set of candidate cameras. For example, the one or more tracking cameras in which the target object is captured may be filtered from the combined set of candidate cameras by image feature matching to identify the object.

[0023] In various embodiments, the method 200a may further comprise updating the tracking camera prediction model based on the one or more new tracking cameras selected during the reselection of tracking cameras. This will be helpful if the movement of the object is quite different from those determined in the experiments during the training stage. In this case, by updating the latest camera selection results to the tracking camera prediction model, various embodiments advantageously provide better selection of cameras in subsequent predictions.

[0024] In various embodiments, the method 200a may further comprise determining, for said each tracking camera of the initial set of one or more tracking cameras, a status of the tracking camera. The status of the tracking camera may be determined based on tracking history (or tracking data) of the object obtained from the video data obtained from the tracking camera. The tracking history may refer to identified positions of the object in the video data (e.g., previous video frames). The tracking history may be object-camera wise. If one object shows in multiple cameras, then the object will have corresponding multiple fracking histories. Further, the above- mentioned performing reselection of fracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras comprises determining whether a predefined condition is met based on the status of said each tracking camera of the initial set of one or more tracking cameras, and performing the above-mentioned reselection of tracking cameras (based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras) in response to the predefined condition being met. The predefined condition may be set or specified by a user.

[0025] Accordingly, when a user specified condition(s) is met, for each current tracking camera, all cameras corresponding to a group associated to the current tracking camera may be checked to see whether the target object is show'ing. In addition, instead of training the tracking camera prediction model beforehand and keeping it static during the test stage (the static pre-trained tracking camera prediction model may not give good selection of cameras when the movement of the object is hard to predict), various embodiments of the invention update the prediction model dynamically by reflecting the latest camera reselection results. In this way, even when the movement of target object is hard to predict, the prediction model according to various embodiments can handle these changes dynamically and prevent the problem of losing tracking.

[0026] The status of each tracking camera may be continuously monitored. In various embodiments, the above-mentioned determining, for said each tracking camera of the initial set of one or more tracking cameras, a status of the tracking camera comprises determining the tracking camera losing status as a tracking camera based on predicting a remaining time for the object to move out of a field of view of the tracking camera, object size in camera view or moving speed.

[0027] In various embodiments, the predefined condition comprises a threshold total quantity of tracking cameras or a threshold quantity of tracking cameras losing status as a tracking camera. For example, when a predefined condition of camera reselection is triggered, e.g., the quantity of tracking cameras drops below a threshold, reselection of tracking cameras may be performed based on the camera group information.

[0028] In various embodiments, the object comprises a vehicle or a person. The vehicle may be a car, a train, an airplane, in various non-limiting examples. It is understood that the present invention is not limited to the above-mentioned objects.

[0029] FIG. 2B depicts another schematic flow diagram of a method 200b (computer-implemented method) of tracking an object (or target object) using at least one processor according to various embodiments of the present invention. The method 200b comprises obtaining (at 252), from each tracking camera of an initial set of one or more tracking cameras, video data; determining (at 254), for said each tracking camera of the initial set of one or more tracking cameras, a status of the tracking camera based on tracking data of the object obtained from the video data obtained from the tracking camera; determining (at 256), whether a predefined condition is met based on the status of said each tracking camera of the initial set of one or more tracking cameras; in response to the predefined condition being met, performing (at 258), reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras, comprising selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras; obtaining (at 260), from each tracking camera of the new set of tracking cameras, updated video data; and detecting (at 262), for said each tracking camera of the new set of tracking cameras, the object in the updated video data obtained from the tracking camera so as to track the object.

[0030] In various embodiments, the above-mentioned determining, for said each hacking camera of the initial set of one or more tracking cameras, a status of the tracking camera comprises determining the tracking camera losing status as a tracking camera based on predicting a remaining time for the object to move out of a field of view of the tracking camera, object size in camera view or moving speed.

[0031] In various embodiments, the predefined condition comprises a threshold total quantity of tracking cameras or a threshold quantity of tracking cameras losing status as a tracking camera.

[0032] According to various embodiments, the present invention may be applied, for example, in in POI/VOI (person/vehicle of interest) function in solutions using video analysis technology. For example, the POI/VOI tracking is to track the movement of target person/vehicle in a monitored area, like a shopping mall, a car park, etc. The area may be monitored by multiple surveillance cameras which may or may not have overlapping FoV (field of view). By applying various embodiments of the invention, the camera group information and/or tracking camera prediction model (e.g., camera region pair probability table, position-camera pair probability table) may be utilized to efficiently select tracking cameras while preventing the possible missing camera problem. Various embodiments of the invention may be used in smart city solutions using video analysis technology in a non-limiting example, which may improve the value of such solutions with accurate object tracking results.

[0033] FIG. 3A depicts a schematic block diagram of a system 300a for tracking an object according to various embodiments of the present invention, such as corresponding to the method 200a of tracking an object as described hereinbefore according to various embodiments of the present invention.

[0034] The system 300a comprises a memory 304, and at least one processor 306 communicatively coupled to the memory 304 and configured to: select one or more hacking cameras from a plurality of cameras based on a tracking camera prediction model to form an initial set of one or more tracking cameras; perform reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras, comprising selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras: obtain, from each tracking camera of the new set of tracking cameras, video data; and detect, for said each tracking camera of the new set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object.

[0035] It will be appreciated by a person skilled in the art that the at least one processor 306 may be configured to perform the required functions or operations through set(s) of instructions (e.g., software modules) executable by the at least one processor 306 to perform the required functions or operations. Accordingly, as shown in FIG. 3A, the system 300a may further comprise a tracking camera selection module (or circuit) 308 configured to select one or more tracking cameras from a plurality of cameras based on a tracking camera prediction model to form an initial set of one or more tracking cameras; a tracking camera reselection module (or circuit) 310 configured to perform reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras; a video obtaining module (or circuit) 312 configured to obtain, from each tracking camera of the new set of tracking cameras, video data; and an object detection (which may be interchangeably referred to as object tracking) module (or circuit) 314 configured to detect, for said each tracking camera of the new set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object. According to various embodiments, in relation to detecting the object in the video data obtained from the tracking camera so as to track the object, the object detection module 314 may detect object location in the camera view and identify the detected object (e.g., identify whether a detected object corresponds to a target object).

[0036] FIG. 3B depicts another schematic block diagram of a system 300b for tracking an object according to various embodiments of the present invention, such as corresponding to the method 200b of tracking an object as described hereinbefore according to various embodiments of the present invention.

[0037] The system 300b comprises a memory 304, and at least one processor 306 communicatively coupled to the memory 304 and configured to: obtain, from each tracking camera of an initial set of one or more tracking cameras, video data; determine, for said each tracking camera of the initial set of one or more tracking cameras, a status of the tracking camera based on tracking data of the object obtained from the video data obtained from the tracking camera; determine whether a predefined condition is met based on the status of said each tracking camera of the initial set of one or more tracking cameras: in response to the predefined condition being met, perform reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new set of tracking cameras, comprising selecting one or more cameras to include as one or more new tracking cameras in the new set of tracking cameras; obtain, from each tracking camera of the new set of tracking cameras, updated video data; and detect, for said each tracking camera of the new set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object.

[0038] As show'll in FIG. 3B, the system 300b may further comprise a video obtaining module (or circuit) 312 configured to obtain, from each tracking camera of an initial set of one or more tracking cameras, video data, and obtain, from each tracking camera of the new' set of tracking cameras, updated video data; a tracking camera status determination module (or circuit) 316 configured to determine, for said each tracking camera of the initial set of one or more tracking cameras, a status of the tracking camera based on tracking data of the object obtained from the video data obtained from the tracking camera: a tracking camera reselection module (or circuit) 310 configured to determine whether a predefined condition is met based on the status of said each tracking camera of the initial set of one or more tracking cameras, and in response to the predefined condition being met, perform reselection of tracking cameras based on camera group information associated with each tracking camera of the initial set of one or more tracking cameras so as to form a new' set of tracking cameras; and an object detection module (or circuit) 314 configured to detect, for said each tracking camera of the new' set of tracking cameras, the object in the video data obtained from the tracking camera so as to track the object.

[0039] It will be appreciated by a person skilled in the art that the above-mentioned modules (or circuits) are not necessarily separate modules, and two or more modules may be realized by or implemented as one functional module (e.g., a circuit or a software program) as desired or as appropriate without deviating from the scope of the present invention. For example, the tracking camera selection module 308, the tracking camera reselection module 310, the video obtaining module 312, the object detection module 314 and/or the tracking camera status determination module 316 may be realized (e.g., compiled together) as one executable software program (e.g., software application or simply referred to as an “app”), which for example may be stored in the memory 304 and executable by the at least one processor 306 to perform the functions/operations as described herein according to various embodiments.

[0040] In various embodiments, the system 300a corresponds to the method 200a as described hereinbefore with reference to FIG. 2A, therefore, various functions/operations configured to be performed by the least one processor 306 may correspond to various steps or operations of the method 200a described hereinbefore according to various embodiments, and thus need not be repeated with respect to the system 300a for clarity and conciseness. Similarly, the system 300b corresponds to the method 200b as described hereinbefore with reference to FIG. 2B, therefore, various functions/operations configured to be performed by the least one processor 306 may correspond to various steps or operations of the method 200b described hereinbefore according to various embodiments, and thus need not be repeated with respect to the system 300b for clarity and conciseness. In other words, various embodiments described herein in context of the methods are analogously valid for the respective systems (e.g., which may also be embodied as devices).

[0041] For example, in various embodiments, the memory 304 may have stored therein the tracking camera selection module 308, the tracking camera reselection module 310, the video obtaining module 312, the object detection module 314 and/or the tracking camera status determination module 316, which respectively correspond to various steps or operations of the method 200a and/or the method 200b as described hereinbefore, which are executable by the at least one processor 306 to perform the corresponding functions/operations as described herein.

[0042] A computing system, a controller, a microcontroller or any other system providing a processing capability may be provided according to various embodiments in the present disclosure. Such a system may be taken to include one or more processors and one or more computer-readable storage mediums. For example, the system 300a and/or system 300b described hereinbefore may include a processor (or controller) 306 and a computer-readable storage medium (or memory) 304 which are for example used in various processing carried out therein as described herein. A memory or computer- readable storage medium used in various embodiments may be a volatile memory, for example a DRAM (Dynamic Random Access Memory) or a non-volatile memory, for example a PROM (Programmable Read Only Memory), an EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or a flash memory, e.g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory).

[0043] In various embodiments, a “circuit” may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing software stored in a memory, firmware, or any combination thereof. Thus, in an embodiment, a “circuit” may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g., a microprocessor (e.g., a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor). A “circuit” may also be a processor executing software, e.g., any kind of computer program, e.g., a computer program using a virtual machine code, e.g., Java. Any other kind of implementation of the respective functions which will be described in more detail below may also be understood as a “circuit” in accordance with various alternative embodiments. Similarly, a “module” may be a portion of a system according to various embodiments in the present invention and may encompass a “circuit” as above, or may be understood to be any kind of a logic-implementing entity therefrom.

[0044] Some portions of the present disclosure are explicitly or implicitly presented in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the data processing arts to convey most effectively the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities, such as electrical, magnetic or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

[0045] Unless specifically stated otherwise, and as apparent from the following, it will be appreciated that throughout the present specification, discussions utilizing terms such as “selecting”, “obtaining”, “determining”, “detecting”, “identifying”, “performing”, or the like, refer to the actions and processes of a computer system, or similar electronic device, that manipulates and transforms data represented as physical quantities within the computer system into other data similarly represented as physical quantities within the computer system or other information storage, transmission or display devices. [0046] The present specification also discloses a system (which may also be embodied as a device or an apparatus) for performing the operations/functions of the methods described herein. Such a system may be specially constructed for the required purposes, or may comprise a general purpose computer or other device selectively activated or reconfigured by a computer program stored in the computer. The algorithms presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose machines may be used with computer programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate.

[0047] In addition, the present specification also at least implicitly discloses a computer program or software/functional module, in that it would be apparent to the person skilled in the art that the individual steps or operations of the methods described herein may be put into effect by computer code. The computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming languages and coding thereof may be used to implement the teachings of the disclosure contained herein. Moreover, the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the scope of the invention. It will be appreciated by a person skilled in the art that various modules described herein (e.g., the tracking camera selection module 308, the tracking camera reselection module 310, the video obtaining module 312, the object detection module 314 and/or the tracking camera status determination module 316) may be software module(s) realized by computer program(s) or set(s) of instructions executable by a computer processor to perform the required functions, or may be hardware module(s) being functional hardware unit(s) designed to perform the required functions. It will also be appreciated that a combination of hardware and software modules may be implemented.

[0048] Furthermore, one or more of the steps or operations of a computer program/module or method described herein may be performed in parallel rather than sequentially. Such a computer program may be stored on any computer readable medium. The computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general-purpose computer. The computer program when loaded and executed on such a general-purpose computer effectively results in an apparatus that implements the steps or operations of the methods described herein.

[0049] In various embodiments, there is provided a computer program product, embodied in one or more computer-readable storage mediums (non-transitory computer-readable storage medium), comprising instructions (e.g., the tracking camera selection module 308, the tracking camera reselection module 310, the video obtaining module 312, the object detection module 314 and/or the tracking camera status determination module 316) executable by one or more computer processors to perform a method 200a and/or method 200b of tracking an object as described hereinbefore with reference to FIGS. 2A and 2B. Accordingly, various computer programs or modules described herein may be stored in a computer program product receivable by a system (e.g., a computer system or an electronic device) therein, such as the system 300a and/or system 300b as shown in FIGS. 3A and 3B, for execution by at least one processor 306 of the system 300a and/or system 300b to perform the required or desired functions.

[0050] The software or functional modules described herein may also be implemented as hardware modules. More particularly, in the hardware sense, a module is a functional hardware unit designed for use with other components or modules. For example, a module may be implemented using discrete electronic components, or it can form a portion of an entire electronic circuit such as an Application Specific Integrated Circuit (.ASIC). Numerous other possibilities exist. Those skilled in the art will appreciate that the software or functional module(s) described herein can also be implemented as a combination of hardware and software modules.

[0051] In various embodiments, the above-mentioned computer system may be realized by any computer system (e.g., portable or desktop computer system), such as a computer system 400 as schematically shown in FIG. 4 as an example only and without limitation. Various methods/operations or functional modules (e.g., the tracking camera selection module 308, the tracking camera reselection module 310, the video frame obtaining module 312, the object identification module 314 and/or the tracking camera status determination module 316) may be implemented as software, such as a computer program being executed within the computer system 400, and instructing the computer system 400 (in particular, one or more processors therein) to conduct the methods/functions of various embodiments described herein. The computer system 400 may comprise a computer module 402, input modules, such as a keyboard 404 and a mouse 406, and a plurality of output devices such as a display 408, and a printer 410. The computer module 402 may be connected to a computer network 412 via a suitable transceiver device 414, to enable access to e.g. the Internet or other network systems such as Local Area Network (LAN) or Wide Area Network (WAN). The computer module 402 in the example may include a processor 418 for executing various instructions, a Random Access Memory (RAM) 420 and a Read Only Memory (ROM) 422. The computer module 402 may also include a number of Input/Output (I/O) interfaces, for example I/O interface 424 to the display 408, and I/O interface 426 to the keyboard 404. The components of the computer module 402 typically communicate via an interconnected bus 428 and in a manner known to the person skilled in the relevant ail.

[0052] It will be appreciated by a person skilled in the art that the terminology used herein is for the purpose of describing various embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a”, "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises” and/or "comprising", or the like such as “includes” and/or “including”, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

[0053] In order that the present invention may be readily understood and put into practical effect, various example embodiments of the present invention will be described hereinafter by way of examples only and not limitations. It will be appreciated by a person skilled in the art that the present invention may, however, be embodied in various different forms or configurations and should not be construed as limited to the example embodiments set forth hereinafter. Rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present invention to those skilled in the art.

[0054] In various example embodiments, multiple cameras (or plurality of cameras) may be used to view a monitored area, in which an object (e.g., target object) is moving. FIG. 5 illustrates an exemplary process flow 500 of tracking the object according to various example embodiments of the present invention.

[0055] At 510, tracking cameras in which the target object is showing may be selected (corresponding to selecting one or more tracking cameras from a plurality of cameras as described hereinbefore). In order to select the tracking cameras, the location of the target object may first be detected. The location of the target object may be a point or region in a camera view (e.g., two-dimensional (2D) space) of a corresponding camera, or a place or area (i.e., geographical location) in the real world (e.g., three- dimensional (3D) space). The camera view may be a video frame captured by the camera. For example, an object detection algorithm may be run on the camera view of a corresponding camera to automatically detect the object’s location, manually select the location on the camera view of the corresponding camera, or parse sensor signals sent from the target object. Once the location of the object is detected or determined, one or more candidate cameras may be selected by using one or more camera selection models (corresponding to the tracking camera prediction model as described hereinbefore), which defines a mapping from the object’s location to candidate cameras, or region of camera views corresponding to one or more of the candidate cameras.

[0056] FIGS. 6A-6B show two examples of the camera selection model (or tracking camera prediction model), respectively, according to various example embodiments of the present invention. FIG. 6A illustrates camera region pair probability tables 610 and 620 which shows how likely the target object will show (or be captured) in a camera region pair (a pair of region of camera views corresponding to one or more of the plurality of cameras). In various example embodiments, the whole camera view (e.g., the whole or entire frame captured by the camera) may be regarded or taken as one region as illustrated in table 620 (e.g., denoted by camera CI, camera C2, camera C3, camera C4, camera C5). It should be noted that although each entry of the example camera region pair probability table shows how many times the object showed or is captured in a camera pair during experiments performed in the learning stage (or training stage) rather than a probability value, it is understood that the probability value may be easily obtained by dividing each entry with the diagonal entry in the same row. With the camera region pair probability table, candidate cameras may be easily selected by referring to large entries (except diagonal entry) within the same row as a given camera region, e.g., given region of a camera view corresponding to a camera, (step 714 in FIG. 7). For example, according to the table 610, when the target object is detected in region R2 of camera Cl, “C1..R2”, the largest entry in this row (except diagonal entry) is in column “C2...R1”, so the corresponding candidate camera will be camera C2. In various example embodiments, the top N entries from the row may be selected as candidate cameras, or entries by some threshold may be selected.

[0057] FIG. 6B illustrates a position -camera pair probability table 650 according to various example embodiments. The position-camera pair table show's how likely the target object will show (or be captured) in each camera if it is found in some predefined position in the real world. In this case, the position is a place in the monitored area. Similar to the camera region pair probability table, although the position-camera pair probability table 650 show's a count number in each table entry, it is understood that the probability value may be easily obtained by dividing each entry with a total value in the corresponding row' (total count number from all cameras in the row'). When the target object is detected in some position P, one can easily select candidate cameras as large entries (except the column “Total”) of row P. For example, when the object was detected in position P2, according to the position-camera pair probability table 650, the largest entry is camera “C3”, thus, “C3” will be selected as a candidate camera. Like the camera region pair probability table, the top N entries may be selected or entries defined by some predefined threshold may be selected. In various other example embodiments, each column of the position-camera pair probability table 650 may also be a region in a camera view' of a corresponding camera.

[0058] Accordingly, both the camera region probability table and position-camera pair table define the mapping from a location to candidate cameras, or region of camera view's corresponding to one or more of the candidate cameras. Such mapping relationship can be learned by conducting experiments. In various other example embodiments, the mapping relationship may be built manually according to experience. In various example embodiments, time difference may be introduced in the mapping.

[0059] FIG. 7 depicts an exemplary schematic diagram of a method 700 for selecting one or more tracking cameras from a plurality of cameras based on a tracking camera prediction model according to various example embodiments of the present invention. In various example embodiments, candidate cameras may be selected according to the camera region pair probability (710, 712, 714) and according to the position-camera pair probability table (720, 722, 724), respectively. For example, the candidate cameras selected according to the camera region pair probability form a first set of candidate cameras. The candidate cameras selected according to the positioncamera pair probability table forms second set of candidate cameras. The candidate cameras selected based on the camera region probability table may be different from those selected based on the position-camera pair probability table. In other words, the candidate cameras in the first set of candidate cameras may be different from the candidate cameras in the second set of candidate cameras.

[0060] At 730, the candidate cameras selected based on the camera region probability table may be combined with the candidate cameras selected based on the position-camera pair probability table to form a single set of candidate cameras (corresponding to the combined set of candidate cameras as described hereinbefore). In other words, the candidate cameras from the first set of candidate cameras and from the second set of candidate cameras may be combined to form the single set of candidate cameras. In various example embodiments, the candidate cameras selected based on the camera region probability table may be combined with the candidate cameras selected based on the position-camera pair probability table by creating a union set which contains all the candidate cameras. FIG. 8 shows another example method 800 for combining the candidate cameras selected based on multiple camera selection models. In various example embodiments, the candidate cameras may be combined by assigning a score to each candidate camera and selecting one or more of the candidate cameras with the highest scores (e.g., N number of candidate cameras with top N scores).

[0061] At 810, a score for each camera may be initialized by zero. At 820 (e.g., 820a, 820b), for each candidate camera of each set of candidate cameras obtained based on a respective camera selection model, the score of the candidate camera may be increased by a specified value. For example, the increment may be a fixed value for every camera in the same set of candidate cameras, as illustrated in FIG. 8. Alternatively, the increment may be computed or specified differently for each candidate camera of the same set of candidate cameras such as by using the probability value, in a non-limiting example. At 830, candidate cameras may be selected according to the corresponding scores of the candidate cameras. In various example embodiments, the candidate cameras with top N scores may be selected as illustrated in FIG. 8. Alternatively, the candidate cameras may be selected by its associated scores based on some predefined threshold.

[0062] Referring back to FIG. 7 , it should be noted that the candidate cameras selected at 730 are only candidate cameras, which means that the target object may or may not show in each candidate camera. In various example embodiments, the selected candidate cameras may be filtered to obtain the one or more tracking cameras, in which the target object shows or is captured. At 740, the selected candidate cameras may be filtered to obtain the one or more tracking cameras. In other words, the one or more tracking cameras in which the target object shows may be selected from the selected candidate cameras. The one or more tracking cameras in which the target object is captured may be selected by image feature matching. FIG. 9 depicts an exemplary schematic diagram of a method 900 for selecting one or more tracking cameras from candidate cameras by image feature matching, according to various example embodiments of the present invention.

[0063] As shown in FIG. 9, for each candidate camera, image features for each (candidate) object detected in the candidate camera in video data obtained from the candidate camera may be extracted and these extracted features may be compared with the target object’s features. For example, multiple candidate objects may be detected from a video frame, and feature matching may be performed to identify the target object. If feature matching is successful, the candidate camera will be nominated as a hacking camera.

[0064] Referring back to FIG. 5, once the one or more tracking cameras are selected, at 520, object tracking may be performed for each tracking camera. The object tracking may include detecting, for each tracking camera, the object in video data obtained from the tracking camera. The object tracking task may include object localization subtask. For example, tracking history may be used to predict the object's next location, and compare with all the locations of the detected object in a current video frame. Many sophisticated object tracking algorithms may be used to perform the object hacking such as, but not limited to, the Kalman filter, SORT. During the object hacking for each tracking camera, the method 500 may continuously check whether the target object moved to a new location or not. At 530, a determination as to whether the object’s location changed may be made. If the target object’s location has changed (e.g., moved to a new location such as a new region of view), it means that the object may become visible (or captured) in other cameras and these cameras need to be added as tracking cameras. New cameras (i.e., cameras not in the existing or current one or more tracking cameras selected) in which the object may be visible or captured may be included as tracking cameras by repeating the step 510. In other words, the tracking camera prediction model may be run again to see whether other tracking cameras need to be added. This tracking process may continue until the target object disappear in all cameras, i.e., there is no tracking cameras anymore. [0065] In various example embodiments, during the object tracking for each tracking camera, the method 500 may also continuously update the tracking cameras status. At 540, for each tracking camera, the status of the tracking camera may be determined or updated. The definition of tracking camera status may be highly related to the conditions for camera reselection at step 550. In other words, the tracking camera status provides all necessary and latest information for evaluating the conditions for camera reselection at step 550. FIG. 10 shows an example method 1000 of determining or updating the status of each tracking camera, and determining or evaluating the conditions for camera reselection so as to trigger camera reselection. In various example embodiments, the camera reselection condition may be defined using two values: (i) the number of tracking cameras and (ii) the number of losing cameras (from the tracking cameras) in which the tracking may be lost within a certain time threshold. Thus, during the object tracking process, these two values may be continuously updated.

[0066] Referring to FIG. 10, at 1010, for each tracking camera, the location of the object may be detected. The location of the object may be detected by using any of object detection algorithms, such as but not limited to, Yolo V3, Faster R-CNN. Here the object location refers to the position within the camera view of the tracking camera. At 1020, a moving speed in the camera view (e.g., 10 pixels per second) of the target object may be predicted based on the object’s tracking history. The tracking history may refer to identified positions of the object in previous video frames. The tracking history may be object-camera wise. If one object shows in multiple cameras, then the object will have corresponding multiple tracking histories. By comparing the current location of the object with an edge of the camera view, and given the moving speed of the object, one can estimate how long it takes for the object to move out of the camera’s FoV (field of view) at 1030. In other words, the remaining time R for the object to move out of the field of view of the tracking camera may be predicted or determined. At 1040, if the remaining time R is less than a threshold, the current tracking camera may be nominated or regarded as a losing camera. At 1070, by comparing the number of tracking cameras and the number of losing cameras to their threshold, respectively, a determination may be made as to whether the camera reselection should be triggered.

[0067] It should be noted that the trigger condition of camera reselection (e.g., at step 550 in FIG. 5) may be freely defined. For example, the trigger condition for camera reselection may be defined based on the number (quantity) of tracking cameras, size of the object in the camera view of a corresponding tracking camera, object’s moving speed, etc. The principle is that when the target object is likely to show in some cameras which are not included in current tracking cameras, the condition(s) should be prone to be triggered.

[0068] Referring back to FIG. 5, at 560, reselection of tracking cameras based on camera group information may be performed.

[0069] In various example embodiments, the cameras of the plurality of cameras may be divided into different groups according to properties of each camera. For example, cameras which are near to each other may be categorized as one group. It should be noted that one camera may belong to multiple groups. FIG. 11 shows an example of camera group information according to various example embodiments.

[0070] FIG. 12 shows a process flow 1200 of reselecting tracking cameras based on camera group information according to various example embodiments. At 1230, all cameras in the same group with each current tracking camera may be selected as candidate group-related cameras. At 1240, new tracking cameras may be filtered (or selected) from the candidate group-related cameras. The new tracking cameras may be filtered in the same manner as step 740 in FIG. 7. At 1250, the new tracking cameras selected may be added to the (current) one or more tracking cameras to form a new set of tracking cameras.

[0071] Referring back to FIG. 5, after updating the tracking cameras (i.e., forming a new set of tracking cameras based on the reselection of tracking camera based on camera group information), at 570, the tracking camera prediction model may be updated according to the latest tracking cameras (e.g., new tracking camera(s)). In various example embodiments, the tracking camera prediction model may be updated with respect to entries related to the new tracking cameras obtained from the reselection of tracking cameras based on camera group information. FIGS. 13A-13B illustrate schematics for updating the camera region pair probability table and the positioncamera pair probability table, respectively, according to various example embodiments. For example, entries may be updated to include increased count or probabilities of an object being captured in each pair of tracking cameras or each pair of camera view regions corresponding to a tracking camera. In another example, entries may be updated to include increased count or probabilities of an object being captured in a camera of a camera and location pair in every tracking camera.

[0072] Referring back to FIG. 5, in various example embodiments, the order of steps 520 to 530 may be exchanged with steps 540 to 570. FIG. 14 illustrates another exemplary process flow 1400 of tracking the object according to various example embodiments of the present invention. The process 1400 is similar to the process 500 described with respect to FIG. 5, except that the order of steps 520 to 530 is exchanged with steps 540 to 570. Referring to the process flow 1400, before the object tracking at step 520, reselection of tracking cameras at step 560 may be performed after selecting the one or more tracking cameras based on the tracking camera prediction model at step 510. For example, the process 1400 may first select the one or more tracking cameras based on the tracking camera prediction model at step 510, and if few tracking cameras are selected, reselection of tracking cameras may be triggered at step 550, and reselection of tracking cameras may be performed at step 560.

[0073] In various example embodiments, the step 550 of determining or evaluating the conditions for camera reselection so as to trigger camera reselection in FIG. 5 and FIG. 14 need not be performed. In such case, the reselection of tracking cameras will always be performed. FIG. 15 illustrates another exemplary process flow 1500 of tracking the object according to various example embodiments of the present invention. The process 1500 may perform reselection of tracking cameras 560 based on the camera group information each time one or more tracking cameras are selected based on the tracking camera prediction model at step 510. In other words, reselection of tracking cameras 560 will always be triggered after calling the tracking camera prediction model.

[0074] In various example embodiments, step 570 of updating the tracking camera prediction model in FIG. 5, FIG. 14, and FIG. 15 need not be performed, so that the tracking camera prediction model will not be updated during the object tracking process.

[0075] FIG. 16 depicts a simplified system 1600 according to various example embodiments of the present invention. The system 1600 may connect a plurality of cameras, cornputer(s) on which the method may be running as a program, and database which stores data.

[0076] While embodiments of the invention have been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the invention as defined by the appended claims. The scope of the invention is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.