Technical Field

The present invention relates to a system and method for distinguishing personnel and visitors entering a store in systems that count visitors by means of image processing technologies.

Background of the Invention

Today, there are systems which enable to conduct sales statistics by learning the number of visitors of stores and essentially uses image processing technologies. The said systems compile images of customers who enter a store upon being placed to locations which are determined suitably for taking images in stores and ensure that the images compiled are processed within the system or in external servers. Numbers of visitors who visit a store are measured from images compiled by image processing technologies and the relationship between the numbers of sales realized and the number of visitors are determined.

Visual methods or radio frequency recognition methods are used to distinguish personnel from visitors in stores that particularly employ personnel in a number comparable with the number of visitors, wherein visitor mobility is observed equivalently to personnel mobility or low error rate is required for determining the number of visitors.

Nevertheless, using methods of facial recognition technology and detection of reflecting material -which are among visual methods enabling to distinguish personnel- is not sufficient for personnel exception due to the fact that visual detection errors are seen and these may lead to violations of privacy that may occur upon facial information of personnel are stored rawly. It is not possible to use methods of radio frequency recognition -another method used for personnel exception- in a meaningful way for personnel exception because active radio frequency broadcasts based on Wi-Fi or Bluetooth which enable to identify personnel are exposed to software or hardware limitations of electronic devices enabling to perform related broadcasts and belonging to related personnel or they do not have a signal level and quality that will enable to detect location information of personnel.

Considering the deficiencies of visitor counting systems in the state of the art and methods of personnel exception in the state of the art which are applied in the said systems, it is understood that there is need for a computer-applied method for distinguishing personnel and visitors entering a store in systems that count visitors by means of image processing technologies, by running a multi-step decisionmaking mechanism; and a system which can apply this method.

The Chinese patent document no. CN106463006, an application in the state of the art, discloses a personnel authentication and tracking system. The said system receives numbers of personnel visually by means of image processing technology in a detection region, enables to identify radio frequency identification tags kept by personnel by means of a radio frequency sensor, and compares the number of personnel determined visually with the number of tags identified by the radio frequency sensor.

Summary of the Invention

An objective of the present invention is to realize a system and method for distinguishing personnel and visitors entering a store in systems that count visitors by means of image processing technologies, by running a multi-step decisionmaking mechanism.

Detailed Description of the Invention

“A System and Method of Personnel Exception in Visitor Count” realized to fulfil the objective of the present invention is shown in the figure attached, in which:

Figure l is a schematic block diagram of the inventive system.

Figure l is a flowchart of the inventive method.

The components illustrated in the figures are individually numbered, where the numbers refer to the following:

1. System

2. Image sensor

3. Memory element

4. Processor

5. Communication element

6. Server

100. Method

Y: Decision-making software

The inventive system (1) for distinguishing personnel and visitors entering a store in systems that count visitors by means of image processing technologies, by running a multi-step decision-making mechanism comprises:

- at least one image sensor (2) which is configured to view personnel and visitors entering a store and to process them by means of image processing technologies; - at least one memory element (3) which is configured to save height information, vector facial information, beacon information of each person viewed and visual markers in the form of colours of clothes, special images, patterns specific to a store temporarily for a predetermined period of time;

- at least one processor (4) which is configured to establish data communication with the memory element (3) and to manage the memory element (3); and

- at least one communication element (5) which is in communication with the memory element (3) and the processor (4) and configured to establish data communication; and

- at least one server (6) which is configured to save and run the decisionmaking software (Y); to save height information, vector facial information, beacon information of each personnel and visual markers in the form of colours of clothes, special images, patterns as a personnel identification information; to ensure that persons are distinguished as visitor and personnel by receiving the data sent by the communication element (5) and then feeding it to the decision-making software (Y); and to conduct shift and break follow-ups of the personnel independently or in an integrated way.

The image sensor (2) included in the inventive system (1) is configured to view personnel and visitors entering a store and to process them by means of image processing technologies. In a preferred embodiment of the invention, the image sensor (2) is configured to create a depth map in the area wherein visitors and personnel are viewed and to parse visitors and personnel from other mobile and stationary objects around on the basis of depth map information. In a preferred embodiment of the invention, the image sensor (2) is positioned in such a way that visitors and personnel can be viewed from a bird's-eye view. In a preferred embodiment of the invention, it is ensured by the image sensor (2) to determine a beacon in a one-dimensional or two-dimensional barcode structure that is placed onto clothes of personnel viewed by the image sensor (2) such that it can be seen from a bird's-eye view, and to analyse the beacon determined upon identification, and to parse the information coded within the beacon. In a preferred embodiment of the invention, it is ensured by the image sensor (2) to determine height information of the personnel viewed, upon being processed by the depth information created by the image sensor (2). In a preferred embodiment of the invention, it is ensured by the image sensor (2) to determine vector facial information of the personnel viewed, by means of image processing technologies. In a preferred embodiment of the invention, it is ensured by the image sensor (2) to determine colours of clothes, special images, patterns and similar visual beacons of the personnel viewed, by means of image processing technologies.

The memory element (3) included in the inventive system (1) is configured to save height information, vector facial information, beacon information of each person viewed and visual markers in the form of colours of clothes, special images, patterns specific to a store temporarily for a predetermined period of time.

The processor (4) included in the inventive system (1) is configured to establish data communication with the memory element (3) and to manage transmission of the data stored in the memory element (3) periodically to the communication element (5) and deletion of the transmitted data from the memory element (3).

The communication element (5) included in the inventive system (1) is configured to periodically receive the data stored from the memory element (3) and to transmit these data to the server (6) by a command received from the processor (4).

The server (6) included in the inventive system (1) is configured to save and run the decision-making software (Y); to save height information, vector facial information, beacon information of each personnel and visual markers in the form of colours of clothes, special images, patterns as a personnel identification information; to ensure that persons are distinguished as visitor and personnel by receiving the data sent by the communication element (5) and then feeding it to the decision-making software (Y); and to conduct shift and break follow-ups of the personnel independently or in an integrated way. In a preferred embodiment of the invention, the server (6) enables to compare the beacon information, the height information, the vector facial information, and the visual beacons in the form of colours of clothes, special images, pattern information -which are transmitted by the communication element (5) periodically- with the pre-recorded personnel identification information by means of a vote of confidence method according to classical statistical algorithms within the decision-making software (Y) or iterative algorithms such as machine learning models, deep learning models and artificial intelligence models; and to mark persons who have the highest percentage of confidence as the personnel. Thereby, the number of persons who are marked as the personnel by means of the highest percentage of confidence can be distinguished from the number of visitors, upon tips to be visually determined between visitors and personnel are evaluated in a plurality of steps and according to a percentage of confidence. In a preferred embodiment of the invention, the server (6) can determine the time spent in the store and the time spent outside the store, the time spent with the units within the store by the parsed personnel and their number and time of interaction with customers, in a compatible way with the periods when the data are processed; and conduct shift, break, unit and interaction time follow-ups of the personnel. In one embodiment of the invention, follows-up of shift, break, unit and interaction time of the personnel determined by the server (6) can be conducted in an integrated way upon calculation by adding the total shift, break, unit and interaction time of all personnel and dividing the average shift, break, unit and interaction time and the total shift and break time into the number of personnel. In one embodiment of the invention, shift, break, unit and interaction time follow-ups of the personnel determined by the server (6) can be conducted by examining shift, break, unit and interaction time of each personnel independently.

The inventive computer-applied method (100) for distinguishing personnel and visitors entering a store in systems that count visitors by means of image processing technologies, by running a multi-step decision-making mechanism comprises steps of: saving height information, vector facial information, beacon information of each personnel and colours of clothes, special images and patterns specific to a store, as a personnel information (101);

- viewing personnel and visitors entering a store from a bird's-eye view and processing them by means of image processing technologies (102);

- determining a beacon in a one-dimensional or two-dimensional barcode structure that is placed onto clothes of personnel such that it can be seen from a bird's-eye view (103);

- analysing the beacon determined upon identification, and parsing the information coded within the beacon (104);

- determining height information of the personnel viewed, by processing the depth information (105);

- determining vector facial information of the personnel viewed (106);

- determining visual markers in the form of colours of clothes, special images, patterns of the personnel viewed (107); comparing the beacon information, the height information, the vector facial information, the special images, the pattern information -which are determined visually- with the pre-recorded personnel information, by means of a vote of confidence method according to classical statistical algorithms or iterative algorithms (108);

- distinguishing the viewed personnel from the visitors by sending the information of personnel receiving high vote of confidence to external servers (109); - conducting shift, break, unit and interaction time follow-ups of the personnel independently or in an integrated way (110).

In the inventive method (100), height information, vector facial information, beacon information of each personnel and colours of clothes, special images and patterns specific to a store, are saved into the server (6) as a personnel information

(101); personnel and visitors entering a store are viewed from a bird's-eye view by the image sensor (2) and processed by means of image processing technologies

(102). A beacon in a one-dimensional or two-dimensional barcode structure that is placed onto clothes of personnel such that it can be seen from a bird's-eye view, is determined by the image sensor (2) (103) and the beacon determined by the image sensor (2) is analysed upon identification, and the information coded within the beacon are parsed (104). Height information of the personnel viewed by the image sensor (2) is determined by processing the depth information (105); their vector facial information is determined (106); visual markers in the form of colours of their clothes, special images, patterns are determined (107). The memory element (3) saves height information, vector facial information, colours of clothes of the personnel, visual markers in the form of special images, patterns temporarily for a predetermined period of time. The visually determined data which are sent from the communication element (5) to the server (6) upon being stored by the memory element (3) under the control of the processor (4), are compared with the personnel information within the decision-making software (Y) by means of a vote of confidence method according to classical statistical algorithms or iterative algorithms (108) and the viewed personnel are distinguished from the visitors by determining the personnel receiving high vote of confidence (109). Shift, break, unit and interaction time follow-ups of the personnel are conducted by the server (6) independently or in an integrated way (110).

Within these basic concepts; it is possible to develop various embodiments of the inventive “System (1) and Method (100) of Personnel Exception in Visitor Count”; the invention cannot be limited to examples disclosed herein and it is essentially according to claims.