CONVEYOR. CONTROLLING APPARATUS AND METHOD

Field of the Invention

[0001]

The present invention relates to a conveyor controlling apparatus and a conveyor controlling method for controlling a conveyor in accordance with environment parameters with respect to the conveyor.

Background of the Invention

[0002]

In recent years, automatic systematisation in logistics has been steadily progressing, and technologies related to automating warehouse management and handling of conveyance items such as checked-luggage and parcels at airports have been developed. The technologies related to automated systems in logistics include, for example, US Patent No. 5165520 Publication, US Patent Application Publication No. 2018/0148271 and US Patent No. 7055672 Publication.

[0003]

Nevertheless, it is still common to input conveyance items to automatic conveyance systems by hand. For example, in conventional airports, it is common that the checking-in of passengers' luggage is manually handled by ground handling staff. However, even if the ground-handling staff are well trained, there will invariably by situations where the luggage is presented in an inappropriate configuration, or portions of the luggage are in an awkward position (for example, the handle of a suitcase may be extended, or the zipper on a duffel bag may open). This problem is also prevalent in self-drop luggage kiosks.

[0004]

The checked-luggage can be in transit on a baggage handling system (BHS) and on its way to being loaded onto a departing plane. Therefore, to ensure that the loading of the checked-luggage is seamless and does not cause any delay to the departing plane, it is vital to ensure that further checks and measures are in place to ensure that all checked-luggage satisfy a 'forwarding condition' before being loaded onto the plane.

The forwarding condition is mainly subject to capability of the BHS. Accordingly, if a conveyance item does not meet the forwarding condition, it may cause problems in the BHS such as conveyance jams.

[0005]

Thus, a solution is required that automatically detects any checked- luggage that does not satisfy the forwarding condition. This solution must be capable of clearly identifying the problematic luggage (among all the other luggage) so that appropriate counter measures can be quickly taken (i.e. an operator can manually fix or adjust the position or configuration of the luggage). The solution should also be robust enough to accept feedback and learn and adapt to updating standards for identification of defective luggage.

[0006]

Another problem is that an operator or a passenger, either accidentally or intentionally, enters an area, in which humans must not be allowed to enter (i.e. a keep-out zone), near an automatic conveyance system. This kind of incident may lead to a disruption of operation of the automatic conveyance system.

[0007]

An object of the present invention is to provide a conveyor controlling apparatus and a conveyor controlling method have been created in light of such problems to improve the reliability of conveyance systems having conveyors.

Summary of the Invention

[0008]

A conveyor controlling apparatus for controlling a conveyance object on a conveyor in accordance with a conveyance state of the conveyance object, the apparatus comprises: an upstream capturing device configured to capture an upstream image of the conveyance object when the conveyance object is in a first upstreamassessment zone on the conveyor and while the conveyance object is conveyed by the conveyor in a direction towards the upstream capturing device; a downstream capturing device configured to capture a downstream image of the conveyance object in a second downstreamassessment zone on the conveyor, the second downstream-assessment zone being set further downstream when compared to the first upstreamassessment zone; an image-analysing module configured to determine, based on the upstream image and/or the downstream image, whether the conveyance state of the conveyance object satisfies a forwarding condition; and a conveyor-controlling module configured to control the conveyor based on a result of the determination carried out by the image-analysing module; wherein if the image-analysing module determines, based on the upstream image, that the conveyance state of the conveyance object does not satisfy the forwarding condition, then the conveyor-controlling module stops the conveyor such that the conveyance object is positioned in the second downstream-assessment zone, and wherein if the image-analysing module determines, based on the downstream image, that the conveyance state of the conveyance object satisfies the forwarding condition, then the conveyor-controlling module restarts the conveyor.

[0009]

The conveyor controlling apparatus, wherein the image-analysing module is further configured to implement a learning model, the learning model adapted to define an assessment standard for the conveyance state, and the image-analysing module utilises the learning model in the determination of whether the conveyance state of the conveyance object satisfies a forwarding condition.

[0010]

The conveyor controlling apparatus, further comprises: an overriding module configured to execute an overriding operation, which is an operation forcibly restarts the conveyor, even when the imageanalysing module determines based on the downstream image that the conveyance state of the conveyance object does not satisfy the forwarding condition, in response to an overriding command; and a training module configured to update the learning model based on the overriding command and the upstream-side and/or downstream images of the conveyance object whenever the overriding operation is executed by the overriding module, wherein the overriding command includes a reason of execution of the overriding operation.

[0011]

The conveyor controlling apparatus, wherein the upstream capturing device and the downstream capturing device are adjacent to one another and orientated in opposing directions.

[0012]

The conveyor controlling apparatus further comprises: a main body; a pillar extending upwardly from the main body; and a head unit provided at a top end of the pillar, wherein the upstream capturing device and the downstream capturing device are mounted in the head unit.

[0013]

The conveyor controlling apparatus further comprises: an upstream lighting device configured to illuminate the first upstreamassessment zone in which the upstream image is captured by the upstream capturing device; and a downstream lighting device configured to illuminate the second downstream-assessment zone in which the downstream image is captured by the downstream capturing device.

[0014]

The conveyor controlling apparatus, wherein the upstream lighting device and the downstream lighting devices are mounted in the head unit.

[0015]

The conveyor controlling apparatus, wherein a zone-separation distance, which is a distance between the first upstream-assessment zone and the second downstream-assessment zone, and the zone-separation distance is adjusted according to a processing capability of the image-analysing module.

[0016]

The conveyor controlling apparatus further comprises: an alarm device configured to provide a visual and/or aural notification of the result of the determination carried out by the image-analysing module. [0017] A conveyor controlling method for controlling a conveyance object on a conveyor in accordance with a conveyance state of the conveyance object, the method comprises steps of: capturing an upstream image of the conveyance object by an upstream capturing device when the conveyance object is in an first upstreamassessment zone on the conveyor and while the conveyance object is being conveyed by the conveyor in a direction towards the upstream capturing device; capturing a downstream image of the conveyance object by a downstream capturing device in a second downstream-assessment zone on the conveyor, the second downstream-assessment zone being set further downstream when compared to the first upstream-assessment zone; determining, based on the upstream image and/or the downstream image, whether the conveyance state of the conveyance object satisfies a forwarding condition; and controlling the conveyor based on a result of the determination step, wherein if the determination step determines, based on the upstream image, that the conveyance state does not satisfy the forwarding condition, then the controlling step stops the conveyor such that the conveyance object is positioned in the second downstream-assessment zone, and wherein if the determination step determines, based on the downstream image, that the conveyance state satisfies the forwarding condition, then the controlling step restarts the conveyor.

[0018]

The controlling method, further comprises steps of: providing a learning model defining an assessment standard to the conveyance state; and recognising the conveyance state of the conveyance object based on the learning model and the upstream image and/or the downstream image, wherein the determining step determines whether the conveyance state, which is recognised in the recognising step, satisfies the forwarding condition. [0019]

The controlling method further comprises steps of: executing an overriding operation, which is an operation forcibly restarts the conveyor, even when the determining step determines, based on the downstream image, that the conveyance state of the conveyance object does not satisfy the forwarding condition, in response to an overriding command; and updating the learning model based on the overriding command and the upstream image and/or the downstream image of the conveyance object whenever the overriding operation is executed, wherein the overriding command includes a reason of execution of the overriding operation.

[0020]

The controlling method, wherein the upstream capturing device and the downstream capturing device are adjacent to one another and orientated in opposing directions.

[0021]

The controlling method further comprises steps of: providing a main body; providing a pillar extending upwardly from the main body; providing a head unit at a top end of the pillar; and providing the upstream capturing device and the downstream capturing device in the head unit.

[0022]

The controlling method further comprises steps of: providing an upstream lighting device configured to illuminate the first upstream-assessment zone in which the upstream image is captured by the upstream capturing device; and providing a downstream lighting device configured to illuminate the second downstream-assessment zone at which the downstream image is captured by the downstream capturing device.

[0023]

The controlling method, wherein the upstream and downstream lighting devices are mounted in the head unit.

[0024]

The controlling method further comprises a step of: adjusting a zone-separation distance, which is a distance between the first upstream-assessment zone and the second downstream-assessment zone, according to a processing capability of the determining step.

[0025]

The controlling method further comprises a step of: visually and/or aurally notifying the result of the determining step. [0026]

A conveyor controlling apparatus for controlling a conveyor in accordance with environment parameters with respect to the conveyor, the apparatus comprises: a capturing device for capturing an environmental image related to the conveyor; an image-analysing module for determining, based on the environmental image captured by the capturing device, whether an operational condition of the conveyor is satisfied; and a conveyor-controlling module for controlling the conveyor based on a result of the determination carried out by the image-analysing module, wherein if the image-analysing module determines that the operational condition is not satisfied, then the conveyor-controlling module stops the conveyor.

[0027]

The conveyor controlling apparatus, wherein the capturing device captures an object image of a conveyance object on the conveyor when the conveyance object is in an assessment zone on the conveyor, the object image is the environmental image with respect to the conveyance object, the image-analysing module determines based on the object image captured by the capturing device whether a conveyance state of the conveyance object satisfies a forwarding condition, the forwarding condition is the operational condition with respect to the conveyance object, and if the image-analysing module determines based on the object image that the conveyance state of the conveyance object does not satisfy the forwarding condition, then the conveyor-controlling module stops the conveyor such that the conveyance object is positioned within the assessment zone. [0028]

The conveyor controlling apparatus, wherein when the conveyor is stopped by the conveyor-controlling module, if the image-analysing module determines based on the object image that the conveyance state of the conveyance object satisfies the forwarding condition, then the conveyor-controlling module restarts the conveyor.

[0029]

The conveyor controlling apparatus, wherein the assessment zone includes a first assessment zone, and a second assessment zone which is provided in a downstream direction with respect to the first assessment zone, the first assessment zone is an area in which the object image is captured as a first image by the capturing device while the conveyance object is conveyed by the conveyor, the second assessment zone is an area in which the object image is captured as a second image by the capturing device while the conveyance object is stationary on the conveyor when the conveyor is stopped by the conveyor-controlling module, the conveyor-controlling module stops the conveyor such that the conveyance object is positioned within the second assessment zone if the image-analysing module determines based on the first image that the conveyance state of the conveyance object does not satisfy the forwarding condition, and when the conveyor is stopped by the conveyor-controlling module, if the image-analysing module determines based on the second image that the conveyance state of the conveyance object satisfies the forwarding condition, then the conveyor-controlling module restarts the conveyor.

[0030]

The conveyor controlling apparatus, wherein the capturing device includes a first image-capturing device for capturing the first image, and a second image-capturing device for capturing the second image, the first assessment zone is defined on the conveyor provided in an upstream direction with respect to the first image-capturing device, and the second assessment zone is defined on the conveyor provided in the downstream direction with respect to the second image-capturing device. [0031]

The conveyor controlling apparatus, wherein the image-analysing module is further configured to implement a learning model, the learning model defines an assessment standard for the conveyance state, and the image-analysing module utilises the learning model in the determination of whether the conveyance state of the conveyance object satisfies the forwarding condition.

[0032]

The conveyor controlling apparatus according to any one of claims 2 to 5, further comprises: an overriding module for executing an overriding operation, which is an operation to forcibly restart the conveyor, even when the image-analysing module determines based on the object image that the conveyance state of the conveyance object does not satisfy the forwarding condition, in response to an overriding command; and a training module for updating the learning model based on the overriding command and the object image of the conveyance object when the overriding operation is executed by the overriding module, wherein the overriding command must include an overriding reason for execution of the overriding operation.

[0033]

The conveyor controlling apparatus, further comprises: an intrusion camera as the capturing device for capturing an intrusion image; and an intrusion-analysing module for determining based on the intrusion image captured by the intrusion camera whether an intrusion state of the keep- out zone satisfies an intrusion condition, wherein the intrusion image is the environmental image with respect to a keep- out zone which is an area including at least the conveyer, the intrusion condition, which is the operational condition with respect to the keep-out zone, is a condition that any human is not detected in the keep-out zone, if the intrusion-analysing module determines that the intrusion state does not satisfy the intrusion condition, then the conveyor-controlling module stops the conveyor.

[0034]

The conveyor controlling apparatus, wherein when the conveyor is stopped by the conveyor-controlling module because of the determination by the intrusion-analysing module that the intrusion condition was not satisfied, if the intrusion-analysing module determines based on the intrusion image that the intrusion state satisfies the intrusion condition, then the conveyor-controlling module restarts the conveyor. [0035]

The conveyor controlling apparatus, further comprises: a main body in which has the image-analysing module and the conveyor-controlling module; a pillar extending upwardly from the main body; and a head unit, which is provided at a top end of the pillar, housing the capturing device.

[0036]

A conveyor controlling method for controlling a conveyor in accordance with environment parameters with respect to the conveyor, the method comprises steps of: preparing a capturing device; first capturing an environmental image related to the conveyor by the capturing device; first determining, based on the environmental image captured in the first capturing step, whether an operational condition of the conveyor is satisfied; and controlling the conveyor based on a result of the determination carried out by the first determining step, wherein if the determination step determines based on the environmental image that the operational condition is not satisfied, then the controlling step stops the conveyor.

[0037]

The method, wherein the first capturing step captures an object image of a conveyance object on the conveyor when the conveyance object is in an assessment zone on the conveyor, the object image is the environmental image with respect to the conveyance object, the first determining step determines, based on the object image captured in the first capturing step, whether a conveyance state of the conveyance object satisfies a forwarding condition, the forwarding condition is the operational condition with respect to the conveyance object, and if the first determining step determines based on the object image that the conveyance state of the conveyance object does not satisfy the forwarding condition, then the controlling step stops the conveyor such that the conveyance object is positioned within the assessment zone.

[0038]

The method further comprises steps of: second capturing the object image of the conveyance object in the assessment zone by the capturing device when the conveyor is stopped by the stopping step; second determining based on the object image captured in the second capturing step whether the conveyance state of the conveyance object satisfies the forwarding condition; and restarting the conveyer if the second determining step determines that the conveyance state satisfies the forwarding condition.

[0039]

The method, wherein the first capturing step captures the object image as a first image when the conveyance object is within a first assessment zone which is a part of the assessment zone, the second capturing step captures the object image as a second image when the conveyance object is within a second assessment zone which is another part of the assessment zone and is provided in a downstream direction with respect to the first-assessment zone, the first determining step is carried out by determining, based on the first image captured in the first capturing step, whether the conveyance state of the conveyance object in the first assessment zone satisfies the forwarding condition, the second determining step is carried out by determining, based on the second image captured in the second capturing step, whether the conveyance state of the conveyance object in the second assessment zone satisfies the forwarding condition, the controlling step stops the conveyor such that the conveyance object is positioned within the second assessment zone if the first determining step determines that the conveyance state of the conveyance object in the first assessment zone does not satisfy the forwarding condition, and the restarting step restarts the conveyer if the second determining step determines based on the second image, when the conveyor is stopped by the stopping step, that the conveyance state of the conveyance object in the second assessment zone satisfies the forwarding condition.

[0040]

The method, further comprises steps of: providing a first image-capturing device, as a part of the capturing device, for capturing the first image, and providing a second image-capturing device, as another part of the capturing device, for capturing the second image, wherein the first assessment zone is defined on the conveyor provided in an upstream direction with respect to the first image-capturing device, and the second assessment zone is defined on the conveyor provided in the downstream direction with respect to the second image-capturing device. [0041]

The method, further comprises a step of: providing a learning model, which defines an assessment standard for the conveyance state, being utilised in the determination of whether the conveyance state of the conveyance object satisfies the forwarding condition. [0042]

The method, further comprises steps of: executing an overriding operation, which is an operation forcibly restarts the conveyor, even when the second determining step determines that the conveyance state of the conveyance object in the second assessment zone does not satisfy the forwarding condition, in response to an overriding command; and updating the learning model based on the overriding command and the object image when the overriding operation is executed by the executing step, wherein the overriding command must include an overriding reason for execution of the overriding operation.

[0043]

The method, further comprises steps of: preparing an intrusion camera as the capturing device for capturing an intrusion image; intrusion-determining, based on the intrusion image captured by the intrusion camera, whether an intrusion state of the keep-out zone satisfies an intrusion condition; wherein the intrusion image is the environmental image with respect to a keep- out zone which is an area including at least the conveyer, the intrusion condition, which is the operational condition with respect to the keep-out zone, is a condition that any human is not detected in the keep-out zone, and if the intrusion-determining step determines that the intrusion state does not satisfy the intrusion condition, then the controlling stops the conveyor. [0044]

The method, wherein when the conveyor is stopped by the conveyor-controlling module because of the determination by the intrusion-determining step that the intrusion condition was not satisfied, if the intrusion-determining step determines based on the object image that the intrusion state satisfies the intrusion condition, then the conveyor-controlling module restarts the conveyor. [0045]

The method, further comprises steps of: providing a main body; providing a pillar extending upwardly from the main body; and providing a head unit, which houses the capturing device, at a top end of the pillar.

Brief Description of the Drawings

[0046]

FIG 1 is a schematic view showing a conveyor controlling apparatus according to first and third embodiments of the present invention when the apparatus is installed near a belt conveyor of which the conveying direction is in the right direction.

[0047]

FIG 2 is a schematic diagram showing a part of an airport in which the conveyor controlling apparatuses according to the first to third embodiments of the present invention are installed.

[0048]

FIG 3 is a schematic perspective left-front view showing the conveyor controlling apparatus according to the first embodiment of the present invention.

[0049]

FIG 4 is a schematic block diagram showing a configuration of the conveyor controlling apparatus according to the first embodiment of the present invention.

[0050]

FIG 5 is a schematic plan view (Ai-Ai cross-sectional view of FIG 7) mainly showing a bottom face of a sensor head of the conveyor controlling apparatus according to the first embodiment of the present invention.

[0051]

FIG 6 is a schematic front view of the conveyor controlling apparatus according to the first embodiment of the present invention.

[0052]

FIG 7 is a schematic rear view of the conveyor controlling apparatus according to the first embodiment of the present invention.

[0053]

FIG 8 shows a schematic left view of the conveyor controlling apparatus according to the first embodiment of the present invention when the apparatus is installed adjacent to a belt conveyor.

[0054]

FIG 9 is a schematic diagram showing examples of conveyance objects managed by the conveyor controlling apparatuses according to the first to third embodiments of the present invention.

[0055]

FIG 10 is a schematic view showing an indication on a touch screen of the conveyor controlling apparatus according to the first to third embodiments of the present invention while an image analysis is in progress.

[0056]

FIG 11 is a schematic view showing an example of indication on the touch screen of the conveyor controlling apparatus according to the first to third embodiments of the present invention when the forwarding condition is not satisfied.

[0057]

FIG 12 is a schematic view showing an indication on the touch screen of the conveyor controlling apparatus according to the first to third embodiments of the present invention while entering of an overriding reason is awaited. [0058]

FIG 13 is a schematic view showing an example of indication on the touch screen of the conveyor controlling apparatus according to the first to third embodiments of the present invention when the overriding reason is entered.

[0059]

FIG 14 is a main flowchart schematically showing an operation of the conveyor controlling apparatus according to the first embodiment of the present invention.

[0060]

FIG 15 is a sub-flowchart schematically showing an operation of the conveyor controlling apparatus according to the first embodiment of the present invention.

[0061] FIG 16 is a schematic view showing the conveyor controlling apparatus according to the first and third embodiment of the present invention when the apparatus is installed near the belt conveyor of which the conveying direction is in the left direction.

[0062]

FIG 17 is a schematic left view of the conveyor controlling apparatus according to a first variant of the first embodiment of the present invention when the apparatus is installed adjacent to a belt conveyor.

[0063]

FIG 18 is a schematic block diagram showing a configuration of the conveyor controlling apparatus according to a second variant of the first embodiment of the present invention.

[0064]

FIG 19 is another main flowchart schematically showing an alternative operation of the conveyor controlling apparatus according to the first embodiment of the present invention.

[0065]

FIG 20 is a schematic block diagram showing another configuration of the conveyor controlling apparatus according to a third variant of the first embodiment of the present invention, when the conveyor controlling apparatus is installed near a boundary between two conveyors.

[0066]

FIG 21 is a schematic view showing a conveyor controlling apparatus according to a second embodiment of the present invention when the apparatus is installed near a belt conveyor of which a conveyance object is in a first upstream-assessment zone.

[0067]

FIG 22 is a schematic view showing a conveyor controlling apparatus according to the second embodiment of the present invention when the apparatus is installed near a belt conveyor of which a conveyance object is in a second upstream-assessment zone.

[0068]

FIG 23 is a schematic block diagram of the conveyor controlling apparatus according to the second embodiment of the present invention. [0069]

FIG 24 is a schematic front view of the conveyor controlling apparatus according to the second embodiment of the present invention.

[0070]

FIG 25 shows a schematic left view of the conveyor controlling apparatus according to the second embodiment of the present invention when the apparatus is installed adjacent to a belt conveyor.

[0071]

FIG 26 is a schematic perspective left-back view partially showing the conveyor controlling apparatus according to the second embodiment of the present invention.

[0072]

FIG 27 is a schematic plan view showing a bottom face of a head unit of the conveyor controlling apparatus according to the second embodiment of the present invention.

[0073]

FIG 28 is a schematic view showing an intrusion image captured by the conveyor controlling apparatus according to the first to third embodiments of the present invention.

[0074]

FIG 29 is a schematic view showing an intrusion image captured by the conveyor controlling apparatuses according to the first to third embodiments of the present invention, showing a case where a keep-out zone is reflected in the intrusion image. [0075]

FIG 30 is a main flowchart schematically showing an operation of the conveyor controlling apparatus according to the second embodiment of the present invention.

[0076]

FIG 31(a) is a schematic diagram showing the operation of the conveyor controlling apparatus according to the second embodiment of the present invention, showing a state immediately after a part of a conveyance object enters a first assessment zone.

FIG 31(b) is a schematic diagram showing the operation of the conveyor controlling apparatus according to the second embodiment of the present invention, showing a state in which entire conveyance object enters the first assessment zone.

FIG 31(c) is a schematic diagram showing the operation of the conveyor controlling apparatus according to the second embodiment of the present invention, showing a state in which the conveyance object is stationary in a second assessment zone.

FIG 31(d) is a schematic diagram showing the operation of the conveyor controlling apparatus according to the second embodiment of the present invention, showing a state in which the conveyance object is further forwarded in a downstream direction from the second assessment zone.

[0077]

FIG 32 is a flowchart schematically showing an operation of the conveyor controlling apparatuses according to the second and third embodiments of the present invention, showing a need-of-care control.

[0078]

FIG 33 is a flowchart schematically showing an operation of the conveyor controlling apparatuses according to the second and third embodiments of the present invention, showing an intrusion control. [0079]

FIG 34 is a schematic view showing an example of display on a display unit of the conveyor controlling apparatuses according to the first to third embodiments of the present invention.

[0080]

FIG 35 is a schematic front view of the conveyor controlling apparatus according to the third embodiment of the present invention.

[0081]

FIG 36 is a schematic plan view showing a bottom face of a head unit of the conveyor controlling apparatus according to the third embodiment of the present invention.

[0082]

FIG 37 is a schematic block diagram of the conveyor controlling apparatus according to the third embodiment of the present invention.

[0083]

FIG 38(a) is a schematic diagram showing the operation of the conveyor controlling apparatus according to the third embodiment of the present invention, showing a state immediately after a part of a conveyance object enters a first assessment zone.

FIG 38(b) is a schematic diagram showing the operation of the conveyor controlling apparatus according to the third embodiment of the present invention, showing a state in which entire conveyance object enters the first assessment zone.

FIG 38(c) is a schematic diagram showing the operation of the conveyor controlling apparatus according to the third embodiment of the present invention, showing a state in which the conveyance object is stationary in a second assessment zone.

FIG 38(d) is a schematic diagram showing the operation of the conveyor controlling apparatus according to the third embodiment of the present invention, showing a state in which the conveyance object is further forwarded in a downstream direction from the second assessment zone.

[0084]

FIG 39 is a main flowchart schematically showing an operation of the conveyor controlling apparatus according to the third embodiment of the present invention.

[0085]

FIG 40 is an example of a schematic list of rejection categories may be used by the conveyor controlling apparatuses according to the first to third embodiments of the present invention.

Description of the Preferred Embodiments

[0086]

The first embodiment of the present invention will now be described with reference to the accompanying main drawings FIGs 1 to 20.

[0087]

As shown in FIG 1, a conveyor supervisor 1000 (conveyor controlling apparatus Baz), which is installed in the vicinity of a belt conveyor BC, assesses whether a conveyance state of the conveyance object Cob on the belt conveyor BC satisfies a predetermined forwarding condition. The definition of the 'conveyance object' Cob includes all kinds of objects on the belt conveyor BC. [0088]

That is, not only ordinary objects (e.g. bags or parcels), which are normally carried by the belt conveyor BC, but also other non-ordinary objects, which should theoretically not be carried by the belt conveyor BC, shall also be included in the definition of 'conveyance object' Cob.

[0089]

The conveyor supervisor 1000 is designed for an airport AP. The airport

AP has a non-restricted area (so-called Front of House (FOH)) and a restricted area (so-called Back of House (BOH)) as shown in FIG 2. The conveyor supervisor 1000 can be utilised in both the FOH and the BOH. In this airport AP, each of the conveyor supervisors 1000 is installed at the points shown as "Bazi", "B _aZ 2", and "B _aZ 3" in FIG 2, but in the present embodiment, one of the conveyor supervisors 1000 installed in the place shown as "B _az i" in FIG 2 will be described.

[0090]

An automatic baggage handling system (BHS) Bap installed in the airport AP mainly has an introduction part Bi, a main part Bm, and a transition part Bt.

[0091]

The introduction part Bi has a group of conveyors such as weighing conveyors (not shown in FIGs), holding conveyors (not shown in FIGs) and collector conveyors BCioi installed in first and second islands IS1 and IS2 provided in the FOH in the airport AP. Each of the first and second islands IS1 and IS2 has a plurality of self-baggage drop machines SBD.

[0092]

Each self-baggage drop machine SBD has a weighing conveyor.

Each of the holding conveyors, which is provided at a downstream side of each of the weighing conveyors, receives checked-luggage (conveyance object Cob) forwarded from the weighing conveyor, and automatically drops the forwarded checked-luggage into the collector conveyor BCioi at an appropriate timing.

[0093]

The collector conveyor BCioi is a belt conveyor that continuously runs substantially at a constant speed (e.g. 30 m/min) for conveying the checked- luggage dropped from the holding conveyor to the downstream-side.

[0094]

The main part Bm is the core part of the automatic baggage handling system Bap provided in the BOH of the airport AP. The main part Bm has a group of conveyors such as main conveyors, tilt-tray conveyors, and make-up conveyors.

[0095]

The transition part Bt, which is a part provided between the introduction part Bi and the main part Bm in the BOH of the airport AP, has a transport conveyor BC102 installed near an inlet of BOH. [0096]

The transport conveyor BC102, which is a belt conveyor disposed between a downstream end of the introduction part Bi and an upstream end of the main part Bm, forwards the checked-luggage (conveyance object Cob) sent from the collector conveyor BC101 to the main part Bm.

Each of the conveyors of the automatic baggage handling system Bap is controlled by a conveyor controller CTR.BC (see FIG 4).

[0097]

The conveyor supervisor 1000 is installed in a position abutting onto the collector conveyor BC101 (see FIGs 1 and 8). The conveyor supervisor 1000 assesses whether a conveyance state of the conveyance object Cob carried by the collector conveyor BC101 satisfies a forwarding condition. The forwarding condition includes specific requirements (e.g. size, orientation, and posture of baggage) in terms of the capability of the automatic baggage handling system Bap.

[0098]

As shown in FIGs 3, 6, 7, and 8, the conveyor supervisor 1000 is mainly comprises a main body 101, a pillar 102, and a sensor head (head unit) 103. [0099]

The main body 101 is a pedestal portion of which a bottom face 116 is placed on a floor FL and vertically extends upwards. Further, as shown in FIG 4, a primary Al computer 301, a secondary Al computer 308, a network hub 302, a system computer 303, and an I/O controller 304 are included in the main body 101.

A top face 113 of the main body 101 is formed as a plane extending diagonally downward from a back face 112 to a front face 111. A touch screen (display unit) 305 is provided on the top face 113.

[0100]

The pillar 102 is a metal member extending upward from the main body 101. A status indicator 306 is provided on a front face 117 of the pillar 102.

Further, the pillar 102 is slidably fixed to the main body 101 so that the height of the sensor head 103 is adjustable as shown by the bi-directional arrow Ash in FIGs 6 and 7.

[0101]

The sensor head 103 is an assembly unit fixed to the top end of the pillar 102. As shown in FIG 5, a left three-dimensional camera (capturing device/ upstream capturing device 201) 203, a left RGB camera (capturing device/ upstream capturing device 201) 204, a left light (upstream lighting device) 207, a right three-dimensional camera (capturing device/ downstream capturing device 202) 205, a right RGB camera (capturing device/ downstream capturing device 202) 206, and a right light (downstream lighting device) 208 are fixed onto a bottom face 118 of the sensor head 103.

[0102]

Each of the left RGB camera 204 and the right RGB camera 206 is a Global Shutter RGB type camera with a fixed manual focus lens.

[0103]

Further, as shown in FIG 8, a tip end 119 of the sensor head 103 projects further behind the back face 112 of the main body 101. Hence, when the conveyor supervisor 1000 is installed in such a manner that the back face 112 abuts onto the collector conveyor BCioi, the equipment provided under the bottom face 118 of the sensor head 103 is located above the collector conveyor BC101.

[0104]

Each of the left three-dimensional camera 203 and the left RGB camera

204 captures an image of the conveyance object Cob as an upstream image (first image) when the conveyance object Cob is in an first upstreamassessment zone (upstream assessment zone) Zi _u on the collector conveyor BCioi and while the conveyance object Cob is conveyed by the collector conveyor BCioi in a direction (see the arrow Ac in FIG 4) towards the left three- dimensional camera 203 and the left RGB camera 204.

[0105]

The upstream image captured by the left three-dimensional camera 203 is transmitted in real time to the primary Al computer 301 as data processed three-dimensionally (3D point cloud data) about the conveyance object Cob. The upstream image captured by the left RGB camera 204 is simply forwarded in real time to the primary Al computer 301 as raw data that is not subjected to special processing.

[0106]

As shown in FIG 4, each of the right three-dimensional camera 205 and the right RGB camera 206 captures an image of the conveyance object Cob in a second downstream-assessment zone (downstream assessment zone) Z ₂ d on the collector conveyor BCioi as a downstream image (second image).

[0107]

The downstream image captured by the right three-dimensional camera

205 is transmitted in real time to the primary Al computer 301 as data processed three-dimensionally (3D point cloud data) about the conveyance object Cob. The downstream image captured by the right RGB camera 206 is simply forwarded in real time to the primary Al computer 301 as raw data that is not subjected to special processing.

[0108] As shown in FIG 5, the left three-dimensional camera 203 and the right RGB camera 206 are substantially disposed back to back under the bottom face 118 of the sensor head 103. In addition, the left RGB 204 and the right three-dimensional camera 205 are substantially disposed back to back under the bottom face 118 of the sensor head 103.

[0109]

In the present embodiment, the conveying direction Ac of the collector conveyor BCioi is the right direction (see FIG 1) with respect to the conveyor supervisor 1000. Accordingly, lenses of the left three-dimensional camera 203 and the left RGB camera 204 both face towards an upstream direction (see an arrow Au in FIG 5). On the other hand, lenses of the right three- dimensional camera 205 and the right RGB camera 206 both face towards a downstream direction (see an arrow Ad in FIG 5).

[0110]

As shown in FIGs 1 and 4, the first upstream-assessment zone Z _iu on the collector conveyor BCioi is a virtual area that approximately corresponds to a photographable area of the cameras towards the upstream direction Au (i.e. the left three-dimensional camera 203 and the left RGB camera 204 in this embodiment).

The second downstream-assessment zone Z ₂ d on the collector conveyor BCioi is another virtual area that approximately corresponds to a photographable area of the cameras towards the downstream direction Ad (i.e. the right three-dimensional camera 205 and the right RGB camera 206 in this embodiment).

[0111]

Further, the first upstream-assessment zone Z _iu is provided in the upstream direction Au (i.e. the left side in the present embodiment) with respect to the sensor head 103. On the other hand, the second downstreamassessment zone Z ₂ d is provided in the downstream direction Ad (i.e. the right side in the present embodiment) with respect to the sensor head 103.

This means that the first upstream-assessment zone Z _iu is set in the upstream direction Au with respect to the second downstream-assessment zone Z2d. The second downstream-assessment zone Z ₂ d is set in the downstream direction Ad with respect to the first upstream-assessment zone Zlu.

[0112]

Each of the first upstream-assessment zone Z _iu and the second downstream-assessment zone Z _2d is set as a stationary area regardless of running and stopping of the collector conveyor BCioi. [0113]

A zone-separation distance (see a sign Dz in FIG 4), which is a gap distance between the first upstream-assessment zone Zi _u and the second downstream-assessment zone Z ₂ d, is set in accordance with the processing power of the primary Al computer 301.

The zone-separation distance Dz can be adjusted by changing the height of the sensor head 103, as shown by the arrow Ash in FIG 6.

In addition, the zone-separation distance Dz can be adjusted by individually changing mounting angle of the left three-dimensional camera 203, the left RGB camera 204, the right three-dimensional camera 205, and the right RGB camera 206, with respect to the sensor head 103.

For example, if the processing power of the primary Al computer 301 is relatively higher, then the zone-separation distance Dz may be set relatively shorter. On the other hand, if the processing power of the primary Al computer 301 is relatively lower, then the zone-separation distance Dz may be set relatively longer.

[0114]

As shown in FIG 5, the left light 207 is a lighting unit that illuminates the first upstream-assessment zone Zi _u so that the left three-dimensional camera 203 and the left RGB camera 204 clearly capture the upstream image of the conveyance object Cob.

The right light 208 is another lighting unit that illuminates the second downstream-assessment zone Z2d so that the right three-dimensional camera 205 and the right RGB camera 206 clearly capture the downstream image of the conveyance object Cob.

[0115]

As shown in FIG 4, the primary Al computer (image-analysing module) 301 is a single-board computer that is capable of neural network processing based on image data. The primary Al computer 301 is communicably connected to the network hub 302. [0116]

The primary Al computer 301 is also communicably connected to each of the left three-dimensional camera 203, the left RGB camera 204, the right three-dimensional camera 205, and the right RGB camera 206. The Al computer 301 executes image-processing applications for image classification, object detection in images and division of images, in addition to the neural network processing.

[0117]

Based on a learning model 307 stored in a storage device of the primary Al computer 301, the primary Al computer 301 analyses the upstream image and/or the downstream image, and recognises the 'conveyance state' with regard to the conveyance object Cob on the collector conveyor BCioi. Further, the primary Al computer 301 determines whether the recognised conveyance state satisfies the 'forwarding condition' of the automatic baggage handling system Bap.

In the meantime, the determination on the conveyance state of the conveyance object Cob in the first upstream-assessment zone Zi _u is referred to as the 'primary determination'. On the other hand, the determination on the conveyance state of the conveyance object Cob in the second downstreamassessment zone Z ₂ d is referred to as the 'secondary determination'.

[0118]

The learning model 307 defines an assessment standard. Specifically, the learning model 307 is a machine learning model generated based on a vast amount of raw data (training data), in which a number of conveyance objects Cob are photographed individually, and a learning data set corresponding to the factors of the conveyance objects Cob. The learning model 307 is stored in the storage device of the primary Al computer 301.

[0119]

In addition, the primary Al computer 301 (training module) updates the learning model 307, in response to an overriding command containing an overriding reason entered by an operator via the touch screen 305 based on the entered overriding reason and the upstream image and/or the downstream image.

[0120]

In the meantime, the 'conveyance state' includes various factors with regard to the conveyance object Cob (e.g. dimensions, orientation, type, condition of accessories, presence or absence of overlapping, distance from other conveyance objects Cob, presence or absence of transport tub, open/close state, presence or absence of baggage tag, and so on).

[0121]

For example, when a conveyance object Cob is conveyed by the collector conveyor BCioi, the primary Al computer 301, based on the learning model 307 and the upstream image and/or the downstream image, recognises the type of the conveyance object Cob as a suitcase (hard bags), a duffel bag (soft bags), a backpack (soft bags), a sports bag (soft bags), or others.

[0122]

In addition, when a conveyance object Cob has wheels (movable parts), the primary Al computer 301 recognises whether the conveying direction Ac of the collector conveyor BCioi and a projecting direction of the wheels protrude from the conveyance object Cob are substantially the same. Further, the primary Al computer 301 recognises whether the orientation of the conveyance object Cob is either upright or lying flat. As shown in FIG 9, wheels 901 of a suitcase Cobl protrudes towards in a direction that is the same as the conveying direction Ac of the collector conveyor BCioi.

[0123]

Further, the primary Al computer 301 recognises whether an auxiliary piece such as a handle (movable parts) of a conveyance object Cob is appropriately stowed in the conveyance object Cob. In this regard, FIG 9 shows a handle 902 of a suitcase Cob2 is extended and not properly retracted into the suitcase Cob2, and orientation of the suitcase Cob2 is upright.

[0124]

In addition, FIG 9 shows a suitcase Cob3 that is lying on the collector conveyor BCioi (i.e. the orientation of the suitcase Cob3 is proper), but a handle 903 of the suitcase Cob3 is extended and not properly retracted into the suitcase Cob3.

[0125]

The primary Al computer 301 also recognises whether one conveyance object Cob overlaps another conveyance object Cob (see suitcases Cob4 and Cob5 in FIG 9).

[0126]

In addition, the primary Al computer 301 recognises whether a distance (see the arrow DI in FIG 9) between one conveyance object Cob and another conveyance object Cob is shorter than a threshold distance.

[0127]

In addition, if a baggage tag is affixed to a conveyance object Cob, the primary Al computer 301 recognises the affixed baggage tag. [0128]

In addition, if there is relatively large damage to a conveyance object Cob, the primary Al computer 301 recognises the damage on the conveyance object Cob.

[0129]

Further, the primary Al computer 301 recognises whether a conveyance object Cob is placed in a transport tub.

[0130]

The primary Al computer 301 also recognises whether a shoulder strap is properly bundled if a conveyance object Cob has the shoulder strap.

[0131]

The primary Al computer 301 also recognises whether a conveyance object Cob is properly closed (i.e. the open/close state of the conveyance object Cob).

[0132]

That is, the primary Al computer 301 assesses a conveyance object Cob as an 'improper-state object', which is an object that is not in a state that can be conveyed, if the primary Al computer 301 recognises at least one of:

(i) the size of the conveyance object Cob exceeds the prescribed size (height, width and depth);

(ii) the orientation of the conveyance object Cob is upright;

(iii) the conveying direction Ac of the collector conveyor BCioi and the projecting direction of wheels of the conveyance object Cob are substantially the same;

(iv) a handle is not appropriately retracted in the conveyance object Cob;

(v) there is no baggage tag affixed to the conveyance object Cob;

(vi) the conveyance object Cob is recognised as a soft bag that is not placed in a transport tub;

(vii) a shoulder strap of the conveyance object Cob is not properly bundled; and

(viii) the conveyance object Cob is not properly closed.

[0133]

The primary Al computer 301 also assesses a conveyance object Cob as the 'improper-state object' if the primary Al computer 301 recognises that there are extraneous items (i.e. passports, books, magazines, documents and so on) left on the conveyance object Cob on the collector conveyor BCioi. [0134]

In addition, the primary Al computer 301 assesses a conveyance object Cob on the collector conveyor BCioi as an 'unknown object', if the primary Al computer 301 cannot recognise the conveyance object Cob.

[0135]

Further, the primary Al computer 301 determines that the conveyance state of the conveyance object Cob on the collector conveyor BCioi does not meets the forwarding condition, if the conveyance object Cob is assessed as at least one of the 'improper-state object' and the 'unknown object'. On the other hand, the primary Al computer 301 determines that the conveyance state of the conveyance object Cob on the collector conveyor BCioi meets the forwarding condition, if the conveyance object Cob is assessed as neither the 'improper-state object' nor the 'unknown object'.

[0136]

The primary Al computer 301 and the system computer 303 ascertain the conveying direction Ac of the collector conveyor BCioi based on the installation information entered by an operator through the touch screen 305. Alternatively, the primary Al computer 301 and the system computer 303 may determine that the conveying direction Ac of the collector conveyor BCioi is either rightwards (see FIG 1) or leftwards (see FIG 16) based on the upstream image and/or the downstream image.

[0137] As shown in FIG 4, the network hub 302 is a network switch that supports the TCP/IP protocol having power-over- Ethernet (POE) function. The network hub 302 is communicably connected to each of the primary Al computer 301, the secondary Al computer 308, and the system computer 303. [0138]

The system computer 303 is a computer that holistically control the conveyor supervisor 1000. The system computer 303 is communicably connected to each of the network hub 302, the I/O controller 304, and the touch screen 305.

[0139]

The system computer 303 displays the processing status by the primary Al computer 301 on the touch screen 305. For instance, FIG 10 shows an indication on the touch screen 305 while the image analysis process by the primary Al computer 301 is ongoing.

[0140]

When the primary Al computer 301 determines that the conveyance state of the conveyance object Cob does not meet the forwarding condition, then the system computer 303 displays the problems of the conveyance state on the touch screen 305. For example, an indication on the touch screen 305 is shown in FIG 11 when the primary Al computer 301 recognises: wheels of the conveyance object Cob projects to the conveying direction Ac; the conveyance object Cob requires a transport tub; and orientation of the conveyance object Cob is upright.

[0141]

The system computer 303 has an external-communication module (remote accessing module) which is not shown in the FIGs. The system computer 303 is communicably connected to the Internet 904 via the externalcommunication module. This allows remote operators to connect to the system computer 303 via the Internet 904 and an external-communication module in order to remotely control the conveyor supervisor 1000.

[0142]

The external-communication module (communication module) of the system computer 303 also communicates with other external systems (e.g. SCADA systems, systems under the control of the International Air Transport Association (IATA), and other cargo systems).

[0143]

The system computer 303 has a diagnosing module (diagnosing module) that executes a diagnostic program for self-diagnosing the conveyor supervisor 1000.

The system computer 303 also has a logging module (logging module) that records the activity of the conveyor supervisor 1000.

In the meantime, the diagnostic module and the logging module are not shown in the FIGs.

[0144]

The system computer 303 turns on the status indicator (alarm device) 306 via the I/O controller 304 in accordance with the results of the primary and secondary determinations performed by the primary Al computer 301. As a result, it is possible to visually notify operators of the conveyance state of the conveyance object Cob.

[0145]

The touch screen 305 is an interface device provided on the top face 113 of the main body 101. The touch screen 305 is communicably connected to the system computer 303. The touch screen 305 visually notifies various information to an operator in accordance with the commands from the system computer 303. Further, the touch screen 305 receives inputs from the operator and forward the inputs entered by the operator to the system computer 303.

[0146] The I/O controller 304 is a control unit that is communicably connected to the system computer 303 and the conveyor controller CTR.BC of the automatic baggage handling system Bap. Hence, the I/O controller 304 controls the collector conveyor BCioi via the conveyor controller CT BC in accordance with control commands received from the system computer 303. [0147]

That is, the collector conveyor BCioi is controlled by the I/O controller 304, which works in accordance with the control commands transmitted by the system computer 303 according to the results of the primary and/or secondary determinations.

[0148]

If the primary Al computer 301 determines that the conveyance state of the conveyance object Cob, which is recognised based on the learning model 307 and the upstream image, 'satisfy' (i.e. when the primary determination is a positive result), then the system computer 303 does not actively control over the I/O controller 304. Hence, in this case, the collector conveyor BCioi continues normal operation. As a result, the conveyance object Cob passes through the second downstream-assessment zone Z ₂ d, and continues travelling in the downstream direction Ad.

[0149]

On the other hand, if the primary Al computer 301 determines that the conveyance state of the conveyance object Cob, which is recognised based on the learning model 307 and the upstream image, 'does not satisfy' (i.e. when the primary determination is a negative result), the system computer 303, via the I/O controller 304 and the conveyor controller CT BC, controls the collector conveyor BCioi in such a matter that the conveyance object Cob is stopped in the second downstream-assessment zone Z ₂ d second downstream-assessment zone Z ₂ d. This allows the right three-dimensional camera 205 and the right RGB camera 206 to capture the downstream image of the conveyance object Cob in a stationary state.

[0150]

If the primary Al computer 301 determines that the conveyance state of the conveyance object Cob, which is recognised based on the learning model 307 and the downstream image, 'does not satisfy' (i.e. when the secondary determination is a negative result), the system computer 303 does not actively control over the I/O controller 304. Hence, in this case, the stationary state of the collector conveyor BCioi continues, so the conveyance object Cob is held as it is in the second downstream-assessment zone Z ₂ d second downstreamassessment zone Z _2d . [0151]

Further, if the secondary determination is a negative result, the system computer 303 executes a 'need-of-care control'. This need-of-care control includes a control for indicating the problems of the conveyance state of the conveyance object Cob on the touch screen 305, and another control for turning on the status indicator 306. Details of this need-of-care control will be described later using FIG 15. [0152]

On the other hand, if the primary Al computer 301 determines that the conveyance state of the conveyance object Cob, which is recognised based on the learning model 307 and the downstream image, is 'satisfactory' (i.e. when the secondary determination is a positive result), then the system computer 303 sends a resume signal to the I/O controller 304. Afterwards, the I/O controller 304 restarts, via the conveyor controller CTR.BC, the collector conveyor BCioi in the stationary state in response to the resume signal received from the system computer 303. As a result, the conveyance object Cob on the collector conveyor BCioi travels in the downstream direction Ad. [0153]

In the meantime, a situation in which the conveyance state of the conveyance object Cob does not satisfy the forwarding condition in the first upstream-assessment zone Z _iu , but then satisfies the forwarding condition in the second downstream-assessment zone Z _2d , it could generally be assumed that the conveyance condition of the conveyance object Cob has been corrected with an action taken by an operator because of executing the need- of-care control which enables the operator to recognise that the conveyance object Cob held in the second downstream-assessment zone Z ₂ d does not meet the forwarding condition.

[0154]

However, even if the primary Al computer 301 determines that the conveyance state of the conveyance object Cob does not meet the forwarding condition in the secondary determination, the I/O controller 304 receives the resume signal from the system computer (overriding module) 303 when the system computer 303 executes an 'overriding operation'.

[0155]

This overriding operation is an operation that is executed when an input of the overriding command to the touch screen 305 is completed. That is, when the input of the overriding command to the touch screen 305 is completed, the system computer 303 transmits the resume signal to the I/O controller 304. Then, the I/O controller 304, which received the resume signal, resumes the collector conveyor BCioi in the stationary state via the conveyor controller CTR.BC SO that the conveyance object Cob travels in the downstream direction Ad.

[0156]

In other words, this overriding operation is carried out under a situation where the primary Al computer 301 determines that the conveyance state of the conveyance object Cob suspended in the second downstream-assessment zone Z _2d still does not meet the forwarding condition, but an operator completed the input of the overriding command into the touch screen 305. [0157]

Then, when this overriding operation is executed, the system computer 303 restarts the collector conveyor BCioi via the I/O controller 304 and the conveyor controller CTR.BC in order to send the conveyance object Cob in the downstream direction Ad.

Nevertheless, prior to executing the overriding operation (i.e. for completion of inputting the overriding command), it is required to input a reason (overriding reason) into the touch screen 305.

[0158]

As shown in FIG 12, each of the check boxes for corresponding to the problem of the conveyance state of the conveyance object Cob is indicated on the touch screen 305. Accordingly, an operator, who saw the indication on the touch screen 305, checks an actual state of the conveyance object Cob, which is temporarily stopped in the second downstream-assessment zone Z _2d , and input the overriding reasons into the touch screen 305 by entering ticks into the corresponding checkboxes.

[0159]

For example, assuming that the primary Al computer 301 recognises that the size of a conveyance object Cob held in the second downstreamassessment zone Z _2d exceeds the predetermined specified size (height, width and depth), as a result, the primary Al computer 301 determines that the conveyance object Cob is deemed as an improper-state object, and therefore the primary Al computer 301 determines that the conveyance state of the conveyance object Cob does not satisfy the forwarding condition, but the operator has confirmed that the size of the conveyance object Cob stopped in the second downstream-assessment zone Z _2d is actually within the predetermined specified dimensions.

In this case, the operator enters a tick into the check box of 'Acceptable size', which is one of the overriding reasons displayed on the touch screen 305 as shown in FIG 13, and then touches a "Submit" button indicated on the touch screen 305, otherwise the overriding operation will not be executed. In other words, it is mandatory to include the overriding reason to complete inputting the overriding command.

[0160]

When the overriding operation is executed, the system computer 303 forwards the overriding reason entered into the touch screen 305 to the primary Al computer 301. The primary Al computer 301, which received this overriding reason, updates the learning model 307 based on the upstream and downstream images corresponding to the conveyance object Cob subjected to the overriding operation and the overriding reason received from the system computer 303.

[0161]

The states of the learning model 307 can be restored as needed. For example, backup data of the learning model 307 may be periodically saved and stored in the storage device of the primary Al computer 301. This affords the ability to restore the learning model 307 based on the backup data of any restore points. This is particularly useful in the event of any 'incorrect training data' which had been fed into and has corrupted the learning model 307.

Alternatively, more simply, a factory default version of backup data of the learning model 307 may be stored in the storage device of the primary Al computer 301. According to this arrangement, the learning model 307 can be restored to the factory default state at any time.

[0162]

In other words, even if operators repeatedly enter the overriding reason incorrectly, the learning model 307 can be corrected by restoring the learning model 307 as appropriate.

[0163]

The collector conveyor BCioi has a first photo-eye sensor 209 near the first upstream-assessment zone Zi _u . Further, the collector conveyor BC101 has a second photo-eye sensor 211 near the second downstream-assessment zone Z _2d .

[0164]

Each of the first and second photo-eye sensors 209 and 211 is communicably connected to the I/O controller 304.

Hence, the system computer 303 and the primary Al computer 301 may detect the conveyance object Cob in the first upstream-assessment zone Z _iu by the first photo-eye sensor 209.

[0165]

Likewise, the system computer 303 and the primary Al computer 301 may detect the conveyance object Cob in the second downstream-assessment zone Z _2d by the second photo-eye sensor 211.

[0166]

With the above-mentioned arrangement, the conveyor controlling apparatus according to the first embodiment of the present invention provides the following advantages.

[0167]

The assessment of the conveyance object Cob by the conveyor supervisor 1000 is mainly performed in accordance with a main flowchart shown in FIG 14.

At step S001, when a conveyance object Cob, which is conveyed in the downstream direction Ad by the collector conveyor BCioi, enters into the first upstream-assessment zone Zi _u , then the left light 207 illuminates the first upstream-assessment zone Zi _u and the left three-dimensional camera 203 and the left RGB camera 204 capture an image of the conveyance object Cob as the upstream image.

Then, the primary Al computer 301 performs the image recognition using the learning model 307 for the captured upstream image. [0168]

Afterwards, at step S005, the primary Al computer 301 determines whether the conveyance state of the conveyance object Cob on the collector conveyor BCioi satisfies the forwarding condition based on the recognition result of the upstream image.

[0169]

If the result of the determination at step S005 is positive (see Yes route from step S005), the system computer 303 does not actively control the conveyor controller CTR.BC- That is, in this case, the collector conveyor BCioi continues normal operation. Accordingly, the conveyance object Cob on the collector conveyor BCioi passes through the second downstream-assessment zone Z2d and is further carried in the downstream direction Ad.

[0170]

On the other hand, if the result of the decision at step S005 is negative (see No route from step S005), the system computer 303 instructs the conveyor controller CT BC SO that the conveyance object Cob on the collector conveyor BCioi is stopped within the second downstream-assessment zone Z ₂ d (step S010).

[0171]

At step S015, an image of the conveyance object Cob stationary in the second downstream-assessment zone Z _2d is captured as the downstream image by the right three-dimensional camera 205 and the right RGB camera 206. At this time, the right light 208 illuminates the second downstreamassessment zone Z _2d .

Afterwards, the primary Al computer 301 performs the image recognition using the learning model 307 for the captured downstream image.. [0172]

Thereafter, at step S020, the primary Al computer 301 determines whether the conveyance state of the conveyance object Cob on the collector conveyor BC101 satisfies the forwarding condition based on the recognition result of the downstream image.

If the determination result in this the step S020 is positive (see Yes route from step S020), the system computer 303 sends the resume signal to the I/O controller 304. The I/O controller 304 received the resume signal sends the resume request to the conveyor controller CTR.BC. The conveyor controller CT BC received this resume request restarts the collector conveyor BCioi in the stationary state (step S025). As a result, the conveyance object Cob on is conveyed further in the downstream direction Ad.

[0173]

On the other hand, if the determination at step S020 is negative (see No route from step S020), the system computer 303 does not actively control the I/O controller 304 and the conveyor controller CT BC. That is, in this case, the stationary state of the collector conveyor BCioi continues. As a result, the conveyance object Cob on the collector conveyor BCioi is kept holding as it is in the second downstream-assessment zone Z ₂ d.

[0174]

At this time, at step S030, the system computer 303 executes the 'need- of-care control'. In this need-of-care control, as shown in FIG 15, the step S045 and the step S050 are executed in parallel.

[0175]

Specifically, at step S045, the system computer 303 indicates the problems of the conveyance state of the conveyance object Cob on the touch screen 305 (see FIG 11, for example). This allows an operator to promptly know the problems of the conveyance object Cob held in the second downstream-assessment zone Z ₂ d.

[0176]

At step S050, the system computer 303 turns on the status indicator

306 via the I/O controller 304. This visually provides a call to an operator's attention even if he/she is a short distance away from the conveyor supervisor 1000. That is, it is possible to notify the operator that the conveyance object Cob held in the second downstream-assessment zone Z ₂ d is requiring some assistance.

[0177]

Afterwards, at step S035 of FIG 14, the touch screen 305 waits for an input of the overriding command from an operator. As described above, in order to complete this overriding command input, it is required to input the overriding reason into the touch screen 305. That is, in this the step S035, the operator must enter the overriding reason (see FIG 13) into the touch screen 305, and touch the 'Submit' button on the touch screen 305 to complete the input of the overriding command. Only then is the overriding operation executed (Yes route from step S035).

[0178]

On the other hand, when the input of the overriding command has not been completed (see No route from step S035), the conveyance object Cob on the collector conveyor BCioi is held as it is at the second downstreamassessment zone Z _2d .

[0179]

Thereafter, when the input of the overriding command is completed and the overriding operation is executed (see Yes route from step S035), then at step S040, the system computer 303 forwards the overriding reason entered via the touch screen 305 to the primary Al computer 301. The primary Al computer 301 updates the learning model 307 based on the upstream and downstream images of the conveyance object Cob, which was subject to the overriding operation, and the overriding reason received from the system computer 303.

[0180]

Then, at step S025, the system computer 303 transmits the resume signal to the I/O controller 304. The I/O controller 304 received the resume signal, via the conveyor controller CTR.BC, restarts the collector conveyor BCioi in the stationary state. As a result, the conveyance object Cob is further carried in the downstream direction Ad.

[0181]

The present invention is not limited to the above-described embodiment and the variant thereof, and can be variously modified and implemented. [0182]

As shown in FIG 18, a CCTV camera 212 may be added to the conveyor supervisor 1000 described in the above embodiment. For example, the CCTV camera 212 is communicably connected to the network hub 302. The CCTV camera 212 is supplied electric power from the network hub 302 with POE. Although there are no restrictions on the installation position of the CCTV camera 212, it may be preferable to put the CCTV camera 212 at the sensor head 103.

The CCTV images captured by the CCTV camera 212 are recorded and saved to a data storage server SSVR which is communicably connected to the network hub 302 of the conveyor supervisor 1000.

[0183]

The CCTV images captured by the CCTV camera 212 are processed by a secondary Al computer 308 which is installed in the main body 101 of the conveyor supervisor 1000. The secondary Al computer 308 is a single-board computer that is capable of neural network processing based on image data. The secondary Al computer 308 is communicably connected to the network hub 302.

[0184]

The secondary Al computer 308 is used to detect a suspicious person (e.g. a person who has intruded into an area near the conveyor supervisor 1000 or on the collector conveyor BCioi and a child who has accidentally strayed on the collector conveyor BCioi) by analysing and recognising people captured in the CCTV image, based on a CCTV learning model 308 stored in the storage device of the secondary Al computer 308 and the CCTV images taken by the CCTV camera 212.

The analysis and recognition results of the CCTV images obtained by the secondary Al computer 308 are stored in the data storage server SSVR.

Accordingly, the recognition result of the CCTV images and the CCTV images stored on the data storage server SSVR can be checked as necessary.

Further, when the secondary Al computer 308 detects a suspicious person based on the CCTV images, the system computer 303 may urgently stop the collector conveyor BCioi via the I/O controller 304 and the conveyor control unit CTR.BC.

Also, when the secondary Al computer 308 detects a suspicious person, the system computer 303 may send a signal, which is for notifying that the suspicious person has been detected, to other external systems (e.g. a security system of the airport).

[0185]

The primary Al computer 301 may determine the conveying direction Ac of the collector conveyor BCioi based on the upstream image and/or the downstream image. That is, the primary Al computer 301 judges the conveying direction Ac of the collector conveyor BCioi is either rightwards (see FIG 1) or leftwards (see FIG 16) based on the upstream image and/or the downstream image.

[0186]

In the above embodiment, if the result of the determination at the step S005 shown in FIG 14 is positive (see Yes route from step S005 in FIG 14), the right three-dimensional camera 205 and the right RGB camera 206 does not capture an image (i.e. the downstream image), but it is not limited to this configuration. For example, as shown in FIG 21, even if the primary Al computer 301 determines that a conveyance object Cob on the collector conveyor BCioi satisfies the forwarding condition (see Yes route from step S005 in FIG 19), then the right three-dimensional camera 205 and the right RGB camera 206 may capture an image of the conveyance object Cob as the downstream image.

According to this arrangement, the learning model 307 can be further trained based on a large number of the downstream images.

[0187]

As shown in FIG 20, even if the collector conveyor has a multiple of conveyors such as a first conveyor BCioi-i and a second conveyor BC101-2, the conveyor supervisor 1000 is compatible with these conveyors BC101-1 and BC101- 2 without problems.

That is, in the configuration as shown in FIG 20, the first conveyor BC101- 1 can be operated independently of the second conveyor BC101-2 even while a conveyance object Cob is stopped in the second downstream-side assessment zone Z2d on the second conveyor BC101-2. In other words, even when the operation of the second conveyor BC101-2 is suspended, the first collector conveyor BC101-1 can be operated.

Accordingly, even when the conveyance object Cob is stopped in the second downstream-assessment zone Z ₂ d, it is possible to suppress an impact onto handling process carried out by the automatic baggage handling system Bap.

[0188]

As described in detail above, according to the conveyor controlling apparatus and method of the present invention, it is possible to assess appropriately whether the conveyance state of various types of conveyance objects meets different forwarding conditions for each automatic conveyance system so that the conveyance objects are precisely controlled in accordance with the assessed conveyance state. [0189]

Specifically, according to the conveyor controlling apparatus and method of the present invention, an operator can be swiftly made aware of a conveyance object, which is halted in the second downstream-assessment zone having some problems that does not meet the forwarding condition.

Further, it is also possible to quickly and automatically resume temporarily halted forwarding of the conveyance object in the downstream direction once the problems of the conveyance object are fixed by the operator as the conveyor is automatically restarted when satisfying of the forwarding condition is determined based on the downstream image.

[0190]

In addition, it is possible to improve the accuracy of determination on the conveyance state by recognising the conveyance state of the conveyance object based on the learning model and the upstream image and/or the downstream image, and determining whether the recognised conveyance state satisfies the forwarding condition.

[0191]

In addition, it is possible to improve automatically the accuracy of the learning model by simply carrying out the normal handling procedures for the conveyance object.

That is, an operator is simply required to input the reason why the operator needs to execute the overriding operation, but in fact, the learning model is updated based on this reason (i.e. the overriding reason) and the upstream and/or downstream images. Accordingly, the accuracy of the learning model can be automatically improved without requiring educating operators on how to update the learning model.

[0192]

In addition, it is possible to easily and reliably capture the upstream and downstream images by arranging the upstream capturing device and the downstream capturing device being adjacent to one another and orientated in opposing directions.

[0193]

Further, it is possible to capture clearly the upstream image and the downstream image of the conveyance object from above by mounting the upstream capturing device and the downstream capturing device in the head unit.

[0194]

Further, it is possible to capture clearly the upstream image and the downstream image by illuminating the first upstream-assessment zone and the second downstream-assessment zone.

[0195]

In addition, it is possible to capture easily and clearly the upstream and downstream-side images by providing the upstream and downstream-side lighting devices at the head unit.

[0196]

In addition, the zone-separation distance is adjustable according to the processing capability related to the assessment of the conveyance state.

For example, if the processing power of the image-analysing module is relatively higher, then the zone-separation distance may be set relatively shorter. On the other hand, if the processing power of the image-analysing module is relatively lower, then the zone-separation distance may be set relatively longer.

[0197]

Specifically, in the above-described first embodiment, the collector conveyor BCioi continues its operation while the primary Al computer 301 determines whether the conveyance state of the conveyance object Cob on the collector conveyor BCioi satisfies the forwarding condition based on the learning model 307 and the upstream image (i.e. while the primary determination is carried out). As a result, the conveyance object Cob continues to move in the downstream direction Ad even while the processing of the primary determination is ongoing. However, in this case, the right three-dimensional camera 205 and the right RGB camera 206 could not capture the conveyance object Cob, if a result of the primary determination came out after the conveyance object Cob has passed the second downstreamassessment zone Z2d and further moved in the downstream direction Ad. [0198]

For this reason, the zone distance is set taking into account the processing power of the image-analysing module (primary Al computer 301). [0199]

In addition, it is possible to seek quickly and reliably a possible assistance from an operator by visually and/or aurally notifying an operator of a result of the determination whether the conveyance state of the conveyance object meets the forwarding condition.

This also allows the operator to do other work until the operator gets a visual and/or aural notification that the forwarding condition has not been satisfied. Accordingly, the work efficiency of the operator may be improved.

[0200]

The second embodiment of the present invention will now be described using mainly FIGs 21 to 34.

[0201]

As shown in FIGs 21 and 22, a conveyor supervisor 2000 (conveyor controlling apparatus Baz), which is installed in the vicinity of a belt conveyor BC, assesses whether a conveyance state of the conveyance object Cob on the belt conveyor BC satisfies a predetermined forwarding condition.

[0202]

The conveyor supervisor 2000 is designed for an airport AP in the same manner of the conveyor supervisor 1000 according to the first embodiment. The conveyor supervisor 2000 can also be utilised in both the FOH and the BOH of the airport AP. In this airport AP, each of the conveyor supervisors 2000 is installed at the points shown as "Bazl", "Baz2", and "Baz3" in FIG 2, but in the present embodiment, one of the conveyor supervisors 2000 installed in the place shown as "Bazl" in FIG 2 will be described.

[0203]

The conveyor supervisor 2000 is installed in a position abutting onto the collector conveyor BCioi (see FIG 25). The conveyor supervisor 2000 assesses whether a conveyance state of the conveyance object Cob carried by the collector conveyor BCioi satisfies a forwarding condition which is subjected to the automatic baggage handling system Bap.

[0204]

As shown in FIGs 24 to 26, the conveyor supervisor 2000 comprises a main body 2101, a pillar 2102, and a sensor head (head unit) 2103.

[0205]

The main body 2101 is a pedestal portion of which a bottom face 2116 is placed on a floor FL and vertically extends upwards. Further, as shown in FIG 23, a primary computer 2301, an Al vision processor 2311, an Ethernet switch 2302, a secondary computer 2308, and an I/O controller 2304 are included in the main body 2101.

A DC power supply unit is also provided in the main body 2101, but illustration is omitted.

A top face 2113 of the main body 2101 is formed as a plane extending diagonally downward from a back face 2112 to a front face 2111. A touch screen (display unit) 2305 is provided on the top face 2113.

[0206]

The pillar 2102 is a member extending upward from the main body 2101. A status indicator 2306 is provided on a front face 2117 of the pillar 2102. Further, the pillar 2102 is slidably fixed to the main body 2101 so that the height of the sensor head 2103 is adjustable as shown by the bi-directional arrow Ash in FIG 24.

[0207]

The sensor head 2103 is an assembly unit fixed to the top end of the pillar 2102. As shown in FIG 27, a base-plate 2113 is fixed to the bottom face 2118 of the sensor head 2103. A left three-dimensional camera (capturing device/ upstream capturing device) 2203, a left RGB camera (capturing device/ upstream capturing device) 2204, and a left light (upstream lighting device) 2207 are individually and adjustably mounted to the base-plate 2113 of the sensor head 2103 via respective ball joints 2212LI, 2212i_2,.and 2212L3. [0208]

A CCTV camera (intrusion camera) 2309 is a security camera and its frame rate can be adjusted.

The images (intrusion image) captured by the CCTV camera 2309 are transmitted to the secondary computer 2308 via the Ethernet switch 2302 as image stream data. [0209]

The CCTV camera 2309 is adjustably mounted to the base-plate 2113 via a ball-joint 2212 _M I and a swing-shaft 2212 _M 2. In this example shown in FIGs 25 and 27, the CCTV camera 2309 is directed toward the rear direction Ab of the conveyor supervisor 2000. However, the monitoring-focal direction can be adjusted as needed so that the CCTV camera 2309 selectively monitors the upstream direction Au or the downstream direction Ad. [0210]

Further, as shown in FIG 25, a front edge 2119 of the sensor head 2103 projects further behind the back face 2112 of the main body 2101. Hence, when the conveyor supervisor 2000 is installed in such a manner that the back face 2112 abuts onto the collector conveyor BCioi, the equipment provided under the bottom face 2118 of the sensor head 2103 is located above the collector conveyor BCioi.

[0211]

Each of the left three-dimensional camera 2203 and the left RGB camera 2204 captures an image of the conveyance object Cob as a 'first image' when the conveyance object Cob is in an first upstream-assessment zone Zi _u on the collector conveyor BCioi and while the conveyance object Cob is conveyed by the collector conveyor BCioi in a direction (see the arrow Ac in FIG 23) towards the left three-dimensional camera 2203 and the left RGB camera 2204. [0212]

In addition, each of the left three-dimensional camera 2203 and the left RGB camera 2204 captures an image of the conveyance object Cob stationary in a second upstream-assessment zone Z _2u on the collector conveyor BCioi as a 'second image'. [0213]

Each of the images captured by the left three-dimensional camera 2203 is transmitted in real time to the primary computer 2301 as data processed three-dimensionally (3D point cloud data) about the conveyance object Cob. Each of the images of the conveyance object Cob captured by the left RGB camera 2204 is transmitted in real time to the primary computer 2301 as raw data that is not subjected to special processing.

[0214]

In the present embodiment, the conveying direction Ac of the collector conveyor BCioi is the right direction (see FIGs 21 and 22, for example) with respect to the conveyor supervisor 2000. Accordingly, each capturing direction of the left three-dimensional camera 2203 and the left RGB camera 2204 is directed towards an upstream direction (see an arrow Au in FIG 27). [0215]

The left RGB camera 2204 is a Global Shutter RGB type camera with a fixed manual focus lens which is directed to the assessment zone Za including both of the first upstream-assessment zone Z _iu and the second upstreamassessment zone Z _2u .

The left RGB camera 2204 is communicably connected to the primary computer 2301 with a USB3 or similar cable.

The image data transmitted from the left RGB camera 2204 is forwarded by the primary computer 2301 to the Al vision processor 2311 for the Al analysis carried out by the Al vision processor 2311.

[0216]

The Al vision processor 2311 is a computer dedicated for neural net artificial intelligence (Al) processing.

The Al vision processor 2311 has a memory in which an image-analysing module 2401 and a learning model 2405 are stored.

The Al vision processor 2311 receives the data of the images captured by each of the left three-dimensional camera 2203 and the left RGB camera 2204 from the primary computer 2301.

[0217]

The image-analysing module 2401, which is a software module, implements the learning model 2405 which has been preliminarily trained.

The image-analysing module 2401 assesses and determines whether the conveyance state of the conveyance object Cob satisfies the forwarding condition based on the received image data and the trained learning model 2405.

[0218]

The learning model 2405 defines an assessment standard. Specifically, the learning model 2405 is a machine learning model generated based on a vast amount of raw data (training data), in which a number of conveyance objects are photographed individually, and a learning data set corresponding to the factors of the conveyance objects. The Al vision processor 2311 transmits results of the assessment and determination process to the primary computer 2301.

[0219]

The training module 2404 of the primary computer 2301 updates the learning model 2405, when the overriding operation (described later) is executed, based on the overriding reason entered into the touch screen 2305 and the image corresponding to the conveyance object Cob subjected to the overriding operation.

[0220]

The training module 2404 restores the states of the learning model 2405 as needed.

For example, backup data of the learning model 2405 may be periodically saved and stored in the storage device of the primary computer 2301. This affords the ability to restore the learning model 2405 based on the backup data of any restore points. This is particularly useful in the event of any 'incorrect training data' which has been fed into and has corrupted the learning model 2405.

Alternatively, a factory default version of backup data of the learning model 2405 may be stored in the storage device of the primary computer 2301. According to this arrangement, the learning model 2405 can be restored to the factory default state at any time.

[0221]

In other words, even if operators repeatedly enter the overriding reason incorrectly, the training module 2404 can correct the learning model 2405 by restoring the training module 2404 as appropriate.

[0222]

As shown in FIG 23, the assessment zone Za on the collector conveyor BCioi is an area that approximately corresponds to a photographable area of the cameras towards the upstream direction Au (i.e. the left three-dimensional camera 2203 and the left RGB camera 2204 in this embodiment).

The assessment zone Za has the first upstream-assessment zone Z _iu and the second upstream-assessment zone Z _2u . [0223]

The assessment zone Za is provided in the upstream direction Au (i.e. the left side in the present embodiment) with respect to the sensor head 2103 shown in FIG 24.

Each of the first upstream-assessment zone Z _iu and the second upstream-assessment zone Z _2u is set as a stationary area regardless of running and stopping of the collector conveyor BCioi.

[0224]

The first upstream-assessment zone Zi _u is provided in the upstream direction Au with respect to the second upstream-assessment zone Z _2u .

The first upstream-assessment zone Z _iu is an area in which an image of the conveyance object Cob (i.e. object image) is captured as a 'first image' by each of the left three-dimensional camera 2203 and the left RGB camera 2204 while the conveyance object Cob is conveyed by the collector conveyor BCioi. [0225]

As schematically shown in FIG 23, a first photo-eye sensor 2209 is provided beside an upstream edge of the first upstream-assessment zone Z _iu .

The first photo-eye sensor 2209 is an optical sensor which detects that the conveyance object Cob has entered the first upstream-assessment zone Ziu.

[0226]

The first photo-eye sensor 2209 is connected to the I/O controller 2304 with a digital I/O cable.

Further, the I/O controller 2304 is communicably connected to the primary computer 2301 via the Ethernet switch 2302.

Hence, the primary computer 2301 can immediately detect that the conveyance object Cob enters into the first upstream-assessment zone Zi _u by the first photo-eye sensor 2209.

[0227]

The second upstream-assessment zone Z _2u is provided in the downstream direction Ad with respect to the first upstream-assessment zone Ziu.

The second upstream-assessment zone Z _2u is an area in which the object image is captured as a 'second image' by each of the left three-dimensional camera 2203 and the left RGB camera 2204 when the conveyance object Cob is stationary on the collector conveyor BCioi since the collector conveyor BCioi is stopped by the primary computer 2301.

Further, in this second upstream assessment zone Z _2U , an operator can attend on the conveyance object Cob stationary on the collector conveyors BCioi.

[0228]

A second photo-eye sensor 2211 is provided beside an upstream edge of the second upstream-assessment zone Z _2u .

The second photo-eye sensor 2211 is an optical sensor detecting that the conveyance object Cob has entered the second upstream-assessment zone Z _2u .

[0229]

The second photo-eye sensor 2211 is connected to the I/O controller 2304 with a digital I/O cable.

Further, the I/O controller 2304 is communicably connected to the primary computer 2301 via the Ethernet switch 2302.

Hence, the primary computer 2301 can immediately detect that the conveyance object Cob enters the second upstream-assessment zone Z _2u by the second photo-eye sensor 2211.

[0230] A length of the first upstream-assessment zone Zi _u (i.e. a first-zone length - see a sign 'L _Z i' in FIG 23) can be adjustably set in accordance with a processing power of the primary computer 2301 and the Al vision processor 2311 as well as a running speed of the collector conveyor BCioi.

The first-zone length Lzi can be adjusted by changing the height of the sensor head 2103, as shown by the arrow Ash in FIG 24.

In addition, the first-zone length Lzi can be adjusted by individually changing mounting angle of the left three-dimensional camera 2203 and the left RGB camera 2204 with respect to the sensor head 2103.

[0231]

As an example, if the processing power of the primary computer 2301 and the Al vision processor 2311 is relatively higher, then the first-zone length Lzi may be set relatively shorter with considering the length of the assessment zone Za and the length of the second upstream-assessment zone Z _2u . On the other hand, if the processing power of the primary computer 2301 and the Al vision processor 2311 is relatively lower, then the first-zone length Lzi may be set relatively longer with considering the length of the assessment zone Za and the length of the second upstream-assessment zone Z _2u .

As another example, if the running speed of the collector conveyor BCioi is relatively slower, then the first-zone length Lzi may be set relatively shorter. On the other hand, if the running speed of the collector conveyor BCioi is relatively faster, then the first-zone length Lzi may be set relatively longer. [0232]

As shown in FIG 27, the left light 2207 is a LED lighting unit that illuminates the first upstream-assessment zone Z _iu and the second upstreamassessment zone Z _2u so that the left three-dimensional camera 2203 and the left RGB camera 2204 can clearly capture the first and second images of the conveyance object Cob.

[0233] As shown in FIG 23, the primary computer 2301 is a main computer of the conveyor supervisor 2000.

[0234]

The primary computer 2301 is connected to the Ethernet switch 2302 with an Ethernet cable. Hence, the primary computer 2301 is communicably connected to the I/O controller 2304 via the Ethernet switch 2302.

The primary computer 2301 is connected to each of the left three- dimensional camera 2203 and the left RGB camera 2204 with a USB3 or similar cable.

The primary computer 2301 is connected to the touch screen 2305 with a HDMI cable.

The primary computer 2301 is communicably connected to the Internet as well as other external systems.

[0235]

The primary computer 2301 has a storage device in which software modules such as a conveyer-controlling module 2402, an overriding module 2403 and a training module 2404 are stored.

[0236]

The conveyer-controlling module 2402 stops the collector conveyor BCioi via a conveyor controller (not shown in FIGs) of the automatic baggage handling system Bap such that the conveyance object Cob is positioned stationary within the second upstream-assessment zone Z2 _U if the imageanalysing module 2401 of the Al vision processor 2311 determines based on the first image that the conveyance state of the conveyance object Cob does not satisfy the forwarding condition.

Further, via the conveyor controller of the automatic baggage handling system Bap, the conveyer-controlling module 2402 restarts the collector conveyor BCioi, which is stopped by the conveyer-controlling module 2402, if the image-analysing module 2401 determines based on the second image that the conveyance state of the conveyance object Cob satisfies the forwarding condition.

In the meantime, the determination on the conveyance state of the conveyance object Cob in the first upstream-assessment zone Zi _u is referred to as the 'primary determination'. On the other hand, the determination on the conveyance state of the conveyance object Cob in the second upstreamassessment zone Z _2u is referred to as the 'secondary determination'.

[0237]

The overriding module 2403 executes an 'overriding control' that forcibly restarts the collector conveyor BCioi from the stationary state via the conveyor controller of the automatic baggage handling system Bap. This overriding operation is an operation that is run when an input of the overriding command to the touch screen 2305 is completed.

That is, when the input of the overriding command to the touch screen 2305 is completed, the overriding module 2403 executes the overriding control. [0238]

In other words, this overriding operation is carried out in a situation where the image-analysing module 2401 of the Al vision processor 2311 determined that the conveyance state of the conveyance object Cob did not satisfy the forwarding condition, but an operator completed inputting the overriding command into the touch screen 2305.

[0239]

As shown in FIG 12, each of the check boxes for corresponding to the problems of the conveyance state of the conveyance object Cob is indicated on the touch screen 2305. Accordingly, an operator, who saw the indication on the touch screen 2305, checks an actual state of the conveyance object Cob, which is temporarily stopped in the second upstream-assessment zone Z _2u , and inputs the at least one or more overriding reasons into the touch screen 2305 by entering a tick or ticks into the corresponding checkbox or checkboxes. [0240]

For example, assuming that the image-analysing module 2401 of the Al vision processor 2311 recognises that the size of a conveyance object Cob exceeds the predetermined specified size (height, width and depth), as a result, the image-analysing module 2401 determines that the conveyance state of this conveyance object Cob does not satisfy the forwarding condition, but the operator has confirmed that the size of this conveyance object Cob stopped is actually within the predetermined specified dimensions.

In this case, the operator enters a tick into the check box of 'Acceptable size', which is one of the overriding reasons displayed on the touch screen 2305 as shown in FIG 13, and then touches a "Submit" button indicated on the touch screen 2305, otherwise the overriding operation will not be executed. In other words, it is mandatory to include the overriding reason to complete inputting the overriding command.

[0241]

The training module 2404 updates the learning model learning model 2405, in response to an overriding command containing an overriding reason entered by an operator via the touch screen 2305 based on the entered overriding reason and the first image and/or the second image.

[0242]

The training module 2404 sequentially records the results of the primary and secondary determinations by the conveyor control unit 2402 and the first and second images corresponding to these determinations into a storage server (not shown in FIGs). This storage server is an external server that is communicably connected to the conveyor supervisor 2000.

[0243]

The 'conveyance state' includes various factors with regard to the conveyance object Cob (e.g. dimensions, orientation, type, condition of accessories, presence or absence of overlapping, distance from other conveyance objects Cob, presence or absence of transport tub, open/close state, presence or absence of baggage tag, and so on).

[0244]

For example, when a conveyance object Cob is conveyed by the collector conveyor BCioi, the image-analysing module 2401 of the Al vision processor 2311, based on the learning model 2405 and the first image and/or the second image, recognises the type of the conveyance object Cob as a suitcase (hard bags), a duffel bag (soft bags), a backpack (soft bags), a sports bag (soft bags), or others.

[0245]

In addition, when a conveyance object Cob has wheels (movable parts), the image-analysing module 2401 of the Al vision processor 2311 recognises whether the conveying direction Ac of the collector conveyor BCioi and a projecting direction of the wheels protrude from the conveyance object Cob is substantially the same. Further, the image-analysing module 2401 recognises whether the orientation of the conveyance object Cob is either upright or lying. As shown in FIG 9, wheels 901 of a suitcase Cobl protrudes towards in a direction that is the same as the conveying direction Ac of the collector conveyor BCioi.

[0246]

Further, the image-analysing module 2401 of the Al vision processor 2311 recognises whether an auxiliary piece such as a handle (movable parts) of a conveyance object Cob is appropriately stowed in the conveyance object Cob. In this regard, FIG 9 shows a handle 902 of a suitcase Cob2 is extended and not properly stowed away into the suitcase Cob2, and orientation of the suitcase Cob2 is upright.

[0247]

In addition, FIG 9 shows a suitcase Cob3 that is lying on the collector conveyor BCioi (i.e. the orientation of the suitcase Cob3 is proper), but a handle 903 of the suitcase Cob3 is extended and not properly stowed away into the suitcase Cob3.

[0248]

The image-analysing module 2401 also recognises whether one conveyance object Cob overlaps another conveyance object Cob (see suitcases Cob4 and Cob5 in FIG 9).

[0249]

In addition, the image-analysing module 2401 recognises whether a distance (see the arrow DI in FIG 9) between one conveyance object Cob5 and another conveyance object Cob6 is shorter than a threshold distance.

[0250]

If a baggage tag is affixed to a conveyance object Cob, the imageanalysing module 2401 may recognise the affixed baggage tag.

[0251]

If there is relatively large damage to a conveyance object Cob, the image-analysing module 2401 may recognise the damage on the conveyance object Cob.

[0252]

Further, the image-analysing module 2401 recognises whether a conveyance object Cob is placed in a transport tub.

[0253]

The image-analysing module 2401 also recognises whether a shoulder strap is properly bundled/stowed if a conveyance object Cob has the shoulder strap.

[0254]

The image-analysing module 2401 may recognise whether a conveyance object Cob is properly closed (i.e. the open/close state of the conveyance object Cob).

[0255] That is, for example, the image-analysing module 2401 of the Al vision processor 2311 assesses that a conveyance state of a conveyance object Cob does not satisfy the forwarding condition (i.e. the conveyance state corresponds at least one of the rejection categories), if the image-analysing module 2401 recognises at least one of:

(i) the size of a conveyance object Cob exceeds the prescribed size (height, width and depth);

(ii) the orientation of a conveyance object Cob is upright;

(iii) the conveying direction Ac of the collector conveyor BCioi and the projecting direction of wheels of a conveyance object Cob are substantially the same;

(iv) a handle is not appropriately stored in a conveyance object Cob;

(v) there is no baggage tag affixed to a conveyance object Cob;

(vi) a conveyance object Cob is recognised as a soft bag that is not placed in a transport tub;

(vii) a shoulder strap of a conveyance object Cob is not properly bundled/stowed;

(viii) a conveyance object Cob is not properly closed;

(ix) there are extraneous items (i.e. passports, books, magazines, documents and so on) left on a conveyance object Cob; and

(x) a conveyance object Cob on the collector conveyor BCioi is an 'unknown object'.

[0256]

The conveyer-controlling module 2402 of the primary computer 2301 displays the processing status by the image-analysing module 2401 of the Al vision processor 2311 on the touch screen 2305. For instance, FIG 10 shows an indication on the touch screen 2305 while the image analysis process by the image-analysing module 2401 is ongoing.

[0257] When the forwarding condition is not met by the conveyance state of the conveyance object Cob, then the conveyer-controlling module 2402 of the primary computer 2301 displays the problems of the conveyance state on the touch screen 2305. For example, an indication on the touch screen 2305 is shown in FIG 11 when the image-analysing module 2401 of the Al vision processor 2311 recognises: wheels of the conveyance object Cob projects to the conveying direction Ac; the conveyance object Cob requires a transport tub; and orientation of the conveyance object Cob is upright.

[0258]

As shown in FIG 24, the touch screen 2305 is an interface device provided on the top face 2113 of the main body 2101. The touch screen 2305 is communicably connected to the primary computer 2301 by the HDMI cable.

The touch screen 2305 visually notifies various information to an operator in accordance with the commands from the touch screen 2305. Further, the touch screen 2305 receives inputs from the operator and forward the inputs entered by the operator to the touch screen 2305.

[0259]

The primary computer 2301 ascertains the conveying direction Ac of the collector conveyor BCioi based on the installation information entered by an operator through the touch screen 2305. Alternatively, the primary computer 2301 may determine that the conveying direction Ac of the collector conveyor BCioi is either rightwards or leftwards based on the first image and/or the second image.

[0260]

As shown in FIG 23, the Ethernet switch 2302 is a network switch that supports the TCP/IP protocol having power-over- Ethernet (POE) function. The Ethernet switch 2302 is communicably connected to each of the primary computer 2301, the Al vision processor 2311, the secondary computer 2308, and the CCTV camera 2309 with an Ethernet cable. [0261]

The status indicator 2306 is an LED light unit that illuminates red, green, or amber for easy identification of the operational state of the conveyor supervisor 2000.

Specifically, the conveyer-controlling module 2402 of the primary computer 2301 controls the status indicator 2306 via the I/O controller 2304 in accordance with the working state of the conveyor supervisor 2000 as summarised below.

Working State LED Illumination

System Unhealthy Amber/ ed Steady

Powered Off Off

System Healthy Green

Non-compliant baggage detected Amber Flashing

Intrusion detected Red Flashing

According to this, it is possible to visually notify operators of the conveyance state of the conveyance object Cob.

[0262]

The I/O controller 2304 is a control unit that is communicably connected with a digital I/O cable to each of the first photo-eye sensor 2209, the second photo-eye sensor 2211, the status indicator 2306, the left light 2207, and a conveyor controller (not shown in FIGs) of the automatic baggage handling system Bap.

The I/O controller 2304 is also connected to the Ethernet switch 2302 with an Ethernet cable. Hence, the primary computer 2301 controls the collector conveyor BCioi via the I/O controller 2304.

[0263]

The secondary computer 2308 is a supplementary computer that controls the processing related to an 'intrusion control' among the processes of the conveyor supervisor 2000. The secondary computer 2308 is connected to the Ethernet switch 2302 with an Ethernet cable.

The secondary computer 2308 has a storage device in which software modules such as an intrusion-analysing module 2407 are stored.

The intrusion-analysing module 2407 implements an intrusion learning model (not shown in FIGs) which has been preliminarily trained for the intrusion control.

[0264]

The secondary computer 2308 continuously receives image stream data fed from the CCTV camera 2309 via the Ethernet switch 2302.

The secondary computer 2308 runs dedicated human detection application algorithms implemented in the intrusion-analysing module 2407.

The intrusion-analysing module 2407 is designed for detecting the presence of humans in a region of interest which corresponds to a 'keep-out zone'. The region of interest is calibrated to the collector conveyor BCioi in view of the image stream captured by the CCTV camera 2309.

The keep-out zone is an area extending along the collector conveyor BCioi and including at least the collector conveyor BCioi.

[0265]

For example, as shown in FIG 28, an infant H _in f may accidentally enter a restricted area in which the collector conveyor BCioi is installed.

In this case, the intrusion-analysing module 2407 of the secondary computer 2308 detects a presence of the infant H _in f within the keep-out zone ZK as shown as shown in FIG 29.

In response to the detection of the infant Hinf, the intrusion-analysing module 2407 of the secondary computer 2308 immediately stops the collector conveyor BCioi via the conveyer-controlling module 2402 of the primary computer 2301.

[0266] With the above-mentioned arrangement, the conveyor controlling apparatus according to the second embodiment of the present invention provides the following advantages.

[0267]

The analysis control of the conveyance object Cob by the conveyor supervisor 2000 is mainly performed in accordance with a main flowchart shown in FIG 30.

Note that, in parallel with the analysis control of the conveyance object Cob shown in FIG. 201, the intrusion control shown in FIG. 207 is also executed, which will be described later.

[0268]

As shown in FIG 31(a), the conveyer-controlling module 2402 recognises that the conveyance object (suitcase) Cob3 has just entered the first upstream-assessment zone Z _iu in response to detection by the first photoeye sensor 2209 that the conveyance object Cob3 which is conveyed from the upstream direction Au to the downstream direction Ad by the collector conveyor BCioi (Yes route from step S205 in FIG 30).

Immediately subsequent to this, the conveyer-controlling module 2402 stays on standby to make a very short delay for ensuring that the entire conveyance object Cob enters the first upstream-assessment zone Z _iu (step S210).

[0269]

At this time, the conveyer-controlling module 2402 of the primary computer 2301 obtains the first image of the conveyance object Cob3 captured by each of the left three-dimensional camera 2203 and the left RGB camera 2204 (step 215). During this time, the collector conveyor BCioi is still running, hence, the conveyance object Cob3 keeps travelling in the downstream direction Ad.

[0270] Then, the image-analysing module 2401 of the Al vision processor 2311 determines whether the conveyance state of the conveyance object Cob3 satisfies the forwarding condition based on the first image captured by each of the left three-dimensional camera 2203 and the left RGB camera 2204 (steps S215 and S220 in FIG 30).

Also, at this time, the training module 2404 records the analysis results of the conveyance object Cob3 carried out by the image-analysing module 2401 of the Al vision processor 2311 and the captured first image of the conveyance object Cob3 in the external storage server (steps S215 and S220 in FIG 30).

[0271]

If the conveyance state of the conveyance object Cob3 satisfies the forwarding condition (Yes route from step S220 in FIG 30), the conveyercontrolling module 2402 does not take action on the automatic baggage handling system Bap. Hence, the collector conveyor BCioi continues running and the conveyance object Cob3 is conveyed to the downstream direction Ad. [0272]

On the other hand, if the conveyance state of the conveyance object Cob3 does not satisfy the forwarding condition (No route from step S220 in FIG 30), the conveyer-controlling module 2402 stops the collector conveyor BCioi via the conveyor controller of the automatic baggage handling system Bap so that the conveyance object Cob3 is stationary within the second upstream-assessment zone Z _2u (step S230 of FIG 30 - see also FIG 31(c)). [0273]

The conveyer-controlling module 2402 recognises that the conveyance object Cob3 has entered the second upstream-assessment zone Z _2u in response to the second photo-eye sensor 2211 detecting the conveyance object Cob3.

Nevertheless, the conveyer-controlling module 2402 is set to make a very short delay to ensure that the entire conveyance object Cob3 enters the second upstream-assessment zone Z _2u (step S225 in FIG 30), then stops the collector conveyor BCioi (step s230).

[0274]

After that, at step S235 of FIG 30, the conveyer-controlling module 2402 executes a 'need-of-care control'. In this need-of-care control, as shown in FIG 32, the steps S400 and S405 are executed in parallel.

[0275]

At step S400, the conveyer-controlling module 2402 indicates on the touch screen 2305 the determined problems with respect to the conveyance state of the conveyance object Cob3 (see FIG 11, for example). This control can quickly and specifically notify an operator that there is some problem with the conveyance object Cob3 that is stopped in the second upstreamassessment zone Z _2u .

[0276]

Further, at the step S405 shown in FIG 32, the conveyer-controlling module 2402 illuminates the status indicator 2306 via the I/O controller 2304 in the colour and pattern that correspond to the working state of the conveyor supervisor 2000.

According to this arrangement, even if an operator is a short distance from the conveyor supervisor 2000, the operator can be visually alerted. In other words, it is possible to swiftly notify the operator that some assistance is required for the conveyance object Cob3 stopped in the second upstreamassessment zone Z _2u .

As a result, the problems of the conveyance object Cob3 are solved (step S240 in FIG 30).

[0277]

At this time, if the operation mode of the conveyor supervisor 2000 with regard to restarting the collector conveyor BCioi is 'automatic' (Automatic route from step S245), the conveyer-controlling module 2402 obtains the second image of the conveyance object Cob3 captured by each of the left three-dimensional camera 2203 and the left RGB camera 2204 (step S270 in FIG 30).

Further, the image-analysing module 2401 of the Al vision processor 2311 determines whether the conveyance object Cob3 satisfies the forwarding condition based on the second image captured by the left three-dimensional camera 2203 and the left RGB camera 2204 (steps S270 and S275 in FIG 30). [0278]

Also, at this time, the training module 2404 records the analysis results of the conveyance object Cob3 carried out by the image-analysing module

2401 of the Al vision processor 2311, the determination results as to whether or not the conveyance state of the conveyance object Cob3 satisfies the forwarding condition, and the captured second image of the conveyance object Cob3 in the external storage server (steps S270 and S275 of FIG 30).

[0279]

If the conveyance state of the conveyance object Cob3 satisfies the forwarding condition (Yes route from S275), the conveyer-controlling module

2402 restarts the collector conveyor BCioi via the conveyor controller of the automatic baggage handling system Bap (step S280). At this time, in the example shown in FIG 31(d), the handle 903 is now retracted into the conveyance object (suitcase) Cob3 and is being conveyed to the downstream direction Ad.

[0280]

On the other hand, if the operation mode of the conveyor supervisor 2000 with regard to restarting the collector conveyor BCioi is 'manual' (Manual route from step S245), the conveyer-controlling module 2402 indicates a 'Force-in' button and a 'Retry' button on the touch screen 2305. FIG 34 shows an example of the display on the touch screen 2305 at this time. [0281]

If the Force-in button is touched by an operator (see Force-in route from step S250 in FIG 30), the overriding module 2403 indicates on the touch screen 2305 check boxes corresponding to the found problems with respect to the conveyance object Cob3 (see FIG 12).

In other words, this situation could be, for example, that although an operator has checked the conveyance object Cob3 at step S240, no problem was actually found, and therefore the operator believes that the conveyance object Cob3 should continue travelling in the downstream direction Ad. [0282]

At this time, the operator appropriately checks the check boxes corresponding to the reasons why there is no problem with the conveyance object Cob3 (i.e. 'overriding reason'), then the operator touches a 'Submit' button indicated on the touch screen 2305 (step S255).

[0283]

Also, at this time, the training module 2404 records the analysis results of the conveyance object Cob3 carried out by the image-analysing module 2401 of the Al vision processor 2311, the determination results as to whether or not the conveyance state of the conveyance object Cob3 satisfies the forwarding condition, the captured first image of the conveyance object Cob3, and the overriding reason entered through the touch screen 2305 in the external storage server (step S260).

In addition, the training module 2404 may apply a learning control to the learning model 2405 based on the entered overriding reason.

[0284]

In short, in a situation where the image-analysing module 2401 of the Al vision processor 2311 has determined that the conveyance state of the conveyance object Cob3 does not satisfy the forwarding condition, but an operator has completed inputting the overriding command through the touch screen 2305, therefore, the overriding module 2403 executes the overriding control to forcibly restart the collector conveyor BCioi via the conveyor controller of the automatic baggage handling system Bap (step S265 of FIG 30).

[0285]

In contrast, at the step S250, if an operator touches the Retry button (see Retry route from step S250), the conveyer-controlling module 2402 obtains the second image of the conveyance object Cob3 captured by each of the left three-dimensional camera 2203 and the left RGB camera 2204. [0286]

In other words, this situation could be, for example, that although an operator has checked the conveyance object Cob3 at the step S240, no problem was actually found, and therefore the operator wants to have the conveyor supervisor 2000 reconfirm the conveyance state of the conveyance object Cob, as a precaution.

As another example situation, an operator has solved the problems of the conveyance object Cob3 at the step S240, and as a precaution, the operator wants to make sure that the correction to the conveyance object Cob3 was appropriate by using the conveyor supervisor 2000.

[0287]

If the Retry button is touched (see Retry route from step S250), the image-analysing module 2401 of the Al vision processor 2311 determines, based on the second image captured by each of the left three-dimensional camera 2203 and the left RGB camera 2204, whether or not the conveyance state of the conveyance object Cob3 satisfies the forwarding condition (steps S270 and S275).

If the conveyance state of the conveyance object Cob3 satisfies the forwarding condition (Yes route from S275), the conveyer-controlling module 2402 restarts the collector conveyor BCioi via the conveyor controller of the automatic baggage handling system Bap (step S280).

[0288]

Note that an 'Accept' button may also be indicated on the touch screen 2305 at the step S250. In this configuration, if the Accept button is touched by an operator, the collector conveyor BCioi is forcibly restarted bypassing the step S255 and reaching the step S265. Nevertheless, executing this control shown as the Accept route from the step S250 is normally prohibited because the conveyance object Cob3 is resultantly conveyed in the downstream direction Ad without an operator inputting the overriding command.

[0289]

'Intrusion control' based on an 'intrusion image' captured by the CCTV camera 2309 will be described.

In this intrusion control, the intrusion-analysing module 2407 of the secondary computer 2308 substantively always analyses the image stream (intrusion image) continuously supplied from the CCTV camera 2309 via the Ethernet switch 2302 (step S300 of FIG 33).

If the intrusion-analysing module 2407 detects the presence of a human within the region of interest (i.e. the 'keep-out zone ZK') within the view of the image stream captured by the CCTV camera 2309, the intrusion-analysing module 2407 immediately stops the collector conveyor BCioi via the conveyercontrolling module 2402 of the primary computer 2301 (step S310).

[0290]

After that, if an operator confirms that there is no human in the keep- out zone Z _K (Yes route from step S315), the intrusion-analysing module 2407 restarts operation of the collector conveyor BCioi via the conveyer-controlling module 2402 of the primary computer 2301.

[0291]

In the example shown in FIG 33, the collector conveyor BCioi is not restarted unless an operator inputs to the touch screen 2305 that there is no person in the keep-out zone Z _K (step S310 via No route from step S315), but it is not limited to this scenario.

For example, if a human presence is detected in the region of interest within the view of the image stream captured by the CCTV camera 2309, but subsequently no human presence is detected in the region of interest, then the intrusion-analysing module 2407 may automatically restart operation of the collector conveyor BCioi via the conveyer-controlling module 2402 of the primary computer 2301.

[0292]

The third embodiment of the present invention will now be described below mainly using FIGs 35 to 39.

[0293]

A conveyor supervisor 3000 according to the third embodiment has substantially the same configuration as the conveyor supervisor 2000 according to the second embodiment. Accordingly, differences between the conveyor supervisors 2000 and 3000 will be mainly described here and redundant description will be omitted.

[0294]

As shown in FIG 35, a sensor head 3103 of the conveyor supervisor 3000 differs from the sensor head 2103 of the conveyor supervisor 2000 in terms of hardware.

Specifically, as shown in FIG 36, the sensor head 3103 includes all of the equipment mounted to the sensor head 2103, plus a right three- dimensional camera 3205, a right RGB camera 3206, and right light 3208. [0295]

The right RGB camera 3206 is also a Global Shutter RGB type camera with a fixed manual focus lens

[0296] Due to such a hardware difference, in this embodiment, as shown in FIG 37, the assessment zone Za is divided into the upstream side and the downstream side with respect to the conveyor supervisor 3000.

The first upstream-assessment zone Zi _u is an area in which an image of the conveyance object Cob (i.e. object image) is captured as the 'first image' by each of the left three-dimensional camera 2203 and the left RGB camera 2204 while the conveyance object Cob is conveyed by the collector conveyor BCioi.

[0297]

The second downstream-assessment zone Z2d is provided in the downstream direction Ad with respect to the first upstream-assessment zone Ziu as well as the conveyor supervisor 3000.

The second downstream-assessment zone Z2d is an area in which the object image is captured as the 'second image' by each of the right three- dimensional camera 3205 and the right RGB camera 3206 when the conveyance object Cob is stationary on the collector conveyor BC101 since the collector conveyor BCioi is stopped by the primary computer 2301. [0298]

The left light 2207 is a LED lighting unit that illuminates the first upstream-assessment zone Z _iu so that the left three-dimensional camera 2203 and the left RGB camera 2204 can clearly capture the first image of the conveyance object Cob.

The right light 3208 is another LED lighting unit that illuminates the second downstream-assessment zone Z2d so that the right three-dimensional camera 3205 and the right RGB camera 3206 can clearly capture the second image of the conveyance object Cob. [0299]

As shown in FIG 37, a primary computer 3301 and an Al vision processor

3311 correspond to the image-analysing module 2401 and the Al vision processor 2311 described in the second embodiment, respectively, and there is no substantial difference in hardware.

However, software modules of the primary computer 3301 are slightly different from each software module of the primary computer 2301 in the second embodiment.

Further, software modules of the Al vision processor 3311 are also slightly different from each software module of the Al vision processor 2311 in the second embodiment.

[0300]

As shown in FIG 37, a conveyer-controlling module 3402, an overriding module 3403, and a training module 3404 are stored in a storage device (not shown in FIGs) of the primary computer 3301 as software modules.

An image-analysing module 3401, which is a software module designed for the Al vision processor 3311, is stored in a storage device (not shown in FIGs) of the Al vision processor 3311.

[0301]

The conveyer-controlling module 3402 stops the collector conveyor BCioi such that the conveyance object Cob is positioned stationary within the second downstream-assessment zone Z2d if the image-analysing module 3401 of the Al vision processor 3311 determines based on the first image that the conveyance state of the conveyance object Cob does not satisfy the forwarding condition.

Further, the conveyer-controlling module 3402 restarts the collector conveyor BCioi, which is stopped by the conveyer-controlling module 2402, if the image-analysing module 3401 determines based on the second image that the conveyance state of the conveyance object Cob satisfies the forwarding condition.

In the meantime, in the present embodiment, the determination on the conveyance state of the conveyance object Cob in the first upstream- assessment zone Zi _u is referred to as the 'primary determination'. On the other hand, the determination on the conveyance state of the conveyance object Cob in the second downstream-assessment zone Z ₂ d is referred to as the 'secondary determination'.

[0302]

The overriding module 3403 executes an 'overriding control' that forcibly restarts the collector conveyor BCioi from the stationary state.

Specifically, this overriding operation is carried out under a situation where the image-analysing module 3401 of the Al vision processor 3311 determined that the conveyance state of the conveyance object Cob was not satisfying the forwarding condition, but an operator completed inputting the overriding command into the touch screen 2305.

[0303]

The Al vision processor 3311, like the Al vision processor 2311, is a computer dedicated for neural net artificial intelligence (Al) processing.

The Al vision processor 3311 receives the data of the first images captured by the left three-dimensional camera 2203 and the left RGB camera 2204 from the primary computer 3301. Also, the Al vision processor 3311 receives the data of the second images captured by the right three-dimensional camera 3205 and the right RGB camera 3206 from the primary computer 3301. [0304]

The image-analysing module 3401 implements the learning model 3405 which has been preliminarily trained for the conveyor supervisor 3000.

The image-analysing module 3401 assesses and determines whether the conveyance state of the conveyance object Cob satisfies the forwarding condition based on the received image data and the trained learning model 3405.

[0305]

The learning model 3405 defines an assessment standard. Specifically, the learning model 3405 is a machine learning model generated based on a vast amount of raw data (training data), in which a number of conveyance objects are photographed individually, and a learning data set corresponding to the factors of the conveyance objects.

The Al vision processor 3311 transmits results of the assessment and determination process to the primary computer 3301.

[0306]

A second photo-eye sensor 3211 is provided beside an upstream edge of the second downstream-assessment zone Z ₂ d.

The second photo-eye sensor 3211 is an optical sensor detecting that the conveyance object Cob has entered the second downstream-assessment zone Z ₂ d.

[0307]

The second photo-eye sensor 3211 is connected to the I/O controller 2304 with a digital I/O cable.

Further, the I/O controller 2304 is communicably connected to the primary computer 3301 via the Ethernet switch 2302.

Hence, the primary computer primary computer 3301 can immediately detect that the conveyance object Cob enters the second downstreamassessment zone Z _2d by the second photo-eye sensor 3211.

[0308]

With the above-mentioned arrangement, the conveyor controlling apparatus according to the third embodiment of the present invention provides the following advantages.

[0309]

The analysis control of the conveyance object Cob by the conveyor supervisor 3000 is mainly performed in accordance with a main flowchart shown in FIG 39.

Note that, in parallel with the analysis control of the conveyance object Cob shown in FIG. 411, the intrusion control shown in FIG. 207 is also executed, but the intrusion control which has already been described in the second embodiment, so will be omitted here.

[0310]

As shown in FIG 38(a), the conveyer-controlling module 3402 recognises that the conveyance object (suitcase) Cob3 has just entered the first upstream-assessment zone Zi _u in response to detection by the first photoeye sensor 2209 that the conveyance object Cob3 which is conveyed from the upstream direction Au to the downstream direction Ad by the collector conveyor BCioi (Yes route from step S205 in FIG 39).

Immediately subsequent to this, the conveyer-controlling module 3402 stays on standby to make a very short delay for ensuring that the entire conveyance object Cob enters the first upstream-assessment zone Zi _u (step S310).

[0311]

At this time, the conveyer-controlling module 3402 of the primary computer 3301 obtains the first image of the conveyance object Cob3 captured by each of the left three-dimensional camera 2203 and the left RGB camera 2204 (step 315). During this time, the collector conveyor BCioi is still running, hence, the conveyance object Cob3 keeps travelling in the downstream direction Ad.

[0312]

Then, the image-analysing module 3401 of the Al vision processor 3311 determines whether the conveyance state of the conveyance object Cob3 satisfies the forwarding condition based on the first image captured by each of the left three-dimensional camera 2203 and the left RGB camera 2204 (steps S315 and S320 in FIG 39).

Also, at this time, the training module 3404 records the analysis results of the conveyance object Cob3 carried out by the image-analysing module 3401 of the Al vision processor 3311 and the captured first image of the conveyance object Cob3 in the external storage server (steps S315 and S320 in FIG 39).

[0313]

If the conveyance state of the conveyance object Cob3 satisfies the forwarding condition (Yes route from step S320 in FIG 39), the conveyercontrolling module 3402 does not perform special control on the automatic baggage handling system Bap. Hence, the collector conveyor BCioi continues running and the conveyance object Cob3 is conveyed to the downstream direction Ad.

[0314]

On the other hand, if the conveyance state of the conveyance object Cob3 does not satisfy the forwarding condition (No route from step S320 in FIG 39), the conveyer-controlling module 3402 stops the collector conveyor BCioi via the conveyor controller of the automatic baggage handling system Bap so that the conveyance object Cob3 is stationary within the second downstream-assessment zone Z ₂ d (step S330 of FIG 39 - see also FIG 38(c)). [0315]

The conveyer-controlling module 3402 recognises that the conveyance object Cob3 has entered the second downstream-assessment zone Z _2d in response to the second photo-eye sensor 3211 detecting the conveyance object Cob3.

Nevertheless, the conveyer-controlling module 3402 makes a short delay to ensure that the entire conveyance object Cob3 enters the second downstream-assessment zone Z _2d (step S325 in FIG 39), then stops the collector conveyor BCioi (step s330).

[0316]

After that, at step S335 of FIG 39, the conveyer-controlling module 3402 executes the 'need-of-care control'. Note that the need-of-care control has been described in the second embodiment with reference to FIG 32, so description thereof will be omitted here.

[0317]

If the operation mode of the conveyor supervisor 3000 with regard to restarting the collector conveyor BCioi is 'automatic' (Automatic route from step S345), the conveyer-controlling module 3402 obtains the second image of the conveyance object Cob3 captured by each of the right three-dimensional camera 3205 and the right RGB camera 3206 (step S370 in FIG 39).

Further, the image-analysing module 3401 of the Al vision processor 3311 determines whether the conveyance object Cob3 satisfies the forwarding condition based on the second image captured by the right three-dimensional camera 3205 and the right RGB camera 3206 (steps S370 and S375 in FIG 39).

[0318]

Also, at this time, the training module 3404 records the analysis results of the conveyance object Cob3 carried out by the image-analysing module

3401 of the Al vision processor 3311, the determination results as to whether or not the conveyance state of the conveyance object Cob3 satisfies the forwarding condition, and the captured second image of the conveyance object Cob3 in the external storage server (steps S370 and S375 of FIG 39).

[0319]

If the conveyance state of the conveyance object Cob3 satisfies the forwarding condition (Yes route from S375), the conveyer-controlling module

3402 restarts the collector conveyor BCioi via the conveyor controller of the automatic baggage handling system Bap (step S380). At this time, in the example shown in FIG 38(d), the handle 903 is now stored in the conveyance object (suitcase) Cob3 and is being conveyed to the downstream direction Ad. [0320]

On the other hand, if the operation mode of the conveyor supervisor 3000 with regard to restarting the collector conveyor BCioi is 'manual' (Manual route from step S345), the conveyer-controlling module 3402 indicates a 'Force-in' button and a 'Retry' button on the touch screen 2305. FIG 34 shows an example of the display on the touch screen 2305 at this time.

[0321]

If the Force-in button is touched by an operator (see Force-in route from step S350 in FIG 39), the overriding module 3403 indicates on the touch screen 2305 check boxes corresponding to the found problems with respect to the conveyance object Cob3 (see FIG 12).

In other words, this situation could be, for example, that although an operator has checked the conveyance object Cob3 at step S340, no problem was actually found, and therefore the operator believes that the conveyance object Cob3 should continue travelling in the downstream direction Ad. [0322]

At this time, the operator appropriately checks the check boxes corresponding to the reasons why there is no problem with the conveyance object Cob3 (i.e. 'overriding reason'), then the operator touches a 'Submit' button indicated on the touch screen 2305 (step S355).

[0323]

Also, at this time, the training module 3404 records the analysis results of the conveyance object Cob3 carried out by the image-analysing module 3401 of the Al vision processor 3311, the determination results as to whether or not the conveyance state of the conveyance object Cob3 satisfies the forwarding condition, the captured first image of the conveyance object Cob3, and the overriding reason entered through the touch screen 2305 in the external storage server (step S360).

In addition, the training module 3404 may apply a learning control to the learning model 3405 based on the entered overriding reason.

[0324] In short, in the situation where the image-analysing module 3401 of the Al vision processor 3311 has determined that the conveyance state of the conveyance object Cob3 does not satisfy the forwarding condition, but an operator has completed inputting the overriding command through the touch screen 3305, accordingly, the overriding module 3403 executes the overriding control to forcibly restart the collector conveyor BCioi via the conveyor controller of the automatic baggage handling system Bap (step S365 of FIG 39).

[0325]

In contrast, at the step S350, if an operator touches the Retry button (see Retry route from step S350), the conveyer-controlling module 3402 obtains the second image of the conveyance object Cob3 captured by each of the right three-dimensional camera 3205 and the right RGB camera 3206. [0326]

In other words, this situation could be, for example, that although an operator has checked the conveyance object Cob3 in the second downstreamassessment zone Z _2d at the step S340, no problem was actually found, and therefore he/she wants to have the conveyor supervisor 3000 reconfirm the conveyance state of the conveyance object Cob, as a precaution.

As another example situation, an operator has solved the problems of the conveyance object Cob3 in the second downstream-assessment zone Z _2d at the step S340, and as a precaution, the operator wants to make sure that the correction to the conveyance object Cob3 was appropriate by using the conveyor supervisor 3000. [0327]

If the Retry button is touched (see Retry route from step S350), the image-analysing module 3401 of the Al vision processor 3311 determines, based on the second image captured by each of the right three-dimensional camera 3205 and the right RGB camera 3206, whether or not the conveyance state of the conveyance object Cob3 satisfies the forwarding condition (steps

S370 and S375).

If the conveyance state of the conveyance object Cob3 satisfies the forwarding condition (Yes route from S375), the conveyer-controlling module 3402 restarts the collector conveyor BCioi via the conveyor controller of the automatic baggage handling system Bap (step S380).

[0328]

Note that an 'Accept' button may also be indicated on the touch screen 2305 at the step S350. In this configuration, if the Accept button is touched by an operator, the collector conveyor BCioi is forcibly restarted bypassing the step S355 and reaching the step S365. Nevertheless, executing this control shown as the Accept route from the step S350 is normally prohibited because the conveyance object Cob3 is resultantly conveyed in the downstream direction Ad without an operator inputting the overriding command.

[0329]

Regarding the intrusion control using the CCTV camera 2309 has been described in the second embodiment, so description thereof will be omitted here.

[0330]

The present invention is not limited to the above-described embodiments and the variants thereof, and can be variously modified and implemented.

[0331]

In the first to third embodiments, the conveyor supervisors 1000/2000/3000 are installed in the airport AP, but is not limited to this scenario. For example, the conveyor supervisors 1000/2000/3000 may be installed in factories, plants, and/or warehouses.

[0332] In the first to third embodiments, the conveyor supervisors 1000/2000/3000 are installed near the collector conveyor BCioi, partly because, normally, there are more ground-handling staff working in the FOH than the BOH. Accordingly, if the conveyor controlling apparatus Baz is installed in the FOH of the airport AP, then the conveyance object Cob may be quickly attended by the ground-handling staff in the FOH.

Nevertheless, the installation point of the conveyor supervisors 1000/2000/3000 is not limited to this location. For example, the conveyor supervisors 1000/2000/3000 may be installed near the transport conveyor BC102 close to the inlet of the BOH in the airport AP.

[0333]

In the first and third embodiments, as shown in FIG 1, the conveying direction Ac of the collector conveyor BCioi is in the right direction with respect to the conveyor supervisors 1000/3000, but it is not limited to this direction. For example, as shown in FIG 16, even if the conveying direction Ac of the belt conveyor is the opposite, the conveyor supervisors 1000/3000 may be compatible.

[0334]

That is, as shown in FIG 16, when the conveying direction Ac of the collector conveyor BCioi is the left-direction with respect to the conveyor supervisors 1000/3000, both of the right three-dimensional camera 205/3205 and the right RGB camera 206/3206 function as the upstream capturing device that captures the upstream image. Likewise, both of the left three- dimensional camera 203/2203 and the left RGB camera 204/2204 function as the downstream capturing device that captures the downstream image. [0335]

In this case, the first upstream-assessment zone Z _iu on the collector conveyor BCioi is a virtual area that approximately corresponds to a photographable area of the cameras towards the upstream direction Au (i.e. each of the right three-dimensional cameras 205/3205 and each of the right RGB cameras 206/3206). Likewise, the second downstream-assessment zone Z ₂ d on the collector conveyor BCioi is another virtual area that approximately corresponds to a photographable area of the cameras towards the downstream direction Ad (i.e. each of the left three-dimensional cameras 203/2203 and each of the left RGB cameras 204/2204).

[0336]

The conveyor supervisors 1000/2000/3000 are robust and can work with belt conveyors, roller conveyors, slat conveyors, mesh conveyors and the like. The conveyor supervisors 1000/2000/3000 can also be adapted to work in scenarios where the conveyor has been replaced by robot-type vehicles such as Automated Guided Vehicles (AGVs). Therefore, again, the conveyor supervisors 1000/2000/3000 are not limited for airports. Applicable industries are not limited to the airline sector, and can be applied to various industries.

[0337]

As a variant of the first to third embodiments, the conveyor supervisors 1000/2000/3000 may recognise a baggage ID printed on a baggage tag which is uniquely affixed to the conveyance object Cob, based on the first image and/or the second image.

[0338]

In this case, the baggage ID recognised by the conveyor supervisors 1000/2000/3000 may be transmitted to other external systems (e.g. systems under the control of the International Air Transport Association (IATA), other cargo systems, and so on).

Further, the first and/or the second images may be transmitted to other external systems along with the baggage ID recognised by the conveyor supervisors 1000/2000/3000.

[0339] Further, the conveyor supervisors 1000/2000/3000 may transmit the first image and/or the second image of the conveyance object Cob, which did not satisfy the forwarding condition, to other external systems.

[0340]

In the first to third embodiments, the LED illumination of the status indicators 306/2306 is changed in accordance with the working state of the conveyor supervisors 1000/2000/3000, but it is not limited to this configuration. For example, the conveyor supervisors 1000/2000/3000 may include a buzzer (not shown in FIGs) as an alarm device for beeping in accordance with the working state of the conveyor supervisors 1000/2000/3000. According to this configuration, it is possible to notify aurally an operator of the operational state of the conveyor.

[0341]

The conveyor supervisors 1000/2000/3000 may be movable. For example, FIG 17 shows the conveyor supervisor 1000 according to the first embodiment has wheels 121 added to the bottom face 116. According to this arrangement, an operator can easily move the conveyor supervisors 1000/2000/3000 to any desired location.

[0342]

The primary computers 301/2301/3301 may determine the conveying direction Ac of the collector conveyor BCioi based on the first image and/or the second image. That is, the conveyor supervisors 1000/2000/3000 may judge the conveying direction Ac of the collector conveyor BCioi is either rightwards (see FIG 1) or leftwards (see FIG 16) based on the first image and/or the second image.

[0343]

In the first and third embodiments, respective first photo-eye sensors 209/2209 are provided beside the first upstream-assessment zone Zi _u , and respective second photo-eye sensors 211/3211 are provided beside the second downstream-assessment zone Z2 _U . Further, in the second embodiment, the first photo-eye sensor 2209 is provided is beside the first upstreamassessment zone Ziu, and the second photo-eye sensor 2211 is provided is beside the second upstream-assessment zone Z _2u .

However, it is not limited to theses configurations.

[0344]

For example, without using the first photo-eye sensors 209/2209, the primary computers 301/2301/3301 according to the first to third can detect an entry of a conveyance object Cob into the first upstream-assessment zone Ziu based on the first image.

Similarly, even without using the second photo-eye sensors 211/3211, the primary computers 301/3301 according to the first and third embodiments can detect an entry of a conveyance object Cob into the second upstreamassessment zone Z _2u based on the downstream image.

Likewise, without using the second photo-eye sensor 2211, the primary computer 2301 according to the second embodiment can detect an entry of a conveyance object Cob into the second downstream-assessment zone Z _2d based on the second image.

[0345]

Nevertheless, by using the first and second photo-eye sensors 209, 221, 2209, 2211 and 3211, the processing load of the conveyor supervisors 1000/2000/3000 may be reduced.

[0346]

In the first to third embodiments, the height of a conveyance object is detected based on the captured image by the three-dimensional camera. However, it is not limited to this configuration.

[0347]

For example, an optical height sensor may be provided beside the conveyor. According to this configuration, it is possible to judge more directly whether or not the height of the conveyance object Cob is beyond the predetermined height limit.

[0348]

In the second embodiment, the image-analysing module 2401 of the Al vision processor 2311 analyses and determines whether the conveyance state of the conveyance object Cob satisfies the forwarding condition.

Likewise, in the third embodiment, the image-analysing module 3401 of the Al vision processor 3311 analyses and determines whether the conveyance state of the conveyance object Cob satisfies the forwarding condition.

However, it is not limited to these configurations.

[0349]

For example, the technology described in the second embodiment may be realised by a single computer that integrates all the features of the primary computer 2301 and the Al vision processor 2311.

Also, the technology described in the third embodiment may be realised by a single computer that integrates all the features of the primary computer 3301 and the Al vision processor 3311.

[0350]

In the first to third embodiments, the secondary computers 308/2308 carry out the intrusion control using the CCTV camera 212/2309. However, it is not limited to these configurations.

[0351]

For example, the technology described in the first to third embodiments may be realised by a single computer that integrates all the features of the primary computers 301/2301/3301 and the secondary computers 308/2308. [0352]

In the first to third embodiments, the case where the conveyor supervisors 1000/2000/3000 are applied to the straight collector conveyor BC101 has been described, but it is not limited to this. For example, the conveyor supervisors 1000/2000/3000 are applicable to curbed conveyors. [0353]

The situations in which a conveyance state of a conveyance object Cob does not satisfy the forwarding conditions as discussed in the first to third embodiments may corresponds to at least one of 'rejection categories' shown in FIG 40.

[0354]

In the first to third embodiments, the data related to analysis/determination of the conveyance object Cob3, the data related to the intrusion control, and the data related to the captured images of the conveyance object Cob3 is recorded to the external storage server, but it is not limited to this configuration.

[0355]

For example, it is also possible to employ a configuration in which the data relating to the analysis/determination of the conveyance object Cob, the intrusion control, and the data related to the captured images of the conveyance object Cob is recorded to an internal storage device (now shown in FIGs) provided in the conveyor supervisor 1000/2000/3000.

[0356]

As described in detail above, according to the control device and control method of the present invention, the conveyor is controlled according to the environmental parameters of the conveyor, so that the reliability of the conveyance system having the conveyor can be improved.

[0357]

In addition, by stopping the conveyor according to different forwarding conditions depending on the conveyance system having the conveyor and the image of the conveyance objects, it is possible to improve the accuracy of flow control for the conveyance objects. Further, it is possible to improve overall operational efficiencies of the conveyance system.

[0358]

Further, when it is later determined that the conveyance state of the conveyance object now satisfies the forwarding condition of the conveyor, the conveyer can be quickly restarted.

[0359]

Also, the conveyance state of the conveyance object may be determined in the first assessment zone, and the conveyor may be stopped so that the conveyance object is positioned within the second assessment zone according to the determination result. Further, the conveyance object within the second basement zone may be further determined, and the conveyor may be restarted depending on the determination result.

[0360]

Note that the first assessment zone may be defined on the upstream side of the capturing device, and the second assessment zone may be defined on the downstream side of the capturing device.

[0361]

Also, the learning model may be used in determining the image of the conveyance object.

[0362]

Moreover, even if it is determined that the conveyance state of the conveyance object does not satisfy the forwarding condition, the control for forcibly restarting the conveyor, that is, the overriding control may be executed. However, when executing this overriding control, the reason for the execution must be included, so that more accurate control of the conveyance object can be realised.

[0363]

Further, it improves the operational reliability of the transport system having the conveyor by immediately stopping the conveyor if any human is detected with in the keep-out zone.

[0364]

Also, the conveyor can be restarted quickly if no humans are detected within the keep-out zone.

[0365]

In addition, by making the conveyor controlling apparatus compact with the main body, the apparatus can be adapted to various types of conveyors.

[0366]

In the claims which follow and in the preceding description of the invention, except where the context requires otherwise due to express language or necessary implication, the word "comprise" or variations such as "comprises" or "comprising" are used in an inclusive sense, i.e. to specify the presence of the stated features but not to preclude the presence or addition of further features in various embodiments of the invention.

[0367]

The reference to any prior art in this specification is not, and should not be taken as, an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge.

[0368]

Reference Signs List

101 main body

102 pillar

103 sensor head (head unit)

203 left three-dimensional camera (upstream capturing device/first capturing device/capturing device)

204 left RGB camera (upstream capturing device/first capturing device/capturing device)

205 right three-dimensional camera (downstream capturing device/second capturing device/capturing device)

206 right RGB camera (downstream capturing device/second capturing device/capturing device)

207 left light (upstream lighting device)

208 right light (downstream lighting device)

212 CCTV camera (intrusion camera/capturing device)

301 primary Al computer (image-analysing module, training module)

303 system computer (overriding module)

304 I/O controller (conveyor-controlling module)

306 status indicator (alarm device)

307 learning model

1000 conveyor supervisor (conveyor controlling apparatus)

2000/3000 conveyor supervisor (conveyor controlling apparatus)

2101 main body

2102 pillar

2103 sensor head (head unit)

2203 left three-dimensional camera (first image-capturing device/capturing device)

2204 left RGB camera (first image-capturing device/capturing device)

2309 CCTV camera (intrusion camera/capturing device)

2401/3401 image-analysing module

2402/3402 conveyer-controlling module

2403/3403 overriding module

2404/3404 training module

2405/3405 learning model

2407 intrusion-analysing module

3205 right three-dimensional camera (second image-capturing device/capturing device)

3206 right RGB camera (second image-capturing device/capturing device)

Ad downstream direction

Au upstream direction

Baz conveyor controlling apparatus

BC conveyer

BCioi collector conveyor (conveyor)

BCioi-i first collector conveyor (conveyor)

BCIOI-2 second collector conveyor (conveyor)

Cob conveyance object

Dz zone-separation distance

Hinf infant (human)

Za assessment zone

Z _K keep-out zone

Ziu first upstream-assessment zone (first assessment zone/upstream assessment zone/assessment zone)

Z _2d second downstream-assessment zone (second assessment zone/downstream assessment zone/assessment zone)

Z _2u second upstream-assessment zone (second assessment zone/upstream assessment zone/assessment zone)