TITLE OF THE INVENTION

CONVEYANCE OBJECT CONTROLLING APPARATUS AND METHOD

Field of the Invention

[0001]

The present invention relates to a conveyance-object controlling apparatus and a conveyance-object controlling method for controlling a conveyance object on a conveyor in accordance with a conveyance state of the conveyance object.

Background of the Invention

[0002]

In recent years, automatic systematisation in logistics has been steadily progressing, and technologies related to automating warehouse management and handling of conveyance items such as checked-luggage and parcels at airports have been developed. The technologies related to automated systems in logistics include, for example, US Patent No. 5165520 Publication, US Patent Application Publication No. 2018/0148271 and US Patent No. 7055672 Publication.

[0003]

Nevertheless, it is still common to input conveyance items to automatic conveyance systems by hand. For example, in conventional airports, it is common that checking-in of passengers' luggage are manually handled by ground handling staff. However, even if the ground-handling staff are well trained, actually, there will invariably by situations where the luggage are presented in an inappropriate configuration, or portions of the luggage are in awkward position (for example, a handle of a suitcase may be extended, or a zipper on a duffel bag may open). This problem is also prevalent in self-drop luggage kiosks.

[0004]

The checked-luggage can be on transit in a baggage handling system (BHS) and on its way to being loaded onto a departing plane. Therefore, to ensure that the loading of the checked-luggage is seamless and does not cause any delay to the departing plane, it is vital to ensure that further checks and measures are in place to ensure that all checked-luggage satisfy a 'forwarding condition' before being loaded onto the plane.

The forwarding condition is mainly subject to capability of the BHS. Accordingly, if a conveyance item does not meet the forwarding condition, it may cause problems in the BHS such as conveyance jams.

[0005]

What is thus required is a solution that automatically detects any checked-luggage that does not satisfy the forwarding condition.

Furthermore, this solution must be also be capable of clearly identifying the problematic luggage (among all the other luggage) so that appropriate counter measures can be quickly taken (i.e. an operator can manually fix or adjust the position or configuration of the luggage). The solution should also be robust enough to accept feedback and learn and adapt to updating standards for identification of defective luggage.

[0006]

An object of the present invention is to provide a conveyance-object controlling apparatus and a conveyance-object controlling method has been created in light of such problems to assess appropriately whether conveyance state of various types of conveyance objects meets the forwarding condition of each automatic conveyance system so that the conveyance objects are precisely controlled in accordance with the assessed conveyance state.

Summary of the Invention [0007]

A conveyance-object controlling apparatus for controlling a conveyance object on a conveyor in accordance with a conveyance state of the conveyance object, the apparatus comprises: an upstream capturing device configured to capture an upstream image of the conveyance object when the conveyance object is in an upstream assessment zone on the conveyor and while the conveyance object is conveyed by the conveyor in a direction towards the upstream capturing device; a downstream capturing device configured to capture a downstream image of the conveyance object in a downstream assessment zone on the conveyor, the downstream assessment zone being set further downstream when compared to the upstream assessment zone; an image-analysing module configured to determine, based on the upstream image and/or the downstream image, whether the conveyance state of the conveyance object satisfies a forwarding condition; and a conveyor-controlling module configured to control the conveyor based on a result of the determination carried out by the image-analysing module; wherein if the image-analysing module determines, based on the upstream image, that the conveyance state of the conveyance object does not satisfy the forwarding condition, then the conveyor-controlling module stops the conveyor such that the conveyance object is positioned in the downstream assessment zone, and wherein if the image-analysing module determines, based on the downstream image, that the conveyance state of the conveyance object satisfies the forwarding condition, then the conveyor-controlling module restarts the conveyor.

[0008] The conveyance-object controlling apparatus, wherein the image-analysing module is further configured to implement a learning model, the learning model adapted to define an assessment standard for the conveyance state, and the image-analysing module utilises the learning model in the determination of whether the conveyance state of the conveyance object satisfies a forwarding condition.

[0009]

The conveyance-object controlling apparatus, further comprises: an overriding module configured to execute an overriding operation, which is an operation forcibly restarts the conveyor, even when the image-analysing module determines based on the downstream image that the conveyance state of the conveyance object does not satisfy the forwarding condition, in response to an overriding command; and a training module configured to update the learning model based on the overriding command and the upstream-side and/or downstream images of the conveyance object whenever the overriding operation is executed by the overriding module, wherein the overriding command includes a reason of execution of the overriding operation.

[0010]

The conveyance-object controlling apparatus, wherein the upstream capturing device and the downstream capturing device are adjacent to one another and orientated in opposing directions.

[0011]

The conveyance-object controlling apparatus further comprises: a main body; a pillar extending upwardly from the main body; and a head unit provided at a top end of the pillar, wherein the upstream capturing device and the downstream capturing device are mounted in the head unit.

[0012]

The conveyance-object controlling apparatus further comprises: an upstream lighting device configured to illuminate the upstream assessment zone in which the upstream image is captured by the upstream capturing device; and a downstream lighting device configured to illuminate the downstream assessment zone in which the downstream image is captured by the downstream capturing device.

[0013]

The conveyance-object controlling apparatus, wherein the upstream lighting device and the downstream lighting devices are mounted in the head unit.

[0014]

The conveyance-object controlling apparatus, wherein a zone-separation distance, which is a distance between the upstream assessment zone and the downstream assessment zone, and the zone-separation distance is adjusted according to a processing capability of the image-analysing module.

[0015]

The conveyance-object controlling apparatus further comprises: an alarming device configured to provide a visual and/or aural notification of the result of the determination carried out by the image-analysing module.

[0016]

A conveyance-object controlling method for controlling a conveyance object on a conveyor in accordance with a conveyance state of the conveyance object, the method comprises steps of: capturing an upstream image of the conveyance object by an upstream capturing device when the conveyance object is in an upstream assessment zone on the conveyor and while the conveyance object is being conveyed by the conveyor in a direction towards the upstream capturing device; capturing a downstream image of the conveyance object by a downstream capturing device in a downstream assessment zone on the conveyor, the downstream assessment zone being set further downstream when compared to the upstream assessment zone; determining, based on the upstream image and/or the downstream image, whether the conveyance state of the conveyance object satisfies a forwarding condition; and controlling the conveyor based on a result of the determination step, wherein if the determination step determines, based on the upstream image, that the conveyance state does not satisfy the forwarding condition, then the controlling step stops the conveyor such that the conveyance object is positioned in the downstream assessment zone, and wherein if the determination step determines, based on the downstream image, that the conveyance state satisfies the forwarding condition, then the controlling step restarts the conveyor.

[0017]

The controlling method, further comprises steps of: providing a learning model defining an assessment standard to the conveyance state; and recognising the conveyance state of the conveyance object based on the learning model and the upstream image and/or the downstream image, wherein the determining step determines whether the conveyance state, which is recognised in the recognising step, satisfies the forwarding condition.

[0018]

The controlling method further comprises steps of: executing an overriding operation, which is an operation forcibly restarts the conveyor, even when the determining step determines, based on the downstream image, that the conveyance state of the conveyance object does not satisfy the forwarding condition, in response to an overriding command; and updating the learning model based on the overriding command and the upstream image and/or the downstream image of the conveyance object whenever the overriding operation is executed, wherein the overriding command includes a reason of execution of the overriding operation.

[0019]

The controlling method, wherein the upstream capturing device and the downstream capturing device are adjacent to one another and orientated in opposing directions.

[0020]

The analysing method further comprises steps of: providing a main body; providing a pillar extending upwardly from the main body; providing a head unit at a top end of the pillar; and providing the upstream capturing device and the downstream capturing device in the head unit.

[0021]

The controlling method further comprises steps of: providing an upstream lighting device configured to illuminate the upstream assessment zone in which the upstream image is captured by the upstream capturing device; and providing a downstream lighting device configured to illuminate the downstream assessment zone at which the downstream image is captured by the downstream capturing device.

[0022]

The controlling method, wherein the upstream-side and downstream lighting devices are mounted in the head unit.

[0023]

The controlling method further comprises a step of: adjusting a zone-separation distance, which is a distance between the upstream assessment zone and the downstream assessment zone, according to a processing capability of the determining step.

[0024]

The controlling method further comprises a step of: visually and/or aurally notifying the result of the determining step.

Brief Description of the Drawings

[0025]

FIG 1 is a schematic view showing a conveyance-object controlling apparatus according to one embodiment of the present invention when the apparatus is installed near a belt conveyor of which the conveying direction is in the right direction.

[0026]

FIG 2 is a schematic diagram showing a part of an airport in which the conveyance-object controlling apparatus according to one embodiment of the present invention is installed.

[0027]

FIG 3 is a schematic perspective left-front view showing the conveyance-object controlling apparatus according to one embodiment of the present invention.

[0028]

FIG 4 is a schematic block diagram showing a configuration of the conveyance-object controlling apparatus according to one embodiment of the present invention.

[0029]

FIG 5 is a schematic plan view (Ai-Ai cross-sectional view of FIG 7) mainly showing a bottom face of a sensor head of the conveyance-object controlling apparatus according to one embodiment of the present invention. [0030]

FIG 6 is a schematic front view of the conveyance-object controlling apparatus according to one embodiment of the present invention. [0031]

FIG 7 is a schematic rear view of the conveyance-object controlling apparatus according to one embodiment of the present invention. [0032]

FIG 8 shows a schematic left view of the conveyance-object controlling apparatus according to one embodiment of the present invention when the apparatus is installed adjacent to a belt conveyor.

[0033]

FIG 9 is a schematic diagram showing examples of conveyance objects managed by the conveyance-object controlling apparatus according to one embodiment of the present invention.

[0034]

FIG 10 is a schematic view showing an indication on a touch panel of the conveyance-object controlling apparatus according to one embodiment of the present invention while an image analysis is in progress.

[0035]

FIG 11 is a schematic view showing an indication on the touch panel of the conveyance-object controlling apparatus according to one embodiment of the present invention when the forwarding condition is not satisfied.

[0036]

FIG 12 is a schematic view showing an indication on the touch panel of the conveyance-object controlling apparatus according to one embodiment of the present invention while entering of an overriding reason is awaited. [0037]

FIG 13 is a schematic view showing an indication on the touch panel of the conveyance-object controlling apparatus according to one embodiment of the present invention when the overriding reason is entered.

[0038]

FIG 14 is a main flowchart schematically showing an operation of the conveyance-object controlling apparatus according to one embodiment of the present invention.

[0039]

FIG 15 is an example showing a recognition result of an upstream image carried out by the conveyance-object controlling apparatus according to one embodiment of the present invention. [0040]

FIG 16 is an example showing a recognition result of a downstream image carried out by the conveyance-object controlling apparatus according to one embodiment of the present invention.

[0041]

FIG 17 is a sub-flowchart schematically showing an operation of the conveyance-object controlling apparatus according to one embodiment of the present invention.

[0042]

FIG 18 is a schematic view showing the conveyance-object controlling apparatus according to one embodiment of the present invention when the apparatus is installed near the belt conveyor of which the conveying direction is in the left direction.

[0043]

FIG 19 is a schematic left view of the conveyance-object controlling apparatus according to the first variant of one embodiment of the present invention when the apparatus is installed adjacent to a belt conveyor.

[0044]

FIG 20 is a schematic block diagram showing a configuration of the conveyance-object controlling apparatus according to the second variant of one embodiment of the present invention.

[0045]

FIG 21 is another main flowchart schematically showing an alternative operation of the conveyance-object controlling apparatus according to one embodiment of the present invention.

[0046]

FIG 22 is a schematic block diagram showing another configuration of the conveyance-object controlling apparatus according to one embodiment of the present invention, when the conveyance-object controlling apparatus is installed near a boundary between two conveyors.

Description of the Preferred Embodiments

[0047]

One embodiment of the present invention will now be described with reference to the accompanying main drawings FIGs 0101 to 0901.

[0048]

As shown in FIG 1, a conveyance-object analyser 1000 (conveyance-object controlling apparatus Baz), which is installed in the vicinity of a belt conveyor BC, assesses whether a conveyance state of the conveyance object Cob on the belt conveyor BC satisfies a predetermined forwarding condition. The definition of the 'conveyance object' Cob includes all kinds of objects on the belt conveyor BC. That is, not only ordinary objects (e.g. bags or parcels), which are normally carried by the belt conveyor BC, but also other unordinary objects, which should theoretically not be carried by the belt conveyor BC, shall also be included in the definition of 'conveyance object' Cob.

[0049]

The conveyance-object analyser 1000 is installed in the non-restricted area (so-called Front of House (FOH)) of an airport AP near the boundary with respect to the restricted area (so-called Back of House (BOH)), as shown in FIG 2. In this airport AP, each of the conveyance-object analysers is installed at the points shown as "Buxi", "Buxz", and "Buxs" in FIG 2, but in the present embodiment, one of the conveyance-object analysers 1000 installed in the place shown as "Buxi" in FIG 2 will be described.

[0050]

An automatic baggage handling system (BHS) Bap installed in the airport AP mainly has an introduction part Bi, a main part Bm, and a transition part Bt.

[0051]

The introduction part Bi has a group of conveyors such as weighing conveyors, holding conveyors and collector conveyors installed in first and second islands IS1 and IS2 provided in the FOH in the airport AP. Each of the first and second islands IS1 and IS2 has a plurality of self-baggage drop machines 5001.

[0052]

Each of self-baggage drop machine 5001 has the weighing conveyor.

Each of the holding conveyors, which is provided at a downstream side of each of the weighing conveyors, receives checked-luggage (conveyance object Cob) forwarded from the weighing conveyor, and automatically drops the forwarded checked-luggage into the collector conveyor 5002 at an appropriate timing.

[0053]

The collector conveyor 5002 is a belt conveyor that continuously runs substantially in a constant speed (e.g. 30 m/min) for conveying the checked-luggage dropped from the holding conveyor to the downstream-side. [0054]

The main part Bm is the core part of the automatic baggage handling system Bap provided in the BOH of the airport AP. The main part Bm has a group of conveyors such as main conveyors, tilt-tray conveyors, and make-up conveyors. [0055]

The transition part Bt, which is a part provided between the introduction part Bi and the main part Bm in the BOH of the airport AP, has a transport conveyor 5003 installed near an inlet of BOH. [0056]

The transport conveyor 5003, which is a belt conveyor disposed between a downstream end of the introduction part Bi and an upstream end of the main part Bm, forwards the checked-luggage (conveyance object Cob) sent from the collector conveyor 5002 to the main part Bm.

Each of the conveyors of the automatic baggage handling system Bap is controlled by a conveyor controller 4060 (see FIG 4).

[0057]

The conveyance-object analyser 1000 is installed in a position abutting onto the collector conveyor 5002 (see FIGs 0101 and 0403). The conveyance-object analyser 1000 assesses whether a conveyance state of the conveyance object Cob carried by the collector conveyor 5002 satisfies a forwarding condition which is subjected to the automatic baggage handling system Bap. [0058]

As shown in FIGs 0201, 0305, 0401, and 0403, the conveyance-object analyser 1000 is mainly comprises a main body 3010, a pillar 3020, and a sensor head (head unit) 3030.

[0059]

The main body 3010 is a pedestal portion of which a bottom face 3017 is placed on a floor 5004 and vertically extends upwards. Further, as shown in FIG 4, a primary Al computer 1030, a secondary Al computer 1040, a network hub 1050, a system computer 1060, and an I/O controller 1130 are included in the main body 3010.

A top face 3013 of the main body 3010 is formed as a plane extending diagonally downward from a back face 3012 to a front face 3011. A touch panel (display unit) 1070 is provided on the top face 3013.

[0060]

The pillar 3020 is a metal member extending upward from the main body 3010. An LED unit 1090 is provided on a front face 3021 of the pillar 3020.

Further, the pillar 3020 is slidably fixed to the main body 3010 so that the height of the sensor head 3030 is flexibly changed as shown as an arrow Ash in FIGs 0305 and 0401.

[0061]

The sensor head 3030 is an assembly unit fixed to the top end of the pillar 3020. As shown in FIG 5, a left three-dimensional camera (upstream capturing device) 1011, a left RGB camera (upstream capturing device) 1012, a left light (upstream-side lighting device) 1081, a right three-dimensional camera (downstream capturing device) 1021, a right RGB camera (downstream capturing device) 1022, and a right light (downstream-side lighting device) 1082 are fixed onto a bottom face 3031 of the sensor head 3030.

[0062]

Further, as shown in FIG 8, a tip end 3032 of the sensor head 3030 projects further behind the back face 3012 of the main body 3010. Hence, when the conveyance-object analyser 1000 is installed in such a manner that the back face 3012 abuts onto the collector conveyor 5002, the equipment provided under the bottom face 3031 of the sensor head 3030 is located above the collector conveyor 5002.

[0063]

Each of the left three-dimensional camera 1011 and the left RGB camera 1012 captures an image of the conveyance object Cob as an upstream image when the conveyance object Cob is in an upstream assessment zone Zu on the collector conveyor 5002 and while the conveyance object Cob is conveyed by the collector conveyor 5002 in a direction (see the arrow Ac in FIG 4) towards the left three-dimensional camera 1011 and the left RGB camera 1012.

[0064]

The upstream image captured by the left three-dimensional camera 1011 is transmitted in real time to the primary Al computer 1030 as data processed three-dimensionally (3D point cloud data) about the conveyance object Cob. The upstream image captured by the left RGB camera 1012 is simply forwarded in real time to the primary Al computer 1030 as raw data that is not subjected to special processing.

[0065]

Each of the right three-dimensional camera 1021 and the right RGB camera 1022 captures an image of the conveyance object Cob in a downstream assessment zone Zd on the collector conveyor 5002.

[0066]

The downstream image captured by the right three-dimensional camera 1021 is transmitted in real time to the primary Al computer 1030 as data processed three-dimensionally (3D point cloud data) about the conveyance object Cob. The downstream image captured by the right RGB camera 1022 is simply forwarded in real time to the primary Al computer 1030 as raw data that is not subjected to special processing.

[0067]

As shown in FIG 5, the left three-dimensional camera 1011 and the right RGB camera 1022 are substantially disposed back to back under the bottom face 3031 of the sensor head 3030. In addition, the left RGB 1012 and the right three-dimensional camera 1021 are substantially disposed back to back under the bottom face 3031 of the sensor head 3030. [0068]

In the present embodiment, the conveying direction Ac of the collector conveyor 5002 is the right direction (see FIG 1) with respect to the conveyance-object analyser 1000. Accordingly, lenses of the left three-dimensional camera 1011 and the left RGB camera 1012 both face towards an upstream direction (see an arrow Au in FIG 5). On the other hand, lenses of the right three-dimensional camera 1021 and the right RGB camera 1022 both face towards a downstream direction (see an arrow Ad in FIG 5). [0069]

As shown in FIGs 0101 and 0205, the upstream assessment zone Zu on the collector conveyor 5002 is a virtual area that approximately corresponds to a photographable area of the cameras towards the upstream direction Au (i.e. the left three-dimensional camera 1011 and the left RGB camera 1012 in this embodiment).

The downstream assessment zone Zd on the collector conveyor 5002 is another virtual area that approximately corresponds to a photographable area of the cameras towards the downstream direction Ad (i.e. the right three-dimensional camera 1021 and the right RGB camera 1022 in this embodiment).

[0070]

Further, the upstream assessment zone Zu is provided in the upstream direction Au (i.e. the left side in the present embodiment) with respect to the sensor head 3030. On the other hand, the downstream assessment zone Zd is provided in the downstream direction Ad (i.e. the right side in the present embodiment) with respect to the sensor head 3030.

This means that the upstream assessment zone Zu is set in the upstream direction Au with respect to the downstream assessment zone Zd. The downstream assessment zone Zd is set in the downstream direction Ad with respect to the upstream assessment zone Zu.

[0071]

Each of the upstream assessment zone Zu and the downstream assessment zone Zd is set as a stationary area regardless of running and stopping of the collector conveyor 5002.

[0072]

A zone-separation distance (see a sign Dz in FIG 4), which is a gap distance between the upstream assessment zone Zu and the downstream assessment zone Zd, is set in accordance with the processing power of the primary Al computer 1030.

The zone-separation distance Dz can be adjusted by changing the height of the sensor head 3030, as shown by the arrow Ash in FIG 6.

In addition, the zone-separation distance Dz can be adjusted by individually changing mounting angle of the left three-dimensional camera 1011, the left RGB camera 1012, the right three-dimensional camera 1021, and the right RGB camera 1022, with respect to the sensor head 3030.

For example, if the processing power of the primary Al computer 1030 is relatively higher, then the zone-separation distance Dz may be set relatively shorter. On the other hand, if the processing power of the primary Al computer 1030 is relatively lower, then the zone-separation distance Dz may be set relatively longer.

[0073]

As shown in FIG 5, the left light 1081 is a lighting unit that illuminates the upstream assessment zone Zu so that the left three-dimensional camera 1011 and the left RGB camera 1012 clearly capture the upstream image of the conveyance object Cob.

The right light 1082 is another lighting unit that illuminates the downstream assessment zone Zd so that the right three-dimensional camera 1021 and the right RGB camera 1022 clearly capture the downstream image of the conveyance object Cob.

[0074]

As shown in FIG 4, the primary Al computer (image-analysing module) 1030 is a single-board computer that is capable of neural network processing based on image data. The primary Al computer 1030 is communicably connected to the network hub 1050.

[0075]

The primary Al computer 1030 is also communicably connected to each of the left three-dimensional camera 1011, the left RGB camera 1012, the right three-dimensional camera 1021, and the right RGB camera 1022. The Al computer 1030 executes image-processing applications for image classification, object detection in images and division of images, in addition to the neural network processing.

[0076]

Based on a learning model 1031 stored in a storage device of the primary Al computer 1030, the primary Al computer 1030 analyses the upstream image and/or the downstream image, and recognises the 'conveyance state' with regard to the conveyance object Cob on the collector conveyor 5002. Further, the primary Al computer 1030 determines whether the recognised conveyance state satisfies the 'forwarding condition' of the automatic baggage handling system Bap.

In the meantime, the determination on the conveyance state of the conveyance object Cob in the upstream assessment zone Zu is referred to as 'primary determination'. On the other hand, the determination on the conveyance state of the conveyance object Cob in the downstream assessment zone Zd is referred to as 'secondary determination'. [0077]

The learning model 1031 defines an assessment standard. Specifically, the learning model 1031 is a machine learning model generated based on a vast amount of raw data (training data), in which a number of conveyance objects Cob are photographed individually, and a learning data set corresponding to the factors of the conveyance objects Cob. The learning model 1031 is stored in the storage device of the primary Al computer 1030. [0078]

In addition, the primary Al computer 1030 (training module) updates the learning model 1031, in response to an overriding command containing an overriding reason entered by an operator via the touch panel 1070 based on the entered overriding reason and the upstream image and/or the downstream image. [0079]

In the meantime, the 'conveyance state' includes various factors with regard to the conveyance object Cob (e.g. dimensions, orientation, type, condition of accessories, presence or absence of overlapping, distance from other conveyance objects Cob, presence or absence of transport tub, open/close state, presence or absence of baggage tag, and so on). [0080]

For example, when a conveyance object Cob is conveyed by the collector conveyor 5002, the primary Al computer 1030, based on the learning model 1031 and the upstream image and/or the downstream image, recognises the type of the conveyance object Cob as a suitcases (hard bags), a duffel bag (soft bags), a backpack (soft bags), a sports bag (soft bags), or others.

[0081]

In addition, when a conveyance object Cob has wheels (movable parts), the primary Al computer 1030 recognises whether the conveying direction Ac of the collector conveyor 5002 and a projecting direction of the wheels protrude from the conveyance object Cob is substantially the same. Further, the primary Al computer 1030 recognises whether the orientation of the conveyance object Cob is either upright or lying. As shown in FIG 9, wheels 5005 of a suitcase Tobi protrudes towards in a direction that is the same as the conveying direction Ac of the collector conveyor 5002. [0082]

Further, the primary Al computer 1030 recognises whether an auxiliary piece such as a handle (movable parts) of a conveyance object Cob is appropriately stowed in the conveyance object Cob. In this regard, FIG 9 shows a handle 5006 of a suitcase Tob2 is extended and not properly stowed away into the suitcase Tob2, and orientation of the suitcase Tob2 is upright. [0083]

In addition, FIG 9 shows a suitcase Tob3 that is lying on the collector conveyor 5002 (i.e. the orientation of the suitcase Tob3 is proper), but a handle 5007 of the suitcase Tob3 is extended and not properly stowed away into the suitcase Tob2.

[0084]

The primary Al computer 1030 also recognises whether one conveyance object Cob overlaps another conveyance object Cob (see suitcases Tob4 and Tob5 in FIG 9). [0085]

In addition, the primary Al computer 1030 recognises whether a distance (see the arrow DI in FIG 9) between one conveyance object Cob and another conveyance object Cob is shorter than a threshold distance. [0086]

In addition, if a baggage tag is affixed to a conveyance object Cob, the primary Al computer 1030 recognises the affixed baggage tag. [0087]

In addition, if there is relatively large damage to a conveyance object Cob, the primary Al computer 1030 recognises the damage on the conveyance object Cob.

[0088]

Further, the primary Al computer 1030 recognises whether a conveyance object Cob is placed in a transport tub.

[0089]

The primary Al computer 1030 also recognises whether a shoulder strap is properly bundled if a conveyance object Cob has the shoulder strap. [0090]

The primary Al computer 1030 also recognises whether a conveyance object Cob is properly closed (i.e. the open/close state of the conveyance object Cob). [0091]

That is, the primary Al computer 1030 assesses a conveyance object Cob as an 'improper-state object', which is an object that is not in a state that can be conveyed, if the primary Al computer 1030 recognises at least one of:

(i) the size of the conveyance object Cob exceeds the prescribed size (height, width and depth);

(ii) the orientation of the conveyance object Cob is upright; (Hi) the conveying direction Ac of the collector conveyor 5002 and the projecting direction of wheels of the conveyance object Cob are substantially the same; and

(iv) a handle is not appropriately stored in the conveyance object Cob;

(v) there is no baggage tag affixed to the conveyance object Cob;

(vi) the conveyance object Cob is recognised as a soft bag that is not placed in a transport tub;

(vii) a shoulder strap of the conveyance object Cob is not properly bundled; and

(viii) the conveyance object Cob is not properly closed. [0092]

The primary Al computer 1030 also assesses a conveyance object Cob as the 'improper-state object' if the primary Al computer 1030 recognises that there are extraneous items (i.e. passports, books, magazines, documents and so on) left on the conveyance object Cob on the collector conveyor 5002.

[0093]

In addition, the primary Al computer 1030 assesses a conveyance object Cob on the collector conveyor 5002 as an 'unknown object', if the primary Al computer 1030 cannot recognise the conveyance object Cob. [0094]

Further, the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob on the collector conveyor 5002 does not meets the forwarding condition, if the conveyance object Cob is assessed as at least one of the 'improper-state object' and the 'unknown object'. On the other hand, the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob on the collector conveyor 5002 meets the forwarding condition, if the conveyance object Cob is assessed as neither the 'improper-state object' nor the 'unknown object'. [0095]

The primary Al computer 1030 and the system computer 1060 ascertain the conveying direction Ac of the collector conveyor 5002 based on the installation information entered by an operator through the touch panel 1070. That is, the primary Al computer 1030 and the system computer 1060 determine that the conveying direction Ac of the collector conveyor 5002 is either rightwards (see FIG 1) or leftwards (see FIG 18) based on the upstream image and/or the downstream image.

[0096]

As shown in FIG 4, the network hub 1050 is a network switch that supports the TCP/IP protocol having power-over- Ethernet (POE) function. The network hub 1050 is communicably connected to each of the primary Al computer 1030, the secondary Al computer 1040, and the system computer 1060.

[0097]

The system computer 1060 is a computer that holistically control the conveyance-object analyser 1000. The system computer 1060 is communicably connected to each of the network hub 1050, the I/O controller 1130, and the touch panel 1070.

[0098]

The system computer 1060 displays the processing status by the primary Al computer 1030 on the touch panel 1070. For instance, FIG 10 shows an indication on the touch panel 1070 while the image analysis process by the primary Al computer 1030 is ongoing.

[0099]

When the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob does not meet the forwarding condition, then the system computer 1060 displays the problems of the conveyance state on the touch panel 1070. For example, an indication on the touch panel 1070 is shown in FIG 11 when the primary Al computer 1030 recognises: wheels of the conveyance object Cob projects to the conveying direction Ac; the conveyance object Cob requires a transport tub; and orientation of the conveyance object Cob is upright.

[0100]

The system computer 1060 has an external-communication module (remote accessing module) which is not shown in the FIGs. The system computer 1060 is communicably connected to the Internet 4050 via the external-communication module. This allows remote operators to connect to the system computer 1060 via the Internet 4050 and an external-communication module in order to remotely control the conveyance-object analyser 1000.

[0101]

The external-communication module (communication module) of the system computer 1060 also communicates with other external systems (e.g. SCADA systems, systems under the control of the International Air Transport Association (IATA), and other cargo systems). [0102]

The system computer 1060 has a diagnosing module (diagnosing module) that executes a diagnostic program for self-diagnosing the conveyance-object analyser 1000.

The system computer 1060 also has a logging module (logging module) that records the activity of the conveyance-object analyser 1000.

In the meantime, the diagnostic module and the logging module are not shown in the FIGs.

[0103]

The system computer 1060 turns on the LED unit (alarming device)

1090 via the I/O controller 1130 in accordance with the results of the primary and secondary determinations performed by the primary Al computer 1030. As a result, it is possible to visually notify operators of the conveyance state of the conveyance object Cob.

[0104]

The touch panel 1070 is an interface device provided on the top face 3013 of the main body 3010. The touch panel 1070 is communicably connected to the system computer 1060. The touch panel 1070 visually notifies various information to an operator in accordance with the commands from the system computer 1060. Further, the touch panel 1070 receives inputs from the operator and forward the inputs entered by the operator to the system computer 1060.

[0105]

The I/O controller 1130 is a control unit that is communicably connected to the conveyor controller 4060 of the system computer 1060 and the automatic baggage handling system Bap. Hence, the I/O controller 1130 controls the collector conveyor 5002 via the conveyor controller 4060 in accordance with control commands received from the system computer 1060.

[0106]

That is, the collector conveyor 5002 is controlled by the I/O controller 1130, which works in accordance with the control commands transmitted by the system computer 1060 according to the results of the primary and/or secondary determinations.

[0107]

If the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob, which is recognised based on the learning model 1031 and the upstream image, 'satisfy' (i.e. when the primary determination is a positive result), then the system computer 1060 does not actively control over the I/O controller 1130. Hence, in this case, the collector conveyor 5002 continues normal operation. As a result, the conveyance object Cob passes through the downstream assessment zone Zd, and continues travelling in the downstream direction Ad.

[0108]

On the other hand, if the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob, which is recognised based on the learning model 1031 and the upstream image, 'does not satisfy' (i.e. when the primary determination is a negative result), the system computer 1060, via the I/O controller 1130 and the conveyor controller 4060, controls the collector conveyor 5002 in such a matter that the conveyance object Cob is stopped in the downstream assessment zone Zd. This allows the right three-dimensional camera 1021 and the right RGB camera 1022 to capture the downstream image of the conveyance object Cob in a stationary state.

[0109]

If the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob, which is recognised based on the learning model 1031 and the downstream image, 'does not satisfy' (i.e. when the secondary determination is a negative result), the system computer 1060 does not actively control over the I/O controller 1130. Hence, in this case, the stationary state of the sending conveyor 5003 continues, so the conveyance object Cob is held as it is in the downstream assessment zone Zd. [0110]

Further, if the secondary determination is a negative result, the system computer 1060 executes a 'need-of-care control'. This need-of-care control includes a control for indicating the problems of the conveyance state of the conveyance object Cob on the touch panel 1070, and another control for turning on the LED unit 1090. Details of this need-of-care control will be described later using FIG 17.

[0111]

On the other hand, if the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob, which is recognised based on the learning model 1031 and the downstream image, 'satisfy' (i.e. when the secondary determination is a positive result), then the system computer 1060 sends a resume signal to the I/O controller 1130. Afterwards, the I/O controller 1130 restarts, via the conveyor controller 4060, the collector conveyor 5002 in the stationary state in response to the resume signal received from the system computer 1060. As a result, the conveyance object Cob on the collector conveyor 5002 travels in the downstream direction Ad. [0112]

In the meantime, a situation in which the conveyance state of the conveyance object Cob does not satisfy the forwarding condition in the upstream assessment zone Zu, but then satisfies the forwarding condition in the downstream assessment zone Zd, it could generally be assumed that the conveyance condition of the conveyance object Cob has been corrected with an action taken by an operator because of executing the need-of-care control which enables the operator to recognise that the conveyance object Cob held in the downstream assessment zone Zd does not meet the forwarding condition. [0113]

However, even if the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob does not meet the forwarding condition in the secondary determination, the I/O controller 1130 receives the resume signal from the system computer (overriding module) 1060 when the system computer 1060 executes an 'overriding operation'. [0114] This overriding operation is an operation that is executed up on when an input of the overriding command to the touch panel 1070 is completed. That is, when the input of the overriding command to the touch panel 1070 is completed, the system computer 1060 transmits the resume signal to the I/O controller 1130. Then, the I/O controller 1130, which received the resume signal, resumes the forwarding conveyer 5003 in the stationary state via the conveyor controller 4060 so that the conveyance object Cob travels in the downstream direction Ad.

[0115]

In other words, this overriding operation is carried out under a situation where the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob suspended in the downstream assessment zone Zd still does not meet the forwarding condition, but an operator completed the input of the overriding command into the touch panel 1070.

[0116]

Then, when this overriding operation is executed, the system computer 1060 restarts the collector conveyor 5002 via the I/O controller 1130 and the conveyor controller 4060 in order to send the conveyance object Cob in the downstream direction Ad.

Nevertheless, prior to executing the overriding operation (i.e. for completion of inputting the overriding command), it is required to input a reason (overriding reason) into the touch panel 1070.

[0117]

As shown in FIG 12, each of the check boxes for corresponding to the problem of the conveyance state of the conveyance object Cob is indicated on the touch panel 1070. Accordingly, an operator, who saw the indication on the touch panel 1070, checks an actual state of the conveyance object Cob, which is temporarily stopped in the downstream assessment zone Zd, and input the overriding reasons into the touch panel 1070 by entering ticks into the corresponding checkboxes.

[0118]

For example, assuming that the primary Al computer 1030 recognises that the size of a conveyance object Cob held in the downstream assessment zone Zd exceeds the predetermined specified size (height, width and depth), as a result, the primary Al computer 1030 determines that the conveyance object Cob is deemed as an improper-state object, and therefore the primary Al computer 1030 determines that the conveyance state of the conveyance object Cob does not satisfy the forwarding condition, but the operator has confirmed that the size of the conveyance object Cob stopped in the downstream assessment zone Zd is actually within the predetermined specified dimensions.

In this case, the operator enters a tick into the check box of 'Acceptable size', which is one of the overriding reasons displayed on the touch panel 1070 as shown in FIG 13, and then touches a "Submit" button indicated on the touch panel 1070, otherwise the overriding operation will not be executed. In other words, it is mandatory to include the overriding reason to complete inputting the overriding command.

[0119]

When the overriding operation is executed, the system computer 1060 forwards the overriding reason entered into the touch panel 1070 to the primary Al computer 1030. The primary Al computer 1030, which received this overriding reason, updates the learning model 1031 based on the upstream and downstream images corresponding to the conveyance object Cob subjected to the overriding operation and the overriding reason received from the system computer 1060.

[0120]

The states of the learning model 1031 can be restored as needed. For example, backup data of the learning model 1031 may be periodically saved and stored in the storage device of the primary Al computer 1030. This affords the ability to restore the learning model 1031 based on the backup data of any restore points. This is particularly useful in the event of any 'incorrect training data' which had been fed into and has corrupted the learning model 1031.

Alternatively, more simply, a factory default version of backup data of the learning model 1031 may be stored in the storage device of the primary Al computer 1030. According to this arrangement, the learning model 1031 can be restored to the factory default state at any time.

[0121]

In other words, even if operators repeatedly enter the overriding reason incorrectly, the learning model 1031 can be corrected by restoring the learning model 1031 as appropriate.

[0122]

The collector conveyor 5002 has an upstream-side optical sensor 4011 near the upstream assessment zone Zu. Further, the collector conveyor 5002 has a downstream-side optical sensor 4012 near the downstream assessment zone Zd.

[0123]

Each of these upstream-side optical sensors 4011 and the downstream-side optical sensor 4012 is communicably connected to the I/O controller 1130.

Hence, the system computer 1060 and the primary Al computer 1030 may detect the conveyance object Cob in the upstream assessment zone Zu by the upstream-side optical sensor 4011.

[0124]

Likewise, the system computer 1060 and the primary Al computer

1030 may detect the conveyance object Cob in the downstream assessment zone Zd by the downstream-side optical sensor 4012.

[0125]

With the above-mentioned arrangement, the conveyance-object controlling apparatus according to the present invention provides the following advantages.

[0126]

The assessment of the conveyance object Cob by the conveyance-object analyser 1000 is mainly performed in accordance with a main flowchart shown in FIG 14.

At step S001, when a conveyance object Cob, which is conveyed in the downstream direction Ad by the collector conveyor 5002, enters into the upstream assessment zone Zu, then the left light 1081 illuminates the upstream assessment zone Zu and the left three-dimensional camera 1011 and the left RGB camera 1012 capture an image of the conveyance object Cob as the upstream image.

Then, the primary Al computer 1030 performs the image recognition using the learning model 1031 for the captured upstream image. An example of the recognition result for the upstream image is shown in FIG 15. [0127]

Afterwards, in step S005, the primary Al computer 1030 determines whether the conveyance state of the conveyance object Cob on the collector conveyor 5002 satisfies the forwarding condition based on the recognition result of the upstream image.

[0128]

If the result of the determination in step S005 is positive (see Yes route from step S005), the system computer 1060 does not actively control the conveyor controller 4060. That is, in this case, the collector conveyor 5002 continues normal operation. Accordingly, the conveyance object Cob on the collector conveyor 5002 passes through the downstream assessment zone Zd and is further carried in the downstream direction Ad. [0129]

On the other hand, if the result of the decision in step S005 is negative (see No route from step S005), the system computer 1060 instructs the conveyor controller 4060 so that the conveyance object Cob on the collector conveyor 5002 is stopped within the downstream assessment zone Zd (step S010).

[0130]

In step S015, an image of the conveyance object Cob stationary in the downstream assessment zone Zd is captured as the downstream image by the right three-dimensional camera 1021 and the right RGB camera 1022. At this time, the right light 1082 illuminates the downstream assessment zone Zd.

Afterwards, the primary Al computer 1030 performs the image recognition using the learning model 1031 for the captured downstream image. An example of the recognition result for the downstream image is shown in FIG 16.

[0131]

Thereafter, in step S020, the primary Al computer 1030 determines whether the conveyance state of the conveyance object Cob on the collector conveyor 5002 satisfies the forwarding condition based on the recognition result of the downstream image.

If the determination result in this step S020 is positive (see Yes route from step S020), the system computer 1060 sends the resume signal to the I/O controller 1130. The I/O controller 1130 received the resume signal sends the resume request to the conveyor controller 4060. The conveyor controller 4060 received this resume request restarts the collector conveyor 5002 in the stationary state (step S025). As a result, the conveyance object Cob on is conveyed further in the downstream direction Ad. On the other hand, if the determination in step S020 is negative (see No route from step S020), the system computer 1060 does not actively control the I/O controller 1130 and the conveyor controller 4060. That is, in this case, the stationary state of the collector conveyor 5002 continues. As a result, the conveyance object Cob on the collector conveyor 5002 is kept holding as it is in the downstream assessment zone Zd.

[0133]

At this time, in step S030, the system computer 1060 executes the 'need-of-care control'. In this need-of-care control, as shown in FIG 17, step S045 and step S050 are executed simultaneously.

[0134]

Specifically, in step S045, the system computer 1060 indicates the problems of the conveyance state of the conveyance object Cob on the touch panel 1070 (see FIG 11, for example). This allows an operator to promptly know the problems of the conveyance object Cob held in the downstream assessment zone Zd.

[0135]

In step S050, the system computer 1060 turns on the LED unit 1090 via the I/O controller 1130. This visually provides a call an operator's attention even if he/she is in a short distance away from the conveyance-object analyser 1000. That is, it is possible to notify the operator that the conveyance object Cob held in the downstream assessment zone Zd requiring some assistance.

[0136]

Afterwards, in step S035 of FIG 14, the touch panel 1070 waits for an input of the overriding command from an operator. As described above, in order to complete this overriding command input, it is required to input the overriding reason into the touch panel 1070. That is, in this step S035, the operator must enter the overriding reason (see FIG 13) into the touch panel 1070, and touch the 'Submit' button on the touch panel 1070 to complete the input of the overriding command, then the overriding operation is executed (Yes route from step S035).

[0137]

On the other hand, when the input of the overriding command has not been completed (see No route from step S035), the conveyance object Cob on the collector conveyor 5002 is held as it is at the downstream assessment zone Zd.

[0138]

Thereafter, when the input of the overriding command is completed and the overriding operation is executed (see Yes route from step S035), then in step S040, the system computer 1060 forwards the overriding reason entered via the touch panel 1070 to the primary Al computer 1030. The primary Al computer 1030 updates the learning model 1031 based on the upstream and downstream images of the conveyance object Cob, which was subject to the overriding operation, and the overriding reason received from the system computer 1060.

[0139]

Then, in step S025, the system computer 1060 transmits the resume signal to the I/O controller 1130. The I/O controller 1130 received the resume signal, via the conveyor controller 4060, restarts the collector conveyor 5002 in the stationary state. As a result, the conveyance object Cob is further carried in the downstream direction Ad.

[0140]

The present invention is not limited to the above-described embodiment and the variant thereof, and can be variously modified and implemented.

[0141] In the above embodiment, the conveyance-object analyser 1000 is installed in the airport AP, but is not limited to this. For example, a conveyance-object controlling apparatus Baz may be installed in factories, plants, and/or warehouses.

[0142]

In the above embodiment, the conveyance-object analyser 1000 is installed near the collector conveyor 5002, partly because, normally, there are more ground-handling staff working in the FOH than the BOH. Accordingly, if the conveyance-object controlling apparatus Baz is installed in the FOH of the airport AP, then the conveyance object Cob may be quickly attended by the ground-handling staff in the FOH.

Nevertheless, the installation point of the conveyance-object analyser 1000 is not limited to this. For example, the conveyance-object analyser 1000 may be installed near the transport conveyor 5003 close to the inlet of the BOH in the airport AP.

[0143]

In the above embodiment, as shown in FIG 1, the conveying direction Ac of the collector conveyor 5002 is in the right direction with respect to the conveyance-object analyser 1000, but it is not limited to this. For example, as shown in FIG 18, even if the conveying direction Ac of the belt conveyor is the opposite, a conveyance-object controlling apparatus Baz may be compatible.

[0144]

That is, when the conveying direction Ac of the collector conveyor 5002 is the I eft- di recti on with respect to the conveyance-object analyser 1000 as shown in FIG 18, both of the right three-dimensional camera 1021 and the right RGB camera 1022 function as the upstream capturing device that captures the upstream image. Likewise, both of the left three-dimensional camera 1011 and the left RGB camera 1012 function as the downstream capturing device that captures the downstream image.

[0145]

In this case, the upstream assessment zone Zu on the collector conveyor 5002 is a virtual area that approximately corresponds to a photographable area of the cameras towards the upstream direction Au (i.e. the right three-dimensional camera 1021 and the right RGB camera 1022). Likewise, the downstream assessment zone Zd on the collector conveyor 5002 is another virtual area that approximately corresponds to a photographable area of the cameras towards the downstream direction Ad (i.e. the left three-dimensional camera 1011 and the left RGB camera 1012). [0146]

The conveyance-object analyser 1000 is robust and can work with belt conveyors, roller conveyors, slat conveyors, mesh conveyors and the like. The conveyance-object analyser 1000 can also be adapted to work in scenarios where the conveyor has been replaced by robot-type vehicles such as Automated Guided Vehicles (AGVs). [0147]

As a variant of the above-described embodiment, the primary Al computer 1030 may recognise a baggage ID printed on a baggage tag which is uniquely affixed to the conveyance object Cob, based on the upstream image and/or the downstream image. [0148]

In this case, the system computer 1060 may forward the baggage ID recognised by the primary Al computer 1030 to other external systems (e.g. systems under the control of the International Air Transport Association (IATA), other cargo systems, and so on).

Further, the system computer 1060 may transmit the upstream and/or the downstream images to other external systems along with the baggage ID recognised by the primary Al computer 1030. [0149]

Further, as a variant of the embodiment described above, the primary Al computer 1030 may transmit, via the system computer 1060 to other external systems, the upstream image and/or the downstream image of the conveyance object Cob, which did not satisfy the forwarding condition. [0150]

In the above embodiment, the upstream-side optical sensor 4011 is provided near the upstream assessment zone Zu. Further, the downstream-side optical sensor 4012 is provided near the downstream assessment zone Zd. However, it is not limited to this configuration. [0151]

For example, without using the upstream-side optical sensor 4011, the primary Al computer 1030 may detect an entry of a conveyance object Cob into the upstream assessment zone Zu based on the upstream image. Similarly, even without using the downstream-side optical sensor 4012, the primary Al computer 1030 may detect an entry of a conveyance object Cob into the downstream assessment zone Zd based on the downstream image. [0152]

Nevertheless, by using the upstream-side optical sensor 4011 and downstream-side optical sensor 4012, the processing load of the primary Al computer 1030 may be reduced. [0153]

As a variant of the above-described embodiment, the system computer 1060 may actuate a buzzer (not shown in FIGs) as an alarming device in accordance with the primary and secondary determinations performed by the primary Al computer 1030. As a result, it is possible to notify aurally an operator of the conveyance status of the conveyance object Cob.

[0154]

Further, as shown in FIG 19, wheels 3018 may be added to the bottom face 3017 of the conveyance-object analyser 1000 described in the above embodiment. According to this arrangement, an operator can easily move the conveyance-object analyser 1000 to the desired locations.

[0155]

Furthermore, as shown in FIG 20, a CCTV camera 6010 may be added to the conveyance-object analyser 1000 described in the above embodiment. For example, the CCTV camera 6010 is communicably connected to the network hub 1050. The CCTV camera 6010 is supplied electric power from the network hub 1050 with POE. Although there are no restrictions on the installation position of the CCTV camera 6010, it may be preferable to put the CCTV camera 6010 at the sensor head 3030.

The CCTV images captured by the CCTV camera 6010 are recorded and saved to a data storage server 4020 which is communicably connected to the network hub 1050 of the conveyance-object analyser 1000.

[0156]

The CCTV images captured by the CCTV camera 6010 are processed by a secondary Al computer 1040 which is installed in the main body 3010 of the conveyance-object analyser 1000. The secondary Al computer 1040 is a single-board computer that is capable of neural network processing based on image data. The secondary Al computer 1040 is communicably connected to the network hub 1050.

[0157]

The secondary Al computer 1040 detects a suspicious person (e.g. a person who has intruded into an area near the conveyance-object analyser 1000 or on the collector conveyor 5002 and a child who has accidentally strayed on the collector conveyor 5002) by analysing and recognising people captured in the CCTV image, based on a CCTV learning model 1041 stored in the storage device of the secondary Al computer 1040 and the CCTV images taken by the CCTV camera 6010. The analysis and recognition results of the CCTV images obtained by the secondary Al computer 1040 are stored in the data storage server 4020.

Accordingly, the recognition result of the CCTV images and the CCTV images stored on the data storage server 4020 can be checked as necessary.

Further, when the secondary Al computer 1040 detects a suspicious person based on the CCTV images, the system computer 1060 may urgently stop the collector conveyor 5002 via the I/O controller 1130 and the conveyor control unit 4060.

Also, when the secondary Al computer 1040 detects a suspicious person, the system computer 1060 may send a signal, which is for notifying that the suspicious person has been detected, to other external systems (e.g. a security system of the airport).

[0158]

The primary Al computer 1030 may determine the conveying direction Ac of the collector conveyor 5002 based on the upstream image and/or the downstream image. That is, the primary Al computer 1030 judges the conveying direction Ac of the collector conveyor 5002 is either rightwards (see FIG 1) or leftwards (see FIG 18) based on the upstream image and/or the downstream image.

[0159]

In the above embodiment, if the result of the determination at step S005 shown in FIG 701 is positive (see Yes route from step S005 in FIG 14), the right three-dimensional camera 1021 and the right RGB camera 1022 does not capture an image (i.e. the downstream image), but it is not limited to this.

For example, as shown in FIG 21, even if the primary Al computer 1030 determines that a conveyance object Cob on the collector conveyor 5002 satisfies the forwarding condition (see Yes route from step S005 in FIG 21), then the right three-dimensional camera 1021 and the right RGB camera 1022 may capture an image of the conveyance object Cob as the downstream image.

According to this arrangement, the learning model 1031 can be trained more based on a large number of the downstream images.

[0160]

As shown in FIG 22, even if the collector conveyor has a multiple of conveyors such as a first conveyor 5002-1 and a second conveyor 5002-2, the conveyance-object analyser 1000 is compatible with these conveyors 5002-1 and 5002-2 without problems.

That is, in the configuration as shown in FIG 22, the first conveyor 5002-1 can be operated independently of the second conveyor 5002-2 even while a conveyance object Cob is stopped in the downstream-side assessment zone Zd on the second conveyor 5002-2. In other words, even when the operation of the second conveyor 5002-2 is suspended, the first collector conveyor 5002-1 can be operated.

Accordingly, even when the conveyance object Cob is stopped in the downstream-side assessment zone Zd, it is possible to suppress an impact onto handling process carried out by the automatic baggage handling system Bap.

[0161]

As described in detail above, according to the conveyance-object controlling apparatus and method of the present invention, it is possible to assess appropriately whether the conveyance state of various types of conveyance objects meets different forwarding conditions for each automatic conveyance system so that the conveyance objects are precisely controlled in accordance with the assessed conveyance state.

[0162]

Specifically, according to the conveyance-object controlling apparatus and method of the present invention, a human operator is enabled to swiftly become aware of a conveyance object, which is halted in the downstream assessment zone having some problems, does not meet the forwarding condition.

Further, it is also possible to quickly and automatically resume temporarily halted forwarding of the conveyance object in the downstream direction once the problems of the conveyance object are fixed by the operator as the conveyor is automatically restarted when satisfying of the forwarding condition is determined based on the downstream image.

[0163]

In addition, it is possible to improve the accuracy of determination on the conveyance state by recognising the conveyance state of the conveyance object based on the learning model and the upstream image and/or the downstream image, and determining whether the recognised conveyance state satisfies the forwarding condition.

[0164]

In addition, it is possible to improve automatically the accuracy of the learning model by simply carrying out the normal handling procedures for the conveyance object.

That is, an operator is simply required to input the reason why the operator needs to execute the overriding operation, but in fact, the learning model is updated based on this reason (i.e. the overriding reason) and the upstream and/or downstream images. Accordingly, the accuracy of the learning model can be automatically improved without educating operators how to update the learning model.

[0165]

In addition, it is possible to easily and reliably capture the upstream and downstream images by arranging the upstream capturing device and the downstream capturing device being adjacent to one another and orientated in opposing directions. [0166]

Further, it is possible to capture clearly the upstream image and the downstream image of the conveyance object from above by mounting the upstream capturing device and the downstream capturing device in the head unit.

[0167]

Further, it is possible to capture clearly the upstream image and the downstream image by illuminating the upstream assessment zone and the downstream assessment zone.

[0168]

In addition, it is possible to capture easily and clearly the upstream and downstream-side images by providing the upstream and downstream-side lighting devices at the head unit.

[0169]

In addition, the zone-separation distance is adjustable according to the processing capability related to the assessment of the conveyance state.

For example, if the processing power of the image-analysing module is relatively higher, then the zone-separation distance may be set relatively shorter. On the other hand, if the processing power of the image-analysing module is relatively lower, then the zone-separation distance may be set relatively longer.

[0170]

Specifically, in the above embodiment, the collector conveyor 5002 continues its operation while the primary Al computer 1030 determines whether the conveyance state of the conveyance object Cob on the collector conveyor 5002 satisfies the forwarding condition based on the learning model 1031 and the upstream image (i.e. while the primary determination is carried out). As a result, the conveyance object Cob continues to move in the downstream direction Ad even while the processing of the primary determination is ongoing. However, in this case, the right three-dimensional camera 1021 and the right RGB camera 1022 could not capture the conveyance object Cob, if a result of the primary determination came out after the conveyance object Cob has passed the downstream assessment zone Zd and further moved in the downstream direction Ad.

[0171]

For this reason, the zone distance is set taking into account the processing power of the image-analysing module (primary Al computer 1030).

[0172]

In addition, it is possible to seek quickly and reliably a possible assistance from an operator by visually and/or aurally notifying an operator of a result of the determination whether the conveyance state of the conveyance object meets the forwarding condition.

This also allows the operator to do other his/her work while the conveyance state of the conveyance object satisfies the forwarding condition until the operator gets a visual and/or aural notification. Accordingly, the work efficiency of the operator may be improved.

[0173]

In the claims which follow and in the preceding description of the invention, except where the context requires otherwise due to express language or necessary implication, the word "comprise" or variations such as "comprises" or "comprising" are used in an inclusive sense, i.e. to specify the presence of the stated features but not to preclude the presence or addition of further features in various embodiments of the invention.

[0174]

The reference to any prior art in this specification is not, and should not be taken as, an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge. [0175]

Reference Signs List

1000 conveyance-object analyser (conveyance-object controlling apparatus)

1011 left three-dimensional camera (upstream capturing device)

1012 left RGB camera (upstream capturing device)

1021 right three-dimensional camera (downstream capturing device)

1022 right RGB camera (downstream capturing device)

1030 primary Al computer (image-analysing module, training module)

1031 learning model

1060 system computer (overriding module)

1081 left light (upstream lighting device)

1082 right light (downstream lighting device)

1090 LED unit (alarming device)

1130 I/O controller (conveyor-controlling module)

3010 main body

3020 pillar

3030 sensor head (head unit)

5002 collector conveyor (conveyor)

5003 transport conveyor (conveyor)

Baz conveyance-object controlling apparatus

BC conveyer

Cob conveyance object

Dz zone-separation distance

Zd downstream assessment zone

Zu upstream assessment zone