The invention refers to a method for detecting a door status of an aircraft door.

Nowadays the docking of the passenger boarding bridge (PBB) to an aircraft is performed manually. An operator is using a joystick, giving immediate operation signals to the drive means. This kind of docking is time consuming and requires a well-trained operator. In case the operator is still busy at another PBB, the docking is delayed, which leads to costly delays in ground handling of aircrafts. Consequently, the market requires a possibility of automatic docking, in which no fully trained and certified operator is needed through the complete docking procedure anymore. In the context of automatization of the procedure in connection with the docking procedure it has found out, that it is helpful to get a confirmation of the status of the aircraft door.

WO 2016 / 038760 A1 discloses a door status detector for an aircraft door. The door closing status detector may be a photodetector such as an infrared sensor. Here the opening and closing of the door is detected by the change of amount of the infrared energy due to reflection and transmission of the infrared light from the detector by the door.

It is the object of the invention to provide an alternative, in particular improved, solution for detecting the door status of an aircraft. This object is solved by a method, an image recognition system and an arrangement according to the main claims; embodiments are subject to the subclaims and the description.

The inventive method for detection a door status of an aircraft door comprising the following steps:

taking a first image of the aircraft door;

- detecting at least a reference indicator in the first image and determining a first position of the reference indicator;

- taking a third image of the aircraft door after taking said first image

- searching for said reference indicator in the third image and detecting a third position of said reference indicator;

- determining that the door status is closed if the third position conforms to the first position.

Within this application the term door status is meant as an indicator whether the door is open or closed. The term door status detector is meant as a detector which can detect the door status. By comparing the third image with the first image consistencies between said images can be obtained. On the basis of these consistencies the door status is detected. If there are no consistencies, the door status may be detected as“not closed”, consequently the door status can be determined as“open”.

In an embodiment the door status is detected as“open”, if it is not possible to detect that the third position conforms to the first position. Here any status which is not“closed” is assessed as“open”, even if the door is merely partially opened.

In an embodiment an optional second image is taken between taking the first image and the third image, and it is searched for the reference indicator; it is determined that the door status is open if the reference indicator is not found in the second image in the first position and/or in the second image at all. Here the door status of an aircraft may be detected, in which the door performs a rotating movement during opening, e.g. the B737. Unless the indicator is not found in its first position, the door is not assessed as being closed.

It is to be understood that using the term“third image” does not require the provision of at least three images. Rather the term“third”, as well as the terms“first” and“second” are to be understood as terms making the individual images mentioned within this invention distinguishable from each other.

The aforementioned embodiments may be based on the principle of exclusion. Since it is not determined that the door status is closed it is concluded that the door is open. However the door open status may not be verified.

In an embodiment a verified door open status is provided, in particular the determination of the open door status is not just based on the principle of exclusion.

In an embodiment is determined that the door is open if the step analyzing delivers the result, that the reference indicator is found in the second image at a second position which is different to the first position, in particular wherein the difference between the first position and the second position is more than 20cm, in particular measured in longitudinal direction of the aircraft. In contrast to the aforementioned principle of exclusion here the open status can be verified by detecting the indicator at a certain position, distant from its original position in the first image. Consequently it can be detected, that the reference indicator has performed a certain amount of movement, which can be put on the same level as an opening movement of the door. In an embodiment before determining the door status, aircraft specific information is retrieved, in particular from database. The aircraft specific information refers to the aircraft, to which the aircraft door belongs. The aircraft specific information may comprise information about indicators, in particular their shape and/or position within the aircraft fuselage. This may help to identify a certain visual structure as a reference indicator.

In an embodiment the retrieved information contains data whether the door is a parallel opening door or a rotating door (door type information). In particular the system may know that the B737 has a rotating door and the A320 has a parallel opening door. An information containing the aircraft type may already implicitly contain the required door type information.

In an embodiment it is detected from the taken images, whether the door is a parallel opening door or a rotating door.

In the inventive method for operating a passenger boarding bridge, the method for detection of the opening/closing status of a door of an aircraft is used as described above. Thereby the first image is taken after the passenger boarding bridge has started docking the aircraft and when the door is closed; the third image is taken after the door has been opened and passengers have entered the aircraft. Now the door status is relevant in particular for starting the next steps for automated undocking. As long as the door status is not detected as closed, in particular undocking must not be started. In particular the passenger boarding bridge is not undocked before it is determined, that the third position of at least one indicator conforms to the first position of said indicator.

The invention is explained in more detail by means of the figures, the figures show:

Fig. 1 an aircraft and a passenger boarding bridge in top view;

Fig. 2 the front area of an aircraft of a first type in side view during the inventive method;

Fig. 3 the front area of an aircraft of a second type in side view during the inventive method.

Figure 1 shows an arrangement, to which the invention can be applied. An aircraft 1 is located at a gate of an airport. A passenger boarding bridge 10 is to be docked to the aircraft 1. An image recognition device 20 is provided. The image recognition device 20 comprises a camera 21 (in particular a stereo camera) which is located at the passenger boarding bridge 1, facing a door 3 of the plane 1, when the aircraft 1 is docked or shortly before docking is finished. The term“camera” comprises also any suitable image recording device. The image recognition device 20 further comprises a computer 23 for analyzing any images taken by the camera 21. The computer 22 may be located within the passenger boarding bridge 1 or may be located at a remote location and may be connected via a data connection 24 to the camera. The computer 22 has access to a database 23. The database 23 may be

incorporated into the computer 22 or may be located remote to the computer 21. If necessary a data connection 24 can be established between the camera 21 and the computer 22 and/or between the computer 22 and the database 23, in particular via a hard wire or via the internet.

Figure 2 shows a front area of an aircraft 1 of a first type in side view during the steps of the method. The aircraft 1 has a fuselage and a door 3a closing a door frame 5 within the fuselage 2.

The image recognition device 20 is acquired for taking several images 6 of the fuselage 2, at least partially covering the door 3a. A first image 6a is taken when the door is still closed (figure 2a). Here the image recognition device 20 detects several reference indicators 4 on the door surface, which may be the contour of a door opening lever 4a, a window ring 4b or a text field 4c.

At first a door is detected within the image. A suitable method for detecting a door within the fuselage of an aircraft by image recognition is described in European patent application 18 382 372.3 and any other patent application claiming its priority, not yet published.

In a step when the door 3 is still closed (figure 2a), at least one of the reference indicators 4, e.g. indicator 4a, is detected and it’s initial, first position Pl=(xl,yl,zl) within an coordinate system x,y,z is stored e.g. in the database 23. Instead of the indicator 4a one or more of the other indicators may be used. Subsequently when the door 3 is opened (figure 2b), the indicators 4a will move together with the door 3 into an opened position. At the original position of the door 3 the empty door frame 5 gets visible, as shown in figure 2b.

Now the image recognition device 20 takes a second image 6b. The image recognition device 20 searches in the second image 6b for the stored reference indicator 4a-c. The image recognition device 20 finds the same indicators 4a in a second position P2 (x2, y2, z2) which is about lm left of the first position. From this positional difference it is determined, that the door 3 is open.

Subsequently when the door 3a is closing, the indicator 4a moves together with the door 3 back into its initial, first position PI, as shown in figure 2c. Now the image recognition device 20 takes a third image 6c and searches herein for the stored reference indicator 4a. The image recognition device 20 finds the same indicator 4a back in the first position PI . Based on this positional identity it is determined, that the indicators 4a is now again in it’s closed position, consequently it is determined, that the door 3a is closed.

The aforementioned procedure is applicable for a first type of aircraft, e.g an Airbus A320.

This type of aircraft has a door 3 performing parallel translation. Here the outer surface of the door always faces in the same direction, i.e. in direction to the camera 21.

In contrast thereto a second type of aircraft, e.g. a Boeing 737, has a door 3b, which performs a rotational movement. Figures 3 shows an area of the aircraft 1 of the second type in side view. During opening the door 3b is turned around about 180°. Consequently when the door 3b is opened, at least some of the indicators 4 are not visible anymore by the image recognition device. It may possible, that a window ring 4b may be still visible, but the appearance may be different to the shape when viewed from the other side. The other indicators 4a, 4c are not visible at all. Instead other components like an emergency chute container 7 or an opening lever 8 may dominate the image 6b taken in the open status of the door 3b (figure 3b).

But as analogously described for the first aircraft type also for the second aircraft type in the situation of figure 2b the image recognition device is not able to detect, that the position of indicators 4a-c detected during the closed status (figure 3a) can be found in their original position when the door 3 is open (figure 3b). Consequently the system cannot confirm that the door 3 is closed, consequently the door 3b is determined as being open.

In an embodiment the first picture 6a is taken when docking is finished or nearly finished, but before the door has been opened for the passengers to leave the aircraft. The second image (or several of second pictures) are taken before undocking has been started. Now it can be determined that the door is still open. The status“door open” leads to a signal, which prevents the passenger boarding bridge from undocking.

Not before the picture recognition device 20 provides a signal“door closed” the passenger boarding bridge may start an automated or semi-automated undocking procedure. Undocking means, that a bridgehead of the passenger boarding bridge is moved away from the fuselage

2.

The camera 21 used within this method may be the same camera which may be used for supporting automated docking and/or automated docking. list of reference signs

1 aircraft

2 aircraft fuselage

3 aircraft door

4 reference indicator

5 door frame

6 area of captured image

7 emergency chute container

8 opening lever

10 passenger boarding bridge

20 image recognition device

21 camera

22 computer

23 database

24 data connection

P position

x,y,z coordinated