The present invention is related to a pinch protection method for a vehicle, a pinch protection system for a vehicle, a corresponding vehicle as well as a computer program product. Closing components of vehicles, such as tailgates, boot lids or passenger doors, in particular sliding doors or gull-wing doors, pose a risk on injury through pinching of body parts between the respective closing surfaces, in particular if the closure component is motorized or controlled automatically. Existing safety mechanisms stop the respective motors when a blockage is detected and thereby reduce the risk of injury. However, these mechanisms require sensing of a significant reaction force before they are activated.

Document US 2004/0040771 A1 describes a liftgate anti-pinch detector utilizing ultrasonic transmitters, which are originally used for distance measurements. The ultrasonic transmitters detect the presence of an object in an environment of the vehicle and movement of the liftgate is blocked in case an object has been detected.

Since this method is based on using distance sensors, the object detection is not selective and, in particular, different types of objects cannot be distinguished.

Furthermore, also the exact position of the object with respect to the tailgate cannot be determined reliably by using ultrasonic sensors.

It is therefore an object of the present invention to provide an improved concept for pinch protection for a vehicle, which further reduces the risk of injury or reduces the severity of a potential injury.

According to the improved concept, this object is achieved by the subject-matter of the independent claims. Preferred embodiments and further implementations are subject- matter of the dependent claims. The improved concept is based on the idea to monitor a pinch zone by means of a camera system and prevent closing of the closure component in case a person is detected in the pinch zone.

According to the improved concept, a pinch protection method for a vehicle is provided. According to the method, a predefined pinch zone of a closure component of the vehicle is monitored by a camera system, in particular of the vehicle. A person, which is present at least in part in the pinch zone, is detected by using a computing system, for example, of the vehicle and/or the camera system, based on the monitoring. A measure that prevents closing of the closure component is initiated by using the computing system based on the detection of the person.

The closure component can be understood as a movable component designed to open or close, respectively, or designed to cover or uncover, respectively, an opening of the vehicle. In particular, the closure component may comprise a tailgate, a boot lid, a hood or a passenger door, in particular a sliding door or a gull-wing door, of the vehicle.

The pinch zone can be understood as an environment of the closure component and/or the opening, wherein an object present in the environment has a high risk of being clamped between closing surfaces of the closure component and the opening, when the closing component is closed, in particular automatically.

In particular, the pinch zone is monitored by the camera system when the closure component is in an open position or in a partly open position, in other words, when the opening is not completely closed by the closure component.

In order to detect the person in the pinch zone, the computing system performs a person detection algorithm, in particular a computer vision algorithm for detecting the person.

Here and in the following, a computer vision algorithm can be understood as an algorithm capable of determining properties or characteristics related to a content of an image by analyzing image data, in particular camera data. In particular, that computer vision algorithm may comprise an object recognition or object classification algorithm, an object detection algorithm, a shape recognition algorithm and/or an object tracking algorithm. The computer vision algorithm may for example be designed as an algorithm trained based on machine learning. The monitoring of the pinch zone includes, in particular, capturing images repeatedly and generating respective camera data or camera signals by the camera system.

Detecting the person based on the monitoring can be understood as detecting the person based on the camera data or camera signals.

The computing system may comprise one or more computing units that may or may not be spatially distributed. In particular, the computing system may comprise circuitry of an imager chip of the camera system, an image signal processor, ISP, of the camera system or the vehicle and/or an electronic control unit, ECU, of the vehicle.

The measure preventing the closing of the closure component may, for example, comprise a measure for preventing a start of an automated closing procedure of the closure component, a measure to stop of an ongoing closing procedure and/or a measure to reverse the closing procedure.

A method according the improved concept allows to react to a potential pinching incident before there is any physical contact between the person and the closure component or the vehicle. Therefore, the latency regarding the pinch protection is reduced and in consequence the potential severity of injury is reduced. Furthermore, the pinch zone can be accurately monitored by the camera system so that the probability to prevent pinching is increased.

By detecting the person in the pinch zone in contrast to merely determining that some object is present in a roughly defined environment of the vehicle, the probability for initiating the measure preventing closing of the closure component even though it is not necessary, that is the probability for false positives, is reduced.

Moreover, in many cases there are no additional hardware costs, since the camera system as well as the computing system may be present in principle in the vehicle for other purposes.

According to several implementations of the method for pinch protection, the closure component is designed as an automated or motorized closure component.

In other words, the vehicle comprises a motor, for example an electric motor, to open and/or close the opening by moving the closure component. In such implementations, the method for pinch protection is particularly beneficial.

According to several implementations, an object is detected in the pinch zone by the computing system based on the monitoring for detecting the person in the pinch zone and a probability for the object being a person is determined by the computing system to be greater than a predefined threshold value based on the monitoring.

In other words, the computing system determines that there is an object in the pinch zone and what is the probability for the object corresponding to a person. This may, for example, be accomplished by a computer vision algorithm. The computing system may then compare the determined probability with the predefined threshold to identify the object as a person. In this way, the probability for false positives is further reduced. According to several implementations, a trained algorithm is used by the computing system to detect the person in the pinch zone.

The trained algorithm is in particular designed as an algorithm based on machine learning and/or computer vision.

The trained algorithm can for example be denoted as person detection algorithm. The trained algorithm is, in particular, designed to detect the object and to determine the probability that the object is the person. In such implementations, the person can be detected very reliably.

The trained algorithm may, for example, be started when the monitoring is started or the monitoring comprises carrying out the algorithm.

According to several implementations, the trained algorithm is based on a trained artificial neural network, in particular a convolutional neural network, CNN.

For training the algorithm, reference image data may be fed to the neural network, wherein the reference image data are known to either contain a person or not. Depending on an output of the neural network, parameters of the neural network, in particular weights of the neural network, are adapted using back-propagation to iteratively improve the performance of the neural network. Since neural networks, in particular CNNs, are known to provide powerful tools for image analysis and object detection as well as object classification, the reliability of person detection and therefore pinch protection may be further improved by such implementations and the probability for false positives may be reduced.

According to several implementations, a control unit of the vehicle is used to generate a control signal for initiating an automatic closing procedure of the closure component.

In particular, the control unit may transmit the control signal to a motor or a drive unit of the vehicle and the motor and/or the drive unit of the vehicle is configured to automatically close the closure component depending on the control signal.

In particular, the control signal may be provided to the computing system, too, for example for information purposes.

The control unit may be implemented separately from the computing system or may be part of the computing system.

According to several implementations, the monitoring of the pinch zone by the camera system is started by the computing system depending on the control signal.

In other words, the monitoring of the pinch zone is started, when the automated closing procedure of the closure component is initiated depending on the control signal.

In such implementations, the camera system may for example be brought into a dedicated monitoring mode of operation for carrying out the respective method steps. Apart from that, the camera may be used for other purposes, such as, for example, advanced driver assistance functions, in particular when the camera system is not in the monitoring mode.

According to several implementations, the person detection algorithm is started depending on the control signal, in particular is started when the automated closing procedure of the closure component is initiated, for example when the camera system is brought into the monitoring mode.

According to several implementations, the automated closing procedure is stopped based on the detection of the person, in particular in order to initiate the measure for preventing the closing of the closure component. Stopping the automated closing procedure may, in particular, comprise preventing the start of the automatic closing procedure, stopping the ongoing automated closing procedure and/or reversing a movement of the closure component at least in part.

According to several implementations, the pinch zone is monitored by the camera system during the automated closing procedure, in particular during the whole closing procedure, until the closing procedure is completed. In this way, safety is further increased.

According to several implementations, a body pose of the person is determined by using the computing system. The measure preventing closing of the closure component is initiated by using the computing system based on the body pose.

The body pose may, for example, be determined by the trained algorithm or by a further trained algorithm, in particular a further trained algorithm based on machine learning and/or computer vision. The further trained algorithm may for example be implemented based on the neural network or based on a further artificial neural network.

The body pose may indicate how high the probability that the person, in particular a limb or another body part of the person, might be hit or clamped by the closure component is, in case the closing procedure would be carried out.

In particular, if, only if or if and only if, it is determined by the computing system that the probability for a clamping or hitting a body part of the person is greater than a predetermined further threshold value based on the determined body pose, the measure for preventing the closing of the closure component is initiated.

By taking into account the body pose of the person, the probability for false positives may therefore be further decreased.

According to several implementations, a probable intention of the person is determined by using the computing system based on the monitoring and the measure preventing closing of the closure component is initiated by using the computing system based on the determined probable intention.

The probable intention may, in particular, include a probable specific movement of the person or a body part of the person and a respective probability for this movement. The probable intention of the person may, for example, be determined by the trained algorithm or the further trained algorithm or based on an additional algorithm. In particular, an optical flow may be determined based on the camera data to determine the probable intention of the person.

The probable intention may, in particular in combination with the body pose of the person, indicate the probability for the person or body part being hit or clamped by the closure component. Therefore, taking into account the probable intention, the probability for false positives may be further decreased.

According to several implementations, an acoustical or visual information signal is generated, in particular by using the computing system, when the measure preventing closing is initiated.

This may be particularly beneficial in implementations, wherein the closure component is designed as an automated or motorized closure component. Such implementations have the advantage that the person or a further person in the environment of the vehicle may be made aware of the reason for stopping the closing procedure.

According to the improved concept also a pinch protection system for a vehicle is provided. The pinch protection system comprises a camera system arrangeable or arranged at or on the vehicle to monitor a predefined pinch zone of a closure component of the vehicle. The pinch protection system comprises a computing system, which is configured to detect a person present in the pinch zone based on the monitoring and to initiate a measure preventing closing of the closure component based on the detection of the person.

Further implementations of the pinch protection system according to the improved concept follow directly from the various implementations of the method according to the improved concept and vice versa. In particular, the pinch protection system according to the improved concept may be configured or programmed to perform a method according to the improved concept or the system performs a method according to the improved concept.

According to the improved concept, also a vehicle including a pinch protection system according to the improved concept is provided. In particular, the camera system is arranged or mounted in or on the vehicle.

According to several implementations of the vehicle, the closure component comprises a, in particular automatic or motorized, tailgate or boot lid of the vehicle.

According to several implementations, in particular implementations where the closure component comprises the tailgate or boot lid, the camera system comprises a camera mounted on a rear part of the vehicle, for example above or below the tailgate or boot lid.

According to several implementations, the closure component comprises a passenger door of the vehicle, in particular an electric, motorized or automated passenger door.

According to several implementations, in particular implementations where the closure component comprises the passenger door, the camera system comprises a camera mounted on a rear-view mirror, in particular an external rear-view mirror, of the vehicle.

Further implementations of the vehicle according to the improved concept follow directly from the various implementations of the method according to the improved concept and vice versa.

According to the improved concept, also a computer program comprising instructions is provided. When the computer program or the instructions are executed by a pinch protection system according to the improved concept, the instructions cause the pinch protection system to carry out a pinch protection method according to the improved concept.

According to the improved concept, also a computer readable storage medium storing a computer program according to the improved concept is provided.

The computer readable storage medium as well as the computer program can be understood as respective computer program products comprising the instructions.

Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations without departing from the scope of the invention. Thus, implementations are also to be considered as encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations from the explained implementations. Implementations and feature combinations are also to be considered as disclosed, which do not have all of the features of an originally formulated independent claim. Moreover, implementations and feature combinations are to be considered as disclosed, in particular by the implementations set out above, which extend beyond or deviate from the feature combinations set out in the relations of the claims.

In the figures

Fig. 1 shows schematically a vehicle with an exemplary implementation of a pinch protection system according to the improved concept;

Fig. 2 shows schematically a vehicle with a further exemplary implementation of a pinch protection system according to the improved concept; Fig. 3 shows a passenger door of a vehicle; and

Fig. 4 shows a vehicle with gull-wing doors.

Fig. 1 shows an exemplary implementation of a vehicle 1 according to the improved concept and a block diagram of a corresponding exemplary implementation of a pinch protection system 7 according to the improved concept. Furthermore, a person 4 is shown.

The pinch protection system 7 comprises a camera system with a camera 3, which is, for example, mounted at a rear part of the vehicle 1 , in particular such that a boot lid 2 of the vehicle 1 lies within the field of view of the camera 3.

The pinch protection system 7 further comprises a computing system 5, which may comprise an ECU of the vehicle 1 and/or an ISP of the camera 3 and/or one or more further processing units of the camera 3 and/or of the vehicle 1. The boot lid 2 of the vehicle 1 is designed as a motorized or automatic boot lid. In particular, the vehicle 1 comprises a motor (not shown) designed to close the boot lid 2 when a respective control signal is generated by a control unit 6 of the vehicle 1 and transmitted to the motor or a drive unit coupled to the motor.

In case the boot lid 2 is open and the control unit 6 generates the control signal to initiate an automated closing procedure of the boot lid 2, the computing system 5 controls the camera 3 to monitor a pinch zone of the boot lid 2. The pinch zone of the boot lid 2 may comprise an environment of the boot lid 2 and/or of the corresponding component of the vehicle 1, in particular closing surfaces of the boot lid 2 and the corresponding component, which touch each other if the boot lid 2 is closed.

By monitoring the pinch zone, the camera 3 generates camera signals representing respective images of the pinch zone.

Based on the camera signals, the computing system 5 carries out a person detection algorithm to determine whether the person 4 is present in the pinch zone.

To this end, the computing system 5 may employ an object detection and/or object classification algorithm, for example based on a CNN.

If, according to the algorithm, the presence of the person 4 in the pinch zone is verified, the computing system 5 initiates a measure for preventing that the boot lid 2 is closed. In particular, the computing system 5 may generate a respective stop signal and transmit the stop signal to the control unit 6 and/or to the motor. The control unit 6 may control the boot lid 2, in particular the motor, to stop the movement of the boot lid 2 or not to start the movement or to reverse the movement of the boot lid 2.

Therefore, the risk of injury from pinching body parts between the closing surfaces is reduced.

In some implementations, the body pose and/or intention of the person 4 may be analyzed using a respective further algorithm by the computing system 5 to predict if there is any intention by the person 4 to put a limb into the pinch zone. Therefore, false detections of the system may be prevented and the motor may be prevented from stopping unnecessarily. In Fig. 2 a further exemplary implementation of the vehicle 1 according to the improved concept containing a further exemplary implementation of the pinch protection system 7 according to the improved concept is shown schematically.

In the situation of Fig. 2, the control unit 6 is designed to control a further motor (not shown) for automatically closing a passenger door 2’ of the vehicle 1. The camera system comprises a camera 3’ mounted, for example, on a rear-view mirror of the vehicle 1 to be able to monitor the pinch zone of the passenger door 2’ within the field of view of the camera 3’.

For the function of the pinch protection system 7, it is referred to the analog explanations with respect to Fig. 1.

Fig. 3 shows exemplarily a passenger door 2 of a car and a person 4 putting its hand in a respective pinch zone 8 between the passenger door 2 and the vehicle body.

In Fig. 4, exemplarily a vehicle 1 comprising gull-wing doors 2 and respective pinch zone 8 is shown.

A pinch protection system or method according to the improved concept may be used in the context shown in Fig. 3 and Fig. 4.

As described, the improved concept provides a suitable means to increase the safety of doors, tailgates or other closure components of vehicles, in particular, if they are electrified or motorized. The improved concept provides a particularly reliable way to prevent such injury.

In some implementations, a computer vision algorithm runs on a camera mounted on the vehicle, for example in the rear of the vehicle, over a rear window on the boot or under the boot or on the rear-view mirrors of the vehicle. The camera is connected to a computing system, which runs a pedestrian detection algorithm. This algorithm is, for example, activated when the tailgate of a boot lid or passenger door is triggered to close. Once a pedestrian is detected within the pinch zone, the algorithm will send a signal to the motor to prevent further movement and thereby reduce risk of injury from pinching body parts between the closing surfaces. In a complex more implementation, a person’s body pose and intention can be analyzed using computer vision to predict if they are intending to put a limb inside the vehicle during closing. This will reduce false detections and thus false stopping of the door closure due to pedestrians in the vicinity that are not intended to interact with the vehicle during the closing procedure. The algorithm continues to run until the tailgate, boot or door is fully closed to avoid injuries over the entire closing procedure. An audible or visual signal can also be provided to ensure that users are aware of the reason for stopping the closing procedure. The improved concept avoids an intrinsic latency as it is given by systems relying on force sensors or the like. The improved concept provides an earlier detection of the risk and may stop the closing procedure in a shorter period and therefore may reduce the potential severity of the injury by stopping the closure particularly early.