ACTIVATION BASED ON STRUCTURED LIGHT DETECTION

FIELD OF INVENTION

[0001] This application is related to vehicle electronics.

BACKGROUND

[0002] Vehicles have a number of power entry systems which may include automatic power lift gates, trunk lids or hoods, sunroofs, sliding doors, and doors. Users may want to use a touch-less activation of the power entry systems upon approaching the vehicle or when they are in the vehicle due to security or convenience reasons.

[0003] Most of the newer vehicles are also equipped with a Radio

Frequency (RF) system that is able to detect and verify if a keyfob/transponder associated with the vehicle is in the immediate proximity of the vehicle and "wake-up" the vehicle.

SUMMARY

[0004] Described herein is a system and method of activating vehicle power entry systems based on structured light detection.

[0005] The system includes a coherent light emitter (CLE), for example a laser or laser diode, and a camera with image processing capability integrated in the same package or a camera connected to an image processing Electronic Control Unit (ECU), and connected to the power entry system controller.

[0006] Upon detection and verification of the transponder/keyfob, when a user approaches the vehicle, the CLE will project a pattern of light on the ground. If the person steps on the image projected on the ground, the camera will detect the change on the projected pattern (structured light detection) and communicate to the power entry system controller to activate the system.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings wherein:

[0008] FIG. 1 is an embodiment of a Modular Automotive Camera Solution

(MACS) module;

[0009] FIG. 2 is an embodiment of an image processing engine (IPE) module that includes a power management module connected to a digital signal processor (DSP) and if applicable or available, a deserializer;

[0010] FIG. 3 is an example block diagram of the vehicle entry power systems activation;

[0011] FIG. 4 is an example flow diagram without a zone of interest;

[0012] FIG. 5 is an example flow diagram with a zone of interest;

[0013] FIG. 6 is an embodiment of a vehicle that enables a person to activate or trigger predetermined functions associated with electromechanical systems using gestures;

[0014] FIG. 7 is an example of a person detected and a CLE turned on; and

[0015] FIG. 8 is an example of a pattern change recognized as a trigger for further actions.

DETAILED DESCRIPTION

[0016] It is to be understood that the figures and descriptions of embodiments of a system and method of activating or triggering predetermined functions for vehicle electromechanical systems based on image recognition and radio frequency systems have been simplified to illustrate elements that are relevant for a clear understanding, while eliminating, for the purpose of clarity, many other elements found in typical vehicle systems. Those of ordinary skill in the art may recognize that other elements and/or steps are desirable and/or required in implementing the present invention. However, because such elements and steps are well known in the art, and because they do not facilitate a better understanding of the present invention, a discussion of such elements and steps is not provided herein. [0017] The non-limiting embodiments described herein are with respect to system and method of activating or triggering predetermined functions for vehicle electromechanical systems based on image recognition and radio frequency systems. Other electronic devices, modules and applications may also be used in view of these teachings without deviating from the spirit or scope as described herein. The system and method of activating or triggering predetermined functions for vehicle electromechanical systems based on image recognition and radio frequency systems may be modified for a variety of applications and uses while remaining within the spirit and scope of the claims. The embodiments and variations described herein, and/or shown in the drawings, are presented by way of example only and are not limiting as to the scope and spirit. The descriptions herein may be applicable to all embodiments of the system and method of activating or triggering predetermined functions for vehicle electromechanical systems based on image recognition and radio frequency systems although it may be described with respect to a particular embodiment.

[0018] Described herein is a system and method of activating vehicle power entry systems based on structured light detection.

[0019] The system includes a coherent light emitter (CLE), for example a laser or laser diode, and a camera with image processing capability integrated in the same package or a camera connected to an image processing Electronic Control Unit (ECU), and connected to the power entry system controller.

[0020] Figure 1 is an embodiment of a Modular Automotive Camera

Solution (MACS) module 100. The MACS module 100 includes a power management module 120 that is connected to a microcontroller 140, an image sensor 125, and if available, a serializer 130. The microcontroller (MCU) 140 may be further connected to a local interconnect network (LIN)/control area network (CAN) transceiver 150 and to the image sensor 125, which in turn may be connected to the serializer 130. Although the LIN/CAN transceiver is shown to describe the embodiments, any communication bus (COMM) may be used. For example the COMM may be a LIN bus, a CAN bus, a hardwired input, or any other automobile communication system or network

[0021] The MACS module 100 receives as inputs a battery voltage 155 and enable signal 160 from a passive entry system 195. The power management module 120 converts the battery voltage 155 and supplies the required voltages to the MCU 140, the image sensor 125, and if available, the serializer 130. The image sensor 125, for example, is a complementary metal— oxide— semiconductor (CMOS) sensor based camera. The enable signal 160 is sent to the power management module 120 and the MCU 140 when identification is authenticated or verified, (for example, a key fob is detected). The MACS module 100 can be configured to output an analog National Television System Committee (NTSC) composite video signal 185 from the image sensor 125 and/or a digital video 180 through a low voltage differential signaling (LVDS) output in the serializer 130. The LIN/ CAN transceiver 150 is configured to communicate via a LIN bus 170 with other vehicle electronic components or modules in the vehicle such as, for example, a passive entry system/module and a power lift gate module (PLGM) 190.

[0022] In general, the MACS module 100 is configured to recognize the presence of a user and to control the opening of, for example, a lift gate. The MACS module 100 is installed as a rear view camera module in the vehicle and works in conjunction with a passive entry system and PLGM. The MACS module 100 functions as a sensor as part of a lift gate drive system/motor control mechanism. The MACS module can also be used in a minimal configuration as an automotive rear view camera or can be used in conjunction with a master module for image processing in driver assist applications like birds eye view, blind spot detection, and the like.

[0023] Figure 2 is an embodiment of an image processing engine (IPE) module 200 that includes a power management module 210 connected to a digital signal processor (DSP) 215 and if applicable or available, a deserializer 220. The DSP 215 may be further connected to a LIN/CAN transceiver 230 and to the deserializer 220. The LIN/ CAN transceiver 230 is configured to communicate via a LIN/CAN bus 245 with other vehicle electronic components or modules in the vehicle such as, for example, a passive entry system/module and a PLGM 190.

[0024] The IPE 200 receives as input a battery voltage 240 and a video information signal 250. The power management module 210 converts the battery voltage 240 and supplies the required voltages to the DSP 215 and if available, the deserializer 220. The video signal 250 is received by the deserializer 220, which in turn sends the video information signal 250 to the DSP 215 for analysis.

[0025] The IPE 200 performs image processing and communicates with both the PLGM 190 and the MACS module 100. The IPE 200 receives the video information signal 250 and based on an image processing algorithm in the DSP 215 decides if a person is standing at the back of the car. The algorithm can be based on light, contour or color gradient changes. The algorithm differentiates between a person standing and a passerby. In case of a positive determination, the IPE 200 communicates to the PLGM 190 via the LIN transceiver 230 over the LIN/CAN bus 245 to open the lift gate.

[0026] Operationally, a person will need to carry the proper identification, i.e. a key fob or other similar identification and be in the field of view of the rearview camera of the MACS module 100. The passive entry system 195 detects and/or authenticates the key fob, and sends an enable signal 160 to the PLGM 190, IPE 200 and the MACS module 100. Upon detection and verification of the transponder/keyfob, when a user approaches the vehicle, the CLE will project a pattern of light on the ground. If the person steps on the image projected on the ground, the camera will detect the change on the projected pattern (structured light detection) and communicate to the power entry system controller to activate the system.

[0027] The power lift gate is used as an exemplification of the power entry system. For example, other power entry systems may include power door locks and a powered sliding back door. The camera with the image processing capability or the camera + image processing ECU assembly will be called a "smart" camera. The "smart" camera is connected to the power lift gate control module, either through the vehicle communication bus, or through a 2 way hardwire connection.

[0028] The CLE may be controlled/connected either by the Power Lift Gate

Module (PLGM) or by the "smart" camera.

[0029] The method described here can work in conjunction with an image/object detection algorithm residing in the "smart" camera that can detect if a person moves behind the vehicle within a predefined zone of interest within the field of view of the camera. The "smart" camera/PLGM will turn on the CLE only if the person is in the zone of interest.

[0030] Figure 3 is an example block diagram of the vehicle entry power systems activation. The vehicle power entry system 300 may include a vehicle communication bus 301, a PLGM 302, a camera with image processing capability or "smart" camera 303, and a laser diode 304. The PLGM 302 is connected to the "smart" camera 303 via a communications bus (COMM) 305. The COMM 305 may be a LIN bus, a CAN bus, a hardwired input, or any other automobile communication system or network. The laser diode 304 may be controlled by either the PLGM 302 or the "smart" camera 303.

[0031] The PLGM 302 will be awaken by the keyfob detection, and in turn will wake up the "smart" camera 303. The "smart" camera 303 (or the PLGM) will turn on the laser diode 304 which will project on the ground a light pattern for a predetermined period of time. The light pattern can be a line, for example, 2 parallel lines or a more complicated image. To keep the coherent light emitter simple but to ensure a robust detection, the pattern should be at least 2 parallel lines.

[0032] When the person steps on the 2 projected lines it will change the 3D shape of the line. The "smart" camera 303 detects the shape change (structured light detection) and signals to the power lift gate controller to activate the lift gate opening. The feedback/acknowledgment of valid detection can be signaled back to the person by various means: turn off laser, blink laser a couple of times, blink vehicle back-up lights, so on. If during the predetermined time the light pattern is not changed the laser is turned off and the system goes back to sleep.

[0033] Figure 4 is an example flow diagram without a zone of interest. In Figure 4, at first the vehicle is asleep (401). The vehicle receives a wake up trigger and user authentication (402). If the user is not authenticated, the vehicle goes back to sleep (401). If the user is authenticated the "smart" camera is woken up (403). A laser timer is then turned on (404). A laser diode is then turned on (405). If the laser timer has not expired (406), a pattern change detection algorithm is run (407).

[0034] If there is a valid structured light detection (408), an acknowledgement/feedback is communicated to the user (409), for example, turning off the laser, blinking the laser a couple of times, blinking the vehicle back-up lights, and the like. Once the acknowledgement/feedback is communicated to the user (409), the lift gate is activated (410). After the lift gate is activated (410), the laser diode is turned off (411), the "smart" camera goes to sleep (412), and the vehicle goes back to sleep (401).

[0035] If there is no valid structured light detection (408), the vehicle checks to see if the laser timer has not expired (406).

[0036] If the laser timer has expired, the laser diode is turned off (411), the

"smart" camera goes to sleep (412), and the vehicle goes back to sleep (401).

[0037] If the zone of interest detection is available, after the light pattern is turned off, the "smart" camera checks again the zone of interest. If the person is still there then the method repeats, starting with the "smart" camera (or the PLGM) turning on the laser diode which will project on the ground a light pattern for a predetermined period of time. If no person is there then the entire system goes back to sleep.

[0038] Figure 5 is an example flow diagram with a zone of interest.

[0039] In Figure 5, at first the vehicle is asleep (501). The vehicle receives a wake up trigger and user authentication (502). If the user is not authenticated, the vehicle goes back to sleep (501). If the user is authenticated the "smart" camera is woken up (503). A zone of interest detection timer is then started (504). If the zone of interest timer has not expired (505), the vehicle checks to see if the person is in the zone of interest (506). If a person is detected in the zone of interest (507), a laser timer is then turned on (508). A laser diode is then turned on (509). If the laser timer has not expired (510), a pattern change detection algorithm is run (5100).

[0040] If there is a valid structured light detection (512), an acknowledgement/feedback is communicated to the user (513), for example, turning off the laser, blinking the laser a couple of times, blinking the vehicle back-up lights, and the like. Once the acknowledgement/feedback is communicated to the user (513), the lift gate is activated (514). After the lift gate is activated (514), the laser diode is turned off (515), the "smart" camera goes to sleep (516), and the vehicle goes back to sleep (501).

[0041] If the zone of interest timer has expired (505), the laser diode is turned off (515), the "smart" camera goes to sleep (516), and the vehicle goes back to sleep (501).

[0042] If there is no person detected in the zone of interest (507), the vehicle checks to see if the zone of interest timer has not expired (505).

[0043] If the laser timer has expired, the laser diode is turned off (515), the

"smart" camera goes to sleep (516), and the vehicle goes back to sleep (501).

[0044] If there is no valid structured light detection (512), the vehicle checks to see if the zone of interest timer has not expired (505).

[0045] Figure 6 is an embodiment of a vehicle 600 that enables a person to activate or trigger predetermined functions associated with electromechanical systems using gestures. The vehicle 600 includes any structure or component including roof, sidewalls, doors, windows, bumpers, seats, mirrors, and the like. The vehicle 600 includes a radio frequency (RF) system 605, and one or more camera modules 610 as described herein above, all of which are connected to one of more ECUs 615. The camera modules 610 are located throughout the vehicle 600 including at the rear, front, side mirrors, rear view mirror and other like locations. The CLE may be integrated or co-located with the camera module 610.

[0046] The RF system 605 may include an RF controller 620 that is configured to transmit and receive RF frequency signals. The RF controller 620 may be configured as a single unit or multiple units. The RF controller 620 may be in communication with one or more RF antennas 625. In an embodiment, communication paths 630 between the RF antennas 625 and RF controller 620 may be hardwired or wireless. The RF antennas 625 may be coupled to the vehicle 600 in any manner. The number of antennas 625 may vary depending on the size, model, type, or any other difference between vehicles. The vehicle 600 may be any motor, rail, aircraft, watercraft or the like vehicle that is consumer, commercial, military or the like. A transponder/keyfob 635 may be used to communicate with the RF controller 620 via the RF antennas 625. The transponder or keyfob 635 may be a passive transponder or keyfob, (e.g., radio frequency identification (RFID) tag), an active transponder or keyfob or a semi- passive transponder or keyfob.

[0047] Figure 7 is an example user detected and the laser diode turned on.

The user 701, either in the zone of detection or by waking up the vehicle with a key fob, activates the laser diode. The laser diode projects an image 702 onto the ground.

[0048] Figure 8 is an example pattern change recognized as a trigger for further actions. When the user 801 interacts with the projected image 802, the pattern changes and activates the power entry system, for example, the lift gate.

[0049] As described herein, the methods described herein are not limited to any particular element(s) that perform(s) any particular function(s) and some steps of the methods presented need not necessarily occur in the order shown. For example, in some cases two or more method steps may occur in a different order or simultaneously. In addition, some steps of the described methods maybe optional (even if not explicitly stated to be optional) and, therefore, may be omitted. These and other variations of the methods disclosed herein will be readily apparent, especially in view of the description of the systems described herein, and are considered to be within the full scope of the invention.

[0050] Although features and elements are described above in particular combinations, each feature or element can be used alone without the other features and elements or in various combinations with or without other features and elements.