TRACKING FOR UNMANNED AERIAL VEHICLES

TECHNICAL FIELD

The subject matter disclosed herein relates to the technical field of autonomous mobile robotic platforms, particularly, integration of autonomous unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) for the purpose of UAV localization, navigation, power and processing supply.

BACKGROUND

Unmanned Aerial Vehicles (UAVs) are remotely controlled, semi-automated or fully automated aircraft that can carry actuators, sensors, communication equipment, or other payloads. More recently, UAVs have been developed for a wide range of applications including product transportation, material handling, security, search and rescue, and military missions.

UAVs have the ability to navigate and detect surrounding objects autonomously using multiple sensors such as LiDAR, Ultrasonic, GPS and vision cameras. Accurate localization of UAVs is an important requirement for efficient UAV navigation in autonomous tasks. Typically, Global Positioning System (GPS) is used to keep track and localize UAVs. However, GPS localization is only possible in properly exposed outdoor environments. Recently, systems and methods such as image-aided navigation, Ultra-Wide Band (UWB) technology and motion capture tracking systems are introduced for accurate indoor positioning of robots including UAVs.

Tracking systems such as motion capture infrared sensors and UWB sensors are particularly interesting for UAV navigation. Specifically, integrated infrared camera tracking systems have realized sub-millimeter accuracies for autonomous navigation of UAVs. Typically, tracking systems are stationary and fixed to a reference location. This feature limits the working range of the tracking system specifically in scenarios where a UAV needs to navigate through nested environments whereby the UAV hides from all the stationary sensors. Regarding motion capture systems that are used for UAVs, each system includes multiple cameras and accessories. Therefore, expensive settings are required to have a motion tracking system for UAVs in a large setup. Moreover, using fixed motion capture systems in outdoor fields is not always possible due to lack of required infrastructures such as electricity.

The flight time of battery powered UAVs are very limited. Using a mobile power supply with multiple battery banks that is tethered to the UAV can increase the flight time dramatically and decrease the UAV’s weight because there is less need to on-board UAV battery. Also, the weight of a UAV could be further decreased by reducing the computational and processing resources such as microcontrollers, drivers, and computers from the tethered UAV and provide these resources through the tether cable from the resources that are placed on a ground mobile station. Moreover, usually, the charging time of a battery is several times longer than its discharge time which subsequently results in long downtime for a UAV in order for its rechargeable battery to be charged again. These shortcomings, result in inefficiency and pause in a task or process being carried out by the UAV. Recently, several solutions have been introduced to provide battery changing and battery charging specifically for battery-operated UAVs.

In this disclosure, a system is proposed for localization of UAVs using a UGV (or more generally a wheeled chassis) that is equipped with tracking sensors such as infrared tracking sensors. The mobile tracking system can also include a central power supply or a central processing unit or a combination of the two in order to increase the flight time and enhance the computation resource of the UAVs. The system composes of a multi -robot system of a UAV, an Unmanned Ground Vehicle (UGV) and a motion capture system affixed to the UGV. The UGV, which could be controlled remotely or semi-autonomously or autonomously provides a mobile reference for the tracking system that can facilitate positioning and navigation of the UAV even in highly nested terrains. Additionally, the UAV could be tethered to the UGV in which case the UGV could be used to: 1- supply extended power for the UAV, 2- provide extended computational resources for the UAV, 3- increase the UAV’s payload weight and volume because there are no battery and computational resources installed on the UAV, 4- provide extended storage for possible detachable UAV payloads, and 5- increase the safety and security of the people working around the UAV and the safety of the UAV itself by means of physical attachment.

Also a system and method is introduced to reduce or eliminate the downtime of a task or process that is being carried out by a battery-operated UAV. Example application of such invention could be in agricultural picking (harvesting) where the UAVs of the current invention can pick particular vegetables and fruits and deliver them to UGVs for further storage and handling. Another example could be in warehousing applications where UAVs can pick up boxed and items in a warehouse inventory and deliver to corresponding UGVs for storage and handling.

SUMMARY OF THE INVENTION

According to a first disclosed aspect, a system is provided for accurate localization and tracking of one or more UAVs. The system comprises: a wheeled chassis; at least three pose detecting sensors disposed on the wheeled chassis and configured to operably measure the pose of one or more UAVs; and a processing unit operably in communication with the pose detecting sensors; wherein the processor calculates the pose of the one or more unmanned aerial vehicles with respect to the wheeled chassis using the measurements provided by the at least three position detection sensors. The system may be configured to include the processing unit disposed on the wheeled chassis.

The wheeled chassis may further include the processing unit and a landing structure disposed on the wheeled chassis and configured to allow landing of the one or more UAVs on the wheeled chassis. The wheeled chassis may be configured to include self-propelled wheels and provide selfdriving capabilities. The wheeled chassis may be configured to autonomously follow the one or more UAVs to maintain the one or more UAVs in a working range of the pose detecting sensors. The wheeled chassis may further be configured to autonomously avoid obstacles. In other embodiments the wheeled chassis may be configured to move along a pre-determined paths or raillike passages and aisles while in other embodiments the wheeled chassis may move freely inside a working space by manual commands from a remote operator. In either embodiment, the UGV could be equipped with plurality of sensors such as infrared (IR), ultrasonic, LiDAR, vision cameras and Ultra-Wide Band (UWB) sensors to provide autonomy for the UGV.

The system may further include a power source configured to supply power for the one or more UAVs. The power source may be disposed on the wheeled chassis or may be located otherwise.

The system may further include a tethering mechanism disposed on the wheeled chassis and configured to operably connect the at least one UAV to the wheeled chassis for at least one of the following purposes: providing extended power source for the at least one UAV; and providing extended processing source for the at least one UAV.

The wheeled chassis may further include: a frame disposed on the wheeled chassis; and at least one charging pad disposed on the frame and operably connected to the power source, the charging pad configured to charge a UAV’s battery once the UAV is landed on the charging pad. The charging pad may charge the UAV battery either through conductive contact or through contactless induction. The number of the charging pads in the system may be equal to or greater than one plus the ratio of the UAV’s battery charging time over UAV’s battery discharge time.

The advantages of the disclosed aspect may include providing a mobile pose tracking system which can follow a UAV and always can detect the UAV in the active tracking range of the pose tracking system. The mobile system eliminates the need for installation of fixed tracking systems, particularly, in large spaces such as warehouses. In these cases, a mobile tracking system, such as the system proposed in this disclosure, can reduce the infrastructure costs for UAV tracking. Moreover, the run-time and processing resources of a UAV is considerably improved by tethering the UAV to the wheeled chassis and providing extended power and computational resources to the UAV.

In accordance with another disclosed aspect, a method is presented for UAV pose tracking the method comprising: measuring pose data of one or more UAVs using at least 3 pose detecting sensors disposed on a wheeled chassis; transmitting the measured pose data to a processing unit; and using the processing unit, calculating the pose of the one or more UAVs with respect to the wheeled chassis. The wheeled chassis in the method may be an autonomous mobile robot configured to autonomously move within an environment. The UAV pose tracking method may further comprise: detecting by the processing unit, if one or more UAVs are moving out of a working range of the pose detecting sensors; and in response to one or more UAVs moving out of the working range, generating steering command signals for the wheeled chassis to move such that the one or more UAVs are placed in the working range of the pose detecting sensors.

In accordance with another disclosed aspect, a system is presented to perform a mission with zero or minimum downtime, the system comprising: an electric power source; a wheeled chassis; a frame disposed on the wheeled chassis and configured to house a plurality of UAVs with rechargeable batteries; a plurality of charging pads disposed on the frame and operably connected to the power source, the charging pads configured to recharge the battery of the UAVs; a first UAV performing the mission; and a second UAV standing-by on the charging pads; wherein, in response to a low-battery status of the first UAV, the second UAV takes off to continue performing the mission and the first UAV lands on a charging pad from the charging pads to recharge the battery of the first UAV. The power source of the system may be disposed on the wheeled chassis or may be remote from the wheeled chassis.

In accordance with yet another disclosed aspect, there is provided a method for performing a task with zero or minimum downtime, the method comprising: carrying a plurality of battery-operated UAVs on a wheeled chassis toward a location to perform the task; flying a first UAV and causing the first UAV to perform the task; continue performing the task until the first UAV indicates a low-battery status; swapping the low-battery UAV with a full-battery UAV by navigating the first UAV to the wheeled chassis and landing the first UAV on a charging pad of the wheeled chassis, and flying a second UAV with a fully charged battery; instructing the second UAV to continue performing the task; and continue swapping a low-battery UAV with a full-battery UAV until the task is done.

This summary is not an extensive overview intended to delineate the scope of the subject matter that is described and claimed herein. The summary represents aspects of the subject matter in a simplified form to provide a basic understanding thereof, as a prelude to the detailed description that is provided below. Further features and details of the invention will be explained or will be apparent in the course of the following illustrations and descriptions.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more comprehensive understanding of the nature and advantages of the disclosed subject matter, as well as the preferred modes of use thereof, reference should be made to the following detailed description, read in conjunction with the accompanying drawings.

Figure 1 A and IB is a perspective and block diagram view of a mobile tracking system according to a first disclosed embodiment.

Figure 2 is a side cross sectional view of the mobile tracking system of Figure 1. Figure 3 is a perspective section view of the tethered cable of Figure 1.

Figure 4 is a perspective view of a UAV mobile power supply system according to another disclosed embodiment.

Figure 5 is a side view of the system of Figure 4.

Figure 6 is a side view of an embodiment of a UAV of the system of Figure 4.

Figure 7A to 7C are different embodiments of the system of Figure 4.

Figure 8 is a flowchart representing block of codes regarding operation of the system of Figure 4.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Referring to Figure 1A, a mobile localization system is shown generally at 100, according to the first disclosed embodiment. The mobile localization system 100 comprises: a wheeled chassis 102; multiple pose detection beacons 130 disposed on the wheeled chassis 102 and configured to provide measurements related to the pose of one or more unmanned aerial vehicles (UAVs) 140 and 141. The pose measurements comprise measurements related to the position, orientation, or both position and orientation of the UAVs 140 and 141. The system 100 further comprises a processor 150 (shown in Figure IB) operably connected to the pose detection beacons 130 and configured to process the pose measurements from the beacons 130 to calculate the position and/or orientation of the UAVs 140 and 141 (i.e. localize the UAVs) with respect to the wheeled chassis 102 in real time. At least 3 pose detection beacons 130 are disposed on the wheeled chassis 102 to properly localize the UAVs with sufficient accuracy.

The pose detection beacons 130 may be 3D positioning sensors such as infrared motion capture cameras or vision-based cameras. In other embodiments, the pose detection beacons 130 may be radio frequency (RF) sensors such as ultrawide band or radar sensors. In the preferred embodiments of the system 100, high speed (high frequency) sensors 130 are used to enable very fast response for the UAV controllers and hence enable fast UAV maneuvers. In the embodiment shown in Figure 1A, high speed infrared (IR) motion capture tracking systems are used in the mobile localization system 100. Examples of such motion capture systems include Optitrack sensors by Naturalpoint Ines. IR motion tracking systems may use passive or active markers (not shown in figures) disposed on the UAVs 140 and 141 to measure and track the pose of the UAVs. Typically, active trackers are more accurate but are required to be powered which complicates the needed operational infrastructure. In other embodiments, RF-based sensors may be used due to their long distance transmission capabilities.

The wheeled chassis includes a chassis or body 104 and multiple wheels 106. The wheels 106 may be powered or idler wheels but in the preferred embodiments the wheels 106 are powered wheels and the wheeled chassis 102 uses the powered wheels 106 to navigate autonomously or alternatively using a remote controller. Alternatively, the wheeled chassis 102 may be configured to move along a pre-determined paths or rail-like passages and aisles. In some embodiments the wheeled chassis may be equipped with a plurality of navigating sensors (not shown in Figure 1) such as infrared (IR), ultrasonic, LiDAR, vision cameras and Ultra-Wide Band (UWB) sensors to provide the wheeled chassis 402 with autonomous navigation capability.

The wheeled chassis 402 may further include landing stations 112 to facilitate landing of UAVs

140 and 141 on top of the wheeled chassis. The wheeled chassis may further include a storage box 114 disposed on chassis 104 to provide storage for the possible UAVs’ 140 and 141 detachable payloads. The storage box 114 may be detachable from the wheeled chassis 102 in some embodiments.

Still referring to Figure 1 A, a tether cable 120 may used to physically attach the UAVs 140 and

141 to the wheeled chassis 102 and may serve as the power and signal transmission line that transmits power and signal to and from the UAVs 140 and 141. In some embodiments the wheeled chassis 102 may include a central power bank or power supply 160 and a central processing unit 150 that provide power and processing resources for the UAVs 140 and 141 using the tether cables 120. The wheeled chassis 102 may further include cable reel coils 122 disposed on the chassis 104 to extend and retract the tether cables 120.

The pose detection sensors 130 evaluate the position and orientation of the UAVs 140 and 141 in the 3D space and transmit the measurements to the central processing unit 150. Proper navigation command signals are then generated in the processing unit 150 and then are transmitted to the UAVs 140 and 141 through the tether cable 120. Signals from the UAVs 140 and 141 such as status of the UAV onboard batteries, sensory data from the UAV onboard sensors such as current, torque and vision feedback, and signals from the UAV onboard processor, may be transmitted to the central processing unit 150 for further processing, particularly, computationally expensive processing.

Referring to Figure IB, a block diagram of the mobile localization system 100 of Figure 1A is shown. In the embodiments shown in Figure IB, the processor 150 is disposed on the wheeled chassis 102. However, in other embodiments the processor 150 may be remote from the wheeled chassis 102 but still should be connected to the pose detection sensors 130 using wireless connection for example. The processor 150 may be connected to pose detection sensors 130 wired or wirelessly. The wheeled chassis 102 may further include a power supply 160, operably connected to pose detection sensors 130 and processor 150 wired or wirelessly, to provide them with power.

Referring to Figure 2, a side cross sectional view of the wheeled chassis 102 is shown. The power bank 160 and the central processing unit 150 are disposed on the wheeled chassis 102. The wheeled chassis may further include a local networking hardware 180 such as a WiFi router disposed on the chassis 104 and connected to the processing unit 150 to facilitate data or signal communication between the processing unit and external devices such as a remote controller 190 (shown in Figure IB) or the UAVs 140 and 141.

Referring to figure 3, a cross section of the tether cable 120 is shown which comprises of a power line 122 and at least one data line 124.

In accordance with another disclosed embodiment a method for UAV pose tracking using a mobile unit is described. The method comprises measuring pose data related to one or more UAVs using at least 3 pose detecting sensors which are disposed on a wheeled chassis; transmitting the measured pose data to a processing unit; and then using the processing unit to calculate the UAV pose with respect to the wheeled chassis at each instant. The pose measurements may include data regarding to the position, orientation, or both related the UAVs.

The pose detecting sensors may be 3D positioning sensors such as infrared motion capture cameras or vision-based cameras. In some embodiments, the pose detecting sensors may be radio frequency (RF) sensors such as ultrawide band or radar sensors. The wheeled chassis may be similar to the system shown in Figure 1 and may have self-driving capabilities which may receive steering commands from the processing unit to follow the UAVs and facilitate continuous pose tracking of the UAVs using the pose detecting sensors. Usually, the pose detecting sensors have a limited range of operation and in order to continuously track the pose of the UAVs, the UAVs should stay in the operable range of the pose detecting sensors. Thus, the method may further include detecting, by the processing unit, if one or more UAVs are moving out of the working range of the pose detecting sensors and accordingly generating, by the processing unit, steering commands for the wheeled chassis to direct the wheeled chassis such that the UAVs are in the operable range of the pose detection sensors.

The wheeled chassis may also receive obstacle avoidance commands from the processing unit to facilitate safe and fluid mobility of the wheeled chassis. The processing unit may receive data related to the surroundings and obstacles around the wheeled chassis using sensory systems, such as LiDAR and vision camera, operably disposed on the wheeled chassis, the UAVs, or both.

Referring to Figure 4, in accordance with another disclosed aspect, there is provided an embodiment of a system 400 for battery charging of one or more UAVs 440 to 443. The system 400 comprises a wheeled chassis 402 comprising of chassis 404, a frame 408 disposed on the chassis 404, and a plurality of charging pads 410 operably disposed on the frame 408. Each charging pad 410 is connected to a central battery bank (not shown in the figures) disposed on the wheeled chassis 402 and each charging pad 410 is configured to charge a UAV battery (not shown in figures) once any of the UAVs 410 to 443 is landed on the charging pad 410. Each charging pad 410 may be an inductive charging pad to cause wireless charging, or a conductive charging pad to cause charging through contact.

The wheeled chassis 402 may further include a processing unit (not shown in the figures) such as a mini computer to control all of the processes in the system 400. The wheeled chassis 402 may include idler wheels or powered wheels. In the embodiment shown in Figure 4, the wheeled chassis 402 has powered wheels 406 to direct the chassis 404 through a workspace.

The system 400 may further include a communication module 480 such as a WiFi module, disposed on the wheeled chassis and operably connected to the processing unit, to facilitate data transfer between the processing unit and each UAV 440 to 443. The communication module 480 may also facilitate data transmission between the processing unit a remote device 490 such as WiFi-enabled user handheld controller for a remote operator 492 of the system. The operator 492 may use the remote device 490 to receive information regarding the status of each UAV or the processing unit, for example, or to send information to each UAV or the processing unit regarding the procedures of a certain task.

The frame 408 may be configured to have a single deck or multiple decks, wherein at each deck, a single or multiple UAVs may be landed. In the embodiment shown in Figure 4, the frame 408 comprises two decks 407 and 409 and each deck may house two UAVs.

In the embodiment shown in Figure 4, the UAV 440 is engaged in performing an assigned task while UAVs 441 to 443 are landed on the charging pads 410 and are operably configured to have their onboard battery (not shown in figures) either in the process of charging or fully charged. In system 400, a charge status is associated with the charge level of the battery of a UAV. The charge statuses 446 to 449 are associated with the UAVs 440 to 443, respectively. The system 400 may be configured to have at least one UAV 441 with a fully charged battery status 447 located on the charring pad 410. Once the onboard battery of the operating UAV 440 is about to fully discharge, the operating UAV 440 communicate with the processing unit, using a wireless communication for example, to land on a vacant charging pad 411 to cause its battery to be charged. Furthermore the processing unit communicates with the UAV 441, the UAV with fully charged battery status 447, to take off and continue the task.

The system 400 may be used to facilitate the UAVs 440 to 443 to collaboratively perform a task or mission continuously and potentially with no down time for UAV battery charging. The system 400 may be configured to autonomously perform a task or mission such as system monitoring, inspection, tracking, and transportation of articles. In the embodiment shown in Figure 4, the plurality of UAVs 440 to 443 are configured to perform scanning all of the barcodes 494 on articles 496 from a plurality of shelves 498 of a warehouse.

In order for the system 400 to perform a task continuously, the number of UAVs, which is identified by N, should be configured to be greater or equal to:

For example, according to Figure 4, the time to fully charge each UAV’s battery is at most 3 times longer than the UAV’s battery discharge time. Hence, at least 4 (N > 3+1=4) UAVs are required to continuously perform the barcode scanning task. In case the number of UAVs N is smaller than Nc, the task will not be performed continuously.

In some embodiments, the system 400 may be equipped with pose detecting sensors such as the pose detection beacons 130 of Figure 1, to enable pose tracking of the UAVs 440 to 443. In such embodiments, the system 400 may provide pose tracking, extended power, and extended processing resource for the UAVs.

Referring to Figure 5, a side view of an embodiment of the system 400 is shown. The wheeled chassis 402 may also include navigation sensors 486, such as LiDAR, Radar, UWB, or camera, disposed on the chassis 404 to assist the system 400 in driving the wheeled chassis 402 to navigate through a workspace. The measurements and data from the navigation sensors 486 may be transmitted to the processing unit where steering commands are generated to autonomously navigate the wheeled chassis 402.

Referring to Figure 6, an embodiment of a UAV, such as UAVs 140 and 141 in Figurel and UAVs 440 to 443 in Figure 4, is shown at 600. The UAV 600 comprises a plurality of propellers 610, a control unit (not shown in images) such as a flight control module, Wi-Fi module 620, landing gears 630, a battery 640, a conductive charging end 650, and at least one attachment 660 to assist the UAV with performing its task. In the embodiment shown in Figure 6, the attachment 660 is a vision camera configured to assist the UAV with the barcode scanning task.

In the embodiment shown in Figure 6, the conductive charging end 650 is configured to make contact with a conductive surface 465 of a conductive charging pad 670, such as the charging pad 410 of Figure 4, once the UAV 30 is landed on the surface 665 and subsequently causes the battery 640 to charge.

Referring to Figure 7A to 7C, several embodiments of a two-deck frame for the system 400 is shown. Referring to Figure 7A, the decks 407 and 409 are rigidly affixed to one another using a support structure 415 at one side of the decks 407 and 409. Referring Figure 7B the decks 407 and 409 are affixed to one another using a support 417 at the middle of the decks 407 and 409. In Figure 7C, the deck 407 is coupled to a linearly actuated system and the deck 407 is configured to be driven in direction 421 to facilitate landing and take-off of the UAVs that are housed on deck 407. The deck 409 is affixed to the chassis 404 using a structure 419.

Referring to Figure 8, according to another aspect, a flowchart disclosing an embodiment of a method for operating the system 400 of figure 4 is presented generally at 800. The method 800 starts at block 802 by carrying a plurality of UAVs on a wheeled chassis to a location of a task. At block 804, a first UAV with a fully charged battery flies and starts the task. Then, the operating UAV continues to perform the task at block 806 while the UAV checks if the task is done at block 808. If the task is not done, the operating UAV checks its battery status at block 810 and if the battery level is not critically low the operating UAV continues the task at block 806. At block 810, if the battery level is critically low, the operating UAV communicates its status with processing unit, such as the processing unit of the wheeled chassis in Figure 4, at block 812 and then, block 814 is proceeded where the UAV navigates toward the wheeled chassis, lands on a vacant charging pad, and starts to charge its battery. At block 816, another UAV with a fully charged battery flies and is set to continue the task at block 806. This procedure in the method 800 is repeated until the task is done at block 818. The presented method 800 could be implemented into the computer readable programs using programing languages such as C, C++, or python.

While specific embodiments have been described and illustrated, such embodiments should be considered illustrative of the invention only and not as limiting the invention as construed in accordance with the accompanying claims.