Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
METHODS AND SYSTEMS FOR PERFORMING REMOTE PRE-FLIGHT INSPECTIONS OF DRONE AIRCRAFT
Document Type and Number:
WIPO Patent Application WO/2022/164885
Kind Code:
A1
Abstract:
Methods and systems are disclosed for completing a remote preflight inspection of an autonomous drone system utilizing a combination of specialized hardware and software. A combination of systems is used together, including cameras, temperature sensors, inertial measurement units (IMUs), computer vision, and automatic anomaly detection.

Inventors:
SOUSA, Andrew (US)
BABCOCK, Eitan (US)
HARVEY, Zach (US)
KARANTZA, Alexander (US)
KONDAPALLI, Charvak (US)
SOMANDEPALLI, Vijay (US)
Application Number:
PCT/US2022/013874
Publication Date:
August 04, 2022
Filing Date:
January 26, 2022
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
AMERICAN ROBOTICS, INC. (US)
International Classes:
B64F5/60; B64C39/02; G07C5/08
Attorney, Agent or Firm:
VALLABH, Rajesh et al. (US)
Download PDF:
Claims:
CLAIMS

1. A computer-implemented method for performing a remote pre-flight inspection of an autonomous drone aircraft, comprising the steps, performed by a computer system, of: receiving a set of detected values of parameters relating to the drone from a plurality of sensors associated with the drone located remotely from the computer system; determining whether the detected values of parameters are within desired predetermined ranges; and notifying an operator of the drone aircraft or preventing further operation of the drone aircraft when any detected values of parameters are not within desired predetermined ranges.

2. The method of claim 1, wherein the plurality of sensors comprise a temperature sensor, a limit switch, a humidity sensor, a voltage sensor, a current sensor, or a positional sensor.

3. The method of claim 1, wherein the step of determining whether the detected values of parameters are within desired predetermined ranges is performed when the drone aircraft is in a pre-takeoff sequence.

4. The method of claim 1, wherein the step of determining whether the detected values of parameters are within desired predetermined ranges is performed prior to powering up vehicle electronics in the drone aircraft.

5. The method of claim 1, further comprising: receiving information from a remote information source relating to weather conditions around a planned flight path of the drone aircraft; determining if the weather conditions would be hazardous for operation of the drone aircraft; and

7 notifying the operator of the drone aircraft or preventing further operation of the drone aircraft when the weather conditions are determined to be hazardous for operation of the drone aircraft.

6. The method of claim 1, further comprising: receiving image data of the drone aircraft or launch area from a camera system; processing the image data using an anomaly detection algorithm to identify anomalies in the drone aircraft or launch area; and notifying the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the drone aircraft or launch area have been identified.

7. The method of claim 1, further comprising: receiving image data of an area surrounding the drone aircraft from a camera system; processing the image data using an anomaly detection algorithm to identify anomalies in the surrounding area; and notifying the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the surrounding area have been identified.

8. The method of claim 7, periodically repeating the steps of receiving image data of an area surrounding; processing the image data using an anomaly detection algorithm; and notifying the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the surrounding area have been identified.

9. A computer system, comprising: at least one processor; memory associated with the at least one processor; and

8 a program stored in the memory for performing a remote pre-flight inspection of an autonomous drone aircraft, the program containing a plurality of instructions which, when executed by the at least one processor, cause the at least one processor to: receive a set of detected values of parameters relating to the drone from a plurality of sensors associated with the drone located remotely from the computer system; determine whether the detected values of parameters are within desired predetermined ranges; and notify an operator of the drone aircraft or preventing further operation of the drone aircraft when any detected values of parameters are not within desired predetermined ranges.

10. The computer system of claim 9, wherein the plurality of sensors comprise a temperature sensor, a limit switch, a humidity sensor, a voltage sensor, a current sensor, or a positional sensor.

11. The computer system of claim 9, wherein determining whether the detected values of parameters are within desired predetermined ranges is performed when the drone aircraft is in a pre-takeoff sequence.

12. The computer system of claim 9, wherein determining whether the detected values of parameters are within desired predetermined ranges is performed prior to powering up vehicle electronics in the drone aircraft.

13. The computer system of claim 9, wherein the program includes further instructions causing the processor to: receive information from a remote information source relating to weather conditions around a planned flight path of the drone aircraft; determine if the weather conditions would be hazardous for operation of the drone aircraft; and

9 notify the operator of the drone aircraft or preventing further operation of the drone aircraft when the weather conditions are determined to be hazardous for operation of the drone aircraft.

14. The computer system of claim 9, wherein the program includes further instructions causing the processor to: receive image data of the drone aircraft or launch area from a camera system; process the image data using an anomaly detection algorithm to identify anomalies in the drone aircraft or launch area; and notify the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the drone aircraft or launch area have been identified.

15. The computer system of claim 9, wherein the program includes further instructions causing the processor to: receiving image data of an area surrounding the drone aircraft from a camera system; processing the image data using an anomaly detection algorithm to identify anomalies in the surrounding area; and notify the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the surrounding area have been identified.

16. The computer system of claim 15, wherein the program includes further instructions causing the processor to: periodically repeat the steps of receiving image data of an area surrounding; processing the image data using an anomaly detection algorithm; and notifying the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the surrounding area have been identified.

17. A system for performing a remote pre-flight inspection of an autonomous drone aircraft, comprising: a plurality of sensors associated with the drone aircraft;

10 a plurality of cameras; and a computer system configured to: receive a set of detected values of parameters relating to the drone from the plurality of sensors; determine whether the detected values of parameters are within desired predetermined ranges; and notify an operator of the drone aircraft or preventing further operation of the drone aircraft when any detected values of parameters are not within desired predetermined ranges; receive information from a remote information source relating to weather conditions around a planned flight path of the drone aircraft; determine if the weather conditions would be hazardous for operation of the drone aircraft; and notify the operator of the drone aircraft or preventing further operation of the drone aircraft when the weather conditions are determined to be hazardous for operation of the drone aircraft; and receive image data of the drone aircraft or launch area from the plurality of cameras; process the image data using an anomaly detection algorithm to identify anomalies in the drone aircraft or launch area; and notify the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the drone aircraft or launch area have been identified.

18. The system of claim 17, wherein the plurality of sensors comprise a temperature sensor, a limit switch, a humidity sensor, a voltage sensor, a current sensor, or a positional sensor.

19. The system of claim 17, wherein determining whether the detected values of parameters are within desired predetermined ranges is performed when the drone aircraft is in a pre-takeoff sequence.

20. The system of claim 17, wherein determining whether the detected values of parameters are within desired predetermined ranges is performed prior to powering up vehicle electronics in the drone aircraft.

11

21. The system of claim 17, wherein the computer system is further configured to: receive image data of an area surrounding the drone aircraft from a camera system; process the image data using an anomaly detection algorithm to identify anomalies in the surrounding area; and notify the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the surrounding area have been identified.

12

Description:
METHODS AND SYSTEMS FOR PERFORMING REMOTE PRE-FLIGHT INSPECTIONS OF DRONE AIRCRAFT

CROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims priority from U.S. Provisional Patent Application No. 63/141,713 filed on 26 January 2021 entitled METHODS AND SYSTEMS FOR COMPLETING A REMOTE PREFLIGHT INSPECTION OF A DRONE, which is hereby incorporated by reference.

BACKGROUND

[0002] The present application relates to methods and systems for performing a remote pre-flight inspection of an autonomous drone aircraft.

[0003] In order to safely operate an autonomous vehicle like a drone aircraft, a diagnostic check should be run prior to takeoff or launch. If any aspect of the drone aircraft or of the surrounding environment is determined to be unsafe, the flight should be postponed until the issue can be resolved. Currently, pre-flight diagnostic checks are performed manually with a human co-located with the vehicle. The human is able to inspect the aircraft and observe the surrounding environment first hand to ensure safe operation.

[0004] A need exists for a method and system for performing pre-flight checks automatically and remotely with a more reliable and consistent set of checks to ensure efficient and safe drone aircraft operation.

BRIEF SUMMARY OF THE DISCLOSURE

[0005] Methods and systems are disclosed for completing a remote preflight inspection of an autonomous drone system utilizing a combination of specialized hardware and software. A combination of systems is used together, including cameras, temperature sensors, inertial measurement units (IMUs), computer vision, and automatic anomaly detection.

[0006] A computer-implemented method in accordance with one or more embodiments is disclosed for performing a remote pre-flight inspection of an autonomous drone aircraft. The method, performed by a computer system, includes the steps of: receiving a set of detected values of parameters relating to the drone from a plurality of sensors associated with the drone located remotely from the computer system; determining whether the detected values of parameters are within desired predetermined ranges; and notifying an operator of the drone aircraft or preventing further operation of the drone aircraft when any detected values of parameters are not within desired predetermined ranges.

[0007] In one or more embodiments, the method further includes the steps of: receiving information from a remote information source relating to weather conditions around a planned flight path of the drone aircraft; determining if the weather conditions would be hazardous for operation of the drone aircraft; and notifying the operator of the drone aircraft or preventing further operation of the drone aircraft when the weather conditions are determined to be hazardous for operation of the drone aircraft.

[0008] In one or more embodiments, the method further includes the steps of: receiving image data of the drone aircraft or launch area from a camera system; processing the image data using an anomaly detection algorithm to identify anomalies in the drone aircraft or launch area; and notifying the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the drone aircraft or launch area have been identified.

[0009] In one or more embodiments, the method further includes the steps of: receiving image data of an area surrounding the drone aircraft from a camera system; processing the image data using an anomaly detection algorithm to identify anomalies in the surrounding area; and notifying the operator of the drone aircraft or preventing further operation of the drone aircraft when any anomalies in the surrounding area have been identified.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG. 1 is a simplified block diagram illustrating an exemplary system for performing remote pre-flight inspections of drone aircraft in accordance with one or more embodiments. [0011] FIG. 2 is a flowchart illustrating an exemplary method for performing remote pre-flight inspections of drone aircraft in accordance with one or more embodiments.

[0012] FIG. 3 is a simplified block diagram illustrating an exemplary computer system used in performing remote pre-flight inspections of drone aircraft in accordance with one or more embodiments.

DETAILED DESCRIPTION

[0013] Various embodiments disclosed herein relate to methods and systems for performing remote pre-flight inspections of drone aircraft.

[0014] In accordance with one or more embodiments, multiple types of pre-flight checks are executed in multiple layers. The checks include fully automated sensor-based checks, advanced information-based checks, and remote visual checks, which can be performed in the various stages or layers of a launch sequence.

[0015] FIG. 1 illustrates an exemplary system 10 for performing remote pre-flight inspections of a drone aircraft 12 in accordance with one or more embodiments. The system 10 includes a set of sensors 14 for performing automated sensor-based checks. The system 10 also includes a set of cameras 16 for use in performing remote visual checks. The system 10 includes a central processor 18 such as a computer system, which receives and processes data from the sensors 14 and cameras 16. The processor 18 also can access external information sources 20, e.g., for weather information.

Automated Sensor Checks

[0016] Automated sensor checks in accordance with one or more embodiments are fully automated and utilize various types of sensors 14 including, but not limited to, temperature sensors, limit switches, humidity sensors, voltage sensors, current sensors, and positional sensors. The sensors 14 are used to ensure that particular drone components are functioning within normal parameters. If any of the sensors 14 detects a value that is out of acceptable limits, the mission is postponed or cancelled and the remote operator is informed. Examples of conditions detected by the sensors include power related anomalies such as low voltage or high current, humidity issues that could indicate unacceptable amounts of precipitation, or an invalid starting position for a piece of equipment detected by a positional sensor.

[0017] In accordance with one or more embodiments, the sensor checks can take place over the course of the pre-takeoff sequence. For instance, some checks are performed prior to powering up vehicle electronics, such as checking the battery status, while other checks are performed after the vehicle is powered up and has had a chance to initialize, such as verifying that the GPS is receiving an accurate position.

Information Checks

[0018] In accordance with one or more embodiments, the processor 18 receives information from the information source 20 that can be used to perform a more advanced check. For example, if the processor 18 is connected to the internet, it can check the local weather conditions and forecasts to ensure the flight will remain safe for the entire expected duration.

[0019] In addition, the processor 18 can use information gained from various sensors to automatically perform a more detailed check. In one example, the camera system 16 is used to capture images of the drone 12. An anomaly detection algorithm is run on the images to automatically find physical issues with the vehicle 12 such as a broken component or presence of a foreign object.

Remote Visual Checks

[0020] In one or more embodiments, remote visual checks are performed on the drone 12. These checks comprise a visual inspection of the drone 12 and the surrounding launch area. A suite of cameras 16 is placed around the system housing the vehicle 12 (e.g., a hangar) such that the camera system 16 has a view of the aircraft 12 from substantially all angles as well as a view of the surrounding area. Software can analyze the images received by the processor 18 from the camera system 16 and detect anomalies on the aircraft 12 as well as the hangar in which the aircraft 12 resides. Images can then be presented to the remote pilot with any identified anomalies overlaid onto the image. In addition, the previous steps could feed into this, highlighting ancillary anomalies to the operator overlaid onto the image. The operator can then ensure that everything is operating within acceptable limits to allow the vehicle to take off.

Launch Monitoring Checks

[0021] In accordance with one or more embodiments, during periods when the vehicle 12 is launching from the hangar, software is run in the background periodically monitoring the surrounding area for unsafe conditions such as, e.g., approaching objects or animals. The operator can be alerted to any detection of an unsafe condition.

[0022] FIG. 2 is a simplified flowchart illustrating an exemplary method 30 for performing remote pre-flight inspections of drone aircraft in accordance with one or more embodiments. The process includes the steps of performing automated sensor checks 32, performing information checks 34, performing remote visual checks 36, and performing launch monitoring checks 38.

[0023] The processes of the central processor 18 described above for completing a remote preflight inspection of an autonomous drone system may be implemented in one or more computer programs executing on a programmable computer system. FIG. 3 is a simplified block diagram illustrating an exemplary computer system 100, on which the computer programs may operate as a set of computer instructions. The computer system 100 includes, among other things, at least one computer processor 102, system memory 104 (including a random access memory and a read-only memory) readable by the processor 102. The computer system 100 also includes a mass storage device 106 (e.g., a hard disk drive, a solid-state storage device, an optical disk device, etc.). The computer processor 102 is capable of processing instructions stored in the system memory or mass storage device. The computer system 100 additionally includes input/output devices 108, 110 (e.g., a display, keyboard, pointer device, etc.), a graphics module 112 for generating graphical objects, and a communication module or network interface 114, which manages communication with sensors 14, cameras 16, information sources 20 and other devices via telecommunications and other networks.

[0024] Each computer program can be a set of instructions or program code in a code module resident in the random access memory of the computer system. Until required by the computer system, the set of instructions may be stored in the mass storage device or on another computer system and downloaded via the Internet or other network.

[0025] Having thus described several illustrative embodiments, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to form a part of this disclosure, and are intended to be within the spirit and scope of this disclosure. While some examples presented herein involve specific combinations of functions or structural elements, it should be understood that those functions and elements may be combined in other ways according to the present disclosure to accomplish the same or different objectives. In particular, acts, elements, and features discussed in connection with one embodiment are not intended to be excluded from similar or other roles in other embodiments.

[0026] Additionally, elements and components described herein may be further divided into additional components or joined together to form fewer components for performing the same functions. For example, the computer system may comprise one or more physical machines, or virtual machines running on one or more physical machines. In addition, the computer system may comprise a cluster of computers or numerous distributed computers that are connected by the Internet or another network.

[0027] Accordingly, the foregoing description and attached drawings are by way of example only, and are not intended to be limiting.