SYSTEM

CROSS REFERENCE TO RELATED APPLICATIONS

[000 !] This present Patent Application claims priority benefit from U.S. Provisional Patent

Application No. 62/689,618 filed on June 25, 2018, the entire content of which is hereby incorporated herein by reference.

BACKGROUND

1. Technical Field

[0002] The present disclosure relates to a task management and communication system, and more specifically to tasking autonomous vehicles to perform tasks identified by other autonomous vehicles when the detecting vehicles are incapable of performing the tasks,

2. Introduction

[0003] Robots, drones, and other autonomous vehicles are becoming more and more prevalent in society. However, because no single robot, drone, or autonomous vehicle can replicate the actions of a sentient human being, the robots, drones, and autonomous vehicles are specialized to perform specific tasks or actions. For example, a drone may be designed and built for the express purpose of delivering groceries to a customer’s home, but may be incapable of retrieving those goods within a warehouse. Similarly, a store robot tasked with verifying current shelf inventories may he capable of detecting a spill within the aisle, but be incapable of cleaning the spil l itself.

SUMMARY

[0004] Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or can be learned by practice of the herein disclosed principles. The features and advantages of the disclosure can be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features of the disclosure will become more fully apparent from die foliowing description and appended claims, or can be learned by the practice of the principles set forth herein.

[0005] An example method configured according to this disclosure can include: receiving, from a first autonomous vehicle a task assignment which the first autonomous vehicle is incapable of performing; identifying, via a processor, a subset of autonomous vehicles within a plurality of autonomous vehicles, the subset of autonomous vehicles being capable of completing the task assignment; transmitting a request to each autonomous vehicle in the subset of autonomous vehicles; receiving at least one response to the request from at least a portion of the subset of autonomous vehicles; selecting, via the processor and based on the at least one response, a selected autonomous vehicle from the subset of autonomous vehicles; and transmitting the task assignment to the selected autonomous vehicle for completion

[0006] An example system configured according to this disclosure can include: a processor; and a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising: receiving, from a first autonomous vehicle a task assignment which the first autonomous vehicle is incapable of performing; identifying a subset of autonomous vehicles within a plurality of autonomous vehicles, the subset of autonomous vehicles being capable of completing the task assignment; transmitting a request to each autonomous vehicle in the subset of autonomous vehicles; receiving at least one response to the request from at least a portion of the subset of autonomous vehicles; selecting, based on the at least one response, a selected autonomous vehicle from the subset of autonomous vehicles; and transmitting the task assignment to the selected autonomous vehicle for completion

[0007] A non~transitory computer-readable storage medium configured according to this disclosure can have instructions store which, when executed by a computing device, cause the computing device to perform operations which include: receiving, from a fust autonomous vehicle a task assignment which the first autonomous vehicle is incapable of performing; identifying a subset of autonomous vehicles within a plurality of autonomous vehicles, the subset of autonomous vehicles being capable of completing the task assignment; transmitting a request to each autonomous vehicle in the subset of autonomous vehicles; receiving at least one response to the request from at least a portion of the subset of autonomous vehicles; selecting, based on the at least one response, a selected autonomous

7 vehicle from the subset of autonomous vehicles; and transmitting the task assignment to die selected autonomous vehicle for completion.

BRIEF DESCRIPTION OF THE DRAWINGS

[00083 FIG. I illustrates an example of a robot in a store identifying a spill;

[0009] FIG. 2 illustrates an example of communications between distinct devices;

[001 Oj FIG. 3 illustrates exemplary iterations of the selection process;

[0011] FIG. 4 illustrates an exemplary _' method embodiment of the present invention; and [0012] FIG. 5 illustrates an exemplary' computer system.

DETAILED DESCRIPTION

[0013] V arious embodiments of the disclosure are described in detail below. While specific implementations are described, it should be understood that this is done for illustration purposes only. Other components and configurations may be used without parting from the spirit and scope of the disclosure.

[0014] The present disclosure addresses a task management system, and particularly a system for managing task distribution between various robots, drones, and autonomous vehicles. While various examples and embodiments may refer specifically to robots, drones, devices, or autonomous vehicles, the invention can be applied to all these categories. As tasks are identified as not capable of completion by the detecting robot, they are transmitted to a central task-management system which identifies a subset of robots which are capable of completing the task, determines the availability of the robots in die subset, and assigns one of those robots in the subset to complete the task.

[0015] Consider the following example, A store robot may be tasked with verifying current shelf inventories, and may also have a camera used for object avoidance when navigating store aisles. During the course of taking inventory, the inventory robot may detect the presence of liquid within the aisle. The inventory robot can classify the liquid as a spill aud determine that the robot does not have the capabilit of cleaning the spill itself. To make this determination, the robot would, upon classifying the detected task into a task type, compare the task type of the detected task to the types of action the robot is capable of performing. With respect to store-specific tasks, other tasks which ma need to be performed can include removing debris (such as glass, cardboard, dirt, or paper), cleaning a dirty floor, or ensuring that a planogram of the store is complied with. However, the types of tasks which can be identified are limitless. In this example, upon determining that the robot cannot clean up the spill itself, the inventory robot then reports the spi ll to a central task-management system and continues to perform its inventory duties (routing around the spill if necessary) in other configurations and examples, the detection robot may be capable of completing the task, but ma be otherwise occupied or unable to complete the task. For example, the detection robot may be assigned another task which has greater urgency, may be low on power level to complete the identified task or to complete the identified task in addition to its assigned task, etc

[00161 While in this example the task is detected using the object avoidance camera of the inventory robot, aspects of this invention can be performed even when the task is detected through other (non-robot based) mechanisms, such as human reporting (e.g., a human reports the spill using a networked device, such as a C-40 or smartphone), a surveillance camera, a security camera, an odor detection device, etc. The information may be text-based (may require an NPL and OCR); audio-based (may require an NPL Image-based required image processing); machine-machine communication (may require an exchangeable codelanguage). In such cases, the information would be reported directly to the central task- management system without a detection robot first determining it could not complete the task. The central task-management system may analyze the information to identify if action is required and what the task is. Otherwise the processes described herein would be similar.

[0017] The central tas -management system is a computer-based system, such as a computer server, which is capable of making determinations regarding the status of available robots and communicating with those robots. First, the central task- management system receives the task information, which may include information such as the location of the task and the type of task which needs to be performed. In some cases, the task information can include image s/pictures, text describing the nature of the task, audio associated with the task, etc. The central task-management system may also analyze the received inputs, for example to identify the task (in some cases the robot may not be able to identify a task, just that an action may be required) or further refine the assignment. The central task-management system then proceeds to determine die best way to fulfill the task by assigning one or more robots to die task.

10018] T o determine the best way to fulfill the task, the central task-management system may identify, from a list of all robots currently within its control, those robots which would be capable of completing the task. For example, there may be three inventory robots, four cleaning robots, and two security robots under the central task-management system’s control. If an inventory robot detects a spi ll, the central task-management system can select a subset (the cleaning robots) of the total robots available based on their capacity to perform the task, [00193 The central task-management system can then determine the availability andor performance capability of the robots within the subset. This can refer to determining if the robot is currently physically capable of performing the task, whereas the prior check can be based on robot configuration. In other words, this check on current availability identifies if the individual robots within the subset are sufficiently charged, functioning, servieedmaintamed, have the necessary cleaning materials required for the task, etc. This check can also determine if the robots are already assigned tasks, or are otherwise engaged and incapable of performing the task at hand.

[0020] T b make these determinations, the central task-management system may communicate with wireless requests to the individual robots. These requests can include a query for the current location of the individual robots, as well as a query' regarding their current status (charge available, other assignments, etc.). The individual robots receive the requests and transmit responses back to the central task -management system.

10021] The central task-management system receives these responses and can use the responses to select which robot will be assigned the task. In one example, the central task- management system has the location of the task to be completed based on the original information from the detecting robot, receives the current locations of the robots capable of completing the task, and assigns the robot which would have the shortest transit time to the task location in other words, the central task-management system receives the responses from the various robots in the subset of capable robots, determines the respective transit times of each robot in the subset to the task location, ranks the robots based on those transit times, and selects the robot with the shortest transit time in that list. |0022j ln another example, the central task-management system may receive information from the subset of robots indicating that one robot is capable of completing the task, but would complete the task at a slower speed than other robots in such an example, the slower robot may be ranked lower than other robots. If combined with transit time, the central task- management system may make the robot selection based on the total time to complete the task, whereas in other cases may make the robot selection based on transit time alone in yet another example, a robot may report that it is closest to the task location and capable of completing the task, but woul not have any charge remaining after the task is accomplished. The central task-management system may determine that it is more operationally efficient for the overall system for that particular robot to not be given the task (perhaps to go charge now) such that it can be available at a later time.

[00233 In some configurations, the central task-management system will“ping” the robots regarding their availability for every task received. The pings may be also sent at regular or irregular intervals. The robots may also automatically report their status to the central task- management system. A database may store the robots associated with the central task- management system, along with the respective capabilities of the robots which are associated with robot in a record.

[0024] ln other configurations the central task-management system can implement efficiencies which result in increased spee in assigning tasks to the robots. Consider when two tasks are received within a time window. For the first task, the central task-management system may not have a record of where the capable robots are currently located and/or their availability, and may transmit requests to the robots regarding their availability. If the second task is recei ved within a threshold amount of time of the first task, the central task- management system can use the first-task reports from the subset of capable robots to make the determination of which robot should be selected for the second task. That is, if robots “A” and“B” are capable and available of completing the first task and the central task- management system assigns robot“A” for the first task, when the second task is received shortly thereafter, the central task -management system can assign robot“B” without needing to ping the remaining robots. This can result in a taster determination of which robot wi ll be assigned the task, and result in reduced power requirements (no longer requiring transmission of requests and responses). 100253 The overall robot selection process used by the central task-management system can be modified overtime based on the responses of the robots, such that the robot se lection process becomes more efficient in receiving tasks and assigning robots for the task completion. Specifically, the central task-management system can receive the responses, store the availability information received from the robots, store information about which robot was selected for the task, and store information about the task completion (such as actual time required to complete the task, did the robot fail in the task, was human intervention needed, etc.). The central task-management system can then identify a pattern in the data to modify future decision making in future iterations. For example, when a particular robot (or type of robot) consistently fails at a specific type of task, the central task- management system can self-modify its code to no longer assign that robot to those tasks In this manner, the central task-management system can use iterative machine learning to become more efficient in the decision making process.

[0026] The disclosure now turns to the specific examples illustrated in the Figures. While specific examples are provided, elements of the examples can be removed, combined, and/or modified as required by a specific implementation.

[0027] FIG. 1 illustrates an example of a robot 104 in a store 100 identifying a spill 106. Ln ibis example, the store 100 has several rows of goods 102 separated by aisles. The detecting robot, robot A 104, has determined that it is incapable of cleaning up the spill 106, However, within the store 100 are two robots, robot B 108 and robot C 1 10, which may be capable of cleaning up the spill 106.

[0028] FIG. 2 illustrates an example of communications between distinct devices, or device categories. In this example, there are three devices: the detection device 202, the device management system 204, and the additional devices 206, The detection device 202 is die device which first detects the task 208. The detection device 202 determines that it is unable to complete the task 210 and transmits the task 212 and/or detected information to the device management system 204.

[0029] The device management system 204 receives the task 214 from the detection device 202 and identifies other devices capable of completing the task 216. This can be done using known informatio about the general capabilitie of the various robots with which the de vice management system 204 communicates, or accessing similar information in a database electrically connected to the device management system 204 Upon identifying additional devices 206 capable of completing the task 216 (which may be a subset of all available devices in communication with the device management system 204) the device management system 204 pings 218 capable devices for (1) their current availability and (2) their current location.

[0030] T he additional devices 206 receive the pings 220, then transmit their availability and location 222 and any other information to the device management system 204. The device management system 204 receives the availability and location information 224, then selects a capable, available device 226 based on the received information. Upon selecting a device, the device management system 204 transmits the task assignment 228 to the selected device. The selected device receives the assignment 230, completes the task 232, and transmits a task completion notice 234 back to the device management system 204 The device management system 204 receives the task completion notice 236, and the task is noted as completed.

10031] FIG. 3 illustrates exemplary' iterations of the selection process performed by the device management system 204. In a first iteration 302, a task is received 306, robots are pinged for availability information 310 (other information, such as location, may be requested as well), and the system makes a selection using an initial algorithm 312. In addition, in this configuration, the information received from the various pinged robots is recorded 314 for use in future iterations. The total time to make the selection is illustrated as it 316. During subsequent iterations 304, the system can receive a task 318, then determine if the new task was received within a time window of the first task 306. If outside the time window, the system will repeat die first iteration process 302. However, if within the time window, the system can make the selection of a robot using the recorded information 424 recorded 314 during the first iteration 302. Additional data may then be recorded 326 and relied upon in future iterations. The total time for the second/subsequent iterations is illustrated as ii 328, which is illustrated as smaller than tj 316 because the subsequent iterations 304 can be processed faster than the first iteration 302.

[0032] In addition, the recorded data about each iteration can be used to modify the selection algorithm. For example, the weights assigned to various factors (availability, location, task proficiency, overall system capacity) may change based on the selections and patterns made by the system. To modif the selection algorithm can require modifications to code and/or processor configurations. That is, if the algorithm is being executed by a processor from memory, modification of the algorithm can require overwriting the memory with the new, more efficient code. If the algorithm is encoded into a processor such as an FPGA (Field- Programmable Gate Array), modification of the algorithm can require a modification to the modifiable processor itself.

[0033] FIG. 4 illustrates an exemplary method embodiment of the present invention. In this example, the system receives, from a first autonomous vehicle a task assignment which the first autonomous vehicle is incapable of performing (402). An example of the task is a cleaning task, such as vacuuming, sweeping, mopping, and disinfecting. Other exemplary tasks may include placing or removing items, taking photographs, detecting specific signals, etc. in one configuration, the task assignment can include: a task location identifying a location of the task assignment, and a task type identifying an action which needs to be performed for completion of the task as ignment.

10034] The system identifies, via a processor, a subset of autonomous vehicle within a plurality of autonomous vehicles, the subset of autonomous vehicles being capable of completing the task assignment (404) and transmits a request to each autonomous vehicle in the subset of autonomous vehicles (406). In one example, the request can include a location quer regarding a current location of each autonomous vehicle in the subset of autonomous vehicles, an a status query regarding a current availability of each autonomous vehicle in the subset of autonomous vehicles. In such configurations, upon receiving the responses of the subset, the selecting can occur by: identifying, based on the current availability of each autonomous vehicle in die subset of autonomous vehicles, available autonomous vehicles capable of completing the task assignment; determining a travel time required for each of the available autonomous vehicles from the current location of each respective autonomous vehicle in the available autonomous vehicles to the location of the task assignment, to yield travel times of the available autonomous vehicles; ranking the available autonomous vehicles based on the travel times of the available autonomous vehicles, to yield ranked available autonomous vehicles; and selecting a fastest time within the ranked available autonomous vehicles as the selected autonomous vehicle.

[0035] The system recei ves at least one response to the request from at least a portion of the subset of autonomous vehicles (408) and selects, via the processor and based on the at least one response, a selected autonomous vehicle from the subset of autonomous vehicles (410) The system then transmits the task assignment to the selected autonomous vehicle for completion (412).

[0036] In one configuration, the transmitting of the request, the receiving of the at least one response, and the transmitting of the task assignment occur using a wireless communication system.

[0037] In one configuration, the method can he further augmented to include: recording the at least one response to the request, to yield recorded data; and modifying a selection process used for the selecting of the selected autonomous vehicle for a subsequent task iteration based on the recorded data.

[0038! With reference to FIG. 5, an exemplary system includes a general-purpose computing device 500, including a processing unit (CPU or processor) 520 and a system bus 5 10 that couples various system components including the system memory 530 such as read-only memory (ROM) 540 and random access memory (RAM ) 550 to the processor 520 The system 500 can include a cache of high-speed memory connected directly with, in close proximity to, or integrated as part of the processor 520. The system 500 copies data from the memory 530 and/or the storage device 560 to the cache for quick access by the processor 520, in this way, the cache provides a performance boost that avoids processor 520 delays while waiting for data. These and other modules can control or be configured to control the processor 520 to perform various actions. Other system memor 530 may' be available for use as well. The memory 530 can include multiple different types of memory with different performance characteristics lt ca be appreciated that the disclosure may operate on a computing device 500 with more than one processor 520 or on a group or cluster of computing devices networke together to provide greater processing capability. The processor 520 can include any general purpose processor and a hardware module or software module, such as module 1 562, module 2 564, and module 3 566 stored in storage device 560, configured to control the processor 520 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. The processor 520 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc A multi-core processor may be symmetric or asymmetric. 100393 The system bus 510 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. A basic input/output (BIOS) stored in ROM 540 or die like, may provide the basic routine that helps to transfer information between dements within the computing device 500, such as during start-up. The computing device 500 further includes storage devices 560 such as a hard disk drive, a magnetic disk drive, an optical disk drive, tape drive or the like. The storage device 560 can include software modules 562, 564, 566 for controlling the processor 520. Other hardware or software modules are contemplated. The storage device 560 is connected to the system bus 510 by a drive interface. The drives and the associated computer-readable storage media provide nonvolatile storage of computer- readable instructions, data structures, progra modules and other data for the computing device 500 In one aspect, a hardware module that performs a particular function includes the software component stored in a tangible computer-readable storage medium in connection with the necessary hardware components, such as the processor 520, bus 510, display 570, and so forth, to cany out the function. In another aspect, the system can use a processor and computer-readable storage medium to store instructions which, when executed by the processor, cause the processor to perform a method or other specific actions. The basic components and appropriate variations are contemplated depending on the type of device, such as whether the device 500 is a small, handheld computing device, a desktop computer, or a computer server.

[0040 j Although the exemplary embodiment described herein employs the hard disk 560, other types of computer-readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, digital versatile disks, cartridges, random access memories (RAMs) 550, and read-only memory (ROM) 540, may also be used in the exemplary operating environment. Tangible computer-readable storage media, computer-readable storage devices, or computer-readable memor devices, expressly exclude media such as transitory waves, energy, carrier signals, electromagnetic waves, and signals per se.

[004Ij To enable user interaction with tire computing device 500, an input device 590 represents any number of input mechanisms, such as a microphone for speech, a touch- sensitive screen lor gesture or graphical input, keyboard, mouse, motion input, speech and so forth. An output device 570 cun also be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems enable a user to provide multipie types of input to communicate with the computing device 500, The communications interlace 580 generall governs and manages the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.

10042] The steps outlined herein are exemplary and can be implemented in any combination thereof, including combinations that exclude, add, or modify certain steps

[0043] Use of language such as“at least one of X, Y, and Z ^M or“at least one or more of X, Y, or Z” are intended to convey a single item (just X, or just Y, or just Z) or multiple item (i.e., {X and Y}, (Y and Z}, or {X, Y, and Z}). “At least one of’ is not intended to convey a requirement that each possible item must be present.

10044] The various embodiments described above are provided by way of illustration only and should not be construed to limit the scope of the disclosure. Various modifications and changes may be made to the principles described herein without following the example embodiments and applications illustrated and described herein, an without departing from the spirit and scope of the disclosure.