CROSS REFERENCE

[0001] This application relates to and claims priority to US Provisional application 62/796,319 filed January 24, 2019 the entirety of which is hereby incorporated by reference.

TECHNICAL FIELD

[0002] This application relates to the field of automated agricultural harvesting equipment using robotic assemblies, mobile harvesting units, remote harvesting, target tracking, and various combinations of related technologies.

BACKGROUND

[0003] The agriculture industry is highly reliant on human pickers to harvest a number of produce, including berries such as strawberries. The reason human pickers are still used today, despite the technological advancements available, is because of the difficulty of identifying a target such as a berry in a field, that is ready to be picked, reaching through the foliage of the plant to grasp that berry, and then carefully removing that berry without damaging it, to package and sell immediately.

[0004] Current automatic harvesting of such delicate and difficult to grasp agricultural targets such as berries, while operating in a harsh outdoor environment did not exist before this application.

SUMMARY

[0005] Systems and methods here may include a vehicle having various subcomponents for harvesting delicate agricultural items such as berries. In some examples, the subcomponents may be automated. In some examples, the vehicle may include a targeting subcomponent and a harvesting subcomponent. In some examples, the targeting subcomponent utilizes multiple cameras to create three dimensional maps of the target and target areas sometimes including the agricultural foliage. In some examples, the targeting subcomponent may include any of various cameras, sensors, or other targeting features to locate and map targets in an automated or semi-automated manner. The system may then determine coordinates of the mapped targets to be passed to the harvesting subcomponent. In some examples, the harvesting subcomponent may include vacuum features which help a nozzle attach to an agriculture target for harvesting. In some examples, the harvesting subcomponent includes padded spoons to aid in removal of the targeted agriculture from the plant, including in some examples, a stem.

[0006] Systems and methods here may include a harvesting vehicle system with a vehicle with a targeting subcomponent and a harvesting subcomponent, the vehicle including at least one motor in communication with wheels mounted to traverse planter bed rows, and alternatively or additionally, the vehicle includes a computing device with a processor and a memory, the computing device in communication with multiple sensors configured to generate and send sensor data regarding agricultural targets to the computing device, wherein the computing device configured to map the agricultural targets using the sensor data, the harvesting subcomponent in communication with the computing device, the harvesting subcomponent including a robotic arm with picker head assembly, the picker head assembly including a vacuum hose in communication with a compressor, the hose terminating in a bellows end and spoons configured to pinch together to remove targets using data received from the computer regarding mapped targets. In some example embodiments, alternatively or additionally, a conveyor belt system is included, mounted to the harvesting subcomponent, the conveyor belt system configured to receive and move targets from the harvesting subcomponent to a packing area of the system.

[0007] Alternatively or additionally, some examples include a turbine and vacuum assembly mounted to a robotic arm of the harvester subcomponent, the turbine and vacuum assembly in communication with the computer, configured to chop and suction material using data from the computer and map. In some examples, alternatively or additionally, the harvesting subcomponent includes a picker head assembly with multiple picker heads mounted to one robotic arm, the picker heads configured to rotate to harvest and deposit agricultural targets. In some examples, alternatively or additionally, a foliage management subcomponent is mounted to the harvester subcomponent, the foliage management subcomponent including at least two belt drives in communication with the computing device, the belt drives including a belt configured to rotate on the belt drives, the belt configured to interact with foliage of the agricultural targets, and bend the foliage to reveal agricultural targets for the sensors. In some examples, alternatively or additionally, the belt drives are synchronized with the at least one motor by the computing device, to move the harvester wheels, such that a relative speed of the foliage and a portion of the belt that is configured to contact the foliage, is close to zero.

[0008] In some examples, alternatively or additionally, the foliage management subcomponent includes a suction blower mounted above the foliage management subcomponent, the suction blower configured to suck ambient air from around the target foliage, up and away from the foliage. In some examples, alternatively or additionally, wherein the foliage management subcomponent includes a second set of at least two belt drives and a belt mounted around the at least two belt drives, configured to rotate on the belt drives, wherein the two combined belts configured to squeeze the foliage to reveal agricultural targets. In some examples, alternatively or additionally, the second set of belt drives are synchronized with the at least one motor by the computing device, to move the harvester wheels, such that a relative speed of the foliage and a portion of the belt that is configured to contact the foliage, is close to zero. In some examples, alternatively or additionally, at least one air blower is mounted on the foliage management subcomponent, the air blower configured to blow air toward the foliage to clear debris. In some examples, alternatively or additionally, the robotic arm includes a second vacuum assembly including a hose, configured to receive the agricultural targets from the picker head assembly and remove the targets from the harvester subcomponent to a packing area of the system. In some examples, alternatively or additionally, a back end computing system is in communication with the harvester computer, the back end computing system configured to allow human users to review sensor data and designate agricultural targets for the harvester subcomponent to harvest. In some examples, alternatively or additionally, the computer uses neuro network logic to make preliminary determinations of targets using the sensor data. In some examples, alternatively or additionally, the computer may allow the human user to determine an angle of attack for the picker head to harvest an agricultural target, using the sensor data.

[0009] Systems and methods here include harvesting agriculture, including a traversing machine with at least two robotic articulating arms attached to it, at least two wheels attached to it, and a computing system with at least a processor and memory, the traversing machine including at least two wheels or tracks configured below an upper frame, wherein the upper frame supports the at least two robotic arms, the robotic arms including at least one seeker subassembly and at least one picker subassembly both in communication with the computer system, the seeker subassembly including at least one sensor, configured to send information regarding potential agricultural targets to the computing system for analysis, the picker subassembly including a vacuum assembly coupled to a nozzle with a terminating end, wherein the terminating nozzle end includes a segmented ported section; the picker subassembly further including two grappling spoons, the grappling spoons configured to pinch together toward the vacuum nozzle. In some examples, alternatively or additionally, the picker subassembly vacuum nozzle is mounted on an extender actuator. In some examples, alternatively or additionally, the baffle section is made of resilient, pliable material. In some examples, alternatively or additionally, the at least one sensor is at least one of a camera, laser, and lidar. In some examples, alternatively or additionally, a foliage management system includes two sets of belt rollers in communication with the computing system, two belts, around each of the respective belt roller sets forming an inside and an outside, the belt roller sets and belts configured on the traversing machine to receive foliage between the two belts, on the respective insides of each, wherein the belt roller sets are synchronized with the at least two wheels or tracks such that a relative motion of an inside of the belts and a surface on which the traversing machine is moving, is approximately zero. In some examples, alternatively or additionally, the foliage management system includes an ambient air vacuum pump in communication with the computing system, the vacuum pump configured above the two sets of belt rollers configured to suck air away from plant beds over which the traversing machine passes.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] For a fuller understanding of the nature and objects of the disclosure, reference should be made to the following detailed description, taken in connection with the accompanying drawings, in which:

[0011] FIG. 1A and IB are diagrams showing example mobile vehicle examples as described in the embodiments disclosed herein.

[0012] FIG. 2 is a diagram showing example picker head example details as described in the embodiments disclosed herein.

[0013] FIG. 3A, 3B and 3C are diagrams showing example extraction hardware example steps as described in the embodiments disclosed herein.

[0014] FIG. 4 is a diagram showing example rotating picker head examples as described in the embodiments disclosed herein.

[0015] FIG. 5 is a diagram showing more example foliage management features as described in the embodiments disclosed herein.

[0016] FIG. 6 is a diagram of an example foliage management system as described in the embodiments disclosed herein.

[0017] FIG. 7 is a diagram showing more example foliage management features as described in the embodiments disclosed herein.

[0018] FIG. 8 is a diagram showing example conveyor belt examples as described in the embodiments disclosed herein. [0019] FIG. 9 is a diagram showing additional example conveyor belt examples as described in the embodiments disclosed herein.

[0020] FIG. 10 is a diagram showing example point-to-point examples as described in the embodiments disclosed herein.

[0021] FIG. 11 is a diagram showing example pneumatic removal examples as described in the embodiments disclosed herein.

[0022] FIG. 12 is a diagram showing example sensor examples as described in the embodiments disclosed herein.

[0023] FIG. 13 is a diagram showing an example networked system which may be used in the embodiments disclosed herein.

[0024] FIG. 14 is a diagram showing an example computing system which may be used in the embodiments disclosed herein.

DETAILED DESCRIPTION

[0025] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a sufficient understanding of the subject matter presented herein. But it will be apparent to one of ordinary skill in the art that the subject matter may be practiced without these specific details. Moreover, the particular embodiments described herein are provided by way of example and should not be used to limit the scope of the invention to these particular embodiments. In other instances, well- known data structures, timing protocols, software operations, procedures, and components have not been described in detail so as not to unnecessarily obscure aspects of the embodiments of the invention.

[0026] Overview

[0027] The systems and methods described here include an automated / semi-automated system with machine(s) that is/are capable of harvesting agricultural targets such as berries with robotic assemblies touching the plants or targets themselves. Example overall systems may include subcomponents such as a seeker or sensor subsystem to find and locate the targets, that works with and informs a picking subsystem to harvest the targets. The overall system(s) may be mounted on wheels or tracks to advance down a row of targets such as agricultural produce so that the seeker subassembly may identify and map the targets while the picker subsystem is used to harvest, gather, and move the targets with robotic pickers.

[0028] In some example embodiments, additionally or alternatively, the seeker subassembly includes a camera or multi-camera system (such as a stereoscopic arrangement) used by a remote operator to locate targets and three dimensionally map them. In such examples, these mapped coordinates may then be queued for harvesting. Additionally or alternatively, in some examples, the harvester subassembly is then able to follow the seeker subassembly and harvest the berries whose mapped locations are queued by the seeker subassembly. In some example embodiments, the harvester subassembly includes at least one robotic arm with multiple degrees of freedom capable of reaching into the foliage of a plant and extracting a target such as a berry. In some examples, the extraction is aided by a vacuum system to hold the targets. In some examples, additionally or alternatively, the extractions is augmented by one or multiple padded spoon graspers, capable of twisting and snapping a berry stem.

[0029] It should be noted that the examples used here describing berry harvesting, or even strawberry harvesting in the written description and/or figures is not intended to be limiting and are merely used as examples. The agricultural targets to which the systems here may identify, map, and ultimately harvest may be any sort including but not limited to berries such as strawberries, blackberries, blueberries, and raspberries, other examples include grapes, figs, kiwi, dragon fruit, or other fruits. Vegetables may be harvested as well, such as Brussel sprouts, tomatoes, peppers, beans, peas, broccoli, cauliflower, or other vegetable. Any type of agricultural target may be harvested using the systems described herein. Additionally or alternatively, the systems and methods here may be used to target and gather non-agri cultural items such as garbage, or be used to take scientific samples such as rocks or minerals in environments or situations where it may be advantageous to avoid human contact or interaction.

[0030] Harvester Subassembly Examples

[0031] In some example embodiments, a harvesting subassembly is included as its own separate vehicle system from the seeker/sensor subassembly. Additionally or alternatively, in some examples, the harvesting subassembly may be in communication with or connected to the seeker subassembly. In some examples, seeker/sensor subcomponents are integrated into the harvesting assembly and one machine incorporates all of the features described herein. The harvesting subassembly may include any number of features that allow for autonomous, semi-autonomous, or human operable harvesting of delicate target agriculture such as berries, as described herein.

[0032] In some examples, either harvesting or seeker/sensor subassembly may be mounted on its own vehicle subassembly with wheels and/or tracks or combination of both, to traverse down a row of agriculture with the seeker subassembly identifying and mapping the target berries and the harvester subassembly gathering targets.

[0033] FIG. 1 A and IB shows figures of example overall traversing harvesting machine to which any of the various subassemblies described herein may be attached, mounted on, and/or coupled. Such a machine may be manned or unmanned, depending on the arrangement and level of automation programmed into it, as described herein. In the example, the main traversing subassembly 152 includes various portions mounted to it including main driving wheels 154 and in some examples, guide wheels 156. In some examples, guide wheels 156 may be canted outward in order to support traversing a raised mound 101 should a mound be configured as shown. In some examples, tank treads or tracks may be used instead of wheelsl54, 156, and/or a combination of wheels and tracks may be used.

[0034] In some examples, sensors may be mounted to the frame 153 and/or chassis 155 of the machine 152 as well as or in alternate of mounting them to a robotic arm. Such sensors may collect sensor data and send it to a computing system on the harvester assembly or offboard to a back end system for processing and target identification. Such target identification may include coordinates of targets to be picked, and those coordinates sent back to the system for robotic picking as described herein.

[0035] In some examples, any number of robotic arms 160 may be mounted to any of various frame portions 153 and/or chassis portions 155 that comprise the overall traversing subassembly 152. Such robotic arms 160 may include picker assemblies, sensors, or combinations of both. It should be noted that many variations of robotic arms 160 may be used in the systems described here, including but not limited to robotic wrists with link and joint combinations with linear and rotational links, gantry robots with linear joints, cylindrical robots connected to rotary base joints, polar robots for twisting, and/or jointed-arm or articulating robots with twisting joints and rotary joints. Any combination of these or other robotic assemblies 160 may be used on the systems described herein to manipulate a picker head and/or sensors for harvesting agricultural targets as described.

[0036] For example purposes, the range 195 of the robotic arm 160 is shown in the FIG. 1A and IB to show that the robotic arm 160 may reach different sides of the row mound 101 where targets may be found and an accumulator for processing targets, such as the example traversing conveyor 178. In example embodiments, the robotic arm 160 may include various numbers of joints thereby allowing for various degrees of freedom to move around and about the plants and rows, taking different angles for the picker assembly on the robotic arm 160 to extract a target. In some examples, the robotic arm 160 may include six degrees of freedom. In some examples, the robotic arm 160 may include seven degrees of freedom, or any other number. In various example embodiments, the robotic arm 160 may be any of various lengths, thereby affecting the range 195 of the arm, which may be tailored to the needs of the particular field or mound or target. In some examples, the robotic arm 160 may include one or more telescoping portions, which may be elongated and/or retracted, thereby affecting the length of that portion and the overall reach 195 of the robotic arm 160.

[0037] In some example embodiments, the robotic arms 160 may be ruggedized in that the tolerances and durability of the arms are developed for outside, dirty employment. In such examples, the robotic arms are not to be operated in pristine factory settings. The systems described here may operate in weather, precipitation, dirt, mud, heat, cold, and in jarring, rough conditions. As such, the bearings, tolerances, and actuators may be made of more durable materials than factory robotic assemblies. In some examples, extra gaskets may be fitted into the various joints to keep dirt out of the more delicate metal couplings and pivoting features of the robotic arms. In such examples, gaskets may be made of rubber, plastic, or ceramic. The robotic arms may be made with fewer joints to minimize the number of potential problems that may occur. The robotic arms may be made of thicker materials, may be heavier, and be rust-proofed, waterproof, weatherized, and/or otherwise reinforced.

[0038] It should be noted that the system in FIG. 1A, IB is shown straddling one row of plants. In some examples, one system may straddle two, three, or wider mounds of plants and the example in FIG. 1A, IB is merely intended to be an example, and not limiting. By making the system wider to straddle a second row, two sets of arms 160 may be used to pick two rows, or three, or four, or whichever number.

[0039] In some examples, multiple robotic arms 160 may fit onto one overall traversing vehicle 152. For example, systems may include a primary picker assembly with a clean-up / redundant picker assembly which operates behind the primary setup. In those examples, up to eight picker arms may be employed, four on the primary and four on the clean-up assembly, with one or two arms operating on each side of two rows. The clean-up system may operate in the same way that the primary system operates, to find targets that the primary system did not harvest, and/or to operate as a redundancy should one or more arms on the primary system malfunction.

[0040] In examples where targets are fruit plants which are harvested many multiple times during a single growing season, often multiple times per week, leaving fruit on a fruit plant may curtail the productivity of the plant. If the plant senses that it still has fruit on it, it may not produce more fruit. This would limit production, so the goal is to remove all of the fruit when ripe. As the bed rows may allow for some fruit to drape over the side of the plastic, and become easily exposed to viewing, other fruit may grow under the foliage, or on top of the bed row tops and be obscured by foliage. Therefore, to find and harvest as much fruit from each plant as possible, it may be necessary to maneuver the foliage to better view and/or harvest fruit targets.

[0041] In some examples, foliage moving arms may be included on the robotic arms 160 to alter, move, displace, and or otherwise gently maneuver the foliage of the plant to better expose the targets such as fruit berries to be picked. In such examples, a bar, or arm, may be pulled across the top of the foliage in order to temporarily move it out of the way for the seeker cameras and/or the harvesting assembly to locate and grapple the target. In some examples, this foliage moving arm on the robotic arm 160 may be maneuvered parallel or substantially parallel to the top of the row bed, and pull across the top of the foliage, bending the plant, but not breaking the plant leaves. This may reveal targets under the foliage, those laying on the top of the row bed, or those caught up in the foliage.

[0042] In some examples, a flexible curtain may be dragged over the foliage, to avoid damage to the foliage, but still pull it out of the way for the seeker and/or harvester to operate. In some examples, this flexible curtain may be a plastic skirt, in some examples, it may be a fringed or sliced skirt. In some examples, it may have fringes that drape over the foliage, and yet flex around the foliage so as not to damage it. As the flexible skirt is pulled over the plants, it helps the seeker subassembly find the targets more easily by limiting the area to be targeted with a clean backdrop. The flexible skirt may be dragged from one side in one direction during a first harvest and the next time the other direction, to avoid biasing or pulling the foliage in the same direction each time.

[0043] In some examples, the overall traversing subassembly 152 may include a transfer conveyor 178. Such a conveyor may include any number of conveyor belts, chains, rope, or other mechanism that can pull materials from one place to another. Such transfer conveyor may be used to collect harvested targets and move them to a packaging subassembly, or storage unit.

[0044] In some examples, either or both the picking and/or harvesting subassemblies or overall harvester 152 may include location sensing and determining devices. In some examples, GPS location sensors may be configured on both or either subassemblies 152 for the computing systems to determine locations and/or steer as discussed in FIG. 13. In some examples, cellular towers and signals may be used for location sensing by the subassemblies. In some examples, inertial navigations systems may be used such as a ring laser gyro, a magnetic gyro and/or any other combination of such with a computing system. In some examples, additionally or alternatively, the cameras in the seeker subassembly and/or other cameras on the harvesting subassembly may be used to identify and track an agricultural row 101 down which the vehicle 152 may be steered. The location sensing and/or steering may be fed into any computing system, either located on the harvesting/seeking systems or remotely, in order to autonomously, semi-autonomously and/or allow for human activated remote steering. Any combination of these or other systems may be used to locate and/or steer the systems here.

[0045] In some examples, the systems and methods may utilize self-steering on row 101 with the intention to utilize a human user droid tender to steer the droid off-row for unloading accumulated berry containers and reloading empty containers, then finally steering the droid back onto a new row to be picked. The droid may have the ability to be converted to full autonomous mode for turnaround at the head lands as well as unloading and loading target containers.

[0046] Picker Head Examples - Vacuum Point of Contact

[0047] In some example embodiments, the harvesting subassembly may include at least one picker head that first interacts with the target in the field to remove or detach the target from the plant it grows on. Such picker heads may be affixed to or be part of, attached to, or otherwise in communication with the robotic arms 160 as discussed in FIG. 1A, IB. In some example embodiments, at least one picker head may be mounted on the end of a robotic harvesting arm, alone or in combination with a sensor and/or lighting system.

[0048] FIG. 2 shows an example picker head assembly, with a front 2A and side 2B view of the same assembly in detail. In the example shown, the main picker head assembly 202 is mounted with two actuators, one actuator for a pincers 204 and one actuator for an extender 206. In some examples, no extender actuator 206 is utilized. In examples where an extender is utilized, the extender actuator 206 moves in and down to move the main vacuum nozzle 203 up and down, relative to the robotic arm (not shown).

[0049] The main nozzle 203 may be a hollow tube which may include a vacuum pump assembly at one end (not shown) and the other end may be used to secure a coupling suction portion 230 to a target 250 such as a berry. In some examples, the coupling portion 230 may include one or more bellows or bellow configurations 232 that allow the coupling portion 230 to stay flexible and malleable to couple with the target 250. The compression nozzle portion 230 may include a malleable hood or coupling section 232 which may include one or more bellow sections, and a rim 234 around an opening 236 to aid in coupling to a target. In some examples, the compression coupling portion 230 is or made up of at least one of, or combination of a neoprene sleeve, a silicone sleeve, a rubber sleeve, or other natural or synthetic material that is soft and flexible. Such a malleable coupling section 232 may be configured to deform or otherwise compress when a target 250 is contacted and may include baffles or other structure that allows for deformation and malleability. Such a deformation or compression may allow for the rim 234 to more easily conform to the target 250 and thereby form a better suction fit for the opening 236.

[0050] In some examples, the compression coupling portion 230 may be 1.250 inches in diameter, in some examples, the nozzle may be between 1.000 and .750 inches in diameter, in some examples, the compression coupling may be 2 inches in diameter. In some examples, the coupling portion 230 may have an effective ported area of 0.44 sq. in. at rest and an expanded area when stretched over the apex of a berry of 1.56 sq. in. But in any case, the nozzle could be customized to any size of intended target.

[0051] In some examples, the compression nozzle portion 230 may include an internal reverse conical mesh to help capture the target 250 yet be as gentle as possible on them. In such examples, the mesh creates an environment where the negative vacuum is acting on a broader surface of the target, thus minimizing the chance of target damage from localized contact to the grappler edges. This mesh may form a cradle for the target to lay in even while being picked, handled, and moved. Such a mesh can be made of silicone materials for durability and flexibility. Alternate materials may be used such as a wire mesh, a plastic mesh, or a combination of wire mesh with plastic coating. Silicon coating may be used on a wire mesh in some example embodiments as well. In some examples, the compression nozzle portion 230 and the opening 234 may be sized for a most average target 250, big enough for the biggest targets and flexible, but able to grasp and vacuum even a smaller target.

[0052] Examples may also include an internal spring system, inside or integrated into the coupling portion 230. Such a spring system may be made of plastic or metal coil(s) that help return the coupling portion 230 back to an extended shape after a target is released by turning off the vacuum and thereby deposited. Additionally or alternately, an iris or camera lens feature may be integrated into the nozzle 230. In such examples, the system may be able to adjust the size of the opening or nozzle end for different sized targets, opening for larger targets, and constricting for smaller targets. In such examples, a coil or spring could be wound tighter for smaller targets and wound looser for larger targets.

[0053] In some examples, the vacuum hose 203 may be connected with the main nozzle 230 to impart a suction or lower than ambient pressure within the nozzle tube 203, and thereby be able to attach to and secure a target 250. In some examples, a vacuum subsystem with a vacuum pump may be mounted on the harvesting subassembly and a vacuum hose may run through or around each harvesting picker robotic arm. In some examples, vacuum subassemblies may be mounted on the robotic arm itself, along with a vacuum hose on the picker head 202.

[0054] In some examples, the amount of suction power that the pneumatic vacuum system / pump may impart through the tube 203, may be 35 inches of negative vacuum. In some examples, 50 inches of negative vacuum may be used. Alternatively or additionally, in some examples, less than 80 inches negative of vacuum may be used so as to avoid damage to the target 250. Alternatively or additionally, in some examples, the amount of suction power may be between 35-50 inches of negative vacuum or between 50-70 inches of negative vacuum may be used. Additionally or alternatively, the vacuum system may be able to reverse from suction to blowing air outward, to clear debris from the bellows, before switching back to a suction mode for harvesting.

[0055] In some examples, a picker head 202 may include two grappler spoons, prongs, extenders, arms, or otherwise structures for grasping a target 212, 214 . In some examples, three spoons may be employed in a similar manner as those examples shown with two as in FIG 2. In some examples, four grappler spoons may be configured in two axes around the picker head 202 assembly. In some examples, alternatively or additionally, the grappler spoons include a hinged and/or spring loaded portion at the end to better cushion the target 250 when pinched. In some examples, the grappler spoons 212, 214 may pivot about the nozzle 203 to impart a twisting motion to snap a berry or other stem as discussed herein.

[0056] Another portion of the example embodiment of FIG. 2 is the grappler spoons 212, 214. The grappler spoons 212, 214 may be configured with the main nozzle 203 between them and be configured to move in a pincer motion toward the nozzle 203 by a robotic actuator 216 and a hinge 218 arrangement as discussed in more detail in FIG. 3A, 3B and 3C. In some example embodiments, the grappler spoons include a cushion 220, 222. In some examples, the cushion 220, 222 may be made of or include closed cell foam, neoprene, gel filled pads, liquid filled pads, open cell foam, layers of foam of different densities, a foam backing with a gel filled pad on top, and/or any combination of the above or other material that may cushion a target 250 when the grappler spoons 212, 214 pinch the target 250. In some examples, the material contacting the target 250 is no more than 20-30 durometer in hardness. [0057] In some examples, a pneumatic trash cleaning air jet 224 may be mounted to the end of the grappler spoon 212, 214 in order to help clear debris. In such examples, air holes may be configured on the end lip of the spoons and face in various directions to direct air toward foliage. In some examples, a line of holes may be configured on the end lip of each grappler spoon 212, 214.

[0058] Example Picker Head and Picking Steps

[0059] FIG. 3A, 3B and 3C show three example snapshots in the multi-step process of target 350 acquisition and grappling using the picker head assembly 302 as described, where each step shows two angles 313, 305 of the same picker head assembly 302, front 313 and side 305. In an example target acquisition, first, 3A, the picker head 302 is directed to a target 350 by a seeker subassembly as discussed herein, using passed coordinates and/or manually steered. Once directed and in place, a robotic arm (860 in FIG. 8) maneuvers the picker head 302 into close proximity of the target 350 where the compression nozzle portion end 330 may attach to the target 350 (as described in FIG. 2) resting on the ground or surface 301 using low pressure imparted by the vacuum nozzle 303. In some examples, this vacuum nozzle 303 may be maneuvered in an extended configuration 344 using the extension actuator 306. In such a configuration, the compression coupling portion 330 may attach, suction, or otherwise temporarily hold the target 350 and thereby secure the target 350 with the vacuum suction through the vacuum tube 303.

[0060] In examples using a pneumatic vacuum, it may be used to first secure the picker head assembly 302 to a target 350 such as a strawberry. This vacuum attachment 303 to a target 350 may allow for the picker assembly 302 to extract a target 350 from foliage, pick it off a stem and/or off a resting surface 301. By securing a vacuum attachment 330 to a target 350 first, further even more grappling, twisting, and/or handling, may be accomplished with spoons 312, 314, as described herein to aid in harvesting and moving targets.

[0061] In such examples, pliable bellows 330 may be positioned on the robotic assembly to within range of a specified target 350 and the bellows section 330 extended 344 to within vacuum range. Through the bellows tube 303, a negative pressure may be generated by a pneumatic vacuum, pump, or other air or pneumatic suction device (not shown) thereby sucking air through the nozzle end 336, through the bellows tube 303 and thereby through the various ports, holes, slots, and/or other shaped voids in the end of the bellows. As such bellows 330 material may be made of pliable material to be able to better conform to the shape of a target more easily to thereby secure a better vacuum hold on the target. [0062] Such a task is made more difficult by the variety of shapes, sizes, and orientation of targets needed to be grappled. Such a task is also made more difficult by the potential of nearby foliage, other targets, stems, dirt, sticks, etc. which could interfere with the vacuum suction, and/or reduce the vacuum pressure that may be applied to a target by the bellows. The purpose of the bellows vacuum system is to create enough of a vacuum attachment to the target such as a strawberry to secure the target from all or as many orientations as possible, including the apex point in examples of a target such as a strawberry.

[0063] In FIG. 3B, the same picker head assembly 302 is shown with a front 313 and side 305, the main nozzle tube 303 may be retracted using the actuator for the extender 306. In some examples, the actuators 306, 304 may be pistons operated by pneumatic and/or hydraulic features in communication with a computing system to operate them. In some examples, the extender actuator 306 may operate in a generally upward/downward motion 340 away from the ground 301 or surface and toward the interior of the picker head assembly 302. Some examples may forego the retraction step and not utilize the extension actuator 306 and/or may not be configured with one. In examples where retraction is utilized, with the target 350 attached to the compression nozzle portion 330 which is now retracted into the picker head assembly 302, the target 350 may be generally aligned with however many grappler spoons 312, 314, for example, are fitted on the picker head assembly 302. In some examples, the compression nozzle portion 330 and thereby the target 350 may be retracted to align with the respective spoon cushion portions 320, 322 of the grappler spoons 312, 314, no matter how many grappler spoons are utilized.

[0064] In some examples, as shown through the FIG. 3 A, 3B and 3C, the spoon actuator 304 may be a piston assembly in communication with a computer to receive instructions and send data and configured to raise and lower a bracket assembly 318 on the picker assembly 302 which may interact with a top end 372, 374 of each spoon arm 312, 314 to pivot each spoon arm 312, 314 about a pivot axis 310, 311, thereby opening and closing, or pinching the two spoons 320, 322 together, and moving them apart. In some examples, the spoon arms 312, 314 may include springs in the pivot axis areas 310, 311, which may bias the spoons in the closed or pinched position, and the bracket 318 may move in relation to the top spoon ends 372, 374 to move down to work against the spring tension to open the spoons, and up to allow the springs to pinch the spoons 312, 314 together. This actuation may take place due to the interaction between ramped or angled portions of the top of the spoon arms 372, 374, and the bracket 318 moving against the spring tension in the pivot axes 310, 311, pivoting each arm 312, 314 about its respective axes 310, 311. [0065] As targets 350 may vary in size and shape, the alignment with the spoons 312, 314, may be obtained by retracting 340 the compression nozzle portion 330 so that the rim 334 of the compression nozzle portion 330 is at a place just above the respective cushion portions 320, 322 of the grappler spoons 312, 314 thereby ensuring that the respective cushion portions 320, 322 of the grappler spoons 312, 314 are able to pinch together to grasp 342 the target 350 without touching the compression nozzle portion 330 when they are in the closed position. Other shapes and sizes of spoon pads 320, 322 may be used to cushion the targets 350 when the pinch er arms 312, 314 are brought together.

[0066] In some examples, sensors may be placed on or near the grappler spoons 312, 314 and/or cushions 320, 322, at the hinges 310, 311, under the pads 320, 322, or other areas, to sense the size and/or shape of the target 350 once gripped. In such examples, a feedback loop may be used from the sensor data to adjust the distance the compression nozzle portion 330 is retracted to align the target 350 with the grappler spoons 312, 314. In some examples, such a sensor may be a light sensor, a laser sensor, a proximity sensor, piezoelectric pressure sensors, and/or a camera to align the target 350 with the grappler spoons 312, 314.

[0067] In some examples, a pneumatic trash cleaning air jet 324 may be mounted to the end of the grappler spoon 312, 314 in order to help clear debris in the field, foliage, and other obstructions in the field to aid in target 350 acquisition. Such an air jet 324 may include one or more nozzles attached to pneumatic pumps and tubes that are able to blast jets of air in various directions, thereby moving, flapping, or otherwise disturbing plants, leaves, dirt, sticks, stems, or other debris that the system is not trying to target, but might be in the way of a target. In some examples, the ends of the grappler spoons 312, 314 themselves may include one or more nozzles, ports, or holes for air jets to blast debris. In some examples, the outsides of the grappler spoons 312, 314, opposite the respective cushion portions 320, 322 may include one or more nozzles or ports, or holes for air jets to blast debris.

[0068] In the example of the third step, FIG. 3C showing the same picker head assembly 302, front 313 and side 305, when the compression nozzle portion 330 and thereby the suctioned target 350 is retracted 340 off the ground or surface 301 and aligned with the grappler spoon cushions 320, 322, the gripper spoons 312, 314 may be actuated by the grappler spoon actuator 304 as described, and squeeze together 342 to grasp the target 350. In some examples, the retractable end of the baffler 332 may retract above or past the point where the pincher spoons 312, 314 may pinch together 342 so as to be able to hold a target 350 with the suction through the pliable baffles section 332 and the spoons 320, 322 at the same time. Only by retracting 340 far enough could a target 350 be secured by both systems at the same time. Such a configuration also allows for a handoff, for example, the suction may be turned off once the retracted portion 332 is secured by the pincher spoons 320, 322.

[0069] In this third configuration 3C, the target 350 may be grasped by the grappler spoons 312, 314, and may still be held by the compression nozzle portion 330 and the vacuum pressure from the main nozzle 303. In such examples, a feedback loop to the onboard or offboard computing systems may be used from the sensor data to adjust the pressure used to squeeze the target 350. In some examples, additionally or alternatively, a single expansion spring (not shown) that may be connected between spoons 312 and 314 may be used. The spring may set the tension that the grappling spoons 312 and 314 may exert onto the target 350. In some examples, additionally or alternatively, the sensors may be piezoelectric pressure sensors on or under the grappler spoon cushions 320, 322. In some examples, additionally or alternatively, the sensors may include cameras to visually detect securing the target 350. In some examples, additionally or alternatively, the sensors may be tension sensors on the grappler spoon actuator 304 to sense the pressure exerted on the closure of the grappler spoons 312, 314. In some examples, additionally or alternatively, the sensors may be tension sensors on the gripper spoons 312, 314 and/or in a hinged portion of the gripper spoons 312, 314 to sense the pressure exerted on the closure of the grappler spoons 312, 314. In some examples, feedback loops may be analyzed by computer systems in communication with the grappler spoon sensors 312, 314, and also in communication with the actuators for the grappler spoons 312, 314 to adjust the pressure on the target 350, which may be used to secure differently sized targets 350 while minimizing damage to larger targets 350 and/or making sure smaller targets 350 are secured.

[0070] In some example embodiments, once the gripper spoons 312, 314 have secured the target 350, the vacuum suction may be turned off by the computer, reduced, or otherwise cut off from the compression nozzle portion 330 which would in turn release the pressure holding the target 350 to the compression nozzle portion 330 but leaving the target 350 in control of the grappler spoons 312, 314. In some examples, the padded grappler spoons 312, 314 may be configured to rotate 333, to thereby flick and turn the target 350 to remove them from their stems and thereby avoid having to cut a stem or plant in any way. This removal process of a target 350 from a stem may be advantageous in the shelf life of the target after harvesting and may be cheaper and easier to accomplish in the field.

[0071] In some examples, this twisting motion 333 may be a 90 degree twist of the grappler picker head assembly 302. In some examples, this may be a 180 degree twist. In some examples, this may be between an 80 degree and 100 degree twist to snap a target 350 stem. In some examples, a snipping element may be used in lieu of or in addition to the snapping, twisting motion of the grappling spoons 312, 314. In such examples, a longer target stem may be desired, and snapping or twisting may remove the stem close to the target 350.

[0072] In some examples, the snapping of the stem by twisting may benefit if the stem of the target 350 plant is pulled out and away from the plant in order to impart a strain on the stem. In such examples, the pulling of the stem first, and then twisting the stem may result in cleaner and/or more accurate stem snaps. In some example embodiments, the exten si on/retr action actuator 306 may include a sensor that may sense resistance as the target 350 is retracted. In some examples, tension sensors may be placed in joints of the robotic arm(s) in communication with the computing systems, to make such a determination.

[0073] In some examples, the twisting motion 333 may be imparted only when the resistance of the retraction of the target 350 meets a particular threshold, as determined by computing systems in communication with such sensors, thereby indicating that the stem of the target 350 is under strain or is otherwise stretched. Such resistance sensors may include a piezoelectric sensor, a strain gage, or other sensor mounted in or on the retraction actuator 306. In some examples, the target may be a berry that includes a calyx portion where a few leaves and the stem attach to the target berry. In some examples, this calyx portion may be identified by the seeker subassembly to help determine which direction to pull the target, normal to the calyx portion. In such examples, the camera and computer system may be able to identify a color variation between the berry itself and the calyx leaves and thereby the stem.

[0074] In snipping examples, a scissors, saw, clipper, or other sharp pincher may be secured to the picker head assembly 3C to cut the stem of the target 350 at a desired length.

[0075] Multi Headed Picker Examples

[0076] In some examples, additionally or alternatively, one robotic arm may include more than one picker head. Such an arrangement may allow for faster picking, as picker heads move between harvesting and dropping targets to be packaged and processed.

[0077] FIG. 4 shows an example of a multi-picker, in this example, a three headed picker rotating assembly mounted on one arm 460which may be configured to harvest and drop targets into a bucket accumulator. In the example of FIG. 4, each robotic arm 460 includes three picker heads 402A, 402B, 402C. The picker heads 402A, 402B , 402C may be configured as described in any of the various example embodiments described herein. In some examples, the picker heads 402 may be variants of one another, for example, different sizes that are configured to harvest a certain sized target. For example, a small, medium, and large picker head 402A, 402B, 402C may be configured on one robotic arm 460 with different diameters on each nozzle or coupling portion.

[0078] As designed, and in use, when the targeting sub-system identifies a target and estimates its size, it may assign the respectively sized one of the three sized picker heads 402A, 402B, 402C to harvest that target. In some examples, the various picker heads 402A, 402B, 402C have different features on them, for example, different air jet blasters for different foliage situations. In some examples, the various picker heads 402A, 402B, 402C may be redundant for maintenance and allow for one or more heads 402A, 402B, 402C to break or malfunction, and then be automatically replaced by a new and functioning picker head assembly 402A, 402B, 402C. In some examples, the functionality of the picker heads 402A, 402B, 402C may be triaged or placed in a hierarchy, so the best, most functional picker head 402A, 402B, 402C is used, and if it breaks further, then the next most capable head 402A, 402B, 402C is used, etc.

[0079] Any of various examples of multiple heads and uses for multiple heads may be employed by the systems described herein. It should be noted that the number of three picker heads is not intended to be limiting and any number of picker heads may be mounted to a single robotic arm. In some examples, each robotic arm 460 could include two picker heads 402 A, 402B. In some examples, each robotic arm 460 could include four or more picker heads 402. In such examples, the picker heads 402A, 402B, 402C are configured onto a rotating portion of the robotic arm 460 to spin and/or rotate 499 to the target area and to a drop off or packing area as described. In some examples the rotation is in a direction perpendicular to the ground, on an axis parallel to the ground as shown in FIG. 4. In some examples, the rotation may be in a carousel fashion, parallel to the ground, with an axis perpendicular to the ground. The example shown is not intended to be limiting.

[0080] In the example, first 4A, one of the multiple picker heads 402A first couples to a target 450. Next, 4B the picker head rotates 499 to bring the first picker head 402A with the coupled target C50 into alignment with whichever accumulator 470 is being used. In some examples, this rotation also brings another picker head assembly 402B into approximate alignment with another target C50. When in position, the first picker head 402A with the target 450 decouples, releases, or otherwise drops off the target 450 while the second picker head 402B couples to another target 450 and the assembly can then spin or rotate again 499.

[0081] Foliage Management Examples

[0082] In some examples, in the field, the targets may grow and rest not only on the sides of the walls of the planter beds, but sometimes on the tops of the planter bed crowns and sometimes within the foliage of the plant itself. In such examples, the goal of finding and harvesting the targets using the systems and methods here may be more difficult than if the targets are more easily presented. In some examples, the foliage may be dense, it may impede the sensors from finding targets. In some examples, alternatively or additionally, the foliage may impede the harvesting pickers from extracting targets, even if they could be found.

[0083] Thus, in such examples, it may be helpful to finding and harvesting more targets, to be able to manipulate the foliage on the plant itself to better reveal any targets that may be growing and/or resting inside or under the foliage.

[0084] In some example embodiments, as described, an additional feature may be a trash grinder. In some examples, impediments such as dead foliage, spoiled targets, or other non targets may get in the way of the seeker subassemblies and/or harvesting subassemblies such as a picker head, from cleanly finding and harvesting targets. In such examples, the system may employ an additional trash disposal system.

[0085] FIG. 5 shows an example robotic assembly with an optional muncher, grinder, or other composting feature head with a target berry 550 and foliage, trash, dead leaves, or other material 552 that is not to be harvested by can get in the way of the targets 550. Such trash 552 may obscure the sensors from the targets to be harvested 550 and may clog the vacuum or picker head assemblies as shown in FIG. 3 A, 3B, etc.. Therefore, a trash muncher features may be used to clear debris 552 that is not desired to be harvested and can impede harvesting.

[0086] In such examples, additionally or alternatively mounted on and or separately from the picker heads and seeker heads, a separate hose 510 or hose used for the trash muncher features with suction capabilities including a separate or shared suction or vacuum pump (not pictured) may include grinding blades 520 that are capable of spinning by way of a motor 560. The vacuum pump and/or motor 560 may be in communication with the computer system as shown in FIG. 14 etc. to be able to turn spinning blades on, turn spinning blades off, speed up rotation, slow down rotation, turn suction on, turn suction off, etc.

[0087] In such a way, the trash material 552 may be suctioned up and be ground up to be collected or ejected 530. In some examples, such grinder blades 520 may be located at the mouth of the vacuum tube 510 as shown. In some examples, additionally or alternatively, blades 521 may be located further up the vacuum tube 510 instead of or in addition to blades 520 at the mouth of the tube. A motor 560 may be used to spin the blades 520, 521 inside the tube 510 and be in communication with he computer to send and receive data including commands to turn on, off, speed up, low down, the spinning blades. [0088] A pump or vacuum system (not shown) may be coupled to the vacuum tube 510 to suck air, trash, and/or impediments up 522 and away from the surface 501. Such air and impediments, after being ground by the spinning blades 520, and/or 521 may be sucked back and out 530 to the ground, away from the plants, or to a collection bin (not shown). Such a vacuum pump (not shown) may be in connection with a computing system onboard to control the vacuum motor to start, stop, speed up, slow down, or any other kind of command. In such a way, a clean-up or preparation step may be added in order to remove impediments to help expose targets before the sensors look for and map targets, and picker heads harvest the targets.

[0089] In some examples, additional trash grinding systems may work in conjunction with the sensor systems to identify impediments for removal. Such systems may utilize the same targeting and coordinate mapping system as the target systems, but instead employ the trash grinding system to remove the impediments instead of harvest the targets. In such a way, an iterative process may be to utilize the sensors to first locate and map impediments for the trash grinding system to remove impediments, then the sensors may be utilized to locate and map targets from the cleaned up plants, to the pass to picker heads to harvest. In such a way, additional targets may be harvested that otherwise might be unseen or too hard to reach.

[0090] Foliage Roller and/or Pneumatic Examples

[0091] Additionally or alternatively, FIG. 6 shows an example system with foliage management features to aid in finding and/or harvesting such targets. As shown in FIG. 6, three views of the same scene are shown, a top down view 601, a side view 603 and an end- on view 605 to the planter bed 660, harvester portions, and plants 620. It should be noted that in FIG. 6, not all of the harvesting assembly is shown, only a portion having to deal with foliage management, and specifically, foliage manipulating rollers 610, 612, pneumatic air jets 630, 632, robotic assembly 662, and pneumatic vacuum 640 and although they appear in the example FIG. 6 as not connected to anything, would be mounted on the frame of the harvester, and are shown in FIG. 6 for illustrative purposes only. It should be noted that all of these features are optional and may be employed on the harvesting system alone or in any combination described here or otherwise. They are not reliant upon one another but may benefit from employment of the other features, meaning any combination of the above may be used, alone or together.

[0092] In some examples, an arrangement of belts and rollers may push or bend the main foliage of a plant 620 to allow the robotic assembly 662 including sensors to more easily observe and/or harvest the targets or berries 602. Such foliage manipulating rollers may be more gentle on a plant foliage 620 than just a bar or pusher, because the speed at which the belts move may be synchronized with the forward movement of the harvesting machine down a row (as shown in FIG. 1A, IB) such that there is imparted no or little shear force on the foliage 620, but instead only the intended pushing or bending force to gently move the foliage 620 to the side.

[0093] In the example views shown in FIG. 6, the harvesting machine may include belts 610, 612, spun by rollers 614, 615, 616, 617, 618, 619 that include pins or cylinders capable of turning mechanically by a chain, gear, or motor, as shown in the top-down view 601 in communication with onboard and/or offboard computer systems. Between and among these rollers may be a belt, sheet, ribbon, or other material 610, 612 kept taught or semi-taught around the rollers 614, 616, 618. The belt 610, 612 may be made of fabric, plastic, woven fibers, rubber, or plastic coated fabric. Such material may be of a thickness that is sturdy for field work, yet gentle on the plant foliage 620.

[0094] In the example of FIG. 6, the rollers 614, 615, 616, 617, 618, 619 are generally arranged perpendicular to the ground 660, pointing upright such that the outside of the belts 610, 612, rotate in the same general direction as the movement of the harvesting machine 650. In this way, the inside of the belts, where the belts touch the plant 620, is moving the opposite way that the machine is moving 650 down a plant bed row. In some examples, this rotation may be synchronized with the forward movement of the machine down a plant bed row, by computers which receive data regarding the speed of the harvester, and thereby the speed of the plants 622 as they are traversed, and the rollers 614, 615, 616, 617, 618, 619 to speed up or slow down at the same time and rate, such that the interior side of the belts 610, 612, are moving at or about the same speed as the plants 620 as the machine traverses the planter rows 660, such that the relative speed between the interior of the belts 610, 612, and the plants 620 is zero, or close to zero. This allows the plants 622 to be touched by the belts 610, 612 with as little disturbance as possible, and without shearing, ripping, or otherwise damaging the foliage 620, and allowing only the bending or pushing of the main foliage 620 to be effected on the plant by the main roller system.

[0095] In some examples, the belts 610, 612, and rollers 614, 615, 616, 617, 618, 619 are configured to interact with the plants 620 by squeezing the main foliage into a narrow opening between the two belts 610, 612. In such a way, the targets or berries 602 may be more easily seen by and/or harvested by the robotic arms 662 as described herein. In some examples, only one belt 610 may be arranged to push aside the foliage on the plant 620 in one direction. In some examples, instead of the two belts 610, 612, being aligned so as to squeeze the plant foliage 620 at the same time, they may be staggered, one after the other such that the foliage of the plant 620 may be pushed to one side by one belt 610, then to the other side by the other belt 612 instead of simultaneously. Any combination of these pushers to one side, and/or squeezing may be utilized.

[0096] It should be noted that the arrangement of rollers 614, 615, 616, 617, 618, 619 may be more or fewer in number than three per belt 610, 612. In such examples, only two rollers may be used to roll a belt, or more such as four rollers may be used to roll a belt. In such examples, and/or with a three roller example, the configuration of roller positions on the machine may be changed such that the shape of the rolling belt may change. In the example of FIG. 6, the three roller belt configuration includes a generally wide triangular shape followed by a generally narrow triangular shape. This allows for the plant 620, when the machine is moving, to be gathered at a wide end by the first, most widely spaced rollers 614, 615, and then pinched into a narrower position between the next rollers 618, 619, and held there through the rest of the traverse, where the robotic arms identify and/or harvest targets 602, before being released at the last rollers 616, 617 when in use.

[0097] It should be noted that side view 603 does not show both belts, but only one belt 612 in either a one belt or staggered belt embodiment, or without the second belt to show the inside of the embodiment for purposes of explanation. As mentioned, any configuration of one, two, staggered, or aligned belts may be utilized as described herein.

[0098] Air Jet Examples

[0099] In some example embodiments, alone or in combination, air jets 630, 632 could be configured to blow ambient air onto the foliage 620 and/or the plant beds 660 to remove trash, clear debris, move foliage 620 or otherwise aid the robotic assembly 662 in finding and harvesting targets 602.

[00100] In such examples, compressors and/or pumps (not shown) may send compressed air down a tube (not shown) and through a nozzle or jet 630, 632 to push or maneuver foliage 620 for sensors and/or harvesting features 662 to better find and harvest targets 602. Such compressors may be positioned in, on, or around the harvesting machine. In some embodiments, alternatively or additionally, such tubes may run down robotic arms 662 and thereby be able to be maneuvered in the same or similar fashion as the seekers and/or harvesting picker heads 662. In some examples, such air jets 630, 632 may be configured next to, adjacent, and/or near the seeker and/or picker head to aid in finding and harvesting targets 602. [00101] It should be noted that the number of air jets 630, 632, does not need to be two as shown, and in fact could be any number of air jets, configured to blow air on the sides of the planter beds 660 and/or targets 602 or plants 620. In some examples, the air jets may utilized the same vacuum system as the vacuum 640 if such a system is used to blow the same air sucked from the top. In some examples, a filter may be used to filter the air sucked by the vacuum 640 to blow by the air jets 630, 632. In some examples, the air jets 630, 632 may include nozzles which concentrate the air flow into a faster speed. In some examples, the air jets 630, 632 may include diffusers, or other features that push air into directions which may be able to be aimed or moved by a motor in communication with a computer.

[00102] More embodiments of Air Jets are described in more detail in FIG. 7. In the example, similar to FIG. 6, three sides of the same plant 720 and plant bed 760 are shown with a top-down 701, end-on 705 and side view 703. In the drawing, the plant 702 is being deflected to the side by a roller setup 710 as described in FIG. 6, or some other kind of arm or wall and harvested by any number of robotic assemblies 762 as described herein. In such examples, various air jets connected to air pumps (not shown) could be arranged on the harvester system to blow ambient air onto the plants 720 and/or plant beds 760 to allow for more easily finding and harvesting the targets 702 by the robotic assembly 762. In the example of FIG. 7, there are air jets on both sides of the plant 720 with one set of air jets 732 arranged to layover the canopy of the plant 720 and comb the foliage. In the example shown, three air jets 732 are so arranged to blow air at the top of the plant canopy 720 and help push the plant to one side. In such examples, any number of air jets with or without nozzles or diffusers may be used, for example, three are shown in FIG. 7 but one, two, three, four, five or more could be used for this purpose. These air jets 732 are generally arranged toward the top of the plant canopy, aimed to push or deflect the plant 720 in a particular direction. Either alone or in combination with a belt as shown in FIG. 6, the purpose of deflecting the plant foliage 720 would be to move debris and material out of the way for the robotic assembly 762 to find and/or harvest targets 702 as described herein.

[00103] In some examples as shown in FIG. 7, another air jet 730 or set of air jets alternatively or additionally, may be configured for another purpose. In such examples, the second grouping of air jets 730 either alone or in combination with the others described herein, may be used as cross-air jets to blow targets off the top of the plant bed 760 and down onto the side to make it easier for the robotic assembly 762 to find and/or harvest targets. In the example of FIG. 7, the cross air jets 730 are arranged lower than the foliage air jets 732 and instead arranged to blow air down and across the top of the plant bed 760. This may induce targets 702 to fall or otherwise move onto the side of the plant bed 760 to make it easier for the robotic assembly 762 to find and/or harvest targets 702. The number of cross air jets could be one as shown, or more, such as two, three, four, five, or more. The arrangement of cross-air jets 730 may be configured onto the harvesting machine in any way, connected to an air pump to blow ambient air onto the plant bed 760 and/or targets 702.

[00104] In some examples, alone or in combination, alternatively or additionally, air jets 730, 732, may be configured as comb-air jets that follow an elliptical path through and around the plant 720. In such examples, the air jets 730, 732, may comb through the leaves and canopy of the plant 720, thus releasing targets. Any arrangement of air jets to manipulate foliage, release targets, or otherwise blow away debris and dead foliage may be used. And any combination of nozzles and/or diffusers may be arranged on the air jets as described herein to either focus the air into a faster moving stream in a nozzle, or widen the stream of air in a diffuser. And number of sub nozzles may be configured on the air jets as well, in order to control the direction of the air jet streams of air, such sub nozzles may include movable parts, controlled by a computer to direct air flow in a desired location. Air pumps pushing air through the nozzles may also be controlled by computers to turn on, turn off, pulse, or otherwise push air toward an intended direction and velocity and/or pressure.

[00105] Vacuum

[00106] Turning back to FIG. 6, in some examples, a high volume suction blower, or vacuum 640 may be attached near the belt system. In such examples, the vacuum system 640 may pull air up and out away from the plant foliage 620. In some examples, the vacuum system 640 may come just before the belts 610, 612 such that the entirety of the foliage 620 may be pulled up or urged upward before being squeezed between belts 610, 612. In such examples, the vacuum 640 may also remove some dirt, debris, dead leaves, bugs, or other unwanted materials. In some examples, the material captured from the vacuum 640 may be blown aside, mulched, or otherwise recycled.

[00107] In some examples, air jets 630, 632 may be used in combination with the belt systems 610. In some examples, the air jets 630, 632 may be staggered to control air positioning at the plants 620, targets 602, and/or plant beds 660.

[00108] Target Accumulator Examples

[00109] In some examples, additionally or alternatively, the harvester subassembly may include any number of target accumulators. In some examples, a conveyor belt system may be used to move accumulated targets from the robotic arm and/or vacuum hose to a storage area and/or a packaging area. In some examples, a bucket system may be used to gather and/or move accumulated targets to a storage area and/or a packaging area.

[00110] FIG. 8 shows an example single headed picker assembly which is configured to drop targets into a multi-conveyor belt accumulator assembly from the top down, and from the side. In FIG. 8, the picker head 802 is shown mounted to a robotic arm 860 which may position the picker head 802 in any of various poses in order to couple to a target 850. In the example, the targets are on a mound 801 which the robot arm 860 traverses from above.

[00111] In use, multi-conveyor system as shown in FIG. 8 is positioned by the robotic arm 860, to extract a target 850 and drop it, and or/ place it onto a first capture conveyor belt 880. In some examples, this capture conveyor 880 is configured to move perpendicular to a mound row 801 and/or otherwise away from the plants and out toward an area which is more open. In some examples, this capture conveyor 880 is a flat conveyor, and wide enough to allow for the movement or rolling of targets, such as four inches wide. In some examples, the speed of this capture conveyor belt 880 may be between 1 in to 5 inches per second. In the example, this capture conveyor 880 may be configured to move targets 850 to a second transfer conveyor belt 878. In some examples, the transfer conveyor 878 is configured to move targets in parallel with a row mound 801, out and away from the ground to be processed.

[00112] In some examples, the transfer conveyor 878 may include fins or other compartmentalized portions to reduce rolling of targets 850 as they move up the transfer conveyor 878 at an angle. In some examples, the transfer conveyor 878 may move targets 850 to a packaging station or other handling area on a main vehicle, or off the main vehicle to an accumulator or other receiver section as described herein. In some examples, the transfer conveyor 878 may be more narrow than the capture conveyor 880, such as between 3 and 5 inches wide. In some examples, the transfer conveyor 878 revolves at speeds of between 5 inches and 10 inches per second.

[00113] In some examples, a brush or other skirt assembly 876 is configured to bridge the capture conveyor 880 and the mound 801.

[00114] FIG. 9 shows an example set of diagrams showing the conveyor belt arrangement from the side, in three steps of an example pick. In such examples, a single picker head 902 is attached in a rotatable manner to the arm 960. In FIG. 9, the first diagram 9A shows the picker head 902 grappling a target 950 as described herein. The capture conveyor belt assembly 980 is shown moving the targets away from the plant 999 and bed and toward another removal arrangement such as a transfer conveyor 978 or other conveyor or bucket system. [00115] 9B shows the picker head assembly 902 retracting the target 950. 9C shows the picker head assembly 902 rotating (arrow) on the arm 960 and dropping or placing the target 950 onto the capture conveyor 980 which moves the target to the transfer conveyor 978. In such examples, the picker head 902 would then rotate back to another target for grappling and extraction.

[00116] Some examples may employ human packers, that is, the harvesting system merely uses conveyor systems as shown in FIG. 1 A, IB, 8, and 10 to transport the targets to a central point on the system for a human to handle and pack into a box, clam shell plastic, or other shipping/ sales container. But some example systems may utilize a more automated system to pack targets. In such examples, similar subsystems to those described for seeking, targeting, and picking, may be employed to deposit targets into a container and arrange the targets in a manner that minimizes empty space and are arranged in a stable arrangement using picker heads. In such examples, a three dimensional mapping of not only the target itself but the other targets which are deposited into a container may be used to maneuver newly placed targets in an arrangement that is stable and uses the space provided. A three dimensional puzzle results in each container, placed by the picker arms or a separate set of picker arms (placer arms) that would operate by picking targets from the conveyor, and depositing them in a container, using the same suction and/or pincher system as described in FIG. 2, 3A, 3B and 3C. In some examples, the grappling spoons may not be necessary to pack targets as only the baffle and nozzle vacuum system may be enough to maneuver them.

[00117] FIG. 10 shows an example which uses either a single transfer conveyor or a bucket instead of a multi-conveyor system. In FIG. 10, a single headed picker assembly which is configured with the picker head 1002 to suction couple to a target 1050 as described herein 10A, then move the target 10B to a bucket, or other accumulator 1070 where the picker head 1002 may then deposit, drop, or otherwise place the target 1050 for further removal or processing. In some examples, the accumulator 1070 is a transfer conveyor. In some examples, the accumulator 1070 includes a package.

[00118] FIG. 11 shows an example which uses a vacuum hose instead of a multi-conveyor system. In FIG. 11, a single headed picker assembly which is configured with the picker head 1102 to suction couple to a target 1150 as described herein 11 A, includes a second vacuum assembly 1190 move the target 1 IB for further removal or processing. In some examples, the pneumatic assembly 1190 includes a vacuum transfer tube 1192 which moves the targets 1150 out for processing. In some examples of the pneumatic transfer system as shown in FIG. 11, no grappling spoons are necessary if the vacuum assembly 1190 is positioned relatively below the picker head 1102. In such examples, the primary picker head vacuum may switch off when it is retracted and positioned over the mouth 1194 of the vacuum assembly 1190 and the target 1150 drops into the vacuum assembly 1190 and transfer tube 1192 by pressure differential suction.

[00119] Seeker/ Sensor Subassemblies

[00120] In some examples, the harvesting described herein is directed by a seeker subassembly that is able to identify targets for harvesting, pass coordinates for the targets to the picker subassembly for extraction. Such seeker subassemblies may include any number of cameras (visible light, thermal, UV or other), radars, lidars, lasers, acoustic location finders, GPS, inertial navigation systems, piezoelectric sensors, and/or any combination of these or other sensors to locate and identify targets.

[00121] In some example embodiments, as discussed herein, the seeker subassembly may include a camera and/or multiple cameras arranged so as to be able to view the target foliage and thereby the target agriculture to be harvested. In some examples, multiple cameras may be arranged on the seeker subassembly such that images taken from the multiple cameras may be processed by a computing system to create three dimensional (3-D) images using machine vision. In some examples, these images are made of pixels and the computing systems is able to identify targets represented by pixels to be harvested and map the targets in three dimensions. In some examples, the cameras may be configured to acquire multi-spectral or hyper-spectral imagery to enable the use of advanced analysis algorithms for evaluating fruit health, quality and ripeness. In some examples, the images gathered may include those of a thermal imaging system for evaluating the temperature of the berry to be harvested. These cameras may comprise of cooled or uncooled sensors generating area-scanned images of at least 640 x 480 pixels. Some embodiments may utilize a single thermopile based sensor to provide an integrated temperature measurement of the mean temperature of the berry.

[00122] In some examples, the target identification is automated by Artificial Intelligence (AI) computing systems working in conjunction with a neural network data base and the camera systems. In some examples, the target identification is aided by a human who is analyzing a visual representation of the image data sent by the camera(s). The human operator may utilize human logic to perform quality control on the berry quality, correcting errors of identification by the AI systems, and to determine grade (i.e. #1 grade, #2 grade, spoiled, immature). In addition, the human operator may determine the best approach angle for the berry grappler to capture the fruit. In some examples, the processing computer may provide visual guidance and suggestions to the human operator as to the quality of the berry using automated algorithms exploiting the spectral and spatial analysis of the target. In some examples, the human may be at a remote location, not on or necessarily near the harvesting machine or field itself, but at a location networked to the camera systems to analyze image data coming from the camera(s) and sent by wireless communications. In some examples, the wireless communications are at least one of cellular, Private Wireless Network, Bluetooth, satellite, radio, and/or any other wireless method using antennae and processing computers. In some examples, a private wireless network will consist of a fixed or portable base station that forms a mesh network with other fixed or portable base stations to provide seamless complete communications coverage of the entire field in which the harvester is operating. In some examples, the fixed or portable base station will connect to the Wide Area Network, or Internet, through a dedicated back-haul connection to an Internet Service Provider (ISP). The backhaul connection may utilize any technology capable of carrying the necessary bandwidth to execute the harvester mission including connections comprising of dedicated wireless point-to-point links, copper twisted-pair links, coaxial based communications or optical fiber technologies.

[00123] In operation, the berry grappler, once it reaches a predetermined off-set distance from the mapped coordinates of the berry to be picked, control of the movement of the robotic arm is handed off to an internal guidance system that will lock onto the targeted berry and fine tune any discrepancies in the logged coordinates that may occur from the forward movement of the harvesting platform. The internal guidance system will utilize a neural network and/or artificial intelligence in conjunction with accumulated data gathered onto a Neuro Network to make decisions.

[00124] To deal with leaves, stems and trash obstructing access to the targeted berry, the berry grappler will have the ability to recognize obstructions and perform a second operation to clear access to the targeted berry. The berry grappler will be equipped with a trash diverter located at its most forward tip of the grappler spoons. The trash diverter may be equipped with air jets, or foliage hook device, or rotating paddle device and other appropriate methods to displace the obstruction. In operation, the berry grappler trash diverter will move in a progressive diameter rotation around the targeted berry location, clearing the obstructions. This second operation will only take place when obstructions are viewed by the stereo cameras.

[00125] In some examples, the three dimensional image data processed by and sent from the camera(s) may allow for a virtual reality environment to be created for a human user. In such examples, a virtual reality headset or display may be utilized by a user, remote or close to the harvester, to identify target agriculture and thereby send the target mapping coordinates to the harvesting machine for harvesting.

[00126] In some examples, the remote human users may be allowed by the computer to determine an angle of attack for the picker head to harvest an agricultural target, using the sensor data. In some examples, that may include image data such that the user may angle the picker head assembly using an interface such as a joystick, touchscreen, or mouse.

[00127] In some examples, data created by the cameras and data created by the human selection of agriculture is stored by a computing device. In such examples, the identification data may be amassed in order to analyze and later create algorithms for neural network engines to process. In such examples, after much data of targeting, identification, and harvesting information is gathered, an neural network engine can be trained may be able to replicate some or all of the human targeting using the three dimensional maps.

[00128] Examples of cameras which may be used in the described systems include stereo vision with resolution of 1920 x 1080 and frame rates of 30 per second. In some examples, different stereo vision camera systems may be utilized, with different resolution and frame rates, these being only examples.

[00129] In some examples, a suite of these or other sensors could be placed on the frame or chassis of the harvester, mounted still, or on motors to swivel or turn. In some examples, a suite of these or other sensors could be placed at the end of a robotic arm such as those shown in FIG. 12. In the example, the sensor 1203 is able to detect the target 1250 in whichever manner the senor operates (light, heat, lidar, acoustic, radar, etc.). In some examples, multiple sensors are placed on a single arm 1260. In some examples, multiple arms 1260 operate with their own or multiple sensors 1203. In some examples, the sensors 1203 are mounted on a rotatable mount, able to move rotate in one, two, three, four, five, six, seven, or more degrees of freedom. In some examples, a robotic arm 1260 includes sensors, picker heads, and/or multiple sensors and/or picker heads, and/or a combination of sensors and/or picker heads. In some examples, the sensors may be stereoscopic sensors, spaced apart but aimed at a similar focal point to provide data for the computing system to create three dimensional models of targets and environment around targets.

[00130] In some examples, a seeker subassembly vehicle may work independently from the harvester subassembly, and in some examples, the two subassemblies are on the same traversing machine. In some examples, the harvesting subassembly has its own wheels and/or tracks or combination of both, to traverse down a row of agriculture and harvest the mapped targets it receives from the seeker subassembly. In some examples, the seeker subassembly vehicle and harvesting subassembly vehicle are able to mate, connect, and/or otherwise work in concert by connection. In some examples, this connection includes a wired connection to allow for target information to be passed from the seeker subassembly to the harvesting subassembly. In some examples, the two subassemblies communicate wirelessly. In some examples, a combination of wired and wireless communication may be arranged.

[00131] Lighting

[00132] In some example embodiments, the seeker subassembly includes various specialized lighting which may be used to find and identify targets. Such lights may be configured on the ends of robotic arms, integrated into robotic arms that include picker heads, or cameras. Examples are shown in FIG. 1 A and IB. Such lights may be fixed onto other sub- assemblies on the seeker assembly and/or harvesting sub-assembly.

[00133] In some examples, such specialized lighting may be configured to emit a certain wavelength or spectrum of wavelengths such as but not limited to visible light, infra-red light, and/or ultra-violet light. In some examples, the lighting may be at a wavelength that excites items to fluoresce. In some example embodiments, light spectrum filters may be used by the cameras described herein to filter out or delete wave lengths of light that would otherwise block out any fluorescent properties reflected or emitted by targets such as berries.

[00134] In some examples, the specialized lighting may be light emitting diodes which are tuned to emit light at a specific frequency. In some examples, that frequency may be a combination of 470nm (blue) and 635nm (red). In some examples, the lights may be LED lights. In some examples, the lights may be incandescent lights. In some examples, the lights may be halogen lights, fluorescent lights, metal-halide, neon, high-intensity discharge lamps, or any permutation or combination of any of the above.

[00135] Mapping and Passing Target Coordinates

[00136] As described herein, in some example embodiments, additionally or alternatively, sensors onboard the harvesting systems such as machine vision camera and computing systems may be used to map target agriculture in three dimensions and pass the coordinates to the harvester subassembly for harvesting. In some examples, these are stereoscopically arranged cameras, with similar field of views to create image data that may be used by the computing systems to create three dimensional maps of targets. These mapping coordinates may be described in a global coordinate system such as Universal Transverse Mercader (UTM), or a local coordinate system frame relative to the coordinate system defined by the three dimensional imaging system on the harvester. In some examples, an C,U,Z coordinate system may be employed using an anchor point in the camera view and/or on the traversing machine itself.

[00137] The various sensors described herein including but not limited to visible light cameras, infrared cameras, ultraviolet cameras, lidars, radars, lasers, or other sensors may be used to scan the produce plants and identify targets. Using the automated, semi-automated, or manual selection processes and systems described herein, the systems could generate coordinates for selected targets. These mapped target coordinates may then be queued in a buffer or database, for the harvester subassembly to harvest. In some examples, after one coordinate is added to the harvesting coordinate queue, more targets may be added to the queue to be harvested in turn. In such examples, the targeting subassembly, machine vision, and target mapping may occur without lag or delay in the handoff from targeting to harvesting, and not be hampered by the limitations of the harvesting subassembly itself. In such a way, in some examples additionally or alternatively, the targeting subassembly may be mounted on a separate vehicle to travel at its own speed and send targeting mapped data to the harvesting subassembly by wireless communications. In some examples, the targeting subassembly may be a part of the overall machine and connected to or in communication with the harvesting subassembly to pass the targeting mapped coordinate queue by wired communications to the harvesting subassembly. In some examples, a cloud or distributed computing resource may be utilized so that the targeting queue may be relayed or sent to the harvesting subassembly wirelessly.

[00138] In some examples, the mapping may be done early or before a harvester machine may come down a row. Additionally or alternatively, in some examples, mapping may be done just before harvesting, on the same machine in some examples to minimize the variables of the berries and/or foliage moving. Any time between target mapping and harvesting may be utilized, depending on the circumstances of the harvest.

[00139] In some examples, mapping information may be stored in a remote server as described in FIG. 13, cloud server, or distributed system, for the purpose of future analysis (post processing) of the imagery to evaluate the condition of the plant. Post processing operations may include an evaluation of the plant for disease, nutrient deficiency, unripe berry inventory, and/or other plant damage. Data gathering and analysis on all types of agricultural specifics may be accomplished using the suite of cameras and/or sensors on the systems described herein. For example, outputs of post processing operations may be utilized to selectively address in-field issues at a plant-local scale that may otherwise require broad remedies using traditional methods. Other outputs of post processing operations may generate statistical data related to observations and measurements that are made while the harvester is operating in the field that can be advantageous to the growers business efforts.

[00140] Planter Subassembly Examples

[00141] In some example embodiments, a subassembly may be utilized which may be used to automate the bed preparation and planting of plants in the beds as an additional or alternative embodiment than the harvesting robotic assemblies described in FIGs. 3A, 3B, 3C, etc. In some examples, such a subassembly may include an arm with a propane torch instead of or in addition to a picker assembly. In some examples, the propane torch includes a circular metal shape within which the torch is lit. In such examples, when the circularly shaped torch is brought within range of a plastic sheet covering a planting bed, the torch may burn a circularly shaped hole in the plastic, but leave no residual flap or other integrity diminishing aspects to the plastic. In such examples, the subassembly may be configured to space the holes for the plantings in the plastic at predetermined distances from one another. In some examples, the subassembly may be configured to place one, two, or three lines of holes on one bed row, depending on how many plants are configured to be placed on one bed row. In some examples, the holes may be side-by-side. In some examples, the holes may be staggered or alternating.

[00142] In some examples, after the robotic arm(s) has burned holes in the plastic covering the bed rows, another arm may be utilized to insert a seedling plant into the dirt under the hole in the plastic. In such examples, camera arrangements may be utilized to identify the holes in the plastic and align the robotic assembly to the center of the hole. In such examples, the robotic arm may then utilize a spade shaped tool to dig into the dirt under the hole a predetermined distance to make room for roots of a seedling plant. In some examples, a spoon or pinch mechanism may be used to grasp and insert the seedling plant in the hole dug by the spade, at a predetermined distance into the dirt, and inside the hole in the plastic. In some examples, the hole may be made with the same tool that plants the seedling. In some examples, this is a two-step process, to make the hole and then plant the seedling. In some examples, this is a one-step process, to make the hole and plant the seedling.

[00143] Computer and Computer Network Examples

[00144] Additionally or alternatively, the systems described here may be used to harvest agricultural targets in an automated, semi-automated, or even manually controlled manner. In some examples, the semi-automated manner may be arranged in a remote setting, allowing for a human to interact with camera views from the harvester to help target the produce. Such an arrangement may be made possible by a network and computer arrangement shown in FIG. 13 and/or FIG. 14.

[00145] The variations on these options depend on how much a remote or local computing system may be programmed to identify and harvest a target. For example, in a fully manually controlled system, a human operator may control the movements of both the seeker system and the harvesting system. In such examples, by remote control using a joystick or other computer driven operating device(s) a human could scan the rows of plants for a target using the camera systems, and even maneuver the robotic arms that the camera systems are connected to, to identify targets, and then use a control system such as a joystick to maneuver the picker head assembly to the target, and then harvest the target as described herein. Such examples would allow for remote operation of the systems such as by wireless control to allow for human controllers to be stationed anywhere in the world, through some kind of wireless uplink.

[00146] The other extreme of control systems would be a fully automated system. In such a system, the traversing machines would move down a row of agricultural targets and the seeker subassembly would use machine learning / artificial intelligence / neural networks / and/or other programming to seek out and identify targets with the seeker subassemblies and then harvest them as described using the picker heads. Such examples would depend on computer algorithms and programs to determine using the inputs from the cameras and sensors, what a target may be and where they are located. For example, a color camera may be used by the computing system to detect a red strawberry amongst the green foliage of the plant it is growing on. Then a laser system could be used to determine a proximate location and range from the system and the computers could use that information to triangulate a three dimensional coordinate system and identify where the target is located in space, relative to the traversing machine. Next, the coordinates could be passed to the harvesting subassembly where the picker heads may attach to and harvest the target strawberry, in some examples using its own sensors such as cameras and lasers.

[00147] The middle-ground option between the fully automated and the manually controlled system would be some variant of semi-automated seeking and harvesting. The degree of semi-autonomy and which portions were automated and which manually controlled could vary from separate subassemblies. For example, the seeker subassembly may be more manually controlled with a human interacting with the cameras and sensors to help identify targets. In some examples, that may include a human interacting with a graphical user interface“GUI” such as a touchscreen to identify a target displayed on the screen. [00148] In any of the above examples of automation, the sensors onboard the harvesting system may be used to create, track and pass coordinates of the targets for harvesting.

[00149] The computing architecture for the harvester could be described as a distributed computing system comprising of elements or processing centers that exist on the harvester, a central server system which may or may not be a cloud based resource and an operator processing system. Each of these processing centers are interconnected through an IP network which may include local private wireless networks, private wide area networks and/or public networks such as the Internet as described in FIGs. 13 and 14. Computational tasks may be divided such that real-time tasks are executed on the local harvester processor, post-processing operations and non-real time computation are executed on the central server and user-interface computation are performed on the operator processing center.

[00150] In example systems described herein, various computing components may be utilized to operate the systems. For example, a communication computing system may allow for remote operation of the machines, sensors may send information to a computing system to help differentiate targets from non-targets, target location and mapping information may be calculated, stored, sent, and utilized between the seeker systems and harvesting systems, steering and driving instructions may be calculated and utilized, machine learning / artificial intelligence / and/or neural networks may be employed by computing systems to find and harvest targets, and any of the other computing operations as described herein. In some examples, alternatively or additionally, a WiFi system / cellular system / Bluetooth system, or any other communication system, with the appropriate antenna system and a processor and memory as described herein, may be used on a subassembly. In some embodiments, alternatively or additionally, the hardware may include a single integrated circuit containing a processor core, memory, and programmable input/output peripherals. In some examples, various computing components may be used in the seeker and/or harvesting subassemblies, as well as the communication systems, control systems, and/or any other portion of the systems described herein.

[00151] FIG. 13 shows an example networked system which could be used in the systems and methods here. In FIG. 13, the computer system 1302 onboard the harvesting system described herein, including process any images from the various sensors including cameras taking images of the targets and plants. Such image data may include pixel data of the captured target images. The computer 1302 could be any number of kinds of computers such as those included in the camera itself, navigation system, robotic arm maneuvering systems, traversing or driving systems, sensor systems, and/or any other another computer arrangement including those examples are described in FIG. 14.

[00152] As shown in FIG. 13, the various computing systems may be in communication with a back end computing system 1330 and/or data storage 1332 to send and receive data regarding the operations of the harvesting systems described herein. For example, as shown in FIG. 13, captured image data of targets may be transmitted to a back end computer system 1330 and associated data storage 1332 for saving and analysis. In some examples, this may include the remote operators who are interfacing with the harvesting systems, selecting targets, providing navigation instruction, steering, and/or overseeing maintenance of the systems. In some examples, the communication may be a wireless transmission 1310 by a radio, cellular or WiFi transmission with associated routers and hubs. In some examples, the transmission may be through a wired connection 1312. In some examples, a combination of wireless and wired transmissions may be used to stream data between the back end 1330 and the harvesting system including cameras, robotic pickers, etc.

[00153] In some examples, the transmission of data may include transmission through a network such as the internet 1320 to the remote operators, back end server computers 1330, and associated data storage 1332. Once at the back end server computer servers 1330 and associated data storage 1332, the pixelated image data may be acted upon by the remote operators to choose targets to harvest. In some examples, the data may be useful to train the neural network, and/or artificial intelligence models as to good targets versus targets to pass up. In such examples, the image and target data may be stored, analyzed, used to train models, or any other kind of image data analysis. In some examples, the storing, analyzing, and/or processing of image data may be accomplished at the computer 1302 which is involved in the original image capture. In some examples, the local computer 1302 and a back end computing system 1330 may split or share the data storing, modeling, analyzing, and/or processing. Back end computer resources 1330 may be more powerful, faster, or be able to handle more data than may be otherwise available at the local computers 1302 on the harvesting machines. In some examples, the networked computer resources 1330 may be spread across many multiple computer resources by a cloud infrastructure. In some examples, the networked computer resources 1330 may be virtual machines in a cloud infrastructure.

[00154] FIG. 14 shows an example computing device 1400 that may be used in practicing example embodiments described herein. FIG. 14 could describe computers such as 1302, 1330 or other systems as described in FIG. 13. Such computing device 1400 may be the back end server systems use to interface with the network, receive and analyzed data, as well as generate remote operator GUIs, additionally or alternatively, it could be a computing system aboard the harvesting system as described, to instruct the robotic assemblies, cameras, communications, etc.. Such computer 1400 may be a device used to create and send data, as well as receive and cause display of GUIs representing data such as back end interfaces for remote operators, camera image analysis, etc.. In FIG. 14, the computing device could be a smartphone, a laptop, tablet computer, server computer, or any other kind of computing device. The example shows a processor CPU 1410 which could be any number of processors in communication via a bus 1412 or other communication with a user interface 1414. The user interface 1414 could include any number of display devices 1418 such as a screen. The user interface also includes an input such as a touchscreen, keyboard, mouse, pointer, buttons, joystick or other input devices. Also included is a network interface 1420 which may be used to interface with any wireless or wired network in order to transmit and receive data. Such an interface may allow for a smartphone, for example, to interface a cellular network and/or WiFi network and thereby the Internet. The example computing device 1400 also shows peripherals 1424 which could include any number of other additional features such as but not limited to cameras, sensors 1425, and/or antennae 1426 for communicating wirelessly such as over cellular, WiFi, NFC, Bluetooth, infrared, or any combination of these or other wireless communications. The computing device 1400 also includes a memory 1422 which includes any number of operations executable by the processor 1410. The memory in FIG. 14 shows an operating system 1432, network communication module 1434, instructions for other tasks 1438 and applications 1438 such as send/receive message data 1440 and/or SMS text message applications 1442. Also included in the example is for data storage 1458. Such data storage may include data tables 1460, transaction logs 1462, user data 1464 and/or encryption data 1470. The computing device 1400 also include one or more graphical processing units (GPUs) for the purposes of accelerating in hardware computationally intensive tasks such as execution and or evaluation of the neural network engine and enhanced image exploitation algorithms operating on the multi-modal imagery collected. The computing device 1400 may also include one or more reconfigurable hardware elements such as a field programmable gate array (FPGA) for the purposes of hardware acceleration of computationally intensive tasks.

[00155] Conclusion

[00156] As disclosed herein, features consistent with the present inventions may be implemented by computer-hardware, software and/or firmware. For example, the systems and methods disclosed herein may be embodied in various forms including, for example, a data processor, such as a computer that also includes a database, digital electronic circuitry, firmware, software, computer networks, servers, or in combinations of them. Further, while some of the disclosed implementations describe specific hardware components, systems and methods consistent with the innovations herein may be implemented with any combination of hardware, software and/or firmware. Moreover, the above-noted features and other aspects and principles of the innovations herein may be implemented in various environments. Such environments and related applications may be specially constructed for performing the various routines, processes and/or operations according to the invention or they may include a computer or computing platform selectively activated or reconfigured by code to provide the necessary functionality. The processes disclosed herein are not inherently related to any particular computer, network, architecture, environment, or other apparatus, and may be implemented by a suitable combination of hardware, software, and/or firmware. For example, various machines may be used with programs written in accordance with teachings of the invention, or it may be more convenient to construct a specialized apparatus or system to perform the required methods and techniques.

[00157] Aspects of the method and system described herein, such as the logic, may be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (“PLDs”), such as field programmable gate arrays (“FPGAs”), programmable array logic (“PAL”) devices, electrically programmable logic and memory devices and standard cell-based devices, as well as application specific integrated circuits. Some other possibilities for implementing aspects include: memory devices, microcontrollers with memory (such as 1PROM), embedded microprocessors, Graphics Processing Units (GPUs), firmware, software, etc. Furthermore, aspects may be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types. The underlying device technologies may be provided in a variety of component types, e.g., metal-oxide semiconductor field-effect transistor (“MOSFET”) technologies like complementary metal-oxide semiconductor (“CMOS”), bipolar technologies like emitter- coupled logic (“ECL”), polymer technologies (e.g., silicon-conjugated polymer and metal- conjugated polymer-metal structures), mixed analog and digital, and so on.

[00158] It should also be noted that the various logic and/or functions disclosed herein may be enabled using any number of combinations of hardware, firmware, and/or as data and/or instructions embodied in various machine-readable or computer-readable media, in terms of their behavioral, register transfer, logic component, and/or other characteristics. Computer- readable media in which such formatted data and/or instructions may be embodied include, but are not limited to, non-volatile storage media in various forms (e.g., optical, magnetic or semiconductor storage media) and carrier waves that may be used to transfer such formatted data and/or instructions through wireless, optical, or wired signaling media or any combination thereof. Examples of transfers of such formatted data and/or instructions by carrier waves include, but are not limited to, transfers (uploads, downloads, e-mail, etc.) over the Internet and/or other computer networks by one or more data transfer protocols (e.g., HTTP, FTP, SMTP, and so on).

[00159] Unless the context clearly requires otherwise, throughout the description and the claims, the words“comprise,”“comprising,” and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in a sense of“including, but not limited to.” Words using the singular or plural number also include the plural or singular number respectively. Additionally, the words “herein,” “hereunder,” “above,” “below,” and words of similar import refer to this application as a whole and not to any particular portions of this application. When the word“or” is used in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.

[00160] Although certain presently preferred implementations of the invention have been specifically described herein, it will be apparent to those skilled in the art to which the invention pertains that variations and modifications of the various implementations shown and described herein may be made without departing from the spirit and scope of the invention. Accordingly, it is intended that the invention be limited only to the extent required by the applicable rules of law.

[00161] The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.

[00162] The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. Etc.