Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
AGRICULTURAL SYSTEM FOR LOW AND HIGH-RESOLUTION SPOT SPRAYING AND METHOD FOR OPERATING SUCH A SYSTEM
Document Type and Number:
WIPO Patent Application WO/2023/100102
Kind Code:
A1
Abstract:
The present invention relates to an agricultural system (100) for spraying an area of a cultivated field (10), including a spraying equipment (200). The spraying equipment comprises a first and a second spray bar (210, 215) extending perpendicularly to the travel direction of the agricultural system (100) when operating. The first spray bar (210) comprises a first array of nozzles (220) separated from each other by a first distance while the second spray bar (215) comprises a second array of nozzles (240) separated from each other by a second distance smaller than said first distance. Consequently, the spatial resolution of spot sprays (242) that may be sprayed by the second array of nozzles (240) is higher than the spatial resolution of spot sprays (222) that may be sprayed by the first array of nozzles (220) when the first and second spray bars (210, 215) are at the same height such that the first and second arrays of nozzles (220, 240) may perform respectively low and high-resolution spot sprays (222, 242).

Inventors:
TANNER STEVE (CH)
Application Number:
PCT/IB2022/061597
Publication Date:
June 08, 2023
Filing Date:
November 30, 2022
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
ECOROBOTIX SA (CH)
International Classes:
A01M7/00; A01C1/00; A01C23/00; A01C23/04; B05B1/20; B05B12/12; B05B15/68
Domestic Patent References:
WO2020157702A12020-08-06
WO2019105651A12019-06-06
Foreign References:
DE102019203394A12020-09-17
US6206406B12001-03-27
DE102007008786A12008-08-28
EP3469905A12019-04-17
DE102018221468A12020-06-18
Attorney, Agent or Firm:
P&TS SA (AG, LTD.) (CH)
Download PDF:
Claims:
Claims

1. Agricultural system (100) for spraying an area of a cultivated field (10), including a spraying equipment (200), the spraying equipment comprising: a first and a second spray bar (210, 215) extending perpendicularly to the travel direction of the agricultural system (100) when operating, wherein the first spray bar (210) comprises a first array of nozzles (220) separated from each other by a first distance while the second spray bar (215) comprises a second array of nozzles (240) separated from each other by a second distance smaller than said first distance such that the spatial resolution of spot sprays (242) that may be sprayed by the second array of nozzles (240) is higher than the spatial resolution of spot sprays (222) that may be sprayed by the first array of nozzles (220) when the first and second spray bars (210, 215) are at the same height such that the first and second arrays of nozzles (220, 240) may perform respectively low and high-resolution spot spraying (222, 242); a first spray bar height control system (230) comprising at least one height actuator (232), and a second spray bar height control system (250) comprising at least one height actuator (252) so as to control independently the height of the first and second spray bars (210, 215) with reference to the ground (12); a camera system (270) comprising one or more camera modules (300) comprising a camera (310) arranged to capture images of objects, for example plants (14) and/or part of plants (16), and of the ground (12), ahead of the first and second spray bars (210, 215) in the travel direction of the agricultural system (100); a processing unit (274) configured to run an image recognition software to identify said objects on the acquired images and to generate a mapping of these objects on the coordinate system of the spray bars, one or more object tracking units (276) configured to continuously track the position of these objects on said mapping, a first and a second nozzles array control unit (280, 282) configured to control each nozzle of respective first and second array of nozzles (220, 240) as to selectively control the nozzles of said first and second arrays of nozzles (220, 240) to perform low and/or high-resolution spot spraying (222, 242) on different objects as a function of the position of these objects on said mapping with respect to the position of the first and second spray bars (210, 215).

2. The agricultural system (100) of claim 1, wherein the spray bar height control system (230) comprises one or more primary height actuators (232) arranged to control the height of both first and second spray bars (210, 215) with reference to the ground (12) and one or more secondary height actuators (252) arranged to move one of said first and second spray bars (210, 215) relative to the other of said first and second spray bars (210, 215).

3. The agricultural system (100) of the preceding claim, wherein the spraying equipment (200) comprises distance measurement sensors (290) arranged to measure the distance from the first and/or the second spray bar (210, 215) to the ground surface (12) to determine an estimated object plane (326), said distance measurement sensors (290) being for example mounted along the first and second spray bars (210, 215), and wherein said distance measurement sensors (290) are arranged to send distance information to the first and a second nozzles array control unit (280, 282) to regulate the height of said first and second spray bars (210, 215) to ground as a function of said distance information.

4. The agricultural system (100) of the preceding claim, wherein the camera module or each camera module (300) further comprises a 3D depth sensor (314) arranged to measure the distance (320) between any point of the objects acquired by the camera (310) and said 3D depth sensor (314) to generate a depth map on which is mapped said objects in the tri-dimensional coordinate system of the first and second spray bars (210, 215), for correction of any horizontal mapping errors (322) caused by the height difference (324) between said estimated object plane (326) and the real object positions to improve the spraying accuracy.

5. The agricultural system (100) of the preceding claim, wherein said 3D depth sensor (314) is a laser scanning system with time of flight (LIDAR) or triangulation, a stereovision system, a time-of-flight camera, or a structured light depth camera.

6. The agricultural system (100) of claim 4, wherein the camera module or each camera module (300) comprises at least two cameras arranged to acquire simultaneously respectively a fist and a second set of images of the ground (12) ahead of the first and second spray bars (210, 215), and a stereovision computing unit (316) configured to compute the depth map of the objects as a function of said first and second sets of images.

7. The agricultural system (100) of any preceding claim, wherein the spraying equipment (200) includes a first and a second fluid distribution systems (510, 520) arranged to provide to respective first and second arrays of nozzles (220, 240) a different chemical mixture at a possibly different pressure, so as to enable the spraying equipment (200) to spray two different chemical preparations in a single passage, for example simultaneously, each chemical preparation including for example herbicide, fungicide, insecticide, fertilizer or nematocide.

8. The agricultural system (100) of any preceding claim, wherein said second array of nozzles (240) comprises several segments (260) comprising each a series of nozzle, the second spray bar height control system (250) comprising one actuator (252) per segment (260) to allow fine control of the nozzles to ground distance of each segment (260) independently from the others.

9. The agricultural system (100) of any preceding claim, wherein the first spray bar (210) comprising said first array of nozzles (220) is adapted to be extended laterally, either by translation or by unfolding of one or several first spray bar extensions (550) equipped with the low-resolution nozzle array (220) and with range measurement sensors (290), so as to provide a larger working width for full spray application to achieve continuous and homogeneous spray application without low and high-resolution spot spraying (222, 242).

10. The agricultural system (100) of any preceding claim, wherein the camera module or each camera module (300) is rotatably mounted on a support of the spraying equipment (200), for example on the first spray bar (210), to provide varying tilt angle of the optical axis of the camera (310) of the camera module or each camera module (300) with reference to the ground (12) to modify the distance between an edge of the captured images of an area of a cultivated field (10) by the camera (310) and the projection on the ground (12) of said first and second spray bars (210, 215).

11. The agricultural system (100) of the preceding claim, wherein said camera module or each camera module (300) is mounted on a mast (600) extending forward from said first and second spray bars (210, 215), the mast (600) comprising a motorized rotation system (620) for controlled, accurate and repeatable rotation of the mast around an axis (610) extending perpendicularly to the travel direction of the agricultural system (100) and allowing multiple positions between two extreme positions, thus offering varying tilt angle for the camera module (300).

12. Method of operating the agricultural system (100) of any preceding claim, using visual odometry to detect and track the tri-dimensional movement of said objects in the tri-dimensional coordinate system of the first and second arrays of nozzles (220, 240) such that the first and second nozzles array control units (280, 282) timely control said first and second

22 array of nozzles (220, 240) according to the position of said objects in said 3D coordinate system, the visual odometry comprising the steps of: i. capturing with the camera (310) a first image of the ground (12) and the objects on the ground; ii. simultaneously capturing with a 3D depth sensor (314) a depth map of the ground and the objects captured in the first image; iii. merging the first image with the depth map to obtain a 3D image; iv. extracting on said 3D image a set of features, such as contour of an object, for features tracking; v. repeating steps i to iv and tracking the movement of the extracted features between two consecutive 3D images as the agricultural system (100) moves along its travel direction to compute the movement of said extracted features in said 3D coordinate system.

13. Method of operating the agricultural system (100) of any claims 1 to 11, wherein the optical axis of the camera (310) of the camera module or each camera module (300) is tilted around its rotation axis (610) to set a first distance (642) and a second distance (632) smaller than the first distance, said distance being the distance between an edge of the captured image of an area of a cultivated field (10) and the projection on the ground of said first and second spray bars (210, 215), and wherein said optical axis is adjusted to

- set the first distance (642) when only low-resolution spot spraying (222) is used, thereby increasing the time for computation and consequently to allow speed increase of the agricultural system (100),

- set the second distance (632) when only high-resolution spot spraying (242) is used, wherein the speed of the agricultural system (100) is limited to allow high-precision mapping of the plants in the ground reference.

14. Method of operating the agricultural system (100) of any claims 1 to 11, wherein the camera module or each camera module (300) includes a first camera (310) sensitive to a first set of spectral bands, a second camera (312)

23 sensitive to a second set of spectral bands, a 3D optical depth sensor (314) sensitive to a third set of spectral bands, one or more irradiation elements (330, 332, 334), the spectral band emission of said one or more irradiation elements (330, 332, 334), combined together, covering the entire spectral bands of the first camera (310), the second camera (312) and the 3D depth sensor (314), each irradiance element being switched on only when the pixels of the cameras (310, 312) or the 3D depth sensor (314) are integrating light using the corresponding spectrum.

15. Method of operating the agricultural system (100) of any claims 1 to 11, wherein the first spray bar height control system (230) regulates the distance between the first spray bar (210) and the ground (12) as a function of a combination of the information provided by distance measurement sensors (290), by 3D depth sensors (314), and by the second spray bar height control system (250), in particular the position, speed and/or acceleration information of the second spray bar (215) and/or each of several segments (260) of said the second spray bar (215).

16. Method of operating the agricultural system (100) of any claims 1 to 11, wherein the second spray bar height control system (250) regulates the distance between the second spray bar (215) and/or each of several segments (260) of said the second spray bar (215) with the ground (12) as a function of a combination of the information provided by distance measurement sensors (290), by 3D depth sensors (314), and by the first spray bar height control system (230), in particular the position, speed and/or acceleration information of the first spray bar (210).

17. Method of operating the agricultural system (100) according to any of claims 1 to 11, wherein the first and second arrays of nozzles (220, 240) are supplied either by a same liquid distribution system, or by their respective liquid distribution systems (510, 520), and wherein the first and second control units (280, 282) are operated to control each nozzle of respective first

24 and second arrays of nozzles (220, 240) to perform on the cultivated field (10) any of the following operations: a. performing low-resolution and high-resolution spot spraying (222, 242) simultaneously, b. performing low-resolution spot spraying (222) while high- resolution spot spraying (242) is not performed, c. performing high-resolution spot spraying (242) while the low- resolution spot spraying (242) is not performed, d. performing a continuous and homogeneous spray application with the first array of nozzles (220) while the second array of nozzles (240) is not used, e. performing a continuous and homogeneous spray application with the second array of nozzles (240) while the first array of nozzles (220) is not used, f. performing a continuous and homogeneous spray application with the first array of nozzles (220) while the second array of nozzles (240) performs high-resolution spot spraying (242), g. performing a continuous and homogeneous spray application with the second array of nozzles (240) while the first array of nozzles (220) performs low-resolution spot spraying (222), h. performing a continuous and homogeneous spray application with the first and second arrays of nozzles.

Description:
Agricultural system for low and high-resolution spot spraying and method for operating such a system

Field of the invention

[001] The present invention concerns the field of agriculture plant treatment and more particularly an agricultural system for low and high- resolution spot spaying. The present invention also concerns a method of operating the agricultural system for multiple spraying modes of agrochemicals.

Description of related art

[002] In conventional agricultural spraying systems, the nozzles are placed on a spray bar aligned orthogonally to the direction of displacement of the vehicle on which it is mounted, and parallel to the surface to be sprayed. The nozzles are aligned on the spray bar, and usually separated to each other by a uniform distance. Their divergent jet allows to apply the liquid on a given surface, which depends on the distance from nozzle to the target, usually the height distance from nozzle to the ground or plants. A conventional spray application is applied as homogeneously as possible on the complete surface under the spray bar, such application being called "full" application in the following description.

[003] Recently, so-called smart sprayers have appeared on the market. They include cameras forward-looking fixed along the spray bar and processing units configured to process images captured by the cameras to recognize the presence of specific plants in front of the spray bar. The nozzles comprise electro valves which are switched on when a plant to be sprayed passes under the corresponding nozzle. The spatial precision and size of the corresponding spot sprays is limited due to the nozzle-to-nozzle distance of a conventional spray bar, and the limited plant mapping precision allowed by the camera system. These smart sprayers have therefore the inconvenient to use large amount of herbicide which are applied indiscriminately on the plants to be treated and to their surroundings which do not need any treatment, thereby wasting a significant amount of herbicide.

[004] There is therefore a need for more precise spot spray applications to reduce the agrochemicals usage. In order to be more precise in spraying, the spray spot size must be smaller, and must be localized with greater accuracy. A smaller spray spot is obtained with a higher nozzle density along the bar, and more narrow spray angles. However, the minimal angle is usually limited due to the need for the nozzle to produce droplets at a correct size. Small jet divergence angles have the tendency to produce too big droplets. Therefore, the only way to reduce the spray spot is to reduce the distance from nozzle to target, usually near the ground. In order to position the activation of the spot spray with greater accuracy, the object to be sprayed must first be mapped accurately in the coordinate system of the sprayer, and then its displacement must be tracked with accuracy so that the nozzle are activated at the right moment and right place.

[005] In a conventional agricultural sprayer equipped with a single spray bar, spanning horizontally on a distance of several tenths of meters, the regulation of the distance of the spray bar to the ground is done actively by a regulation system. This system comprises distance sensors mounted along the spray bar for measuring the vertical distance of the spray bar to ground, and actuators configured to move the spray bar according to the output sensors, usually with a rotation around an axis parallel to the displacement (roll movement) and vertical translation. There exist some more sophisticated regulation schemes, for instance for each half spray bar, but reducing the nozzle to ground distance on a large sprayer is very difficult, as the spray bar height control system of sprayers has limited dynamical capability to follow the ground surface in the presence of shocks in the wheels of the vehicle, caused by bumps or holes in the terrain. Then, controlling a constant distance along the complete spray ramp in the presence of uneven field is very challenging. Finally, the inertial behaviour of a wide spray ramp makes it prone to collision with ground with short nozzle to ground distance.

[006] Reducing the nozzle-to-nozzle distance to improve spray precision has another negative aspect: the flow of each nozzle needs to be reduced accordingly to keep the same applied dose per unit area. Low-capacity nozzles are then not ideal for full applications, and therefore the sprayer capability to perform full spray application, such as application of liquid fertilizer, or fungicide, is reduced. The smart sprayer loses its "universal" usage and two different sprayers are then required: one for full application, where spatial precision is not a concern, and one for spot spray application with the maximal precision in the spot spray operations.

[007] To reach high spraying precision, a critical operation is to localize the objects to be sprayed with high accuracy. This localization is usually done with an image sensor fixed on the spray bar, which is accurately georeferenced on it, usually by means of a geometric calibration, so that any detected object is correctly mapped and its relative displacement in the rear direction can be tracked until it is sprayed. However, in most sprayers, the cameras are tilted so as to look forward, which allows a detection sufficiently early to leave enough time for the image processing to be done. Unfortunately, tilting the camera in the horizontal direction results in poor object mapping due to the absence of distance measurement with a standard image sensor. The distance must be approximated, resulting in important errors. Furthermore, any ground irregularity translates directly into positioning errors. Finally, badly estimated object to camera distance causes also scaling errors in any visual odometry algorithm used to track the relative movement of the sprayer equipment with ground. Brief summary of the invention

[008] An aim of the present invention is therefore to provide an agricultural system for high-resolution spot spraying adapted to detect very small plants, and to perform accurate plant/object mapping and tracking.

[009] Another aim of the present invention is to provide an agricultural system adapted to perform full application of liquid, for instance fertilizer or fungicide, and high-resolution spot spraying application, for instance of non- selective herbicide for selective weeding operation, or insecticide on the crop plants only.

[0010] These aims are achieved, notably by an agricultural system for spraying an area of a cultivated field, including a spraying equipment. The spraying equipment comprise a first and a second spray bar, a first and a second spray bar height control system, a camera system, a processing unit, one or more object tracking units, and a first and a second nozzles array control unit. The first and second spray bar extend perpendicularly to the travel direction of the agricultural system when operating. The first spray bar comprises a first array of nozzles separated from each other by a first distance while the second spray bar comprises a second array of nozzles separated from each other by a second distance smaller than said first distance such that the spatial resolution of spot sprays that may be sprayed by the second array of nozzles is higher than the spatial resolution of spot sprays that may be sprayed by the first array of nozzles when the first and second spray bars are at the same height such that the first and second arrays of nozzles may perform respectively low and high-resolution spot spraying. The first and second spray bar height control system comprise each at least one height actuator so as to control independently the height of the first and second spray bars with reference to the ground. The camera system comprises one or more camera modules having each a camera arranged to capture images of objects, for example plants and/or part of plants, and of the ground, ahead of the first and second spray bars, in the travel direction of the agricultural system. The processing unit is configured to run an image recognition software to identify said objects on the acquired images and to generate a mapping of these objects on the coordinate system of the spray bars. The or each object tracking unit is configured to continuously track the position of these objects on said mapping. The first and second nozzles array control unit are configured to control each nozzle of respective first and second array of nozzles as to selectively control the nozzles of the first and second arrays of nozzles to perform low and/or high-resolution spot spraying on different objects as a function of the position of these objects on said mapping.

[0011] In an embodiment, the spray bar height control system comprises one or more primary height actuators arranged to control the height of both first and second spray bars with reference to the ground and one or more secondary height actuators arranged to move one of said first and second spray bars relative to the other of said first and second spray bars.

[0012] In an embodiment, the spraying equipment further comprises distance measurement sensors arranged to measure the distance from the first and/or the second spray bar to the ground surface to determine an estimated object plane. The distance measurement sensors are for example mounted along the first and second spray bars. The distance measurement sensors are arranged to send distance information to the first and a second nozzles array control unit to regulate the height of said first and second spray bars as a function of said distance information.

[0013] In an embodiment, the camera module or each camera module further comprises a 3D depth sensor arranged to measure the distance between any point of the objects acquired by the camera and said 3D depth sensor to generate a depth map on which is mapped said objects in the tridimensional coordinate system of the first and second spray bars, for correction of any horizontal mapping errors caused by the height difference between said estimated object plane and the real object positions to improve the spraying accuracy.

[0014] In an embodiment, the 3D depth sensor is a laser scanning system with time of flight (LIDAR) or triangulation, a stereovision system, a time-of- flight camera, or a structured light depth camera.

[0015] In an embodiment, the camera module or each camera module comprises at least two cameras arranged to acquire simultaneously respectively a fist and a second set of images of the ground ahead of the first and second spray bars, and a stereovision computing unit configured to compute the depth map of the objects as a function of said first and second sets of images.

[0016] In an embodiment, the spraying equipment includes a first and a second fluid distribution systems arranged to provide to respective first and second arrays of nozzles a different chemical mixture at a possibly different pressure, so as to enable the spraying equipment to spray two different chemical preparations in a single passage, for example simultaneously. Each chemical preparation includes for example herbicide, fungicide, insecticide, fertilizer or nematocide.

[0017] In an embodiment, the first spray bar comprising the first array of nozzles is adapted to be extended laterally, either by translation or by unfolding of one or several first spray bar extensions. These bar extensions are equipped with the low-resolution nozzle array and with range measurement sensors, so as to provide a larger working width for full spray application to achieve continuous and homogeneous spray application without low and high-resolution spot spraying. [0018] In an embodiment, the camera module or each camera module is rotatably mounted on a support of the spraying equipment, for example on the first spray bar, to provide varying tilt angle of the optical axis of the camera of the camera module or each camera module with reference to the ground to modify the distance between an edge of the captured images of an area of a cultivated field by the camera and the projection on the ground of said first and second spray bars.

[0019] In an embodiment, the camera module or each camera module is mounted on a mast extending forward from said first and second spray bars. The mast comprises a motorized rotation system for controlled, accurate and repeatable rotation of the mast around an axis extending perpendicularly to the travel direction of the agricultural system and allowing multiple positions between two extreme positions, thus offering varying tilt angle for the camera module.

[0020] Another aspect of the invention relates to a method of operating the agricultural system using visual odometry to detect and track the tridimensional movement of said objects in the tri-dimensional coordinate system of the first and second spray bars such that the first and second nozzles array control units timely control said first and second array of nozzles according to the position of said objects in said 3D coordinate system. The visual odometry comprises the steps of: i. capturing with the camera a first image of the ground and of the objects on the ground; ii. simultaneously capturing with a 3D depth sensor a depth map of the ground and the objects captured in the first image; iii. merging the first image with the depth map to obtain a 3D image; iv. extracting on said 3D image a set of features, such as contour of an object, for features tracking; v. repeating steps i to iv and tracking the movement of the extracted features between two consecutive 3D images as the agricultural system moves along its travel direction to compute the movement of said extracted features in said 3D coordinate system. [0021] In an embodiment, the optical axis of the camera of the camera module or each camera module is tilted around its rotation axis to set a first distance and a second distance smaller than the first distance. The distance is the distance between an edge of the captured image of an area of a cultivated field and the projection on the ground of said first and second spray bars. The optical axis is adjusted to set the first distance when only low- resolution spot spraying is used, thereby increasing the time for computation and consequently to allow speed increase of the agricultural system. The optical axis is adjusted to set the second distance when only high-resolution spot spraying is used, wherein the speed of the agricultural system is limited to allow high-precision mapping of the plants in the ground reference.

[0022] In an embodiment, the camera module or each camera module includes a first camera sensitive to a first set of spectral bands, a second camera sensitive to a second set of spectral bands, a 3D optical depth sensor sensitive to a third set of spectral bands, and one or more irradiation elements. The spectral band emission of said one or more irradiation elements, combined together, covers the entire spectral bands of the first camera, the second camera and the 3D depth sensor. Each irradiance element is turned on only when the pixels of the cameras or the 3D depth sensor are integrating light using the corresponding spectrum.

[0023] Another aim of the invention is to provide a precise and fast stabilization of the high resoution nozzle array to ground distance in uneven terrains. This is achieved by having two independent active height control systems for each one of the high and low resolution nozzle arrays. The first nozzle array (low resolution) being fixed directly on the first spray bar structure, its distance to ground is set by a first regulator controlling the stability of the first spray bar by means of a first set of actuators, generally controlling the roll angle and the vertical position of the bar. As the distance to ground of this first nozzle array is significant, a very accurate regulation is not needed, and the impact of the terrain irregularities on the spray accuracy do not degrade significantly the precisions of this low resolution spot spray. However, this is different with the second (high resolution) nozzle array, which must be placed and maintained much closer to ground in order to guarantee high spot spray accuracy. This is achieved by means of two elements. The first is to split this high resolution nozzle array into smaller segments, each typically of a few meters long, and each having its own active height control system, so that even in the presence of uneven terrain, the high resolution nozzles follow the surface profile. The second is to pilot independently these height controllers by the information provided by the 3D depth sensors located in front of the corresponding segment. Additional height information provided by the height sensors regulating the first spray bar can also be used.

[0024] Another aim of the invention is to take into account the mechanical coupling existing between the two position-regulated spray bars for the active control of the bars. Indeed, if such a coupling would not be taken into account, any vertical movement of the individual segments of the second spray bar resulting into a force applied to the first spray bar, would inject a perturbation in its height control. However, this perturbation is known, as it is generated by the output of the height controller of each second spray bar segment. Therefore, this knowledge can be used to compensate said perturbation in the controller method of the first spray bar height controller. Conversely, any regulation movement of the first spray bar will translate into a perturbation in the position of each segment of the second spray bar. This perturbation being known, it can be compensated in the controller method of each segment of the second spray bar. This mutually mechanically-coupled system can be described mathematically and the regulation methods of each spray bar height controller can be designed accordingly.

[0025] In an embodiment, the second array of nozzles comprises several segments comprising each a series of nozzle. The second spray bar height control system comprises one actuator per segment to allow fine control of the nozzles-to-ground distance of each segment independently from the others.

[0026] In an embodiment, the first spray bar height control system regulates the distance between the first spray bar and the ground as a function of a combination of the information provided by distance measurement sensors, by 3D depth sensors, and by the second spray bar height control system, in particular the position, speed and/or acceleration information of the second spray bar and/or each of several segments of said the second spray bar.

[0027] In an embodiment, the second spray bar height control system regulates the distance between the second spray bar and/or each of several segments of said the second spray bar with the ground as a function of a combination of the information provided by distance measurement sensors, by 3D depth sensors, and by the first spray bar height control system, in particular the position, speed and/or acceleration information of the first spray bar.

[0028] In an embodiment, the first and second arrays of nozzles are supplied either by a same liquid distribution system, or by their respective liquid distribution systems. The first and second control units are operated to control each nozzle of respective first and second arrays of nozzles to perform on the cultivated field any of the following operations: a. performing low-resolution and high-resolution spot spraying simultaneously; b. performing low-resolution spot spraying while high- resolution spot spraying is not performed; c. performing high-resolution spot spraying while the low-resolution spot spraying is not performed; d. performing a continuous and homogeneous spray application with the first array of nozzles while the second array of nozzles is not used; e. performing a continuous and homogeneous spray application with the second array of nozzles while the first array of nozzles is not used; f. performing a continuous and homogeneous spray application with the first array of nozzles while the io second array of nozzles performs nigh-resolution spot spraying; g. performing a continuous and homogeneous spray application with the second array of nozzles while the first array of nozzles performs low- resolution spot spraying, and h. performing a continuous and homogeneous spray application with the first and second arrays of nozzles

Brief Description of the Drawings

[0029] The invention will be better understood with the aid of the description of embodiment given by way of examples and illustrated by the figures, in which:

Figure 1 shows a schematic lateral view of an agricultural system comprising a spraying equipment according to an embodiment;

Figure 2 shows a schematic perspective view of a portion of the agricultural spraying equipment of Figure 1;

Figure 3 shows a schematic lateral view of the first and second spray bars with the high- and low-resolution nozzle arrays, the camera module and associated systems to process the images and control the spraying equipment according to an embodiment;

Figure 4 shows a schematic diagram of the elements associated to one camera module of the spraying equipment according to an embodiment;

Figure 5a shows a schematic front view of one camera with the various mapping errors due to projection occurring without range sensor;

Figure 5b shows a similar view of Figure 5a with exact mapping when a 3D range sensor is used, according to an embodiment; Figure 6 shows a schematic rear view of the spraying equipment in a folded configuration, according to an embodiment;

Figure 7 shows the spraying equipment of Figure 6 in a deployed configuration,

Figure 8a shows a schematic lateral view of the spray bar with the hi gh- and low-resolution nozzle arrays, when the camera module is oriented to maximize the speed of the spraying equipment, and

Figure 8b shows a similar view when the camera module is oriented to maximize the spray precision.

Detailed description of possible embodiment of the invention

[0030] With reference to Figures 1 and 2, the agricultural system 100 comprises an agricultural spraying equipment 200 that may comprise a first and a second spray bar 210, 215 extending substantially perpendicularly to the travel direction of the agricultural system 100 when operating. The first spray bar 210 comprises a first array of nozzles 220 separated from each other by a first distance while the second spray bar 215 comprises a second array of nozzles 240 separated from each other by a second distance smaller than said first distance. As a result, the spatial resolution of spot sprays 242 that may be sprayed by the second array of nozzles 240 is higher than the spatial resolution of spot sprays 222 that may be sprayed by the first array of nozzles 220 when the first and second spray bars 210, 215 are at the same height. Accordingly, the first and second arrays of nozzles 220, 240 may advantageously perform respectively low and high-resolution spot spraying 222, 242.

[0031] Referring to Figure 3, the agricultural spraying equipment 200 also comprises a first spray bar height control system 230 comprising at least one height actuator 232 (Figure 1), and a second spray bar height control system 250 comprising at least one height actuator 252 so as to control independently the height of the first and second spray bars 210, 215 with reference to the ground 12. As shown in Figure 1, the height from the ground 12 of the first spray bar 210 may be controlled for example by means of height and roll actuators. At the rear of the first spray bar 210 are mounted height actuators 252 supporting the second spray bar 215 with the high-resolution nozzle array 240 so as to allow a different distance between the nozzles of this array and the ground on one side, and between the low- resolution nozzle array and the ground on the other side.

[0032] With reference to Figure 2, the agricultural spaying equipment further comprises a camera system 270 with one or more camera modules 300. Each camera module 300 comprises at least one a camera 310 (Figure 4) arranged to capture images of objects, for example plants 14 and/or part of plants 16, and of the ground 12, ahead of the first and second spray bars 210, 215 in the travel direction of the agricultural system 100. Advantageously, each camera module 300 may comprise other cameras, sensors and light sources as described subsequently with reference to Figure 4 to acquire images of high quality independently of the light conditions. Each camera module 300 is mounted at the end of a camera mast 600 and looking forward as shown in Figure 2.

[0033] Referring to Figure 3, the agricultural spraying equipment comprises a processing unit 274 configured to run an image recognition software to identify the objects on the acquired images and to generate a depth map of these objects on the 3D coordinate system of the spray bars. The spraying equipment further comprises one or more object tracking units 276 and a first and a second nozzles array control unit 280, 282. The tracking unit or each tracking unit 276 is configured to continuously track the position of these objects on the depth map. The first and second nozzles array control units 280, 282 are configured to control each nozzle of respective first and second arrays of nozzles 220, 240 as to selectively control the nozzles of the first and second arrays of nozzles 220, 240 to perform low and/or high- resolution spot spraying 222, 242 on different objects as a function of the position of these objects on the depth map with respect to the position of the first and second spray bars 210, 215.

[0034] The first spray bar height control system 230 is configured to receive height information from a 3D depth sensor 314 of the camera modules 300 and from other distance measurement sensors placed along the first spray bar 210 to measure its distance from the ground 12, and to compute orders for the spray bar height and roll actuators 232 in order to maintain the low resolution nozzle array at a constant distance 234 from ground. The high- resolution nozzle array height control system 250 performs the same function but on the high-resolution nozzle array to maintain a constant distance 254 with ground. The sensor processing unit 274 is configured to perform object detection and to transmit their coordinates to the object tracking unit 276 which tracks their relative displacement until they reach the nozzles. The first and second nozzles array control units 280, 282 transform a plant presence passing under the nozzles into opening and closing orders for the nozzles.

[0035] With reference to Figure 4, data of the 3D depth sensor 314 of the camera module 300 is sent to the processing unit which generates the depth map of the ground and its objects. The camera module 300 may comprise a second camera 312 with a second set of spectral bands 312, for instance in the infrared. In another embodiment, two cameras are used to simultaneously acquire the same images from two distinct positions and to calculate the depth map using a stereovision algorithm 316.

[0036] In an advantageous embodiment, the camera module 300 further comprises irradiation elements 330, 332, 334, whose respective spectral emission band covers at least all the bands of the cameras and 3D optical depth sensor. These irradiation elements provide very powerful light levels to allow the cameras to acquire images with a short exposure time to avoid image blur when the agricultural system 100 travels at high speed. In order to save power consumption, the irradiation elements are configured to emit only during the exposure times of the cameras. The rest of the time, they are inactive.

[0037] The advantages of the 3D depth sensor 314 on the spray precision are illustrated in Figures 5a and 5b. With a standard camera 310, in Figure 5a, the difference of vertical distance from camera to a supposed object plane 326 and camera to the real objects 12, 14, 16 results in wrongly mapped objects in longitudinal 322, lateral and vertical 324 directions. These errors degrade the object mapping and the visual odometry when this latter is used. To avoid these projection errors, using a 3D range sensor 314, as shown in Figure 5a, whose optical axis is located as much as possible near the camera optical axis, allows to obtain the exact distance 320 between a point in the image and the camera. Knowing this distance allows to compute the correct longitudinal, lateral and vertical object mapping from the camera image.

[0038] The best spot spray accuracy is obtained with short nozzle to ground distance. Keeping constant and short distance from nozzle to ground on a spray bar of several tenths of meter is challenging. One embodiment of the invention is to decompose the high-resolution nozzle array 240 into smaller segments 260, each one being mounted on individual height actuators 262 to allow fast regulation capability. The corresponding system is shown in Figure 6, where an uneven terrain 12 is sprayed with high spatial resolution spot sprays 242 and the height of each segment 260 with ground is independently adapted to obtain as constant spray distance as possible.

[0039] The same figure represents another embodiment of the invention, which is to include two liquid distribution systems with their own mixture and pressure. The first supplies the low-resolution nozzle array, which can be either operate in full application mode or in low resolution spot spray mode. The second supplies the high-resolution nozzle array.

[0040] Another aspect of the invention is to provide a higher throughput in full application mode with the same spraying equipment. The proposed innovation is to add extension segments at the extremities of the spray bar, as shown in Figure 7, supporting the low-resolution nozzle array only, and not being equipped with camera modules. By unfolding these segments, the spraying equipment can perform full application with an improved throughput.

[0041] Another aspect of the invention is to provide varying tilt angles of the cameras depending on the desired speed of and accuracy of the chemical application. More particularly, the camera modules 300 are placed on the extremity of a corresponding mast 600 as illustrated in Figures 8a, 8b. Each mast 600 is mounted on a motorized rotation system 620 to pivot about an axis extending along the first spray bar 210.

[0042] When the mast is rotated to extend along a vertical direction (not shown), the first spray bar can be folded by segments without issue regarding collision of the masts with other parts. Figure 8a shows a first scenario where the tilt angle of the mast 600 is set such that the field of view 640 of the camera of the camera module 300 is directed to a first position at a first distance 642 to the first spray bar 210 in forward direction of the agricultural system resulting in an important distance between the rear side of the field of view and the nozzles. This case is desired when fast operation is needed, because this extended distance allows more time for the image processing, which is very often the bottleneck of such systems in terms of speed.

[0043] By setting the tilt angle of the mast as shown in Figure 8b, the field of view 630 is directed to a second position at a second distance 632 to the first spray bar 210 shorter than the first distance 642. The optical axis of the camera being almost vertical in this configuration, the quality of object mapping is better with an increased precision, at the cost of a reduced possible operation speed of the agricultural system due to a shorter distance between the rear side of field of view and the nozzles.

[0044] Another aspect of the invention relates to a method of operating the agricultural system 100 using visual odometry to detect and track the tridimensional movement of objects in the tri-dimensional coordinate system of the first and second array of nozzles 220, 240 such that the first and second nozzles array control units 280, 282 timely control the first and second array of nozzles 220, 240 according to the position of these objects in the above 3D coordinate system.

[0045] The visual odometry according to this method comprises the steps of: i. capturing with the camera 310 a first image of the ground 12 and the objects on the ground; ii. simultaneously capturing with a 3D depth sensor 314 a depth map of the ground and the objects captured in the first image; iii. merging the first image with the depth map to obtain a 3D image; iv. extracting on the 3D image a set of features, such as contour of an object, for features tracking, and v. repeating steps i to iv and tracking the movement of the extracted features between two consecutive 3D images as the agricultural system 100 moves along its travel direction to compute the movement of the extracted features in the 3D coordinate system of the first and second arrays of nozzles 220, 240.

[0046] Another aspect of the invention relates to a method of operating the agricultural system 100 of Figure 1 to perform different spraying operations on the cultivated field 10. The first and the second array of nozzles 220, 240 are supplied either by a same liquid distribution system, or by their respective liquid distribution systems 510, 520 as shown in Figure 6. The first and second control units 280, 282 are operated to control each nozzle of respective first and second array of nozzles 220, 240 to perform in the cultivated field any of the following operations: a. performing low-resolution and high- resolution spot sprays 222, 242 simultaneously; b. performing low-resolution spot spray 222 while high-resolution spot sprays 242 is not performed; c. performing high-resolution spot spray 242 while the low-resolution spot sprays 242 is not performed; d. performing a continuous and homogeneous spray application with the first array of nozzles 220 while the second array of nozzles 240 is not used; e. performing a continuous and homogeneous spray application with the second array of nozzles 240 while the first array of nozzles 220 is not used; f. performing a continuous and homogeneous spray application with the first array of nozzles 220 while the second array of nozzles 240 performs high-resolution spot spray 242; g. performing a continuous and homogeneous spray application with the second array of nozzles 240 while the first array of nozzles 220 performs low-resolution spot spray 222, and h. performing a continuous and homogeneous spray application with the first and second arrays of nozzles 220, 240.