Technical Field

The invention belongs to the field of measurement and the application of artificial intelligence in agriculture. The designation according to the International Patent Classification (IPC) is: G06Q50/02, G06Q10, G01C21/32, A01B79/005 H AO 1C21/005.

Background Art

The invention solves the problem of defining optimal soil sampling points for a robotic platform on a given plot of land on which the mentioned measurements take place.

The invention solves this problem of defining optimal sampling points by applying artificial intelligence algorithms located on the server platform of the system and controlling the hardware system or robotic platform to efficiently sample the land (i.e. to make the soil sampling location to be the best representatives of a certain part of the plot of land).

The invention solves the problem of the current movement of robotic platforms for soil sampling in a way that increases the efficiency of sampling with intelligent selection of sampling locations.

The current state of the art includes a scientific paper entitled Optical Sensing of Nitrogen, Phosphorus and Potassium: A Spectrophotometrical Approach Toward Smart Nutrient Deployment , which differs from the proposed invention by a different principle of measurement, namely, the current invention uses ion-selective electrodes while in the paper it is explained the approach for measurement through the optical principle of detection.

Also to the current state of the art belongs a paper entitled Task-based agricultural mobile robots in arable farming: A review that uses the optical principle of detection, not ion- selective electrodes, and does not mention an intelligent algorithm for selecting plot sampling points.

In addition to the above papers, the state of the art includes the following protected solutions: patent application US20160232621 entitled "Methods and systems for recommending agricultural activities" published on August 11, 2016, which also discusses the algorithms of recommendation, with the invention talking about the algorithm of recommendation optimal soil sampling sites. Also, this patent application does not specify a physical system for automated sampling and analysis of soil. However, the essential difference is reflected in the fact that in the present invention, artificial intelligence algorithms are used to select suitable sampling and analysis points, while in the above application, the results of the soil sample input are used to determine the zones. Also, there is no part in the application about pixel enlargement and cluster mapping in other resolutions, which contribute to finding meaningful zone boundaries from the point of view of further use. The application states that the zones are made from one or more sources (soil analysis, vegetation indices, yield maps, elevation, etc.).

Patent application US20160078375 entitled "Methods and systems for recommending agricultural activities" published on March 17, 2016, as well as applications US20160073573 and US20160078375 also belong to the prior art and are almost the same differences compared to application above US20160232621 compared to the present invention.

Patent application EP3529556A1 entitled “Land mapping and guidance system” published on 28.08.2019. belongs to the state of the art. The difference in relation to the proposed invention is reflected in the fact that the current invention uses an intelligent algorithm to determine the trajectory of the robot, while in the reported system moves through the selected area in the form of a meandering path covering the entire plot area and maps the parameters of interest, without the use of more advanced algorithms and artificial intelligence. The application sees anomalies in the field, slope data, but as a system output there is no fertilization recommendation.

Patent EP 1754405 B1 entitled “Mobile station in combination with an unmanned vehicle” issued on 5 November 2008 belongs to the prior art, but no phosphorus and potassium analysis, acidity analysis, electrical conductivity analysis and no intelligent sampling are performed. These are the basic differences with respect to the present invention.

Patent application US20140379228 A1 entitled “Method and system for optimizing planting operations” published on 25.12.2014. talks about a seeder that does not measure electrical conductivity, it also samples everywhere without the use of artificial intelligence algorithms. Fertilization is done in real time based on the results of the analysis and according to the description in the application; it is done on the fly. In the case of the invention, this is not the case because it describes an intelligent system in the sense that it includes algorithms for path recommendation and fertilizer recommendation that are located on the server and include artificial intelligence in agriculture.

Disclosure of the Invention

The growth of the world population increases the demand for food and arable land needed for its production. Since arable land resources are limited and very often degraded due to poor management, there is a need to improve agro-technical measures, to increase the yield in an environmentally sustainable way. Classical methods for soil analysis include taking on average 15-20, soil samples from a plot of 5 ha area from the depth of 30 cm. Next, the mixed sample is prepared and sent to a laboratory for analysis. The results of the analysis are obtained after 10 to 14 days. It should be noted that classical methods are physically demanding since the soil is sampled with hand tools, they are also time inefficient, and they are not preserving the information about the exact location of individual samples. For these reasons, the results of the analysis obtained deviate from reality, especially in the case of one of the most important nutrients in the soil, such as nitrogen since it is extremely temporally and spatially variable. For the reasons stated, there is a need for a new system and method that processes the results in a short time and gives a real picture in time and space, based on which an adequate fertilizer in optimal quantities can be applied where is needed, and thus a better yield with less investment is achieved.

Sowing and fertilization, as stages of production of a plant species on a certain plot, should be done after the analysis of soil components with adequate methods and systems for assessing soil quality and the presence of nutrients that the plant uses during its growth and development. So far, farmers have applied uniform amounts of fertilizer throughout the plot, which is suboptimal from the point of view of the economy, and unsustainable in the ecological sense. Namely, plants take nutrients from the soil in the quantities they need for proper growth and development, while the rest either evaporates creating greenhouse gases or reaches the groundwater and surface water eventually polluting them. In this way the reproduction of algae is encouraged, leading to a lack of oxygen and thus fish kills in aquatic habitats. These pollutants also affect people. For example, exceeding the allowed concentration of nitrates in water leads to problems in the development of children.

The present invention represents a new system and method for adequate sampling of soil nutrients in a given field (nitrogen primarily, but also other nutrients in the soil). The present invention is characterized by the relatively short time needed to get analysis results since the procedure of soil sampling, sample analysis, and sending the results to the server takes 15-20 minutes per sample. The significance of this solution is reflected in the rational and correct selection of parts of the plot for sampling to determine the percentage of nutrients that need to be compensated. The concentration of inorganic nitrogen, phosphorus, potassium, calcium, magnesium is analyzed in the soil sample. Furthermore, parameters such as but not limited to pH and soil moisture, organic carbon, soil particle size, iron oxide concentration, the composition of minerals, and dissolved salts could be determined as well.

The basic idea of the present invention is the accuracy and quick soil sample analysis through a new method and system of software components that improve the hardware part of the system in the field to adequately create a system of points and an adequate route for soil sampling on a given part of the plot. A soil sampling map of a part of a plot or a whole plot (less often) is a map by which a robotic system moves (route) using the intelligence of the algorithms of the present invention. The selection of sampling points, which are prepared for the robotic platform to move intelligently around the plot, involves a K-mean algorithm and depth of resolution while preserving the information of all pixels up to a certain threshold (the threshold is set to the level of fertilizer spreaders). The invention proposes and implements an innovative way of moving a robotic platform for sampling, i.e., an innovative way of soil sampling for the analysis of nutrient concentration.

The invention is a system of modules that are combined with a robotic platform to make a new system of hardware-software components for a new method of soil sampling using an intelligent selection of points (georeferenced coordinates) that determine the robotic platform's trajectory to take a sample as quickly and accurately as possible, to get the insight of deficiency of the nutrients necessary for normal plant growth and development in the soil.

Brief Description of the Drawings

Figure 1 shows the flow-chart of processes for fabrication of the photovoltaic infrared detector.

Figure 2 shows a deposited P-type semiconductor film composed of compound IV- VI, side view.

Figure 3 shows the temperature on the exposed surface of the film during irradiation of the film with a pulsed radiation source, results of the model.

Figure 4 shows the temperature gradient along the cross section of the film created during irradiation if the exposed surface of the film with a pulsed radiation source, results of the model.

Figure 5 shows a formed photovoltaic infrared detector with a P-N homojunction, side view.

Figure 6 shows schematically formation of P-N homojunction irradiated with a defocused laser beam.

Figure 7 shows the top view of the photovoltaic infrared detector with P-N homojunction. Figure 8 shows the current-voltage curves of a photovoltaic detector in the dark and illuminated by infrared radiation.

Best Mode for Carrying Out of the Invention

There is a need to develop new technologies in agriculture to use the arable land optimally, thus provide adequate amounts of food for the growing world population. In addition to available water, the most important parameters for optimal growth and development of plants, and consequently the yield achieved, are nutrients, where nitrogen plays the most important role. The challenge of monitoring the concentration of nitrogen in the soil is not an easy task since the concentration of nitrogen in the soil varies in time and space, and this variability adversely affects the optimal fertilization, i.e. maximum yield. Precise determination of nitrogen concentration in the soil at a given location is a prerequisite for precise application of fertilizers, adequate savings, and minimal negative impact on the ecological system.

The harmfulness of a larger amount of nitrogen in the soil is reflected in the reduction of oxygen in the waters, which causes the deterioration of fish stocks and reduces the biodiversity of animal species. Furthermore, waters that contain nitrates are no longer suitable for drinking. One part of the nitrogen evaporates and reaches the atmosphere, increasing the negative greenhouse effects. The third negative effect of excess nitrogen is an increase in soil acidity. The present invention significantly reduces the negative effects of increased fertilization on the environment by optimal sampling and subsequent fertilization.

The present invention includes a reliable and fast sampling, measurement, and mapping of nitrates in soil enabled by the system and operation of an autonomous robotic platform that samples soil, prepares a sample by mixing with deionized water, and finally, using a sensor module, determines the amount of nitrate in the prepared soil sample

The present invention enables precise mapping of the amount of nitrate-nitrogen available in soil with a high spatial and temporal resolution, and through a new method and system of algorithms for selecting sampling points.

The system consists of an automated robotic platform on which other functional parts of the system are placed. In the specific case, the platform of the manufacturer Clearpath Robotics Inc. was used, but the invention is not conditioned by the choice of platform, and it can be applied to any other adequate platform, as well as to a tractor or any other agricultural machinery. The robotic system can also navigate itself with the help of GPS, where the presence of a technical person is not necessary, but in most countries, it is mandatory due to the valid legal regulations regarding autonomous vehicles.

In Figure 1 described innovative system for intelligent sampling is presented. The system is composed of robotic system 100, server 111, and input-output module 114.

Robotic system 100 includes robotic platform 101 comprising three basic modules: module 102 for soil sampling, module 105 for the sample preparation, and module 108 for soil sampling analysis.

On server 111 a module 113 for localization is placed, which represents the basic innovative component of the present invention. Furthermore, the control module 112 is placed on the server with the coordination-management role since it communicates with the server part and the robotic platform 101.

The soil sampling module 102 is connected to the anchoring module 103 which allows the robotic platform 101 to be anchored in the field. Successful anchoring allows the robotic platform 101 not to rise when probe 104 enters the ground, as well as to reduce the load on the wheels when probe 104 is pulled out of the ground. The information about successful anchoring is sent to server 111 by the communication module 110 via robotic platform 101. Module 102 for soil sampling is further connected to the soil sampling probe 104. Successful sampling information by soil sampling probe 104 is sent by module 102 to server 111. Information from robotic platform 101 to server 111 is sent via the communication module 110. All the activities are monitored and coordinated by the control module 112 which is also located on server 111.

The sample preparation module 105 prepares the obtained soil sample received from sampling unit 102 by mixing it with the water from the water tank 107 in the container 106 to which it is connected. The amount of added water is determined based on the measurement of the weight of the soil sample. This functionality is integrated into module 105 for sample preparation via force sensors or the like. The sample preparation module 105 has a funnel on top of it, which is placed below the sampling module 102 and from that funnel, the soil sample is directed to a container 106 (mixer) in which it is mixed with deionized water coming from the water tank 107. Deionized water is used since it does not contain any ions that would affect the measurement results of sensor 109, and it is necessary for the proper functioning of sensor 109 which needs an aqueous soil sample solution.

The sample analysis module 108 receives a soil sample from the sample preparation module 105 and uses the sensor 109 to detect the presence of certain ions of nitrate, nitrogen, phosphorus, potassium, calcium, carbon, magnesium, iron, etc., then measures the electrical conductivity and acidity of the soil, moisture, particle size, etc. It is important to note that within the sample analysis module 108, there are also appropriate calibration standards used to calibrate the sensor 109 before the measurement series, thus achieving measurement accuracy and repeatability.

The sensor module 109 uses several probes to characterize soil samples:

• ion-selective electrodes (produced by Clean Growth) for the following ions: Ca2+, C1-, K+, Na+, NH4+, N03-, Mg2+ and P[HP042-],

• probes for measuring electrical conductivity and acidity

It is important to note that the present invention, besides stated parameters could also measure soil moisture content, conduct hyperspectral analysis based on which following parameters can be estimated: moisture, organic matter (organic carbon), particle size, iron oxide concentration, mineral content, dissolved salts, heavy metals and the like.

Data on the concentrations of nitrates and other listed substances and soil characteristics are sent by the sample analysis module 108 to the robotic platform 101 and the information is forwarded via the communication module 110 to server 111 and the control module 112. The control module 112 monitors and coordinates communication efficiency.

The robotic platform itself 101 includes a global positioning system (GPS) of high accuracy and light detection and ranging (LIDAR) system enabling obstacle detection and avoidance. The system 100 of hardware-software components for soil sampling represents a digital platform, which generates optimal sampling locations in the form of coordinate points. The list of location is based on high-resolution satellite or drone images. The main innovation of the present invention is reflected in the intelligent selection of the sampling points by using artificial intelligence algorithms.

As stated above, the system includes an application (user software), input-output parameters module 114, that communicates with module 113 for robot localization. It is already explained that module 113 for localization is placed on server 111 and represents the basic innovation of the present invention since it determines soil sampling locations.

When the farmer sets the coordinates of the parcel via module 114, this data is sent to the localization module 113 which communicates with the control module 112 on further sampling activities with the robotic platform 101. The control module 112 is a module on server 111 which is a control software component. The control module 112 receives sampling points from module 113 and module 112 can change the path due to natural obstacles and the like. The control module 112 on this occasion determines the starting point to which the robotic platform 101 returns after performing sampling at the last point. When the control module 112 confirms the route, it is sent via server 111 with which module 112 communicates to the robotic system 100 which performs the task of soil sampling.

During the task execution, module 112 can monitor the status of platform 101, and have the insight whether it has successfully arrived at the location, sampled, analyzed the land, etc. and this status is refreshed every 5 minutes, or at the request of the control module 112. These statuses are obtained by the robotic platform 101 from the sampling module 102, the preparation module 105, and the analysis module 108. When the robotic system 100 finishes with the last point, a fertilization recommendation can be made in coordination with server 111.

On server 111 there is module 113 of artificial intelligence algorithms, which represents the innovation of the invention in the sense that it provides an intelligent selection of points at which the soil is sampled.

Artificial intelligence is involved in the operation of robotic system 100 of the invention, through the module 113 which estimates GPS locations of sampling points within the field for the analysis of nitrates and other nutrients in the soil. The invention provides solution in the form of the robotic system 100 for the optimal soil sampling on the observed field parcel. The optimal soil sampling implies utilization of advance algorithms of artificial intelligence, implemented within module 113, which determine number of soil samples and their locations within the parcel. Algorithms take as input multispectral images of the observed parcel, obtained from satellite or high-resolution cameras mounted on unmanned aerial vehicles (UAV). From the calculated vegetation indices and their spatial distribution from multispectral images, algorithms divide parcel into zones with different statistics of vegetation indices and generate locations for soil sampling within each zone, where the robotic system 100 takes samples autonomously. In figures below, which present results of zone creation and sampling points generation, a multispectral image from drone is used as an input data for the algorithms.

Robotic platform 101 can be any platform which is capable for autonomous moving within the parcel. GPS coordinates, generated from module 113, for the localization of robot within the parcel are given to robotic platform 101 from the module 112 which is on server 111. Stability of the robotic platform depends on module 103. In addition to the analysis of the presence of nitrogen, potassium, sodium, can be analyzed the presence of calcium, chlorine, nitrates, phosphorus, magnesium or can be measured electrical conductivity of soil, its acidity, humidity, organic carbon, soil particle size, concentration of iron oxide or dissolved salts, or can be determined mineral composition, etc.

After the optimal locations for soil sampling are determined by algorithms, the robotic platform 101 moves across the shortest trajectory within the parcel towards these locations. There are also points on the trajectory, which robotic platform 101 needs to traverse, but there the soil sampling is omitted. Within the invention there is a check on these points of trajectory, where the robotic system 100 comes, will the soil sampling with analysis be conducted or omitted, and this role is dedicated to the control module 112 on server 111. All mentioned steps contain positive or negative outcomes about which robotic platform 101 regularly reports input-output module 114.

In Figure 2 is shown a method of the invention, which consists of: step 200 which consists of retrieving the GPS coordinates of the locations for soil sampling from server, then step 201 where the moving of the robotic platform 101 to the coordinates of points and soils sampling, step 202 preparing for soil sample analysis, step 203 of sample analysis and finally step 204 in which data from the analysis are sent to the server.

Figure 3 presents the innovation of step 200 and illustrates it in more detail through the following steps: step 300 where the boundary of parcel is defined by the user 301 for which soil sampling needs to be done then step 302 for determining the region of interest (ROI) for extracting pixels in the image (satellite or drone image) belonging to parcel boundary from step 301 and step 303 which defines mask for the parcel region from ROI.

Figure 4a illustrates parcel obtained from step 301, and Figure 4b illustrates a mask from step 302 which defines a region of interest. For the extracted pixels from parcel 301, vegetation indices are calculated that can be determined based on available spectral channels. Vegetation indices are used to estimate vegetation, green cover, chlorophyll percentage, etc. The following vegetation indices are used: NDVI-Normalized Difference Vegetation Index, TNDVI-Transformed Normalized Difference Vegetation Index, GNDVI-Green Normalized Difference Vegetation Index, ExG- Excess Green, CIVE-Color Index of Vegetation, TGI- Triangular Greenness Index, GLI- Green leaf index, SAVI-Soil Adjustment Vegetation Index and MSAVI-modified SAVI. NDVI index is a vegetation index of global vegetation analysis, and it is important for the assessment of seasonal and perennial vegetation. Its values are from -1 to 1, usually from 0.3-0.8, and values between 0.2 and 0.3 represent grass areas. This index is negatively affected by soil color and humidity, atmospheric conditions, and dead matter in the plant. It is important, but other indices are also used because they provide better information depending on atmospheric adaptation. The TNDVI-Transformed Normalized Difference Vegetation Index is an index whose value varies from zero to one and this index represents the square root of the NDVI. For example if the value is greater than 0.4 then we have presence of green vegetation. This is followed by the GNDVI-Green Normalized Difference Vegetation Index, which is sensitive to small amounts of chlorophyll and uses the wavelength of the green band, especially the lower ones of 550 micrometers. ExG and CIVE indices are used to estimate the region of interest in the picture where the vegetation cover is located (separation of vegetation from soil in the image). TGI and GLI indices serve as indicators of the presence of chlorophyll in the leaves. ExG, CIVE TGI and GLI are vegetation indices based on responses from the visible part of the spectrum and are used when there is no information from the infrared part of the spectrum. And finally, we have SAVI and MSAVI indexes, where MSAVI is modified SAVI index. SAVI in its formula contains the factor of adjusting the background of the leaves and considers the brightness of the soil. Its values are in range from -1 to 1, and a lower value reflects less vegetation while a higher value reflects larger vegetation. MSAVI has no leaf background adjustment factor.

In the next step 306, the separated part of parcel 302 from the parcel boundary set in 301, is considered as a matrix of pixels in 305 with dimensions axb, and with the third dimension which reflects the values of vegetation indices. That is, the matrix 305 is fulfilled with the values of vegetation indices 304 for each pixel within the selected part of parcel 302, and its rows correspond to pixels while columns represent vegetation indices.

The pixels from the matrix 305 are standardized in step 308 according to the values of vegetation indices, so a new matrix 307 is obtained. The standardization of the matrix results in discarding the rows of the matrix which correspond to the pixels which cover the land without vegetation. The prepared data matrix 307 is further analyzed in step 309 with the K-means clustering method, which result is further presented in different spatial resolutions (from 1 pixel width to the width corresponding to the width of fertilizer spreader). Each pixel from the ROI via the K-mean algorithm should be associated with one of the K zones. This is done with transformation of matrix 307 in K binaiy matrices with zeros and ones where the units indicate the affiliation of the pixels to the cluster. This is done in step 309 and one of K binary matrices is represent as a matrix 310. Clustering in different spatial resolutions in step 309 yields maps of labels 311, 312, 313, i.e., new entities (matrices) of matrix 310 depending on the change in spatial resolution. Diagonal 314 reflects this change in resolution. After association of pixels within the ROI with the one of the K zones (e.g., for k = 2 we have that the pixel 315 is associated with one of the two clusters because it is labeled with 1, i.e., the cluster to which it should be, and not with 0 which is the label of the other cluster) a spatial statistic of labels is calculated. Then based on calculated spatial statistics of pixels with values zero and one in matrix 310 with different spatial resolutions and maps of labels 311, 312, 313, for each pixel from the ROI, the probability with which it belongs to one of the K zones is obtained.

Label maps 311, 312 and 313, i.e., the new entities obtained from matrix 310, are further processed by calculating the number of label occurrences for the zones. In the example below, the analysis was done for two zones (two clusters), where one zone is labeled with 0 and the other with 1, and without pixels covering the soil without vegetation. With the change of spatial resolutions reflected by diagonal shift 314, each pixel 315 is linked to new values which represent the probability of belonging, e.g. 311, 312 and 313 are examples of different resolutions corresponding to the occurrence of numbers 0 and 1 given in columns 319 and 320 following resolutions 311, 312 and 313 or windows of different spatial resolution. It is the pixel 315 who draws the highest probability of belonging to one region, while pixel 316 has the lowest . Based on the frequency of occurrence of 0 and 1 in columns 319 and 320, the final decision is made for the observed pixel 315 whether it belongs to zone 0 or 1 (0 if the cumulative sum of S column 320 is greater than the cumulative sum of S column 319 for the observed resolution 318 and vice versa. The resolution in column 318 goes from 1 to n, which is the threshold or width of the spray while S is the sum of ones and zero related to columns 319 and 320 and is also located at the end of column 318. Spatial resolutions are changing until the threshold for spatial resolution is reached, which is the level of spray coverage n. The frequencies of label occurrence, depending on the spatial resolution, are stored in a new table which in the first column 318 contains the number of spatial resolutions (from 1 to n are the rows of the table, where n is the level of spray coverage, and S is the sum of all zeros or ones). Then in the next two columns of the table are stored the number of zeros and ones for a observed label for a given matrices 311, 312, 313 of matrix 310.

In step 317, consensus is reached. The final zones and the estimation of the probability of pixel belonging to the zones in the ROI are determined based on local histograms of labels (new matrix entities) 311, 312 and 313 from clustering obtained using windows of different sizes (spatial resolutions). Based on the cumulative sum of the appearance of the label, given in columns 319 and 320, through different resolutions, the final decision is made, i.e., map of pixel belonging to the zones (Figure 5). Variability by spatial resolutions and their further consensus led to the creation of spatial cluster boundaries. Pixel 315 based on the cumulative sum of occurrence of ones through different spatial resolutions (sum of S elements of column 319 to given resolution 318) most likely belongs to that cluster labeled with 1, in relation to pixel 316 which will most likely belong to another cluster. In this way, a further consensus takes place, which assigns the most probable label to the observed pixel based on spatial statistics for different spatial resolutions. An additional constraint of sprayer width adjusts the algorithm for the specific application of soil sampling location determination. Considering the width of the atomizer, the obtained label map (Figure 5b) is further analyzed in the same way as the result of clustering primarily, but with only one spatial resolution step equal to the width of the atomizer.

Figure 5a illustrates the result of clustering by the method of K-mean values for K = 2, while Figure 5b illustrates the final classification of pixels into two zones based on the frequencies of label appearance through different spatial resolutions.

In step 317, based on the map of the decision of pixel belonging to the zones and the entered capture of the sprayer, the rasterized boundaries of the zones are calculated, which are adjusted to the movement of the sprayer through the parcel. The final rasterized boundaries are obtained based on the frequency of labels from the decision map in the square region of the dimension equal to the width of the atomizer obtained by the same procedure as for the decision maps.

Figure 6a illustrates the obtained rasterized region boundaries based on the zone decision map.

Figure 6b illustrates a final rasterized map of zones obtained from label statistics calculated over window widths dictated by atomizer width.

In addition, in step 317, sampling points within the zones are generated based on the zone boundaries in such a way that they do not fall on the boundaries of the zone decision map and the rasterized zone map boundaries and optionally not close to the parcel boundaries (user distance). In their environment they have more than 95% of the pixels from the same zone. Figure 7 illustrates the final map of soil sampling points. The obtained sampling points can be modified by the user if necessary (their number or position can be changed).

Industrial Applicability

The invention finds application in smart systems for agriculture.