FIELD OF INVENTION:

The present invention relates to the field of artificial neural networks for determining one or more agri-measures and particularly, to the system and method for obtaining and recommending agri measures at the crop field based on crop related data, field-condition data and environmental data.

BACKGROUND:

Agricultural farms are more susceptible to crop disease, pest and untimely weather conditions which has a serious effect on the yield and quality of the crop. Therefore, the agricultural production requires significant strategy, analysis and continuous monitoring. Most often, agricultural producers or growers (e.g. farmers or others involved in agricultural cultivation) require analysis of certain data in order to make strategic decisions prior to the period of crop cultivation (i.e., growing season). For making such strategic decisions in advance, crop producers and growers need to consider at least some of the following decision constraints like resource costs, historical and projected weather trends, soil conditions, projected risks posed by pests, disease and weather events, and projected market values of agricultural commodities (i.e., crops). Timely determination of such agri-measures help a crop producer or grower to predict and evaluate key agricultural outcomes such as crop yield, energy usage, cost and resource utilization, and farm profitability and many more.

Such analysis and strategy is difficult to accomplish because of numerous technical challenges. Firstly, obtaining reliable information for various considerations of the crop grower is often difficult. Secondly, extracting such information and aggregating it into a usable manner is a time consuming task. Thirdly, the extracted data is not precise enough to be useful to determine strategy. For example, weather data (historical or projected) is often generalized for a large region such as a county or a state. In reality, weather may vary significantly at a much more granular level, such as an individual field. In addition, terrain features may cause weather data to vary significantly in smaller regions. Moreover, crop growers often need to take regular decisions such as when to harvest, provide supplemental fertilizer, and how to mitigate risks posed by pests, disease and weather. Consequently, there is a need to continually monitor various aspects of their crops during the growing season including weather, soil, and field conditions.

Nowadays along with the rise of artificial neural networks which allow a system to take intelligent decision itself, such as systems for detecting crop disease and taking smart decision with respect to other factors affecting the crop yields are in demand.

Chinese applications with publication number CN 107392091 A discloses artificial- intelligence based agricultural crop detection method comprising of obtaining an image sample and constructing a crop disease database; obtaining a neural network model to predict disease prediction result; selecting a disease recognition network for executing detection according to the to-be-detected crop variety and outputting the disease prediction result.

Another Chinese application CN110472784A discloses a system based on real-time diagnosis and real-time processing of diseases and insect pests by real-time monitoring of crop diseases and insect pests, using convolutional neural network algorithm through artificial intelligence combined with disease and insect pest knowledge base diagnosis. However, they do not take into consideration soil data, weather data, etc. in order to evaluate the key agricultural outcome like crop yield and farm profitability. Certain prior arts are solely focused on soil nutrient analysis, but fails to provide accurate monitoring of all such (soil data, weather data and crop data) aspects at a granular level, which is difficult and time consuming.

Accordingly, an intelligent and reliable system for providing continuous and regular monitoring of farm crops to determine the agri-measure based on the crop data, soil data and weather data with precision at granular level is desirable.

The embodiments of the present disclosure provide system and method for determining agri measures by providing prediction results in the form of crop health indices and strategic recommendations related to weather, finance, insurance, etc. in order to maximize crop production. Adoption of such a system and method can further give the confidence index reference for the various external entities like insurance agencies, markets, policy makers and hanks. SUMMARY OF THE INVENTION

The subject matter described herein relates generally to determining one or more agri-measures by using Neural Network based prediction model which can be trained using deep learning systems. In one aspect, an Agricultural Intelligence System based on multi-modal neural network (NN) prediction model with deep learning for determining agri-measures is provided. The Agricultural Intelligence System includes a user device, a plurality of database systems, a prediction model comprising one or more processors and a memory in communication with the processors. The processor is configured to receive one or more inputs in the form of crop related data, retrieve one or more input data from at least one of the databases, retrieve environmental data and a field condition data, wherein the field condition data comprises of field related data, a packages of practice and a soil data, identifying a subset of the plurality of input data associated with the packages of practices, determine a plurality of agri-measures based on the subset of the plurality of input data in the form of indexes for each of the plurality of crop activity options based at least in part on the plurality of field condition data, and provide the predicted results and recommendations based on plurality of agri-measures. The prediction model consists of Exploratory Data Analysis (EDA) module, a learning module and a prediction module which is characterized by the multi modal neural network (NN) based prediction model which undergoes continuous learning by training with the deep learning based layered architecture consisting of various hidden layers forming complex network that learns with a feedback mechanism.

In another aspect, method for determining agri-measures is provided. The method is implemented by an Agriculture Intelligence system. The method includes receiving one or more inputs in a form of crop related data, retrieving a one or more input data from at least one of the databases, retrieving an environmental data and a field condition data, wherein the field condition data comprises of field related data, a packages of practice and a soil data, identifying a subset of the plurality of input data associated with the packages of practices, determining a plurality of agri-measures based on the subset of the plurality of input data in a form of indexes for each of the plurality of crop activity options based at least in part on the plurality of field condition data, and providing the predicted results and recommendations based on plurality of agri-measures.

In a further aspect, computer-readable storage media for recommending agri-measures is provided. The computer-readable storage media has computer-executable instructions embodied thereon. When executed by at least one processor, the computer-executable instructions cause a processor to receive one or more inputs in the form of crop related data, retrieve one or more input data from a at least one of the databases, retrieve an environmental data and a field condition data, wherein the field condition data comprises of field related data, a packages of practice and a soil data , identify a subset of the plurality of input data associated with the packages of practices, determine a plurality of agri-measures data based on the subset of the plurality of input data in a form of indexes for each of the plurality of crop activity options based at least in part on the plurality of field condition data, and provide the predicted results and recommendations based on plurality of agri-measures.

The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present invention, reference is made to the following description of an exemplary embodiment thereof, considered in conjunction with the accompanying drawings, in which:

FIG. 1 is a diagram depicting an Agricultural Intelligence Environment;

FIG. 2 is a block diagram of an Agricultural Intelligence System and its components;

FIG. 3 is a block diagram of Prediction Model comprising of Exploratory Data Analysis (EDA), Learning and Prediction modules used for managing and constructing the model for agri-measures;

FIG. 4 is a diagram depicting Pre-processing operation of Exploratory Data Analysis (EDA) Module; FIG.5 is a diagram depicting Feature Extraction operation of Exploratory Data Analysis (EDA) Module;

FIG. 6 is a network flow diagram used by the prediction model for managing and recommending agri-measures;

FIG. 7 illustrates a block diagram of multi modal neural network based prediction model with various forms of inputs;

FIG. 8 is a block diagram depicting types of inputs considered by the Agriculture Intelligence System;

FIG. 9 is block diagram depicting types of agri-measures outputs generated by the Agriculture Intelligence System;

FIG. 10 is an example method for determining plurality of agri-measures via multi modal neural network based prediction model of the Agriculture Intelligence System;

FIG. 11 is a block diagram depicting offline and online mode of operations performed by multi modal neural network based prediction model;

FIG. 12 is a hierarchal neutral network based prediction model with deep learning for predicting the agri- measures;

Although specific features of various embodiments may be shown in some drawings and not in others, this is for convenience only. Any feature of any drawing may be referenced and/or claimed in combination with any feature of any other drawing.

DETAILED DESCRIPTION OF THE DISCLOSURE

The following detailed description of the embodiments of the disclosure refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements. Also, the following detailed description does not limit the claims.

As used herein, the term “agricultural intelligence services” refers to a plurality of data providers used to aid a user (e.g., a farmer, agronomist or consultant) in managing agricultural services and to provide the user with recommendations of agri-measures. As used herein, the term “grower” is synonymous to “producer” of the crop or agricultural produce and hence used interchangeably. As used herein, the terms “agricultural intelligence service”, “data network”, “data service”, “data provider”, and “data source” are used interchangeably herein unless otherwise specified. In some embodiments, the agricultural intelligence service may be an external data network (e.g., a third- party system). As used herein, data provided by any such “agricultural intelligence services” or “data networks” may be referred to as “input data”, or “source data”.

As used herein, the term “Agricultural Intelligence System” refers to a computer system supported by hardware devices like input devices, one or more processors, memory configured to carry out the methods disclosed. The agricultural intelligence system is connected with a “user device” (e.g., desktop computer, laptop computer, smartphone, personal digital assistant, tablet or other computing device) and a plurality of databases via communication network.

The term “crop related data” or “crop data” can be used interchangeably and include at least one of the past and present crop production information, past and present geographic information, past and present agricultural information, past and present agronomic information, past and present sensor data, in house research data associated with crop production, any other information related to the planting, growing, and harvesting of a crop, and any other crop parameters as described herein.

The term “Crop data” further refers to, at least one of the following data: (a) harvest data (e.g., crop type, crop variety, crop rotation, whether the crop is grown organically, harvest date, Actual Production History (APH), expected yield, yield, crop price, crop revenue, grain moisture, tillage practice, weather information (e.g., temperature, rainfall) to the extent maintained or accessible by the user, previous growing season information), (b) planting data (e.g., planting date, seed(s) type, relative maturity (RM) of planted seed(s), seed population), (c) nitrogen data (e.g., application date, amount, source), (d) pesticide data (e.g., pesticide, herbicide, fungicide, other substance or mixture of substances intended for use as a plant regulator, defoliant, or desiccant), (e) irrigation data (e.g., application date, amount, source), and (f) scouting observations photos, videos, free form notes, voice recordings, voice transcriptions, weather conditions (temperature, precipitation (current and over time), soil moisture, crop growth stage, wind velocity, relative humidity, dew point, black layer)) with respect to the crop.

The term “field condition data” and “field data” are used interchangeably and without limitations refers to (a) field related data (e.g., field name, acreage, tilling status, irrigation status) (b) soil data comprises soil nutrition, soil types, elements of soil, soil moisture, soil composition (e.g., pH level, organic matter (OM), cation exchange capacity (CEC) and precipitation conditions (c) “packages of practice data” that refers to characteristics and conditions of a field that may be used by the Agricultural Intelligence System to manage and recommend agri-measures. Packages of practice data may include, for example, and without limitation, field weather conditions, field workability conditions, growth stage conditions. Packages of practice data is generated based on the crop stages and multiple research parameters.

The term “environmental data” refers to environmental information related to at least one of the farming activities such as weather data, vegetation and effect of climatic conditions on the crops. The term “weather data” refers to weather condition comprises of the temperature, atmospheric pressure, wind, humidity, precipitation, and cloudiness.

As used herein, a “neural network model” or “machine learning prediction model”, or simply “prediction model”, “deep learning” hereinafter refers to any model that uses at least one of the machine learning operations to predict a agri-measures based on inputs comprising crop related information, environmental data, field condition data which includes field related data, packages of practice and soil data, etc. and is trained on information comprising various inputs using one or more deep learning operations.

As used herein, “an agricultural machine” include at least one of the combine, tractor, cultivator, plow, subsoiler, sprayer or other machinery used on a farm to help with farming. Similarly, as used herein “the agricultural machine computing device” include at least one of the planter monitor, planter controller or a yield monitor.

As used herein, “agri-measures prediction information” (or “crop index prediction information”, “prediction of health index”, or simply “prediction information” hereinafter) refer to any measure of an expected crop index, such as crop yield, crop quality, crop value, or any other suitable measure of crop production (including those described herein), and refer to a set of farming operations expected to result in the measure of expected crop production when performed in a specified manner, at a specified time/location, and the like.

As used herein, the term “database” may refer to either a body of data, a relational database management system (RDBMS), or to both. As used herein, a database may include any collection of data including hierarchical databases, relational databases, flat file databases, object-relational databases, object oriented databases, and any other structured collection of records or data that is stored in a computer system. The above examples are examples only, and thus are not intended to limit in any way the definition and/or meaning of the term database. Examples of RDBMS include, but are not limited to including, Oracle® Database, MySQL, IBM® DB2, Microsoft® SQL Server, Sybase®, and PostgreSQL. However, any database may be used that enables the systems and methods described herein.

As used herein, the terms “software” and “firmware” are interchangeable, and include any computer program stored in memory for execution by a processor, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The above memory types are examples only, and are thus not limiting as to the types of memory usable for storage of a computer program.

As used herein, an element or step recited in the singular and proceeded with the word “a” or “an” should be understood as not excluding plural elements or steps, unless such exclusion is explicitly recited. Furthermore, references to “example embodiment” or “one embodiment” of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.

Artificial Neural Networks (ANN) when used in agriculture fields, provide advantages over traditional systems. ANN based models can take multiple inputs, generate reasoning based on mathematical algorithms and can predict results on the basis of reasoning. Additionally such model can be trained instead of programming them thoroughly. Deep learning is a concept which allows Neural Network (NN) based intelligent system to learn from the past data and statistical data in order to generate accurate results. An Agriculture Intelligence System is built by employing multi input neural network prediction model with deep learning operations. This model makes use of hierarchal (layered) architecture consisting of many hidden layers forming complex network that learns with a feedback mechanism. Such deep learning based Agriculture Intelligence System facilitates analysis of data from past experience in the form of data and predict results using mathematical approach to build intelligent machines.

The subject matter described herein relates generally to determining one or more agri-measures by using Neural Network (NN) based prediction model which can be trained using deep learning systems. It works by obtaining one or more of field related data, packages of, practices, soil data, weather data and crop related data and determining via neural network model and predicting agri measures based on information comprising crop related information, field condition information and environmental information. Specifically, one of the embodiments of the invention describes the Agricultural Intelligence system based on multi- tasking multi - modal neural network model which is configured to receive neural network model data inputs as input from different neural network domains/modalities corresponding to different neural network tasks. Example of neural network domains/modalities inputs include speech, images, language, or text. Example of neural network tasks include speech recognition, image classification, object detection, visual analytics, machine translation, or parsing. For instance, the multi task multi- modal neural network model may receive text inputs corresponding to a machine translation task, e.g., an input text segment in an input natural language to be translated into a target natural language, or text inputs corresponding to a parsing task, e.g., an input text segment to be parsed.

Further, the system and method described herein includes (i) receiving a plurality of inputs in a form of crop related data (ii) retrieving a plurality of input data from a at least one of the databases, (iii) retrieving an environmental data which consist of weather data and a field condition data, which comprises of field related data, a packages of practice and a soil data (iv) identifying a subset of the plurality of input data associated with the packages of practices, (v) determining a plurality of agri-measures based on the subset of the plurality of input data in a form of indexes for each of the plurality of crop activity options based at least in part on the plurality of field condition data, and (v) providing the predicted results and recommendations based on plurality of agri-measures in the form of crop health index, farmer rating index, crop production advices, type of crop and/or yields, and other recommendations applicable to field such as fertilizers, pesticides, growth regulators and harvest aids, etc.

Generally, in an agricultural intelligence environments (e.g., farms, groups of farms, and other agricultural cultivation environments), agricultural growers employ significant strategy and analysis to make decisions on agricultural cultivation. In many cases, growers analyze a variety of data to make prior strategic decisions few months before the period of crop cultivation (i.e., growing season). While making such strategic decisions, crop growers must consider at least some of the following decision constraints like fuel and resource costs, historical and projected weather trends, soil conditions, projected risks posed by pests, disease and weather events, and projected market values of agricultural commodities (i.e., crops). Analyzing these decision constraints may help a producer or grower to predict key agricultural outcomes including crop yield, energy usage, cost and resource utilization, and farm profitability.

Despite its importance, such analysis and strategy is difficult to accomplish for a variety of technical challenges. Firstly, obtaining reliable information for the various considerations of the crop grower or producer is often difficult. Secondly, extracting such information and aggregating into a usable manner is a time consuming task. Thirdly, the extracted data is not precise enough to determine strategy. For example, providing the index for the reference to different kinds of benefits to farmers like insurance, credit loans, supplier information, expert opinions from agriculture scientist and recommendation of features such as: weather alert, soil nutrients etc.

Additionally, growers often need to make decisions regularly during the growing season. Such decisions may include: when to harvest, provide supplemental fertilizer, and how to mitigate risks posed by pests, disease and weather. Subsequently, growers require continuous monitoring of various aspects of their crops during the growing season including weather, soil, and crop conditions. Accurately monitoring all such aspects at a granular level is difficult and time consuming. Accordingly, methods and systems for analyzing crop related data, soil data, environmental data and packages of practice to provide strategic recommendations for maximizing crop yield are desirable. As per the present embodiment, the system and method are described herein to facilitate the evaluation, analysis and recommendation of agri-measures in order to suggest ratings to different stakeholders. Recommendations includes field applicable advice such as fertilizers, pesticides, herbicides, fungicide, growth regulators, and harvest aids, crops planted, crop yields, fuel costs, equipment operational data, pest and disease infestations and recommendation for a crop based at least in part upon what crop(s) have been grown in a particular location in the previous year(s) or growing cycles. Farmers would use recommendations to determine long-term patterns, or to predict future performance, type of crop and/or yields based upon similar conditions in the past.

An Agricultural Intelligence Environment 100 is depicted in FIG. 1 which comprises of users (for eg: growers, farmers, producers, etc.) 110, an agricultural intelligence system 120 and various external entities 130 which includes banks and funding agencies, government agencies and policy makers, agricultural and market suppliers, insurance agencies, agriculture scientists etc. Users 110 interact with agricultural intelligence system in order to provide input related to crop data and field data and also to effectively receive recommendations based on agri-measures. Various external entities 130 like banks and financial institutions, insurance agencies, agriculture scientists / experts, Government and policy makers, etc. utilize the predicted agri-measures in the form of crop indices, farmer rating indices and which can become a reference for providing different kinds of benefits to the users (growers, farmers, producers, etc.) 110 like insurance options, credit loans, supplier information, expert opinions from agriculture scientist and recommendation based on type of crops and / or yield, weather and soil data and so on.

In one of the exemplary embodiments, the agricultural intelligence system provides the agricultural intelligence services using a deep learning model. Therefore, the agricultural intelligence system may be implemented using a variety of distinct machine learning models.

An agricultural intelligence system 120 is depicted in FIG. 2 with all its components which includes user 210, network 220, database 230, and prediction model 240 which uses a prediction engine 250 to generate results in the form of agri-measures. The user 210 may interact with the prediction model directly in offline mode and via network 220 in an online mode. In similar fashion, in many examples, the user may access data from data sources (database 230) indirectly via the prediction model, in other examples, the user may directly access the data sources (database 230) via any suitable network connection. Agricultural intelligence system 120 utilizes a prediction model 240 which is based on plurality of neural network models to provide network services. The prediction model 240 is depicted in FIG.3 which consist of three modules: Exploratory Data Analysis (EDA) module 310, learning module 320 and prediction module 330. The EDA module processes the input leaf image 340 (for eg: diseased crop image, crop image, etc.) using the Pre processing operation 350 and extract the necessary features from the processed input using Feature Extraction 360 techniques and pass the information to the learning module 320. The learning module 320 has a repository consisting of image database 370 and training image 380 which helps to store the processed leaf image and train the prediction model by storing the newly found sample in training database for future prediction based on continuous learning. The learning module also consists of the Neural Network (NN) classification model 390 in order to classify the images using classification rules based on principles of neural network with deep learning and passes the classified results to the Prediction module 330 which in turn generates the results in the form of disease prediction, indices and other recommendations. In this manner, the prediction model undergoes continuous learning and provide improvements in the predicted results using neural network model based on deep learning. In case, the system predicts the disease with confident score of 90% plus, then these images are taken weekly once and continues learning is initiated.

In one of the exemplary embodiments, the input leaf image are taken on a daily from the farmers for processing by the Exploratory Data Analysis (EDA) module 310. In order to increase prediction accuracy, the system is trained often with unseen data by the prediction model. So, once 100+ new images are received from the farmers, learning model will process these 100+ images in an automated way and the new model is created. A quality improvement of 0.3% for each 100+ samples on every iteration has been noticed as per the experimental data. As a part of an experiment, 3 iterations have been performed which resulted in an average of 0.3% increase. The experimental results for each iteration is as follow:

Iteration 1

Total Image in the dataset 54303 New images from farmer 996 Previous quality 89.10% New model quality 89.55%

Difference in quality improvement 0.45%

Iteration 2

Total Image in the dataset 55299 New images from farmer 115 Previous quality 89.55% New model quality 89.80%

Difference in quality improvement 0.25%

Iteration 3

Total Image in the dataset 55414 New images from farmer 275 Previous quality 89.80% New model quality 90.10%

Difference in quality improvement 0.30%

Average Quality improvement in all 3 iterations 0.33%

Round off average ~ 0.3%

FIG 4. shows the operations involved in Pre-processing 350 operation of FIG. 3. Sample images 410 are shown which are inputted to the pre- processing operation 350, which consist of Image Size Reduction 420 and Labelling 430. Under Images size reduction 420, size of the original image has been reduced to less than a threshold (for eg: 100 KB) or 25% of the size (whichever gives lower size). This has been done to reduce the training and prediction period. With this approach of image size reduction 420, reduction in processing period from 4 days to 2 days has been noticed, which has further contributed in improving the efficiency of the prediction model. Under Labelling 430, collected images are grouped or classified under pre-defined categories or otherwise, new categories are created. Collected images are tagged, ie. assigned with a keyword to accurately recognize features in a vast collection of images. Tagging is done by industry experts. Subsequently, the feature extraction techniques are applied to obtain features that are useful in classifying and recognition of images. The Feature Extraction 360 technique is depicted in FIG. 5 wherein, an algorithm is applied on processed and labelled input image data and the interesting parts of information from that image are chosen. Image profile is downloaded from web to showcase how the intermediatory images looks like in NN intermittent layers. FIG. 5 shows sample intermediate image downloaded from web.

FIG. 6 shows the network flow diagram of the series of operations performed by the prediction model for managing and recommending the agri-measures. An input data is extracted 620 from the user and the databases. In the next stage, the extracted data is stored in the local/network databases wherein, if the input does not get matched with the existing agriculture database 630 then it is stored in the modelling database 640 and further go to Interface Database Module 650. The data is further analyzed 660 in combination with the different sub sets of packages of practice. The multi-modal NN based prediction model 670 is configured to receive multiple inputs from different neural network domains/modalities (for eg: speech, images, language, or text) corresponding to different neural network tasks (such as speech recognition, image classification, object detection, parsing, etc.).

FIG. 7 shows various forms of inputs 701 considered by the multi modal NN model. Based on this analyzed data, the multi-modal NN based prediction model 670 in coordination with the crop prediction engine 260 processes the inputs, which is joint representation of different modalities and generate prediction result 680 in the form of recommendations and indices to represent the crop health.

As per one of the exemplary embodiments, farmers are on boarded in the platform and facilitated by getting information related to crop detail accounting for acreage and type of crops grown. Once the farmers are on boarded, their crop related data (eg crop image, harvest data, etc.) are mapped against the input application, Packages of practice (POP) and Geo detail. The agricultural intelligence system processes the user data that includes crop related data 810, crop image 830, diseased crop image 840, environmental data 860 (for eg: weather data) and field condition data 850 for the field or field region identified by the field related data, packages of practice 820 and soil data as depicted in FIG. 8. Based on entered crop data, the agricultural intelligence system may recommend farmers as to how to increase mechanization and get benefit from various government policies and insurance agencies. Suggestions, recommendations and analysis generated by such agricultural intelligence system facilitate better farming.

In an exemplary embodiment, an agricultural machine may be coupled to a computing device (“agricultural machine computing device”) that interacts with the agricultural intelligence system which can also be considered as the user device. The agricultural machine and agricultural machine computing device may provide the agricultural intelligence system with crop related data, field condition data and environmental data. When crop related data and field condition data is not provided directly to the agricultural intelligence system via one or more agricultural machines or agricultural machine computing devices that interacts with the agricultural intelligence system, the user may be prompted via one or more user interfaces on the user device (served by the agricultural intelligence system) to input such information. In an embodiment, the user may identify crop related data by accessing a map on the user device (served by the agricultural intelligence system).

The agricultural intelligence system also utilizes environmental data to provide agricultural intelligence services. Environmental data may be obtained from external data sources accessible by the Agricultural intelligence system. Environmental data may also be obtained from internal data sources integrated within the Agricultural intelligence system. Data sources for environmental data may include at least one of the following: weather radar sources, satellite -based precipitation sources, meteorological data sources (e.g., weather stations), satellite imagery sources, aerial imagery sources (e.g., airplanes, unmanned aerial vehicles), terrestrial imagery sources (e.g., agricultural machine, unmanned terrestrial vehicle), soil sources and databases, seed databases, crop phenology sources and databases, and pest and disease reporting and prediction sources and databases.

The multi modal NN based prediction model provides a user with a plurality of agri-measures services as depicted in FIG. 9 in order to generate AI based recommendations and suggestions in form of crop health index 910, farmer rating index 920, crop disease prediction results 930, recommendation 940 and weather alerts 650. Such agricultural intelligence services may be used to recommend courses of action for the user to undertake.

In one of the exemplary embodiments, the agri-measures services may include recommendations at least one of the following: planting advisor, a nitrogen application advisor, a pest advisor, a field health advisor, a harvest advisor, and a revenue advisor. Recommendations applicable to the fields such as fertilizers, pesticides, herbicides, fungicide, growth regulators, and harvest aids, crops planted, crop yields, fuel costs, equipment operational data, pest and disease infestations, etc. are used by farmers to determine long-term patterns, or to predict future performance, type of crop and/or yields based upon similar conditions in the past. Recommendation for a crop, based at least in part upon what crop(s) have been grown in a particular location in the previous year(s) or growing cycles and recommendation of a crop varietal to plant will affect the return on investment for a farmer and maximize profits because he may or may not have to apply specific crop treatments on a crop depending upon what crops have been grown in that location previously. For example, such recommendations not only provide a better overall estimate of how well a particular crop will perform at a given location at a given time, but also to optimize treatments, irrigation, planting and harvesting of a crop.

As noted above, the agricultural intelligence system may be implemented using a variety of distinct computing devices using any suitable network. In an exemplary embodiment, the agricultural intelligence system uses a client-server architecture configured for exchanging data over a network (e.g., the Internet). One or more user devices may communicate via a network with a user application or an application platform. The application platform represents an application available on user devices that may be used to communicate with agricultural intelligence systems. Other example embodiments may include other network architectures, such as peer-to-peer or distributed network environment.

The application platform may provide server-side functionality, via the network to one or more user devices. Accordingly, the application platform may include client side software stored locally at the user device as well as server side software stored at the Agricultural intelligence system. In one of the embodiments, the user device may access the application platform via a web based platform. The user device may transmit data to, and receive data from one or more front-end servers. In an exemplary embodiment, the user device may take user input in the form of requests or field specific data. One or more front-end servers may process the requests and user input in order to determine whether the requests are service requests or content requests. Content requests may be transmitted to one or more content management servers for processing. Application requests may be transmitted to one or more application servers. In an example embodiment, application requests may take the form of a request to provide crop data, packages of practice agricultural intelligence services for one or more specific fields.

In an exemplary embodiment, the application platform may include one or more servers in communication with each other. For example, the agricultural intelligence system may include at least one of the front-end servers, application servers, content management servers, account servers, modeling servers, environmental data servers, and corresponding databases.

FIG. 10 is an example method for determining agri-measures by the multi modal NN model based Agricultural intelligence system 120 (shown in FIG. 1). Method 1001 is implemented by agricultural intelligence system 120 by receiving 1020 a plurality of crop related data from the user device, retrieving 1030 a plurality of input data from a plurality of databases (630, 640), retrieving 1040 a plurality of field condition data (field related data, packages of practice and soil data) and environmental data (weather data). A subset of inputs is determined 1050 in the form of crop data, field data, weather data, etc. based on the subset of the plurality of packages of practice, a plurality of agri-measures is determined 1060 for each of the plurality of field and crop specific data via a multi modal NN model (670) and the predicted results (crop index, farmers rating index, etc.) and recommended field activity option is provided based on plurality of agri-measures.

In some examples, users may be prompted at the user device to answer questions regarding agri measures uses, practices and implementation, in case such information has not already been provided to the agricultural intelligence system. Few examples of parameters are: product type, application date, formulation, rate, acres tested, amount, source, costs, latest growth stage of crop at which pesticide can be applied, application equipment, labor costs, expected crop price as well as current crop data (e.g., planting date, seed(s) type, relative maturity (RM) of planted seed(s), seed population). Accordingly, the multi modal neural network module receives such data from user devices.

FIG. 11 is an example for determining agri-measures by the agricultural intelligence system based on neural networks models in both online and offline state. User can use the system directly in an offline state 1150 via offline NN model 1120 and in an online state 1140 using network, via online NN model 1110. Once the online model 1110 generates the agri-measures, it stores the agri measure in an agriculture database 1130 via network. An online state model 1110 receives feedback 1160 via internet from output generated by the Agriculture intelligence system performs continuous learning for improvement in the predictions. After the online model receive updates through feedback 1160 based on iteration, the latest version of agri-measures will be updated in an offline model 1120 as soon as it is exposed to network, such that the offline model also provides better result.

In application, multi-modal NN model based prediction model produce a health index of crop, including a predicted measure of crop production, health recommendation and a set of farming operations that, when performed, is expected to produce the predicted measure of crop health, disease prediction. In practice, such NN model can use or be trained by any deep learning, machine learning operation or any combination of machine learning operations for predictions of crop production.

NNs are mathematical models are an interconnected network of nodes, where each node assigned to the network represents a neuron. In a network, the neurons play an important role, they accept and process the inputs and create the outputs and the connection between two neurons carries the weights in which the electrical information is encoded implicitly. The electrical information simulates with specific values stored in those weights that enable the networks to have capabilities like learning, generalization, imagination and creating the relationship within the network.

The Agriculture Intelligence System based on multi modal NN model with deep learning is shown in FIG. 12, which operate in a feed-forward mode from the input layer 1210 through the hidden layers 1220 to the output layer 1230. The last layer or the output layer 1230 consist of the nodes usually computed by a non-linear combination of the nodes of input 1210 and hidden layers 1220. The user input is fed as weights from the input layer 1210 which undergoes series of mathematical formulation based on the series of hidden layer 1220 involved in the machine learning algorithms and generate output from the output layer 1230 in form of indices and recommendations. The output data generated is further fed as a feedback 1240 into the Agriculture Intelligence System in the form of input in the next iteration which alters the weights to minimize errors and generate more accurate results subsequently. This process continues for numerous iteration in a very complex manner till the desired accuracy is achieved in terms of threshold. Threshold for predicting any category is set. Only if the prediction score is greater than the threshold, then the system accepts the category as correct else the system throws a result that the category is not trained and it goes to continued learning process. The multi modal NN model with deep learning identifies crop health quality over the course of the season and uses such crop health determinations to recommend scouting or investigation in areas of poor field health. More specifically, the model receives and processes field image data to determine, identify, and provide index values of crop health. The health index values of crop health may range from zero (indicating minimum) to 5 (indicating the maximum). In an example embodiment, the index value has a specific recommendation scheme, so that every value has a predefined point’s health scheme (e.g., 1 show the crop with the lowest relative health index). In one of the embodiments, the multi modal NN model with deep learning may receive one or more of the following data points for each field identified by the user (as determined from crop data): i. A first set of data points includes weather information. Such environmental information includes information related to satellite data, cloud data, online data, and manual data. ii. A second set of data points includes crop data related to field condition data. Such field condition data may include field and soil nutrients, and soil types. iii. A third set of data points includes crop images, previous data generated by the model. iv. A fourth set of data points includes packages of practice related to planting data. Such field- condition data may include, for example, planting dates, seed type, relative maturity (RM) of planted seed, and seed population.

The system and method described herein include at least one of recommendation utilization of agricultural farmers through providing the health index to insurance agencies and bank companies to provide the benefits to farmers; improved selection of method of fertilization and time period for the same; Field specific recommendation from the experts regarding the use of pesticides and fertilizers for the predicted disease; improved selection of types of crops, seeds planted for the given location of soil; and improved selection of time of harvest. Farmers would use recommendation to determine long-term patterns, or to predict future performance, type of crop and/or yields based upon similar conditions in the past. Recommendations generated by the multi modal NN model provide a better overall estimate of how well a particular crop would perform at a given location at a given time, but also to optimize treatments, irrigation, planting and harvesting of a crop, as well as generate a recommendation of a crop varietal to plant in order to maximize profits for a farmer. Thus, various embodiments compile as complete model deals with variety of problems related to decision making faced by a farmer in order to generate data-driven choices regarding a given location, crop, time of year, etc. which in turn will maximize profits from a given piece of land.

In one embodiment, a computer program is provided, and the program is embodied on a computer readable medium. In an exemplary embodiment, the system is executed on a single computer system, without requiring a connection to a server computer. In a further embodiment, the system is being run in a Windows® environment. In yet another embodiment, the system is run on a mainframe environment and a UNIX® server environment. The application is flexible and designed to run in various different environments without compromising any major functionality. In some embodiments, the system includes multiple components distributed among a plurality of computing devices. One or more components may be in the form of computer-executable instructions embodied in a computer-readable medium.

As used herein, the term “non-transitory computer-readable media” is intended to be representative of any tangible computer-based device implemented in any method or technology for short-term and long-term storage of information, such as, computer-readable instructions, data structures, program modules and sub-modules, or other data in any device. Therefore, the methods described herein may be encoded as executable instructions embodied in a tangible, non-transitory, computer readable medium, including, without limitation, a storage device and/or a memory device. Such instructions, when executed by a processor, cause the processor to perform at least a portion of the methods described herein. Moreover, as used herein, the term “non-transitory computer-readable media” includes all tangible, computer-readable media, including, without limitation, non- transitory computer storage devices, including, without limitation, volatile and nonvolatile media, and removable and non-removable media such as a firmware, physical and virtual storage, CD- ROMs, DVDs, and any other digital source such as a network or the Internet, as well as yet to be developed digital means, with the sole exception being a transitory, propagating signal.

The systems and processes are not limited to the specific embodiments described herein. In addition, components of each system and each process can be practiced independent and separate from other components and processes described herein. Each component and process also can be used in combination with other assembly packages and processes. This written description uses examples to disclose the disclosure, including the best mode, and also to enable any person skilled in the art to practice the embodiments, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

It will be understood that the embodiment described herein is merely exemplary and that a person skilled in the art may make variations and modifications without departing from the spirit and the scope of the invention. More particularly, many components of the exemplary embodiment have well known mechanical equivalents. All such variations and modifications are intended to be included within the scope of the invention as defined in the appended claims.