BUSCH KRISTINA (DE)
EGGERS TILL (DE)
NAVARRA-MESTRE RAMON (DE)
HADEN EGON (DE)
ARNHOLD JESSICA (DE)
FISCHER SEBASTIAN (DE)
MARTIN PALMA ANDRES (ES)
KLUKAS CHRISTIAN (DE)
FRIEDEL SWETLANA (DE)
STUERMER-STEPHAN BASTIAN (DE)
HAHN STEFAN (DE)
TRESCH STEFAN (DE)
DE MELO RAMÁSIO FERREIRA ET AL: "Diagnosis of Apple Fruit Diseases in the Wild with Mask R-CNN", 13 October 2020, FOUNDATIONS OF AUGMENTED COGNITION : 7TH INTERNATIONAL CONFERENCE, AC 2013, HELD AS PART OF HCI INTERNATIONAL 2013, LAS VEGAS, NV, USA, JULY 21-26, 2013 ; PROCEEDINGS; [LECTURE NOTES IN COMPUTER SCIENCE; LECT.NOTES COMPUTER], SPRINGER, CHAM, PAGE(S), ISBN: 978-3-030-71592-2, XP047569939
SANKARAN S ET AL: "A review of advanced techniques for detecting plant diseases", COMPUTERS AND ELECTRONICS IN AGRICULTURE, ELSEVIER, AMSTERDAM, NL, vol. 72, no. 1, 1 June 2010 (2010-06-01), pages 1 - 13, XP026995685, ISSN: 0168-1699, [retrieved on 20100409]
NOMI ET AL., NUCLEIC ACIDS RESEARCH, vol. 28, no. 12, 2000, pages e63
ASTIERBRAHABAYLEY, JOURNAL OF THE AMERICAN CHEMICAL SOCIETY, vol. 128, no. 5, 2006, pages 1705 - 10
FOLOGEA ET AL., NANO LETT, vol. 5, no. 10, 2005, pages 1905 - 9
MILLERTANG, CLINICAL MICROBIOLOGY REVIEWS, 2009
| Claims Computer-implemented method for providing training data for a machine learning algorithm for image classifying a pathogen infestation of a plant, comprising the steps: providing image data of a plant or a plant part infested with a pathogen; providing genetic result data of the plant or the plant part to which the image data refers comprising at least information about the type of pathogen; labeling the image data with the genetic result data. Computer-implemented method according to claim 1 , wherein the genetic result data further comprising information on the quantity of the pathogen. Computer-implemented method according to claim 1 or claim 2, wherein the image data is based on at least one RGB-image of the plant or the plant part, wherein the at least one RGB-image is preferably taken against a white background. Computer-implemented method according to any one of the preceding claims, wherein the image data is based on - at least one multispectral image, preferably in the radiation range between 390 nm and 850 nm, in the near infrared range with 800 nm and in the far infrared range (FIR) with 820 nm, wherein the at least one multispectral image is preferably taken against a white background; and/or - at least one hyperspectral image in a radiation range between 390 nm and 1650 nm, preferably up to 850 nm Computer-implemented method according to any one of the preceding claims, wherein the method further comprises the step of filtering the images, trimming and/or marking specific areas of the images before providing the image data. Computer-implemented method according to any one of the preceding claims, wherein the genetic result data is based on a quantitative DNA-analysis tool, preferably PCR-based techniques, isothermal amplification methods and/or a DNA microarray method. Computer-implemented method according to any one of the preceding claims, wherein the pathogen is a fungus, preferably a Phakopsora pachyrhizi uv\Qus. Computer-implemented method according to any one of the preceding claims, wherein the plant is a soybean plant or a part of a soybean plant and the pathogen is a Phakopsora pachyrhizi fu ng us . Use of image data of a plant or a plant part infested with a pathogen in a method according to any one of claims 1 to 8. Use of genetic result data of a plant or a plant part comprising at least information about a pathogen infestation in a method according to any one of claims 1 to 8. Use of training data obtained according to a method according to any one of claims 1 to 8 for training a machine learning algorithm for image classifying a pathogen infestation of a plant. Neural network trained with training data provided according to any one of claims 1 to 8, wherein the training features for the training of the neural network preferably comprise the Normalized Differences Vegetation Index (NDVI) and the Greeness Index (G). A classification system for image classifying a pathogen infestation of a plant, comprising: at least one input interface for providing image data of a plant or plant part; at least one evaluation unit which is set up to feed forward a machine learning algorithm with the image data of a plant or plant part, wherein the machine learning algorithm is trained on the basis of training data provided according to any one of claims 1 to 8; at least one output interface adapted to output a classification result of a pathogen infestation of the plant. A computer implemented method for image classifying a pathogen infestation of a plant, comprising: providing image data of a plant or plant part; feeding a the machine learning algorithm with the image data of a plant or plant part, wherein the machine learning algorithm is trained on the basis of training data provided according to any one of claims 1 to 8; outputting a classification result of a pathogen infestation of the plant. Computer implemented method according claim 14, further comprising the step of providing recommendation data for treating the invested plants with an active ingredient and/or a crop protection product suitable for the classified pathogen infestation of the plant. A computer program element which when executed by a processor is configured to carry out the method of claim 14 or claim 15. |
TECHNICAL FIELD
The present disclosure relates to a computer-implemented method for providing training data for a machine learning algorithm for image classifying a pathogen infestation of a plant.
TECHNICAL BACKGROUND
Pathogens, in particular fungal diseases, are a major problem in commercial agricultural production processes. For many processes, it is important to detect the pathogens as early as possible in order to be able to initiate respective countermeasures, such as treatment with fungicides, as early as possible. Infestation assessments of plant diseases have been carried out visually for many years and require very experienced phytopathologists who can recognize and differentiate the symptoms of the various diseases. Since the visual assessment is influenced by a wide range of biotic and abiotic factors, incorrect assessments cannot be excluded a. Furthermore, even a very experienced phytopathologists is able to detect many diseases only after a certain degree of infestation, so that an early stage infestation is often not detected. Moreover, the classification of the degree of infestation by phytopathologists suffers from the subjective judgement of the individual. This is particularly problematic if statistically relevant effects are to be measured, especially effects that are not very pronounced such as fungicidal effects of fungicidal development candidates. The effect becomes even more pronounced if the infection data is gathered over a prolonged period by more than one phytopathologist. In this case, the standard deviation of the assessments is increased by the respective disease features that are differently ranked by the individuals.
In view of this, it is found that a further need exists to provide a possibility to detect a pathogen on a plant or a part of a plant, in particular a fungal disease, in an early stage infestation. Moreover, a further need exits to provide a method that can objectively classify the degree of infestation of a plant with high confidentiality and low standard deviation.
SUMMARY OF THE INVENTION
In view of the above, it is an object of the present invention to provide a possibility to detect a pathogen on a plant or a part of a plant, in particular a fungal disease, in an early stage infestation. A further object of the present invention is to provide a method that can objectively classify the degree of infestation of a plant with high confidentiality and low standard deviation.
These and other objects, which become apparent upon reading the following description, are solved by the subject matter of the independent claims. The dependent claims refer to preferred embodiments of the invention.
According to a first aspect of the present disclosure, a computer-implemented method for providing training data for a machine learning algorithm for image classifying a pathogen infestation of a plant is provided, comprising the steps: providing image data of a plant or a plant part infested with a pathogen; providing genetic result data of the plant or the plant part to which the image data referred comprising at least information about type of pathogen; labeling the image data with the genetic result data with respect to the pathogen, in particular as concentration, e.g. in relation to plant DNA, mass of plant leaves, etc.
The present disclosure is based on the finding that it is in principle possible to train a machine learning algorithm in such a way that it can reliably detect pathogens, even at an early stage of infestation, by analyzing image data of a plant or part of a plant, and classify the image data according to the extent of the disease. However, it has turned out that such a machine learning algorithm can only make a reliable classification, if the training data are highly reliable. It has also been shown that such a trained machine learning algorithm, e.g. a neural network, in many cases, can detect a pathogen more reliably and earlier than a human being. In order to provide such reliable training data, the present disclosure proposes to provide image data of infested plants and to carry out a genetic analysis, e.g. a DNA and/or a RNA analysis, of these plants. This makes it possible to provide training data of very high quality, which can be labelled with virtually no errors. It has been shown that with such training data a machine learning algorithm can be trained, which in many cases can classify images of infested plants more reliably than a human being. However, it should be noted that the present disclosure is not limited to the fact that only the genetic result data are used for providing the training data. In fact, in addition other data/parameters can be used additionally for labelling the image data.
In this context, it should be noted that the training data obtained according to the present disclosure could be used as training data for different machine learning algorithms, e.g. deep learning model, neural networks with different structures, etc. The image data can be provided in different formats and/or be provided from different images and/or partial images of plants and/or from different plant parts. The genetic result data can be obtained by different genetic analysis methods. In this context, it is only important that the genetic result data are based on the plant or plant part referenced in the respective image data. Notably, in this context, it is only important that the image data are unambiguously assigned to the respective genetic result data, e.g. DN A and/or RNA result data, so that an error-free labeling of the training data can take place. The term pathogen is understood broadly as any living organism, which can cause harm to a plant or can negatively affect the growth or the health of a plant. Pathogens include, but are not limited to, any kinds of weeds, fungi, bacteria, viruses, insect pests, arachnids, nematodes, mollusks, and rodents. In this respect, it is only relevant that such pathogen may be detected, at least in one infestation stage, by means of an image analysis. Preferably, pathogens include fungi, bacteria, viruses, insect pests, arachnids, nematodes, mollusks, and rodents. More preferably, pathogens include fungi, bacteria, and viruses. In an example, the pathogen is a fungus, in particular the Phakopsora pachyrhizi\\xv\<yjs. Such a may affect different plant species, wherein in an example, the plant is a soybean plant and the fungus is Phakopsora pachyrhizi.
The “type of pathogen" includes, but is not limited to, any classification of the pathogen as such and/or of its growth stage(s). Preferably, the “type of pathogen” includes any biological classification of the pathogen. More preferably, the “type of pathogen” includes class, subclass, order, family, genus, species, subspecies, variety, biotype, and mutant of the pathogen. Most preferably, the “type of pathogen” includes species, subspecies, variety, biotype, and mutant of the pathogen. Particularly, the “type of pathogen” additionally includes the biological classification of the pathogen as well as the classification of its growth stage(s).
“Plant part” or “part of a plant” includes, but is not limited to, leaves, cotyledones (seed leaves), roots, stems, flowers, fruits, pollens, xylem, phloem, seeds. Preferably, “plant part” or “part of a plant” include any above-ground parts of a plant. More preferably, “plant part” or “part of a plant” include leaves, cotyledones (seed leaves), stems, flowers, and fruits. Most preferably, “plant part” or “part of a plant” include leaves, cotyledones (seed leaves), and fruits. Particularly, “plant part” or “part of a plant” include leaves and cotyledones (seed leaves).
The invention is particularly suitable for the following combinations of pathogens on plants or plant parts: l/Zj^ospp. (white rust) on ornamentals, vegetables (e. g. A Candida) and sunflowers (e. g. A. tragopogonisy, A/ternaria spp. (Alternaria leaf spot) on vegetables (e.g. A. dauci or A. porri), oilseed rape (A. brassicico/a or brassicae), sugar beets (A tenuis), fruits (e.g. A. grandis), rice, soybeans, potatoes and tomatoes (e. g. A. soiani, A. grandis or A. alternate), tomatoes (e. g. A. soiani or A. alternate) and wheat (e.g. A. triticina)-, Aphanomyces spp. on sugar beets and vegetables; Ascochytas . on cereals and vegetables, e. g. A. tritici (anthracnose) on wheat and A. hordeion barley; Aureobasidium zeae (syn. Kapatiella eae) on corn; Bipolaris and Drechslera spp. (teleomorph: Cochiioboiuss yp.), e. g. Southern leaf blight (£ maydis)o Northern leaf blight B. zeicoia on corn, e. g. spot blotch B. sorokiniana on cereals and e. g. B. oryzae on rice and turfs; Blumeria (formerly Erysiphe graminis (powdery mildew) on cereals (e. g. on wheat or barley); Botrytis cinerea (teleomorph: Botryotinia fuckeiiana. grey mold) on fruits and berries (e. g. strawberries), vegetables (e. g. lettuce, carrots, celery and cabbages); B. squamosa or B. aiiiion onion family), oilseed rape, ornamentals (e.g. Beliptica'), vines, forestry plants and wheat; Bremia lactucae (downy mildew) on lettuce; Ceratocystis (syn. Ophiostoma) spp. (rot or wilt) on broadleaved trees and evergreens, e. g. C. uimi (Dutch elm disease) on elms; Cercospora spp. (Cercospora leaf spots) on corn (e. g. Gray leaf spot: C. zeae-maydis), rice, sugar beets (e. g. C. beticoia), sugar cane, vegetables, coffee, soybeans (e. g. C. sojina or C. kikuchii) and rice; Cladobotryum (syn. Dactylium) spp. (e.g. C. mycophilum (formerly Dactylium dendroides, teleomorph: Nectria albertinii, Nectria rosella syn. Hypomyces rosellus) on mushrooms; Cladosporium spp. on tomatoes (e. g. C. fulvum. leaf mold) and cereals, e. g. C. herbarum (black ear) on wheat; Claviceps purpurea (ergot) on cereals; Cochliobolus (anamorph: Helminthosporium of Bipolaris) spp. (leaf spots) on corn (C. carbonum), cereals (e. g. C. sativus, anamorph: B. sorokiniana) and rice (e. g. C. miyabeanus, anamorph: H. oryzae)-, Coiietotrichum (teleomorph: Giomereiia) spp. (anthracnose) on cotton (e. g. C. gossypii), corn (e. g. C. graminicoia: Anthracnose stalk rot), soft fruits, potatoes (e. g. C. coccodes. black dot), beans (e. g. C. Hndemuthianum), soybeans (e. g. C. truncatumo C. gioeosporioides , vegetables (e.g. C. iagenarium or C. capsici), fruits (e.g. C. acutatum), coffee (e.g. C. coffeanum or C. kahawae) and C. gioeosporioides on various crops; Corticiums^., e. g. C. sasakii (sheath blight) on rice; Corynespora cassiicoia (leaf spots) on soybeans, cotton and ornamentals; Cycioconium spp., e. g. C. oieaginum on olive trees; Cyiindrocarpon spp. (e. g. fruit tree canker or young vine decline, teleomorph: Nectria or Neonectria spp.) on fruit trees, vines (e. g. C. Hriodendri, teleomorph: Neonectria Hriodendri. Black Foot Disease) and ornamentals; Dematophora (teleomorph: RoseHinia) necatrix (root and stem rot) on soybeans; Diaporthe spp., e. g. D. phaseoiorum (damping off) on soybeans; Drechslera (syn. Helminthosporium, teleomorph: Pyrenophora) spp. on corn, cereals, such as barley (e. g. D. teres, net blotch) and wheat (e. g. D. tritici-repentis. tan spot), rice and turf; Esca (dieback, apoplexy) on vines, caused by Formitiporia (syn. PheHinus) punctata, F. mediterranea, Phaeomonieiia chiamydospora (formerly Phaeo- acremonium chiamydosporum), Phaeoacremonium aleophilum and/or Botryosphaeria obtusa, Eisinoes . on pome fruits (£. pyri), soft fruits (£. veneta. anthracnose) and vines (£. ampeiina anthracnose); Entyioma oryzae (leaf smut) on rice; Epicoccum spp. (black mold) on wheat; Erysiphe spp. (powdery mildew) on sugar beets (£. betae , vegetables (e. g. E. pisi), such as cucurbits (e. g. E. cichoracearum), cabbages, oilseed rape (e. g. E. cruciferaruniy, Eutypa lata (Eutypa canker or dieback, anamorph: Cytosporina lata, syn. Liberteiia biepharis) on fruit trees, vines and ornamental woods; ExserohHum (syn. Helminthosporium spp. on corn (e. g. E. turcicum)-, Fusarium (teleomorph: Gibberella spp. (wilt, root or stem rot) on various plants, such as F. graminearum F. cu/morum (root rot, scab or head blight) on cereals (e. g. wheat or barley), F. oxysporumov\ tomatoes, F. soiani^. sp. glycines now s c\. F. virguiiforme and F. tucumaniae and F. brasiiiense each causing sudden death syndrome on soybeans, and F. verticillioides on corn; Gaeumannomyces graminis{\a e-a\\) on cereals (e. g. wheat or barley) and corn; Gibberella spp. on cereals (e. g. G. zeae) and rice (e. g. G. fujikuror. Bakanae disease); Giomereiia cinguiata on vines, pome fruits and other plants and G. gossypii on cotton; Grainstaining complex on rice; Guignardia bidwellii (black rot) on vines; Gymnosporangium spp. on rosaceous plants and junipers, e. g. G. sabinae (rust) on pears; Helminthosporium spp. (syn. Drechsiera, teleomorph: Cochiioboius) on corn, cereals, potatoes and rice; Hemiieia spp., e. g. H. vastatrix (coffee leaf rust) on coffee; isariopsis ciavispora (syn. Ciadosporium vitis) on vines; Macrophomina phaseoiina (syn. phaseoii) (root and stem rot) on soybeans and cotton; Microdochium (syn. Fusarium) nivaie (pink snow mold) on cereals (e. g. wheat or barley); Microsphaera diffusa (powdery mildew) on soybeans; MonHinias ., e. g. M. laxa, M. fructicola and M. fructigena (syn. Monilia spp.: bloom and twig blight, brown rot) on stone fruits and other rosaceous plants; Mycosphaereiia spp. on cereals, bananas, soft fruits and ground nuts, such as e. g. M. graminicoia (anamorph: Zymoseptoria tritic o cn&r Septoria tritici. Septoria blotch) on wheat or M. fijiensis (syn. Pseudocercospora fjiensis. black Sigatoka disease) and M. musicoia on bananas, M. arachidicoia (syn. M. arachidis o Cercospora arachidis), M. berkeieyi on peanuts, M. pision peas and M. brassicioia on brassicas; Peronospora spp. (downy mildew) on cabbage (e. g. P. brassicae), oilseed rape (e. g. P. parasitica), onions (e. g. P. destructor , tobacco P. tabacina) and soybeans (e. g. P. manshuricay Phakopsora pachyrhizi and P. meibomiae (soybean rust) on soybeans; Phiaiophora spp. e. g. on vines (e. g. P. tracheiphiia and P. tetraspora) and soybeans (e. g. P. gregata. stem rot); Phoma Hngam (syn. Leptosphaeria bigiobosa and L. macuiansr. root and stem rot) on oilseed rape and cabbage, P. betae (root rot, leaf spot and damping-off) on sugar beets and P. zeae-maydis (syn. Phyiiostica zeae) on corn; Phomopsis spp. on sunflowers, vines (e. g. P. viticoi can and leaf spot) and soybeans (e. g. stem rot: P. phaseoii, teleomorph: Diaporthe phaseoiorum)-, Physoderma maydis (brown spots) on corn; Phytophthoras^. (wilt, root, leaf, fruit and stem root) on various plants, such as paprika and cucurbits (e. g. P. capsici), soybeans (e. g. P. megasperma, syn. P. sojae), potatoes and tomatoes (e. g. P. infestans. late blight) and broad-leaved trees (e. g. P. ramorum. sudden oak death); Piasmodiophora brassicae (club root) on cabbage, oilseed rape, radish and other plants; Piasmopara spp., e. g. P. viticoia (grapevine downy mildew) on vines and P. haistedii on sunflowers; Podosphaera spp. (powdery mildew) on rosaceous plants, hop, pome and soft fruits (e. g. P. ieucotricha on apples) and curcurbits (P. xanthii)-, Polymyxa spp., e. g. on cereals, such as barley and wheat (P. graminis and sugar beets (P. betae) and thereby transmitted viral diseases; Pseudocercosporella herpotrichoides (syn. Oculimacula yallundae, O. acuformis. eyespot, teleomorph: Tapesia yallundae on cereals, e. g. wheat or barley; Pseudoperonospora (downy mildew) on various plants, e. g. P. cubensis on cucurbits or P. humili on hop; Pseudopezicula tracheiphila (red fire disease or ,rotbrenner’, anamorph: Phia/ophora) on vines; Puccinia spp. (rusts) on various plants, e. g. P. triticina (brown or leaf rust), P. striiformis (stripe or yellow rust), P. horde! (dwarf rust), P. graminis (stem or black rust) or P. recondita (brown or leaf rust) on cereals, such as e. g. wheat, barley or rye, P. kuehnii (orange rust) on sugar cane and P. asparagi on asparagus; Pyrenopeziza spp., e.g. P. brassicae on oilseed rape; Pyrenophora (anamorph: Drechsiera) tritici-repentis (tan spot) on wheat or P. teres (net blotch) on barley; Pyricuiaria spp., e. g. P. oryzae (teleomorph: Magnaporthe grisea. rice blast) on rice and P. grisea on turf and cereals; Pythium spp. (damping-off) on turf, rice, corn, wheat, cotton, oilseed rape, sunflowers, soybeans, sugar beets, vegetables and various other plants (e. g. P. ultim urn or P. aphanidermatum) and P. oiigandrum on mushrooms; Ram uiaria spp., e. g. R. coiio- cygni (Ramularia leaf spots, Physiological leaf spots) on barley, R. areola (teleomorph: Mycosphaereiia areola) on cotton and R. beticoia on sugar beets; Rhizoctonia spp. on cotton, rice, potatoes, turf, corn, oilseed rape, potatoes, sugar beets, vegetables and various other plants, e. g. R. so/ani (root and stem rot) on soybeans, R. so/ani (sheath blight) on rice or R. cereaiis (Rhizoctonia spring blight) on wheat or barley; Rhizopus stoionifer (black mold, soft rot) on strawberries, carrots, cabbage, vines and tomatoes; Rhynchosporium secaiis and R. commune (scald) on barley, rye and triticale; Sarociadium oryzae and S. attenuatum (sheath rot) on rice; Scierotinia spp. (stem rot or white mold) on vegetables (S. minor and S. scierotiorum and field crops, such as oilseed rape, sunflowers (e. g. S. scierotiorum and soybeans, S. roifsii (syn. Atheiia roifsii) on soybeans, peanut, vegetables, corn, cereals and ornamentals; Se tor/aspp. on various plants, e. g. S. glycines (brown spot) on soybeans, S. tritici (syn. Zymoseptoria tritici, Septoria blotch) on wheat and S. (syn. Stagonospora) nodorum (Stagonospora blotch) on cereals; L/ncinuia (syn. Erysiphe) necator (powdery mildew, anamorph: Oidium tuckeri) on vines; Setosphaeria spp. (leaf blight) on corn (e. g. S. turcicum, syn. Heiminthosporium turcicum) and turf; Sphaceiotheca spp. (smut) on corn, (e. g. S. reiiiana, syn. L/stiiago reiiiana. head smut), sorghum und sugar cane; Sphaerotheca fuiiginea (syn. Podosphaera xanthii powdery mildew) on cucurbits; Spongospora subterranea (powdery scab) on potatoes and thereby transmitted viral diseases; Stagonospora spp. on cereals, e. g. S. nodorum (Stagonospora blotch, teleomorph: Leptosphaeria [syn. Phaeosphaeria] nodorum, syn. Septoria nodorum on wheat; Synchytrium endobioticum on potatoes (potato wart disease); Taphrina spp., e. g. T. deformans (leaf curl disease) on peaches and T. pruni (plum pocket) on plums; Thie/aviopsis spp. (black root rot) on tobacco, pome fruits, vegetables, soybeans and cotton, e. g. T. basicola (syn. Chalara eiegans) Tilletia spp. (common bunt or stinking smut) on cereals, such as e. g. T. tritici (syn. T. caries, wheat bunt) and T. controversa (dwarf bunt) on wheat; Trichoderma harzianum on mushrooms, Typhu/a incarnata (grey snow mold) on barley or wheat; Urocystis spp., e. g. U. occulta (stem smut) on rye; Uromyces spp. (rust) on vegetables, such as beans (e. g. U. appendicu/atus, syn.
U. phaseoH), sugar beets (e. g. U. betae o U. beticoia and on pulses (e.g. U. vignae, U. pisi, U. viciae-fabae and U. fabae , UstHago spp. (loose smut) on cereals (e. g. U. nuda and U. avaenae), corn (e. g. U. maydis. corn smut) and sugar cane; Venturia spp. (scab) on apples (e. g. V. inaequaiis) and pears; and Verticiiiium spp. (wilt) on various plants, such as fruits and ornamentals, vines, soft fruits, vegetables and field crops, e. g. V. iongisporum on oilseed rape,
V. dahiiae on strawberries, oilseed rape, potatoes and tomatoes, and V. fungicoia on mushrooms; Zymoseptoria tritici on cereals.
The compounds I and compositions thereof, respectively, are particularly suitable for controlling the following causal agents of plant diseases: rusts on soybean and cereals (e.g. Phakopsora pachyrhizi and P. meibomiae on soy; Puccinia tritici and P. striiformis on wheat); molds on specialty crops, soybean, oil seed rape and sunflowers (e.g. Botrytis cinerea on strawberries and vines, Scierotinia scierotiorum, S. minor and S. roifsii on oil seed rape, sunflowers and soybean); Fusarium diseases on cereals (e.g. Fusarium cuimorum and F. graminearum on wheat); downy mildews on specialty crops (e.g. Piasmopara viticoia on vines, Phytophthora infestans on potatoes); powdery mildews on specialty crops and cereals (e.g. Uncinuia necator on vines, Erysiphe spp. on various specialty crops, Biumeria graminis on cereals); and leaf spots on cereals, soybean and corn (e.g. Septoria tritici and S. nodorum on cereals, S. glycines on soybean, Cercospora spp. on corn and soybean).
In an embodiment, the genetic result data further comprises information about the extent of the pathogen infestation, wherein the information about the extent of the infestation can be provided in different units/formats as long as this allows a quantification of the infestation, e.g. such as the amount of genetic material per biomass, amount of colony-forming units per biomass etc., i.e. any information which allows a direct or indirect quantified conclusion on the grade of infestation. The additional quantitative information on the extent to which the plants and/or plant parts, shown in the image data, are infested with a pathogen can further improve the quality of the training data. Moreover, this may make it possible that a machine learning algorithm, e.g. a deep learning algorithm, trained with such training data may also estimate the extent of infestation when analyzing image data, i.e. in this example, not only the information may be outputted whether and which pathogen infestation may be at hand, but also an estimation of the extent of the pathogen infestation may be outputted.
In an embodiment, the image data is based on at least one RGB-image of the plant and/or the plant part, wherein the at least one RGB-image is preferably taken against a white background. Such RGB-images can be generated by a standard camera or mobile phone. However, it could also be (i) a hyperspectral image, preferably in a radiation range between 390 nm and 1650 nm, more preferably up to 850 nm and/or (ii) a multispectral image, in which the information density is much higher. Thus, alternatively or in addition, the image data may be based on at least one hyperspectral and/or multispectral image. The latter is preferably in the radiation range between about 390 nm and about 850 nm, in the near infrared range with about 800 nm and in the far infrared range (FIR) with about 820 nm, wherein the at least one multispectral image is preferably taken against a white background. In an example, the multispectral image was taken with at least three spectral channels. In one example, one spectral channel is in a visible spectral range, in a near infrared range and in a far infrared range. In a further example, the radiation range is adopted to a specific pathogen or other parameters, like the absorption maxima values of chlorophylls, e.g. which his for chlorophyll a in the red region at 642 nm and in the blue region at 372 nm and for chlorophyll b the values are 626 nm in the red region and 392 nm in the blue region.
In an example, it is possible to base the image data on nine images of the plant or plant part, wherein it is preferred that the image data is based on six side views and three top views of the plant or plant part. However, the present disclosure is not limited to any specific numbers of images used for providing the image data.
In an embodiment, the method further comprises the step of filtering the image(s), trimming and/or marking specific areas of the image(s) before providing the image data. This makes it possible to limit an analysis /the training data to only the relevant parts of a plant. For example, plant pots as well as non-infested primary leaves may be excluded from the data set, so that only the infested parts of the plant are considered.
The infestation of a plant can be tested by any genetic methods. This involves testing for the presence and preferably the quantity of genetic material in plant samples that can be attributed to a pathogen. In other words, the term genetic methods is to be understood broadly and encompasses any method by which the presence of a pathogen can be determined and preferably any method by which also the extent of the pathogen may be determined. In an embodiment, the genetic result data is based on a DNA and/or a RNA analysis, like a sequencing method selected from the group of: PCR-based techniques, e.g. quantitative PCR (“qPCR”), isothermal amplification methods, e.g. loop-mediated amplification (“LAMP”), and/or a DNA analysis methods based on DNA-hybridization.
Over the recent years, promising DNA-analysis methods have become available, which may be applied in the context of the present disclosure: The loop-mediated isothermal amplification technology as first described in Nomi et al. (2000), Nucleic Acids Research, 28 (12), e63 does not require thermocycling equipment as conventionally needed for PCR analysis, and can be applied on-site, such as in the field. Several tests can be performed in parallel, e.g. in microarrays, and the analysis yields results within approximately 30 minutes.
Nanopore-sequencing, as described in Astier, Braha, and Bayley (2006) Journal of the American Chemical Society, 128 (5), 1705-10, or Fologea et al. (2005), Nano Lett. 5 (10), 1905-9., relies on the translocation of a DNA or RNA molecule through a nanopore. If an electric field is applied, an electric current can be observed, whose density across the nanopore surface depends on the nanopore*s dimensions and the composition of the nucleic acid that is passing through the nanopore. The pores are either provided by a protein, such as an alpha-hemolysin or a porin protein, or by solid state inorganic material. Nanopore-based analyses are fast, and several traits can be analyzed in parallel on the DNA- or RNA-leveL
DNA-hybridization approaches use single stranded DNA- or RNA-probe molecules of typically 10-50 bases, which are immobilized to a surface and which are reverse complementary to a given target sequence. The readout can be accomplished electronically, or via optical methods, e.g. fluorescence. DNA-hybridization approaches can be conducted in microarrays as described in Miller & Tang (2009), Clinical Microbiology Reviews, and it is thus possible to analyze a sample on many traits, i.e. genetic sequences that convey a phenotype, in parallel. In general, DNA- hybridization technologies are highly suitable for parallelization, e.g. in microarrays, and ready- to-use microarrays can be applied on-site with acceptable analysis times and equipment requirements.
In summary, a variety of different technologies has emerged as second or third generation DNA- analysis means that can be parallelized, e.g. in microarrays, and only require minor to acceptable equipment, thereby enabling an in-situ analysis of a sample. They can thus be assembled in sensor devices that are capable of analyzing a multitude of genetic sequences in a sample of a plant or plant propagation material. The present teachings can thus provide a method that can be carried out on-site, for example by a farmer or an agronomist, without any further shipping of samples. DNA-result data and information on pathogen infestation can thus be obtained within a timeframe of minutes rather than hours. The proposed teachings are particularly suitable for in-situ detection, i.e., measurement done essentially where the sample of the plant is collected. As used herein, the terms “in-situ” and “on-site” are equivalent and relate the situation that the measurement is performed essentially at the locus where the sample of the plant or the plant propagation material is collected and/or prepared. The same locus comprises both the immediate vicinity to the place where the sampling was carried out (e.g. within a radius of 1000 meters, such as within a radius of 100 meters), but may also relate to the functional or organizational unit where the sampling was carried out, such as a farm, a breeding station, a laboratory, a greenhouse etc.
Accordingly, DNA microarrays may be used in the present disclosure in which probe molecules are immobilized on a "chip". The sequence of the probe molecules is matched to the sequence of certain genes e.g. fungal pests. When a plant sample containing genetic material of a disease is applied to the chip, the pathogen DNA binds to the probe molecules and an optical or electrical signal may be generated. In particular, this technique can be used to test for certain fungicideresistant fungal diseases. The fungicide resistance is often generated by genetic mutations I variations. These are known for many resistances, so that corresponding probe molecules are created and can be immobilized on the chips. By means of such genetic testing of plant material, for example for a fungal disease, such diseases can be detected long before external symptoms become visible. With the proposed genetic screening, training data can be provided, which can include a quantitative statement in addition to the qualitative statement, so that with such high- quality training data a well-functioning machine learning algorithm, e.g. an image classification algorithm, can also be obtained.
In an embodiment, the pathogen is a fungus, preferably a Phakopsora pachyrhizi fungus, wherein in an example, the plant is a soybean plant or a part thereof and the pathogen is a Phakopsora pachyrhizi fungus causing Asian soybean rust disease.
A further aspect of the present disclosure refers to a use of image data of a plant or a plant part infested with a pathogen in a method explained above. A further aspect of the present disclosure refers to a use of DNA result data of a plant or a plant part comprising at least information about a pathogen infestation in a method explained above. A further aspect of the present disclosure refers to a use of training test data obtained according to a method explained above for training a machine learning algorithm, e.g. a neural network or a deep learning model, for classifying image data.
A further aspect of the present disclosure refers to a machine learning algorithm, preferably a neural network, trained with training data provided according to a method described above. The training features for training the machine learning algorithm, e.g. the neural network, preferably comprise the Normalized Differences Vegetation Index (NDVI) and the Greeness Index (G). The Normalized Difference Vegetation Index quantifies vegetation by measuring the difference between near-infrared, which vegetation strongly reflects, and red light, which vegetation absorbs.
A further aspect of the present disclosure refers to a classification system for image classifying a pathogen infestation of a plant, comprising: at least one input interface for providing image data of a plant or plant part; at least one evaluation unit which is set up to feed forward a machine learning algorithm with the image data of a plant or plant part, wherein the machine learning algorithm is trained on the basis of training data provided according to the method explained above; at least one output interface adapted to output a classification of a pathogen infestation of the plant. Such a system could be used by farmers to detect pathogen infestation, e.g. fungal infestation, before external signs become apparent. For example, the farmer can provide different images of a plant to be examined/classified. Moreover, such a system may also be used in the area of "greenhouse automation". Fungicides are tested in the greenhouse by treating infested plants and evaluating the results by means of a credit assessment. In an assessment, an employee manually evaluates the extent of damage to a plant and assigns it a percentage value. This process is naturally subjective and subject to strong fluctuations. Particularly in the case of active ingredient screenings with components that have not yet been optimized with regard to their fungicidal effect, it is difficult in this respect because the significance of the result for the given standard deviation does not always lead to a clear statement. Therefore, a classification system for image classifying a pathogen infestation of a plant may also be used in such application. In other words, the present disclosure provides the possibility that a farmer or an agronomic may take a picture or pictures of his field/plant, feed the trained machine learning algorithm and may then receive the information on pathogen infestation in-situ. Therefore, no shipping of a sample of the infected plant to a laboratory including a lag time in between measurement and information on infection status during which no treatment can occur can be entirely avoided.
A further aspect of the present disclosure refers to a computer-implemented method for image classifying a pathogen infestation of a plant, comprising the steps: providing image data of a plant or plant part; feeding a machine learning algorithm, e.g. a deep learning model, with the image data of a plant or plant part, wherein the machine learning algorithm is trained on the basis of training data provided according to the method explained above; outputting a classification of a pathogen infestation of the plant.
In an embodiment, the classification results of the pathogen infestation of the plant is used to provide a farmer a recommendation on how and with which active ingredient or crop protection product the infested plants may be treated. For example, a database comprising information about active ingredients and there field of application may be provided/used in which a respective query can be made in order to provide such a recommendation about the suitable active ingredient. In this respect, it is also possible that an automatic order for a suitable crop protection product is issued. In addition or as an alternative, it is also possible that based on the classification of the pathogen infestation other steps to be performed by a farmer are recommended in order to prevent a spread of the pathogen infestation in a field (e.g. use of a resistance crusher). In this respect, the present disclosure is not limited to a recommendation in view of the classification of the pathogen, but also further data may be considered (e.g. whether data, crop grow stage data, etc.).
In an embodiment, the classification of the pathogen infestation and/or the recommendation with respect to a treatment of the infested plants, an application map for an application device for applying a crop protection product on the field can be provided. In an example, it can be determined spatially distributed on a field whether and to what extent an infestation is present. Based on this data, it is possible to generate an application map for an application device (e.g. a sprayer, tractor, etc.) indicating/showing where and how much crop protection product is to be applied to the field. Moreover, it is also possible to generate control data for an application machine. Finally, a further aspect of the present disclosure relates to a computer program element which when executed by a processor is configured to carry out a computer-implemented method for image classifying a pathogen infestation of a plant. The computer program element might therefore be stored on a computer unit, which might also be part of an embodiment. This computing unit may be configured to perform or induce performing of the steps of the method described above. Moreover, it may be configured to operate the components of the above described apparatus and/or system. The computing unit can be configured to operate automatically and/or to execute the orders of a user. A computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method according to one of the preceding embodiments. This exemplary embodiment of the present disclosure covers both, a computer program that right from the beginning uses the present disclosure and computer program that by means of an update turns an existing program into a program that uses the present disclosure. Moreover, the computer program element might be able to provide all necessary steps to fulfill the procedure of an exemplary embodiment of the method as described above. According to a further exemplary embodiment of the present disclosure, a computer readable medium, such as a CD-ROM, USB stick or the like, is presented wherein the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section. A computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems. However, the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present disclosure, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the present disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
In the following, the present disclosure is described exemplarily with reference to the enclosed figure, in which
Figure 1 is a schematic view of a method according to the preferred embodiment of the present disclosure; and Figure 2 is an exemplary schematic image, which may be used as basis for providing the image data.
DETAILED DESCRIPTION OF EMBODIMENT
Figure 1 is a schematic view of a method according to the preferred embodiment of the present disclosure. In the following, an exemplary order of the steps according to the present disclosure is explained. However, the provided order is not mandatory, i.e. all or several steps may be performed in a different order or simultaneously.
The method described below can be summarized as follows. In a preparatory step, the plants to be analyzed, for example a soybean plant or a part thereof , are infected with a specific pathogen, for example causing Asian soybean rust. The infected plants or parts of these plants are then photographed to generate image data. Afterwards a genetic analysis with respect to the pathogen and preferably also with respect to the content of the pathogen of the photographed plant is determined, so that the image data can be combined with the results of the genetic analysis.
In a preferred embodiment, the image data is based on at least one RGB-image of the plant and/or the plant part, wherein the at least one RGB-image is preferably taken against a white background. Such RGB-images can be generated by a standard camera or mobile phone. However, it could also be a hyperspectral image and/or a multispectral image, in which the information density is much higher. Thus, alternatively or in addition, the image data may be based on at least one hyperspectral and/or multispectral image. The latter is preferably in the radiation range between about 390 nm and about 700 nm, in the near infrared range with about 800 nm and in the far infrared range (FIR) with about 820 nm, wherein the at least one multispectral image is preferably taken against a white background. In an example, the multispectral image was taken with at least three spectral channels. In one example, one spectral channel is in a visible spectral range, in a near infrared range and in a far infrared range. However, it is also possible to adopt the radiation range to a specific pathogen or other parameters, like the absorption maxima values of chlorophylls, e.g. which his for chlorophyll a in the red region at 642 nm and in the blue region at 372 nm and for chlorophyll b the values are 626 nm in the red region and 392 nm in the blue region. In a step S10, image data of a plant or a plant part infested with a pathogen are provided. The image data may be based on at least one RGB-image of the plant and/or the plant part, wherein the at least one RGB-image is preferably taken against a white background. Such RGB-images can be generated by a standard camera or mobile phone. However, it could also be a hyperspectral image and/or a multispectral image, in which the information density is much higher. Thus, alternatively or in addition, the image data may be based on at least one hyperspectral and/or multispectral image. The latter is preferably in the radiation range between about 390 nm and about 700 nm, in the near infrared range (NIR) with about 800 nm and in the far infrared range (FIR) with about 820 nm, wherein the at least one multispectral image is preferably taken against a white background. In an example, the multispectral image was taken with at least three spectral channels. In one example, one spectral channel is in a visible spectral range, in a near infrared range and in a far infrared range. It is further preferred that the image data is based on nine or more images of the plant or plant part, e.g. six side views and three top views of the plant or plant part. It is further preferred that the image(s) are filtered or trimmed or that specific areas of the image(s) are marked before providing the image data. This makes it possible to limit an analysis /the training data to only the relevant parts of a plant. In figure 2, such a marking of the image is shown excluding areas from the image data, which appear not relevant for the training data. In figure 2, a soybean plant 10 is shown, wherein the pot and the cotyledons (seed leaves) 12 were extracted and not included in the image extraction. Only the primary leaves 14, which have been infested were included in the training data. Trifoliate leaves emerged after inoculation had also been trimmed.
In an example, the digital imaging of the plants, e.g. the in figure 2 shown soybean plant, was carried out with an imaging box. The imaging box contained two CCD cameras, which took images in different wavelength ranges. For example, an RGB camera may took images of the plants from a side view (“lateral view”) and a multispectral camera may took images of the plant from above (“zenithal view”). The wavelengths of the multispectral camera are preferably in the red-green-blue or visible radiation range with 390 nm to 700 nm, in the near infrared range (NIR) with 800 nm and in the far infrared range (FIR) with 820 nm. However, it is also possible to use one or more hyperspectral images and/or to adopt the radiation range to a specific pathogen or other parameters, like the absorption maxima values of chlorophylls, e.g. which his for chlorophyll a in the red region at 642 nm and in the blue region at 372 nm and for chlorophyll b the values are 626 nm in the red region and 392 nm in the blue region.
In a step S20, genetic result data of the plant or the plant part to which the image data referred are provided. The genetic result data may be obtained by different genetic analysis methods. In this context, it is only important that the genetic result data are based on the plant or plant part referenced in the respective image data. For example, a genetic sequencing method, e.g. realtime quantitative PCR, loop-mediated isothermal amplification or a DNA microarray method. In an example, this involves testing for the presence and the quantity of genetic material in plant samples that can be attributed to a pathogen.
In a step S30, the image data are labeled with the genetic result data, e.g. the image data of the plant or plant part are tagged with the genetic result data. By means of such labeled data, different machine learning models, e.g. deep learning models, may be trained resulting in a very reliable classification algorithm for image classifying a pathogen infestation of a plant.
The present disclosure has been described in conjunction with a preferred embodiment as examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims. Notably, in particular the steps S10 to S30 can be performed in any order, i.e. the present invention is not limited to a specific order of these steps. Moreover, it is also not required that the different steps are performed at a certain place or at one place, i.e. each of the steps may be performed at a different place using different equipment/data processing units. In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.
REFERENCE SIGNS
S10 providing image data of a plant or a plant part infested with a pathogen
S20 providing genetic result data of the plant or the plant part to which the image data referred comprising at least information about type of pathogen
S30 labeling the image data with the genetic result data
10 soybean plant
12 cotyledones (seed leaves)
14 primary leaves
Next Patent: SWITCHING AN AC/DC CONVERTER TO A DC GRID