Research

Explainable Machine Learning

Machine learning methods have been remarkably successful for a wide range of application areas in the extraction of essential information from data.
An exciting and relatively recent development is the uptake of machine learning in the natural sciences, where the major goal is to obtain novel scientific insights and discoveries from observational or simulated data.
A prerequisite for obtaining a scientific outcome is domain knowledge, which is needed to gain explainability, but also to enhance scientific consistency. To approach explainable machine learning in the context of natural sciences, 3 core elements must be considered: transparency, interpretability, explainability.

Research partners:
Institute for Numerical Simulation, University of Bonn
Department of Electrical and Computer Engineering, University of Massachusetts Amherst

  • [PDF] R. Roscher, B. Bohn, M. F. Duarte, and J. Garcke, “Explainable machine learning for scientific insights and discoveries,” Ieee access, vol. 8, iss. 1, pp. 42200-42216, 2020.
    [Bibtex]
    @Article{Roscher2020explainable,
    Title = {Explainable Machine Learning for Scientific Insights and Discoveries},
    Author = {Roscher, Ribana and Bohn, Bastian and Duarte, Marco F. and Garcke, Jochen},
    Journal = {IEEE Access},
    Year = {2020},
    Number = {1},
    Pages = {42200-42216},
    Volume = {8},
    Date-added = {2019-05-26 21:15:07 +0200},
    Date-modified = {2019-05-26 21:22:58 +0200}
    }
  • [PDF] [DOI] J. Kierdorf, J. Garcke, J. Behley, T. Cheeseman, and R. Roscher, “What identifies a whale by its fluke? on the benefit of interpretable machine mearning for whale identification,” Isprs annals of photogrammetry, remote sensing and spatial information sciences, vol. V-2-2020, p. 1005–1012, 2020.
    [Bibtex]
    @Article{kierdorf2020whale,
    AUTHOR = {Kierdorf, J. and Garcke, J. and Behley, J. and Cheeseman, T. and Roscher, R.},
    TITLE = {What Identifies a Whale by its Fluke? On the Benefit of Interpretable Machine Mearning for Whale Identification},
    JOURNAL = {ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences},
    VOLUME = {V-2-2020},
    YEAR = {2020},
    PAGES = {1005--1012},
    URL = {https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/1005/2020/},
    DOI = {10.5194/isprs-annals-V-2-2020-1005-2020},
    Owner = {ribana},
    Timestamp = {2020.05.06}
    }
  • [PDF] [DOI] R. Roscher, B. Bohn, M. Duarte, and J. Garcke, “Explain it to me – facing remote sensing challenges in the bio- and geosciences with explainable machine learning,” Isprs annals of photogrammetry, remote sensing and spatial information sciences, vol. V-3-2020, p. 817–824, 2020.
    [Bibtex]
    @Article{roscher2020explain,
    AUTHOR = {Roscher, Ribana and Bohn, Bastian and Duarte, Marco and Garcke, Jochen},
    TITLE = {Explain it to me - Facing Remote Sensing Challenges in the Bio- and Geosciences with Explainable Machine Learning},
    JOURNAL = {ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences},
    VOLUME = {V-3-2020},
    YEAR = {2020},
    PAGES = {817--824},
    URL = {https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-3-2020/817/2020/},
    DOI = {10.5194/isprs-annals-V-3-2020-817-2020},
    Owner = {ribana},
    Timestamp = {2020.05.06}
    }


Machine Learning for Plant Phenotyping

Applications such as yield estimation and forecasting are of special interest in agriculture. Consequently, it is important for both scientists and farmers to study traits of the crop and the reaction to different stress factors. To this end, understanding the plant’s phenotype, i.e., the result of environmental influences such as abiotic and biotic stress factors occurring during plant’s development and growth on its genome, is essential.

Research partners:
All partners in the Cluster of Excellence PhenoRob
Julius Kühn-Institut, Siebeldingen

  • [PDF] L. Zabawa, A. Kicherer, L. Klingbeil, A. Milioto, R. Töpfer, H. Kuhlmann, and R. Roscher, “Detection of single grapevine berries in images using fully convolutional neural networks,” in Cvpr workshop on computer vision problems in plant phenotyping, 2019.
    [Bibtex]
    @InProceedings{zabawa2019,
    Title = {Detection of Single Grapevine Berries in Images Using Fully Convolutional Neural Networks},
    Author = {Zabawa, Laura and Kicherer, Anna and Klingbeil, Lasse and Milioto, Andres and T{\"o}pfer, Reinhard and Kuhlmann, Heiner and Roscher, Ribana},
    Booktitle = {CVPR Workshop on Computer Vision Problems in Plant Phenotyping},
    Year = {2019},
    Date-added = {2019-05-26 21:11:29 +0200},
    Date-modified = {2019-05-26 21:23:12 +0200}
    }
  • [DOI] A. Kicherer, R. Roscher, K. Herzog, W. Förstner, and R. Töpfer, “Image based evaluation for the detection of cluster parameters in grapevine,” in Acta horticulturae, 2014.
    [Bibtex]
    @InProceedings{Kicherer2014Evaluation,
    Title = {Image based Evaluation for the Detection of Cluster Parameters in Grapevine},
    Author = {Kicherer, Anna and Roscher, Ribana and Herzog, Katja and F\"orstner, Wolfgang and T\"opfer, Reinhard},
    Booktitle = {Acta horticulturae},
    Year = {2014},
    Bdsk-url-1 = {http://www.actahort.org/books/1082/1082_46.htm},
    Bdsk-url-2 = {https://doi.org/10.17660/ActaHortic.2015.1082.46},
    Doi = {10.17660/ActaHortic.2015.1082.46},
    Url = {http://www.actahort.org/books/1082/1082_46.htm}
    }
  • [PDF] [DOI] R. Roscher, K. Herzog, A. Kunkel, A. Kicherer, R. Töpfer, and W. Förstner, “Automated image analysis framework for high-throughput determination of grapevine berry sizes using conditional random fields,” Computers and electronics in agriculture, vol. 100, p. 148–158, 2014.
    [Bibtex]
    @Article{Roscher2014Automated,
    Title = {Automated image analysis framework for high-throughput determination of grapevine berry sizes using conditional random fields},
    Author = {Roscher, Ribana and Herzog, Katja and Kunkel, Annemarie and Kicherer, Anna and T{\"o}pfer, Reinhard and F{\"o}rstner, Wolfgang},
    Journal = {Computers and Electronics in Agriculture},
    Year = {2014},
    Pages = {148--158},
    Volume = {100},
    Bdsk-url-1 = {https://doi.org/10.1016/j.compag.2013.11.008},
    Doi = {10.1016/j.compag.2013.11.008},
    Publisher = {Elsevier}
    }


Machine Learning for Plant Disease Detection

plant disease detection

With a limited amount of arable land, increasing demand for food induced by growth in population can only be meet with more effective crop production and more resistant plants. Crop science is an important research field as it is concerned with the aspect of providing a sufficient amount of food – now and in the future. Since crop plants are exposed to many different stress factors, it is relevant to investigate those factors as well as their behavior and reactions. One of the most severe stress factors are diseases, resulting in a high loss of cultivated plants. In our research, we exploit various machine learning methods to solve application tasks such as forecasting and detection.

Research partners:
INRES-Pflanzenkrankheiten und Pflanzenschutz, University of Bonn
Julius Kühn-Institut, Siebeldingen

  • [PDF] A. Foerster, J. Behley, J. Behmann, and R. Roscher, “Hyperspectral plant disease forecasting using generative adversarial networks,” in International geoscience and remote sensing symposium, 2019.
    [Bibtex]
    @InProceedings{foerster2019,
    Title = {Hyperspectral plant disease forecasting using generative adversarial networks},
    Author = {Foerster, A. and Behley, J. and Behmann, J. and Roscher, R.},
    Booktitle = {International Geoscience and Remote Sensing Symposium},
    Year = {2019},
    Date-added = {2019-05-26 21:01:17 +0200},
    Date-modified = {2019-05-26 21:07:38 +0200}
    }
  • [PDF] L. Strothmann, U. Rascher, and R. Roscher, “Detection of anomalous grapevine berries using all-convolutional autoencoders,” in International geoscience and remote sensing symposium, 2019.
    [Bibtex]
    @InProceedings{strothmann2019,
    Title = {Detection of anomalous grapevine berries using all-convolutional autoencoders},
    Author = {Strothmann, Laurenz and Rascher, Uwe and Roscher, Ribana},
    Booktitle = {International Geoscience and Remote Sensing Symposium},
    Year = {2019},
    Date-added = {2019-05-26 21:09:56 +0200},
    Date-modified = {2019-05-26 21:25:51 +0200}
    }
  • [DOI] R. Roscher, J. Behmann, A. Mahlein, J. Dupuis, H. Kuhlmann, and L. Plümer, “Detection of disease symptoms on hyperspectral 3D plant models,” in ISPRS annals of photogrammetry, remote sensing and spatial information sciences, 2016, p. 89–96.
    [Bibtex]
    @InProceedings{Roscher2016detection,
    Title = {Detection of Disease Symptoms on Hyperspectral {3D} Plant Models},
    Author = {Roscher, Ribana and Behmann, Jan and Mahlein, Anne-Kathrin and Dupuis, Jan and Kuhlmann, Heiner and Pl{\"u}mer, Lutz},
    Booktitle = {{ISPRS} Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences},
    Year = {2016},
    Pages = {89--96},
    Abstract = {We analyze the benefit of combining hyperspectral images information with 3D geometry information for the detection of Cercospora leaf spot disease symptoms on sugar beet plants. Besides commonly used one-class Support Vector Machines, we utilize an unsupervised sparse representation-based approach with group sparsity prior. Geometry information is incorporated by representing each sample of interest with an inclination-sorted dictionary, which can be seen as an 1D topographic dictionary. We compare this approach with a sparse representation based approach without geometry information and One-Class Support Vector Machines. One-Class Support Vector Machines are applied to hyperspectral data without geometry information as well as to hyperspectral images with additional pixelwise inclination information. Our results show a gain in accuracy when using geometry information beside spectral information regardless of the used approach. However, both methods have different demands on the data when applied to new test data sets. One-Class Support Vector Machines require full inclination information on test and training data whereas the topographic dictionary approach only need spectral information for reconstruction of test data once the dictionary is build by spectra with inclination.},
    Bdsk-url-1 = {https://www.researchgate.net/publication/303975671_On_the_Benefit_of_Topographic_Dictionaries_for_Detecting_Disease_Symptoms_on_Hyperspectral_3D_Plant_Models},
    Bdsk-url-2 = {https://doi.org/10.5194/isprs-annals-III-7-89-2016},
    Doi = {10.5194/isprs-annals-III-7-89-2016},
    Url = {https://www.researchgate.net/publication/303975671_On_the_Benefit_of_Topographic_Dictionaries_for_Detecting_Disease_Symptoms_on_Hyperspectral_3D_Plant_Models}
    }
  • [DOI] R. Roscher, J. Behmann, A. Mahlein, and L. Plümer, “On the benefit of topographic dictionaries for detecting disease symptoms on hyperspectral 3D plant models,” in Workshop on hyperspectral image and signal processing, 2016.
    [Bibtex]
    @InProceedings{Roscher2016Topographic,
    Title = {On the Benefit of Topographic Dictionaries for Detecting Disease Symptoms on Hyperspectral {3D} Plant Models},
    Author = {Roscher, Ribana and Behmann, Jan and Mahlein, Anne-Kathrin and Pl{\"u}mer, Lutz},
    Booktitle = {Workshop on Hyperspectral Image and Signal Processing},
    Year = {2016},
    Abstract = {We analyze the benefit of using topographic dictionaries for a sparse representation (SR) approach for the detection of Cercospora leaf spot disease symptoms on sugar beet plants. Topographic dictionaries are an arranged set of basis elements in which neighbored dictionary elements tend to cause similar activations in the SR approach. In this paper, the dictionary is obtained from samples of a healthy plant and partly build in a topographic way by using hyperspectral as well as geometry information, i.e. depth and inclination. It turns out that hyperspectral signals of leafs show a typical structure depending on depth and inclination and thus, both influences can be disentangled in our approach. Rare signals which do not fit into this model, e.g. leaf veins, are also captured in the dictionary in a non-topographic way. A reconstruction error index is used as indicator, in which disease symptoms can be distinguished from healthy plant regions.nThe advantage of the presented approach is that full spectral and geometry information is needed only once to built the dictionary, whereas the sparse reconstruction is done solely on hyperspectral information.},
    Bdsk-url-1 = {http://ieeexplore.ieee.org/document/8071690/},
    Bdsk-url-2 = {https://doi.org/10.1109/WHISPERS.2016.8071690},
    Doi = {10.1109/WHISPERS.2016.8071690},
    Url = {http://ieeexplore.ieee.org/document/8071690/}
    }


Multi-Modal Neural Networks for Analyzing Earth Observational Data

Global climate change plays an essential role in our daily life and is one of the most important topics, nowadays. Thus, the understanding, monitoring and prediction is essential to overcome related challenges. Neural networks are a powerful mean to solve tasks such as classification, detection and regression, where they show promising results reaching accuracies superior to classical and shallow state-of-art machine learning algorithms. However, so far they have been barely explored in the context of Earth observation data. In our research, we use the current advances in the deep learning area and adapt the methods for several applications in remote sensing.

Research partners:
Satellite Geodesy Group, University of Bonn

  • [PDF] [DOI] L. Drees, J. Kusche, and R. Roscher, “Multi-modal deep learning with sentinel-3 observations for the detection of oceanic internal waves,” Isprs annals of photogrammetry, remote sensing and spatial information sciences, vol. V-2-2020, p. 813–820, 2020.
    [Bibtex]
    @Article{drees2020multimodal,
    AUTHOR = {Drees, Lukas and Kusche, Jürgen and Roscher, Ribana},
    TITLE = {Multi-Modal Deep Learning with Sentinel-3 Observations for the Detection of Oceanic Internal Waves},
    JOURNAL = {ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences},
    VOLUME = {V-2-2020},
    YEAR = {2020},
    PAGES = {813--820},
    URL = {https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/813/2020/},
    DOI = {10.5194/isprs-annals-V-2-2020-813-2020},
    Owner = {ribana},
    Timestamp = {2020.05.06}
    }
  • [PDF] K. Franz, R. Roscher, A. Milioto, S. Wenzel, and J. Kusche, “Ocean eddy identification and tracking using neural networks,” in IEEE international geoscience and remote sensing symposium (IGARSS), 2018.
    [Bibtex]
    @InProceedings{Franz2018,
    Title = {Ocean Eddy Identification and Tracking using Neural Networks},
    Author = {Franz, Katharina and Roscher, Ribana and Milioto, Andres and Wenzel, Susanne and Kusche, J{\"u}rgen},
    Booktitle = {{IEEE} International Geoscience and Remote Sensing Symposium ({IGARSS})},
    Year = {2018},
    Bdsk-url-1 = {https://arxiv.org/pdf/1803.07436.pdf},
    Url = {https://arxiv.org/pdf/1803.07436.pdf}
    }
  • [PDF] [DOI] A. Braakmann-Folgmann, R. Roscher, S. Wenzel, B. Uebbing, and J. Kusche, “Sea level anomaly prediction using recurrent neural networks,” in Proc. conference on big data from space, 2017, pp. 297-300.
    [Bibtex]
    @InProceedings{Braakmann-Folgmann2017,
    Title = {Sea level anomaly prediction using recurrent neural networks},
    Author = {Braakmann-Folgmann, Anne and Roscher, Ribana and Wenzel, Susanne and Uebbing, Bernd and Kusche, J\"urgen},
    Booktitle = {Proc. Conference on Big Data from Space},
    Year = {2017},
    Pages = {297-300},
    Abstract = {Sea level change, one of the most dire impacts of anthropogenic global warming, will affect a large amount of the world's population. However, sea level change is not uniform in time and space, and the skill of conventional prediction methods is limited due to the ocean's internal variabi-lity on timescales from weeks to decades. Here we study the potential of neural network methods which have been used successfully in other applications, but rarely been applied for this task. We develop a combination of a convolutional neural network (CNN) and a recurrent neural network (RNN) to analyse both the spatial and the temporal evolution of sea level and to suggest an independent, accurate method to predict interannual sea level anomalies (SLA). We test our method for the northern and equatorial Pacific Ocean, using gridded altimeter-derived SLA data. We show that the used network designs outperform a simple regression and that adding a CNN improves the skill significantly. The predictions are stable over several years.},
    Bdsk-url-1 = {http://publications.jrc.ec.europa.eu/repository/bitstream/JRC108361/jrc180361_procbids17.pdf},
    Bdsk-url-2 = {https://doi.org/10.2760/383579},
    Doi = {10.2760/383579},
    Url = {http://publications.jrc.ec.europa.eu/repository/bitstream/JRC108361/jrc180361_procbids17.pdf}
    }
  • [PDF] [DOI] R. Roscher, B. Uebbing, and J. Kusche, “STAR: spatio-temporal altimeter waveform retracking using sparse representation and conditional random fields,” Remote sensing of environment, vol. 201, p. 148–164, 2017.
    [Bibtex]
    @Article{Roscher2017STAR,
    Title = {{STAR}: Spatio-Temporal Altimeter Waveform Retracking using Sparse Representation and Conditional Random Fields},
    Author = {Roscher, Ribana and Uebbing, Bernd and Kusche, J\"urgen},
    Journal = {Remote Sensing of Environment},
    Year = {2017},
    Pages = {148--164},
    Volume = {201},
    Bdsk-url-1 = {https://doi.org/10.1016/j.rse.2017.07.024},
    Doi = {10.1016/j.rse.2017.07.024}
    }


Deep Self-taught learning

Self-taught learning by sparse representation

Self-taught learning (STL) has become a promising paradigm to exploit large amounts of unlabeled data for feature learning and classification. It utilizes both labeled and unlabeled data without the requirement that both sets have to share the same distribution and the same land use and land cover classes. In our research, we analyze the applicability of STL for the interpretation of remote sensing images. Moreover, we combine STL and the concepts of deep learning to learn valuable feature representations for remote sensing image classification.

  • [PDF] [DOI] A. Bettge, R. Roscher, and S. Wenzel, “Deep self-taught learning for remote sensing image classification,” in Proc. conference on big data from space, 2017, pp. 301-304.
    [Bibtex]
    @InProceedings{Bettge2017,
    Title = {Deep self-taught learning for remote sensing image classification},
    Author = {Bettge, Anika and Roscher, Ribana and Wenzel, Susanne},
    Booktitle = {Proc. Conference on Big Data from Space},
    Year = {2017},
    Pages = {301-304},
    Abstract = {This paper addresses the land cover classification task for remote sensing images by deep self-taught learning. Our self-taught learning approach learns suitable feature representations of the input data using sparse representation and undercomplete dictionary learning. We propose a deep learning framework which extracts representations in multiple layers and use the output of the deepest layer as input to a classification algorithm. We evaluate our approach using a multispectral Landsat 5 TM image of a study area in the North of Novo Progresso (South America) and the Zurich Summer Data Set provided by the University of Zurich. Experiments indicate that features learned by a deep self-taught learning framework can be used for classification and improve the results compared to classification results using the original feature representation.},
    Bdsk-url-1 = {http://publications.jrc.ec.europa.eu/repository/bitstream/JRC108361/jrc180361_procbids17.pdf},
    Bdsk-url-2 = {https://doi.org/10.2760/383579},
    Doi = {10.2760/383579},
    Url = {http://publications.jrc.ec.europa.eu/repository/bitstream/JRC108361/jrc180361_procbids17.pdf}
    }
  • [DOI] R. Roscher, S. Wenzel, and B. Waske, “Discriminative archetypal self-taught learning for multispectral landcover classification,” in IAPR workshop on pattern recogniton in remote sensing (PRRS), 2016, p. 1–5.
    [Bibtex]
    @InProceedings{roscher2016discriminative,
    Title = {Discriminative archetypal self-taught learning for multispectral landcover classification},
    Author = {Roscher, Ribana and Wenzel, Susanne and Waske, Bj\"orn},
    Booktitle = {{IAPR} Workshop on Pattern Recogniton in Remote Sensing ({PRRS})},
    Year = {2016},
    Pages = {1--5},
    Bdsk-url-1 = {http://ieeexplore.ieee.org/document/7867022/},
    Bdsk-url-2 = {https://doi.org/10.1109/PRRS.2016.7867022},
    Doi = {10.1109/PRRS.2016.7867022},
    Url = {http://ieeexplore.ieee.org/document/7867022/}
    }
  • [DOI] R. Roscher, C. Römer, B. Waske, and L. Plümer, “Landcover classification with self-taught learning on archetypal dictionaries,” in IEEE international geoscience and remote sensing symposium (IGARSS), 2015, p. 2358–2361.
    [Bibtex]
    @InProceedings{Roscher2015Selftaught,
    Title = {Landcover classification with self-taught learning on archetypal dictionaries},
    Author = {Roscher, Ribana and R\"omer, Christoph and Waske, Bj\"orn and Pl\"umer, Lutz},
    Booktitle = {{IEEE} International Geoscience and Remote Sensing Symposium ({IGARSS})},
    Year = {2015},
    Pages = {2358--2361},
    Bdsk-url-1 = {https://doi.org/10.1109/IGARSS.2015.7326282},
    Doi = {10.1109/IGARSS.2015.7326282}
    }