Deep Learning-Based Panicle Detection by Using Hyperspectral Imagery

Ruya Xu, Purdue University

Abstract

Sorghum, which is grown internationally as a cereal crop that is robust to heat, drought, and disease, has numerous applications for food, forage, and biofuels. When monitoring the growth stages of sorghum, or phenotyping specific traits for plant breeding, it is important to identify and monitor the panicles in the field due to their impact relative to grain production. Several studies have focused on detecting panicles based on data acquired by RGB and multispectral remote sensing technologies. However, few experiments have included hyperspectral data because of its high dimensionality and computational requirements, even though the data provide abundant spectral information. Relative to analysis approaches, machine learning, and specifically deep learning models have the potential of accommodating the complexity of these data. In order to detect panicles in the field with different physical characteristics, such as colors and shapes, very high spectral and spatial resolution hyperspectral data were collected with a wheeled-based platform, processed, and analyzed with multiple extensions of the VGG-16 Fully Convolutional Network (FCN) semantic segmentation model. In order to have correct positioning, orthorectification experiments were also conducted in the study to obtain the proper positioning of the image data acquired by the pushbroom hyperspectral camera at near range. The scale of the DSM derived from LiDAR that was used for orthorectification of the hyperspectral data was determined to be a critical issue, and the application of the Savitzky-Golay filter to the original DSM data was shown to contribute to the improved quality of the orthorectified imagery. Three tuned versions of the VGG-16 FCN Deep Learning architecture were modified to accommodate the hyperspectral data: PCA&FCN, 2D-FCN, and 3D-FCN. It was concluded that all the three models can detect the late season panicles included in this study, but the end-to-end models performed better in terms of precision, recall, and the F-score metrics . Future work should focus on improving annotation strategies and the model architecture to detect different panicle varieties and to separate overlapping panicles based on an adequate quantities of training data acquired during the flowering stage.

Degree

M.Sc.

Advisors

Crawford, Purdue University.

Subject Area

Remote sensing|Aerospace engineering|Agronomy|Artificial intelligence|Computer science|Food Science|Logic|Robotics|Transportation

COinS