Journal Article

Manifold and spatiotemporal learning on multispectral unoccupied aerial system imagery for phenotype prediction

Huggins, Trevis D., Edwards, Jeremy D., McClung, Anna M., Bellis, Emily S., Hashem, Ahmed A., Farag, Fared, Causey, Jason L.

2024

Overview

Timeseries data captured by unoccupied aircraft systems (UASs) are increasingly used for agricultural applications requiring accurate prediction of plant phenotypes from remotely sensed imagery. However, prediction models often fail to generalize well from one year to the next or to new environments. Here, we investigate the ability of various machine learning (ML) approaches to improve yield prediction accuracy in new environments from multispectral timeseries imagery acquired on a set of rice (Oryza sativa L.) experiments with different management treatments and varieties. We also trained deep learning models that perform automated feature extraction and compared these against a suite of other approaches. We observed similar performance on a held‐out growing season for a spatiotemporal model (a three‐dimensional convolutional neural network) trained on raw images compared to simpler workflows using dimension reduction of manually extracted features from temporal imagery (i.e., vegetation indices and image texture properties). Manifold learning on raw imagery was better suited for the prediction of phenological traits due to the preservation of local structure in image embeddings at some time points. Together, these results highlight the competitiveness of classical ML approaches for UAS image analysis alongside computationally expensive deep learning models. Along with a new benchmark dataset for rice, our results help extend the toolkit for UAS image analysis, contributing to improved phenotype prediction in plant breeding and precision agriculture applications. Core Ideas Linear yield prediction models based on multitemporal drone imagery transfer poorly to new growing seasons. Workflows using nonlinear models improve generalization to new growing seasons. Manifold learning shows promise for developmental trajectory inference and phenological traits. A new public benchmark dataset of multitemporal UAS imagery for rice is made available.

• Publisher: John Wiley & Sons, Inc,Wiley
• Subject: Agricultural production / Agricultural research / aircraft / automation / Competitiveness / Crop yield / Cultivars / data collection / Datasets / Deep learning / Genetic diversity / Hypotheses / image analysis / Image processing / Machine learning / Neural networks / New varieties / Nitrogen / Observational learning / Oryza sativa / phenology / phenotype / Phenotypes / Plant breeding / precision agriculture / prediction / Prediction models / Remote sensing / Rice / Seasons / texture / time series analysis / Vegetation / yield forecasting