Journal Article

Fossil image identification using deep learning ensembles of data augmented multiviews

Hou, Chengbin, Shi, Yukun, Lin, Xinyu, Huang, Hanhui, Lv, Hairong, Xu, Sheng, Fan, Junxuan

2023

Overview

Identification of fossil species is crucial to evolutionary studies. Recent advances from deep learning have shown promising prospects in fossil image identification. However, the quantity and quality of labelled fossil images are often limited due to fossil preservation, conditioned sampling and expensive and inconsistent label annotation by domain experts, which pose great challenges to training deep learning‐based image classification models. To address these challenges, we follow the idea of the wisdom of crowds and propose a multiview ensemble framework, which collects Original (O), Grey (G) and Skeleton (S) views of each fossil image reflecting its different characteristics to train multiple base models, and then makes the final decision via soft voting. Experiments on the largest fusulinid dataset with 2400 images show that the proposed OGS consistently outperforms baselines (using a single model for each view), and obtains superior or comparable performance compared to OOO (using three base models for three the same Original views). Besides, as the training data decreases, the proposed framework achieves more gains. While considering the identification consistency estimation with respect to human experts, OGS receives the highest agreement with the original labels of dataset and with the re‐identifications of two human experts. The validation performance provides a quantitative estimation of consistency across different experts and genera. We conclude that the proposed framework can present state‐of‐the‐art performance in the fusulinid fossil identification case study. This framework is designed for general fossil identification and it is expected to see applications to other fossil datasets in future work. Notably, the result, which shows more performance gains as train set size decreases or over a smaller imbalance fossil dataset, suggests the potential application to identify rare fossil images. The proposed framework also demonstrates its potential for assessing and resolving inconsistencies in fossil identification. 摘 要 化石物种的鉴定对进化研究学至关重要。近年来，深度学习在化石图像识别方面的研究进展表现出了广阔的前景。然而，由于化石保存和采样的限制，以及领域专家较少、鉴定结果具有不一致性，已标记的化石图像其数量与质量往往受到限制，这给基于深度学习的图像分类模型的训练带来了很大的挑战。 为了应对这些挑战，我们遵循群体智慧的思想，提出了一种多视角集成学习框架，该框架收集了每张化石图片的原图(O)、 灰度图(G)和骨架图(S)，从多个视角提取每张图像的不同特征，分别训练相应的基模型，然后通过软投票做出最终决策。 在目前最大的fusulinid图像数据集(共2400张图像)上的实验结果表明，集成OGS三个视角的模型性能始终优于使用单一视角模型，并且与集成OOO三个相同视角的模型相比，获得了更好或相当的性能。此外，随着用于训练数据的减少，所提框架的性能增益也越大。对于人类专家的识别一致性评估表明，OGS在数据集原始标签以及两位人类专家重新识别标签的两种场景下，都获得了最高的一致性。模型验证集的性能定量地评估了不同专家或不同的属之间一致性。 我们得出的结论是，本文所提出的方法可以在fusulinid化石鉴定案例研究中呈现最先进的性能。该方法是为一般化石识别而设计的，并有望在未来的工作中应用于其他化石数据集。值得注意的是，研究结果表明，OGS在较小或较不平衡化石数据集上显示出了更明显的性能提升，因而在识别稀有化石图像上有较高的潜在应用价值。此外，本文所提出的方法也具有评估和解决化石鉴定不一致性方面的潜力。

• Publisher: John Wiley & Sons, Inc,Wiley
• Subject: Accuracy / Annotations / Consistency / Datasets / Deep learning / ensemble / fossil identification / Fossils / fusulinid / Genera / Identification / identification inconsistency / Image classification / Image quality / Labels / Machine learning / Morphology / Neural networks / palaeoecology / Taxonomy