Disentangling the Effects of Data Augmentation and Fine-Tuning in Herbal Plant Image Classification

Abstract

Image-based herbal plant identification requires reliable evaluation to distinguish improvements attributable to model architecture, data augmentation, and fine-tuning. This study investigated these factors through a controlled comparison of EfficientNetV2-B0, ResNet-50, and MobileNetV3-Small using 3,500 images from ten herbal plant classes. An image audit combined exact-content hashing, perceptual-similarity screening, and recorded pair decisions to construct image groups before partitioning. The resulting training, validation, and test sets contained 2,450, 530, and 520 images, respectively, with no detected overlap between recorded groups or identical image content. Each architecture was evaluated under four scenarios combining augmentation and backbone fine-tuning, with three random seeds, yielding 36 experimental runs. Performance was assessed using classification metrics, paired ablation contrasts, confusion matrices, parameter counts, and inference latency. Fine-tuning consistently improved macro-F1, whereas augmentation reduced performance with frozen backbones and produced modest improvements after fine-tuning. ResNet-50 with augmentation and fine-tuning achieved the highest mean accuracy of 99.29% and macro-F1 of 99.30%, although its numerical advantage over EfficientNetV2-B0 did not establish statistical superiority. MobileNetV3-Small achieved 98.47% macro-F1 with approximately 1.53 million parameters, supporting further investigation for resource-constrained applications. These findings demonstrate that augmentation effectiveness depends on the adaptation strategy and provide guidance for selecting herbal-image classifiers.