This work highlights the medicinal flora, which is very essential for the conservation of biodiversity and the improvement of health throughout the world. More specifically, it underlines the need for accurate classification of medicinal plant species for their effective conservation and proper use. The complexity of plant features and a lack of annotated datasets make them difficult for traditional classification methods. To address this issue, a deep learning-based framework is proposed in the research for classifying images related to medicinal plants using convolutional neural networks (CNNs). In this framework, a CNN architecture with residual and inverted residual block configurations is selected, and a set of data augmentation is applied to improve the dataset. Concerning feature selection, it adopts Binary Chimp Optimization and serial feature fusion regarding accuracy and speed. Experiments show that the proposed framework significantly outperforms conventional methods in the accurate classification of medicinal flora, and it suggests possible extensions for the identification of other plant species. This study provides evidence of the potential that deep learning models have in improving and automating identification and classification procedures for medicinal plants when integrated with botanical studies.