weed detection research paper

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Sensors (Basel)

Review of Weed Detection Methods Based on Computer Vision

Zhangnan wu.

1 Department of Information Science, Xi’an University of Technology, Xi’an 710048, China; nc.ude.tuax.uts@3000280912 (Z.W.); nc.ude.tuax@bxgnak (X.K.); nc.ude.tuax.uts@0100280912 (Y.D.)

2 Chinese Academy of Agricultural Mechanization Sciences, Beijing 100083, China; moc.621@ihsoboahz

Xiaobing Kang

Yuanyuan ding, associated data.

Not applicable.

Weeds are one of the most important factors affecting agricultural production. The waste and pollution of farmland ecological environment caused by full-coverage chemical herbicide spraying are becoming increasingly evident. With the continuous improvement in the agricultural production level, accurately distinguishing crops from weeds and achieving precise spraying only for weeds are important. However, precise spraying depends on accurately identifying and locating weeds and crops. In recent years, some scholars have used various computer vision methods to achieve this purpose. This review elaborates the two aspects of using traditional image-processing methods and deep learning-based methods to solve weed detection problems. It provides an overview of various methods for weed detection in recent years, analyzes the advantages and disadvantages of existing methods, and introduces several related plant leaves, weed datasets, and weeding machinery. Lastly, the problems and difficulties of the existing weed detection methods are analyzed, and the development trend of future research is prospected.

1. Introduction

At present, many smart agriculture tasks, such as plant disease detection, crop yield prediction, species identification, weed detection, and water and soil conservation, are realized through computer vision technology [ 1 , 2 , 3 ]. Weed control is an important means to improve crop productivity. Considerable literature has proposed precise variable spraying methods to prevent waste and herbicide residual problems caused by the traditional full-coverage spraying [ 4 ]. To achieve precise variable spraying, a key issue that should be solved is how to realize real-time precise detection and identification of crops and weeds.

Methods for realizing field weed detection by using computer vision technology mainly include traditional image processing and deep learning. When weed detection is conducted with traditional image-processing technology, extracting features, such as color, texture, and shape, of the image and combining with traditional machine learning methods, such as random forest or Support Vector Machine (SVM) algorithm, for weed identification are necessary [ 5 ]. These methods need to design features manually and have high dependence on image acquisition methods, preprocessing methods, and the quality of feature extraction. With the improvement in computing power and the increase in data volume, deep learning algorithms can extract multiscale and multidimensional spatial semantic feature information of weeds through Convolutional Neural Networks (CNNs) due to their enhanced data expression capabilities for images, avoiding the disadvantages of traditional extraction methods. Therefore, they have attracted increasing attention from researchers.

Several reviews on the application of machine learning in agriculture [ 6 ] and an overview of using deep learning methods to achieve agricultural tasks have been presented [ 7 ]. They have either provided a comprehensive overview of the methods applied in the entire agricultural field [ 8 ] or conducted the latest research on a certain type of technology for a specific task [ 9 ]. For example, Koirala et al. [ 10 ] summarized the application of deep learning in fruit detection and yield estimation, including the problem and solution to fruit being occluded in imaging. However, they focused only on detection and yield estimation and disregarded other agricultural tasks that contain a large number of objects, such as weed detection. Kamilaris et al. [ 7 ] reviewed the application of deep learning in agriculture, involving many studies in the fields of weed identification, land cover classification, plant identification, fruit counting, and crop type classification. Nevertheless, it was only a summary of the current situation in weed detection. Yuan et al. [ 11 ] elucidated the research progress in field weed identification at home and abroad and the advantages and disadvantages of various segmentation, extraction, and identification methods. Nonetheless, few discussions were presented about the use of deep learning methods to solve the problem of weed identification. Hasan et al. [ 12 ] provided a comprehensive review of weed detection and classification research but focused on methods based on deep learning.

Traditional and deep learning-based weed detection methods have their own advantages. Traditional weed detection methods require small sample sizes, have low requirements on graphics processing units, and can be used in agricultural machinery and equipment at a low cost. This paper mainly reviews the related methods for weed detection in recent years from the perspectives of traditional machine learning (ML) methods and deep learning and briefly discusses the pros and cons of the methods. The datasets of weed identification and detection and leaf classification are summarized, and the problems faced in field weed detection under different conditions are analyzed. This paper provides a certain reference to other scholars to further their research on weed detection algorithms based on computer vision and achieve intelligent weed control and related areas of research and application.

2. Public Image Datasets

Many public and annotated image datasets are available in the field of computer vision, such as ImageNet [ 13 ], COCO [ 14 ], Pascal VOC [ 15 ], and Open Images [ 16 ]. The use of these datasets enables the effective evaluation of the performance of object detection, classification, and segmentation algorithms. Although the kinds and quantities of these datasets are considerable, these datasets are mainly composed of natural scenes and network images and cannot be directly applied to precision agricultural visual tasks. In the study of the method of using computer vision technology to detect weeds, field weed image datasets are critical for the construction of an algorithm and the test of its effect. In fact, public plant image datasets that can be used for precision agriculture tasks should be based on plants or their leaves, but few public datasets meet this requirement [ 17 ]. Researchers face a series of problems, such as few databases and poor algorithm mobility. When researchers use different datasets for specific weed detection algorithms, evaluating different methods on the basis of the results of published literature is difficult or impossible. As computer vision and machine learning continue to impact agriculture, the number of public image datasets designated for specific agriculture tasks has gradually increased since 2015, effectively promoting the development of computer vision technology in precision agriculture. Table 1 lists several common datasets related to the field of weed detection and identification. Part of datasets contain leaf-level ground truth or pixel-level annotations, which can be widely used for weed detection, species identification, and leaf segmentation. The publication of increasing standard datasets will help further break the bottleneck of algorithm research on weed detection tasks.

Public weed image datasets and their features.

Figure 1 shows four typical plant dataset images, representing different situations: (a) demonstrates the images of a target plant segmented from a cluttered background, (b) presents plant leaves with a white background, (c) shows unsegmented maize, and (d) depicts crops and weeds on land.

Four typical plant datasets: ( a ) Grass-Broadleaf database [ 19 ], with images of soybean, broadleaf weed, grass, and soil; ( b ) Flavia dataset [ 28 ]; ( c ) plant seedlings dataset [ 20 ]; ( d ) food crops and weeds dataset [ 26 ].

Table 2 further compares the results of different methods under the same dataset. The comparison results of the three methods are listed under each typical dataset. It can be seen that with the continuous development of the algorithm, the accuracy is getting higher and higher.

Comparison of different methods under the same typical dataset.

3. Traditional Machine Learning Weed Detection Methods

In the early stage, many scholars used machine learning algorithms combined with image features to conduct weed recognition tasks, achieving the purpose of weed detection. These traditional ML methods require a small sample size and short training time; they also have a low requirement for graphics processing units. They can be used in agricultural machinery and equipment at a low cost, providing an effective method and approach for realizing plant identification and weed detection based on image-processing technology.

These intelligent technologies rely on the continuous development of machine vision technology. Machine vision technology uses a series of image-processing methods to extract the shallow features of weeds and then sends them to a classifier for detection. Initially, crops or weeds are identified by calculating the texture, shape, color, or spectral features of images. For example, Le et al. [ 38 ] realized the distinction between corn and single species of weeds on the basis of Local Binary Pattern (LBP) texture features and SVM. Chen et al. [ 39 ] proposed a multi-feature weed reverse location method in a soybean field on the basis of shape and color features. Zhu et al. [ 40 ] proposed a classification method for five kinds of weeds in farmland on the basis of shape and texture. Zhang et al. [ 41 ] conducted a comparative analysis of the gray distribution of each component in the color space of RGB, HSV, and HIS of common weeds in a field at the pea seedling stage. They proposed a method for weed segmentation and extraction in complex background based on R-B color difference features. Some scholars have used plant height [ 42 ] or location information [ 43 , 44 , 45 ] to improve the identification accuracy, but these methods are easily affected by vibration or other uncontrolled motion in practical application [ 46 ]. Moreover, some research has focused on using a single feature to identify plants, which has low accuracy and poor stability.

To deal with the problems of a complex field environment and the low accuracy and poor stability of a single feature, some scholars have also proposed to integrate multiple features to improve the accuracy. For instance, He et al. [ 47 ] integrated multisource recognition information of different features, such as plant leaf shape, fractal dimension, and texture. They combined the good classification and promotion capabilities of SVM in the case of small samples and the advantages of Dempster–Shafer evidence theory of incomplete and uncertain information. Compared with single-feature recognition, this multi-feature decision fusion recognition method has better stability and a higher recognition accuracy. Sabzi et al. [ 5 ] proposed a machine vision prototype based on video processing and meta-heuristic classifiers based on Gray-level Co-occurrence Matrix (GLCM), color feature, texture feature, invariant moment, and shape feature. They used them to identify and classify 4299 samples from potatoes and five weed species online, achieving high accuracy. Deng et al. [ 48 ] integrated the color, shape, and texture features of weed images with a total of 101-dimensional features to solve the problem of the low recognition accuracy of a single feature of weeds in a rice field. Tang et al. [ 44 ] used a combination of vertical projection and linear scanning in corn farms under different lighting conditions to identify the centerline of crop rows. This method only recognizes crop rows, all plants among rows are identified as weeds regardless of their type, and it is unsuitable for identifying different types of weeds. On the whole, these studies have provided effective methods and approaches for realizing plant recognition and weed detection based on image-processing technology in the early stage. However, most of the studies are only for the identification of different plant leaves rather than the precise detection of crops or weeds in a field. Few studies exist on the identification and location of plants and weeds in a complex practical background in a field, and the identification and detection of weeds in actual farmland require further research.

Table 3 lists some literature on the identification or classification of plant leaves by using traditional ML methods. These methods achieve their purpose in specific plant leaves and detection background, but they are unsuitable for large-scale rapid detection or classification of images in a natural environment.

Research status and problems of traditional machine learning methods.

Using drone images to classify vegetation and detect weeds on a large scale has become a hot spot. Object-Based Image Analysis (OBIA) classification has been replacing traditional classification methods like the pixel-based approach. The difficulty lies in setting the optimal combination of parameters. In order to solve this problem, Torres-Sánchez et al. (2015) [ 49 ] used unmanned aerial vehicle (UAV) images of different herbaceous row crops to develop an automatic thresholding algorithm under the OBIA framework, and research the influence of multiple parameters on vegetation classification, making the algorithm allow unsupervised classification. UAVs are less constrained by field conditions that may restrict the access and movement of operators or ground vehicle-based platforms, and can monitor weed areas on a large scale. Furthermore, UAV imagery offers high image resolution and high flexibility in terms of timing of image acquisition. The high image resolution allows detection of low weed densities. Therefore, such methods will have broad prospects in high-input agriculture.

3.1. Traditional Features and Their Advantages and Disadvantages for Common Weed Detection

Most of the traditional weed detection methods based on image processing utilize the feature differences between plant leaves and weeds to distinguish them. This article mainly discusses the traditional image features and their advantages and disadvantages for the detection and recognition of four features of weeds: texture, shape, spectrum, and color.

3.1.1. Texture Features

Texture features are regional features that reflect the spatial distribution among pixels, which have been widely used in image classification [ 56 , 57 , 58 ]. Plant leaves are usually flat, and different leaves have diverse vein texture and leaf surface roughness information. The texture information can be used to distinguish crops and weeds effectively. Texture feature methods can mainly be divided into four categories: (1) statistical method, (2) structural method, (3) model-based method, and (4) transform-based method [ 59 ]. The most common texture feature descriptors used in weed detection include GLCM [ 60 ] and Gray-level Gradient Co-occurrence Matrix (GGCM) based on statistical texture analysis methods, LBP based on structural texture analysis methods, fractal dimension based on model methods, and Gabor based on transformation methods. The LBP feature can reflect the microstructure among pixels, and the improved LBP feature has the advantages of rotation and translation invariance. In essence, the Gabor feature has the effect of allowing the information of a certain frequency band to pass through it, and the remaining sub-information is filtered out. GLCM usually contains 10 statistics, which can reflect the spatial correlation of gray values of any two points in an image. GGCM considers the gradient information on the basis of GLCM, and it mainly has 15 statistics. The fractal dimension uses the self-similarity between local and whole research objects, and its methods include “blanket” algorithm, fractal Fourier, and box-counting dimension [ 61 ].

A large amount of texture information in crop and weed leaves plays an important role in recognition and classification tasks [ 62 ]. For example, Bakhshipour et al. [ 63 ] extracted 52 texture features (GLCM features in four directions) from wavelet multiresolution images for weed segmentation. Ishak et al. [ 52 ] used the combination of Gabor wavelet (GW) and gradient field distribution (GFD) to extract a new feature vector set based on directional texture features to classify weed species. Mustapha et al. [ 64 ] constructed a method based on texture feature extraction, which extracts texture features from field images composed of wide and narrow leaf weeds. However, these techniques cannot reliably and accurately perform classification tasks in complex natural scenarios, such as high weed density, overlapping, or obscured weeds and crops.

3.1.2. Shape Features

Shape features play an important role in image analysis for weed detection. They mainly include shape parameters, region-based descriptors, and contour-based descriptors. Generally, shape parameters include 11 kinds: perimeter, area, diameter, minor axis length, major axis length, eccentricity, compactness, rectangularity, circularity, convexity, and solidity. These parameters are the most intuitive, easy to implement, and unaffected by lighting. Region-based descriptors include Hu moment invariants and two-dimensional Fourier descriptors (FDs). Hu moment invariants are a shape descriptor proposed by Hu (1962) [ 65 ]. They are a normalized function based on shape boundary and its internal region information and contain seven invariant moment parameters in total. They are independent of geometric translation, scaling, or rotation and are robust to noise. Two-dimensional FDs describe the shape region by establishing feature points in the region plane and carrying out Fourier transforms on rows and columns at the same time. Contexture-based descriptors mainly include spatial position descriptor, curvature scale descriptor, and one-dimensional FD.

These shape features have been successfully applied in the species recognition task of plant leaf images [ 66 , 67 , 68 ]. For example, Pereira et al. [ 69 ] used five shape descriptors, namely, beam angle statistics, FD, Hu moment invariants, multiscale fractal dimension, and Tensor Scale Descriptor (TSD), in shape analysis to describe the contour shape of aquatic weeds. Bakhshipour and Jafari [ 51 ] extracted four major shape factors, Hu moment invariants, and FDs to distinguish weeds and crops on different classifiers. Chen et al. [ 39 ] used eight shape features and Hu moment invariants combined with color features to detect weeds in a soybean field.

Different species of plants have distinct shape features, but the shape of the leaves can be distorted by disease, insects, and even human and mechanical damage. Most research is conducted under the ideal condition of specific leaves without background. In a field environment, problems of overlap or occlusion of plant leaves occur. Therefore, the task of weed identification is difficult to complete by only basing on shape features. They should be combined with other features to improve accuracy.

3.1.3. Spectral Features

Spectral features are an effective method to distinguish plants with different leaf colors. When the spectral reflectance of weeds is remarkably different from that of crops [ 70 ], weeds and crops can be distinguished using spectral features. The spectral features are robust to partial occlusion and tend to decrease in calculation [ 71 ]. Some scholars have applied visible light and near-infrared spectra (Vis–NIR) [ 72 , 73 ], multispectral/hyperspectral imaging [ 74 ], and fluorescence [ 75 ] in the detection of different plants.

Pignatti et al. [ 76 ] distinguished corn crops and weeds by using the contents of chlorophyll and carotenoid retrieved using spectral indices or by inverting PROSAIL (coupled PROSPECT and SAIL radiative transfer models, [ 77 ]), as well as the species of weeds. Some scholars have also used Vis–NIR to classify weeds in crops, but studies are limited to laboratory feasibility studies and rely extensively on stoichiometry to select effective wavelengths and establish calibration models [ 78 , 79 ]. Elstone et al. [ 80 ] achieved good results in the identification of weeds and crops by using RGB and multispectral images in a lettuce field. However, weeds in plateau tropical conditions have different shapes and grow in large blocks, such that detecting them is difficult. Spectral sensors (spectrometers) can be used to measure the reflection intensity of multiple wavelengths and provide sufficient information to distinguish vegetation from soil. Nevertheless, they hardly distinguish species, especially in the early growth stages when crops and weeds have similar reflective characteristics [ 81 , 82 ].

During the growth and development stages of plants, the interaction between light and observed geometry and leaf angle distribution, as well as the variability of the spectral features of plant species, can affect hyperspectral detection. Capturing a multispectral image, hence, depends on the climatic conditions of the day, which changes the reflectivity of plants due to the amount of light absorbed. Although research on the identification of crop weeds by using sensitive spectral bands has achieved encouraging results, the accuracy is low under the condition that the spectral difference between crops and weeds is unobvious or the leaf reflection is affected by moisture, plant disease, growth period, and other factors [ 83 ]. Therefore, a combination of multiple features, such as shape and texture features, should be considered [ 84 ].

3.1.4. Color Features

The accuracy of color-based detection highly depends on the plant being studied and its color differences. Color is insensitive to the adjustment of scale, size, and position. In addition, it can provide information about unusable objects. It is a common method used to segment plants from the background by using the difference in color features. Hamuda et al. [ 85 ] summarized the advantages and disadvantages of plant segmentation for color index-based methods. Tang et al. [ 86 ] proposed to modify the color component ( 2 G − R − B ) and use the excessive green component E x G = 2 G − R − B of the RGB color space to segment. Ghasab et al. [ 87 ] and Zhao et al. [ 88 ] used the color moments of the RGB color space (including mean, standard deviation, and skewness) to represent the color features of plant leaves. Rasmussen et al. [ 89 ] used the color difference between green weeds and senescent cereals to propose a simple, semi-automatic, and robust procedure for weed detection in pre-harvest cereals, which has strong practical significance.

In addition, R, G, and B components have a high degree of correlation, which is suitable for color display but not for segmentation and analysis [ 90 ]. Therefore, many methods transform images from the RGB color space to other color spaces, such as HIS, HSV, Lab, and YCrCb. Tang et al. [ 44 ] used the YCrCb color space C g   C g = G − y to describe the green features of green crops under different illumination conditions. Hamuda et al. [ 91 ] believe that the HSV color space is more in line with human color perception than other color spaces and has strong robustness to illumination changes. The HSV color space was used to distinguish weeds, soil, and other residues in cauliflower fields under actual field conditions. Guo et al. [ 92 ] utilized 18 color features (r, g, b; Y, Cb, Cr; H, S, L; H, S, V; L*, a*, b*; L*, u*, v*), which were defined in 6 color spaces (RGB, YCbCr, HSL, HSV, CIEL*a*b*, and CIEL*u*v*). Knoll et al. [ 93 ] and Jin [ 94 ] also utilized different color spaces.

Color is the most unstable feature used for plant identification. When the color difference is unobvious, color-based methods may not be able to distinguish weeds from crops accurately. These methods can be affected by leaf disease, plant seasonal changes in color, or different lighting conditions. Table 4 compares the advantages and disadvantages of four common image features for weed detection.

Comparison of the advantages and disadvantages of four common features.

3.2. Multi-Feature Fusion

The similarity between weeds and crops makes using a single image feature to detect weeds and crops almost impossible. The commonly used image features can achieve the purpose of weed detection, but the experimental accuracy is low and the stability is poor in a nonideal environment due to the complex interference factors in the actual field. Table 4 indicates that the four features are from different perspectives and complement one another in function. To improve the experimental accuracy, researchers have successively used the method of multi-feature fusion to solve the problem of weed detection.

Ghazali et al. [ 95 ] combined statistical GLCM, structural method fast Fourier transform, and scale-invariant feature transform and achieved more than 80% accuracy in the real-time weed control system of an oil palm plantation. Li et al. [ 96 ] used a method based on the combination of shape analysis and spectral angle matching to identify weeds in watermelon fields. Shape and spectral features were used separately, excluding texture features. Chowdhury et al. [ 97 ] focused on vegetation classification on the basis of features extracted from a local binary model and GLCM and classified images in accordance with the density of grass to highlight the images with potential fire risks on both sides of the road. Tang [ 98 ] constructed a leaf texture feature extraction algorithm based on GGCM and an improved leaf color feature extraction algorithm combining K-means and SVM for plant leaf recognition. However, the problems of extracting leaf images and performing threshold segmentation under a complex background remain. He et al. [ 47 ] extracted three types of features of plant leaf shape, texture, and fractal dimension on the basis of field plant image processing. Compared with single-feature recognition, the multi-feature decision making fusion recognition method has better stability and higher accuracy, but it does not analyze the problem of feature selection. Chen et al. [ 99 ] studied the method of multi-feature fusion based on field weed detection at the corn seedling stage to analyze the selection of common feature descriptor combinations. On the basis of 6 feature descriptors commonly used in recent years (rotation-invariant LBP, HOG, GLCM, GGCM, Hu moment invariant, and Gabor), 18 multi-feature groups were formed. The combination of rotation-invariant LBP feature and GGCM showed the highest accuracy. Experiments have also proven that the average accuracy of multi-feature fusion is not necessarily higher than that of single-feature fusion. Nursuriati et al. [ 100 ] used three single features, namely, shape, color, and texture, or fusion features of Malaysian herbal plant leaves for identification experiments. The experimental results showed that when the three features were fused, the average accuracy was highest, followed by the average accuracy when using only the texture features. When shape features are combined with texture features, the average accuracy decreased. Lin et al. [ 101 ] studied the feasibility of integrating spectral, shape, and texture features to identify corn and seven kinds of weeds. They found that from the perspective of accessibility of crop/weed discriminant features, spectral and shape features can be used as the optimal features to develop weed identification. Nonetheless, such a method has not been applied in a complex natural environment, and the method needs further research. Yang et al. [ 37 ] proposed a new shape feature, MTD, which was combined with the LBP–HF texture feature for leaf classification and retrieval tasks. This method is efficient and suitable for large-scale plant species identification. However, its features should be designed manually and cannot be learned automatically, and other important features of leaves are not utilized.

In conclusion, these multi-feature fusion methods can solve the problem of weed detection and improve the accuracy of experiments, but some problems have not been completely solved. For example, for many interference factors under nonideal conditions, the accuracy and stability of experiments should be further improved.

3.3. Classifier

SVMs and Artificial Neural Networks (ANNs) have been widely used in crop and weed classification [ 102 , 103 ]. SVMs can solve the problems of nonlinear and high-dimensional pattern recognition and have good performance in dealing with small-sample problems and nonlocal minimum problems. ANNs have a strong learning capability and can classify untrained data [ 63 ]. Other algorithms often involved in the literature include K-nearest neighbor (KNN) [ 104 ] and random forest [ 105 , 106 ], naive Bayesian algorithm [ 107 , 108 ], Bayesian classifier [ 109 ], and AdaBoost [ 110 , 111 ].

In recent years, relevant scholars have continued to study the use of various classifiers to identify and classify weeds. For instance, Jeon et al. [ 112 ] used a weed detection and image-processing algorithm based on ANN to distinguish weeds and crops in the soil background under uncontrolled outdoor light. Chen et al. [ 113 ] used an improved KNN weed image classification method combined with GW and regional covariance Lie group structure to classify four kinds of broad-leaved weed images. The overall recognition accuracy was 93.13%. Ahmed et al. [ 84 ] used SVM to identify 6 weeds in a dataset of 224 images, and the optimal combination of its extractor could achieve 97.3% accuracy. Rumpf et al. [ 114 ] proposed a sequential classification method and used three different SVM models to distinguish not only weeds and barleys but also weeds of monocotyledon and dicotyledon plants.

Some literature has utilized multiple classifiers. For example, Bakhshipour and Jafari [ 51 ] evaluated the performance of using SVM and ANN based on shape features in accordance with the detection problem of four common weeds in sugar beet fields. The results showed that the overall accuracy of SVM was 95.00%, higher than that of ANN (i.e., 92.92%). Miao et al. [ 115 ] proposed a method based on image segmentation and reconstruction to solve the problems of low recognition accuracy and invalid shape feature in the recognition process of overlapping leaves. The recognition results in different classifiers, such as SVM, KNN, DT, and naive Bayes, were compared using 78-dimensional features, such as color features, LBP texture features, and fractal box dimensions. The best was SVM. Ashraf et al. [ 116 ] developed two kinds of rice field image classification technologies based on the density of weeds. The first method was to use GLCM combined with SVM to achieve a precision of 73%, and the second method was to use invariant scale and rotation moment based on a random forest classifier to achieve a precision of 86%. The limitation of the two methods is that they do not target other types of weeds, such as broadleaf weeds and sedges. Pantazi et al. [ 117 ] implemented a machine vision-based method that can identify 10 types of weeds, including corn plants and specific species. This method uses a Gaussian classifier, a self-organizing feature map (SOFM), an SVM, and an autoencoder as the four hybrid classifiers. However, this method can only recognize four weeds with a maximum accuracy of over 90%. When applied in the field, the system error is relatively large.

In summary, scholars have focused on improving classifiers based on machine vision or the corresponding image features of plants, which is of great significance to improve the accuracy. They can utilize the sample features in the case of small samples and do not require high hardware. They are conducive to practical deployment and play an important role in weed identification or classification in common scenes.

4. Weed Detection and Identification Methods Based on Deep Learning

The great progress and popularization of image-capturing devices have made capturing images easy. Meanwhile, the cost of computer hardware has been greatly reduced, and the computing power of GPU has been remarkably improved. Deep learning has been extended to the agricultural field [ 118 , 119 , 120 ]. Methods based on deep learning have achieved good results in weed detection and classification [ 121 ]. Although traditional ML methods are easy to understand and many improvements have been made, most of them are verified in low-density images. Occlusion, clustering, and changing lighting conditions in a natural environment remain major challenges in detection and localization [ 122 ].

Deep learning has a unique network feature structure, and features extracted using various deep learning methods are more effective than manually extracted features. Higher-level features can be obtained by learning local features from the bottom and then synthesizing those features from the top. Diverse features at different levels can correspond to various tasks. In the field of weed detection, deep learning methods use spatial and semantic feature differences to realize the identification and detection of crops and weeds and effectively improve the accuracy of weed identification and detection. In recent years, commonly used deep learning networks to solve the problem of weed detection include CNNs and fully convolutional networks (FCNs). Various methods in semi- and unsupervised fields have also emerged to reduce the labeling cost. In many cases, classification results obtained using these deep learning algorithms are better than those generated using traditional algorithms [ 123 ]. The use of traditional algorithms to classify different types of crops with high accuracy is still difficult. Deep learning methods need to rely on a large number of datasets for training, and the difficulty of collecting crop and weed images also demonstrates the disadvantages of deep learning methods for weed identification.

4.1. Weed Detection and Identification Methods Based on CNNs

CNNs are increasingly used in weed detection, and methods based on deep CNNs have achieved good results in weed detection and classification. For instance, Dyrmann et al. [ 124 ], Yu et al. [ 125 ], and Olsen et al. [ 21 ] used such methods. Potena et al. [ 126 ] adopted two different CNNs to process RGB and NIR images to identify crops and weeds rapidly and accurately. A lightweight CNN was used for fast and robust vegetation segmentation, then a deeper CNN was used to classify the extracted pixels between crops and weeds. Beeharry and Bassoo [ 127 ] evaluated the performance of two weed detection algorithms based on UAV images, ANN and AlexNet. The experimental results showed that the accuracy of AlexNet in weed detection was more than 99%, whereas the accuracy of ANN on the same dataset was 48%. Ramirez et al. [ 128 ] established an aerial image weed segmentation model and compared it with SegNet and U-Net. The research results showed that the data balance and better spatial semantic information made the experimental results more accurate. Patidar et al. [ 129 ] proposed an improved Mask RCNN model to extract early cranesbill seedlings. These weeds can be used as herbal medicines for rheumatic disease. The proposed method enabled the weeds to be completely separated from the original image to obtain complete nutrients and increase yield. You et al. [ 130 ] proposed a semantic segmentation method for weed crop detection based on deep neural networks (DNNs). Four additional components were integrated to improve the segmentation accuracy, which provided enhanced performance for weeds of arbitrary shape in a complex environment. These methods do not rely on image preprocessing and data conversion and can independently obtain useful feature information in images. The recognition accuracy is better than that of manually designed features under traditional ML methods.

CNN frameworks, such as AlexNet [ 19 ], ResNet [ 131 , 132 ], VGG [ 133 ], Google [ 134 ], U-Net, MobileNets, and DenseNet [ 135 ], are also widely used in weed detection. These methods stand out from other conventional index-based methods. For example, Chechliński et al. [ 135 ] measured four different plants in diverse growing places and light conditions, and their custom framework combined U-Net, MobileNets, DenseNet, and ResNet.

4.2. Weed Detection and Identification Methods Based on FCNs

FCNs are algorithms that automatically learn features and implement forward and reverse processes in an end-to-end manner. In recent years, FCNs have made great achievements in computer vision [ 136 ] and remote sensing applications [ 137 , 138 ]. Dyrmann et al. [ 139 ] proposed a method to detect weeds in color images automatically by using an FCN under severe occlusion. Huang et al. [ 140 ] captured a high-resolution UAV image over a rice field and adopted an FCN for pixel-level classification. Ma et al. [ 25 ] proposed a SegNet semantic segmentation method based on FCNs for the problem of weed detection in rice fields. Compared with the classic FCN model and U-Net model, the proposed method exhibited significantly higher accuracy and could effectively classify the pixels of rice seedlings, background, and weeds in rice field images. To control weeds in the early stages of growth, Fu et al. [ 141 ] proposed a segmentation method based on FCNs for high-resolution remote sensing images. On the basis of the VGG16 CNN model, a pretrained FCN was used to fine-tune the object data. This method could effectively improve the segmentation effect. FCNs were used to solve semantic-level image segmentation and pixel-level classification of images, which further developed the problem of weed segmentation. However, this method only classified each pixel without considering the relationship among pixels.

4.3. Weed Detection and Identification Methods Based on Semi- and Unsupervised Feature Learning

Supervised deep neural networks rely on artificially annotated data; even with the use of rotation and cropping data enhancement techniques, at least hundreds of annotated images are still required for supervised training. Relevant scholars began to study semi-supervised learning with only a small amount of labeled data and unsupervised feature learning without data labeling [ 142 , 143 ]. Hu et al. [ 34 ] proposed a new image-based deep learning architecture called Graph Weed Network (GWN). The purpose is to identify multiple types of weeds from RGB images collected from complex pastures. GWN can be regarded as a semi-supervised learning method, which alleviates the complex annotation task. The evaluation on the DeepWeeds dataset reached the highest accuracy of 98.1% at the time. Jiang et al. [ 144 ] proposed semi-supervised GCN–ResNet101 to improve the recognition accuracy of crops and weeds in a limited labeled dataset, combining the advantages of CNN features and the semi-supervised learning capability of the graph. Tang et al. [ 145 ] combined k-means unsupervised feature learning with the advantages of multilayered and refined CNN parameters as a pretraining process for the identification of weeds in soybean seedlings. This method replaces the random initialization weights of traditional CNN parameters, which effectively proves that this method is more accurate than randomly initialized convolutional networks. Bah et al. [ 146 ] proposed an automatic learning method for weed detection in the UAV images of bean and spinach fields, which was based on CNN and an unsupervised training dataset. Experimental results proved that the performance of this method was close to that of supervised data labeling. Ferreira et al. [ 33 ] tested two latest unsupervised deep clustering algorithms by using two public weed datasets. They proposed to use semiautomatic data labeling for weed identification. Compared with manually marking each image, semiautomatic data labeling could reduce the marking cost by hundreds of times. Then, NMI and unsupervised clustering accuracy indexes were used to evaluate the performance of pure unsupervised clustering. The use of unsupervised learning and clustering on agricultural issues will continue to be the direction of continuous development.

4.4. Other Deep Learning Methods

Researchers have proposed various other deep learning methods to solve the problem of weed detection and achieved good results. For example, Sadgrove et al. [ 147 ] proposed the Color Feature Extreme Learning Machine (CF-ELM). It is an implementation of the Extreme Learning Machine (ELM, which is a single-layer feed-forward neural network. It has a partially connected hidden layer and a fully connected output layer and uses three color inputs instead of the standard grayscale input. The authors used the inputs in three different color systems of HSV, RGB, and Y’UV to test and compare the accuracy and time consumption with those of the standard grayscale ELM. The proposed method performed well on three datasets: weed detection, vehicle detection, and population detection. It is highly suitable for use in agriculture or pastoral landscape. Abdalla et al. [ 148 ] compared three transfer learning methods based on VGG16 for semantic segmentation of high-density weed and oilseed rape images. Annotated images were trained end to end through the extensive use of data enhancement and transfer learning. The fine-tuning was based on the VGG16 encoder for feature extraction, and shallow machine learning classifiers were used for segmentation. Raja et al. [ 149 ] proposed a real-time online weed detection and classification algorithm based on crop signal for lettuce. The spraying mechanism was combined with a machine vision system to realize the classification task in the case of high weed density and achieve the purpose of precise spraying of weeds with herbicides. Khan et al. [ 150 ] proposed a small-cascaded encoder-decoder (CED-NET) architecture to distinguish crops from weeds, in which each level of the encoder and decoder network was independently trained for crop or weed segmentation. This network was compared with other state-of-the-art networks in four public datasets. The experiment proved that it was superior to U-Net, SegNet, FCN-8s, and DeepLabv3.

All in all, in order to further compare deep learning methods. Table 5 summarizes the five architectures and comparison experiment group. The five frameworks are Convolutional Neural Networks, Regional Proposal Networks, Fully Convolutional Networks, Graph Convolutional Networks, and Hybrid Networks. The order of comparison experiment accuracy is the order in “Comparison group”. Among them, Osorio et al. only gave “Precision” but not “Accuracy”. This is different from the calculation formula of “Accuracy”. The specific calculation formula needs to check the current work is classification recognition or semantic segmentation. Researchers could refer to the review written by Hasan et al. [ 12 ], which described 23 evaluation metrics by different researchers of the related works.

Comparison of the typical deep learning methods.

5. Weeding Machinery

In addition to the intelligent detection of weeds based on computer vision technology and to achieve spraying the target variable, autonomous agricultural robots that continuously improve the accuracy and efficiency have also been widely used in weeding fields. Researchers have relied on powerful computer vision and mechanical techniques to design various fully automated weed control robots. Robotic weeding uses computer vision to detect crops and weeds and selectively applies herbicides to the detected weeds [ 133 ] or eliminates weeds among rows [ 153 , 154 ] to achieve the purpose of precision agriculture. Raja et al. [ 155 ] proposed a weed knife control system based on a robot vision-based 3D geometric detection algorithm. Corresponding mechanical knife devices were also designed for automatic control of weeds in tomato and lettuce fields, which could work efficiently in a high-weed density environment. The system proposed by Kounalakis et al. [ 123 ] was mainly used to detect a specific plant on grassland, which would cause health, yield, and quality problems if eaten by animals. The implementation of this method relied on the design of a robot platform that could accurately detect the plant. The research of Chechliński et al. [ 135 ] mapped a weeding device, which would be installed behind a tractor, and the weeding tool would be installed behind a camera. The weeding tool could be replaced with insecticide or steam nozzles. Compared with traditional methods, intelligent weeding machines and equipment save manpower, are efficient, and can increase productivity. The future development direction of agricultural machinery will be to develop more efficient and multitask automatic machinery and equipment.

6. Discussion

6.1. various weed detection tasks.

The tasks of weed detection are diverse. Through literature analysis, they are mainly reflected in the following aspects:

Plant leaves in different backgrounds: ( a ) is the plant leaf image taken in a controlled laboratory environment; ( b ) is the plant leaf image obtained from the Deepweeds dataset [ 21 ], which is shot on-site to capture the true view of the whole plant).

• (2) Different datasets and evaluation indicators. At present, few public datasets are available. Consequently, many studies have been conducted on the basis of self-built datasets. Even if the main body of some datasets is the same crop, the portability of the algorithm is poor under different growth periods, illumination, and actual field backgrounds. Relevant evaluation indicators are not comparable due to the different basis of the dataset developed by the algorithm. The actual performance is difficult to determine.

6.2. Multiple Complex Factors Affect Weed Detection

The natural properties of weeds are complex, with a wide variety of species, wide distribution, numerous leaf shapes and sizes, and random growth, forming various texture features. In the bud stage of weeds, most plants are small in size, vary in appearance, and have high germination density. As a result, accurate statistics is difficult to perform. The main factors affecting the performance of weed detection are as follows:

• (1) The influence of different growth stages. Most plants change their leaf morphology, texture, and spectral characteristics in different seasons or growth and development stages.
• (2) The influence of changing light conditions. When light conditions are different, the shade of the plant canopy and the angle of the sun will affect the color of the vegetation. Some scholars have used the ultra-green index and the Otsu algorithm to solve the problems caused by ambient light. In particular, Atrand et al. [ 156 ] solved the problems by using camera filters and different types of cameras. HIS color model was also applied, and grayscale images with H component were generated to reduce the impact of uneven lighting on color images [ 157 ].
• (3) Influence of overlapping leaves and occlusion. The accurate segmentation of plants is a challenging task. In complex actual field images, overlapping leaves, occlusions, leaf shadows, dead leaves, and damaged leaves will make it impossible to segment the leaves effectively when processing the images.
• (4) Bottleneck of weed detection. Factors, such as hardware, algorithm complexity, and plant density, limit the actual detection speed or accuracy. Hence, fast image processing and accurate weed identification remains extremely important challenges.

7. Summary and Outlook

This article reviews the work of researchers using traditional machine learning and deep learning methods in computer vision technology in recent years. Four traditional characteristics and their advantages and disadvantages in traditional ML methods are analyzed. The respective characteristics of related work based on deep learning algorithms are introduced. Related public datasets and weeding machinery are also presented. Lastly, the future work of weed detection is prospected. In the past two decades, weed detection has made great progress. On the basis of traditional machine learning methods and deep learning-based weed detection methods, high levels of automatic weed detection and weeding have been achieved using various platforms and mechanical equipment. These methods have laid a good foundation for achieving high efficiency and precise weeding in the future. In the future, weed detection and related fields will have the following development trends:

• (1) Further research on semi- or unsupervised feature learning will be a hotspot of weed detection in the future. Researchers have obtained good results in diverse specific background, but they still lack generality and robustness. Deep learning-based methods show an encouraging promise, but the large number of labeled samples increases the manual requirements. The verification and comparison of new development algorithms also require sufficient sample size and corresponding ground truth datasets. Compared with various weeds, field crop images are relatively easy to obtain. In view of the above reasons, weed detection methods based on semi- or unsupervised feature learning will continue to be a popular research topic in the future.
• (2) With the use of the technology of weed detection and accumulation to develop an automatic crop guidance system, agricultural operations can be carried out in various aspects, such as harvest, weeding, spraying, and transportation. Automatically guided agricultural vehicles do not fatigue and reduce the labor intensity of the operator, thus improving efficiency and safety. However, at present, few methods and devices meet the high requirements of practical applications. Considerable work should be done to develop equipment with high performance and cost efficiency.
• (3) Traditional and deep learning methods have their own advantages. In the future, the advantages of the two methods should be fully utilized for further research. To improve the level of weed detection and weeding, solutions have been proposed to solve the difficult practical problems, such as plant illumination, overlapping leaves, occlusion, and classifier or network structure optimization.

Acknowledgments

The authors thank the editors and anonymous reviewers for providing helpful suggestions to improve the quality of this manuscript.

Author Contributions

Authors contributed as follows: thereof. Conceptualization, Y.C., Z.W., B.Z. and Y.D.; Methodology, Y.C., Z.W. and X.K.; Investigation, Z.W., Y.C., B.Z. and X.K.; Writing Original Draft Preparation, Z.W. and Y.C.; Writing Review and Editing, Z.W. and Y.C.; Supervision, Y.C., B.Z., X.K. and Y.D. All authors have read and agreed to the published version of the manuscript.

This work is supported by National Key R&D Program of China (Grant No. 2017YFD0700500, 2018YFD0700400), the Scientific Research Program funded by Shaanxi Provincial Education Department (Program No. 20JY053), the Key Research and Development Program of Shaanxi (Grant No. 2019 GY-080).

Institutional Review Board Statement

Informed consent statement, data availability statement, conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: semi-supervised weed detection for rapid deployment and enhanced efficiency.

Abstract: Weeds present a significant challenge in agriculture, causing yield loss and requiring expensive control measures. Automatic weed detection using computer vision and deep learning offers a promising solution. However, conventional deep learning methods often require large amounts of labelled training data, which can be costly and time-consuming to acquire. This paper introduces a novel method for semi-supervised weed detection, comprising two main components. Firstly, a multi-scale feature representation technique is employed to capture distinctive weed features across different scales. Secondly, we propose an adaptive pseudo-label assignment strategy, leveraging a small set of labelled images during training. This strategy dynamically assigns confidence scores to pseudo-labels generated from unlabeled data. Additionally, our approach integrates epoch-corresponding and mixed pseudo-labels to further enhance the learning process. Experimental results on the COCO dataset and five prominent weed datasets -- CottonWeedDet12, CropAndWeed, Palmer amaranth, RadishWheat, and RoboWeedMap -- illustrate that our method achieves state-of-the-art performance in weed detection, even with significantly less labelled data compared to existing techniques. This approach holds the potential to alleviate the labelling burden and enhance the feasibility and deployment speed of deep learning for weed detection in real-world agricultural scenarios.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Weed Detection Using Deep Learning: A Systematic Literature Review

Affiliations.

• 1 Big Data Analytics Laboratory, Department of Computer Science, School of Mathematics and Computer Science, Institute of Business Administration, Karachi 75270, Pakistan.
• 2 School of Science, Computing and Engineering Technologies, Swinburne University of Technology, Melbourne 3122, Australia.
• 3 School of Engineering and Technology, Central Queensland University, Melbourne 3000, Australia.
• 4 Faculty of Computer and Information Systems, Islamic University of Madinah, Medina 42351, Saudi Arabia.
• PMID: 37050730
• PMCID: PMC10098587
• DOI: 10.3390/s23073670

Weeds are one of the most harmful agricultural pests that have a significant impact on crops. Weeds are responsible for higher production costs due to crop waste and have a significant impact on the global agricultural economy. The importance of this problem has promoted the research community in exploring the use of technology to support farmers in the early detection of weeds. Artificial intelligence (AI) driven image analysis for weed detection and, in particular, machine learning (ML) and deep learning (DL) using images from crop fields have been widely used in the literature for detecting various types of weeds that grow alongside crops. In this paper, we present a systematic literature review (SLR) on current state-of-the-art DL techniques for weed detection. Our SLR identified a rapid growth in research related to weed detection using DL since 2015 and filtered 52 application papers and 8 survey papers for further analysis. The pooled results from these papers yielded 34 unique weed types detection, 16 image processing techniques, and 11 DL algorithms with 19 different variants of CNNs. Moreover, we include a literature survey on popular vanilla ML techniques (e.g., SVM, random forest) that have been widely used prior to the dominance of DL. Our study presents a detailed thematic analysis of ML/DL algorithms used for detecting the weed/crop and provides a unique contribution to the analysis and assessment of the performance of these ML/DL techniques. Our study also details the use of crops associated with weeds, such as sugar beet, which was one of the most commonly used crops in most papers for detecting various types of weeds. It also discusses the modality where RGB was most frequently used. Crop images were frequently captured using robots, drones, and cell phones. It also discusses algorithm accuracy, such as how SVM outperformed all machine learning algorithms in many cases, with the highest accuracy of 99 percent, and how CNN with its variants also performed well with the highest accuracy of 99 percent, with only VGGNet providing the lowest accuracy of 84 percent. Finally, the study will serve as a starting point for researchers who wish to undertake further research in this area.

Keywords: deep learning; machine learning; systematic literature review; weed detection.

Publication types

• Systematic Review
• Agriculture / methods
• Artificial Intelligence
• Crops, Agricultural
• Deep Learning*
• Plant Weeds
• Weed Control* / methods

Grants and funding

• n/a/Research Deanship, Islamic University of Madinah, Kingdom of Saudi Arabia

ORIGINAL RESEARCH article

Weed25: a deep learning dataset for weed identification.

• 1 Key Laboratory of Agricultural Equipment for Hilly and Mountain Areas, College of Engineering and Technology, Southwest University, Chongqing, China
• 2 Key Laboratory of Modern Agricultural Equipment and Technology (Jiangsu University), Ministry of Education, School of Agricultural Engineering, Jiangsu University, Zhenjiang, China
• 3 Interdisciplinary Research Center for Agriculture Green Development in Yangtze River Basin, Southwest University, Chongqing, China
• 4 National Citrus Engineering Research Center, Chinese Academy of Agricultural Sciences and Southwest University, Chongqing, China

Weed suppression is an important factor affecting crop yields. Precise identification of weed species will contribute to automatic weeding by applying proper herbicides, hoeing position determination, and hoeing depth to specific plants as well as reducing crop injury. However, the lack of datasets of weeds in the field has limited the application of deep learning techniques in weed management. In this paper, it presented a dataset of weeds in fields, Weed25, which contained 14,035 images of 25 different weed species. Both monocot and dicot weed image resources were included in this dataset. Meanwhile, weed images at different growth stages were also recorded. Several common deep learning detection models—YOLOv3, YOLOv5, and Faster R-CNN—were applied for weed identification model training using this dataset. The results showed that the average accuracy of detection under the same training parameters were 91.8%, 92.4%, and 92.15% respectively. It presented that Weed25 could be a potential effective training resource for further development of in-field real-time weed identification models. The dataset is available at https://pan.baidu.com/s/1rnUoDm7IxxmX1n1LmtXNXw ; the password is rn5h.

Introduction

Weed suppression is one of the greatest factors affecting crop production. The weeds could compete with crops for water, light, fertilizer, growth space, other nutrients, etc ., resulting in reduction of crop yield and production quality ( Khan et al., 2021 ). It could also be the host of many pathogens and insects, which would damage crop plants. According to a survey, the worldwide annual loss of crop production caused by weed suppression was 13.2%, which was equivalent to the annual food ration for one billion human beings ( Yuan et al., 2020 ). Thus, weed control plays a vital role in crop management and food security.

Common weed control methods include manual, biological, chemical, and mechanical weeding, etc . ( Marx et al., 2012 ; Stepanovic et al., 2016 ; Kunz et al., 2018 ; Morin, 2020 ; Andert, 2021 ). Manual weeding provides the most precise management of weeds in the field. However, the labor intensity and cost are too high to make it feasible for large-scale cultivation. Biological weeding is safe and friendly to the environment as it brings little injury to non-target organisms, while it usually requires a long period to rebuild the eco-system. Chemical weeding is the most common approach of weed control, mainly through spraying of chemical herbicides. However, the overuse of herbicides has caused many issues, such as environmental pollution, pesticide residues, and weed resistance. According to the survey, 513 biotypes of 267 species of weeds have developed resistance to 21 types of herbicides in various cropland systems ( Heap, 2022 ). Thus, the application of technologies such as precise spraying or mechanical weed management on specific weeds will be of great significance to avoid the over-input of herbicide. Due to the concept of organic agriculture, automatic mechanical weeding is gradually attracting more attention ( Cordill and Grift, 2011 ). It realized weed control without chemical input and saved much fuel as unnecessary tillage could be avoided. However, as the weed identification accuracy is not high enough, unexpected damage to the plant–soil system has been one of the most important barriers to the application of intelligent mechanical weeding ( Swain et al., 2011 ; Gašparović et al., 2020 ). Therefore, it is imperative to improve the identification accuracy of weeds in the fields.

In weed identification research, several traditional machine learning methods have been applied based on image processing techniques, including support vector machine (SVM) ( Bakhshipour and Jafari, 2018 ), decision tree ( Bakhshipour and Zareiforoush, 2020 )-based random forest algorithm ( Gašparović et al., 2020 ), and K-nearest neighbor (KNN) classifiers ( Pallottino et al., 2018 ). In these algorithms, the color, texture, shape spectrum, and other characteristics of weed images should be extracted with complex hand-crafting. Thus, similar weed species could not be distinguished if the weed image extraction was incomplete or if there were occluded features.

In 2006, Hinton et al. (2006) proposed the concept of deep learning, pointing out that the structure of deep learning networks was deep and closely connected. The larger datasets would be trained by increasing the speed of algorithms. In recent years, deep learning technology is developing rapidly, showing high accuracy and robustness in the field of image identification ( Peteinatos et al., 2020 ). In particular, ImageNet, a large-scale, multi-variety dataset containing 3.2 million images, presented that large-scale datasets played an important role in improving the identification accuracy of the trained models using deep learning algorithms ( Russakovsky et al., 2015 ). However, both the image amount and the weed species of existing datasets for deep learning-based weed identification model training are in a small scale.

In practice, the weeds should be controlled at the growth stage between three and six leaves so that the crops could occupy the dominance in further growth competition. Conventional algorithms for weed identification used image processing technology to extract the image features of weeds, crops, and background. Bakhshipour et al. (2017) presented a model to distinguish sugar beets and weeds by using wavelet texture features. The principal component analysis was used to select 14 of the 52 extracted texture features. It demonstrated that wavelet texture features could be effectively distinguished between crops and weeds despite many occlusions and overlapping leaves. The color feature-based models could only identify crops and weeds that had obvious differences of pixel values in RGB matrix or other parameter matrixes generated from that. Generally, the color feature was applied in combination with other features—for example, Kazmi et al. (2015) proposed a method which fused surface color and edge shape for leaf detection and vegetation index integration. The vegetation index was integrated into local features by obtaining the accuracy of 99.07%. However, although conventional image processing methods could distinguish weeds and crops, it was difficult to distinguish the weeds in different species.

Deep learning network can form abstract high-level attributes, which will benefit weed identification, rather than the conventional machine vision network using low-level attributes such as color, shape, or texture. As is known, deep learning technique has improved in terms of accuracy and generalization capabilities in the current target detection models. The prevalent target detection networks are composed of Faster R-CNN, Single Shot Detector, You Only Look Once (YOLO) model, etc . ( Redmon et al., 2016 ; Ren et al., 2017 ; Quan et al., 2022 ). Dyrmann et al. (2016) used convolutional neural networks to identify 22 different plants with a total of 10,413 images. The result showed that the higher classification accuracy took place in the weed species which consisted of a larger number of image resources. Thus, weed identification based on deep learning technology requires sufficient datasets.

There have been some large datasets for object detection model training, such as PASCAL VOC ( Everingham et al., 2010 ), ILSVRC ( Russakovsky et al., 2015 ), COCO ( Lin et al., 2014 ), etc . Nevertheless, most of the large and open-access datasets consisted of objects in common life—for example, the PASCAL VOC was composed of 24,000 images in 20 categories such as cats, dogs, cars, etc . However, relevant weed datasets were not involved. Many scholars have created some weed datasets for the identification of weeds in specific plots, which usually contained just a few categories and were in small scales. Haug and Ostermann (2015) collected and produced a labeled and available dataset with 60 images at the Organic Carrot Farm. Dyrmann et al. (2016) collated images consisting a total of 10,413 images of 22 crops and weeds from six different datasets in the earlier periods, with an average of 400 images per species. Giselsson et al. (2017) published a dataset of about 960 plants from 12 plant species at different growth stages. Jiang et al. (2018) established a dataset of four species of weeds with 1,200 images for each. Peng et al. (2019) extracted 1,000 images of weeds associated with cotton fields from the video for research, including goosegrass, purslane, and nutgrass. Meanwhile, most of these datasets are not open-access. Olsen et al. (2019) gathered a total of 17,509 labeled datasets of eight species of weeds from Australian ranches, DeepWeeds, which was a large and publicly available dataset of pasture weeds. Sudars et al. (2020) presented a public dataset including 1,118 images of six food crops and eight weeds. On average, each category contained 80 images. Tretyakova et al. (2020) sorted out a plant dataset containing 24,284 images of 329 plant species, with an average of 73 images for each category, which mainly incorporated grain, spring and winter crops, economic crops, and weeds, so it was not a purely weed dataset. Khan et al. (2020) proposed a study of four publicly available datasets, including the Rice Sowing and Weed Dataset, the BoniRob Dataset, the Carrot Crop and Weed Dataset, and the Rice Millet Dataset. In order to quickly identify cotton field weeds, Fan et al. (2021) collected 4,694 pictures in a cotton field including seven types of associated weeds, such as field thistle, crabgrass, and purslane. The datasets are valuable and available for testing algorithms. However, most of them only cover specialized crops or weeds, which are often limited to specific growth stages. Meanwhile, in view of the current research on weed detection in farmlands, many researchers tried to cultivate some small samples in the laboratory and expanded the data through data enhancement, mainly by expanding, enlarging, shrinking, and rotating the original image. Thus, there is currently a lack of open-access large weed datasets.

To enable better training resources for applying computer vision technology in weed identification, we provided the dataset Weed25 in this paper. This dataset contains 14 families, including 25 species of weeds. The image amount of each weed species was nearly 500–550. It could meet various training requirements for either classification or detection models. Some of the weed images in Weed25 are shown in Figure 1 . Compared to the farmland weed dataset in the existing literature, the Weed25 dataset is larger and more diverse. Due to the long period of collection, the morphology of a variety of weeds at different growth periods was included. The hypothesis is that, with Weed25, the identification accuracy would be significantly improved using the common deep learning training model.

Figure 1 Images of partial samples in the dataset.

Materials and methods

This section mainly introduced the data acquisition, classification, and labeling of the Weed25 dataset.

Image acquisition

The image resources of Weed25 were acquired from fields and lawns in Chongqing, China, on 25 weed species which are prevalent in East Asia. The images were taken between October 2021 and August 2022. Images were taken at a height and angle of approximately 30–50 cm and 60°–90°, respectively, with a digital camera (Nikon D5300 SLR, Japan) or a smartphone (Huawei Enjoy 9S, China), which means that the shooting angle was as vertical to the weed as possible.

As sunlight intensity and angle would have impacts on the accuracy of subsequent weed identification, the weed images were taken at different time points—between 9:00 and 17:00—on sunny, cloudy, and rainy days, respectively. Therefore, the light conditions of this weed dataset could represent that in the natural complex environment. In practice, the issues of mutual occlusion and interweaving of weed leaves could bring difficulty to the image acquisition. Meanwhile, to collect the images of weeds at different growth stages, some species of the weeds were selected for greenhouse cultivation. Majority of the weed images were collected when they were at the growth stage of two to nine leaves (BBCH 12–19). Pictures of pigweed at seedling stage, three-leaves stage, four-leaves stage, and eight-leaves stage, respectively, are presented in Figure 2 . It could be seen that there were significant differences in the morphology of this species of weed at different growth stages.

Figure 2 Morphological appearance of purslane at different growth stages.

Most species of grass weeds in the field are very similar in appearance and shape, such as barnyard grass, green foxtail, crabgrass, and other Gramineae plants, as shown in Figure 3 . The unique color and similar characteristics of weeds in terms of shape will bring some difficulty in identifying the weeds.

Figure 3 Similarity of grass weeds in the field.

Classification of dataset structures

The classification of weeds in Weed25 was conducted mainly with reference to Primary color ecological map for identification and control of weeds in farmland ( Ren et al., 2018 ) and Farmland Weed Identification Primary Color Atlas ( Hun et al., 2012 ). Weed25 consisted of 25 weed species from 14 families. Because each family was made up of many species of weeds, the different families were classified as a general family. The different weeds under each family were classified as a specific species of the general family—for example, Weed25 was mainly composed of barnyard grass of Gramineae , billygoat weed and cocklebur of Compositae , and pepper grass of Cruciferaceae. Gramineae , Asteraceae , and Cruciferaceae were the general families in this classification system. The specific weed included in the classification belonged to a general family. The main hierarchy is shown in Figure 4 . The main occurrence areas and crops of these weeds are summarized and listed in Table 1 .

Figure 4 Classification structure hierarchy of Weed25 dataset. The dataset includes 25 weed species from 14 families.

Table 1 Some of the main crops with weed growth characteristics and hazards.

Data annotation

All images in Weed25 were manually annotated and verified by three weed scientists. LabelImg was selected as the annotated software, which was a visual graphical image annotation tool created in Python environment. The labels were generated as COCO files for further training.

Description and division of datasets

The Weed25 dataset contained 14,023 images in 25 categories, which was more diverse than the existing dataset, with most of the weed images collected from the field. The collected weed dataset was divided into training, validation, and test datasets with a ratio of 6:2:2. Specifically, all images of Weed25 were divided into 8,409 training images, 2,807 validation images, and 2,807 testing images, as shown in Table 2 . For object detection, all dataset images labeled were divided into 9,955 images as the training dataset and 4,068 images as the validation dataset.

Table 2 Division of training/validation/testing (denoted as Train/Val/Test) datasets.

Comparison with other datasets

To show the advantages of Weed25 in terms of species diversity and species average (species diversity: the number of all weed species in the dataset, abbreviated as diversity, that was characterized in this paper by the number of species; species average: the mean of the image number of each weed, abbreviated as diversity average, that was used in this paper to characterize the average number of weeds), Weed25 was compared with several existing datasets related to weed identification, as shown in Table 3 . In terms of diversity, the largest dataset ( Tretyakova et al., 2020 ) was comprised of 329 categories, while the smallest dataset ( Jiang et al., 2018 ) had only four categories. Although the dataset created by Tretyakova et al. (2020) contained 329 categories, the images were not only weeds but also plants such as grains and economic crops. The species average of this dataset was 73, which was usually not sufficient for model training. Moreover, most datasets were not open-access ( Jiang et al., 2018 ; Sudars et al., 2020 ; Tretyakova et al., 2020 ). Majority of the existing datasets had a certain imbalance on species diversity and evenness ( Giselsson et al., 2017 ; Sudars et al., 2020 ; Fan et al., 2021 ), which led to difficulties in practical applications. According to the previous survey, we created Weed25 in the fields such as farmland and lawn. The diversity and average should be more reasonable compared with the existing weed datasets.

Table 3 Weed25 compared with other weed datasets.

Evaluation test

To verify whether the image in Weed25 could be applied for weed identification model training, several deep learning detection networks were employed for the model training with this dataset.

Training platform

The device for deep learning model training was a desktop workstation with a processor of Ryzen threadripper 3970x 32-core processor ×64 (AMD®, California, USA). The running memory was 32G. The graphics processing unit was GeForce RTX3060ti (NVIDIA Corporation, Santa Clara, CA, USA). The Pytorch deep learning framework that supports numerous neural network algorithms was processed under Ubuntu20.4.

Evaluation indicators

In this study, the precision (P), recall (R), and mean average precision (mAP) were used as the evaluation indexes of the trained target detection models. The value range of the three indexes is [0, 1]. Meanwhile, the average of the harmonization of precision and recall (F1 score) was also introduced to reconcile the average evaluation, where precision represents the ratio between the number of correctly detected weeds and predicted weeds of a certain species. Recall represented the proportion of targets for a class of weeds in the sample that were correctly predicted. The specific evaluation calculation formula is as follows:

where TP represents the number of samples correctly divided into positive samples, FP represents the number of incorrectly divided positive samples, and FN represents the number of incorrectly divided negative samples.

The average precision indicates the detection effect of the detection network on a certain category of targets. The larger the value is, the better the overall detection effect will be. The average precision is mainly reflected in the precision–recall curve (also known as the PR curve). In the PR plot, the horizontal axis is the recall rate, which reflects the ability to cover the positive sample, and the value of the vertical axis reflects the precision of predicting the positive sample. The calculation of the average precision is taken as the integral of the precision and recall curve on [0,1]:

The mean of average precision represents the mean of the average precision of all categories in the dataset. It is calculated as the ratio of the sum of the average precision of all categories to the number of all categories:

The F1 value is a comprehensive evaluation index based on accuracy and recall, which is defined as the average of the harmonization of precision and recall:

Deep-learning-based detection

To verify the application capacity of Weed25 in weed detection, the YOLO models based on the single-stage detection of convolutional neural networks as well as the two-stage detection of regional convolutional neural network Faster R-CNN were selected as the training algorithm. The main difference was that Faster R-CNN used the most advanced regional recommendation box extraction algorithm Region Proposal Network (RPN). The feature map of the image can be extracted using the feature extraction network. It will be shared by the RPN network and the Faster R-CNN network. Finally, the position of the candidate box was obtained by anchor regression, while the YOLO model transforms object detection into an end-to-end regression problem and improves the detection real-time. The thresholds IoU of 0.5 and batch size of 4 were adjusted for YOLOv3, YOLOv5, and Faster R-CNN. Each model was trained for 100 epochs.

Test results

The training results are listed in Table 4 . It presented that the YOLO model training indicators using Weed 25 were generally acceptable. The difference of mAP between YOLOv5 and YOLOv3 was very small as the values were 92.4% and 91.8%, respectively. The precision was 88.0% and 89.0%, respectively. Moreover, the recall for both reached 99.0%. It showed that Weed25 was available for the YOLO models. For the sake of excluding the advantages of the YOLO model on the training results, Faster R-CNN was employed for the training as well. The results showed that the mAP of Faster R-CNN network was 92.15% ( Figure 5 ), which was lower than the mAP of the YOLOv5 networks. It indicated that Weed25 would be capable for precision weed identification model training in future studies.

Table 4 Weed identification model training results using YOLO and Faster R-CNN networks with Weed25.

Figure 5 mAP curve of Faster R-CNN (the meaning of mAP_0.5 is that when IoU was set to 0.5, the average precision (AP) of all images in each category would be calculated. Then, the AP of all categories was calculated; that was mAP).

Figure 6 presents the training results of YOLO networks. It could be seen that the box_loss, obj_loss means, and cls_loss means of the training and validation datasets during the training of the model were constantly decreasing. The average precision under mAP_0.5 was constantly increasing. The mAP_0.5 of both YOLOv3 and YOLOv5 was close to 0.9. It indicated that the training effect was good with the dataset Weed25.

Figure 6 Graph of the You Only Look Once model training results (Train/val box_loss: the bounding box loss of the training dataset or validation dataset; the smaller the box is, the more accurate. Train/val obj_loss: train or val is speculated to be the mean loss of the target detection, and the smaller the target detection is, the more accurate the detection. Train/val cls_loss: train or validation is speculated to be the mean of classification loss, and the smaller the classification is, the more accurate).

Figure 7 presents the confusion matrixes summarizing the identification performance of the YOLOv3 and YOLOv5 models. It could be seen that the classification accuracy of 18 weed species in the YOLOv3 model and 19 weed species in the YOLOv5 model was higher than 0.9. In particular, the identification accuracy of Ceylon spinach reached 1.0. It showed that the model had good recognition capability of this weed. Among the six weed species with a classification accuracy less than 0.9 in the YOLOv5 model, it was found that the classification accuracy of crabgrass and green foxtail in the Gramineae family was 0.76 and 0.85, respectively. For the majority of incorrect identification cases, they were predicted as background maps, which showed that the background would have some interference with the detection results. An impressive identification case occurred on the horseweed. With as less as 192 pictures for training and validation in total, the classification accuracy of horseweed reached 0.85. There might be two main reasons that could contribute to this result. Firstly, the appearance features of horseweed were significant. Secondly, the images of the horseweed were not disturbed and occluded by other weeds. Meanwhile, the classification accuracy of Asiatic smartweed with 490 images was as low as 0.56. The features of Asiatic smartweed were not significant as it was a vine weed growing in the soil during the process of collection. On the other hand, the area of soil was larger than the area of weeds in most Asiatic smartweed images. That might be the reason for the incorrect generalization of this weed into the background image. The weed identification results of the YOLO models are shown in Figure 8 , which displays the identification of weeds in a complex context.

Figure 7 Confusion matrix of You Only Look Once models.

Figure 8 Effect diagram of weed prediction of the You Only Look Once model (in the notes above the detection box, the name of the weed is on the left, and the precision of the weed is on the right).

In Figure 5 , the mAP of training result using Faster R-CNN network is displayed. Faster R-CNN used RPN to generate high-quality regional suggestion boxes, which could effectively improve the identification accuracy of weeds. It was found that the mAP of weeds was 92.15%, with a steady upward trend. The mAP tended to be convergent when the number of iterations was greater than 50. In addition, the identification results of all the weed species are shown in Table 5 . The AP of Asiatic smartweed was 62.92%, and the precision was only 20%. The average precision of velvetleaf reached 99.70%.

Table 5 Training results of Faster R-CNN (precision represents the ratio between the number of correctly detected weeds and predicted weeds of a certain type).

Although there are many kinds of weeds in the field, we have just collected 25 weed species in this paper. However, many other weeds could also affect crop production. The proposed weed dataset is still insufficient. Therefore, more species of weeds in the crop field will be appended to improve the intelligent weeding technology in the future.

Through the training results of YOLO models and Faster R-CNN, it was found that the AP of weeds such as Asiatic smartweed was not high. For such weeds, more image resources should be collected. Meanwhile, it would be of great significance to take pictures of these weeds avoiding a large-scale background. Because of the narrowness and insignificant appearance features of grasses, the identification of Gramineae weeds was relatively difficult. Quan et al. (2022) used the trained model to identify the broadleaf and Gramineae weeds in the maize fields, resulting in an average accuracy of 94.70% and 87.10%, respectively. It could be seen from the confusion matrix ( Figure 7 ) that the predicted values of crabgrass and green foxtail were both also lower than 0.9 in our research. Furthermore, the classification accuracy of crabgrass was just 0.76. Although the classification accuracy of barnyard grass reaches 0.93, barnyard grass could be easily misjudged as crabgrass with the possibility of 0.01. Otherwise, sunlight could also cause some influence on weed identification. Li et al. (2021) collected 1,000 images of weeds on sunny, cloudy, and rainy days. In that study, the parameters of the feature extraction network ResNet-101 were optimized using the feature extraction network. The training results showed that the rate of identification reached 96.02% on sunny days and 90.34% on rainy days. Moreover, the overlapping of leaves was also a main issue, reducing the identification accuracy at present. Pahikkala et al. (2015) identified the species of mixed sativa and dandelions based on different textures. It indicated that leaf textures should be specifically considered when identifying weeds under harsh conditions as overlapping was commonly existing in the images. Thus, factors such as different intensities of illumination and the overlapping leaf have a great impact on weed identification. Methods on overcoming the problems were proposed, but they still lack versatility and robustness ( Wang et al., 2019 ).

While the images of Weed25 were collected under different light intensities, the overlapping leaves and light intensity had not yet been considered for investigation as to their impact on weed classification accuracy. Therefore, subsequent studies would be continued focusing on the investigation of the photography factors which affect the weed classification accuracy. The precise weed identification in farmlands remains a challenge, which limits the development and application of intelligent weeding machines based on deep learning algorithms.

In this paper, we created a dataset of weeds, Weed25. The images of the dataset were mainly collected from farmlands and lawns. A total of 14,035 images including 25 different weed species were included. Compared with the existing weed dataset, Weed25 contains the weeds that are prevalent in fields. It has the advantages in diversity and average. In addition, YOLOv3, YOLOv5, and Faster R-CNN were employed to train weed identification models using the Weed25 dataset. The average precision was 91.8%, 92.4%, and 92.15% respectively, which indicated that the proposed dataset has the capability for further development of precise weed identification models, which would contribute to the application of intelligent weed control technology in practice.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors without undue reservation.

Author contributions

PW, YT, LW, and HL contributed to conception and design of the study. YT and FL organized the database. PW, YT, FL, and QN performed deep learning and the statistical analysis. YT wrote the first draft of the manuscript. PW, QN, and HL wrote sections of the manuscript. All authors contributed to the article and approved the submitted version.

Acknowledgments

This research was funded by the National Natural Science Foundation of China (grant numbers 32201651 and 32001425), the Natural Science Foundation of Chongqing, China (grant numbers cstc2020jcyj-msxmX0459 and cstc2020jcyj-msxmX0414), the Fundamental Research Funds for the Central Universities (SWU-KT22024), and the Open Funding of the Key Laboratory of Modern Agricultural Equipment and Technology (Jiangsu University; grant number MAET202105). The authors would like to appreciate Prof. Dr. Xinping Chen, Prof. Dr. Yunwu Li, Prof. Dr. Wei Qian, Prof. Dr. Nannan Li, Zhantu Zheng, Yu Xia, Long Wan, and Chengrui Xu for technical support.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Andert, S. (2021). The method and timing of weed control affect the productivity of intercropped maize (Zea mays l.) and bean (Phaseolus vulgaris l.). Agriculture 11, 380. doi: 10.3390/agriculture11050380

CrossRef Full Text | Google Scholar

Bakhshipour, A., Jafari, A. (2018). Evaluation of support vector machine and artificial neural networks in weed detection using shape features. Comput. Electron. Agric. 145, 153–160. doi: 10.1016/j.compag.2017.12.032

Bakhshipour, A., Jafari, A., Nassiri, S. M., Zare, D. (2017). Weed segmentation using texture features extracted from wavelet sub-images. Biosyst. Eng. 157, 1–12. doi: 10.1016/j.biosystemseng.2017.02.002

Bakhshipour, A., Zareiforoush, H. (2020). Development of a fuzzy model for differentiating peanut plant from broadleaf weeds using image features. Plant Methods 16, 153. doi: 10.1186/s13007-020-00695-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Cordill, C., Grift, T. E. (2011). Design and testing of an intra-row mechanical weeding machine for corn. Biosyst. Eng. 110, 247–252. doi: 10.1016/j.biosystemseng.2011.07.007

Dyrmann, M., Karstoft, H., Midtiby, H. S. (2016). Plant species classification using deep convolutional neural network. Biosyst. Eng. 151, 72–80. doi: 10.1016/j.biosystemseng.2016.08.024

Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., Zisserman, A. (2010). The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338. doi: 10.1007/s11263-009-0275-4

Fan, X., Zhou, J., Xu, Y., Li, K., Wen, D. (2021). Identification and localization of cotton seedling weeds based on optimized faster r-CNN. J. Agric. Machinery 52, 26–34. doi: 10.6041/jissn.1000-1298.2021.05.003

Gašparović, M., Zrinjski, M., Barković, Đ., Radočaj, D. (2020). An automatic method for weed mapping in oat fields based on UAV imagery. Comput. Electron. Agric. 173, 105385. doi: 10.1016/j.compag.2020.105385

Giselsson, T. M., Jørgensen, R. N., Jensen, P. K., Dyrmann, M., Midtiby, H. S. (2017) A public image database for benchmark of plant seedling classification algorithms . Available at: http://arxiv.org/abs/1711.05458 (Accessed May 30, 2022).

Google Scholar

Haug, S., Ostermann, J. (2015). “A Crop/Weed field image dataset for the evaluation of computer vision based precision agriculture tasks,” in Computer vision - ECCV 2014 workshops lecture notes in computer science . Eds. Agapito, L., Bronstein, M. M., Rother, C. (Cham, Switzerland:Springer International Publishing), 105–116. doi: 10.1007/978-3-319-16220-1_8

Heap, I. (2022) The international herbicide-resistant weed database . Available at: www.weedscience.org .

Hinton, G. E., Osindero, S., Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554. doi: 10.1162/neco.2006.18.7.1527

Hun, Z., Yuan, L., Chen, S. (2012). Farmland weed identification primary color atlas (Beijing: China Agriculture Press Co., Ltd).

Jiang, H., Wang, P., Zhang, Z., Mao, W., Zhao, B., QI, P. (2018). A rapid method for identifying weeds in maize field based on convolutional network and hash code. J. Agric. Machinery 49, 30–38. doi: 10.6041/j.issn.1000-1298.2018.11.004

Kazmi, W., Garcia-Ruiz, F., Nielsen, J., Rasmussen, J., Andersen, H. J. (2015). Exploiting affine invariant regions and leaf edge shapes for weed detection. Comput. Electron. Agric. 118, 290–299. doi: 10.1016/j.compag.2015.08.023

Khan, A., Ilyas, T., Umraiz, M., Mannan, Z. I., Kim, H. (2020). CED-net: Crops and weeds segmentation for smart farming using a small cascaded encoder-decoder architecture. Electronics 9, 1602. doi: 10.3390/electronics9101602

Khan, S., Tufail, M., Khan, M. T., Khan, Z. A., Anwar, S. (2021). Deep learning-based identification system of weeds and crops in strawberry and pea fields for a precision agriculture sprayer. Precis. Agric. 22, 1711–1727. doi: 10.1007/s11119-021-09808-9

Kunz, C., Weber, J. F., Peteinatos, G. G., Sökefeld, M., Gerhards, R. (2018). Camera steered mechanical weed control in sugar beet, maize and soybean. Precis. Agric. 19, 708–720. doi: 10.1007/s11119-017-9551-4

Liebman, M., Mohler, C., Staver, C. (2001). Ecological management of agricultural weeds (New York, USA:Cambridge University Press). doi: 10.1017/CBO9780511541810

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., et al. (2014). “Microsoft COCO: Common objects in context,” in Computer vision – ECCV 2014 lecture notes in computer science . Eds. Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (Cham: Springer International Publishing), 740–755. doi: 10.1007/978-3-319-10602-1_48

Li, K., Xu, Y., Zhou, J., Fan, X., Wei, Y. (2021). A seedling-stage weed identification method based on faster r-CNN and data enhancement. J. Xinjiang Univ. (Natural Sci. Edition) (Chinese English) 38, 450–456. doi: 10.13568/j.cnki.651094.651316.2020.06.03.0001

Marx, C., Barcikowski, S., Hustedt, M., Haferkamp, H., Rath, T. (2012). Design and application of a weed damage model for laser-based weed control. Biosyst. Eng. 113, 148–157. doi: 10.1016/j.biosystemseng.2012.07.002

Morin, L. (2020). Progress in biological control of weeds with plant pathogens. Annu. Rev. Phytopathol. 58, 201–223. doi: 10.1146/annurev-phyto-010820-012823

Olsen, A., Konovalov, D. A., Philippa, B., Ridd, P., Wood, J. C., Johns, J., et al. (2019). DeepWeeds: A multiclass weed species image dataset for deep learning. Sci Rep 9, 2058. doi: 10.1038/s41598-018-38343-3. arXiv:1810.05726.

Pahikkala, T., Kari, K., Mattila, H., Lepistö, A., Teuhola, J., Nevalainen, O. S., et al. (2015). Classification of plant species from images of overlapping leaves. Comput. Electron. Agric. 118, 186–192. doi: 10.1016/j.compag.2015.09.003

Pallottino, F., Menesatti, P., Figorilli, S., Antonucci, F., Tomasone, R., Colantoni, A., et al. (2018). Machine vision retrofit system for mechanical weed control in precision agriculture applications. Sustainability 10, 2209. doi: 10.3390/su10072209

Peng, M., Xia, J., Peng, H. (2019). Efficient identification of weeds in cotton field under complex background of faster r-CNN fused with FPN. Trans. Chin. Soc. Agric. Eng. 35, 202–209.

Peteinatos, G. G., Reichel, P., Karouta, J., Andújar, D., Gerhards, R. (2020). Weed identification in maize, sunflower, and potatoes with the aid of convolutional neural networks. Remote Sens. 12, 4185. doi: 10.3390/rs12244185

Quan, L., Jiang, W., Li, H., Li, H., Wang, Q., Chen, L. (2022). Intelligent intra-row robotic weeding system combining deep learning technology with a targeted weeding mode. Biosyst. Eng. 216, 13–31. doi: 10.1016/j.biosystemseng.2022.01.019

Redmon, J., Divvala, S., Girshick, R., Farhadi, A. (2016). “You only look once: Unified, real-time object detection,” in 2016 IEEE conference on computer vision and pattern recognition (CVPR) (Las Vegas, NV, USA: IEEE), 779–788. doi: 10.1109/CVPR.2016.91

Ren, S., He, K., Girshick, R., Sun, J. (2017). Faster r-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149. doi: 10.1109/TPAMI.2016.2577031

Ren, P., Li, P., Huang, L. (2018). Primary color ecological map for identification and control of weeds in farmland (Beijing, China:China Agricultural Science and Technology Press). International Standard Book Number ISBN: 9787511637567.

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al. (2015). ImageNet Large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252. doi: 10.1007/s11263-015-0816-y

Stepanovic, S., Datta, A., Neilson, B., Bruening, C., Shapiro, C., Gogos, G., et al. (2016). The effectiveness of flame weeding and cultivation on weed control, yield and yield components of organic soybean as influenced by manure application. Renew. Agric. Food Syst. 31, 288–299. doi: 10.1017/S1742170515000216

Sudars, K., Jasko, J., Namatevs, I., Ozola, L., Badaukis, N. (2020). Dataset of annotated food crops and weed images for robotic computer vision control. Data Brief 31, 105833. doi: 10.1016/j.dib.2020.105833

Swain, K. C., Nørremark, M., Jørgensen, R. N., Midtiby, H. S., Green, O. (2011). Weed identification using an automated active shape matching (AASM) technique. Biosyst. Eng. 110, 450–457. doi: 10.1016/j.biosystemseng.2011.09.011

Tretyakova, A., Grudanov, N., Kondratkov, P., Baranova, O., Luneva, N., Mysnik, Y., et al. (2020). A database of weed plants in the European part of Russia. Biodiver. Data J. 8, e59176. doi: 10.3897/BDJ.8.e59176

Wang, A., Zhang, W., Wei, X. (2019). A review on weed detection using ground-based machine vision and image processing techniques. Comput. Electron. Agric. 158, 226–240. doi: 10.1016/j.compag.2019.02.005

Yuan, H., Zhao, N., Chen, M. (2020). Research progress and prospect of field weed recognition based on image processing. J. Agric. Machinery 51, 323–334.

Keywords: Weed25, weed dataset, deep learning, weed identification, weed species

Citation: Wang P, Tang Y, Luo F, Wang L, Li C, Niu Q and Li H (2022) Weed25: A deep learning dataset for weed identification. Front. Plant Sci. 13:1053329. doi: 10.3389/fpls.2022.1053329

Received: 25 September 2022; Accepted: 24 October 2022; Published: 30 November 2022.

Reviewed by:

Copyright © 2022 Wang, Tang, Luo, Wang, Li, Niu and Li. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Pei Wang, [email protected] ; Hui Li, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Improving Weed Detection Using Deep Learning Techniques

• Conference paper
• First Online: 21 April 2021
• Cite this conference paper

• Shriti Gupta 13 ,
• Mohit Kumar 14 ,
• B. Govinda Satyanarayana 15 ,
• Sonia Aribam 16 ,
• Smita Tiwari 17 ,
• Rohit Kumar Kaliyar 17 ,
• Mohit Agarwal 17 &
• Nisha Ahuja 17  

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 177))

428 Accesses

1 Citations

In recent years, weeds are responsible for agricultural losses. To get rid of this problem, the farmers have to uniformly spray the whole field with the weedicides which require a huge quantity of weedicides. The process of spraying weedicides affects the environment. Weed detection in dense culture is a plant science problem that is important for field robotics where the detection of weed is currently a challenge so that the use of phytochemical products on crops can be reduced. To control and prevent specific weeds, a method of detecting the weed is presented in this paper. By collecting the plants and weeds datasets which are grayscale images, data is divided into training, validation, and testing datasets and then transported to the convolutional neural network. Based on the knowledge gained by the model, it can detect the weeds among plants. Utilization of a pre-trained VGG16 model for weed detection in dense cultures demonstrated improved performance compared to state of the art without the need for large datasets and high computational power for training.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Tejeda AI, Castro RC (2019) Algorithm of weed detection in crops by computational vision. In: 2019 international conference on electronics, communications and computers (CONIELECOMP). IEEE, pp 124–128

Google Scholar  

Chen L, Zhang JG, Su HF, Guo W (2010) Weed identification method based on probabilistic neural network in the corn seedlings field. In: 2010 international conference on machine learning and cybernetics, 11 July 2010, vol 3. IEEE, pp 1528–1531

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

Zhang C, Kovacs JM (2012) The application of small unmanned aerial systems for precision agriculture: a review. Precision Agric 13(6):693–712

Article   Google Scholar  

Lottes P, Khanna R, Pfeifer J, Siegwart R, Stachniss C (2017) UAV-based crop and weed classification for smart farming. In: 2017 IEEE international conference on robotics and automation (ICRA) 29 May 2017. IEEE, pp 3024–3031

Ibrahim Z, Sabri N, Mangshor NA (2018) Leaf Recognition using texture features for herbal plant identification. Indonesian J Electr Eng Comput Sci 9(1):152–156

Bakhshipour A, Jafari A, Nassiri SM, Zare D (2017) Weed segmentation using texture features extracted from wavelet sub-images. Biosys Eng 1(157):1–2

Barrero O, Rojas D, Gonzalez C, Perdomo S (2016) Weed detection in rice fields using aerial images and neural networks. In: 2016 XXI symposium on signal processing, images and artificial vision (STSIVA), Aug 2016. IEEE, pp 1–4

Rasti P, Ahmad A, Samiei S, Belin E, Rousseau D (2019) Supervised image classification by scattering transform with application to weed detection in culture crops of high density. Remote Sensing 11(3):249

Kamilaris A, Prenafeta-Boldú FX (2018) Deep learning in agriculture: a survey. Comput Electron Agric 1(147):70–90

Lottes P, Behley J, Milioto A, Stachniss C (2018) Fully convolutional networks with sequential information for robust crop and weed detection in precision farming. IEEE Robot Autom Lett 3(4):2870–2877

Bah MD, Hafiane A, Canals R (2018) Deep Learning with unsupervised data labeling for weeds detection on UAV images. arXiv preprint arXiv:1805.12395 . 31 May 2018

Wang S, Han Y, Chen J, Pan Y, Cao Y, Meng H, Zheng Y (2018) A deep-learning-based low-altitude remote sensing algorithm for weed classification in ecological irrigation area. In: International conference on intelligent technologies and applications. Springer, Singapore, pp 451–460

Bah MD, Hafiane A, Canals R (2018) Deep learning with unsupervised data labeling for weed detection in line crops in UAV images. Remote Sensing 10(11):1690 (2018)

Download references

Author information

Authors and affiliations.

Computer Science Engineering, Mody University of Science and Technology, Sikar, Rajasthan, India

Shriti Gupta

Computer Science Engineering, UIET Punjab University, Chandigarh, Punjab, India

Mohit Kumar

Computer Science Engineering, SRKR Engineering College, Andhra Pradesh, Bhimavaram, India

B. Govinda Satyanarayana

Computer Science Engineering, NIT Manipur, Imphal, Manipur, India

Sonia Aribam

Computer Science Engineering, Bennett University, Uttar Pradesh, Greater Noida, India

Smita Tiwari, Rohit Kumar Kaliyar, Mohit Agarwal & Nisha Ahuja

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Smita Tiwari or Rohit Kumar Kaliyar .

Editor information

Editors and affiliations.

SRM Institute of Science and Technology Delhi-NCR Campus, Modinagar, Uttar Pradesh, India

Rajendra Prasad Mahapatra

Indian Institute of Technology Delhi, New Delhi, Delhi, India

B. K. Panigrahi

Indian Institute of Technology Roorkee, Roorkee, Uttarakhand, India

Brajesh K. Kaushik

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper.

Gupta, S. et al. (2021). Improving Weed Detection Using Deep Learning Techniques. In: Mahapatra, R.P., Panigrahi, B.K., Kaushik, B.K., Roy, S. (eds) Proceedings of 6th International Conference on Recent Trends in Computing. Lecture Notes in Networks and Systems, vol 177. Springer, Singapore. https://doi.org/10.1007/978-981-33-4501-0_16

Download citation

DOI : https://doi.org/10.1007/978-981-33-4501-0_16

Published : 21 April 2021

Publisher Name : Springer, Singapore

Print ISBN : 978-981-33-4500-3

Online ISBN : 978-981-33-4501-0

eBook Packages : Intelligent Technologies and Robotics Intelligent Technologies and Robotics (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Advancing Weed Detection in Agricultural Landscapes Using Computer Vision

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.