Deep Learning in Agri-Food Systems

1. Introduction

Deep Learning (DL) is a subset of Machine Learning, a subset of Artificial Intelligence. Deep Learning employs a variety of neural network (NN) architectures and algorithms for pattern recognition and data analysis. This soft computing technique is bringing revolutions in data science, as it can be used to analyze vast volumes of data in all possible forms viz. text, audio, 2D/3D image, and videos into meaningful insights. Recently, the agri-food industry has also benefited mainly from the development in this field. In food processing units, deep Learning is being implemented in machine vision systems for inspecting inline processes, quality of raw materials, and finished goods.

At the farm level, deep learning technology has been successfully applied in autonomous robotic systems tasked with picking agro-commodities with desired characteristics. Deep learning is also being used to monitor vegetation using low-altitude aerial systems coupled with a wide variety of sensors for data collection. This technique is favorable over other AI tools as it is versatile with the type of input data and its performance is directly proportional to the amount of data fed. The nature of data sourced from agro, and biological systems is complex as it can vary from continuous spectral information to spatial RGB images or a combination of both. Thus, deep neural network architectures such as the artificial and convolutional NN is the preferred tool for agricultural data analysis.

Modern-day food industries are required to follow several stringent food safety laws to avoid legal consequences due to negligence in hygiene and quality. A single case of the presence of foreign materials can tarnish the producer’s reliability amongst its consumers.

The consumption of even minimally processed foods is expected to rise manifolds in the coming years as the population is expected to rise by up to three billion in the coming three decades. Food businesses, therefore, have a great opportunity to capture this growth, if they can avoid negative publicity meanwhile, due to the lack of robust methods and quality control techniques. Thus, quality evaluation of the food materials in the whole food value chain is given the highest priority by the producers. Even though in the long run, as per Philip Crosby, maintaining a set of quality standards in the facility is beneficial to eliminate costs of operation due to rework and recall, but elaborate analysis of complex biological systems coupled with high volume in a modern-day unit is an elaborate, labor-intensive, and high resource-consuming task. To solve this issue, deep learning is a tested, reliable, and cost-effective method to automate this cumbersome ritual. In this article, DL architectures are introduced along with a few case studies in agri-food systems.

2. Artificial Neural Network

 

The artificial neural network (ANN) architecture is the simplest model for DL. It is a computational system composed of the systematic interconnection of many simple processing neurons operating in parallel whose function is determined by network structure, connection strengths, and the processing performed at the neurons . They are designed to mimic the human brain’s nervous system, to compute sophisticated computations for a specific task. The ANN architecture comprises several layers – an input layer where the data are provided, hidden layers where classification tasks are done, and the output layer predict the output. For an ANN to predict future data, it must be trained first, further which it becomes able to predict similar patterns. The basic information processing unit of an ANN is the neuron. An artificial neuron receives inputs (x_i) from many such neurons through input links having weight (w_i), finds a weighted sum (y_in = Σw_ix_i) to get a single input quantity (y_in), and computes an output (y) using an activation function (f()) The output is then transmitted to many other neurons through the output links. ANNs are applied in pattern recognition, function approximation, prediction/forecasting, optimization, and control.

3. Convolutional Neural Network

CNN is a specialized type of NN designed to be applied in computer vision tasks like image classification and object detection. Convolutional layers are the fundamental building blocks of CNNs. This layer performs an operation called a ‘convolution’. A convolution is a linear operation that involves the multiplication of a set of weights with the input. In the CNN architecture, convolutional layers and pooling layers are responsible for extracting hidden characteristics out of image pixels while the fully connected layer is responsible for classification.

4. Application in the Agri-Food Systems

Several researchers have implemented these algorithms to solve crucial problems in the food industry such as defects classification, shelf-life prediction, dietary assessment, food grading/sorting,  inspection, etc. The process of development and the forms of models that would yield from DL implementation in the agri-food system is not significantly different from those for other systems. However, the information and knowledge fed to the system as input data will help to characterize the individual systems and their features. Thus, the tools available for DL implementation in any platform remain valid for the analysis of agri-food systems . One of the most common issues plants face nowadays is the quality assurance of raw materials. Raw materials like fruits and vegetables often have a non-visible defect in their surface, but due to improper storage conditions, may get affected by internal pest infestation and pathological disorders. For instance, plantains with preexisting internal damage conditions when stored at optimum conditions (2% O₂ and 5 – 10% CO₂ at 10°C, 90% RH) will bypass a trained and experienced eye. This, however, won’t be an issue if there were a machine vision system in place. During the installation of the system, it would be trained with a comprehensive image dataset of thousands of hyperspectral images using a suitable NN architecture that yields optimum performance metrics. Hyperspectral images are 3-dimensional images that provide both spectral and spatial information. Many properties of fresh food – like Brix, dry-matter, internal damage, firmness – are not visible . These specialized camera systems can help in identifying these properties non-destructively and making an informed decision using a suitable DL model to correlate the spectral data with internal quality parameters, which even a trained master sorter may not be capable of.

4.1. Deep Learning in Quality Control

Quality control is a costly process, but a critical step in the food industry. Traditional techniques are time-consuming and hence machine vision systems are being installed for better performance. In recent times, several industries, as well as academic units, have investigated the performance of models suitable for specialized tasks. For instance, Liu et al. developed a 91.1% accurate DL model to monitor the quality of cucumbers in a pickle processing line.

Similarly, Vasumathi et al.  employed a 98.17% accurate DL technique to sort pomegranate fruits into normal and abnormal based on the color, number of fruit spots, and shape of it. Adulteration of food is also a major cause of concern in maintaining food safety. Al-Sarayreh et al. developed a model to detect lamb meat adulteration with pork, beef, and fat. They studied the deep spectral-spatial features in hyperspectral images of the sample and reported an overall accuracy of 94.4% invariant of the status of the meat. Quality assessment of nuts like Almonds is a very tedious laboratory process that involves wet laboratory techniques using Soxhlet apparatus, which can take up to 12 hours.

To find a reliable alternative, Han et al. developed a quick and non-invasive method for quality estimation of nuts which are categorized by peroxide values (a quality indicator) using hyperspectral imaging with DL classification. They reported accuracy of 93.48%. In recent times, due to progress in computing powers and the development of imaging hardware, more studies are being carried out to develop better models for solving problems of the agri-food industry.

4.2. Deep Learning in Cleaning processing equipment

Cleaning processing equipment requires a lot of time and resources, including water. To address this problem, research engineers at the University of Nottingham have developed a system that uses DL to reduce cleaning time and resources by 20 – 40%. The system, which they call Self-Optimizing CIP, uses ultrasonic sensing and optical fluorescence imaging to measure food residue and microbial debris in a piece of equipment and then optimize the cleaning process.

4.3. Deep Learning in Farming

Post-harvest loss is a major concern amongst farmers. The major causes are infestations and infections, lack of nutrition in the soil, spoilage due to improper storage conditions, and premature/postmature harvesting. Due to the bulk handling of agro-commodities, defects can spread throughout the entire lot if there’s no constant monitoring for indications. In this regard, Deep learning shows great potential in mitigating these issues. For instance, Behera et al. [10] developed a 100% accurate non-destructive method to classify papaya fruit on its maturity status. They trained the network using only 300 images categorized into three classes of maturity.

Identifying diseases in the plants is important in preventing them from further spreading. Park et al. [11] developed a model for strawberry leaves to classify healthy, powdery mildew and gray mold rot fruit with 92% accuracy. These models can be integrated into autonomous vehicles for picking fruits with desired characteristics. Thanks to deep learning and other supporting techniques such as chemometrics, it is now possible to engineer devices that can automatically sort and grade potatoes in terms of size for different applications, apples in levels of sweetness, plantains in levels of resistant starch content, etc.

5. Challenges in implementation

As opposed to the popular belief from science fiction literature, achieving general artificial intelligence is still a long journey. Till then, it is important to develop specialized models for completing specialized tasks. Even though a fully functional model does not require any special assistance while in operation, the initial setup cost is significantly high. In terms of the complexity of the overall deep learning-enabled system in the industry, model selection is the most important step in this process. The first challenge is to collect relevant and un-skewed data. Data that is specialized for the process but also generalized enough to avoid overfitting. The second challenge is to choose a NN architecture that will yield acceptable and sensible results. These challenges can be addressed using techniques such as hyperparameter tuning, regularization, increasing the dataset, and transfer learning.

6. References

• • V. Fausett, Fundamentals of neural networks: architectures, algorithms, and applications. Pearson Education India, 2006.
  • O’Shea, K. and Nash, R., 2015. An introduction to convolutional neural networks, arXiv preprint arXiv; pp.1511.08458.
  • Begum, Ninja, and Manuj Kumar Hazarika. “Artificial Intelligence in Agri-Food Systems—An Introduction.” Internet of Things and Analytics for Agriculture, Volume 3. Springer, Singapore, 2022. 45-63.
  • Wageningen Food and Biobased research – Computer Vision and Robotics for the agri-food industry.
  • Liu, Y. He, H. Cen, and R. Lu, “Deep feature representation with stacked sparse auto-encoder and convolutional neural network for hyperspectral imaging-based detection of cucumber defects,” Transactions of the ASABE, vol. 61, no. 2, p. 425-436, 2018.
  • Vasumathi and M. Kamarasan, “An effective pomegranate fruit classification based on CNN-LSTM deep learning models,” Indian Journal of Science and Technology, vol. 14, no. 16, pp. 1310-1319, 2021
  • Al-Sarayreh, M. M Reis, W. Qi Yan, and R. Klette, “Detection of red-meat adulteration by deep spectral-spatial features in hyperspectral images,” Journal of Imaging, vol. 4, no. 5, p. 63, 2018
  • Han, Z. Liu, K. Khoshelham, and S. H. Bai, “Quality estimation of nuts using deep learning classification of hyperspectral imagery,” Computers and Electronics in Agriculture, vol. 180, p. 105868, 2021.
  • Escrig, Josep, et al. “Monitoring the cleaning of food fouling in pipes using ultrasonic measurements and machine learning.” Food Control 116 (2020): 107309.
  • K. Behera, A. K. Rath, and P. K. Sethy, “Maturity status classification of papaya fruits based on machine learning and transfer learning approach,” Information Processing in Agriculture, 2020.
  • Park, E. JeeSook, and S.-H. Kim, “Crops disease diagnosing using image-based deep learning mechanisms,” in 2018 International Conference on Computing and Network Communications (CoCoNet). IEEE, 2018, pp. 23-26.