AI Case Study

Makoto Cucumber Farm automatically sorts cucumbers based on shape, length and distortion using deep machine vision

Cucumbers and their market price vary according to colour, shape, quality and freshness with no industry-wide classification scheme. The Japanese family farm, Makoto, spent months training staff to classify cucumbers into nine categories followed by intensive manual labour. They developed a deep neural network on TensorFlow to identify and sort cucumbers in real-time. They created the sorting classification based on training from the manual labelling of 7,000 images of cucumbers. It resulted in sorting based on shape, length and level of distortion of the cucumber but not on colour, texture, scratches and prickles.

Industry

Basic Materials

Agriculture

Project Overview

"The system uses Raspberry Pi 3 as the main controller to take images of the cucumbers with a camera, and in a first phase, runs a small-scale neural network on TensorFlow to detect whether or not the image is of a cucumber. It then forwards the image to a larger TensorFlow neural network running on a Linux server to perform a more detailed classification."

Reported Results

"Even with this low-res image, the system can only classify a cucumber based on its shape, length and level of distortion. It can't recognize color, texture, scratches and prickles."

Technology

"Makoto used the sample TensorFlow code Deep MNIST for Experts with minor modifications to the convolution, pooling and last layers, changing the network design to adapt to the pixel format of cucumber images and the number of cucumber classes.

'When I did a validation with the test images, the recognition accuracy exceeded 95%. But if you apply the system with real use cases, the accuracy drops down to about 70%. I suspect the neural network model has the issue of overfitting (the phenomenon in neural network where the model is trained to fit only to the small training dataset) because of the insufficient number of training images.'

The second challenge of deep learning is that it consumes a lot of computing power. The current sorter uses a typical Windows desktop PC to train the neural network model. Although it converts the cucumber image into 80 x 80 pixel low-resolution images, it still takes two to three days to complete training the model with 7,000 images.

'Even with this low-res image, the system can only classify a cucumber based on its shape, length and level of distortion. It can't recognize color, texture, scratches and prickles,' Makoto explained. Increasing image resolution by zooming into the cucumber would result in much higher accuracy, but would also increase the training time significantly."

Function

Operations

General Operations

Background

Cucumbers vary according to colour, shape, quality and freshness. "Straight and thick cucumbers with a vivid color and lots of prickles are considered premium grade and command much higher prices on the market."

'Sorting cucumbers is as hard and tricky as actually growing them.' 'The sorting work is not an easy task to learn. You have to look at not only the size and thickness, but also the color, texture, small scratches, whether or not they are crooked and whether they have prickles. It takes months to learn the system and you can't just hire part-time workers during the busiest period. I myself only recently learned to sort cucumbers well.'

"There are also some automatic sorters on the market, but they have limitations in terms of performance and cost, and small farms don't tend to use them."

Benefits

Data

"To train the model, Makoto spent about three months taking 7,000 pictures of cucumbers sorted by his mother, but it’s probably not enough."