Japanese farmer builds AI system to sort cucumbers
A Japanese farmer who left his city job as an engineer to return to his parents' cucumber farm has created an AI system to potentially replace the laborious task of sorting the vegetables manually. Makoto Koike used Google's TensorFlow to build the AI system, which at present has an accuracy rate of only 70% so each cucumber still has to be checked by hand – mostly by Koike's mother, who sometimes sorts up to 4,000 cucumbers a day. However, Koike hopes to improve his system enough to relieve his mother of her duties in the future.

Not much about Makoto Koike’s adult life suggests that he would be a farmer. Trained as an engineer, he spent most of his career in a busy urban section of Aichi Prefecture, Japan, near the headquarters of the Toyota Motor Corporation, writing software to control cars. Koike’s longtime hobby is tinkering with electronic kits and machines; he is not naturally an outdoorsy type. Yet, in 2014, at the age of thirty-three, he left his job and city life to move to his parents’ cucumber farm, in the greener prefecture of Shizuoka. “I thought I was getting old,” Koike told me. “I wanted to be close to my home and my family.”

The Koikes have been growing cucumbers in Kosai, a town wedged between the Pacific Ocean and the brackish Lake Hamana, for nearly fifty years. Their crop, which fills three small greenhouses, grows year-round. Koike’s father, Harumi, plants the seeds; Koike oversees their cultivation; and his mother, Masako, sorts the harvest. This last job is particularly important in Japan, which is famously discerning about its produce. Nice strawberries can fetch several dollars apiece in some markets, and a sublime cubic watermelon can go for hundreds. Vegetables hold a less privileged place than fruits, but supermarkets rarely stock produce that is at all irregular in shape or size. The Koikes send their better cucumbers, the ones that are straight and uniform in thickness, to wholesalers. The not-so-perfect ones go to local stands, where they are sold at half price. (“They taste the same,” Koike said.) Masako judges the vegetables one by one, separating them into bins. Though she devotes only half a second to each cucumber, the task takes up most of her work time; on some days, she goes through around four thousand of them.

The laborious process of categorizing the cucumbers had remained essentially the same for decades, until last spring, when Koike began developing a new approach. It was inspired, in part, by articles he read about AlphaGo, the first computer program ever to beat a human master of the game of Go. Developed by Google DeepMind, the program relied on deep learning, a method for making computations by arranging basic processing units into complex, layered networks, rather like the way that billions of neurons work together to produce the incomparable (for now) intelligence of the human brain. In the past several years, deep learning has proved exceptionally useful for finding patterns in big piles of data; it has been incorporated into Facebook’s facial-recognition algorithms, Amazon Alexa’s language processing, and autonomous cars’ navigation systems. In AlphaGo’s case, the program was fed thirty million images of positions from real games, which it used to help determine which kinds of moves work best. Koike hoped that a similar strategy might help him sort his family’s cucumbers.

Makoto Koike with his parents, Harumi and Masako, at the family’s cucumber farm, in Shizuoka Prefecture, Japan. The Koikes have been growing cucumbers for nearly fifty years. Photograph Courtesy Makoto Koike

Advanced A.I. techniques, including deep learning, have traditionally been the province of specialized researchers and moneyed software companies. Recently, though, some of the tech world’s biggest players—including Google, Facebook, Microsoft, Amazon, Yahoo, Baidu, Yandex, and various universities—have released free, open-source versions of their tools, making A.I. accessible to small-time programmers who aren’t well-versed in the field, such as Koike. For his project, he used TensorFlow, which Google released to the public in 2015. He began by building a custom photo stand, which allowed him to photograph each cucumber from three angles. Then, to analyze the images, he adapted a popular piece of TensorFlow software used for recognizing handwritten numerals. Before he could turn the A.I. loose, though, Koike had to train it. He captured seven thousand photos of cucumbers that his mother had already sorted, then used the data to teach his software to recognize which vegetables belonged in which categories. Finally, he built an automated conveyor-belt system to move each cucumber from the photo stand to the bin designated by the program.

Koike completed his machine last year, and it works—to some degree. It sorts cucumbers with an accuracy of seventy per cent, which is low enough that they must subsequently be checked by hand. What’s more, the vegetables still need to be placed on the photo stand one by one. Koike’s mother, in other words, is in no immediate danger of being replaced, and thus far, she and her husband are none too impressed. “They are quite severe,” Koike said. “ ‘Oh, it’s not useful yet,’ ” they tell him. Tech enthusiasts, meanwhile, have had decidedly more positive reactions, and Koike has been invited to events such as the Maker Faire, in Tokyo, and the CeBIT expo, in Hanover, Germany. There are machines that work better and faster than Koike’s, but they are industrial-sized and -priced affairs. Before the democratization of deep learning, it would have been difficult for someone like him to design such an effective device himself.

Koike sees his system as an encouraging proof of concept, and he is currently at work on a new version, which he hopes will be capable of analyzing more than one cucumber at a time. He also plans to build a gentler conveyor system, to preserve the fragile prickles on the vegetables’ skin, which are considered a sign of freshness. He expects that, within a few years, his A.I. sorter will be nearly as accurate as his mother, freeing her up to do something else. Either way, he told me, he is back in Kosai for the long haul. “That’s the plan,” he said. “I’ll probably die as a farmer.” By the time that happens, the work may look very different.