Deep Learning-Based Self-Driving Car: JetBot with NVIDIA AI Board to Deliver Items at Agricultural Workplace with Object-Finding and Avoidance Functions

In this study, we attempt to develop a deep learning-based self-driving car system to deliver items (e.g., harvested onions, agri-tools, PET bottles) to agricultural (agri-) workers at an agri-workplace. The system is based around a car-shaped robot, JetBot, with an NVIDIA artificial intelligence (AI) oriented board. JetBot can find diverse objects and avoid them. We implemented experimental trials at a real warehouse where various items (glove, boot, sickle (falx), scissors, and hoe), called obstacles, were scattered. The assumed agri-worker was a man suspending dried onions on a beam. Specifically, we developed a system focusing on the function of precisely detecting obstacles with deep learning-based techniques (techs), self-avoidance, and automatic delivery of small items for manual agri-workers and managers. Both the car-shaped figure and the deep learning-based obstacles-avoidance function differ from existing mobile agri-machine techs and products with respect to their main aims and structural features. Their advantages are their low costs in comparison with past similar mechanical systems found in the literature and similar commercial goods. The robot is extremely agile and easily identifies and learns obstacles. Additionally, the JetBot kit is a minimal product and includes a feature allowing users to arbitrarily expand and change functions and mechanical settings. This study consists of six phases: (1) designing and confirming the validity of the entire system, (2) constructing and tuning various minor system settings (e.g., programs and JetBot specifications), (3) accumulating obstacle picture data, (4) executing deep learning, (5) conducting experiments in an indoor warehouse to simulate a real agri-working situation, and (6) assessing and discussing the trial data quantitatively (presenting the success and error rates of the trials) and qualitatively. We consider that from the limited trials, the system can be judged as valid to some extent in certain situations. However, we were unable to perform more broad or generalizable experiments (e.g., execution at mud farmlands and running JetBot on non-flat floor). We present experimental ranges for the success ratio of these trials, particularly noting crashed obstacle types and other error types. We were also able to observe features of the system’s practical operations. The novel achievements of this study lie in the fusion of recent deep learning-based agricultural informatics. In the future, agri-workers and their managers could use the proposed system in real agri-places as a common automatic delivering system. Furthermore, we believe, by combining this application with other existing systems, future agri-fields and other workplaces could become more comfortable and secure (e.g., delivering water bottles could avoid heat (stress) disorders).