Job planning and supervisory control for automated earthmoving using 3D graphical tools

Planning tools and algorithms are presented for enabling supervisory control of automated earthmoving performed by a robotic wheel loader. Interactive 3D graphical objects are rendered over a worksite model, and allow a remote human operator to specify high-level plans for pile transfer and area clearing jobs. These are automatically translated into lower-level plans for the machine to follow and are displayed graphically back to the operator, who then mostly monitors work but can intervene in a supervisory capacity. The tools were developed and tested using Matlab, and were able to guide simulated jobs to completion. Outdoor manually-driven tests using snow were also conducted to verify that heightmaps from a 3D laser rangefinder could be used to correctly track progress and generate commands using the same tools and algorithms. Augmented Reality versions of the tools present a concept of how they could be used in real-world applications.