A vision based P300 Brain Computer Interface for grasping using a wheelchair-mounted robotic arm

In this paper, we present a novel vision based interface for selecting an object using a Brain Computer Interface (BCI), and grasping it using a robotic arm mounted to a powered wheelchair. As issuing commands through BCI is slow, this system was designed to allow a user to perform a complete task using the robotic system via the BCI issuing as few commands as possible, without losing concentration on the stimuli or the task. A scene image is captured by a camera mounted on the wheelchair, from which a dynamically sized non-uniform stimulus grid is created using edge information. Dynamically sized grids improve object selection efficiency. Oddball paradigm and P300 event related potentials (ERP) are used to select stimuli, the stimuli being each cell in the grid. Once selected, object segmentation and matching is used to identify the object. Then the user, using BCI, chooses an action to be performed on the object via the wheelchair mounted robotic arm (WMRA). Tests on 6 healthy human subjects validated the functionality of the system. An average accuracy of 85.56% was achieved for stimuli selection over all subjects. With the proposed system, it took the users an average of 5 commands to grasp an object. The system will eventually be useful for completely paralyzed or locked-in patients for performing activities of daily living (ADL) tasks.