Award-winning Work Information & Description
UlBot-object recognition and grasping robot control based on multi modal brain computer interface
Description
Based on the current situation of brain-computer interface and humanoid robot, this project will design a brain-controlled robot control system with hybrid brain-computer interface for the problems of low classification accuracy and few output commands of single-mode brain-computer interface, traditional brain-controlled robot system relying too much on brain-controlled operation, and insufficient fusion of nonlinear features of network used for target recognition, the motion control system of the robot forward, backward, turn left and turn right and the mechanical control system of grasping and releasing the ball are built to improve the accuracy of EEG signal and the accuracy of target recognition. The realization of this project can provide help for disabled patients in family services and personnel working in dangerous environments, which is practical and feasible. In the aspect of brain-computer interface, this system adopts non-invasive SSVE988P P300 hybrid brain-computer interface system. In the stimulation paradigm, we adopt the method based on cross-frequency modulation, which increases the characteristic difference between various stimulation buttons and improves the classification accuracy. In the asynchronous detection algorithm of SSVEP and P300, we classify the mixed EEG signals directly into the trained cnn lstm neural network model and the SVM algorithm model, and classify them according to the mixed characteristics of concurrent SSVEP P300. After classification, the corresponding command result is output according to the result and sent to the robot port. In the humanoid robot system, Yolo algorithm is improved for target recognition, and a YOLOv7-RAR model, namely YOLOv7-RAR algorithm, is proposed, which solves the problem of insufficient nonlinear feature fusion in the network and weak positioning of small targets, and also improves the reasoning speed of the model and the speed of the model reasoning, which can effectively learn and converge in a deeper network; Through CVTS target tracking, the NAO robot can capture and track the object in real time; the ultrasonic transmitter on the chest of the robot sends a signal and receives the ultrasonic signal through the ultrasonic receiver. After the signal is processed by the processor, the length of the robot from the object in front is fed back, so that the NAO robot can grasp and avoid obstacles. In the aspect of brain-controlled humanoid robot system, the sensors on the robot are fully utilized to realize machine intelligent decision-making. Users can understand the surrounding environment through multi-modal information fed back by robot sensors and cameras, which reflects the advantages of human brain in decision-making and perception, as well as the advantages of robots in response speed and computational intelligence, so that the machine itself can sense emergency situations and make decisions autonomously, thus enhancing the stability of BRF system.