Multi-sensor fusion technology refers to the acquisition of a variety of information and the comprehensive processing and optimization of information technology. Multi-sensor fusion technology can effectively reduce useless and wrong information and obtain effective and correct parts according to the internal relations and laws of information. Compared with a single sensor, the use of multi-sensor fusion technology can enhance the accuracy and credibility of the data, thereby improving the robustness and reliability of the entire system.

In recent years, with the improvement of computer hardware and software, virtual reality technology is booming. However, there is still a lack of suitable input devices. This topic combines virtual reality with Kinect and Leap motion, and proposes a virtual reality interaction method based on multi-sensor. Compared with the traditional way of interaction, this kind of interaction technology can let people get rid of complex input devices and let the body interact with the virtual environment naturally.

This topic applies this interactive way of multi-sensor fusion to virtual chemical experiments. The discipline of chemistry not only attaches importance to theoretical knowledge, but also emphasizes the practical ability of hands-on experiments. Compared with the traditional information-based teaching methods, the virtual laboratory method proposed in this topic has many advantages such as simulation, safety, economy, reusability, and so on. It is also closer to the requirements of hands-on operation in chemical experiments.

The basic content of this topic is to develop a virtual laboratory based on multi-sensor. Among them are the following innovative modules:

1. Virtual reality module: to achieve a virtual chemical experiment environment. It is proposed to use the wireless head-mounted virtual reality terminal GearVR and create virtual scenes in the Unity3D engine.

2. Action recognition module: recognize the user's current body action, and its action real-time feedback to the virtual experiment scene, interact with the scene. The Kinect somatosensory camera is intended to be used.

3. Gesture recognition module: Identify the user's current subtle gestures, and feedback their actions to the virtual experiment scene in real time, and interact with the virtual experiment instrument. It is intended to use the somatosensory controller Leap Motion.

4. Wireless communication module: due to the use of wireless virtual reality head-mounted terminal, it is necessary to use wireless network communication to send sensor data to the terminal. It is proposed to adopt the LAN communication mode based on UDP protocol.

5. Chemical experiment module: simulate real chemical experiment operation and chemical reaction. The project is highly targeted and wide-ranging. Reduce the chemical experiments that are harmful to the human body and the environment, and the user can personally operate and understand the reaction process.

6. At present, there is no commercialized virtual science project in the market, and the audience is wide.