With the development of science and technology, the popularity of smart home, but it is difficult for deaf people to enjoy these. With the gradual popularization of smart home, the wide application of voice interactive control, its convenience greatly improves the user experience, which is one of the concrete manifestations of science and technology for the benefit of mankind. But this scientific and technological dividend is difficult to enjoy for the special group-the deaf-mute. In view of this, this project designs a set of hardware and software integration system for the deaf-mute group, which enables the deaf-mute to replace speech recognition with gesture recognition, and then control the smart home, so that the special group can easily control the smart home system. The system can support as many as a dozen single instruction, can also achieve a combination of instructions, to achieve dozens of instruction operation effect, to meet the control of conventional smart home units, its response speed, and even in the experience beyond the voice control. Even non-deaf people, after mastering the usage of gesture instructions, will give priority to using this system instead of using voice interaction control in specific application scenarios. This project is a set of hardware and software for the deaf group, which can make the deaf easily control the smart home. This project will first capture gestures through the camera, and then use the YCbCr space to determine the skin color, and then extract and gray-scale the detected contours. After that, the Blaze Palm can return hand key points with high precision and create models with gestures. Finally, Media Pipe calculates and matches the gesture representation and calculates the result.