With the rapid development of network technology represented by 5G technology, the Internet has been widely used in industry, finance, education and other fields. However, while the Internet brings various conveniences to our lives, it also brings various security problems. The number and types of various Trojan horse programs, worms, viruses and other malicious software are also increasing rapidly, which is for Internet users. Security has brought more problems and hidden dangers. In order to protect the privacy information of ordinary users in network activities, encrypted transmission has become a widely used way. However, encrypted transmissions do not guarantee security, and malware developers have begun to use TLS to hide their malicious activities from detection. In order to better maintain network security, we propose the identification, classification and prediction of malicious traffic based on LightGBM algorithm. Firstly, the traffic data set obtained by packet capture is preprocessed, secondly, on the basis of analyzing the difference between malicious traffic and normal traffic, the characteristic dimension is reduced, and the characteristic factor set which can obviously distinguish normal traffic from malicious traffic is extracted, and finally, the encrypted traffic is identified, classified and predicted by LightGBM algorithm. The test results show that the identification and classification accuracy of malicious traffic based on LightGBM algorithm is high, and it has better robustness and wider applicability on the basis of maintaining considerable classification accuracy. In addition, we have developed malicious traffic identification, classification, and prediction systems based on front-end languages, python, React frameworks, and express frameworks.