Award-winning Work Information & Description
Explainable image text description model based on multi-model fusion -- a new vision of e-commerce in the digital econom
Description
Under the wave of digital economy, the traditional commodity display scheme is difficult to attract users to browse and buy. E-commerce platforms urgently need to improve the way of commodity display to enhance user experience. This team proposes an interpretable image text description model based on multi-model fusion. CLIP model with analytic attention mechanism is used to extract advanced features of input image. The prediction results of BLIP2, ConvNeXtxxlarge and CLIP model after LoRA fine tuning are applied to integrate and fuse to generate accurate, detailed and attractive commodity text description. In the operation practice of e-commerce, we design an end-to-end strong interaction web page, B- end and c-end users upload pictures, call the model to automatically extract the key features of goods and predict, get the text description. Based on the multi-modal algorithm, combined with the latest design elements and popular trends, the model can generate attractive and detailed text descriptions to enhance the competitiveness of the commodity market. At the same time, the interpretability of the model guarantees the traceability of the generated text description, so that users can make personalized customization and adjustment. Therefore, the model can significantly improve the quality of product description and optimize the overall user experience. The platform developed in this work can bring a new application experience for e-commerce under the background of artificial intelligence.