Journal Article

Research on user granularity-level personalized social text generation technology

Gao, J T, Ma, R, Yang, L D, Gao, Y B

2022

Overview

With the introduction of large-scale pre-trained language models, breakthroughs have been made in text generation technology research. On this basis, in order to assist users to complete personalized creation, this paper proposes a user-level fine-grained control generation model. First, we design the Encoder-Decoder framework based on the GPT2 structure, and model and encode the user’s static personalized information on the Encoder side. Then add a bidirectional independent attention module to receive the personalized feature vector on the Decoder side. The attention module in the original GPT2 structure captures the dynamic personalized features in the user text, namely writing style, expression way, etc. Next, the scores of each attention module are weighted and fused to participate in the subsequent decoding to automatically generate social texts constrained by the user’s personalized feature attributes. However, the semantic sparsity of the user’s basic information will cause occasional conflicts between the generated text and some personalized features. Therefore, we use the Alignment module to perform the secondary enhancement and generation of consistent understanding between the output data of the Decoder and the user’s personalized features, and finally realize the personalized social text generation. Experiments show that compared with the GPT2 baseline model, the fluency of the model is improved by 0.3%-0.6%, and on the basis of no loss of language fluency, the social text generated by the model can have significant user personalization characteristics, among which personalization and consistency the two evaluation indicators of sexuality both increased significantly by 8.4% and 9%.

• Publisher: IOP Publishing
• Subject: Coders / Customization / Decoding / Encoders-Decoders / Modules / Physics