摘要
文本分类作为自然语言领域中的重要任务之一,广泛应用于问答系统、推荐系统以及情感分析等相关任务中。为了提取文本数据中的复杂语义特征信息并捕获全局的图信息,提出一种融合图嵌入和BERT(bidirectional encoder representation from Transformers)嵌入的文本分类模型。该模型引入双级注意力机制考虑不同类型节点的重要性以及同一类型不同相邻节点的重要性,同时采用BERT预训练模型获得包含上下文信息的嵌入并解决一词多义的问题。该模型把所有单词和文本均视为节点,为整个语料库构建一张异构图,将文本分类问题转化为节点分类问题。将双级注意力机制与图卷积神经网络进行融合,双级注意力机制包含类型级注意力和节点级注意力。类型级注意力机制捕获不同类型的节点对某一节点的重要性,节点级注意力机制可以捕获相同类型的相邻节点对某一节点的重要性。将BERT模型获得的文本中局部语义信息与经图卷积神经网络得到的具有全局信息的图嵌入表示相结合,得到最后的文本嵌入表示,并完成文本分类。在4个广泛使用的公开数据集上与7个基线模型进行对比实验,结果表明本文模型提高了文本分类的准确性。
Text classification is one of the crucial tasks in the field of natural language and is widely used in related tasks such as question answering system,recommendation system,and sentiment analysis.In order to extract complex semantic feature information in text data and capture global graph information,a text classification model based on bidirectional encoder representation from Transformers(BERT)and a dual-level attention mechanism is proposed in this article.This model introduces a two-level attention mechanism to consider the importance of different types of nodes and the importance of dif-ferent neighboring nodes of the same type.At the same time,a BERT pre-training model is used to obtain embeddings con-taining contextual information and solve the problem of polysemy.This method treats all words and texts as nodes,builds a heterogeneous graph for the entire corpus,and transforms the text classification problem into a node classification problem.The dual-level attention mechanism is then integrated with the graph convolutional neural network.The dual-level attention mechanism includes type-level attention and node-level attention.The type-level attention mechanism captures the importance of different types of nodes to a certain node.The node-level attention mechanism can capture the importance of neighbor nodes of the same type to a certain node.Then,the local semantic information in the text obtained by the BERT model is combined with the graph embedding representation with global information obtained by the graph convolutional neural network to obtain the final text embedding representation and complete the text classification.Comparative experiments were conducted with seven baseline models on four widely used public data sets.The results showed that the proposed model improved the accuracy of text classification.
作者
常慧霞
李孝忠
CHANG Huixia;LI Xiaozhong(College of Artificial Intelligence,Tianjin University of Science and Technology,Tianjin 300457,China)
出处
《天津科技大学学报》
2025年第1期72-80,共9页
Journal of Tianjin University of Science & Technology
关键词
文本分类
图卷积神经网络
注意力机制
BERT
text classification
graph convolutional neural network
attention mechanism
BERT
