Semantic knowledge LDA with topic vector for recommending hashtags: Twitter use case

Topic Modeling encompasses a set of techniques for text clustering and tag recommendation with significant advantages such as unsupervised learning. Based on Latent Dirichlet Allocation (LDA) topic modeling, every single word is related to a set of topics with different weight. The weights are further estimated in order to determine the semantic relation between the words and the rest of the documents. Apparently the chief drawback of topic modeling techniques, specifically LDA, lies on their incapability in clustering short texts in which semantic relation between words is neglected. This issue is deemed more severe when analyzing social networks such as Twitter wherein short texts are the case. It is assumed that semantic relation between a document and the target short text helps obtain efficient clustering of short texts via topic modeling. Hence, the current paper proposes a method of topic modeling named Semantic Knowledge LDA based on semantic relations between the words in tweets from Twitter social network based on the co-occurrence of words. Additionally, we propose a method of hashtag recommender system based on topic vector (TV) text similarity, named TV based Hashtag Recommender System (TVHRS). Accordingly, we applied our word co-occurrence LDA (SKLDAC) method together with WordNet lexical database to cluster the short texts from Twitter. The clustered topics are later used as the repository for the proposed hashtag recommender system. The proposed system of both SKLDA and TVHRS were applied to a set of 12,309,911 real tweets for testing purposes. When comparing the components of the proposed system to the existing methods, we recorded higher performance in terms of precision, recall and F-Score of 0.551, 0.682 and 0.526, respectively.