User review analysis of dating apps based on text mining pmc


User review analysis of dating apps based on text mining
Qian Shen, Conceptualization , Data curation , Formal analysis , Investigation , Methodology , Software , Supervision , Validation , Visualization , Writing – original draft , Writing – review & editing , Siteng Han, Conceptualization , Data curation , Formal analysis , Investigation , Methodology , Visualization , Writing – original draft , Yu Han, Data curation , Formal analysis , Validation , Visualization , Writing – original draft , and Xi Chen, Formal analysis , Funding acquisition , Methodology , Project administration , Resources , Supervision , Writing – review & editing *
User review analysis of dating apps based on text mining
Qian Shen, Conceptualization , Data curation , Formal analysis , Investigation , Methodology , Software , Supervision , Validation , Visualization , Writing – original draft , Writing – review & editing , Siteng Han, Conceptualization , Data curation , Formal analysis , Investigation , Methodology , Visualization , Writing – original draft , Yu Han, Data curation , Formal analysis , Validation , Visualization , Writing – original draft , and Xi Chen, Formal analysis , Funding acquisition , Methodology , Project administration , Resources , Supervision , Writing – review & editing *
Qian Shen
School of Statistics, Xi’an University of Finance and Economics, Xi’an, Shaanxi, China
Find articles by Qian Shen
Siteng Han
School of Statistics, Xi’an University of Finance and Economics, Xi’an, Shaanxi, China
Find articles by Siteng Han
Yu Han
School of Statistics, Xi’an University of Finance and Economics, Xi’an, Shaanxi, China
Find articles by Yu Han
Xi Chen
School of Statistics, Xi’an University of Finance and Economics, Xi’an, Shaanxi, China
Find articles by Xi Chen
Viacheslav Kovtun, Editor
School of Statistics, Xi’an University of Finance and Economics, Xi’an, Shaanxi, China
Vinnytsia National Technical University, UKRAINE
Corresponding author.
Competing Interests: NO authors have competing interests.
Received 2022 Nov 2; Accepted 2023 Mar 20.
Copyright © 2023 Shen et al
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Associated Data
S1 Graphical abstract: (TIF)
GUID: F192FA1A-6735-46AC-80AA-73E7633C0D15
Attachment: Submitted filename: Response to reviewers.pdf
GUID: 910F5E2E-3FB2-40B0-A100-54C76F71C8E1
Abstract
With the continuous development of information technology, more and more people have become to use online dating apps, and the trend has been exacerbated by the COVID-19 pandemic in these years. However, there is a phenomenon that most of user reviews of mainstream dating apps are negative. To study this phenomenon, we have used topic model to mine negative reviews of mainstream dating apps, and constructed a two-stage machine learning model using data dimensionality reduction and text classification to classify user reviews of dating apps. The research results show that: firstly, the reasons for the current negative reviews of dating apps are mainly concentrated in the charging mechanism, fake accounts, subscription and advertising push mechanism and matching mechanism in the apps, proposed corresponding improvement suggestions are proposed by us; secondly, using principal component analysis to reduce the dimensionality of the text vector, and then using XGBoost model to learn the low-dimensional data after oversampling, a better classification accuracy of user reviews can be obtained. We hope These findings can help dating apps operators to improve services and achieve sustainable business operations of their apps.
1 Introduction
Maybe the term ‘online dating’ sounded weird in the 1990s, but today we have become accustomed to it. Mobile phones are ‘virtual bars’ in people’s pockets, allowing singles to socialize wherever they want. At least 200 million people worldwide use digital dating services every month, a study of Smith and Duggan [1] found that one in ten Americans has used online dating websites or mobile dating apps; sixty-six percent of online daters have met someone they know through dating websites or apps, and 23 percent have met spouses or long-term partners through these sites or apps. One of Statista’s report [2] claimed that in 2020, there would be 44.2 million online dating service users in United States. The company’s digital market outlook estimated that number will increase to 53.3 million by 2025. Due to the COVID-19 pandemic since 2020, many activities of people have shifted from offline to online. It has also led to a significant increase in the frequency of online dating app users using them. Chisom’s research work [3] suggested that loneliness and boredom due to adhering to the stay at home policy in the age of COVID-19, there was a rapid increase of online dating apps especially on Tinder and had in so many ways. In other words, dating apps have very good market prospects at present.
However, a good market prospect also means that there will be cruel competition among enterprises behind it. For operators of dating apps, one of the key factors in keeping their apps stable against the competitions or gaining more market share is getting positive reviews from as many users as possible. In order to achieve this goal, operators of dating apps should analyze the reviews of users from Google Play and other channels in a timely manner, and mine the main opinions reflected in the user reviews as an important basis for formulating apps’ improvement strategies. The study of Ye, Law and Gu [4] found significant relationship between online consumer reviews and hotel business performances. This conclusion can also be applied on apps. Noei, Zhang and Zou [5] claimed that for 77% of apps, taking into account the key content of user reviews when updating apps was significantly associated with an increase in ratings for newer versions of apps.
For user reviews of apps presented in a textual state, we believe that text mining models can be used to analyze these reviews. Some researchers such as M Lee, M Jeong and J Lee [6] have studied the impact of online user negative reviews on consumers’ choice when booking a hotel through text mining. Latent Dirichlet Allocation (LDA) was proposed by Blei et al. [7]. Since then, topic models based on LDA have become one of the key research areas of text mining. LDA is very widely used in the commercial fields. For example, Wahyudi and Kusumaningrum [8] have used an LDA-based topic model to perform sentiment analysis on user reviews of online shopping malls in Indonesia in their study.
Most of the sentences that people speak every day contain some kinds of emotions, such as happiness, satisfaction, anger, etc. We tend to analyze the emotions of sentences according to our experience of language communication. Feldman [9] thought that sentiment analysis is the task of finding the opinions of authors about specific entities. Operators of dating apps usually collect user feelings and opinions through questionnaires or other surveys within the websites or apps. For many customers’ opinions in the form of text collected in the surveys, it is obviously impossible for operators to use their own eyes and brains to watch and judge the emotional tendencies of the opinions one by one. Therefore, we believe that a feasible method is to first build a suitable model to fit the existing customer opinions that have been classified by sentiment tendency. In this way, the operators can then obtain the sentiment tendency of the newly collected customer opinions through batch analysis of the existing model, and conduct more in-depth analysis as needed.
At present, many machine learning and deep learning models can be used to analyze text sentiment which is processed by word segmentation. In the study of Abdulkadhar, Murugesan and Natarajan [10], LSA (Latent Semantic Analysis) was firstly used for feature selection of biomedical texts, then SVM (Support Vector Machines), SVR (Support Vactor Regression) and Adaboost were applied to the classification of biomedical texts. Their overall results show that AdaBoost performs better compared to two SVM classifiers. Sun et al. [11] proposed a text-information random forest model, which proposed a weighted voting mechanism to improve the quality of the decision tree in the traditional random forest for the problem that the quality of the traditional random forest is difficult to control, and it was proved that it can achieve better results in text classification. Aljedani, Alotaibi and Taileb [12] have explored the hierarchical multi-label classification problem in the context of Arabic and propose a hierarchical multi-label Arabic text classification (HMATC) model using machine learning methods. The results show that the proposed model was superior to all the models considered in the experiment in terms of computational cost, and its consumption cost is less than that of other evaluation models. Shah et al. [13] constructed a BBC news text classification model based on machine learning algorithms, and compared the performance of logistic regression, random forest and K-nearest neighbor algorithms on datasets. The results show that logistic regression classifier with the TF-IDF Vectorizer feature attains the highest accuracy of 97% for the data set. Jang et al. [14] have proposed an attention-based Bi-LSTM+CNN hybrid model that takes advantage of LSTM and CNN and has an additional attention mechanism. Testing results on Internet Movie Database (IMDB) movie review data showed that the newly proposed model produces more accurate classification results, as well as higher recall and F1 scores, than single multilayer perceptron (MLP), CNN or LSTM models and hybrid models. Lu, Pan and Nie [15] have proposed a VGCN-BERT model that combines the capabilities of BERT with a lexical graph convolutional network (VGCN). In their experiments with several text classification datasets, their proposed method outperformed BERT and GCN alone and was more effective than previous studies reported.
However, in practice when the text contains many words or the numbers of texts are large, the word vector matrix will obtain higher dimensions after word segmentation processing. Therefore, we should consider reducing the dimensions of the word vector matrix first. The research of Vinodhini and Chandrasekaran [16] showed that dimensionality reduction using PCA (principal component analysis) can make text sentiment analysis more effective. LLE (Locally Linear Embedding) is a manifold learning algorithm that can achieve effective dimensionality reduction for high-dimensional data. He et al. [17] believed that LLE is very effective in dimensionality reduction of text data.
Currently, there are fewer text mining studies on user reviews of apps that people use every day, but this field has caught the attention of researchers [18]. Much of the research on dating apps now focuses on psychology and sociology, with minority of studies looking at dating apps from a business perspective. The study by Ranzini, Rosenbaum and Tybur [19] found that Dutch people are more likely to choose Dutch people as potential partners when using dating apps, while Dutch people with higher education are more likely to choose potential partners with higher education backgrounds when using dating apps. Tran et al. [20] found that users of dating apps had significantly higher odds of unhealthy weight-control behaviors than those who had not used dating apps. Rochat et al. [21] used cluster analysis to study the characteristics of Tinder users. The results show that Tinder users participating in the study could be reasonably divided into four groups, and the users of each group were different in gender, marital status, depression and usage patterns. Tomaszewska and Schuster [22] compared perceptions related to sexuality of dating app users and non-dating app users, namely their risky sexual scripts and sexual self-esteem, and their risky and sexually assertive behaviors. Results showed that dating app users had more risky sexual scripts and reported more risky sexual behaviors than non-dating app users. In addition, male dating app users had lower sexual self-esteem and were more accepting of sexual coercion than male non-dating app users. Lenton et al. [23] studied the relationship between social anxiety and depressive symptoms of dating app users and their degree of dating app use, they found that dating app user social anxiety and depressive symptoms were positively correlated with their level of dating app use, and that these symptoms predicted that men were less likely to initiate contact with people matched by dating apps, but not women.
In some research work, researchers have proposed methods or tools to help operators of apps, websites, hotel etc. to analyze user reviews. Considering that user reviews for apps are valuable for app operators to improve user experience and user satisfaction, but manually analyzing large numbers of user reviews to get useful opinions is inherently challenging, Vu et al. [24] proposed MARK, a keyword-based semi-automated review analysis framework that can help app operators analyze user reviews more effectively to get useful input from users. Jha and Mahmoud [25] proposed a novel semantic approach for app review classification, it can be used to extract user needs from application evaluations, enabling a more efficient classification process and reducing the chance of overfitting. Dalal and Zaveri [26] proposed a view mining system for binary and fine-grained sentiment classification that can be used for user reviews, and empirical studies show that the proposed system can perform reliable sentiment classification at different granularity levels. Considering that a large number of user reviews need to be explored, analyzed, and organized to better assist website operators in making decisions, Sharma, Nigam and Jain [27] proposed an aspect-based opinion mining system to classify reviews, and empirically demonstrated the effectiveness of this system. Considering that hotel managers in Bali can gain insight into the perceived state of the hotel through hotel user reviews, Prameswari, Surjandari and Laoh [28] used text mining methods and aspect-based sentiment analysis in their research to capture hotel user opinions in the form of emotions. The results show that the Recursive Neural Tensor Network (RNTN) algorithm performs well in classifying the sentiment of words or aspects. As a result, we wish to applying machine learning models on mining user reviews of dating apps. In this way, operators of apps can better manage their user review data and improve their apps more effectively.
Considering the increasing popularity of dating apps and the unsatisfactory user reviews of major dating apps, we decided to analyze the user reviews of dating apps using two text mining methods. First, we established a topic model based on LDA to mine the negative reviews of mainstream dating apps, analyzed the main reasons why users give negative reviews, and put forward corresponding improvement suggestions. Next, we built a two-stage machine learning model that combined data dimensionality reduction and data classification, hoping to obtain a classification that can effectively classify user reviews of dating apps, so that app operators can process user reviews more effectively.
2 Data acquisition and research design
2.1 Data acquisition
At present, there are several dating apps that are widely used, such as the famous Tinder and Okcupid. Since most users download these apps from Google Play, we believed that app reviews on Google Play can effectively reflect user feelings and attitudes toward these apps. All the data we used are from reviews of users of these six dating apps: Bumble, Coffee Meets Bagel, Hinge, Okcupid, Plenty of Fish and Tinder. The data are published on figshare.com [29], we promise that sharing the dataset on Figshare complies with the terms and conditions of the sites from which data was accessed. Also, we promise that the methods of data collection used and its application in our study comply with the terms of the website from which the data originated. The data include the text of the reviews, the number of likes the reviews get, and the reviews’ ratings of the apps. At the end of May 2022, we have collected a total of 1,270,951 reviews data. First of all, in order to prevent the impact on the results of text mining, we first carried out text cleaning, deleted symbols, irregular words and emoji expressions, etc.
Considering that there may be some reviews from bots, fake accounts or meaningless duplicates among the many reviews, we believed that these reviews can be filtered by the number of likes they get. If a review has no likes, or just a few likes, it can be considered that the content contained in the review is not of sufficient value in the study of user reviews, because it can’t get enough commendations from other users. In order to keep the size of data we finally use not too small, and to ensure the authenticity of the reviews, we compared the two screening methods of retaining reviews with a number of likes greater than or equal to 5 and retaining reviews with a number of likes greater than or equal to 10. Among all the reviews, there are 25,305 reviews with 10 or more likes, and 42,071 reviews with 5 or more likes.
In order to maintain a certain generality and generalizability of the results of the topic model and classification model, it is considered that relatively more data is a better choice. Therefore, we selected 42,071 reviews with a relatively large sample size with a number of likes greater than or equal to 5. In addition, in order to ensure that there are no worthless comments in the filtered comments, such as repeated negative comments from robots, we randomly selected 500 comments for careful reading and found no obvious worthless comments in these reviews. For these 42,071 reviews, we plotted a pie chart of reviewers’ ratings of these apps, and the numbers such as 1,2 on the pie chart means 1 and 2 points for the app’s ratings.
Looking at Fig 1 , we find that the 1-point rating, which represents the worst review, accounts for the majority of the reviews on these apps; while all of the percentages of other ratings are all less than 12% of the reviews. Such a ratio is very shocking. Most of the users who reviewed on Google Play were very dissatisfied with the dating apps they were using.