The Sentiment Analysis of Fintech Users Using Support Vector Machine and Particle Swarm
Optimization Method
1stWarjiyono, 2ndSopian Aji, 3rdFandhilah, 4thNunung Hidayatun, 5thHusni Faqih, 6thLiesnaningsih
1,3,4,5Study Program of Information System Bina Sarana Informatika University
Jakarta, Indonesia
2Study Program of Information System STMIK Nusa Mandiri
Jakarta, Indonesia
6Studi Program of Informatics Technical Muhammadiyah Tangerang University
Tangerang, Indonesia
1warjiyono.wrj@bsi.ac.id, 2sopian.sop@nusamandiri.ac.id, 3fandhilah.fnd@bsi.ac.id,
4nunung.ntn@bsi.ac.id, 5husni.hnf@bsi.ac.id, 6liesnaningsih@gmail.com
Abstract— This research was conducted to analyze the sentiment of Fintech users in Tegal City, especially the Ovo application since it currently has a very massive market and promotion. Users choosing the Fintech application generally consider convenience, security, transaction suitability, convenience, and cashback. The problem is that the user's trust in Fintech is still in doubt. Currently the application provides reviews to share their experiences. With the number of reviews displayed, it needs analysis that can classify the review into positive or negative classes. This research is using experimental method. Data is taken from Google Play in the OVO Application. The method used is the method of Support Vector Machine (SVM) and Particle Swarm Optimization (PSO). Data processing is using the Application RapidMiner Studio 7.6.003.
The result is that the Ovo application gets the best value with an accuracy of 82.33%. So that it can be concluded that the results of the review might convince users of other positive customer experiences while negative experiences can be used as a contribution to the thinking of the next product development to be better so they can compete.
Keywords—fintech, review, SVM, PSO I. INTRODUCTION
The development of smartphone technology today is extraordinary. Smartphones have become a primary need for every human being to fulfill their daily communication and information needs. Internet users in Indonesia in 2017 amounted to 143.26 million people out of 262 million Indonesians and 50.08% smartphone users as many as 131 million people [1]. Indonesia has a large capital to support the development of fintech, namely the number of middle- class people that reaches 45 million people, and the total internet users that reach 150 million [2]. For this reason, Indonesia is the fourth-largest smartphone user in the world after China, India, and America [3].
Millennials make a smartphone as a place to store various kinds of data such as bank data, credit card data and other payment data stored in a financial application. With just one click, the financial transaction will be smooth
without having to pay in cash. This is the choice of millennials since it is easy and fast.
Financial Technology (FinTech) is a financial technology, which provides financial services by utilizing software, smartphones and modern technology. FinTech is easier, cheaper and smarter in managing and sending money. In addition, FinTech reduces other costs, makes it easier to make payment transactions and helps small businesses to benefit and become a promising business model [4]. Factors that support the use of fintech are in the form of comfort, security, transaction suitability, convenience, while the inhibitors are the implementation of technology, costs, and infrastructure readiness [5].
Based on the observations made by researchers, FinTech which is commonly and intensively used by the Indonesian people to pay for purchases of goods and services is Gopay, Ovo. It is seen from the Play Store, Gopay has been downloaded more than 50 million with 2 million reviews, while Ovo has been downloaded more than 10 million with 252 thousand more reviews. Ovo and Gopay compete with each other. Ovo is being heavily promoted, a lot of collaboration with businesspeople and users can collect many points that can be used for transactions in other goods and services.
In Indonesia, the use of FinTech is still relatively low even though users and the ability to use smartphones have increased [6].
For those who have never used FinTech services, deficiencies and violations that occur in FinTech services can reduce the trust and willingness to use them so that they need to maximize benefits and minimize the potential danger [7].
The need for Fintech companies to make a marketing campaign to have an impact on increasing public awareness and trust to use FinTech [8].With the internet, someone will easily express their feelings and emotions through ratings and reviews. Since this enhances textual data, there is a need to analyze concepts of expressing sentiments and calculating insights for exploring business [9].
The 7th International Conference on Cyber and IT Service Management (CITSM 2019)
The progress of FinTech products, depending on innovative applications, can develop the business ecosystem and can collaborate throughout the business ecosystem, especially in Indonesia. If not, then Fintech is at risk of failure due to low adoption rates and very tight competition [10].
The evolution of FinTech for beginners is faced with challenges for regulators and market participants, especially in balancing the potential benefits of innovation with the possible risks of a new approach [11].
Sentiment analysis aims to group user reviews into positive or negative opinions by referring to feelings, emotions, opinions, and attitudes. The classification of sentiment analysis is divided into three levels, namely the document level, sentence level, and aspect level [12].
In this study, the sentiment analysis only takes the sentence level. The method used is the method of Support Vector Machine (SVM) and Particle Swarm Optimization (PSO).
Sentiment analysis is a process to determine whether sentences tend to be positive, negative or neutral [13].
Mining opinion does not pay attention to the topic of the text but focuses more on the expression depicted from the opinion text. This determines comments in online forums, blogs, or comments relating to a particular topic (products, books, films, etc.) including positive, negative or neutral opinions [14].
The results of this research will identify or classify whether Ovo is classified as good or not FinTech based on user reviews. Positive reviews can convince users to use while negative experiences can be used as contributions of ideas for the development of the next product to be better so they can compete.
II. METHODOLOGY
This research is using the experimental method. Data is taken from Google Play in the OVO Application. The collected data will be selected first to eliminate data that is not in accordance with the research. After that enter the data filtering stage, namely grouping data taken from user reviews to be separated into positive categories and negative categories.
Support Vector Machine (SVM) is a supervised learning method that analyzes data and recognizes patterns used for classification [15]. This research uses the Support Vector Machine (SVM) method based on Particle Swarm Optimization (PSO) to analyze the sentiment of Fintech users especially the OVO application by measuring the accuracy of experimental data using the RapidMiner Studio 7.6.003 application.
To conduct an experiment the researcher uses comment data or reviews from the Google Play OVO application.
Then obtained 300 data reviews consisting of 150 positive reviews and 150 negative reviews. Positive review data and negative review data with the same amount are stored in their respective files. Each review data is stored in a notepad application with extension.txt.
Then the data will be processed using the RapidMiner Application. In the RapidMiner application for text processing, the researcher uses Tokenize, Transform Cases, Stopwords (dictionary). Testing is done by selecting the feature selection of Support Vector Machine (SVM) based on Particle Swarm Optimization (PSO) aiming to obtain
accuracy value. From the accuracy value of the equation of the matrix confusion model the proportion of the number of predictive values is obtained and ROC Curve so that we can see the ROC chart with the value of AUC (Area Under Curve).
Fig. 1. Research Framework
Fig 1, explains that the framework in this study begins by taking a review of OVO application users from https://play.google.com/store/apps/details?id=ovo.id&show AllReviews=true, then processing text using tokenize, changing case, stopwords (directory). After that, the feature selection stage is done using Particle Swarm Optimization (PSO) and Support Vector Machine (SVM). Then the accuracy of the results is obtained from processing the data.
III. RESULT AND DISCUSSION
Before Research is carried out using the application of RapidMiner Studio 7.6.003 using the computer specifications of Intel Core i3 CPU 1.90 GHz, 10 GB RAM, and the Microsoft Windows 8.1 Professional 64-bit operating system.
The results of the study using Text Processing in table 1 that were traversed, consist of Tokenize, Transform Cases, Stopwords (dictionary). The processing results are as follows:
TABLE I. TEXT PROCESSING USING TOKENIZE,TRANSFORM CASES, STOPWORDS (DICTIONARY)
Text Review Tokenize Transform Cases Stopwords (Dictionary) Son of a bitch
application, the server is easy to go down, it is important funds for people, as a Grab Driver it is a trap, OVO CHEATS
Son of a bitch application, the server is easy to go down, it is important funds for people, as a Grab Driver it is a trap
Son of a bitch application, the server is easy to go down, it is important funds for people, as a Grab Driver it is a trap
Son of a bitch application, the server is easy to go down, it is important funds for people, as a Grab Driver it is a trap
In table 1, it appears that Text Processing from a negative text review will turn into a positive text review. In the text review through Tokenize, all punctuation marks will be removed like punctuation using dots, exclamation points and others, then the Transform Cases of all text reviews that use uppercase letters will change to lowercase letters and the last Stopwords (Dictionary), the negative vocabulary or
words will be eliminated like a bastard, stupid, shit and others.
The classification process for determining a sentence a member of the positive or negative class is based on calculating the probability of the SVM formula. if the positive class is greater than the negative class, then the sentence is included in the positive class. And if the positive class is smaller than the negative class, then the sentence is included in the negative class.
The first test of the research was carried out using the Support Vector Machine (SVM) algorithm to get accuracy which we can see as follows:
TABLE II. ACCURACY ALGORITHM SUPPORT VECTOR MACHINE
(SVM)
true Negatif true Positif
pred. Negatif 81 5
pred. Positive 69 145
In table 2, for negative review data of OVO application, 145 is classified into positive reviews in accordance with predictions made with the SVM method and 5 data are predicted to be positive reviews but it is found that the prediction result is negative reviews.
For the negative review data of OVO application, 81 are classified as negative reviews in accordance with predictions made with the SVM method and 69 data are predicted as negative reviews but it is found that the prediction result is positive reviews with accuracy using Support Vector Machine (SVM) with accuracy: 75.33% +/- 7.77% (micro: 75.33%).
The results of the study using the Support Vector Machine (SVM) algorithm in RapidMiner Studio are as follows:
Fig. 2. Modeling Validation Testing Support Vector Machine
The results of the ROC Curve in the Support Vector Machine (SVM) study can be seen as follows:
Fig. 3. Modeling Validation Testing Support Vector Machine (SVM) In Fig 3, the result of the ROC Curve Support Vector Machine (SVM) study in the ROC graph study has the AUC value of 0.929 +/- 0.042 (micro: 0.929) (positive class:
Positive).
Then the second test is done by using the Support Vector Machine-Based Particle Swarm Optimization (PSO) algorithm:
TABLE III. SUPPORT VECTOR MACHINE-BASED PARTICLE SWARM
OPTIMIZATION (PSO)
true Negatif true Positif
pred. Negatif 101 4
pred. Positive 49 146
In table III, for negative review data of OVO application, 146 is classified into positive reviews in accordance with predictions made with the SVM method and 4 data are predicted to be positive reviews but it is found that the prediction result is negative reviews.
For negative review data of OVO application, 101 is classified as negative reviews in accordance with predictions made with the SVM method and 49 data are predicted as negative reviews but it is found that the prediction result is positive reviews.
The result of this second test is Particle Swarm Optimization (PSO)-based Support Vector Machine (SVM) based on accuracy observation to be better with accuracy value: 82.33% +/- 7.00% (micro: 82.33%). Thus the result of the sentiment analysis review of Support Vector Machine can increase the value of accuracy after merging with Particle Swarm Optimization (PSO).
The results of the study using the Support Vector Machine-Based Particle Swarm Optimization (PSO) algorithm in RapidMiner Studio are as follows:
The 7th International Conference on Cyber and IT Service Management (CITSM 2019)
Fig. 4. Modeling Validation Testing Support Vector Machine-Based Particle Swarm Optimization (PSO)
The results from the ROC Curve on the research of Support Vector Machine-Based Particle Swarm Optimization (PSO) are as follows:
Fig. 5. ROC Curve Support Vector Machine-Based Particle Swarm Optimization (PSO)
In Fig 5, the result of the research is Supporting ROC Curve Vector Machine-Based Particle Swarm Optimization (PSO) in ROC graph research with AUC value: 0.925 +/- 0.031 (micro: 0.925) (positive class: Positive).
IV. CONCLUSION
Model testing by using Support Vector Machine-Based Particle Swarm Optimization (PSO) using sentiment analysis review data from 300 data reviews in Indonesian text consist of 150 positive reviews and 150 negative reviews. The resulting model gets the first test accuracy result done by using Support Vector Machine with the value of accuracy: 75.33% +/- 7.77% (micro: 75.33%) which is then tested again with the second test by combining Support Vector Machine-Based Particle Swarm Optimization ( PSO) with the best value with accuracy of 82.33% +/- 7.00%
(micro: 82.33%). Then it can be concluded that the testing of sentiment analysis using the Support Vector Machine will be even better by combining Support Vector Machine-Based Particle Swarm Optimization (PSO). So that the customers can make the right choice and convince users from other customer experiences through a review of the Product or Application. Then the Product or Application party is open to positive or negative opinions from customers and can provide empirical evidence for related theories so that it can be used as a contribution to thinking for the next product development to be better.
REFERENCES
[1] Anoname, “Penetrasi & Perilaku Pengguna Internet Indonesia,”
Jakarta, 2017.
[2] Y. Ananta, “Ada 150 Juta Pengguna Internet, OJK Dorong Peran Fintech,” CNBC Indonesia, 2019. [Online]. Available:
https://www.cnbcindonesia.com/fintech/20190310101921-37- 59704/ada-150-juta-pengguna-internet-ojk-dorong-peran-fintech.
[Accessed: 21-May-2019].
[3] I. Rahmayani, “Indonesia Raksasa Teknologi Digital Asia,”
Kementrian Komunikasi Dan Informatika Republik Indonesia, 2015.
[Online]. Available:
https://www.kominfo.go.id/content/detail/6095/indonesia-raksasa- teknologi-digital-asia/0/sorotan_media. [Accessed: 03-May-2019].
[4] A. Klein, “The Coming ‘FinTech’ Revolution,” Democracy Journal,
2016. [Online]. Available:
https://democracyjournal.org/magazine/42/the-coming-fintech- revolution/. [Accessed: 03-May-2019].
[5] D. Luckandi, “Analisis Transaksi Pembayaran Menggunakan Fintech Pada UMKM di Indonesia : Pendekatan Adaptive Structuration Theory,” Universitas Islam Indonesia, 2018.
[6] N. N. Khafiyah, “Pengaruh Persepsi Mahasiswa mengenai Uang Elektronik Terhadap Minat Menggunakan Aplikasi OVO,”
Universitas Islam Negeri Syarif Hidayatullah, 2019.
[7] R. Bates, “Banking on the Future : An Exploration of Fintech and The Consumer Interest,” 2017.
[8] S. Saksonova and I. Kuzmina-Merlino, “Fintech as Financial Innovation – The Possibilities and Problems of Implementation,”
Eur. Res. Stud. J., vol. XX, no. 3, pp. 961–973, 2017.
[9] A. Tripathy, A. Agrawal, and S. K. Rath, “Classification of Sentimental Reviews Using Machine Learning Techniques,”
Procedia - Procedia Comput. Sci., vol. 57, pp. 821–829, 2015.
[10] A. Teja, “Indonesian Fintech Business : New Innovations or Foster and Collaborate in Business Ecosystems ?,” Asian J. Technol.
Manag., vol. 10, no. 1, pp. 10–18, 2017.
[11] D. Arner, J. Barberis, and R. Buckley, “The Evolution of FinTech : A New Post-Crisis Paradigm?,” SSRN Electron. J., no. January 2016.
[12] K. Schouten, F. F.-I. T. on Knowledge, and U. 2016, “Survey on aspect-level sentiment analysis,” Ieeexplore.Ieee.Org, vol. 28, no. 3, pp. 813–830, 2016.
analysis of twitter posts,” Expert Syst. Appl., vol. 40, no. 10, pp.
4065–4074, 2013.
[14] J. Peñalver-Martínez, Isidro and Garcia-Sanchez, Francisco and Valencia-García, Rafael and Rodríguez-García, Miguel and Moreno, Valentín and Fraga, Anabel and Sánchez-Cervantes, “Feature-based
[15] A. Samad, H. Basari, B. Hussin, I. G. Pramudya, and J. Zeniarja,
“Opinion Mining of Movie Review using Hybrid Method of Support Vector Machine and Particle Swarm Optimization,” Procedia Eng., vol. 53, pp. 453–462, 2013.