Next is text processing, labeling based on the rating on the dataset. If the rating is >= 5 then it is labeled 1 which means it is positive, while the movie rating <= 5 is labeled 0 which means it is negative. The labeling is directly added in the dataset column. Here the labeling is classified as positive and negative and is named Positive Data and Negative Data.
Table 4.2.3.1 Positive Data
ID Title Review Rating Label TF-IDF
1 Not Bad I do not get the terrible
reviews 5 1 0,11895
2
What are all the bad reviews about
is it woke thing
It is great 7 1 0,15273
3 Cinderella The movie is nice 7 1 0,14733
6 A Descent Effort It is great enough 6 1 0,13085
7 Greenland I like it 7 1 0,14627
8 Mulan The movie is good 5 1 0,14973
9 A Quite Place II It is bad 7 1 0,15273
12 Wonder Woman The movie is great 6 1 0,0996
15 Combat It is nice 7 1 0,14733
17 Tenet Nice 5 1 0,1236
18 Cruella Like it 6 1 0,1495
62
ID Title Review Rating Label TF-IDF
20 Twilight Terrible movie 7 1 0,14835
Table 4.2.3.2 Negative Data
ID Title Review Rating Label TF-IDF
4 Bare-bones killer
shark film The movie is bad 4 0 0,14973
5 Terrible story, dialogue, and CGI
I do not like the
terrible movie 4 0 0,12464
10 Spiderman I do not like 4 0 0,1344
11 Superman The cgi is bad 4 0 0,1355
13 Ava It is good 4 0 0,14733
14 Great White=Jaws
Lite The CGI is not great 4 0 0,1396
16 Frozen Not good 4 0 0,1317
19 Fast & Furious It is terrible 4 0 0,15723
In text processing, the author column is omitted because it is not needed in this project.
Then in text processing, a data split is done. The data is divided into training data and testing data. Split data made 70% for training data from positive data and negative data and 30% for testing data from positive data and negative data. In split, data create 4 new arrays containing training data, training labels, testing data, testing labels. Training and testing data were selected randomly based on positive and negative data. Here's the training and testing:
63
Table 0.3 Training Data Positive
ID Title Review Rating TF-IDF
1 Not Bad I do not get the
terrible reviews 5 0,11895
2
What are all the bad reviews about is it woke
thing
It is great 7 0,15273
3 Cinderella The movie is nice 7 0,14733
6 A Descent Effort It is great enough 6 0,13085
7 Greenland I like it 7 0,14627
8 Mulan The movie is good 5 0,14973
9 A Quite Place II It is bad 7 0,15273
12 Wonder Woman The movie is great 6 0,0996
Table 0.4 Training Data Negative
ID Title Review Rating TF-IDF
4 Bare-bones killer shark
film The movie is bad 4 0,14973
5 Terrible story, dialogue, and CGI
I do not like the
terrible movie 4 0,12464
10 Spiderman I do not like 4 0,1344
11 Superman The cgi is bad 4 0,1355
13 Ava It is good 4 0,14733
64
Table 0.5 Training Label
ID Title Review Rating Label TF-IDF
1 Not Bad I do not get the
terrible reviews 5 1 0,11895
2
What are all the bad reviews about
is it woke thing
It is great 7 1 0,15273
3 Cinderella The movie is nice 7 1 0,14733
6 A Descent Effort It is great enough 6 1 0,13085
7 Greenland I like it 7 1 0,14627
8 Mulan The movie is good 5 1 0,14973
9 A Quite Place II It is bad 7 1 0,15273
12 Wonder Woman The movie is great 6 1 0,0996
4 Bare-bones killer
shark film The movie is bad 4 0 0,14973
5 Terrible story, dialogue, and CGI
I do not like the
terrible movie 4 0 0,12464
10 Spiderman I do not like 4 0 0,1344
11 Superman The cgi is bad 4 0 0,1355
13 Ava It is good 4 0 0,14733
From the positive training data and negative training data, they are combined to be given a label which becomes a training label which can be seen in Table 4.2.3.5.
65
Table 0.6 Testing Data Positive
ID Title Review Rating TF-IDF
15 Combat It is nice 7 0,14733
17 Tenet Nice 5 0,1236
18 Cruella Like it 6 0,1495
20 Twilight Terrible movie 7 0,14835
Table 0.7 Testing Data Negative
ID Title Review Rating TF-IDF
14 Great White=Jaws
Lite The CGI is not great 4 0,14733
16 Frozen Not good 4 0,1236
19 Fast & Furious It is terrible 4 0,1495
Table 0.8 Testing Label
ID Title Review Rating Label TF-IDF
15 Combat It is nice 7 1 0,14733
17 Tenet Nice 5 1 0,1236
18 Cruella Like it 6 1 0,1495
20 Twilight Terrible movie 7 1 0,14835
14 Great White=Jaws
Lite The CGI is not great 4 0 0,14733
16 Frozen Not good 4 0 0,1236
19 Fast & Furious It is terrible 4 0 0,1495
66
From testing positive data and testing negative data, they are combined to be given a label which becomes a testing label which can be seen in Table 4.2.3.8.
Furthermore, based on positive-negative training data and positive-negative data testing, the calculation of positive training counts, negative training counts, positive testing counts, and negative testing counts is carried out. The calculation grouped positive negative training and positive-negative testing itself. The positive training count counts the amount of data that is labeled positively on the training data, while the negative training count counts the amount of data that is labeled negatively on the training data. Positive count testing counts the number of positive data on testing data and negative count testing counts on the number of negative data on testing data. Here are the results of count training and count testing:
Count Training Positif = 8 Count Training Negatif = 5 Count Testing Positif = 4 Count Testing Negatif = 3 4.2.4. Feature Extraction
In feature extraction, Word Dict calculations are performed, namely grouping positive data and negative data. To count each the same number of words in positive reviews and negative reviews. Data is lower case and split by space. The first dictionary is created, namely, posWord which contains the number of occurrences of words in positive data, and the second negWord dictionary contains the number of occurrences of words in negative data. For more details, see the table below:
67
Table 4.2.4.1 Word Dict
Word The number of the same word appears in the data
Total Words on Positive Data (1)
Total Words on Negative Data (0)
i 4 2 2
do 3 1 2
not 4 1 3
get 1 1 -
the 8 4 4
terrible 4 2 2
reviews 1 1 -
movie 6 4 2
is 8 7 5
great 4 2 1
cgi 2 - 2
bad 3 1 2
like 4 2 2
enough 1 1 -
it 8 6 2
good 3 1 2
nice 3 2 -
In the table above, is the number of each word that appears in the positive data and negative data. The number of occurrences of the word is stored as a dictionary type. For example, [nice, 1] = 3, [good, 1] = 2, [nice, 0] = 2.
68
Furthermore, the calculation of feature extraction is carried out. Because computers do not deal with text, and only understand the language of numbers. So the author converts the data into vectors so that it can be entered into the logistic regression algorithm. The data is converted into a 3-dimensional (X) vector containing, rating, TF-IDF score, posWord dictionary (number of positive words in positive data for each id), and negWord dictionary (number of negative words in negative data for each id).
For example id 1: 'i do not get the terrible reviews'
i do not get the terrible reviews
Number of word in positive data 2 1 1 1 4 2 1
Number of word in negative data 2 2 3 0 4 2 0
Total occurrence of words in positive data = 13 Total occurrence of words in negative data = 13
The matrix has a rating, TF-IDF in it, the number of words in the positive data, the number of words in the negative data.
Matrix id 1 = [5 0.11895 13 13]
This calculation is repeated for each id based on the training data. Then the results of the X matrix from the training data:
X = [[5 0.11895 13 13], [7 0,15273 15 8], [7 0,14 733 17 11], [6 0,13085 16 8], [7 0,14627 10 6], [8 0,149733 16 13], [7 0,15273 14 9], [6 0,0996 17 12], [4 0,14973 16 13], [4 0,12464 16 17], [4 0,1344 6 9], [4 0,1355 12 13], [4 0,14733 14 9]]
69
4.2.5. Implementation of Logistic Regression Algorithm
The next step according to the design is the implementation of the Logistic Regression algorithm. The intercept, xData, yData, and weight calculations are used for the next step. Then the calculations for intercept, xData, yData, and weight are as follows:
a. The intercept is initialized to 1 throughout the training data, the length of the training data is 13 data. So it is made 13 to the right.
Intercept = [1 1 1 1 1 1 1 1 1 1 1 1 1]
b. xData combines intercept vertically and X.
xData = [[1 5 0.11895 13 13]
[1 7 0,15273 15 8]
[1 7 0,14733 17 11]
[1 6 0,13085 16 8]
[1 7 0,14627 10 6]
[1 8 0,14973 16 13]
[1 7 0,15273 14 9]
[1 6 0,0996 17 12]
[1 4 0,14973 16 13]
[1 4 0,12464 16 17]
[1 4 0,1344 6 9]
[1 4 0,1355 12 13]
[1 4 0,14733 14 9]]
c. Weight is set to 0 (as many as 5) in length according to the number of contents of the matrix in 1 row and will be updated later.
Weight = [0 0 0 0 0]
70 d. yData is obtained from training labels.
yData = [1 1 1 1 1 1 1 1 0 0 0 0 0]
In the next step, the author creates a function for the training model that requires parameters LR (Learning Rate), iteration, intercept, xData, weight, and yData. This function is used to calculate sigmoid, loss, and gradient descent. Later it will also update the weight to fit into the data. The new weight is used to predict the label. This function is used for each iteration, here is the implementation of 1 iteration:
1st iteration:
First calculate the Sigmoid:
The sigmoid function returns a value from 0 to 1. If the output is greater than or equal to 0.5, it is classified as positive data. Otherwise, it is classified as negative data. Sigmoid depiction can be seen in Figure 4.2.5.1.
Figure 0.1 Curve of Sigmoid Function
In the graph above, it will return a value between 0 to 1. If the value of x (z) goes to positive infinity then the predicted value of y will be 1 and if it goes to negative infinity then the predicted value of y becomes 0.
Find the z-value first:
𝑧 = 𝑥 × 𝑤
Function 4.2.5: Z
71 X = xData
W = weight Then the z value:
Z = [0 0 0 0 0 0 0 0 0 0 0 0 0]
After getting the z value, implement it to the sigmoid formula:
𝑠𝑖𝑔𝑚𝑜𝑖𝑑 = 1 1 + 𝑒−𝑧
Function 4.2.5.1: Sigmoid
Sigmoid calculation:
𝑠𝑖𝑔𝑚𝑜𝑖𝑑 = 1
1 + 𝑒−(0) = 0.5
Sigmoid = [0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0. 5 0.5]
The next step is to calculate the loss value, find the optimal parameters, and minimize errors from the model to get the best predictions. The function for count loss can be seen in Function 4.2.5.2.
Loss :
𝑙𝑜𝑠𝑠 = [((−𝑦 log ℎ)) − ((1 − 𝑦) log 1 − ℎ))]
Function 4.2.5.2: loss
h = sigmoid y = yData
Calculate the log h along the sigmoid, then the result is:
log h = log0.5
= [-0.69314718 -0.69314718 -0.69314718 -0.69314718 -0.69314718 -0.69314718 - 0.69314718 -0.69314718 -0.69314718 -0.69314718 -0.69314718 -0.69314718 - 0.69314718]
72 After getting log h, implementation to loss formula:
Loss = ([-1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 0] * [-0.69314718 - 0.69314718 -0.69314718 -0.69314718 -0.69314718 -0.69314718 -0.69314718 - 0.69314718 -0.69314718 -0.69314718 -0.69314718 -0.69314718 -0.69314718]) – ([1-1 1-1 1-1 1-1 1-1 1-1 1-1 1-1 1-0 1-0 1-0 1- 0 1-0] * [1-(-0.69314718) 1-( -0.69314718) 1-(-0.69314718) 1-(-0.69314718) 1-( -0.69314718) 1-( - 0.69314718) 1-(-0.69314718) 1-(-0.69314718) 1-(-0.69314718) 1-(-0.69314718) 1-(-0.69314718) 1-(-0.69314718) 1-(-0.69314718)])
= ([0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718]) – ([0 0 0 0 0 0 0 0 1 1 1 1 1] * [1.69314718 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718])
= ([0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718]) – ([0 0 0 0 0 0 0 0 1.69314718 1.69314718 1.69314718 1.69314718 1.69314718])
= [0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 0.69314718 -1 -1 -1 -1 -1]
Then look for gradient descent which updates the weights by minimizing loss. The gradient descent formula can be seen in Function 4.2.4, the gradient descent formula is obtained from the loss derivative.
Gradient Descent :
𝑔𝑟𝑎𝑑𝑖𝑒𝑛𝑡 𝑑𝑒𝑠𝑐𝑒𝑛𝑡 =(ℎ − 𝑦). 𝑋𝑇 𝑙𝑒𝑛(𝑦)
Function 4.2.5.3: Gradient Descent
73 h = sigmoid
y = yData
len(y) = lenght of yData 𝑋𝑇 = xData transpose
Calculate the value of (h-y) first:
h-y = ([0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5]) – ([1 1 1 1 1 1 1 1 0 0 0 0 0])
= [-0. 5 -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 0.5 0.5 0.5 0.5 0.5]
Then transpose matrix xData:
𝑋𝑇 = [[1 1 1 1 1 1 1 1 1 1 1 1 1]
[5 7 7 6 7 8 7 6 4 4 4 4 4]
[0.11895 0,15273 0,14733 0,13085 0,14627 0,14973 0,15273 0,0996 0,14973 0,12464 0,1344 0,1355 0,14733]
[13 15 17 16 10 16 14 17 16 16 6 12 14]
After that, just implement the gradient descent formula which can be seen in Function 4.2.5.3.
Gradient Descent = [−1.2 −16.5 −0.203295 −27 −9]
13
= [-0.9231 -1.2692 -0.0156 -2.0769 -0.6923]
Then gradient descent is obtained, the weight update is performed. The new weight is used to predict the label. This update weight is also calculated for each iteration.
Update Weight :
𝑛𝑊𝑒𝑖𝑔ℎ𝑡 = 𝑜𝑊𝑒𝑖𝑔ℎ𝑡 − 𝑙𝑟 . 𝑔𝑟𝑎𝑑𝑖𝑒𝑛𝑡 𝑑𝑒𝑠𝑐𝑒𝑛𝑡
Function 4.2.5.4: Update Weight
nWeight = new weight oWeight = old weight lr = learning rate
74 So the update weight calculation:
oWeight = [0 0 0 0 0]
Lr = 0.1
Gradient Descent = [-0.9231 -1.2692 -0.0156 -2.0769 -0.6923]
Lr * Gradient Descent = [-0.09231 -0.12692 -0.00156 -0.20769 -0.06923]
Update Weight = ([0 0 0 0 0]) – ([-0.09231 -0.12692 -0.00156 -0.20769 - 0.06923])
= [0.09231 0.12692 0.00156 0.20769 0.06923]
The next step is to predict the class label. Calculate the new sigmoid first using the new weight. The sigmoid function can be seen in Function 4.2.5.1.
New Sigmoid:
Z = xData * nWeight X = xData
75 xData = [[1 5 0.11895 13 13]
[1 7 0,15273 15 8]
[1 7 0,14733 17 11]
[1 6 0,13085 16 8]
[1 7 0,14627 10 6]
[1 8 0,14973 16 13]
[1 7 0,15273 14 9]
[1 6 0,0996 17 12]
[1 4 0,14973 16 13]
[1 4 0,12464 16 17]
[1 4 0,1344 6 9]
[1 4 0,1355 12 13]
[1 4 0,14733 14 9]]
New Weight = [0.09231 0.12692 0.00156 0.20769 0.06923]
Z
=
[[1 5 0.11895 13 13]
[1 7 0,15273 15 8]
[1 7 0,14733 17 11]
[1 6 0,13085 16 8]
[1 7 0,14627 10 6]
[1 8 0,14973 16 13]
[1 7 0,15273 14 9]
[1 6 0,0996 17 12]
[1 4 0,14973 16 13]
[1 4 0,12464 16 17]
* [0.09231 0.12692 0.00156 0.20769 0.06923]
76 [1 4 0,1344 6 9]
[1 4 0,1355 12 13]
[1 4 0,14733 14 9]]
Z = [[4.32706 4.65018 5.27324 4.73092 3.47326 5.33094 4.51172 5.21548 4.82326 5.10014 2.46941 3.99248 4.13095]]
The sigmoid calculation for the first value of z:
𝑠 = 1
1 + 𝑒−(4.32706) = 0.98696582
Then the result of the sigmoid (result before):
Result Before = [[0.98696582 0.99053064 0.99489918 0.99125873 0.99125873 0.96991728 0.99518379 0.98913968 0.99459752 0.9920236 0.99394104 0.92196933 0.98188048 0.98418648]]
After that, find the result after from the result before with a threshold. If result before >=
threshold then True, if result before <= threshold then False.
Threshold = 0.5 (dibuat 0.5)
Result After = [[True True True True True True True True True True True True True True]]
The next step is to make a label prediction, the value is made 0 first as long as the result before.
Label Prediction = [[0 0 0 0 0 0 0 0 0 0 0 0 0 0]]
Then compare the Result After with the predicted label. If the result after 'True', then it is labeled 1, if 'False' then it is labeled 0.
So the final result of the label prediction:
Prediction Label = [[1 1 1 1 1 1 1 1 1 1 1 1 1 1]]