But for the case of gesture recognition with kinect sensor, (Maddox, 2011) method of dynamic time warping mengagaskan modified with step function (slope constraint).
Modification aims to increase the sensitivity of pattern recognition. Here is a dynamic time warping pembahasaan method that has been modified by (Maddox, 2011):
In the first series of frames in the input and reference patterns of movement going in reverse. It aims to detect the difference in distance in advance of the final conditions of movement patterns, thereby more quickly identify the pattern of movements performed by the user. The value of A and B after the reverse can be seen in the following equation below:
A = [ ]
B = [ ]
After doing the steps above, made three DTW matrix is a matrix, the matrix slopeN, and matrix slopeM. DTW matrix is used as a weighting matrix of cost accumulation, while the matrix slopeN and slopeM used to identify the pathways that have been skipped.
The initial value of all elements on the DTW matrix is positive infinity, except for elements on the DTW matrix (0.0) is 0. While the initial value of all matrix elements and slopeM slopeN is 0.
slope-N
slope-M
After determining the initial matrix, the next step is to calculate a new value for DTW matrix with the following algorithm:
Algoritma perhitungan nilai matriks DTW dengan slope constrain 1: // n filled with a number of the set input
2: // m filled with a number of the set input
3: dtw [] // making DTW matrix with dimensions 4: slopeN [] // making SlopeN matrix with dimensions 5: slopeM [] // making SlopeM matrix with dimensions
6: for do
11: dtw // Create a Boundary condition as in equation (2)
11: for do
12: for j do
13: if dtw(i,j−1) < dtw(i−1,j−1) & dtw(i,j−1) < dtw(i−1,j) &
slopeN(i,j−1)<maxslope then
// / / line conditions (i, j-1) has the lowest cost of other pathways and pathways have not been (i, j-1) can still be passed.
14 : (i, j) c(i – 1, j –1) + (i, j – 1) // cost path through the (i, j-1) 15: slopeN(i, j) slopeN(i, j – 1) + 1 // marker lane (i, j-1) has passed 16: slopeM(i, j) ) 0 // marker lane (i, j-1) in the reset
17: elseif dtw(i−1,j) < dtw(i−1,j−1) & dtw(i−1,j) < dtw(i,j−1) &
slopeN(i,j−1)<maxslope then
// line conditions (i-1, j) has the lowest cost from the other lane and the path has not been (i-1, j) is still passable.
18: (i, j) c(i – 1, j–1) + (i-1, j) // cost path through the (i-1, j)
28: return dtw // return DTW matrix accumulation
After performing the calculation of DTW matrix, to find the cost of the optimal warping path is used the following equation:
Here is the search algorithm of the optimal warping Algoritma perhitungan DTW
1:
2: // n filled with a number of the set input 3: // m filled with a number of the set reference
4: for do
5: if ( (i, m) < ) then
6: = [i, m] // looking for cost optimal warping 7: endif
8: end for
9: / m // calculate the average 10: return cost
cost obtained will be compared with the value of tolerance (threshold) that have been determined. If the cost is still within the tolerance limit, then the movement will be done with the data on reference.
Here is a comparison between the use of dynamic time warping method that does not use the slope and dynamic time warping constraint that uses the slope
Input = { 1, 3, 7, 4}
Reference = { 3, 1, 3, 4, 10, 11, 4}
(p1)
(p2)
(p3)
Optimal path warping DTW = = 9 (p4)
Optimal path warping DTW with slope =
= 12 (p5) In the calculation of (p1), calculated using the cost matrix equation (1). The next matrix will be calculated accumulated cost matrix cost is to get the optimal warping path. In the calculation of (p2), the search for the optimal warping path is always looking for the lowest value of the range of motion without having horizontal and vertical. While in the calculation (p3) percarian warpting optimal path is always looking for the lowest score by
the movement of the horizontal and vertical limits. Can be seen in the calculation (p4) and (p5) that the value of the calculation (p4) will be smaller than the calculation (p5). This is because by using the slope limits, greater sensitivity of the matching movement.
When the pattern of movement the user is not recognized, then the accuracy will be 0%.
Meanwhile, if the motion is identified, it will show the accuracy of user movement pattern similarity with the reference motion pattern with the following equation:
Evaluation
On program evaluation, testing the user with 3 times 5 trials for each movement pattern 6. The data of test results can be seen in table 1 below:
Table 1 Table 5 tests on user
Noo Nama Pola gerakan Akurasi
Percobaan 1 Percobaan 2 Percobaan 3
1 Kristian Aji Nugroho Saya 70.1% 70.3% 70.1%
Bisa 72.3% 71.1% 75.2%
Mendengar 72.7% 71.1% 73.4%
Melihat 70.5% 70.3% 71.6%
Dengan 0.0% 71.2% 70.5%
Membaca 71.4% 0.0% 72.3%
2 Tarenjit Saya 72.2% 70.3% 72.7%
Bisa 71.1% 72.3% 70.1%
Mendengar 70.2% 70.4% 71.2%
Melihat 72.1% 70.1% 71.2%
Dengan 70.2% 73.2% 0.0%
Membaca 0.0% 0.0% 70.2%
3 Maha Andar Pasaribu Saya 71.2% 70.3% 72.5%
Bisa 70.3% 72.4% 73.1%
Mendengar 72.2% 72.5% 75.2%
Melihat 71.2% 72.2% 71.2%
Dengan 70.2% 0.0% 71.0%
Membaca 0.0% 0.0% 71.3%
4 Andi Putra Saya 73.3% 70.3% 72.7%
Bisa 71.4% 71.4% 74.2%
Mendengar 73.5% 71.5% 74.1%
Melihat 72.2% 74.1% 74.1%
Dengan 70.2% 70.3% 0.0%
Membaca 71.2% 0.0% 72.2%
5 Leonardo Wibowo Saya 71.2% 72.2% 72.5%
Bisa 71.2% 72.2% 70.7%
Mendengar 71.2% 70.0% 70.7%
Melihat 71.2% 70.3% 70.6%
Dengan 71.2% 0.0% 0.0%
Membaca 0.0% 0.0% 72.3%
Based on the results of testing the program in Table 4.1, the percentage of each pattern recognition can be calculated by the following equation:
The test program conducted on 5 user as much as 3 times for each movement pattern.
Therefore, experiments for a total of 15 kali.Sehingga percentage movement is the movement pattern recognition can be seen in Table 2.
Table 2 Table percentage movement pattern recognition
No Pola gerakan Presentase pengenalan
1 Saya 100%
2 Bisa 100%
3 Mendengar 100%
4 Melihat 100%
5 Dengan 60%
6 Membaca 46.6%
From Table 4.1 it can be seen in 30 attempts, there occurred 22 times value of accuracy. This indicated that the level of learning (learning curve) increases with the number of user trials. But there are unisex decrease in accuracy rate of learning sign language. This can be caused by several factors such as:
1. Hardware factors: inability to identify kinect sensor digital frame in the event of crossing the frame body (eg arms crossed).
2. Factor movement patterns: Patterns of complicated movements, so the user is difficult to follow the pattern of motion of the reference.
3. User factors: The ability of users to absorb the information provided by the program.
To find out whether the program can help the process of learning sign language, the survey conducted terhadapat 5 user. Here are the results of users' opinions about whether the program helps in learning sign language
Table 3 Table of test results if the program helps the learning
No Nama Membantu pembelajaran bahasa isyarat
1 Kristian Aji Nugroho Ya
2 Tarenjit Ya
3 M Maha Andar Pasaribu Ya
4 Andi Putra Ya
5 Leonardo Wibowo Ya
In Table 3 we can see that, users find it helpful to learning sign language program was good in knowing about the meaning of the sign language and practice.
On testing for 5 users, the accuracy of the movement pattern similarity with the reference motion is still below 80%, this is Because the data is retrieved when the user does not by know about the condition of movement patterns will be studied, so this is something new to do user. Therefore, the accuracy level of similarity was below 80%, but the user as do the patterns of movement, then the accuracy of the similarity of movement patterns will be even dengna Greater reference.
PENUTUP
After doing the analysis and design of sign language learning application program, it can be concluded that:
1. The program can identify and can provide learning about sign language to the user.
2. A combination of hardware and kinect sensor dynamic time warping method is able to identify patterns of movement of the human body based framework with a different time series.
3. This program can provide guidance in the learning process (learning curve).
Some suggestions can be submitted to the other researchers are:
Need to do more research on the learning curve using the program.
1. Further development is recommended to use a database to store reference data, so that the stored reference motion pattern is more structured.
2. Further development is recommended to create a program that can be used by multiple users.
3. Further development is recommended to assemble the skeleton movement pattern recognition is a member of another body.
4. Further development is recommended to perform the training process to get the movement pattern recognition with an accuracy that is more accurate.