Testing the Population Administration Website Application Using the Black Box Testing
Boundary Value Analysis Method
Abstract—The population administration website application is a computer platform that makes it easy for the government to integrate population data collected by the Department of Population and Civil Registration with the Central Statistics Agency. This application was created to maximize the validity, accuracy, and quality of data. Each module in the application was subject to testing to ensure that it was running according to the expected functionality. This aimed to see the level of errors that occurred in the software. In this study, application testing was carried out with Black Box Testing using the Boundary Value Analysis (BVA) technique, which was to determine the lower and upper limit values based on the data being tested. The BVA test results showed that the application was able to handle data, both normal and abnormal, with a success percentage of 90.9%.
Keywords—Black Box Testing, BVA, Website Application, Population Administration.
I. INTRODUCTION
The population administration website application is a computer platform that makes it easy for the government to integrate population data collected by the Department of Population and Civil Registration with the Central Statistics Agency. This application was created to maximize the validity, accuracy, and quality of data. This application has been running in one of the cities in Indonesia, namely the city of Tegal, wherein this application the population data collection process is carried out in stages in a bottom-up manner, namely: the head of the RT (Rukun Tetangga) carries out population data input with variables that have been determined by the government, then validated by the head of RW (Rukun Warga), Kelurahan, and Kecamatan.
Furthermore, validation and verification are carried out by the Department of Population and Civil Registration [1]. Data that has been validated and verified by the Department of Population and Civil Registration can be directly retrieved and utilized by the Central Bureau of Statistics as an integrated data set with high validity and accuracy. Data integration is a way to minimize data redundancy and duplication [2].
The process of developing a population administration website application has been successfully carried out using the Scrum framework, with stages: sprint planning meetings, sprint backlogs, sprint retrospectives, sprint reviews, and daily meetings conducted by the team members to complete the project [3]. The team members in question are the product
owner, the Development Team, and the Scrum Master [4].
During the application work period and when it entered the sprint retrospective stage, the application was tested to find errors/bugs that occurred, so that when it was at the sprint review stage or up to the implementation stage, errors would not be found again. [5] This test was conducted to maintain the quality of built application software to last and make production costs effective so that the application would not be thrown away due to production failure factors [6].
The population administration website application is able to load six users, namely RT, RW, Kelurahan, Kecamatan, Disdukcapil, and BPS. Each user has a module, which is as described in table I below:
TABLE I. APPLICATION MODULE
No Service Module
1 RT NIK Resident
Family Member Resident Area
2 RW RT
3 Kelurahan RW 4 Kecamatan Kelurahan 5 Disdukcapil Kecamatan
Camat Headman Productive Age Non-Productive Age Head of Disdukcapil
6 BPS Head of BPS
Ideally, each module in the application is subject to testing to ensure that it runs according to the expected functionality. It aims to see the level of errors that occur in the software [7]. Moreover, this website application is complex and will be more integrated into life, so testing is the primary approach to find and minimize any risk[8].
In this study, application testing was carried out by using Black Box Testing to check the functionality of the software application, to observe the fundamental aspects of the software, and to check its suitability with user needs. [9] The technique applied in Black Box Testing in this study was BVA to determine the lower and upper limit values based on the data being tested [10]. With principles that: (1) The most common mistakes were during the input process; (2) BVA worked on the input process.[11]
M. Nishom
Department of Informatics Engineering Politeknik Harapan Bersama
Tegal, Indonesia
https://orcid.org/0000-0003-0765-1044 Ginanjar Wiro Sasmito
Department of Informatics Engineering Politeknik Harapan Bersama
Tegal, Indonesia
https://orcid.org/0000-0002-3941-1769
II. RELATED STUDY
Black Box Testing has been used in the development of the concept of VR-based virtual testing technology, by placing a focus on the systematic study of VR-based virtual testing technology based on a macro perspective, thus establishing the basic concepts and theoretical models and implementation methods used. In this study, Black Box testing is focused on the external structure of the program, without considering the internal logic structure, which can test according to the software interface and software functions [12].
In a study conducted by Yulianton et al. 2020, Black Box Testing was used to detect vulnerabilities in web applications by combining them with Dynamic Analysis and Static Analysis. It is believed that this framework can give better results than if each method is used separately. It is because the strengths of each method are used to overcome the weaknesses of the other methods [13].
Black Box Testing is also used for testing the TLS (Transport Layer Security) protocol, which is one of the most widely used security protocols on the internet. In practice, the TLS protocol continued to experience bugs and security vulnerabilities. It was due to the complexity of the protocol, which made implementing TLS difficult. Therefore, differential testing was implemented on the Black Box for testing the implementation of the handshake protocol on TLS [14].
BVA was applied to test the Augmented Reality function regarding piece recognition in Indonesia with the Cloud method on android mobile devices. The results of testing the distance between the marker object and the Android mobile device in the cloud using the camera showed that the higher the augmentable value of the target image and the more the number of detected marker features, the easier the image will be traced by AR. If the distance between the camera and the real object gets further away, the virtual object cannot be displayed[15].
In an article written by Feng, 2010, it was introduced that the tree method function is used to design test cases on input parameters with constraints. With three input parameters X, Y, and Z, it is generally assumed that Y is a function of X, then Z is a function of X and Y. Based on the geometric theory, that approach is a generalization of BVA in Black Box Testing.[16]
III. METHODOLOGY
Testing of the population administration website application is conducted by the Black Box Testing method using the BVA technique. The BVA technique is used to determine the lower and upper limit values based on the data to be tested. Testing the lower and upper limit values is conducted through several predetermined stages for each field in a software application.[17].
Fig. 1. Technical Design of BVA
Black Box Testing with the BVA technique is as follows [11]:
1. If the input conditions are in the range of x and y values, then a test case should be created with sample data x-1, x, y, y + 1
2. If the input conditions use a number of values, then the test case should be made the sample data is minimum -1, minimum, maximum, maximum + 1 3. Perform steps 1 and 2 for the output process 4. If the data already has an input limit (for example,
the array is set to a maximum of 10), then test cases are created on that limit.
In testing this application, there are several steps made, such as problem identification, test data selection, input test data into the system. The testing process uses BVA, performs test data calculations, and documentation of test results. The research flow is in accordance with the following figure 2:
Fig. 2. Research Flow
IV. RESULT AND DISCUSSION
The Black Box testing method is conducted by inputting data in each existing field, either by inputting valid data or free data which does not match the valid data. The Black Box testing method is applied to the population administration website application using the BVA technique. This application has six users and several modules. However, as the case in this study, the results of testing one of the functionality/modules will be discussed, namely:
“Kelurahan”.
The “Kelurahan” function consists of one data entry panel, as shown in Figure 3. In this form, there are eleven data entry fields, namely: Name of Province, Name of City/District, Kecamatan, Kelurahan, NIK, Name, Username, Email, Password, Repeat Password, Address.
START
PROBLEM IDENTIFICATION
TEST DATA SELECTION
TEST DATA BOUNDARY
VALUE ANALYSIS
TEST
TEST RESULT CALCULATION
DOCUMENTATION
FINISHED
Fig. 3. Kelurahan Form
Fig. 4. Figure 4: Kelurahan Table Structure
Based on the Kelurahan form in Figure 3, the eleven fields on the form were tested. The sample data rules used were normal data, minimum data, maximum data, and maximum data +1.
TABLE II. TEST OF PROVINCE NAME FIELD
Sample Data Result Estimate
Result Conclusion Middle Java TRUE TRUE Success
FALSE FALSE Success East Java FALSE FALSE Success
District of Ogan Komering Ulu
Selatan
FALSE FALSE Success
TABLE III. TEST OF CITY/DISTRICT NAME FIELD
Sample Data Result Estimate
Result Conclusion Tegal True True Success
False False Success District Of
Penukal Abab Lematang Ilir
False False Success
Slawi True True Success
TABLE IV. TEST OF KECAMATAN FIELD
Sample Data
Result Estimate
Result Conclusion East Tegal TRUE TRUE Success
FALSE FALSE Success Margadana TRUE TRUE Success Silo Karno
Doga FALSE TRUE Failed The test results of the Province Name field in Table II produced four successful sample data that can be handled by the Province Name. The success rate for the Province Name field is 100%. Table III provides four successful sample data which can be handled by the City/District Name field. The success rate for the City/District Name field is 100%. In Table IV, which also produces sample data, the success rate for the Kecamatan field is 75%.
TABLE V. TEST OF KELURAHAN FIELD
Sample
Data Result
Estimate Result Conclusion Debong TRUE TRUE Success
FALSE FALSE Success Pegirikan Lor FALSE FALSE Success
Tegalsari TRUE TRUE Success TABLE VI. TEST OF NIKFIELD
Sample Data
Result Estimate
Result Conclusion 19750541130
28601
TRUE TRUE Success FALSE FALSE Success 19908680016
597844
FALSE FALSE Success 19008711876
40
TRUE FALSE Failed
TABLE VII. TEST OF NAME FIELD
Sample Data
Result Estimate
Result Conclusion Budi TRUE TRUE Success
FALSE FALSE Success Johan123 FALSE FALSE Success
Nurlaeli TRUE TRUE Success Table V provides four successful sample data which can be handled by the Kelurahan field. The success rate for the Kelurahan field is 100%. Table VI provides successful sample data that can be handled by the NIK field. The success rate for the NIK field is 75%. Table VII provides four successful sample data that can be handled by the Name field with the 100% success rate for the Name field.
TABLE VIII. TEST OF USERNAME FIELD
Sample
Data Result
Estimate Result Conclusion Gunawan TRUE TRUE Success
FALSE FALSE Success Tri Ardi TRUE TRUE Success
Gendra Alim Trio Prasojo
Bangkara
FALSE FALSE Success
TABLE IX. TEST OF EMAIL FIELD
Sample Data Result
Estimate Result Conclusion dega@gmail.co
m TRUE TRUE Success FALSE FALSE Success Slamet@ymailc
om TRUE FALSE Failed mohammadfikri
hidayattullah@g mail.com
FALSE FALSE Success
TABLE X. TEST OF PASSWORD FIELD
Sample Data Result Estimate
Result Conclusion jangantanya TRUE TRUE Success
FALSE FALSE Success bukansayasajaya
ngtahu FALSE FALSE Success kosongan00 FALSE TRUE Failed Table VIII provides four successful sample data which can be handled by the Username field. The success rate for the Username field is 100%. Table IX provides successful sample data that can be handled by the Email field. The success rate for the Email field is 75%. Table X provides successful sample data that can be handled by the Password field with the 75 % success rate for the Password field.
TABLE XI. TEST OF REPEAT PASSWORD FIELD
Sample Data Result
Estimate Result Conclusion Sayatahu TRUE TRUE Success
FALSE FALSE Success jangantanyasaya
lagiya FALSE FALSE Success 0samakosong TRUE TRUE Success
TABLE XII. TEST OF ADDRESS FIELD
Sample Data Result
Estimate Result Conclusion Jln.Mataram TRUE TRUE Success
FALSE FALSE Success Kluwuttimur TRUE TRUE Success
*apa??ituya##$$
@aku FALSE FALSE Success Table XI provides four successful sample data which can be handled by the Repeat Password field. The success rate for the Repeat Password field is 100%. Table XII provides four successful sample data that can be handled by the Address field with the 100% success rate for the Address field.
Based on the BVA test for each field in the Kelurahan form that has been conducted according to those listed in Table II to Table XII, the recapitulation of the average BVA test results can be shown in table XIII below:
TABLE XIII. RECAPITULATION OF AVERAGE BVATEST RESULTS
No Field Rate of Success (%) 1 Province Name 100 2 City/District Name 100
3 Kecamatan 75
4 Kelurahan 100
5 NIK 75
6 Name 100
7 Username 100
8 Email 75
9 Password 75
10 Repeat Password 100
11 Address 100
Average 90.9 Based on the recapitulation of the BVA test results according
to the data in Table 13, it can be seen that there is a margin of error in the test results of 9.1% out of the total success value of 90.9%.
The graph of recapitulation of the BVA test results for each table is shown in Figure 5. Whilst comparing the success of the BVA test for each table can be shown according to Figure 6.
Fig. 5. Graph of Recapitulation of BVA Test Results of Each Table
Fig. 6. Comparison of the Success of BVA Test Results
V. CONCLUSION
The BVA test results showed that the application was able to handle data, both normal and abnormal, with a success percentage of 90.9%. Four fields need to be improved to improve application performance in processing data in normal and abnormal conditions.
REFERENCES
[1] G. W. Sasmito and M. Nishom, “Development of Web-Based Application in Population Administration System Using Scrum
100%100%
75%
100%
75%
100%100%
75% 75%
100%100%
0%
20%
40%
60%
80%
100%
120%
Recapitulation of BVA Test Results
100%, 7 75%, 4
50%, 0 25%, 00%, 0
Comparison of Success of BVA Test Results
100% 75% 50% 25% 0%
Framework,” Int. J. Web Appl., vol. 11, no. 4, p. 125, 2019, doi:
10.6025/ijwa/2019/11/4/125-135.
[2] T. W. Kang and C. H. Hong, “A study on software architecture for effective BIM/GIS-based facility management data integration,”
Autom. Constr., vol. 54, pp. 25–38, 2015, doi:
10.1016/j.autcon.2015.03.019.
[3] K. Schwaber and J. Sutherland, “The Scrum Guide: The Definitive The Rules of the Game,” Scrum.Org and ScrumInc, no. November, p. 19, 2017, doi: 10.1053/j.jrn.2009.08.012.
[4] G. W. Sasmito and A. Fauzan, “The implementation of scrum framework for developing fundraising mobile applications,” Int. J.
Adv. Sci. Technol., vol. 29, no. 1, 2020.
[5] A. Lawanna, “The Theory of Software Testing,” no. June 2012, 2014.
[6] O. Dahiya, K. Solanki, and A. Dhankhar, “RISK-BASED TESTING : IDENTIFYING , ASSESSING , MITIGATING &
MANAGING,” vol. 11, no. 3, pp. 192–203, 2020.
[7] M. Krichen and S. Tripakis, “Conformance testing for real-time systems,” Form. Methods Syst. Des., vol. 34, no. 3, pp. 238–304, 2009, doi: 10.1007/s10703-009-0065-1.
[8] R. J. Van Emous, “Towards Systematic Black-Box Testing for Exploitable Race Conditions in Web Apps,” 2019.
[9] Navita and Meenu, “Study and Analysis of Software Testing,” no.
December, pp. 6674–6678, 2015.
[10] B. Neate, “Boundary Value Analysis,” vol. http://www, pp. 1–15, 2006.
[11] S. et al. Xu, “A comparative study on black-box testing with open source applications,” in 17th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), 2016, doi:
10.1109/SNPD.2016.7515953.
[12] S. Tang, “Research on Black Box Testing Technology Based on Virtual Reality,” vol. 81, no. Iccse, pp. 99–103, 2017, doi:
10.2991/iccse-17.2017.17.
[13] H. Yulianton, A. Trisetyarso, W. Suparta, B. S. Abbas, and C. H.
Kang, “Web Application Vulnerability Detection Using Taint Analysis and Black-box Testing,” IOP Conf. Ser. Mater. Sci. Eng., vol. 879, p. 012031, 2020, doi: 10.1088/1757-899x/879/1/012031.
[14] A. Walz, Andreas and Sikora, “Exploiting Dissent: Towards Fuzzing-Based Differential Black-Box Testing of TLS Implementations,” IEEE Trans. Dependable Secur. Comput., vol.
17, no. 2, pp. 278–291, 2020, doi: 10.1109/TDSC.2017.2763947.
[15] R. Dewi Agushinta, Hustinawaty, I. Jatnika, and H. Medyawati,
“Boundary value analysis testing on augmented reality of Indonesian fruit recognition at Mekarsari tourist park using cloud method on android mobile devices,” J. Phys. Conf. Ser., vol. 1196, no. 1, 2019, doi: 10.1088/1742-6596/1196/1/012060.
[16] W. Feng, “A Generalization of Boundary Value Analysis for Input Parameters with Functional Dependency,” in 2010 IEEE/ACIS 9th International Conference on Computer and Information Science, 2010, doi: 10.1109/ICIS.2010.39.
[17] M. Maheshwari, “Boundary Value Analysis for Input Variables with Functional Dependency,” 2017.
