International Conference on Internet Studies, August 16-17, 2014, Singapore
EVALUATION ON WEB ACCESSIBILITY, POPULARITY
AND VISIBILITY IN INDONESIAN E-GOVERNMENT
Hanum Putri Permatasari1 Dessy Wulandari Asfary Putri2
Abdus Syakur3 Anacostia Kowanda4
1, 2, 3, 4 Gunadarma University, Indonesia
1, 2, 3, 4 {hanum, dessy_wap, syakur, anacos}@staff.gunadarma.ac.id
ABSTRACT
Today, web accessibility, popularity and visibility are becoming important in many countries, especially for government that trust on the web for serving full of information easily reached by their citizens, then called e-government. This study is to evaluate the web accessibility on ministry and state supreme agency websites in Indonesia based on conformance level of WCAG 2.0 by W3C. Furthermore, this study used an approach for measuring data traffic analysis web as popularity using Alexa and Majestic SEO to examine visibility of the web. Data analysis with statistical test using ANOVA. The study concluded that web accessibility on state supreme agency websites in Indonesia lower than ministry websites. Through the result, both popularity and visibility on ministry websites gained higher rank than state supreme agency websites. Ministry and state supreme agency websites in Indonesia still requires improvement of accessibility, popularity, and visibility.
Keyword: web accessibility, web popularity, web visibility, e-government, indonesia
1. INTRODUCTION
development status of telecommunication infrastructure, and inherent human capital. First, to arrive at a set of online service index values, the researchers assessed each
country’s national website, including the national central portal, e-services portal and e-participation portal, as well as the websites of the related ministries of education, labour, social services, health, finance, and environment as applicable. In addition to being assessed for content and features, the national sites were tested for a minimal level of web content accessibility as described in the Web Content Accessibility Guidelines of the World Wide Web Consortium (W3C). Second, the telecommunication infrastructure index is an arithmetic average composite of five indicators: estimated internet users per 100 inhabitants, number of main fixed telephone lines per 100 inhabitants, number of mobile subscribers per 100 inhabitants, number of fixed internet subscriptions per 100 inhabitants, and number of fixed broadband facilities per 100 inhabitants. The International Telecommunication Union is the primary source of data in each case. Third, the human capital index is a weighted average composite of two indicators: adult literacy rate and the combined primary, secondary, and tertiary gross enrolment ratio, with two thirds weights assigned to adult literacy rate and one third weight assigned to the gross enrolment ratio. The United Nations Educational, Scientific and Cultural Organization is the main source of data for both indicators. All data gaps were filled either using data from the 2010 UNDP Human Development Report or using proxy indicators from other authoritative sources such as official UNICEF figures from its public report or World Bank figures on its website. According to the 2012 United Nations E-government Survey Rankings which was released on 29th February 2012, the Republic of Korea is the world leader (0.9283) followed by the Netherlands (0.9125), the United Kingdom (0.8960) and Denmark (0.8889), with the United States, Canada, France, Norway, Singapore and Sweden close behind. Indonesia gained in the world rankings from 109 to 97 out of 193 countries in 2012. Fig. 1 presents e-government development in countries with populations larger than 100 million.
2. THEORETICAL BACKGROUND
2.1 Web Accessibility
World Wide Web Consortium (W3C) determines 19 countries have their own legislations or policies about web accessibility. Unfortunately, it has not been implemented yet in Indonesia especially by the governments. Today, web accessibility is becoming important in many countries, especially for government that trust on the web for serving full of information easily reached by their citizens. Many governments especially ministry websites in Indonesia are poorly designed and have accessibility obstacles that prevent people with disabilities from using them.
W3C has provided guidelines for web accessibility under the name of Web Accessibility Initiative (WAI), and this initiative has launched the Web Content Accessibility Guidelines (WCAG 1.0 and WCAG 2.0). How WCAG 2.0 is different from WCAG 1.0, based W3C [19], WCAG 2.0 applies more broadly to different types of web technologies and to more advanced technologies. It is designed to apply as technologies develop in the future. The WCAG 2.0 requirements are more precisely testable with automated testing and human evaluation. This allows WCAG 2.0 to be more easily used where specific requirements and conformance testing are necessary, such as in design specifications, purchasing, regulation, and contractual agreements. Thus, this study has examined the web accessibility on ministry and state supreme agency websites in Indonesia. Besides accessibility, web visibility also plays important roles especially in universities. In particular, web visibility of university is very significant in order to improve their university rankings. Such Webometrics accounted 50% for visibility indicator. This study also to examine the correlation between accessibility, popularity, and visibility indicator on ministry and state supreme agency websites in Indonesia.
2.2 Related Research
Table 1. Related Research on Web Accessibility
ministries that are within the scope of duties; (5) Ministerial Institute; and (6) State Supreme Agencies.
Table 2. The Classification of Indonesian Government
No. Category (1)
1 Ministry of Home Affairs
2 Ministry of Foreign Affairs
3 Ministry of Defense
No. Category (3)
1 Ministry of National Development Planning
2 State Minister for Research and Technology
3 State Minister for Cooperatives Small and Medium Enterprises
4 State Minister for State Owned Enterprise
5 State Minister for Acceleration Development Backward Regions
6 State Minister for Environment
7 State Minister for The Empowerment of State Apparaturs and Bureaucracy Reform
8 State Minister for Public Housing
9 State Minister for Women Empowerment and Child Protection
10 State Minister for Youth and Sports Affairs
11 Ministry of the State Secretariat
No. Category (2)
1 Ministry of Justice and Human Right
7 Ministry of Public Work
8 Ministry of Manpower and Transmigration
9 Ministry of Agriculture
10 Ministry of Forestry
11 Ministry of Maritime and Fisheries Affairs
12 Ministry of Energy And Mineral Resources
13 Ministry of Health
14 Ministry of Education and Culture
15 Ministry of Social Services
16 Ministry of Religious Affairs
17 Ministry of Tourism and Creative Economics
No. Category (5)
1 Cabinet Secretariat
2 Attorney General
No. Category (4)
1 Coordinating Ministry for Political, Legal and Security Affairs
2 Coordinating Ministry for Economic Affairs
3. METHODOLOGY
The study was conducted with the main focus on evaluation of accessibility, popularity and visibility on ministry and state supreme agency websites in Indonesia. For the first classification, this study selected 36 ministries that consisting of 5 categories and 8 state supreme agency websites in Indonesia. Second classification with detail comparison of variable research and the result of ANOVA test are based on coordinating ministry consisting three groups and 8 state supreme agency websites in Indonesia. Evaluation of the first phase was to make sure those websites run well. Next, validate the web using the W3C online validator [17]. Evaluation of web accessibility using WCAG 2.0. Automatic evaluation software tool [18] that was used in this study for validating WCAG 2.0, such as aChecker [14], recommended by W3C. The official ministry and state supreme agency websites were evaluated for WCAG 2.0 conformance Level A, AA and AAA. This study used an approach for data traffic analysis web using Alexa [1], include global rank, id rank and reputation link. Majestic SEO [7] was used for offering special function that search for matches only in web elements such as referring domain, external backlinks, citation flow and trust flow. Data collection and measurement were conducted on May 18-20 2014.
4. RESULT AND DISCUSSION
The object of this study consisted of 36 ministries that was classified first into 5 categories (as mentioned in literature) and 8 state supreme agency websites in Indonesia. Validating a website is important to check for valid links in order to interpret well on search engines and also users. So, the first step was to validate the web using the W3C online validator. Fig. 2 shows the average number of parsing errors in ministry and state institution websites. The most common mistakes of parsing error from W3C validator were missing attribute for element html or false in writing HTML script, incorrect nesting of elements, without using alternative name of image or
No. Category (6)
1 People's Consultative Assembly
2 The Indonesian House of Representative
3 The Regional Representative Council
4 ConstitutionalCourt
5 Supreme Court
6 The AuditBoard of RI
7 President
form. XHTML 1.0 transitional was document type that mostly used by ministry and state supreme agency websites. XHTML 1.0 is older than HTML5. Few websites used document type as a HTML 4.01 Transitional and even HTML5.
Figure 2. Average Number of Parsing Errors
After validating the websites using W3C validator, each website was manually checked against the WCAG 2.0 success criteria by using previously aChecker, automatic evaluation software tool that was used in this study for validating WCAG 2.0. WCAG 2.0 has three levels of conformance are defined: A (lowest), AA, and AAA (highest). Conformance Level: one of the following levels of conformance is met in full [20].
Level A: For Level A conformance (the minimum level of conformance), the web page satisfies all the Level A Success Criteria, or a conforming alternate version is provided.
Level AA: For Level AA conformance, the web page satisfies all the Level A and Level AA Success Criteria, or a Level AA conforming alternate version is provided.
Level AAA: For Level AAA conformance, the web page satisfies all the Level A, Level AA and Level AAA Success Criteria, or a Level AAA conforming alternate version is provided.
There were several ministries in level A (5 ministries), AA (8 ministries), AAA (5 ministries) conformance that passed or no known problems detected. Five ministries that passed to all level conformance were, The Ministry of Defence, The Ministry of Transportation, The Ministry of Manpower and Transmigration, The Ministry of Tourism and Creative Economics then The Ministry of Development of Disadvantaged Regions. The state supreme agency website which had least known problems in all level conformance was The People's Consultative Assembly of the Republic of Indonesia.
Figure3. Average Failed of Success Criteria
Alexa traffic ranking is a website indexing engine that attempts to measure popularity of websites. The lower the Alexa ranking, the more popular of website is viewed. In Table 3 we could see that the average of Majestic SEO and Alexa Rank in Ministries and State Institution. The highest rank elements in Alexa were achieved by ministry on category number one (1) for global rank; rank followed by country was on category number two (2); and number of reputation link was on category number two (2). The following are explanation about the rank elements in Alexa:
- Global, an estimate of website's popularity. The rank is calculated using a combination of average daily visitors to a website and pageviews on a website over the past 3 months. The site with the highest combination of visitors and pageviews is ranked #1.
- ID, an estimate of website's popularity in a specific country (Indonesia). The rank by country is calculated using a combination of average daily visitors to a website and pageviews on a website from users from that country over the past month. The site with the highest combination of visitors and pageviews is ranked #1 in that country (Indonesia).
seen by users in the Alexa traffic panel are not counted. Multiple links from the same site are only counted once.
Majestic SEO is a link intelligence tools for SEO and internet page rank and also marketing. It is used for Webometrics research to evaluate the impact indicator. The quality of the contents is evaluated through a "virtual referendum", counting all the external inlinks that the University webdomain receives from third parties. Those links are recognizing the institutional prestige, the academic performance, the value of the information, and the usefulness of the services as introduced in the webpages according to the criteria of millions of web editors from all over the world. As it is seen from the table below (Table 3), the average of Majestic SEO and Alexa Rank in ministries and state supreme agency. The highest elements of Majestic SEO were achieved by ministry on category number one (1) for referring domain; external backlinks was on category number two (2); citation flow was on category number two (2); and trust flow was on category number three (3). The following are explanation about Majestic SEO elements:
- Referring Domain, also known as "ref domain", is a domain from which a backlink is pointing to a page or link.
- External Backlinks, also referred to in SEO as a external "inlink".
- Citation Flow, is a Majestic SEO Flow Metric, which is weighted by the number citations to a given URL, or Domain.
- Trust Flow, is a Majestic SEO Flow Metric, which is weighted by the number of clicks from a seed set of trusted sites to a given URL, or Domain.
Through the result, either in Alexa ranking or Majestic SEO, the ministry with category number two mostly gained the highest rank of all elements. These elements were rank by country and number of reputation link (Alexa ranking) also external backlinks and citation flow (Majestic SEO).
Table 3. Average of Majestic SEO and Alexa Rank
1 2 3 4 5 6
Majestic SEO
Referring
Domain 605 514 324 200 235 368
External
Backlinks 97,437 163,997 65,964 35,551 51,671 62,463 Citation
Flow 33 38 34 24 37 34
1 2 3 4 5 6
Alexa Traffic
Global
Rank 172,146 361,059 558,262 1,011,557 242,423 529,372
ID Rank 5,042 1,638 10,749 40,720 4,268 14,797
Reputation
Link 636 1,599 1,087 417 783 766
In addition besides ministry that handle government affairs of above, there is also Coordinating Ministry is in charge of the ministries in terms of synchronization and coordination of affairs of the ministries that are within the scope of duties. That Coordinating Ministry is classified into three groups: (1) Coordinating Ministry for Political, Legal and Security Affairs consisting of 6 ministries; (2) Coordinating Ministry for Economic Affairs consisting of 15 ministries; and (3) Coordinating Ministry for People's Welfare consisting of 9 ministries. State supreme agency into fourth group in this study consisting of 8 agencies including the website of president and vice president. Comparison of variable value in this study from the group of Coordinating Ministry and State Supreme Agency are presented in the figure 4 below.
Table 4. Group of Coordinating Ministry
No. Coordinating Ministry for Political, Legal and Security Affairs
1 Ministry of Home Affairs
2 Ministry of Foreign Affairs
3 Ministry of Defense
4 Ministry of Justice and Human Right
5 Ministry of Communication and Informatics
6 State Minister for The Empowerment of State Apparaturs and Bureaucracy Reform
No. Coordinating Ministry for People's Welfare
1 Ministry of Health
2 Ministry of Education and Culture
3 Ministry of Social Services
4 Ministry of Religious Affairs
5 Ministry of Tourism and Creative Economics
6 State Minister for Environment
8 State Minister for Public Housing
9 State Minister for Youth and Sports Affairs
No. Coordinating Ministry for Economic Affairs
1 Ministry of Finance
2 Ministry of Energy And Mineral Resources
3 Ministry of Industry
4 Ministry of Trade
5 Ministry of Agriculture
6 Ministry of Forestry
7 Ministry of Transportation
8 Ministry of Maritime and Fisheries Affairs
9 Ministry of Manpower and Transmigration
10 Ministry of Public Work
11 State Minister for Research and Technology
12 State Minister for Cooperatives Small and Medium Enterprises
13 State Minister for Acceleration Development Backward Regions
14 Ministry of National Development Planning
15 State Minister for State Owned Enterprise
URL, or domain” and trusts flow as “one of the Majestic SEO flow metrics, the which is weighted by the number of clicks from a seed set of trusted sites to a given URL, or
domain”.
The low traffic conditions was one measure of website performance as mentioned by Chander and Kush [11] which related to the traffic analysis with usability and popularity on government websites in India. Popularity of the government websites that was still low also expressed by Silfianti, Suhendra, and Suryadi [15] for local government website then Silfianti and Suhatril [16] for the provincial government website in Indonesia. The results of ANOVA analysis showed that the differences variable in this study between ministry and state supreme agency were not significantly as shown in the table 5 below.
Table 5. The Result of ANOVA Test
Sum of Squares df Mean Square F Sig.
Parsing Errors Between Groups 97998.113 3 32666.038 .487 .693
Within Groups 2548346.291 38 67061.745
Total 2646344.405 41
Parsing Warnings Between Groups 62884.504 3 20961.501 .612 .611
Within Groups 1300781.782 38 34231.100
Total 1363666.286 41
Referring Domain Between Groups 472837.399 3 157612.466 1.181 .330
Within Groups 5070542.220 38 133435.322
Total 5543379.619 41
External Backlinks Between Groups 2.825E10 3 9.417E9 .587 .627
Within Groups 6.096E11 38 1.604E10
Total 6.379E11 41
Citation Flow Between Groups 259.759 3 86.586 1.231 .312
Within Groups 2673.384 38 70.352
Total 2933.143 41
Trust Flow Between Groups 272.948 3 90.983 1.008 .400
Within Groups 3430.195 38 90.268
Total 3703.143 41
Global Rank Between Groups 3.320E12 3 1.107E12 1.819 .160
Within Groups 2.313E13 38 6.086E11
Total 2.645E13 41
Reputation Link Between Groups 4861463.345 3 1620487.782 .960 .421
Within Groups 6.414E7 38 1687831.752
Total 6.900E7 41
KnownProblems Between Groups 1857.730 3 619.243 .310 .818
Within Groups 75984.175 38 1999.584
The results of ANOVA test generally showed that ministry and state supreme agency websites relatively have the same performance seen from the 9 parameters that investigated. Ministry and state supreme agency websites in Indonesia still requires improvement of accessibility, popularity, and visibility. This can be done by increasing the quantity and quality of content, diversity of service features, as well as technical aspects that related to design and web programming, as suggested also by Hermana and Silfianti [3].
5. CONCLUSION
The study concluded that web accessibility in Indonesian e-government especially on state supreme agency websites lower than ministry websites. The most common mistakes of parsing error from W3C validator were missing attribute for element HTML or false in writing HTML script, incorrect nesting of elements, without using alternative name of image or form. Many specific aspects mentioned on result and discussion section based conformance level of WCAG 2.0. There were several ministries in level A (5 ministries), AA (8 ministries), AAA (5 ministries) conformance that passed or no known problems detected. Five ministries that passed to all level conformance were the Ministry of Defence, the Ministry of Transportation, the Ministry of Manpower and Transmigration, the Ministry of Tourism and Creative Economics then State Minister for Acceleration Development Backward Regions. The state supreme agency websites which had least known problems in all level conformance was the People's Consultative Assembly of the Republic of Indonesia. Through the result, both popularity and visibility on ministry websites gained higher rank than state supreme agency websites. Ministry and state supreme agency websites in Indonesia still requires improvement of accessibility, popularity, and visibility. This can be done by increasing the quantity and quality of content, diversity of service features, as well as technical aspects that related to design and web programming.
6. REFERENCES
[1] Alexa, “Traffic Statistic Web,” 2014, http://www.alexa.com/
[2]B. Al Mourad and F. Kamoun, “Accessibility evaluation of dubai e-government
websites: Findings and implications,” Journal of E-Government Studies and Best Practices, pp. 1-15, 2013.
[3] B. Hermana and W. Silfianti, “Evaluating E-government Implementation by Local Government: Digital Divide in Internet Based Public Services in Indonesia,” International Journal of Business and Social Science, vol. 2 no. 3, 2011.
[4] E.-J. Park, Y.-W. Lim, and H.-K. Lim, “A study of web accessibility of websites built in html5-focusing on the top 100 most visited websites,” International Journal of Multimedia and Ubiquitous Engineering, vol. 9, no. 4, pp. 247-256, 2014.
[6] M. R. Patra, A. R. Dash, and P. K. Mishra, “A quantitative analysis of wcag 2.0 com-pliance for some indian web portals.” International Journal of Computer Science, Engineering & Applications, vol. 4, no. 1, 2014.
[7] Majestic SEO, “Backlink Checker & Site Explorer,” 2014, http://www.majesticseo.com/
[8] President of the Republic of Indonesia, “Presidential Decree No. 47 Year 2009 on Formation and Organization of the Ministry of State,” 2009, http://www.dephut.go.id/uploads/files/Perpres_47_2009.pdf
[9] President of the Republic of Indonesia, “Act of the Republic of Indonesia No. 14
Year 2008 on Public Information Openness,” 2008,
http://pppl.depkes.go.id/_asset/_regulasi/UU14th2008_ttg_KIP.pdf
[10] R. Goncalves, J. Martins, J. Pereira, M. A.-Y. Oliveira, and J. J. P. Ferreira,
“Enterprise web accessibility levels amongst the forbes 250: Where art thou o
virtuous leader?” Journal of business ethics, vol. 113, no. 2, pp. 363-375, 2013. [11] S. Chander and A. Kush, “E-Governance Web Portals Assessment of Two States,”
International Journal of Advanced Research in Computer Science and Software
Engineering, vol. 2, issue 2, February 2012.
[12] S. M. Lee, S.-G. Hong, D.-H. An, and H.-M. Lee, “Disability users evaluation of
the web accessibility of sns,” Service Business, pp. 1-24, 2013.
[13] The UN E-Government Survey, “E-Government for the People,” ISBN: 978-92-1-123190-8, 2012,
http://unpan1.un.org/intradoc/groups/public/documents/un/unpan048065.pdf [14] Web Accessibility Checker, “AChecker,” 2014, http://achecker.ca
[15] W. Silfianti, A. Suhendra, and H.S. Suryadi, “Performance Evaluation of Indonesian Local Govenment Website: Analysis of Web Content, Traffic and Webmetric,” Proceeding of Global Management Conference – Bali, Indonesia April-May 2010.
[16] W. Silfianti and R.J. Suhatril, “Do Indonesian Province Website Rich and Popular?,”World of Computer Science and Information Technology Journal, vol. 1, no.6, p.253-259, 2011.
[17] W3C, “W3C HTML Validation Tools,” 2014, http://validator.w3.org/
[18] W3C, “Complete List of Web Accessibility Evaluation Tools,” 2014, http://www.w3.org/WAI/ER/tools/complete
[19] W3C, “How WCAG 2.0 Differs from WCAG 1.0,” 2014, http://www.w3.org/WAI/WCAG20/from10/diff.php
