The Effect of Free Virtual Private Networks in the Conductivity of Online Classes in De La Salle University - Senior High School
Simone Franceska Emanuelle M. Capio1*, Rafael Migel P. Dichoso1, Miguel Alfonso D. Guerrero1, and Rafael Ira R. Villanueva1
1De La Salle University Integrated School (Manila)
Katrina Ysabel Solomon1,Research Adviser
1De La Salle University Integrated School (Manila)
Abstract: Virtual Private Networks (VPNs) provide users with online anonymity by sending network traffic through an encrypted tunnel to a server controlled by a VPN service company.
With the abrupt shift of educational and work modes to an online setting, VPN services have been utilized to secure data privacy on the web. Consequently, with online synchronous classes becoming a part of the “new normal,” students are more prone to certain online attacks or breaches. Hence, protective software applications such as VPNs have been in high demand.
However, despite the numerous much-uncovered benefits VPN services offer regarding privacy and security, they have also been found to complicate network paths and affect application performance. Considering this information, this study was conducted to measure the impact of free Virtual Private Networks on network performance during online synchronous classes conducted in areas within the Greater Manila Area. In particular, researchers performed experiments through online synchronous class simulations using the Zoom application software to benchmark various commercially free VPNs on the available local network setups. Through the simulations conducted for online synchronous classes, Hotspot Shield VPN has been deemed the best and recommended free VPN for all network plans utilized in the research. The data from this research could be highly beneficial to VPN developers, Internet Service providers, students, instructors, and even educational institutions in managing the impact of the use of VPNs as a means of online privacy, to Network Performance in an online learning setup, particularly online synchronous sessions within the Greater Manila Area.
Keywords:ISP; VPN; audio; screen sharing; online class; Zoom application software
1. INTRODUCTION
A Virtual Private Network (VPN) connection creates a secure link between users and the Internet in general. VPNs do this by employing encryption and authentication protocols such as OpenSSL, TLS, HMAC, and others. As the Internet has become increasingly important in people’s daily lives, cybersecurity dangers have grown in prominence. As a result, the majority of Internet users have
turned to VPN services to protect their personal information online. However, while VPNs offer important privacy and security benefits, they can also muddle network paths, increase network latency, and negatively influence application performance. With the sudden shift to online classes, both teachers and students have been unprepared for the cybersecurity risks accompanying this type of setup.
In a test for throughput conducted by Wu & Xiao
(2019), a significant decrease in network performance was found with the use of multiple VPN servers at the same time - a phenomenon known as a chain cascade. When using Voice over Internet Protocol (VoIP), these network performance issues might emerge as delayed audio transmission and poor audio quality.
Bahnasse et al. (2021) established that in voice end-to-end delay, the use of IPsec tunneling resulted in 222ms of latency, compared to 180ms without. VoIP jitter was measured to be 17ms without IPsec and 25ms with tunneling. Loading an HTTP page took 148 milliseconds without IPSec tunneling and 216 milliseconds with it.
Slusky (2020) found that the adoption of new intrusive technologies for online examination proctoring increased students’ privacy concerns. A viable but limited solution was the use of a VPN. All network ports on a student’s computer except the one required can be closed by a VPN-enabled application. VPN protocols ideal for online learning setups are Internet Key Exchange version 2 (IKEv2), OpenVPN, and WireGuard.
Despite there being a generous amount of literature available discussing network performance, VPNs, and cybersecurity as a whole, there is much-uncovered ground concerning these topics, especially within the context of online synchronous classes in the Philippines. The emergence of the COVID-19 pandemic saw schools worldwide switch almost entirely to online learning platforms. As a result, lectures, exams, and many other activities usually requiring face-to-face interactions are instead conducted through online conferencing applications, mainly through the Zoom conferencing platform (Version 5.9.1; Yuan, 2021). A growing concern for cybersecurity has led many to consider using VPNs as an extra layer of privacy. Despite their benefits, VPNs are known to add additional latency to an Internet connection and can cause noticeable drops in connection speed. In a real-world setting, this may cause degradation in audio quality when using online conferencing applications. Adding to the fact that unstable Internet connections are already a prevalent issue in the Philippines, it is even more critical to test the impact using VPNs can have on a user's connection and their online conferencing experience as a whole. A user's Internet Service Provider (ISP) and network configuration may also change how a client interacts with a VPN server. It is also important to consider that most students are limited in terms of budget and
often do not have disposable cash to purchase a premium VPN service. It is thus essential that only free, commercially available VPN services are tested.
This study examines the feasibility of using free VPNs for online synchronous classes across different network setups within the Greater Manila Area with the following objectives: to assess the audio quality, and screen sharing quality perceived during Zoom-based online synchronous classes in the Greater Manila Area when using the VPNs, and to establish the most optimal VPN configurations catering to different network setups (ISPs and Network Plans) for a specific online class scenario.
Performance benchmarks were carried out with specific network configurations in the Greater Manila Area, using devices located in Metro Manila and Cavite. Hide.me, Hotspot Shield, and ProtonVPN are the VPNs to be installed.
Following that, audio quality and screen sharing quality was measured in terms of latency (ms), jitter (ms), and packet loss (%). The Zoom Dashboard Statistics page was used to obtain data for the audio quality, and screen sharing quality metrics.
2. METHODOLOGY
2.1Simulation of Online Synchronous Sessions
Online synchronous sessions involving a total of four participants were simulated in order to test the impact of VPN use in real-world settings. In the said simulations, one participant acting as the “teacher” used Zoom’s screen share function to broadcast a prepared slideshow presentation to the other participants. The “teacher” also had their microphone enabled to stream audio as if in a lecture. Accordingly, the three remaining participants acting as the “students” had their microphones muted. All participants had their cameras disabled. The “students”, each under a different ISP, collected performance metrics for each trial. Performance for each of the three VPNs was tested in this manner, including a test case with no VPN enabled. Each trial accordingly was repeated twice for more accurate performance analysis.
Overall, eight experiment trials were conducted with one trial with no VPN, as well as one trial for each VPN altogether being on a different date than the other.
The Zoom’s statistics page was monitored to keep track of audio quality, and screen sharing quality metrics data when conducting online class simulations on the student
participants’ machines, namely their desktop or laptop.
The results for each VPN and their corresponding scenarios were compared to Zoom Video Communications, Inc. threshold values (see Table 1). Any performance metrics with values less than the thresholds were deemed unsuitable for the scenario under consideration.
2.2 Research Design
Metrics were recorded in real-time for each benchmark trial, which lasted a total of 15 minutes with data collected in 5-minute intervals. Accordingly, each benchmark trial represents an online synchronous class simulation.
The results of the online class scenario simulations with and without a VPN were compared in terms of the designated audio and screen sharing metrics.
Table 1 indicates the recommended values for audio, video, and screen sharing metrics to which performance metrics from the Zoom statistics page will be compared against:
Table 1
Recommended Values Specified for Audio and Screen Sharing Metrics
Metric Recommended Values
Latency 150ms or less
Jitter 40ms or less
Packet Loss 2% or less
In essence, latency refers to the amount of time it takes for a single data packet to successfully travel to its destination across a connection (Feamster & Livingood, 2019). It is a measure of delay also often referred to as “lag,”
which can occur in both audio and video calls.
Accordingly, Duman & Eliiyi (2021) defined jitter as the variation in the time delay between when a signal is transmitted and when it is received across a network. This means that as latency increases, so does the jitter.
Lastly, in the process of sending data across a network, small units of data called packets are deployed and received. When one of these packets fails to reach its
intended destination, it is called packet loss. This is calculated as the percentage of packets lost with respect to the packets initially sent (Duman & Eliiyi, 2021).
With each trial having been performed twice, the arithmetic mean of the values for the metrics which were collected during the 15-minute benchmarking trial were recorded as the reference value for a specific metric. After recording the values of the metrics throughout the benchmarking trials, a list of the specific VPNs recommended for particular online synchronous class scenarios was created.
3. RESULTS AND DISCUSSION
The specific online class scenario was conducted with both the cameras of the teacher and students turned off in the Zoom meeting. Accordingly, only the microphone of the teacher is enabled to facilitate the lecture while the microphone of the students would remain muted.
3.1. Converge Xtra Plan 1800
Hide.me and ProtonVPN had higher latency values for both audio and screen sharing during the trial data points, falling short of the threshold value of 300 ms and below (Tables 2 and 3). As a result, when compared to all test cases that met Zoom's recommended metric values for jitter, the jitter for screen sharing when using ProtonVPN was significantly higher. In terms of audio packet loss, all test cases fell short of the recommended values. Similarly, no packet loss was observed for screen sharing in any of the test cases. Hotspot Shield had the least negative impact for audio packet loss as well as audio and screen sharing latency of all the VPNs tested, though not using a VPN is recommended for this scenario because values for either latency or packet loss failed to meet the threshold values set for all test cases using commercially available VPNs. However, if a VPN is to be used, Hotspot Shield would be the best VPN for the situation.
Table 2
Audio Metrics for Simulated Lecture C w/ SS for Converge ISP
VPN No
VPN Hide.me HSS Proton VPN
Latency (ms)
T1 237.00 329.00 211.67 330.33 T2 272.00 365.00 243.33 383.00 AVG 254.50 347.00 227.50 356.67 Jitter (ms)
T1 5.33 4.00 5.33 6.00
T2 10.67 4.33 4.67 7.33
AVG 8.00 4.17 5.00 6.67
Packet Loss (%)
Average
T1 2.90 1.37 1.40 17.23
T2 4.40 0.83 2.20 0.00 AVG 3.65 1.10 1.80 8.62 Max
T1 12.13 8.00 4.00 25.67
T2 11.80 4.53 8.27 0.00
AVG 11.97 6.27 6.14 12.84
Table 3
Screen Sharing Metrics for Simulated Lecture C w/ SS for Converge ISP
VPN No VPN Hide.me HSS Proton VPN
Latency (ms)
T1 220.33 375.67 176.67 551.00 T2 269.67 360.33 220.33 349.67 AVG 245.00 368.00 198.50 450.34 Jitter
(ms)
T1 7.67 4.00 3.00 68.67
T2 16.67 3.00 7.67 9.33
AVG 12.17 3.50 5.34 39.00 Packet
Loss
(%) Average
T1 0.00 0.00 0.00 0.00 T2 0.00 0.00 0.00 0.00 AVG 0.00 0.00 0.00 0.00
Max
T1 0.00 0.00 0.00 0.00 T2 0.00 0.00 0.00 0.00 AVG 0.00 0.00 0.00 0.00
3.2. Globe at Home Unli Fiber Plan 2099
For latency, both ProtonVPN and Hide.me performed worse than Hotspot Shield and the No VPN test case. Accordingly, ProtonVPN had higher values for jitter compared to other VPNs, although still meeting the recommended values for jitter. Despite showing no packet loss on the second trial, ProtonVPN performed the worst across all audio metrics. Hotspot Shield, however, performed as well as, if not at times better than, the No VPN test case, meeting most recommended values for audio metrics.
For screen share metrics Hide.me performed the worst, failing to meet the recommended latency values on both trials. ProtonVPN also largely failed to meet this threshold on the first trial only. In terms of jitter, ProtonVPN had the highest average value compared to its counterparts (see Table 5), it also failed to meet the threshold on the first trial. Furthermore, all screen sharing packet loss values remained 0% across all VPN test cases. Though Hide.me and ProtonVPN failed to meet the recommended values for several metrics, Hotspot Shield performed better on some of the metrics than the No VPN test case. For this reason, Hotspot Shield is recommended.
Table 4
Audio Metrics for Simulated Lecture C w/ SS for Globe ISP
VPN No
VPN Hide.me HSS Proton VPN
Latency (ms)
T1 202.67 383.00 268.33 447.33 T2 297.00 322.00 281.00 265 AVG 249.84 352.50 274.67 365.17 Jitter (ms)
T1 1.00 1.33 1.67 3.00 T2 1.00 1.00 1.00 3.33 AVG 1.00 1.17 1.34 3.17 Packet Average T1 0.00 0.00 0.10 14.13
Loss (%)
T2 7.57 0.00 0.00 0.00 AVG 3.79 0.00 0.05 7.07 Max
T1 0.00 0.00 0.50 25.53
T2 13.17 0.00 0.00 0.00
AVG 6.59 0.00 0.25 12.77
Table 5
Screen Sharing Metrics for Simulated Lecture C w/ SS for Globe ISP
VPN No VPN Hide.me HSS Proton VPN
Latency (ms)
T1 201.67 396.33 266.67 917.67 T2 283.00 322.00 281.00 264.33 AVG 242.34 359.17 273.84 591.00 Jitter
(ms)
T1 0.67 1.67 1.33 54.67
T2 7 0.67 3.00 3.67
AVG 3.84 1.17 2.17 29.17
Packet Loss
(%)
Average
T1 0.00 0.00 0.00 0.00 T2 0.00 0.00 0.00 0.00 AVG 0.00 0.00 0.00 0.00
Max
T1 0.00 0.00 0.00 0.00 T2 0.00 0.00 0.00 0.00 AVG 0.00 0.00 0.00 0.00
3.3. PLDT Unli Fibr Plan 2699
In terms of latency, both ProtonVPN and Hide.me performed markedly worse than Hotspot Shield and the no VPN test case. ProtonVPN also had noticeably worse jitter than the other VPNs, although still meeting recommended values for jitter. From the data that was gathered ProtonVPN on average performed worse in terms of packet loss, although it should be noted that ProtonVPN showed minimal packet loss in the second trial. Despite this inconsistency, ProtonVPN regardless performed the worst across all audio
metrics. Both Hotspot Shield and the no VPN test case performed similarly, meeting the most recommended values for audio metrics.
In the matter of screen share metrics, ProtonVPN again performed the worst, failing to meet recommended latency values across both trials. Hide.me also failed to meet this threshold on the second trial only. ProtonVPN also performed worse in terms of jitter, failing to meet the threshold on the first trial. Finally, screen sharing packet loss values remained at 0% across all test cases. Though Hide.me and ProtonVPN failed to meet recommended values for several metrics, Hotspot Shield again performed similarly to the no VPN test case, thus Hotspot Shield is recommended.
Table 6
Audio Metrics for Simulated Lecture C w/ SS for PLDT ISP
VPN No
VPN Hide.me HSS Proton VPN
Latency (ms)
T1 180.00 248.33 277.67 395.33 T2 269.00 352.00 229.00 335.00 AVG 224.50 300.17 253.34 365.17 Jitter (ms)
T1 1.00 2.33 2.67 23.67
T2 1.00 1.00 1.00 1.00
AVG 1.00 1.67 1.84 12.34
Packet Loss (%)
Average
T1 0.10 0.00 0.00 19.17
T2 0.13 0.00 0.27 0.00 AVG 0.12 0.00 0.14 9.59 Max
T1 0.60 0.00 0.00 27.83
T2 0.50 0.00 1.07 0
AVG 0.55 0.00 0.54 13.92
Table 7
Screen Sharing Metrics for Simulated Lecture C w/ SS for PLDT ISP
VPN No VPN Hide.me HSS Proton VPN Latency (ms) T1 179.00 284.00 277.00 754.00
T2 238.00 352.00 320.00 341.00 AVG 208.50 318.00 298.50 547.50 Jitter
(ms)
T1 1.00 1.00 2.67 64.00
T2 1.33 1.00 2.67 3.33
AVG 1.17 1.00 2.67 33.67
Packet Loss
(%)
Average
T1 0.00 0.00 0.00 0.00 T2 0.00 0.00 0.00 0.00 AVG 0.00 0.00 0.00 0.00
Max
T1 0.00 0.00 0.00 0.00 T2 0.00 0.00 0.00 0.00 AVG 0.00 0.00 0.00 0.00
3.4. Comparative Analysis of the Three VPNs
Class simulations conducted under the Converge Xtra Plan 1800 were observed to have experienced audio and screen sharing “lags” accompanied by significantly high values for packet loss, failing to meet the threshold values set even without a VPN activated. Thus, activating each VPN per trial accordingly resulted in significantly high values for latency and packet loss. In particular, although each VPN did not affect the screen sharing packet loss, all test cases fell short of the recommended values, particularly for audio packet loss. However, benchmarking trials with Hide.me, and ProtonVPN activated performed worse, recording latency values for both audio and screen sharing surpassing the threshold limits set by zoom. Thus, out of the VPNs used, Hotspot Shield was observed as the VPN to have the least negative impact on the network plan. However, due to the high latency and packet loss values recorded for all the test cases, the use of VPN under the network plan is not recommended.
Conversely, the No VPN test case met the most recommended values for audio and screen sharing metrics for Globe at Home Unli Fiber Plan 2099. When VPNs are activated, it was observed that both ProtonVPN and Hide.me performed worse than Hotspot Shield and the No VPN test case in terms of latency. However, ProtonVPN was noted to have poor performance for audio latency, jitter, and packet loss, as well as screen-sharing jitter with its recorded values,
as well as screen-sharing jitter, being higher than the other VPNs. Consequently, Hide.me recorded poor values for screen sharing latency values on both benchmarking trials.With Hotspot Shield performing better than the other 2 VPNs utilized, it is therefore recommended.
Lastly, benchmarking trials through the PLDT Unli Fibr Plan 2699 network plan likewise have recorded stable network data for the No VPN test case. However, it was likewise observed that both ProtonVPN and Hide.me recorded poor values for latency, whilst trials activating ProtonVPN have recorded the worst values for audio and screen sharing metrics particularly failing to meet the recommended latency values across both trials. Screen sharing packet loss values accordingly remained at 0% across all test cases. Hence, Hotspot Shield met the most recommended values for audio and screen sharing metrics, it is recommended.
Overall, through benchmarking trials for all network plans, common trends with regard to VPN performance have been identified. Primarily, ProtonVPN has recorded the worst performance during benchmarking sessions for all 3 network plans. Accordingly, Hide.me VPN performs better than ProtonVPN yet still poorly performs in terms of audio and screen sharing latency for most trials.
Lastly, HotspotShield VPN is the recommended VPN for all benchmarking trials, having the least negative impact for each network plan primarily when conducting online synchronous sessions.
4. CONCLUSION
VPNs provide users with online anonymity by sending network traffic through an encrypted tunnel to a server controlled by a VPN service company. However, VPNs have also been found to complicate network paths and affect application performance. Hence, in this study, we evaluated the effects of 3 VPNs, namely, Hide.me, Hotspot Shield, and Proton VPN, to examine the feasibility of using free VPNs for online synchronous classes across different network setups within the Greater Manila Area mainly by analyzing its impacts on audio quality and screen sharing quality. In particular, the researchers were able to assess the performance of the selected free VPNs by activating them while conducting simulations for online class lectures conducted through Zoom. To do this, audio, and video metrics were recorded through the Zoom Statistics
Dashboard, and the recorded metrics were compared to threshold values for online conferencing officially recommended by Zoom. After analyzing how each of the VPNs affected the different performance metrics set for online conferencing, the researchers used the collected data to recommend a specific VPN from the selection, particularly for each ISP the participating researchers have utilized. For online class participants subscribing to the Converge Xtra Plan 1500, it is recommended to partake in online classes without the use of VPNs due to its negative effects on network speed, yet may opt to use VPNs when needed particularly, Hotspot Shield VPN is recommended.
Consequently, online class participants subscribed to Globe at Home Unli Fiber Plan 2099 are likewise recommended to use Hotspot Shield VPN when attending online classes.
Lastly, online class students subscribed to the PLDT Unli Fibr Plan 2699 are recommended to also use Hotspot Shield VPN for online classes as well. Overall, through the simulations conducted for online synchronous classes, Hotspot Shield VPN has been deemed the best and recommended free VPN for all network plans utilized in the research. By utilizing the recommended VPNs, the online data traffic of users could be protected from external access.
The results of the study were interpreted through the data gathered by the researchers when measuring network metrics from their devices with their respective network setups for an online class. In particular, local ISPs available within the Philippines namely, Converge, Globe, and PLDT were utilized by each corresponding researcher to assess VPN performance in accordance with the measured audio and screen sharing metrics of their respective network plans. Accordingly, the online synchronous class simulations are limited to online discussions wherein only the teacher is active. Likewise, the data gathering process was only done twice for each VPN.
Hence, further benchmarking experimentations of the specified free VPNs particularly involving other ISPs, local or international, are highly recommended to further add information to the study. Accordingly, the simulation of different online class activities such as class recitation, or group presentation is highly recommended to assess the effectiveness of VPNs on other online learning activities.
Lastly, it is recommended for researchers to utilize more benchmarking experimentation trials for each VPN for more accurate data analysis.
5. REFERENCES
Bahnasse, A., & EL KAMOUN, N. (2017).
Performance Evaluation of Web-based Applications and VOIP in Protected Dynamic and Multipoint VPN. InComputing Conference.333
Feamster, N., & Livingood, J. (2019). Internet speed measurement: Current challenges and future recommendations.arXiv preprint
arXiv:1905.02334.
Duman, İ , Eliiyi, U . (2021). Performance Metrics and Monitoring Tools for Sustainable Network Management .Bilişim Teknolojileri Dergisi , 14 (1) , 37-51 . DOI: 10.17671/gazibtd.780504
Slusky, L. (2020). Cybersecurity of Online Proctoring Systems.Journal of International Technology and Information Management, 29(1), 56–83.
Wu, Z., & Xiao, M. (2019, May). Performance Evaluation of VPN with Different Network Topologies. In2019 IEEE 2nd International Conference on Electronics Technology (ICET)(pp. 51-55). IEEE.
Yuan, E.S. (2022). Zoom Meetings [Computer Software].
Retrieved from https://zoom.us
Zoom system requirements: Windows, macOS, Linux.
(2022).
https://support.zoom.us/hc/en-us/articles/20136202 3-Zoom-system-requirements-Windows-macOS-Li nux