Data Services The selected KPIs for data tests achieved by operators with the split per the aggregation type are presented in Figure 1. Those show differences between operators for browsing, data transfer and YouTube video watching.
Fig. 1. Selected KPI from data test
Results for FDTT HTTP DL (400 MB file transmission) tests are presented in Figure 2.
Fig. 2. KPIs results for FDTT HTTP Capacity Transfer - Download (FDTT HTTP DL)
Large Cities:
All operators had near perfect session success rates. Vip mobile had a much higher average data rate, with 10 Mbps lead over the other two operators. All three mobile operators had a low share of samples with a data rate below 4 Mbps. All three operators achieved high data rates due to the Carrier Aggregation (CA) usage. Vip mobile achieved the best result, despite Telenor having the highest CA usage. The Telekom Srbija network had a similar data rate performance as Telenor, even though Telekom Srbija had the lowest CA usage amongst all operators.
Small Cities:
In Small Cities, the success rate was nearly the same for all competitors. CA usage was much smaller compared to Large Cities, with only Telenor and Vip mobile having some availability of CA. Vip mobile achieved the highest data rates.
Roads:
Telenor took the clear lead in terms of success rate, with Vip mobile following closely and Telekom Srbija remaining behind by 1.6 percent. In terms of data rate, Vip mobile had an advantage over competitors. 4G roll-out on the Roads was significantly lower compared to the other aggregations, except for Vip mobile which had around 92% of LTE usage. Telenor and Telekom Srbija stayed behind with an average data rate lower by 10 Mbps and higher share of samples below 4 Mbps threshold.
Large Cities
Results for FDTT HTTP DL throughputs in Large Cities are shown in Figure 3. They show the average and maximum network capabilities in delivering the highest continuous throughput. Vip mobile achieved the highest average data rate, followed by Telekom Srbija and Telenor.
Fig. 3. FDTT HTTP DL throughputs in Large Cities
Small Cities
Results for FDTT HTTP DL throughputs in Small Cities are presented in Figure 4. Vip mobile took the clear lead in delivering the highest average and maximum throughputs in Small Cities.
Fig. 4. FDTT HTTP DL throughputs in Small Cities
Roads
Results for FDTT HTTP DL throughputs on the Roads are presented in Figure 20.
Fig. 5. FDTT HTTP DL throughputs on the Roads
Results for FDTT HTTP UL (400 MB file transmission) tests are presented in Figures 6.
Fig. 6. KPIs results for FDTT HTTP Capacity Transfer – Upload (FDTT HTTP UL)
Large Cities:
All operators had near perfect overall success rates and very high 4G usage. Vip mobile had the best average data rate, with Telekom Srbija being second and Telenor last. Telekom Srbija was able to achieve high average data rates, despite having almost 10% of samples below 2 Mbps threshold.
Small Cities:
The success rate in Small Cities was still on a near perfect level. The average data rate values dropped for Telekom Srbija and Telenor, but the rank remained the same. Vip mobile was still in the lead. In the Telekom Srbija network there were 13.7% of low data rate samples (below 2 Mbps threshold) measured.
Roads:
The success rate for all operators was marginally worse compared to the cities, but still on a very good level. Only Vip retained high 4G usage on the Roads.
Results for HTTP DL 3 MB tests are presented in Figure 7.
Fig. 7. KPIs results for HTTP File Transfer - Download (HTTP DL 3 MB)
Large Cities:
All operators achieved very good session success rates, Telenor and Vip mobile leading with 99.8% and 99.7%, respectively. Vip mobile had the shortest average session time. The good result of Vip mobile was caused by higher LTE utilization and smallest percentage of samples with low data rates. Telekom Srbija had higher average data rates than Telenor, but the large number of samples with a data rate less than 4 Mbps caused the average session time to be similar in these two networks.
Small Cities:
Similarly to the Large Cities, Vip mobile had the best overall results, with Telekom Srbija being second and Telenor third. Despite having a perfect, 100% success rate, Telenor had the lowest data rate and the longest average session time. Telekom Srbija had the highest number of samples with data rates less than 4 Mbps.
Roads:
As expected, session success rates on Roads were lower compared to cities. Overall, Vip mobile had the best result with the shortest session time and a good success rate. Telenor had very a good success rate, but an average low data rate, resulting in long session times. Telekom Srbija had the lowest success rates and scored second in session time and data rate results.
Results for HTTP UL 1 MB tests are shown in Figure 8.
Fig. 8. KPIs results for HTTP File Transfer - Upload (HTTP UL 1MB)
Large Cities:
All mobile operators had very good session success rates, achieving exactly the same result. Telenor had the best data rate and average session time, due to the almost 100% of 4G usage. Telekom Srbija, despite having better data rates than Vip mobile, had much longer session times, mainly due to large amounts of samples with very slow data rates – 8.8% below the 2 Mbps threshold.
Small Cities:
In Small Cities, Telenor had a clear lead with perfect session success rate and very short session time. Vip mobile was second with Telekom Srbija being far behind. Telekom Srbija again had a high share of samples (10.1%) with a very low data rate (below 2 Mbps threshold).
Roads:
As expected, the success rates on Roads were lower compared to cities. Telenor again was leading in success rates, but Vip mobile took the lead in session time, mainly due to a very low amount of low data rate samples. Telekom Srbija scored the worst, having over 26% of samples below the 2Mbps threshold.
Tests results for HTTP Browsing tests are presented in Figure 9.
Fig. 9. KPIs results for HTTP Browsing tests
Large Cities:
Success rates for both live and reference Kepler web pages were on a very good level for all three operators, with the reference Kepler page having slightly better results, mainly due to the smaller page size. All three operators had a very high 4G utilization resulting in low session times. Telekom Srbija, despite having the lowest success rate, had the best average session times for browsing. Vip mobile was far behind the competition in the reference Kepler page session time results, but second in the live pages session time results.
Small Cities:
In Small Cities, Telekom Srbija was having the lowest success rates, but the shortest session times. Vip mobile took the lead in the Live Pages session times.
Roads:
Success rates were much lower on the Roads compared to the cities. Telenor and Telekom Srbija had a large 3G usage, while Vip mobile retained its result with over 93% of samples in 4G. Telekom Srbija was far behind the competition in success rates for both Kepler and Live Pages. Telenor had the best success rates.
YouTube video quality tests were performed according to ITU Recommendation J.343.1 which is relevant for the assessment of quality at the user end. The model measures the visual effect of spatial and temporal degradations as a result of video coding, erroneous transmission or video rescaling. The J.341.1 model is a no reference method, what allows assessing the quality of any content, even from unknown sources. The live streaming content was used to observe and measure quality of the video with real time data transmission to avoid buffering of whole clip and playing it from memory. As a result, the model provides a subjective assessment of visual quality on the 1 to 5 mean opinion score scale for video (VMOS).
YouTube test results of testing the quality of live stream video transmission are presented in Figure 10.
Fig. 10. KPIs results for YouTube video tests
Large Cities:
Telenor and Vip mobile had very similar results in success rates and time needed to display the first picture of the YouTube video. Telekom Srbija was close behind with slightly lower results. Telenor had the best rate of playouts without interruptions, with Telekom Srbija being the second and Vip mobile the third. All three operators had a very good average VMOS. Telenor had a slightly higher VMOS than their competitors due to the higher share of samples in full HD resolution (1080p) and no samples below a 480p resolution.
Small Cities:
All three operators had very good success rates, with Vip mobile being slightly ahead of the competition. Telenor achieved the shortest time to display the first picture of the YouTube video. The playout rate for Telenor and Vip mobile was very similar and on a good level - over 97.5% of video playouts were without any interruption. Average VMOS was very good for all three operators.
Roads:
Telekom Srbija was behind the competition in success rates. An increase in time to display the first picture of the video could be observed for Telekom Srbija and Telenor. Playout rates degraded mostly for Telekom Srbija and Telenor, while Vip mobile retained a good 95.7% level of video playouts without interruption. Telekom Srbija and Telenor VMOS degraded compared to the cities results, mainly due to lower 4G usage. Vip mobile was clearly the best with over 94% of video samples in full HD and VMOS result of 3.86.
Ping test results are presented in Figure 11.
Fig. 11. Ping test – Duration
Large Cities:
Average latency was on a very good level for all three operators of around 30 milliseconds, which is a value typical for 4G networks. Telenor had the best result, partially due to no samples with a very high duration.
Small Cities:
Average latency was on a very good level for all three operators. Compared to Large Cities, latency increased in Telekom Srbija and Telenor networks.
Roads:
On the Roads, the average ping duration degraded significantly for Telekom Srbija, to a value of 62 milliseconds, which is closer to the average latency expected for 3G networks (around 100 milliseconds). Telenor and Vip mobile retained an excellent level of latency of 35 milliseconds.
