The selected KPIs for data tests achieved by operators with the split per the aggregation type are presented in Figure 1. Those show differences between operators for browsing, data transfer and YouTube video watching.
Fig. 1. Selected KPI from data test
Results for Big file transfer (400 MB file transmission) tests are presented in Figure 2.
Fig. 2. KPIs results for Big file Transfer - Download
Large Cities:
All operators had near perfect session success rates, only Telenor achieved 100% success rate. Telenor with highest average data rate of 93Mbps, Telekom Srbija and Vip mobile behind with data rate respectively 66Mpbs and 61Mpbs. All three mobile operators had a low share of samples with data rate below 4 Mbps. All three operators achieved high data rates due to the Carrier Aggregation (CA) usage.
Small Cities:
In Small Cities, the success rate was nearly the same for all competitors. Telenor achieved the highest average data rate, Vip mobile was second and Telekom Srbija was third.
Roads:
Telekom Srbija took the lead in terms of success rate, with Telenor following and Vip mobile remaining behind. In terms of data rate, Vip mobile had an advantage over competitors. Thanks to the highest usage of Carrier Aggregation, Vip mobile achieve much better results than competitors in the matter of average data rate, being 8Mbps faster than Telenor which was second (46.52 Mbps).
Large Cities
Results for Big file DL throughputs in Large Cities are shown in Figure 3. They show the average and maximum network capabilities in delivering the highest continuous throughput. Telenor achieved the highest average data rate, followed by Telekom Srbija and Vip Mobile. Telenor achieved the highest peak data rate.
Fig. 3. Big file DL throughputs in Large Citiess
Small Cities
Results for Big file DL throughputs in Small Cities are presented in Figure 4. Telenor achieved the highest average data rate, followed by Vip mobile and Telekom Srbija. Telenor achieved the highest peak data rate.
Fig. 4. Big file DL throughputs in Small Cities
Roads
Results for Big file DL throughputs on the Roads are presented in Figure 5. Vip mobile achieved the highest average data rate of 55.28Mbps, followed by Telenor and Telekom Srbija. Telenor achieved the highest peak data rate.
Fig. 5. Big file DL throughputs on the Roads
Results for Big file UL (400 MB file transmission) tests are presented in Figure 6.
Fig. 6. KPIs results for Big file Transfer – Upload
Large Cities:
All operators had near perfect overall success rates and very high 4G usage. Telenor had the best average data rate. All three operators had a low percentage of samples below 2 Mbps.
Small Cities:
The success rate in Small Cities was still on a near perfect level. The average data rate values were lower compared to the ones in Large Cities.
Roads:
The success rate for Telekom Srbija and Telenor was on a good level. Vip mobile was behind the competition. The average data rate was much lower than in the cities. Vip mobile had the most samples with low throughput (almost 10%).
Results for Small file DL 3 MB tests are presented in Figure 7.
Fig. 7. KPIs results for Small File Transfer - Download
Large Cities:
All operators achieved excellent session success rates. Telekom Srbija had the shortest average session time and the highest average data rate. Vip mobile had more samples with low throughput comparing to the competition.
Small Cities:
Similarly to the Large Cities, Telekom Srbija had the best overall results, with Telenor being second and Vip mobile third. All operators had low number of samples with data rate below 4Mbps.
Roads:
As expected, session success rates on Roads were lower compared to cities. Overall, Telekom Srbija had the shortest session time and highest average data rate. Telenor scored second in the matter of average data rate, but due to the highest percent of samples with data rate below 4 Mbps, had the longest session time.
Results for Small file UL 1 MB tests are shown in Figure 8.
Fig. 8. KPIs results for Small file Transfer - Upload
Large Cities:
In Large Cities, all operators achieved almost perfect session success rate. Telekom Srbija and Telenor on very similar level, Vip mobile behind the competitors due to highest number of samples with data rate below 2Mbps.
Small Cities:
In Small Cities, all operators had very good success rates. Telenor had the highest data rate and the smallest percent of samples with data rate below 2Mbps which lead to shortest session time. Telekom Srbija and Vip mobile behind respectively.
Roads:
As expected, the success rates on Roads were lower compared to cities. Overall, Telekom Srbija was leading with highest average data rate and best success rate. Despite the above due to high number of samples with data rate below 2Mbps, Telekom Srbija was behind the Telenor in session time. Vip Mobile was with session success rate below expectations, with result of 93.41%.
Tests results for Web Browsing tests are presented in Figure 9.
Fig. 9. KPIs results for Web Browsing tests
Large Cities:
Success rates for both live and reference web pages were on a very good level for Telekom Srbija and Telenor. Vip mobile was having DNS server issues which lead to poor success rate. Average session time on similar level for all operators with lead of Telekom Srbija in Live pages and Telenor in Reference page.
Small Cities:
In Small Cities, similarly to large cites, the results were very good for Telekom Srbija and Telenor. The VIP mobile success rate was impacted with DNS server issue.
Roads:
Success rates were lower on the Roads compared to the cities. Telekom Srbija was the best overall, with the shortest session time for reference page and the best session success rate. The VIP mobile success rate was impacted with DNS server issue.
YouTube video quality tests were performed according to ITU Recommendation J.343.1, which is relevant for the assessment of quality at the user end. The model measures the visual effect of spatial and temporal degradations as a result of video coding, erroneous transmission or video rescaling. The J.341.1 model is a no reference method, which allows assessing the quality of any content, even from unknown sources. The live streaming content was used to observe and measure quality of the video with real time data transmission to avoid buffering of whole clip and playing it from memory. As a result, the model provides a subjective assessment of visual quality on the 1 to 5 mean opinion score scale for video (VMOS).
YouTube test results of testing the quality of live stream video transmission are presented in Figure 10.
Fig. 10. KPIs results for YouTube video tests
Large Cities:
Telekom Srbija and Telenor had very similar results in success rates and the number of playouts without interruptions. The VIP mobile success rate was impacted with DNS server issue. Telekom Srbija had the best average video quality, Telenor and Vip mobile on tied even.
Small Cities:
In Small Cities, similarly to Large Cities, operators had very good results for playouts without interuptions an video quality (VMOS). Vip mobile presented the best average video quality. The VIP mobile success rate was impacted with DNS server issue.
Roads:
On Roads, similarly to other aggregations, the session success rate achieved by Telekom Srbija and Telenor was on a good level. Slight degradation in number of playouts without interruptions could be noticed. The VIP mobile success rate was impacted with DNS server issue.
Ping test results are presented in Figure 11.
Fig. 11. Ping test – Duration
Large Cities:
Average latency was on a very good level for all three operators, below 20 ms, which is very good for 4G networks. Telenor had the best results.
Small Cities:
Average latency was on a very good level for all three operators. Telenor is the first with 23ms, Vip mobile and Telekom are behind.
Roads:
Average latency was on a very good level for all three operators. All operators achieved the same level of ping duration of 24 ms.
