The selected KPIs for data tests achieved by operators with the split per the aggregation type are presented in Figure 1. Those show differences between operators for data transfer, messaging, browsing and YouTube video watching.
Fig. 1. Selected KPI from data test
Results for Small file DL 3 MB tests are presented in Figure 2.
Fig. 2. KPIs results for Small File Transfer - Download
Big Cities:
All operators achieved excellent session success rates. A1 Srbija had the highest average data rate. Yettel had the second result both in the average and P10/P90 values. Telekom Srbija was slightly behind.
Medium Cities:
All operators had the excellent success rate. A1 Srbija again had the best overall average throughput results, while Yettel was second and Telekom Srbija the last. Telekom Srbija was slightly better in P10 than Yettel.
Small Cities:
Again, all three operators had excellent success rates. A1 Srbija had the highest average data rate followed by Telekom Srbija. Yettel took the last position showing worse averages and P10/P90 values out of all three operators.
Highways:
All operators had very good success rates reaching good throughputs on the average. A1 Srbija had the highest average data rate followed by Telekom Srbija. Yettel took the last position showing worse averages and P10/P90 values out of all three operators.
Main Roads:
Telekom Srbija had the best success rate with A1 Srbija and Yettel following right behind. For the average data rate, A1 Srbija took the lead on the average and P10/P90 values. Telekom Srbija had the second best throughput results and Yettel was the last.
Rural Roads:
Telekom Srbija had the best success rate. For A1 Srbija and Yettel, success rates were lower in rural areas, however A1 Srbija managed to achieve best average throughput results followed by Telekom Srbija and Yettel.
Results for Small file UL 1 MB tests are shown in Figure 3.
Fig. 3. KPIs results for Small file Transfer - Upload
Big Cities:
In Big Cities, all operators achieved perfect session success rates. Telekom Srbija had the best result in terms of the average UL throughputs, Yettel had the second best result and A1 Srbija was slightly behind the competitors.
Medium Cities:
Similarly to the Big Cities, all operators had very good success rates. All operators almost equal on the average throughput results.
Small Cities:
Again, all operators achieved excellent session success rates Telekom Srbija had the best result on the average UL throughputs, Yettel had the second best result and A1 Srbija was slightly behind the competitors.
Highways:
All operators had very good success rates. Telekom Srbija had the best result on the average, P10 and P90 values. A1 Srbija and Yettel were slightly behind.
Main Roads:
Telekom Srbija had the best success rate. Yettel had the second and A1 Srbija had the worst result. For the average data rate, Telekom Srbija took the lead.
Rural Roads:
Telekom Srbija had the best success rate. A1 Srbija and Yettel, had lower success rates in rural areas than Telekom Srbija. All three operators achieved much lower throughputs than in other aggregations.
Tests results for Web Browsing tests are presented in Figure 4.
Fig. 4. KPIs results for Web Browsing tests
Big Cities:
Success rates for web pages were on a very good level for all three operators. Average browsing duration time was at a similar level for all three operators with small advantage of Yettel.
Medium Cities:
In Medium Cities, similarly to Big Cities, the success rates were very good for all three operators. Telekom Srbija with slightly longer average browsing duration than the competitors.
Small Cities:
Again, all operators achieved excellent session success rates. For the average duration, Yettel with the shortest browsing duration. Telekom Srbija and A1 Srbija were almost equal.
Highways:
All operators had very good success rates. Telekom Srbija, Yettel and A1 Srbija had the very similar browsing duration on the average.
Main Roads:
Telekom Srbija had the best success rate with Yettel second and A1 Srbija slightly behind. Telekom Srbija and Yettel had slightly better average result than A1 Srbija for the average browsing duration.
Rural Roads:
Telekom Srbija had the best success rate. A1 Srbija and Yettel, had the success rates lower in rural areas, both reaching worse average browsing durations than Telekom Srbija.
YouTube video quality tests were performed according to ITU Recommendation J.343.1, which is relevant for the assessment of quality at the user end. The model measures the visual effect of spatial and temporal degradations as a result of video coding, erroneous transmission or video rescaling. The J.341.1 model is a no reference method, which allows assessing the quality of any content, even from unknown sources. The live streaming content was used to observe and measure quality of the video with real time data transmission to avoid buffering of whole clip and playing it from memory. As a result, the model provides a subjective assessment of visual quality on the 1 to 5 mean opinion score scale for video (VMOS).
YouTube test results of testing the quality of live stream video transmission are presented in Figure 5.
Fig. 5. KPIs results for YouTube video tests
Big Cities:
Success Rates were very good for all three operators in YouTube tests. All operators had the same average VMOS results. Yettel had slightly better Video access time.
Medium Cities:
In Medium Cities, similarly to Big Cities, Success Rates were on a very good level for all three operators. Both Video Quality MOS and Video Access Times were on similar, very good levels for all three operators.
Small Cities:
Again, In Small Cities, Success Rates were on a very good level for all three operators. Again, both Video Quality MOS and Video Access Times were on similar good levels for all three operators.
Highways:
All operators had very good success rates. Telekom Srbija had slightly better Video Access Time.
Main Roads:
Telekom Srbija had the best success rate while Yettel second and A1 Srbija slightly behind. VMOS were almost equal for all three operators. Telekom Srbija had better Video Access Time than the competitors.
Rural Roads:
Telekom Srbija had the best success rate. A1 Srbija and Yettel had lower success rates in rural areas, resulting worse Video Access Times , than Telekom Srbija.
Messaging tests results are presented in Figure 6.
Fig. 6. Messaging tests
Big Cities:
In Big Cities, all three operators had almost perfect session success rates. A1 Srbija had the shortest average duration, Telekom Srbija had the second best result and Yettel was behind the competitors.
Medium Cities:
Similarly to the Big Cities, all three operators had almost perfect session success rates. Again, A1 Srbija had the shortest average duration, Telekom Srbija came second and Yettel had the longer session time than the competitors.
Small Cities:
In Small Cities, all three operators had almost perfect session success rates. A1 Srbija again had the shortest average duration, followed by Yettel. Telekom Srbija had the worst average duration in this aggregation.
Highways:
All three operators had almost perfect session success rates. Telekom Srbija and A1 Srbija had the shortest average durations. Yettel had the longest average duration.
Main Roads:
Telekom Srbija had the best success rate. A1 Srbija was the second and Yettel was the last. Again, Telekom Srbija and A1 Srbija had the shortest messaging average duration, leaving Yettel behind.
Rural Roads:
Telekom Srbija had the best success rate. A1 Srbija and Yettel had thesuccess rates much lower in rural areas, both reaching longer average messaging durations than Telekom Srbija.
The selected KPIs for data tests achieved by operators with the split per the aggregation type are presented in Figure 7. Those show differences between operators for data transfer, messaging, browsing and YouTube video watching.
Fig. 7. Selected KPI from data test
Results for Small file DL 3 MB tests are presented in Figure 8.
Fig. 8. KPIs results for Small file Transfer - Download
Railways:
Telekom Srbija had the best success rate followed by Yettel and A1 Srbija. Overall, the session success rates are much lower than in other aggregations mainly due to the tunnels and mountainous areas which lead to the weak coverage. A1 Srbija had the best average result for the average data rate.
Hot Spots:
All operators achieved excellent session success rates in the Hot Spots aggregation. Overall, A1 Srbija had the highest average data rate followed by Yettel which scored second and Telekom Srbija with slightly worse result.
Results for Small file UL 1 MB tests are shown in Figure 9.
Fig. 9. KPI results for Small file Transfer - Uploads
Railways:
Overall, the session success rates are much lower than in other aggregations mainly due to the tunnels and mountainous areas which lead to the weak coverage. Telekom Srbija had the best success rate followed by Yettel. A1 Srbija had the poorest success rate result.
Hot Spots:
All operators achieved excellent session success rates in the Hot Spots aggregation. Overall, Telekom Srbija had the highest average data rate followed by Yettel and A1 Srbija.
Tests results for Web Browsing tests are presented in Figure 10.
Fig. 10. KPIs results for Web Browsing tests
Railways:
Overall, the session success rates are much lower than in other aggregations mainly due to the tunnels and mountainous areas which lead to the weak coverage. Telekom Srbija again had the best session success rate and Yettel had the shortest average Browsing Session Duration.
Hot Spots:
All operators achieved excellent session success rates where Yettel achieved shortest average session duration, followed by Telekom Srbija and A1 Srbija.
YouTube video quality tests were performed according to ITU Recommendation J.343.1, which is relevant for the assessment of quality at the user end. The model measures the visual effect of spatial and temporal degradations as a result of video coding, erroneous transmission or video rescaling. The J.341.1 model is a no reference method, which allows assessing the quality of any content, even from unknown sources. The live streaming content was used to observe and measure quality of the video with real time data transmission to avoid buffering of whole clip and playing it from memory. As a result, the model provides a subjective assessment of visual quality on the 1 to 5 mean opinion score scale for video (VMOS).
YouTube test results of testing the quality of live stream video transmission are presented in Figure 11.
Fig. 11. KPIs results for YouTube video tests
Railways:
The session success rates are lower than in other aggregations mainly due to the tunnels and mountainous areas. Telekom Srbija had better session success rate than Yettel and A1 Srbija. VMOS was almost equal for all operators. Telekom Srbija had better Video Access Time than the competitors.
Hot Spots:
All operators achieved excellent session success rates. All three operators had equal average VMOS results. Telekom Srbija was slightly better in the average Video Access Time than the competitors.
Messaging tests results are presented in Figure 12.
Fig. 12. Messaging tests
Railways:
Overall, the session success rates are much lower than in other aggregations mainly due to the tunnels and mountainous areas which lead to the weak coverage. Telekom Srbija had the best Messaging Activity Success Rate. A1 Srbija and Telekom Srbija had shorter average Messaging duration than Yettel.
Hot Spots:
All three operators had very good Session Success Rates. A1 Srbija and Telekom Srbija had shorter average Messaging duration than Yettel.
