Voice Service The results of the main KPIs for voice tests achieved by operators with the split per aggregation type are presented in Figure 1.
Fig. 1. Results of main KPIs for voice tests
Large Cities:
Telenor had the best success rate (almost 98%), with Telekom Srbija in close second and Vip mobile a bit further behind. The shortest average call setup time was in Telekom Srbija (4.5 s), with Telenor second and Vip mobile third. The best average speech quality (MOS - Mean Opinion Score) was achieved by Telenor with a 3.76 score (scale from 1 to 5), with Vip mobile second and Telekom Srbija third. Telekom Srbija had the highest percentage of samples with low MOS score (below 2.3) – over 7%. The other two operators have much better results, with 3.1% for Telenor and 2.74 for Vip mobile.
Small Cities:
The results in Small Cities were similar in terms of operator relative scoring as in Large Cities. Telenor had the best call success rate and achieved the highest average sample quality score. Telekom Srbija had the shortest Call Setup Time and was second in Call Success Rate, but came third in voice sample quality (lowest average and most samples with poor quality). Vip mobile was third in Call Success Rate with the longest Call Setup Time, but had the least number of samples with very low MOS score.
Roads:
The Call Success Rate on Roads was much lower in general then in Cities, with Telekom Srbija having the best score, Telenor coming second, and Vip mobile third. Telenor and Telekom Srbija had average Call Setup Time below 5 seconds, while Vip mobile had over 6 seconds. On the other hand, Vip mobile had the highest average MOS score and the lowest amount of poor quality speech samples. Telekom Srbija had the worst speech quality results while Telenor was second in the ranking.
In Table 1 are shown voice failed calls and dropped calls scores for all three operators in different aggregations.
In Large Cities all operators meet the expected 3% fail rate. Telekom Srbija and Telenor had good Call setup failure rates around 1.5%, while Vip mobile has a much higher rate of 2.81%.
Vip mobile had the lowest percentage of dropped calls, only 0.37%. Telenor was close second with 0.49% and Telekom Srbija was third with 0.80%
In Small Cities, all operators met the minimum expected call success rate - less than 3% of both failed and dropped calls. Most of the failed calls were in Vip mobile, with Telekom Srbija second. Telenor had the best results, below 1% of failed calls.
The percentage of dropped calls in Small Cities was very low for all three mobile operators. The worst result was achieved in the Telenor network and was equal to 0.17%, which still should be considered as a very good result.
As expected, the results on Roads were worse than within the cities. There was no specific region with a concentrated number of failed calls, as those events were distributed throughout the country. Vip mobile had the most fails (over 5%), with Telenor and Telekom Srbija further ahead (below 4%).
The dropped calls rate of Telekom Srbija and Telenor were close to each other. The lowest percentage of dropped calls was measured for the Vip mobile network and was equal to 1.01%, which is 0.5 percent better than the competition.
Table 1. Voice Failed Calls and Dropped Calls
Figure 2 below presents average call setup time values for all three operators in different aggregations.
Fig. 2. Average call setup time values
All three operators implemented fast alerting functionality and achieved comparable results. Telekom Srbija has the best results, with call setup time about 1s faster than Vip mobile. Telenor was second and Vip mobile third.
VoLTE (Voice over 4G) technology is currently not supported in the Serbian market and therefore when the mobile device is connected to the 4G network and used to make or receive voice calls, the device falls back to the 3G or 2G network to complete the call. This procedure is known as the CSFB (Circuit Switched Fall Back) and is part of the call setup procedure. All 3 mobile operators mostly fall-back calls to 3G.
Telenor had the shortest CSFB duration, with an average around 0.6 seconds. Vip mobile had a very similar result, while Telekom Srbija was further behind, with an average longer than 1 second.
Distribution of CSFB duration in Large Cities for all three mobile operators is presented in Figure 3:
Fig. 3. CSFB duration in Large Cities
Figure 4 below presents average MOS values for all three operators in different aggregations.
Fig. 4. Average MOS values
There is a variety of speech coding methods allowing transmission of speech effectively. The use of codecs is determined by the quality of radio signal, available network technology and network capacity. Codecs offering the best speech quality require good radio conditions, reasonable radio resources and support of the codec by the network and mobile devices at both ends of the connection. In cases where the network capacity is limited, mobile operators use codecs with more efficient compression of the speech, but also providing worse speech quality. Distribution of the codecs used has a direct influence on the voice quality represented by MOS value (from 1 to 5, the best being 5). The AMR WB (Adaptive Multi-Rate Wideband) codec offers the best speech quality.
Telenor achieved the best speech quality in all city aggregations, followed by Vip mobile and Telekom Srbija. The results correspond to the use of network technologies and voice codecs. Telenor used AMR WB codec for more than 90% of all samples. The usage of AMR WB codec in the Vip mobile and Telekom Srbija networks is slightly lower, resulting in a lower speech quality achieved.
