In the last few years, there are many discussions across the world regarding the 2G/3G Network sunset/shutdown. Many operators have either already shut down their 2G/3G or are planning to in the near future.
The drivers behind this move are several with the main ones re-using frequencies to newer technologies like 4G and 5G and reducing costs and effort in maintaining legacy radio networks.
This move will affect end-users in a variety of ways and during this year’s Mobile World Congress in Barcelona SmartViser assessed the true impact of this change. We have performed testing at the event of the Audio MOS quality on different radio networks and using WhatsApp. At the time of the event (28/02/22 – 03/03/22), there were no roaming VoLTE agreements for the networks we have checked VDF UK, O2 UK, Orange FR.
The map below gives an overview of the VoLTE roaming agreements per country and there are none in place for Spain.
Testing Configuration
Testing was conducted at the event with an Orange France SIM roaming on the Orange Spain network. The device used was an LG Velvet. All testing was performed calling a Samsung A52 with an SFR France SIM roaming on the Movistar network.
viSer test automation was used to ensure all the KPIs are collected, reliability and repeatability of the testing. Together with ViSer test automation the add on viSer Audio MOS solution using the POLQA algorithm was also used for the accurate measurement of the audio quality.
Test Sets:
1) Cellular Calls 2G
2) Cellular Calls 3G
3) WhatsApp Calls on 4G Network
The Network condition was very good:
2G (Blue): RxLev ~ -55 dBm
3G (Black): RSCP ~ -60 dBm
4G (Green): RSRP ~ -75 dBm
The diagram below shows the signal strength and timeline.
Audio MOS Testing Results
The average Audio MOS score was a surprise, with a big difference between each option.
2G Cellular calls had an average score of 2.85* (Blue)
3G Cellular Calls had an average score of 3.04* (Black)
WhatsApp Calls using a 4G network had an average score of 4.05* (Green)
*POLQA™ MOS scores:
Score 1: means that despite great effort, it is impossible to understand what is being said during the call (the call repeatedly cuts in and out)
Score 2: means the quality is not good and a lot of effort is needed to understand
Score 3: indicates an acceptable level of quality and fairly understandable conversation
Score 4: is a good quality level, understandable with minor interference noise
Score 5: is an excellent quality level, understandable discussion without interference noise
WhatsApp call used an average bandwidth of 14 kbps:
Performance Testing Conclusion
The results collected from our team show that it is imperative for the industry to prepare fully for the 2G/3G network shutdowns.
It will be important to educate and encourage end-users to upgrade their devices to VoLTE/VoWiFi capable and Mobile Network operators will need to significantly accelerate the VoLTE roaming agreements across the world.
Covid has played a significant role in slowing the progress of putting these agreements in place as roaming traffic had almost disappeared during the lockdowns. The rate of network shutdowns though continued as planned and there is now this gap in Quality of Experience (QoE) that need to be addressed.
As Tommy Ljunggren from Telia Company has highlighted the necessity for action:
“My recommendation to the Operators is to get going now and start with the first roaming agreements, first roaming implementations, because when you have done the first you will learn so much that you can very quickly go to the second, the third, the fourth”
For business users travelling to Spain for the Mobile World Congress in 2022 WhatsApp calls and potentially other OTT apps were the best option over cellular calls.
For end-users whose device does not support VoLTE or if they are using a feature phone then the cellular audio MOS calls are expected to be of poor or average quality. Especially in countries/networks where 3G is already switched off the only option will be 2G calls and from our experience in Barcelona, a lot of effort is required to understand the conversation.
For the latest list on 2G/3G network shutdown please check: https://www.smartviser.com/post/2g-3gnetworkshutdown
GSMA has created a toolkit to address known VoLTE roaming issues and also provide a summary of industry plans to decommission 2G/3G networks. The very useful toolkit can be found: https://www.gsma.com/aboutus/workinggroups/key-areas/volte-roaming
Don't miss our next performance testing reports:
Battery impact between 4G and 5G (UK & France Operators Report).
Private Networks Testing
MVNO vs MNO QoE UK Report
VoLTE key Testing items for OEMs, MNOs, MVNOs
Join our SmartViser Mailing List. Stay up-to-date with all the latest SmartViser testing, Performance Results and Reports. Just Subscribe to get interesting content to your inbox.
Contact us for a Free Trial of viSer Test Automation
Please message us to arrange a chat on how test automation or our test services can help you.
Susie Siouti is the Chief Commercial Officer for SmartViser helping organisations in the Telecommunications industry offer superior end-user quality of experience and service with the introduction of innovative test automation products. Susie has 20 years of experience in the Telecoms industry and in that time has led teams across the world testing and certifying mobile devices together with responsibility for managing a full range of after-sales activities. Susie joined SmartViser in 2016, part of the internal steering committee, and is working with many organisations aiming to create value by offering better quality of products and services with a strong focus on continuous improvement of energy performance and efficiency.
Comments