VoWiFi presents a new set of conditions, and a new set of challenges, when it comes to evaluating device performance under laboratory conditions.

Guest author

May 18, 2016

8 Min Read
ee wifi calling

Telecoms.com periodically invites expert third parties to share their views on the industry’s most pressing issues. In this piece Dharen Ells Business Development Manager at Spirent discusses the pros and cons of using WiFi to offload pressure on mobile networks.

Who to blame when your premium smartphone drops a call or sounds really cheap? It worked fine back home – so it must be the lousy network!

Customers still find it more natural to blame the network rather than admit they might have chosen the wrong phone. This despite the fact that, back in 2009, a respected European test lab, Broadband Testing, put several of the best handsets to the test and found that their performance varied widely under a range of everyday operating conditions, such as walking, moving in a car or train, or handing over between cells. Since that report, the industry has taken its mobile testing a lot more seriously.

This might just be history, except that Voice over WiFi (VoWiFi) adds a new complication: can handover between the mobile and WiFi networks happen seamlessly and without loss of quality? Get it wrong, and the mobile provider will likely be blamed. Get it right, and the provider can deliver first class service and reduce pressure on the mobile network at a lot less cost. VoWiFi presents a new set of conditions, and a new set of challenges, when it comes to evaluating device performance under laboratory conditions.

The role of VoWiFi

Nearly 50 million hotspots were deployed globally by January 2015, an 80 per cent increase since 2013. The US alone had over 30 million hotspots deployed by multiple service operators by the end of last year. This spread of WiFi coverage across in urban environments – plus the low cost of deploying WiFi – have contributed to an explosion of data exchange over Wi-Fi. Nearly 80% of all mobile data is now exchanged over Wi-Fi networks.

WiFi spectrum is free and unlicensed, with bandwidths over 500 MHz available. Spectrum use is efficient, because its short-range coverage means that frequencies can be re-used over short distances to enhance service coverage, and keep customers happy. So it is hardly surprising that industry groups – including operators, device manufacturers and OEM vendors – want to roll out VoWi-Fi as quickly and effectively as possible.

Effective rollout means ensuring that customers enjoy high-performance and a seamless transition to and fro between LTE and Wi-Fi. So these industry groups must find optimal ways to test and measure the Key Performance Indicators (KPIs) of VoWi-Fi performance in both the field and lab environments.

Fail to prepare? Prepare to fail!

The ultimate test of any service will be made by the user base – and that can throw up some nasty surprises. Thorough pre-testing of a service becomes essential to avoid embarrassing failure and the attendant bad publicity and reputation damage. So the test engineer must develop tests as close as possible to that ultimate user test, in order to ensure that the service roll-out is a credit to the mobile service provider as well as the handset.

You cannot be sure about overall performance and quality of experience if you merely test individual functions because, even without the addition of fancy apps, normal usage can include several functions happening simultaneously. For example: during a business voice call you check the e-mail to see if the document under discussion has arrived; or you invite someone for lunch and start searching for nearby restaurants and e-mail them a map. There are also hidden demands of network connectivity that depend on the actual handset: some save battery power by logging on and off the network, and phones can be handing over between several cells in its active set at one time.

There are three types of test scenario:

  • Testing in the field is nearest to the user experience, but consistent coverage of every permutation of terrain, weather, cityscape and movements between cells and between networks is not feasable.

  • A test network can be purpose built to incorporate a lot of these different conditions, but it would be very expensive to build a truly comprehensive test environment.

  • Testing in the laboratory clearly wins in terms of lower cost, convenience and repeatability, but it does require sophisticated equipment and expert advice to model and record every aspect of real life network, environment and usage variation.

Fortunately, there are sophisticated laboratory test devices that can emulate a whole portfolio of operating conditions, individually and in combination. These include: movement at varying speeds, handover between cells or networks, interference from other networks or electric devices, severe weather conditions, reflections from buildings and attenuation from distance or shadowing, and the added complexity of multiple antennas. The best laboratory test equipment also allows real-world network loads to be recorded and reproduced, or multiplied: if a particular usage pattern is causing problems in the network or poor handset performance then, rather than having to analyse the pattern itself the test team simply record it and feed it into their test scenario.

So, for example, a test for WiFi handover could emulate any number of operating conditions while the WiFi signal level is being raised or lowered, forcing the authorised device to handover to WiFi or back to mobile signals while the relevant QoE elements are being monitored.

What to look for in VoWiFi handover

There are two main areas to please or annoy the user:

  • How reliably the call performs as you handover between WiFi and mobile

  • Then the actual voice quality that is delivered.

The performance can be degraded by a slow handover, the setup time needed and the need to authenticate. The handover from LTE to Wi-Fi depends on whether the Wi-Fi power level is high enough to maintain a stable link – however that power level is measured by the device and there is no common standard yet. If the measurement is not accurate enough, the device may hand over to Wi-Fi, find the signal is too weak, then drop the call or keep toggling between mobile and WiFi, causing jitter and degrading QoE. The call setup time to WiFi can also be delayed if there is a mismatch between codecs – resulting in a longer negotiation to find the “lowest common denominator”. So there is need to test handover under many different conditions to make sure it is reliable, and this is only practical if the laboratory can automate its test procedures.

Testing for authentication involves the network issuing a series of challenges to test whether the device is authorized to offload to the Wi-Fi access point. It requires assessing the ability of the mobile device to keep the call active and not drop the voice call completely.

Voice quality can also be adversely affected by varying codec types between devices, making inter-codec compatibility a critical performance factor. Other degrading factors include packet loss over WiFi, and a high percentage of lost packets can cause delay between the speaker uttering a word and the listener hearing it. Delays greater than 200ms reduce QoE and varying delays cause jitter. When audio packets overlap at the receiver, speech may be choppy, garbled, or even suffer intermittent gaps.

It is best to seek advice from experienced test specialists, but certain test capabilities must be available. For example:

  • Interactive testing to flexibly adjust test parameters and settings to model network behaviour as accurately as possible for a variety of scenarios.

  • Custom test script development by programming the different states between which the network elements can automatically switch. It also helps to automate different test scenarios.

  • Creating a software-based IMS agent provides a reference model against which to compare other devices requiring audio quality performance testing. Using an agent that behaves like a device reduces the need to always have two compatible devices to perform end-to-end testing.

  • Automation – test campaigns related to conformance requirements or carrier test performance evaluation can typically span several days, so you need test automation capability for efficient use of manpower. Once recorded, tests can run automatically out of working hours or round the clock to make optimal use of the laboratory facilities.

  • Device-to-device interoperability – to ensure compatibility between codecs and better overall voice performance, you need full end-to-end testing between devices. This allows you to test across different firmware versions on the same phone or different codec implementations across different OEM vendors or different models from the same vendor, as well as detecting incompatibilities between devices when offloading to WiFi.

Keep testing

The focus has been on pre-testing before migration to VoWiFi, but the testing imperative does not stop there. Every new handset or app that is launched could lead to unexpected challenges, let alone major changes such as network upgrades. As VoWi-Fi becomes more widely deployed, additional test areas will grow in importance, such as assessing the impact of noise and fading on the radio link and the impact of traffic on network delays, jitter and packet losses. Another challenge is the variation in QoE when devices interact with access points of different capacities, like a home Wi-Fi access point versus an enterprise access point.

Test solutions need to be flexible, robust and configurable to accurately model the various interactions between the network and the device for accurate VoWi-Fi device performance evaluation. Expert advice is also needed to pull together all the many factors that can lead to customer dissatisfaction and churn.

There is no doubt that Mobile Service Operators face a challenging task in keeping their subscribers happy and ensuring first class service at all times. But there are professional services available, together with sophisticated, automated test solutions that will ease the uncertainty and enable smooth, reliable and high quality service for the customer.


Dharen-Ells-Spirent-150x150.jpgDharen Ells has more than fifteen years’ experience in the telecommunications industry dealing with both fixed-line and wireless technologies. In recent years, Dharen has been working with Service Providers, Mobile Device and Chipset Vendors to advise on best practice for testing audio quality in the areas of VoLTE and VoWiFi.

Read more about:


You May Also Like