By Usman S. Goni

The need for CDMA networks constantly to expand and evolve is putting pressure on wireless engineers to guarantee system performance. What tools and methods are available to help them monitor and maintain network quality? Usman S. Goni, LCC International Inc.

The explosive growth in wireless networks due to increased subscriber demand has put pressure on cellular and PCS operators to differentiate their service offerings in order to remain competitive. With the increased number of service providers, the wireless marketplace of today is no longer dictated to by the operator but by the subscriber who has several options in choosing for a wireless service. The wireless engineer is therefore faced with issues of network test and performance measurements in order to address customer complaints and as part of network performance monitoring to maintain quality of service (QoS).

Test procedures for RF engineering of CDMA wireless networks have evolved since the initial deployment of IS-95 based systems. In this initial phase, few test and measurement tools were available and signal strength logged via drive-tests through pre-selected routes around the service area was used to determine the system coverage performance. The system quality of service was often assessed using measured frame error rate (FER) on forward and reverse links as well as call set-up success and call drop rates.

The key requirement of an optimum test and measurement tool is a comprehensive data collection and analysis system that supports the needs of management, engineering, operations and marketing services for providers. Above and beyond this, however, an optimum test and measurement tool should provide a means of evaluating quality of service from the subscriber's perspective as well as track it over time. To remain competitive, it is essential for an operator to compare the QoS among all systems in the market to determine areas of competitive strength and to find out where improvements are required. To ensure its effectiveness, benchmarking needs to be part of a scheduled operational process since CDMA network capacity and coverage performance tends to vary as the network subscriber size changes, in addition to the dynamics of the local environment.

A known standard measurement of forward link system coverage has been to monitor the signal strength of the pilot channel as a function of the total interference density in the CDMA carrier band, popularly known as Ec/Io. Since the Ec/Io metric provides a relative measure of interference within the system, and also because the mobile uses the Ec/Io metric to lock or remain on the CDMA system, it is used to determine the extent of the system footprint on the forward path. The measurement process involves a data logging system with or without diagnostic monitor capability, and data collection is performed using the CDMA phone.

Although several system performance metrics such as Ec/Io, call drops and initiation failures could be analyzed from the collected data, the phone-based measurement process has some limitations. Firstly, the data collected is largely controlled by the phone's dependence on the network, including the limited range of PNs that the phone can scan. Thus, a scan receiver-based Ec/Io measurement that is network-independent allows the determination of key metrics for performance optimization. Using both the scan receiver and the phone will provide the key to isolating pilot pollution, as well as the capability to determine neighbor list alerts and to build an optimum neighbor list and search window sizes. Figure 1 shows a typical measurement set-up for CDMA wireless network performance evaluation and optimization.

It is not yet as straightforward to measure reverse link frame error rate (FER) without resorting to setting up a call trace at the switch for the test mobile. FER, which, by definition, refers to the number of frames received in error as compared to the total number of transmitted frames, has been the quantitative measure of voice quality on a CDMA network. The usual FER level used on a CDMA network that corresponds to an acceptable speech quality was in the range two to three percent or better. However, measurement duration that will be used to quantify the FER network performance is not an established standard, and therefore measurement results for FER are often subject to interpretation.
Significance

Clearly, the significance of this measure in the statistical sense depends on the number of frames transmitted. Thus, for measurement purposes, the interval within which information is transmitted and used to measure the quality of the network affects the accuracy and reliability of the result. However, a voice quality-based measurement technique can now be used to objectively determine the quality of not only CDMA but also any wireless network. Using an equivalent aural quality score (AQS), accurate measurement of voice quality can be obtained in real time. Thus evaluation of voice quality in this way greatly supplements FER scores to provide a true evaluation of subscribers' perception of the network call quality.

In addition, power control transmit gain adjust provides another dimension to proper evaluation of reverse link system performance. The gain adjust parameter is derived via post-processing of logged phone-based measurement data, using a comprehensive analysis and parsing tool such as OptimEyes, shown in Figure 2. The more positive the gain adjust is, the greater the possibility of reverse link noise rise and hence the more power required for the mobile to ensure the attainment of target Eb/No at the best serving sector.

Quality benchmarking is another important part of the test and measurement process. In a competitive environment where several wireless services are offered, a network operator may need to evaluate his/her network to determine the need for increased service performance, in order to establish a stronger subscriber base. This is the process typically described as quality benchmarking.

The test and measurement tools employed for benchmarking are designed to track changes in the network performance as perceived by the subscriber, and hence to provide a more objective approach to quantifying system quality performance. All the wireless networks in the market are evaluated by collecting measurement data via drive testing using test phones registered on all the networks.

Congestion period
Before embarking on the tests, system-wide drive routes that were used for system-wide optimization of the network, or other new routes, are selected. The period of the measurements should span through the system busy hour to allow the collection of data that represents the network congestion period.

Other measurement tools that are employed for these tests include audio quality assessment modules, scan receivers, specialized data collection software and computer-based data logging systems. A relative assessment of the wireless network performance for all the operators in the same market can then be compared on the basis of subscribers' perceived quality. In this way, a more comprehensive method is provided for operators to measure their performance and determine areas of improvement.

In addition to the transparency of the benchmarking technique to the air interface technology of the various operators' networks, the benchmarking process is for the first time, the optimal approach in determining the balance between the forward and reverse links of a CDMA wireless network [1]. It is well known that the air interface specifications for IS-95 forward and reverse links are entirely different. Thus, network performance engineers often analyze CDMA forward and reverse link balance by comparing outages in specific geographic bins. For a given bin, link imbalance is said to occur when either the forward or reverse links is in outage. However, the accuracy of this measurement process depends on the size of the bin and the data reduction process used. Hence, a more accurate performance link in a CDMA network could be determined through simultaneous evaluation of reverse and forward link AQS.

Next, we turn to system performance engineering. Traffic measurements and reporting are an essential part of CDMA network performance engineering. The capacity performance of the network is usually determined from a detailed analysis of traffic usage on each sector over a period of time, usually during the peak traffic hour.

Since coverage of the system is impacted by increased usage beyond a given nominal design load, traffic carried by sectors (or cells) is usually monitored to determine optimum loading levels beyond which enhancements to the capacity are incorporated to maintain quality.

The measurement tools required are usually part of the vendors' offering, which comes as a suite of features associated with the MSC or as a direct interface to the OMC. However, there are network performance management tools not tied to one particular vendor-type infrastructure, that could provide service measurements, performance analysis and reporting. These network traffic measurement tools are used to automatically collect and monitor valuable network performance statistics, analyze key network performance indicators, identify current and potential performance problems, investigate problem causes, and forecast network growth to aid in optimal planning.

A key metric for CDMA network planners is determining channel elements overhead due to soft handoff. Although a key feature of CDMA, soft handoff overhead becomes a major constraint to forward link capacity, especially for sites in dense urban core areas, owing to excessive cell overlaps caused by the need to provide greater in-building signal penetration. A network performance measurement tool can be configured to determine sectors with excessive channel element overhead and hence determine corrective actions to preclude capacity problems.

In the early days, CDMA network performance engineering relied on deriving raw traffic data and performing detailed spreadsheet-based traffic analysis, trending and growth planning. It is inefficient to manually monitor such network problems as congestion, drops on the air interface, and fixed network utilization, on a daily basis. Automated performance measurement will allow network operation engineers and planners to forecast medium-to-long term site and network growth, as well as evaluate usage patterns and enhance overall network-wide performance.
Maintaining quality

As CDMA networks worldwide continue to mature, supporting an estimated subscriber base of about 23 million, tools for system performance tests and measurements will continue to evolve to meet the challenges of maintaining network quality. The demand for faster system rollout coupled with the need for advanced service features in a rapidly expanding wireless market has put greater pressure on CDMA wireless network operators to seek quick and automated response solutions to network problems. To fulfil these needs, test and measurement tools with intelligent network diagnostics capabilities are required. Test and measurement equipment manufacturers are therefore faced with the tasks of improving on the earlier version of their CDMA software and hardware products, as well as continuing to innovate and develop new products offerings to meet these demands.

It is anticipated that future CDMA based wireless networks supporting high-speed data and multi-media services would require more intelligent network test and measurement tools. One major challenge would be to monitor and track traffic pattern and variations for the different 3G services to be offered. Test and measurement products for such services are currently being conceptualized.

Usman S. Goni, Ph.D., is a Senior Principal Engineer with Engineering Services Group LCC International Inc., a leader in wireless communications services, test and measurement products.

Email: usman_goni@lcc.com

[1] Nandu Gopalakrishnan, CDMA Voice Quality-Based Optimization, Wireless Business Technology, January 1998