Effective Performance Metrics for Multimedia Mission-critical Communication Systems

Received: 6th January 2021; Accepted: 11th February 2021; Published: 1st April 2021 Abstract: Mission-critical Communication Systems that are adaptable for use with the latest generation of multimedia services are crucial for system users. To determine the set of requirements that need to be hardcoded into such systems, a clear distinction between mission-critical and non-mission-critical systems is required. Moreover, the users of services provided by such systems are very different to those of current mobile commercial communication systems. These differences give rise to a set of challenges that need addressing to facilitate migration from existing systems to those now being proposed. One such challenge relates to the performance of the IP Multimedia Subsystem (IMS) registration process. This is a crucial consideration for mission-critical systems, particularly in large-scale systems where thousands or even millions of users may seek to access the system in disaster scenarios. This paper presents an evaluation of IMS and Session Initiation Protocol (SIP) performance metrics and Key Performance Indicators (KPIs). Moreover, it articulates a proposed study that will seek to address some of the challenges identified.


Introduction
'Mission Critical' is defined by TETRA and The Critical Communication Association (TCCA) as a function whose failure would lead to catastrophic consequences that would place public order or public security at risk. A system that provides such critical functionality must have suitable inbuilt functionality, interoperability, security, and the wherewithal to maintain its availability. Missioncritical users are those with responsibility for the welfare, health, security and safety of the public. Law enforcement forces, firefighters, emergency and medical services, rescue services, military forces, utility staff members, and transport services members, are all example of mission-critical users. The concept of a Mission-critical Communication Systems (MCCS) refers to the hardware, software and communication facilities, that allow mission-critical users to communicate with each other and liaise with command centres securely and dependably for the sake of providing missioncritical services, wherever and whenever the services require special communication solutions.
Many governmental and non-governmental organisations (NGOs) play critical roles in the operation of mission-critical systems. The services provided by such organisation are of a type that makes them less tolerant to execution or operational errors. Such services are referred to as missioncritical services. The criticality of such services implies that they have a set of special requirements that distinguish them from other services. They should be available anytime, anywhere, within the registration process successfully. SRD is the time required to get a reply from the server-side regarding the requested call setup from the user side; it is counted for both successful and unsuccessful call requests. If the call requested were successfully set then the call setup time would simply be the SSD plus the acknowledgement sending time. The SDD is the time difference between sending the BYE message from the user side and the time of receiving 200OK confirmation from the server. In this paper, the call setup delay is used to measure the system performance due to its importance in real-time multimedia services in general and in mission-critical communications specifically. The QoE for SIP-based systems is affected significantly by the call setup value. The mean time of call setup values can reach up to 800 ms [6] ). However, for LTE-based mission-critical systems (designed to work as a replica for traditional dedicated mission-critical systems such as TETRA), the average call setup time needs to be within 500 ms delay. Figure 2 shows the average access time of different technologies. The call setup time is usually a multiple of the access time due to the enforced handshaking process.

IP Multimedia Subsystem (IMS)
Legacy mobile communication networks and the next generation systems can provide end-users with new sets of applications and services. There is a need for a system that sits on top of the access technology domain to provide the required signalling for multimedia services. IP Multimedia Subsystems (IMS) form the core part of Next Generation Networks (NGNs). They play an important role in helping the service operators merge the multimedia services in cellular networks and to provide end-users with key features and services within Quality of Service (QoS) levels set by operators. These multimedia services such as messaging, instant voice, video conferencing, group management, and push services, all rely on IMS to control the signalling and the state of the service before initiation, during the service and after service termination. IMS can be integrated in nonmultimedia based systems as well [7]. IMS was designed and standardised by 3GPP to provide multimedia services over mobile communication technologies beyond GSM [8]. The IMS is used for delivering the IP multimedia services between users and service providers; it gains its importance as an architectural framework for multimedia communications, it can also be integrated in other frameworks for robot messaging and logging [9]. The SIP is the principal signalling protocol used within an IMS to create, modify and destroy multimedia sessions. As defined by the standard, an IMS functions as an interface between the service/application layer and the transport layer, enabling the service providers and operator to control the user QoS based on its subscription profile. Moreover, it works as a hub point for the entire SIP signalling that needs to take place before, during, and after the call. To ensure overall integrity, the different control functions are connected through a set of interfaces.
Users' subscription-related information is stored in the Home Subscriber Server (HSS) that performs authentication and authorisation functions for users. The HSS is also responsible for updating user registration status records. The Call Session Control Functions (CSCF) are responsible for handling the SIP signalling messages and packets in the IMS. The Proxy CSCF (P-CSCF) is the entry point into the IMS system, and all SIP messages flow through the P-CSCF. The P-CSCF may also apply security or compression algorithms to the received traffic in addition to providing quality of service control and bandwidth management. The interrogating CSCF (I-CSCF) is one of the main elements of the IMS systems. It is used during the registration process when the UE does not know which Serving CSCF (S-CSCF) should receive the request. The I-CSCF interrogates the HSS to obtain the address of the appropriate S-CSCF that should process the request.
The S-CSCF performs session control functions that interface with the HSS to check and download the user profile information. It assigns the Application Server (AS) for the user for further services and enforces the operator policy control. The SIP AS has a SIP interface with the S-CSCF and is used to host specific IMS services. After the registration process is completed and the S-CSCF is allocated to the UE, the I-CSCF is no longer used for any further communication. All future communication happens between the UE, the P-CSCF, and the S-CSCF.
In the next sections, the performance of IMS, which is critical in affecting the overall system performance due to its hierarchal position in the stack and core functional role, is discussed.

IMS Performance Issues
After migrating from the circuit-switched Second Generation Mobile Networks (2G) toward the packet-switched domain of fourth-generation (4G) communication networks and beyond, the need for supporting multimedia services in all IP network infrastructure is essential to ensure the convergence of data, multimedia services, and mobile networks technologies. IP Multimedia Subsystems (IMS) developed by the Third Generation Partnership Project are a key part of Next Generation Networks (NGN) that are responsible for providing and controlling the multimedia services in the packet-switched domains. As defined by the standard, the IMS operates as an interface between the service/application layer and the transport layer that enables the service providers and operator to control the user QoS based on its subscription profile. Moreover, it works as a hub point for the entire SIP signalling that needs to take place before, during, and after the call. For this purpose, there are different functions that are connected by interfaces to ensure operational integrity.
In the protocol stack, the IMS resides between the application layer and the transport layer (see figure 3). Therefore, the performance of the IMS affect mission-critical applications that have strict requirements due to their special operating nature and associated tasks. End-end delay of the access technology components along with the IMS entities affect the end-to-end signalling, and QoE offered to the end-user. Having a generic system that has IMS core network serving both legacy mobile users www.aetic.theiaer.org and mission-critical system users will introduce overload, especially at the IMS side. Statistical analysis of the IMS queues helps in measuring the system performance [10]. The performance of the IMS is affected by the individual performance of each entity inside it. The P-CSCF is the first entity that interacts with the User Element (UE) and forwards the SIP messages to other control functions in the IMS. It is also used to apply security or compression algorithms to the received traffic in addition to monitoring quality of service and bandwidth management. The I-CSCF is a SIP server that is assigned by the HSS to the user when the I-CSCF requires it. The S-CSCF has interfaces with the HSS to perform session control, download the user profile information, assign the Application Server (AS) for further services, and to enforce the operator policy control. The SIP AS has a SIP interface with the S-CSCF, and is used to host specific IMS services. Some gateways function as interfaces with other domains.

IMS Registration Performance
In the IMS registration, the user requests authorisation to gain access to the IMS services. Figure  4 shows the registration steps in IMS. First, the UE sends a SIP Register request to the P-CSCF, which forwards it to the I-CSCF. Then the I-CSCF sends a User Authentication Request (UAR) to the Home Subscriber Station (HSS), using the Diameter protocol, to check the user profile for authorisation process and to determine the S-CSCF address allocated to the user. Then, the HSS replies with the User Authentication Answer (UAA) to the I-CSCF and authorises the user. After retrieving the S-CSCF details from the HSS, the I-CSCF forwards the SIP Registration request to the S-CSCF, which in turn sends a Multimedia Authentication Request (MAR) to download user authentication data to the HSS. The HSS then replies with the Multimedia Authentication Answer (MAA) to the S-CSCF, which in turn responds to the user using a SIP 401 Unauthorised response that embed an authentication challenge within it for the UE. The UE then generates another SIP Register request following the same steps as described earlier. This time the authentication process finishes and the S-CSCF sends a Server Assignment Request (SAR) to the HSS, using the Diameter protocol, which replies with a Server Assignment Answer (SAA) using the same protocol. Finally, the S-CSCF sends a SIP 200OK message to the UE to complete the registration process.
As mentioned before, the Registration Request Delay (RRD) is the time needed for the registration process to complete. In this case, that is the time difference between the first SIP Register message sent by the UE and the time the SIP 200 OK message is received. Figure 5 shows the transition diagram of the IMS Registration process. www.aetic.theiaer.org During the registration process, the end-user needs to be subscribed to the IMS to be authorised to make calls and initiate other services instantly during the call. The registration process is required before starting calls (see figure 6) and reregistration is needed for the SIP device to re-authenticate itself after receiving a notification from the SIP server. This is required to avoid compromising the SIP device.
The delay in the registration process is one of the SIP performance metrics and criteria that was defined by IETF in RFC 6076 [4], due to the previous lack of a SIP benchmarking criteria to define the baseline performance of SIP signalling. RFC 6076 defines the performance metrics for the SIP in VoIP applications in order to provide key performance indicators and Service Level Agreement (SLA) indicators for best network resources utilisation and best end-user Quality of Experience (QoE). The main metrics defined in RFC 6076 are the Registration Request Delay (RRD), Ineffective Registration Attempts (IRA), Session Request Delay (SRD), and Session Disconnect Delay (SDD). RRD is the time needed for the user to finish the registration process successfully and plays a major role in the overall QoS.
www.aetic.theiaer.org In addition to the delay in the registration process, the rate at which registration messages are sent to the server is of great importance. This is especially true in mission-critical operations where the number of users or devices that request access to the SIP/IMS server may be high enough to affect or even deteriorates the performance of the service. Different real-world scenarios for registering users/devices over LTE have been presented by other researchers. For example, in one scenario, an Airbus A380 carrying a thousand passengers was simulated after landing at Heathrow airport, with each passenger initiating IMS Registrations over the LTE at a rate of ten registrations per second. In the second scenario, a regional power outage was restored at a big city causing a flurry of IMS registration requests to the network, a hundred registrations per second was expected in this scenario [11]. In both scenarios, it was important to handle all the requests successfully and finish the registration process within acceptable time limits regardless of the received traffic. This successful registration is especially important in mission-critical system that by their nature, should be always available, resilient and reliable.
PoC is one of the mission-critical services supported by IMS. It is designed to operate as a replica of Push To Talk (PTT) service in conventional dedicated mission-critical communication systems such as TErristrial Trunked RAdio (TETRA) networks and P25. Therefore, the investigation of the IMS added delay in the registration process is of great importance for mission-critical services which are less delay tolerant, especially if operating over a generic communication system that is not dedicated to mission-critical communications.
Some studies that analysed IMS registration over WiMax and 3G interworking architectures show that the access technology along with IMS design affect the registration signalling delay [12]. In one study, a performance analysis of IMS signaling, including registration signalling, in multimedia networks was carried out. The study showed that the maximum signalling delay is affected by both the number of User Elements (UEs) and the available bandwidth available in the core network. However, the study was limited to a maximum of 400 users [13]. Other researchers have proposed a general IMS registration protocol for wireless networks in order to reduce the registration delay by combining both the registration processes with the access technology and IMS network [14]. A reregistration procedure modification was proposed in order to reduce the transmission delay in the IMS [15]. Reregistration is needed either in order to refresh the registration state of the user or to update the user registration profile due to change in user capabilities. Moreover, when the impact of SIP signalling load over IMS was investigated, it was found that the SIP signalling load, including registration, has a significant impact on the Key Performance Indicators (KPI) for IMS [16].
This paper investigates the delay in the registration process introduced by the IMS for increasing numbers of UEs. Moreover, it also articulates the effect of increasing the load over the KPI and related QoE.

SIP Key Performance Indicators
SIP key performance indicators are important to measure end-to-end and core network performance, Software-Defined Networking (SDN) and Network Function Virtualization (NFV) rely www.aetic.theiaer.org is the ratio of failed session setup requests and the total number of session requests. Finally, the Session Completion Ratio (SCR) is the ratio of successful completed sessions and the total number of sessions. This paper considers both the RRD and IRA as KPI metrics that reflect the performance of the SIP Registration process.

IMS Key Performance Indicators
The criteria for carrying out IMS performance evaluation using test-beds has been defined by the European Telecommunications Standards Institute [18]. Moreover, 3GPP in their technical standard [19] have defined a set of KPIs for IMS that can be classified in to three categories: A) Accessibility KPIs: which are a set of metrics that reflect the accessibility of the IMS for the users. They are: 1) Initial Registration Success Rate (IRSR) of S-CSCF: reflects the accessibility performance provided by the IMS and measures the percentage of attempted user initial registrations (UR.AttInitReg) that were successful (UR.SuccInitReg). The percentage is calculated as follows: 2) Mean Session Setup Time: measures the meantime for session setup starting from when the SIP Invite message is sent to the P-CSCF until the reception of the 200OK.
3) Session Establishment Success Rate: is calculated by the I-CSCF using two values:-the ratio of successful session initiation requests to the total number of session initiation attempts; the ratio of successful session termination attempts compared to the total session termination attempts.
The type notation in the nominator stands for SIP 180 and SIP 200OK message types.

4) Third-Party Registration Success
Rate: this KPI is obtained by dividing the number of successful third party registration procedures by the attempted third party registration procedures.  * 100% The QoE, which measures the quality of the service as perceived by end-users, is affected by the KPI values of both the SIP and IMS. In order to improve the QoE, the corresponding KPI values need to be continuously monitored and improved. The KPI measure, if combined together, will help in determining the failure points in the IMS network and be indicative of how to improve things so as to avoid the bottleneck and single point failure in case of traffic overload scenarios.
The KPI may also be used as a Load Detection Function (LDF). The LDF may be used to support load balancing for the S-CSCF and, therefore, reduce the service request latency.

Voice Applications
There are different types of applications and data formats used in mission-critical communication system. These include voice, video, and push messages, IP-based voice and video applications over SIP is widely used and deployed in many systems [20]. Some of these applications are termed "killer applications" due to their bandwidth requirements, especially at the core and access point side, which are considered bottlenecks in the overall system. Another application, Push To Talk (PTT) enables the caller to push a button to initiate the sending of a voice burst to the callee side. This particular application uses half-duplex channels to save bandwidth and minimise the need for extra bandwidth. There is a need for addressing users scalability concerns that may emerge due to twoway or full-duplex communication methods which will be a challenge for future broadband mobile communication systems if PTT over Cellular (PoC) are adopted in the future. However the adoption of PoC introduced additional delays especially when applied over LTE which experience more delays that need to be analysed [21]. Voice over IP (VoIP) is used for the delivery of voice bursts over IP networks. Due to the emergence of All-IP-Networks, VoIP has gained significant importance as the best voice streaming solution for voice calls. Voice Codec is used at the subscriber side to convert the analogue voice waves to digital pulses and vice versa. There are different codec types based on the selected sampling rate, data rate, and implemented compression algorithm. In order to determine the bandwidth requirements of VoIP connections, there is a need to identify common VoIP Codecs and their www.aetic.theiaer.org associated characteristics (table 1 shows the Bandwidth Requirements of different Codecs [22]. VoIP services are considered bandwidth-killing applications for mobile broadband Access Technologies that provide end-users with services similar to those of DSL providers in the traditional wired networks. However, the wireless access mechanism introduces additional challenges and constraints on the data rate as well as delay requirements. Therefore, a bandwidth requirements study for VoIP was needed to ensure that the mission-critical service that use VoIP operate as expected and to determine how best to increase the scalability of the networks within acceptable QoS levels. A comparison between the real the theoretical codec values is important indication of the quality of the call and network performance [23].

Multimedia Services in Mission-Critical Systems Challenges
Based on the earlier discussion, there are two different deployment options for the missioncritical systems, either to have a dedicated system or a commercial system. Unfortunately, there are drawbacks for both options and many trade-offs that control the design and deployment process. What is important to note is that at the nascency of dedicated mission-critical systems, voice and simple text messages were the only services used by PPDR for their critical missions. Simply during the early stages of the evolution of mobile communication networks, operators only thought of providing voice communication between users. Hence, when dedicated mission-critical communications were deployed, the standardisation bodies designed the whole system only for voice and low data rate services. However, due to the exponential increase of mission-critical data used to support different services and tasks, the need for broadband data along with the traditional basic voice service has become vital for the mission-critical communication systems.
Due to the increased demands for more bandwidth, the need for an access technology that is capable of providing broadband connectivity and scalability, along with broadband network support in the core network, is of crucial importance for any mission-critical communication system.

Access Technology Challenges
As discussed earlier, there is an increasing demand for more bandwidth for mission-critical services. This implies that systems not only need to satisfy the general requirements of missioncritical services, but also a more scalable system to serve more users. Moreover, the users' broadband services should be running as it should be during crises and worst-case scenarios.
Increasing the bandwidth of the system is not a simple task in the physical layer of any communication system especially as there are multiple factors and trade-offs that complicate the evolution process of any communications system. From a business perspective, mobile network operators need to utilise every Hertz of the invested licensed frequency band to help ensure maximum profit. Technically speaking, the frequency spectrum should be utilised so to achieve the highest possible bandwidth to support the broadband services. Hence, the need to deploy methods for the best frequency exploitations such as: There are trade-offs between the frequency, transmission power, coverage area, and cell size. In addition, there is a trade-off between the frequency and maximum system throughput based on the Bit Error Rate value. Finally, the system throughput controls the downlink and uplink capacity of the resource blocks, which limits the maximum number of simultaneous users and the scalability of the whole system.
Current operational 4G mobile communications networks are widely deployed using the Long Term Evolution (LTE) standard articulated by 3GPP. While the LTE standard can be part of a public safety system, it is important to note that previously existing trade-offs still cause issues within the access technology abstract. However, the LTE or other technologies dedicated technologies can provide reasonable solutions in the physical and MAC layers to maximise the capacity of the system.

System Architecture Design Challenge
Mission-critical communication systems are composed of different entities and interfaces to ensure end-to-end connectivity and overall system integrity. End-end users may be connected to different network domains and access technologies, hence the need for interfaces and gateways that provide transparent connectivity between the different domains. In addition, there is a need for standardised interfaces that connect devices regardless of manufacturer. Finally, there is a need for data plane and control plane protocols that ensure the connectivity and data delivery between endusers. Figure 7 shows the general layered architecture model. Regardless of the details of any missioncritical communication system, all systems share the same hierarchal architectural structure. Endusers are the ones holding the mobile device that is running the applications that interact with the required services through the available access technology. The access technology used to reach the core network can be either a single predetermined access technology or multiple access technologies. Clearly, having more than one supported access technology to access the core network will support the reliability requirement of requested mission-critical service.
Regardless of the access technology used, the need to have a common platform working between the access technology and the mission-critical application servers is a necessity for two reasons. Firstly, the platform needs to work as intermediary between the control signalling and data streaming signalling domains. Secondly, the platform is needed to provide seamless connectivity for the endusers along with the application servers regardless of the access technology being used. Moreover, it should be noted that there is no clear distinction between the data domain and control domain signalling protocols.

End-to-End QoS Challenge
Having different abstracts, in a hierarchal manner for the architecture of the system as described previously, will introduce not only integration challenges but also cross-layer service performance optimisation challenges. Such structure is important to improve the quality of service for the whole end-to-end mission-critical communication system and to satisfy the mission-critical requirements mentioned previously.

System Performance Modelling and Validation Challenge
Modelling a system and simulating its behaviour, while changing a set of dynamic parameters, is useful in providing an understanding of its overall functionality and responses. Verification and validation are essential parts of the readability and availability testing process.
The verification process of a mission-critical system starts with a model design (or framework) analysis. The system design needs to be checked against all specifications before the system is implemented and built. For a newly proposed system, the technical reports and the standardisation bodies in the field are responsible for setting the guidelines and set of specifications need to be followed by system engineers. This helps ensure a more realistic and accurate model. The lack of finalised standards imposes many challenges in the verification and validation steps.

Proposed Solutions
The challenges outlined in the previous section highlight the need for a further study to focus on the cross-layer communications between different system abstracts. The result of this study should help ensure that calls, such as mission-critical emergency calls, with higher priority class, pass through the different entities of the system toward the end-user while at the same time satisfying all mission-critical service requirements. In addition, there is a need to develop a pre-emptive scheduling algorithm capable of deciding which type of calls need to be dropped in order to free up allocated resources at times when system capacity is being exceeded. To facilitate the development of such a pre-emptive scheduling algorithm, a thorough evaluation of overall system capacity with different scenarios that combine a number of users along with the priority class of the service will need to be undertaken.
The proposed study will need to develop a framework for the mission-critical communication system. Moreover, the different layers and interfaces for both data plane and control plane need to be demonstrated. Furthermore, the layered architecture model of the system and the details for each layer need to be investigated and clarified.
A cross-layer optimisation and integration of the signalling used in both control and data domains need to be proposed as part of a general framework. Finally, in order to test the proposed framework and the overall proposed system architecture in addition to validating the proposed cross layer optimisation, the whole system needs to be modeled using simulation tools. The Quality of Service performance measures need to be compared with the performance metrics of a test-bed for the mission-critical system that will be used as a benchmark.
One of the main objectives for this study is to propose a cross layer optimisation solutions between both the interfaces and protocols running in the mission-critical system to help facilitate the enhancement of the overall performance and its associated metrics.

Conclusions
This paper articulates some of the many challenges that need to be addressed with respect to current commercial mobile communication systems. The main goal is migrating to a generic system that is better in terms of performance than current dedicated mission-critical communication systems and, at the same time, more scalable to serve increasing number of commercial mobile users. The performance of SIP and IMS, as a core part of the generic proposed system, has significant implication on the system's overall performance. The registration process, as an example, is considered a good www.aetic.theiaer.org AETiC 2021, Vol. 5, No. 2 13 way to measure the system response time as most of the system entities are involved during the registration process. Similar to the registration, there are a set of KPIs for both the IMS and SIP that can be considered as a reference points to measure the system performance at different layers and abstracts of the end-to-end system. The challenges and requirements that need to be addressed satisfied in any proposed alternative have also been articulated.