With the development of 5G applications, there is a need for 5G IP networks to shift from transporting consumer-oriented eMMB services to transporting integrated services that focus on industry-oriented uRLLC deterministic services, so as to support IT and OT integration for 5G-driven industrial Internet. At the same time, 5G service cloudification and function virtualization bring extensive and differentiated deterministic transport scenarios and requirements, which pose challenges on large-scale IP networks. To meet the diversified deterministic transport requirements, large-scale IP networks first need to have an advanced network architecture.
Deterministic IP Network Overview
Deterministic Networking (DetNet) refers to the technology that provides determinism in the IP network. It has three goals:
—Minimum and maximum end-to-end latency from source to destination, and bounded jitter (packet delay variation).
—Low packet loss ratios, such as zero congestion loss.
—An upper bound on out-of-order packet delivery.
To achieve the above objectives, multiple techniques need to be used in combination, including explicit routes, service protection and resource allocation defined in DetNet. Resource allocation operates by isolating, reserving and guaranteeing resources. Explicit routes control forwarding paths to be not affected by topology events. Service protection further provides guarantees for packet loss ratio and out-of-order delivery in cases of link and node failure.
A deterministic IP network requires a completely redesigned architecture that includes network resource layer management, deterministic IP routing layer management, and deterministic service layer management. A typical deterministic IP network is shown in Fig. 1.
From the perspective of deterministic service requirements, deterministic QoS in the network can be divided into five levels:
—Level-1: bandwidth guarantee.
It includes basic bandwidth guarantee and a certain degree of tolerance to packet loss. There is no requirement for the latency and jitter. Typical services include FTP downloads.
—Level-2: jitter guarantee.
It includes a jitter of less than 50 ms, a delay of less than 300 ms. Typical services include synchronous voice services, such as voice call.
—Level-3: latency guarantee.
It includes a delay of less than 50 ms, a jitter of less than 50 ms. Typical services include real-time communication services, such as video, production monitoring.
—Level-4: low latency and low jitter guarantee.
It includes a delay of less than 20 ms, a jitter of less than 5 ms. Typical services include video interaction services, such as AR/VR, holographic communication, cloud video and cloud games.
—Level-5: ultra-low latency and jitter guarantee.
It includes a delay of less than 10 ms, a jitter of less than 100 us. Typical services include production control services, such as power protection and remote control.
It is ideal to use appropriate deterministic transport resources and technologies to precisely meet the deterministic service requirements classified as above. Moreover, when a large-scale IP network needs to provide the integrated transport of substantial diversified deterministic services, an advanced network architecture is needed to support its continuous evolution.
Large-Scale Deterministic IP Network Architecture
For a large-scale IP network to support determinism, the following issues need to be considered. A large-scale IP network has a large number of hops and high link delay, which makes it difficult to achieve network-wide precise time synchronization, and it cannot maintain the state of each traffic flow when the traffic is differentiated, dynamic and concurrent. It may span across multiple domains with heterogeneous forwarding planes. A large-scale IP network has various deterministic service requirements, such as those from 5G uRLLC services. The current time-sensitive networking (TSN) technology is mainly applied to L2 Ethernet, and cannot be applied to IP network. The deterministic architecture defined for IP networks faces various challenges in precisely guaranteeing the deterministic end-to-end transport.
Network 5.0 technology, defined by China's Network 5.0 Industry and Technology Innovation Alliance, can address the above problems. In it, a large-scale deterministic IP network can provide determinism in the resource layer, routing layer and service layer respectively to establish a unified large-scale deterministic IP network architecture (Fig. 2). The resource layer maintains the deterministic resources of the entire network, and model the deterministic resources to form deterministic links. The routing layer computes deterministic routes based on the deterministic links to provide deterministic transport capabilities. The service layer maps flows to deterministic routes according to the specified traffic characteristics and performs traffic monitoring and shaping.
Differentiated deterministic services require the networks to provide different deterministic capabilities. The resources related to deterministic capabilities are also differentiated. Deterministic resources refer to the resources that meet the deterministic indicators of a node and link processing (such as link bandwidth, queues, and scheduling algorithms). It is necessary to make unified modeling for heterogeneous deterministic resources to form unified deterministic links with different levels. A deterministic link can be a sub-network that provides deterministic transmission or a point-to-point (P2P) link. When the existing resources in a network are insufficient to meet the SLA requirements, a virtual network needs to be created.
Traditional routes only guarantee reachability and bandwidth. The paths change with the topology events. They cannot meet the deterministic requirements. To meet the requirements of different types and levels of deterministic services, it is necessary to create deterministic routes with different SLA levels based on the deterministic links.
Deterministic routes can be based on a strict or loose explicit path. The former is applicable to centralized scenarios with one or more controllers, and the latter is applicable to distributed scenarios without controllers. In the centralized scenario, one or several controllers may compute one or more paths with deterministic SLAs in advance according to the typical traffic specification (T-SPEC) based on the collected deterministic resources, or compute dynamically according to the T-SPEC as required by the services. It is suggested to generate two non-intersecting paths with very close delay to achieve 1+1 protection, and to perform replication on the ingress PE and elimination on the egress PE. In the distributed scenario, deterministic loose routes are computed through routing protocols. IGP is used to compute deterministic routes based on deterministic delay metric intra-domain, and BGP is used to compute deterministic routes based on accurate credit of delay/jitter across domains.
Deterministic QoS includes delay and jitter determinism. The delay and jitter indicators can be considered together or separately. It is necessary to implement admission control and traffic policing at the ingress PE node based on the SLAs of deterministic service flows, and map the service flows to deterministic routes to achieve the final goal of deterministic QoS.
Deterministic QoS means that the end-to-end delay/jitter of flows with T-SPEC in the network will be strictly limited within a bounded range on the basis of deterministic resources and routes. As different service levels have different requirements for delay and jitter, the resources and routing mechanisms used for mapping service flows to deterministic routes are also different. For example, the extremely low delay and jitter can be guaranteed by L1 channel, so as to avoid the excessive intra-node delay contributed by too many hops of intermediate nodes at L3. Or in the customized virtual network, the bounded delay and jitter can be guaranteed by forwarding along the paths composed of links based on the ATS or CQF scheduling algorithm. Once the traffic is monitored and shaped on the ingress PE, the intermediate node does not need to maintain the QoS state; however, different scheduling algorithms have different shaping effects.
Deterministic 5G transport networks will gradually extend from LAN deterministic networks to MAN and WAN deterministic networks. The large-scale deterministic IP network architecture can meet the evolution requirements of this development trend for IP networks, and allows intelligent 5G IP networks to implement unified transport of emerging, differentiated and deterministic services in the future.