With the advent of Time Sensitive Networking (TSN), Ethernet networking received new functionality to schedule the transmission of information. In contrast to strict priority and round robin approaches, TSN allows for the exact control of the departure time of a packet from every node. While this may sound like a minor addition, TSN has become an essential feature to allow for the coexistence of different traffic types on one common connection, and that’s true for small bandwidth links as well as high capacity links.
An analogy to public transport explains the new features. In the city where I live in Germany, the city government is determined to optimize our local public transport. Optimization means to improve timeliness, which translates into predictability for passengers without reducing the capacity for individual users. This is achieved by optimized terminals which allow changes to the sequence of buses to meet scheduled departure times and with intelligent traffic lights on the junctions between the stations. Any remaining capacity still on the streets can be filled with private transport in a best effort manner.
Traffic on public streets is a time-sensitive system. There are many parallels to Time Sensitive Networking on shared Ethernet links.
A solid transport system follows a network-wide schedule. The schedule is calculated to match the service requirements for which users pay. A schedule is maintained centrally and results in timetables for each station and terminal from where buses depart as well as for each junction. The network can be split roughly in three domains: Centralized Configuration, Terminals / (End) Stations, Connections and Bridges as shown in Figure 1.
Figure 1: Domains in a system with a network wide schedule
The list below categorizes the technical methods in TSN and the terms at the end of each bullet defines the corresponding element of IEEE 802.1Q-2018. Industrial Automation uses all these methods.
- Centralized Configuration
- Identifies traffic relations and bridges between them and calculates a schedule
à Central Network Configuration (CNC) - Defines the departure and arrival time for each service and each station
à Central User Configuration (CUC)
- Identifies traffic relations and bridges between them and calculates a schedule
- Terminals / (End) Stations
- All stations are synchronized, i.e. they work with the same time of day
à Generalized Precision Time Protocol (gPTP) - Stations know which conveyance belongs to which service and has awareness of the valid timetable for departures
à Stream Identification - The sequence of vehicles can be sorted at the station, so that their individual timetables are met
à Queues
- All stations are synchronized, i.e. they work with the same time of day
- Connections and Bridges
- Traffic lights have awareness of timetables
à Gate Control List (GCL) - The network can stop other traffic in an urgent case like emergency cars or fire brigades
à Pre-emption and Interspersing Traffic - The network knows methods and algorithms to control unplanned peaks
à Per Stream Filtering and Policing (PSFP)
- Traffic lights have awareness of timetables
The upcoming standard IEC/IEEE 60802 defines the profile for the selection of functions and their dimensioning for industrial applications. The term “TSN-IA” has been introduced for this profile. Two Conformance classes are defined: Conformance Class A (ccA) for feature-rich devices, Conformance Class B (ccB) for resource constrained devices. Both, ccA and ccB can coexist in the same network and interwork with each other.
Note: This article concentrates on features from IEEE 802.1Q-2008. TSN also includes features beyond this scope, for example Seamless Redundancy according to IEEE 802.1CB, also known as Frame Replication and Elimination (FRER) which is not discussed here.
Centralized Configuration
For a time-aware network, a common schedule is essential. The Central Network Configuration (CNC) has information about traffic types and it can retrieve information about all stations, bridges and all neighbourhoods (topology discovery). Also, travel times between network elements are known. There is exactly one active CNC per network.
Similar to a network in public transport, a CNC can calculate a schedule that allows traffic types to reach their service qualities and to use the existing network resources in an optimal way. This is a service that is executed asynchronously, i.e. before the traffic starts.
A continuous connection between the CNC and the bridges exists and allows the access to topology information and the download of updated configurations to the bridges.
The Central User Configuration (CUC) communicates with each station. It takes care to send the right service at the right time. These services can include:
- Traffic with strict deadline for arrival
- Cyclic traffic with latency bounds
- AVB traffic with bandwidth requirements
- Control traffic with strict priority and reliability requirements
- Best effort that is sent when possible, but that also may be discarded.
These services are transported in different traffic types that are listed in Figure 2
Figure 2: Different traffic types share one link and keep their aervice characteristic
The list above is simplified, but it illustrates well that each of the traffic types must be managed differently at each station. In the analogy to public transport, express buses, standard services and other individual traffic may coexist on a street, but they experience different management at the stations. In contrast to CNCs, multiple CUCs can be in a network. That is comparable to multiple carriers who share the same streets.
IEEE’s upcoming Conformance Classes play a role here: Conformance Class A (ccA) can handle eight queues to manage the different services. ccB can handle four queues. Conformance Classes define the size of the stations and the bridge. ccA has more complexity, more memory, bigger size on silicon than ccB, in consequence it can handle more different traffic types independently.
Terminals, (End) Stations
Stations are nodes that send and receive units to the connections. The term “end station” might be misleading at the first look, but it explains itself in the analogy to a transport services that can start and end at every stop.
Services follow a schedule, and that’s why each connected end station must by synchronized to the same time. The underlying protocols are basically known from IEEE 1588, and stations often act as network elements with ordinary clock or transparent clock.
A microcontroller that is embedded in the station has a prioritized service called Generalized Precision Time Protocol (gPTP). Two adjacent stations can achieve an accuracy of some 10 ns, between all stations in a network a few hundred nanoseconds should not be exceeded. Wireless networks may have larger skews.
Because stations are both a source and a destination for traffic, it is their responsibility to label the transport units correctly. This label is a tag in the Ethernet Frame (Layer 2). Most common is the use of VLAN Priorities (Priority Code Point, PCP) to identify the type of traffic. In newer proposals, more options for tagging streams are also discussed so that any pattern in the Ethernet frame can be used (IEEE 802.1CBdb). Beyond this, there are also methods to use the Layer 3 (IP packet) with IP Interception to determine the traffic type.
The common term to assign characteristics to a frame is Stream Identification. Every single frame must be scanned at every station for the tag to determine how to handle it at the egress point of the station.
Stations sort the Ethernet Frames and send scheduled traffic exactly at the right time to meet the given schedule. Up to eight parallel Queues at the egress point of a station that can contain multiple Ethernet frames allow this. Time-controlled gates let data leave the queue at the right time.
Figure 3: Queues enable to set individual departure times for each traffic type
This all must be done for every single egress packet and may result in high processing load for the station. The use of stream identification, queue management and time-controlled gates are normally in dedicated logic to offload software and increase time accuracy. Such network elements use programmable logic in FPGAs, or microcode programmable SoCs to realize this function. FPGA vendors such as Xilinx have dedicated blocks for this in their IP catalogue.
Connections and Bridges
Connections and Bridges build the infrastructure to connect stations. In the example with public transport, connections are streets and bridges are junctions where traffic from multiple sources can be merged.
While the cabling between bridges is static, bridges actively handle the traffic distribution between multiple connections. Stream identification is required there like it is in stations. At every egress port, Ethernet Frames must pass Time Controlled Gates that follow a Gate Control List (GCL) as illustrated in Figure 4. Intervals to control the gates are in the range of a few hundred microseconds to some milliseconds. Opening times for gates range from 50 µs to Milliseconds.
Figure 4: Gate Control List (GCL) for time-aware gates
A gate that is open for an unnecessary long time reduces the capacity of the network. That is why bridges normally know exactly when a scheduled frame arrives. Open gates with high time accuracy is a hardware function in modern TSN-enabled Systems on Chip (SoC), as well as some FPGAs. Often bridges are Conformance Class A, even if all connected stations are Conformance Class B.
Back to the analogy with traffic lights: Switching to “green” requires that other paths are stopped early enough to avoid collisions. A guard band ensures that there is enough time that traffic from the other direction can pass entirely before the next gate opens. A safe guard band also results in lost bandwidth.
TSN introduces Pre-emption and Interspersing Traffic to reduce this loss of throughput. Traffic with lower priority can be cut in smaller fragments, so that the guard band becomes very small. This allows traffic with higher priority to be transported, even if a long frame with lower priority is already using the connection.
This all works well if there is no overload with high priority traffic in the network. An overload may be the result of a crashed application that exceeds its traffic limits. Per Stream Filtering and Policing (PSFP) can apply metrics and filter traffic according to policies. This is a network security feature that keeps the network operational.
Conclusion
It is astonishing how well our methods from traffic management for physical routes and the practices in TSN correlate. The provisioning of paths through a scheduled network helps to use the network resources optimally while remaining capacity can be used for best effort traffic.
Time awareness is required at every station and at every bridge in the network. Because every frame must be identified and be managed according to its service to achieve the right departure time at egress ports, TSN requires determinism. TSN-enabled Systems on Chip and FPGAs can be programmed to provide the features in digital logic with low or no frame jitter, and these programmable solutions ensure that extensions can be upgraded in existing networks to keep your network running at optimal speed and efficiency.
By Michael Zapke, Industrial Marketing Lead, Industrial, Vision, Healthcare & Sciences, Xilinx
