Every version of Linux ships with a process called “tc” that is short for “Traffic Control”. This process works with the Linux kernel to determine how data packets will be placed onto the local network. It contains queuing systems and other mechanisms that control how packets are received and transmitted on the Network Interface Cards.
By default, traffic control consists of a single queue which collects packets and de-queues them as fast as the underlying hardware can accept them. This is a First In First Out mechanism (FIFO) that has been given the name “pfifo_fast”.
All network links are serial in the way that data is received and transmitted so queues are required to manage data flow.
Queues can also be used to treat types of network traffic differently. Non real time network traffic can be slowed down to allow delay sensitive network traffic to have more bandwidth. This gives the people using the network the responsiveness that they demand while still allowing the gaps to be filled in with server traffic.
There are many different types of queues available and any number of queues can be assembled to create solutions for common network challenges like the ones listed below.
– Limit the total bandwidth to a fixed amount.
– Limit the amount of bandwidth available to a specific network user.
– Maximize network throughput for a specific network protocol (Like TCP)
– Reserve bandwidth for a specific application (Like VoIP)
– Adjust latency for a specific class of network traffic.
– Ensure that a specific type of network traffic is dropped (A policer).
When traffic control has been properly configured networks become more predictable. Predictability however comes with a price and that price for the most part is complexity. Traffic control also is dependent on the other users and routers on the network. Before we proceed we should define a few components of a traffic control system.
Definitions:
Queues
A queue is a location or buffer where packets wait to be transmitted. A queue does not make a promise with respect to when a packet will be serviced and is usually configured to be a fixed size.
Flows
A flow is a set of packets that represent a communication between two network connected devices. Flows can have a protocol associated with them like TCP or UDP.
Tokens and Buckets
In order to control the flow of data onto a network one commonly used mechanism is to generate tokens at a desired rate and to then place packets onto the network only when tokens are available. When the queue is empty the tokens are placed in a bucket for future use. The size of the burst of future traffic is then controlled by the bucket size. Controlling the flow of traffic using tokens and a bucket is considered shaping.
A traffic shaper
Is a device that contains buffers and some number of queues that adjust the flow of traffic onto or between network segments.
A Policer
Is a shaper without buffers. Flow is controlled by throwing packets away.
A Classifier
A classifier is a mechanism that allows a filter to inspect a packet and key on different characteristics. A common classifier used by “tc” is the “u32 classifier” which allows a filter to identify things like a UDP packet type.
A Filter
A traffic control filter is a mechanism that glues together other system components. A filter must contain a classifier phrase and may contain policer phrase.
A “qdisc”
“qdisc” is short for “Queuing Discipline”. When ever an OS/Kernel needs to send a packet to an interface it sends it to a “qdisc”. “qdisc” is a generic term to describe the many different types of queues.
A Handle
Every class and classful qdisc requires a unique identifier within “tc”. This unique identifier is known as a handle and has two parts. The major number and a minor number can be assigned arbitrarily by the user in accordance with the following rules.
major/first number
This parameter is completely free of meaning to the kernel, however all objects in the traffic control structure with the same parent must share a major handle number. Conventional numbering schemes start at 1 for objects attached directly to the root qdisc.
minor/second number
This parameter unambiguously identifies the object as a qdisc if minor is 0. Any other value identifies the object as a class. All classes sharing a parent must have unique minor numbers.
NOTE: The special handle ffff:0 is reserved for the ingress qdisc.
The handle is used as the target in classid and flowid phrases of tc filter statements.
Queue Types
There are a number of different types of Queues that come standard with “tc” but we are only going to consider the most useful ones for TeleCollaboration. Queues can be classful or classless. The primary difference is that queues that support classes can contain multiple qdiscs. If a queue is classful, the kernel can dequeue a packet from any of the internal qdisc’s. Examples of classless queues would be pfifo_fast, red(Random Early Detection), sfq(Stochastic Fairness Queuing) and tbf(Token Bucket Filter). Examples of classful queues would include cbq(Class Based Queuing), htb(Hierarchy Token Bucket) and prio.
pfifo_fast
This is the most common type of queue and is the default queue for “tc”. It is a simple First In First Out buffer where no packet gets special treatment. This queue typically has three “bands” where “Type Of Service” bits can be used to place traffic in the bands. Band 0 packets are written out first and only when band 0 is empty will band 1 packets be written. The “pfifo_fast” queue is considered to be classless so you cannot add other qdiscs to it.
tbq (Token Bucket Filter)
This is a simple queue that uses tokens and buckets to regulate the flow of packets onto the network. Short bursts are allowed and controlled by the bucket size.
Htb (Hierarchical Token Bucket)
This is a classful queue that allows for an arbitrary number of token buckets arranged in a hierarchy. Users can define the size of the buckets, the rate of token generation and whether the queues are nested. The variable “ceil” sets the maximum bandwidth that the class is allowed to consume. (The master bucket)
For more on TC and being a good network citizen, see the white paper section of this web site.