HFSC traffic shaping for QOS on DD-WRT
The situation:
To have a backup to the backups of my computer offsite, I decided to trial BackBlaze. Their willingness to share parts of their storage architecture as well as the way encryption is employed attracted me to the company.
The software has its own integrated bandwidth management. With its default settings, BackBlaze did a pretty good job of keeping my connection responsive, but the transfer rate was nowhere near as high as it could be. I wanted to tell BackBlaze to go as fast as it could and move the bandwidth management logic to my router.
The goal is to max my upload while not completely killing the speed of downloads and latency of the connection. This can be tricky on asymmetrical connections like DSL and Cable Modems that are common here in the US.
The problems with the builtin DD-WRT (an excellent 3rd party firmware) QOS:
- HTB QOS as implemented in the firware did not seem limit latency enough. This may just be an HTB issue and not one with DD-WRT.
- There is no option to prioritize outgoing TCP ACK packets. Due to the nature of TCP/IP, your download is significantly slowed if your upload is clogged and ACK packets are not being sent out.
- The HFSC implementation is flawed and not very usable.
To make everything work, I would have to use a custom script on the router. Unfortunately, HFSC is not exactly well documented. I was lucky to be able to borrow from the work of others and adapt it to work on DD-WRT.
Thanks to these websites for collectively providing me enough information to get this accomplished:
- Infotage.net – The black art of traffic shaping provided me with links to a lot of the other sites listed here.
- HFSC Scheduling with Linux appears to be a definitive site of the theory of HFSC. I’m going to need to re-read this one in order to improve my setup.
- HFSC and VoIP – Maciej Bliziński is where I got the script that I modified for my own purposes. Most of the script’s settings are left unchanged. I will be revising why certain classes are the way they are. Since I am not using VoIP, I may be able to better optimize everything.
- QoS Linux with HFSC – voip-info.org has a script that if I paid more attention to, I would have wasted much less time. I was setting the outbound interface as $(nvram get wan_ifname) as other DD-WRT example scripts showed. This came out to eth1. I wasted a lot of time on this problem. Use ppp0 is using a DSL modem with PPPoE.
- http://wsched.sourceforge.net/examples.html provided example commands that helped me troubleshoot. The problem is that DD-WRT does not give any error messages. (Probably to save space to fit in the tiny amount of memory on these routers.)
- Packet-Shaping HOWTO provided example commands that helped me create and troubleshoot tc classes and filters.
I feel there is no point in throttling your downstream using your router. If the packet has already traveled over the last mile to your router, why reject it and cause it to be re-transmitted later. I am mainly interested in managing outgoing traffic.
Here is the modified script I am using based very, very closely upon this script. It is modified to run on DD-WRT, but will likely work on other linux-based routers.:
#!/bin/ash
# Modified to run under DD-WRT
# http://www.morph3ous.net/2009/11/15/hfsc-traffic-shaping-for-qos-on-dd-wrt
# Go to Administration and then commands
# Paste script in and click on the save firewall button
# Reboot router and test
#
###### Script originally from
# Maciej Bliziński, http://automatthias.wordpress.com/
#
# References:
# http://www.voip-info.org/wiki/view/QoS+Linux+with+HFSC
# http://www.nslu2-linux.org/wiki/HowTo/EnableTrafficShaping
# http://www.cs.cmu.edu/~hzhang/HFSC/main.html
######
# Specify the uplink as 85 or 90 percent of your actual upload speed
# Uplink and downlink speeds
# removed throttling downlink
UPLINK=425
###### From original script, but code has been disabled further below
# IP addresses of the VoIP phones,
# if none, set VOIPIPS=""
VOIPIPS=""
######
# Interactive class: SSH Terminal, DNS, RDP
INTERACTIVEPORTS="22 23 53 3389"
# VoIP telephony
#VOIPPORTS="5060:5100 10000:11000 5000:5059 8000:8016 5004 1720 1731"
VOIPPORTS=""
# WWW, jabber and IRC
BROWSINGPORTS="80 443 8080"
# The lowest priority traffic: eDonkey, Bittorrent, etc.
P2PPORTS="110 25 21 143 445 137:139 4662 4664 6881:6999"
# Device that connects you to the Internet
#DEV=$(nvram get wan_ifname)
DEV=ppp0
# clean up in case re-running
# Reset everything to a known state (cleared)
tc qdisc del dev $DEV root > /dev/null 2>&1
tc qdisc del dev $DEV ingress > /dev/null 2>&1
# Flush and delete tables
iptables -t mangle --delete POSTROUTING -o $DEV -j THESHAPER > /dev/null 2>&1
iptables -t mangle --flush THESHAPER 2> /dev/null > /dev/null
iptables -t mangle --delete-chain THESHAPER 2> /dev/null > /dev/null
# start setting up QOS
# Traffic classes:
# 1:2 Interactive (SSH, DNS, ACK, Quake)
# 1:3 Low latency (VoIP) < - currently being used for Netflix movie streaming from Roku box
# 1:4 Browsing (HTTP, HTTPs)
# 1:5 Default
# 1:6 Low priority (p2p, pop3, smtp, etc)
# add HFSC root qdisc
tc qdisc add dev $DEV root handle 1: hfsc default 5
# add main rate limit class
tc class add dev $DEV parent 1: classid 1:1 hfsc \
sc rate ${UPLINK}kbit ul rate ${UPLINK}kbit
# Interactive traffic: guarantee realtime full uplink for 50ms, then
# 5/10 of the uplink
tc class add dev $DEV parent 1:1 classid 1:2 hfsc \
rt m1 ${UPLINK}kbit d 50ms m2 $((5*$UPLINK/10))kbit \
ls m1 ${UPLINK}kbit d 50ms m2 $((7*$UPLINK/10))kbit \
ul rate ${UPLINK}kbit
# VoIP: guarantee full uplink for 200ms, then 3/10
#tc class add dev $DEV parent 1:1 classid 1:3 hfsc \
# sc m1 ${UPLINK}kbit d 200ms m2 $((3*$UPLINK/10))kbit \
# ul rate ${UPLINK}kbit
# Netflix: guarantee 1/2 uplink for 200ms, then 3/10
tc class add dev $DEV parent 1:1 classid 1:3 hfsc \
sc m1 $((1*$UPLINK/2))kbit d 200ms m2 $((3*$UPLINK/10))kbit \
ul rate ${UPLINK}kbit
# Browsing: Don't guarantee anything for the first second, then
# guarantee 1/10
tc class add dev $DEV parent 1:1 classid 1:4 hfsc \
sc m1 0 d 1s m2 $((1*$UPLINK/10))kbit \
ul rate ${UPLINK}kbit
# Default traffic: don't guarantee anything for the first two seconds,
# then guarantee 1/20
tc class add dev $DEV parent 1:1 classid 1:5 hfsc \
sc m1 0 d 2s m2 $((1*$UPLINK/20))kbit \
ul rate ${UPLINK}kbit
# Default traffic: don't guarantee anything for the first 10 seconds,
# then guarantee 1/20
tc class add dev $DEV parent 1:1 classid 1:6 hfsc \
sc m1 0 d 10s m2 $((1*${UPLINK}/20))kbit \
ul rate ${UPLINK}kbit
# add THESHAPER chain to the mangle table in iptables
iptables -t mangle --new-chain THESHAPER
iptables -t mangle --insert POSTROUTING -o $DEV -j THESHAPER
# had to change all iptables rules below to use mark instead of classify. tc filters then pick these up and move to the proper queue
# in the future this should all be done with tc filters as this is somewhat of a hack and there is presumably at least slightly more overhead
# put packets marked by iptables in the right queues using tc filters
tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 2 fw flowid 1:2
tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 3 fw flowid 1:3
tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 4 fw flowid 1:4
# none needed for 1:5 as traffic lands there by default
tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 6 fw flowid 1:6
# To speed up downloads while an upload is going on, put short ACK
# packets in the interactive class:
iptables -t mangle -A THESHAPER \
-p tcp \
-m tcp --tcp-flags FIN,SYN,RST,ACK ACK \
-m length --length :64 \
#-j CLASSIFY --set-class 1:2
-j MARK --set-mark 2
# put large (512+) icmp packets in browsing category
iptables -t mangle -A THESHAPER \
-p icmp \
-m length --length 512: \
#-j CLASSIFY --set-class 1:4
-j MARK --set-mark 4
# ICMP (ip protocol 1) in the interactive class
iptables -t mangle -A THESHAPER \
-p icmp \
-m length --length :512 \
#-j CLASSIFY --set-class 1:2
-j MARK --set-mark 2
setclassbyport() {
port=$1
CLASS=$2
iptables -t mangle -A THESHAPER -p udp --sport $port -j MARK --set-mark $CLASS
iptables -t mangle -A THESHAPER -p udp --dport $port -j MARK --set-mark $CLASS
iptables -t mangle -A THESHAPER -p tcp --sport $port -j MARK --set-mark $CLASS
iptables -t mangle -A THESHAPER -p tcp --dport $port -j MARK --set-mark $CLASS
}
for port in $INTERACTIVEPORTS; do setclassbyport $port 2; done
for port in $VOIPPORTS; do setclassbyport $port 3; done
for port in $BROWSINGPORTS; do setclassbyport $port 4; done
for port in $P2PPORTS; do setclassbyport $port 6; done
for VOIP in $VOIPIPS
do
iptables -t mangle -A THESHAPER --src $VOIP -j MARK --set-mark 3
iptables -t mangle -A THESHAPER --dst $VOIP -j MARK --set-mark 3
done
# Prioritize Netflix streaming from Roku by IP address and put into the VoIP class
# note that device's IP must either be set manually or a static DHCP reservation set on the router
iptables -t mangle -A THESHAPER -p tcp --dport 80 -s 192.168.1.15 -j MARK --set-mark 3
iptables -t mangle -A THESHAPER -p tcp --dport 443 -s 192.168.1.15 -j MARK --set-mark 3
iptables -t mangle -A THESHAPER -p udp --dport 80 -s 192.168.1.15 -j MARK --set-mark 3
iptables -t mangle -A THESHAPER -p udp --dport 443 -s 192.168.1.15 -j MARK --set-mark 3