Iptables basics

Yesterday I tweeted “hey, I learned some stuff about iptables today”! A few people replied “oh no, I’m sorry”. iptables has kind of a reputation for being hard to understand (and I’ve also found it intimidating) so I wanted to write down a few things I learned about iptables in the last few days. I don’t like being scared of things and understanding a few of the basics of iptables seems like it shouldn’t be scary!

I have been looking at Kubernetes things, and Kubernetes creates 5 bajillion iptables rules, so it has been time to learn a little bit about iptables.

The best references I’ve found for understanding iptables so far have been:

the iptables man page
iptables.info (now seemingly at https://www.frozentux.net/iptables-tutorial/iptables-tutorial.html) (which is GREAT, it explains all kinds of stuff like “what does MASQUERADE even mean” that is not explained in the iptables man page)

how to view what iptables stuff you have set up

iptables has a bunch of er, “tables” in it. These are places you can put iptables rules. They’re used at different times during packet processing. There’s a diagram here, from this “traversing of tables and chains” page.

The first surprising thing I learned about iptables is that to look at all the iptables rules you have to run 4 commands

sudo iptables -L # there's an implicit `-t filter` here,
                 # this just lists the filter table
sudo iptables -L -t nat
sudo iptables -L -t mangle
sudo iptables -L -t raw

This isn’t super fun, I find it annoying to have to run 4 commands to see all the iptables rules on my computer. I started running sudo iptables-save which generates a dump of all the iptables rules I have. I like being able to see everything by running one command!

$ sudo iptables-save
*nat
:PREROUTING ACCEPT [1664:324261]
:INPUT ACCEPT [1629:320166]
:OUTPUT ACCEPT [36545:10406977]
:POSTROUTING ACCEPT [34390:10034797]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
COMMIT
# Completed on Wed Jun  7 21:25:14 2017
# Generated by iptables-save v1.6.0 on Wed Jun  7 21:25:14 2017
*filter
:INPUT ACCEPT [2078627:2180604942]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1617291:254682158]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j
ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION -j RETURN
COMMIT

You can see on my laptop right now that I have a bunch of stuff in the filter table and some more stuff in the nat table. And that it’s all generated by Docker.

Okay, now that we’re able to look at iptables rules, how do we read them? And how do we write our own?!

the nat & filter tables

The only tables I’ve seen Kubernetes use so far are the nat and filter tables. The filter table mostly makes sense to me – you can set it up as a firewall to drop some packets, and read the man page to understand the syntax. In the example above there are a bunch of things in the FORWARD chain that apply to forwarded packets. I’m not sure how the idea of a “forwarded packet” applies to Docker yet (I think this documentation page “Understand container communication” is relevant), but I’ll leave it there for now.

But the nat table!! I have learned a few things about that! I’m going to break down a specific nat table rule that I found in the Kubernetes source code because it took me a while to understand what it was doing. Here it is:

/usr/sbin/iptables -w -t nat -A POSTROUTING -o eth0 -j MASQUERADE ! -d ${CONTAINER_SUBNET}

The first time (and possibly second & third times) I saw this my eyes kind of glazed over. I googled iptables masquerade like 5 times and was like “why doesn’t the iptables man page even say the word masquerade at all?!?“.

But it actually turned out to be important to know what this meant, so, now I know!! Let’s break down the easy options first

-w: this takes out an exclusive lock so that it can’t run concurrently with other iptables executions. Makes sense.
-A: just means “add a rule”
POSTROUTING is the name of a chain, it’s basically the last step in iptables processing
-o eth0: means “the output interface is eth0” – so the packet is going out of the computer (and not for example being bridged to a Docker network interface)

Those aren’t so bad! Now let’s deal with the complicated part – -j MASQUERADE ! -d $CONTAINER_SUBNET

-j MASQUERADE means “execute the MASQUERADE rule on this packet”. But what is the MASQUERADE rule?

SNAT & MASQUERADE

The problem this is trying to solve is – packets in Kubernetes that are sent from pods have pod IP addresses on them (which are different from the “real” IP address on the host). This is fine, but if you send a packet to a computer outside of your cluster, they won’t know what that IP address means or how to route traffic back to it.

This is a lot like the problem you have when you’re on a home network (your IP is 192.168.x.x), and you want to talk to hosts in the outside world. They don’t know what 192.168.x.x means!

The iptables.info documentation for SNAT says:

This is what we want, for example, when several hosts have to share an Internet connection. We can then turn on ip forwarding in the kernel, and write an SNAT rule which will translate all packets going out from our local network to the source IP of our own Internet connection. Without doing this, the outside world would not know where to send reply packets, since our local networks mostly use the IANA specified IP addresses which are allocated for LAN networks. If we forwarded these packets as is, no one on the Internet would know that they were actually from us. The SNAT target does all the translation needed to do this kind of work, letting all packets leaving our LAN look as if they came from a single host, which would be our firewall.

This sounds perfect! We need to set up a SNAT iptables rule! Except with SNAT you need to specify which IP address you want to rewrite the source IP address to. So MASQUERADE lets you just rewrite packets to the host’s IP address.

iptables.info actually explains what MASQUERADE means, unlike the iptables man page. With this reference, it’s much easier to understand iptables incantations! Yay!

having the right iptables rules is important

iptables is kind of frustrating because if you don’t have the right rules you can end up in situations like “oh well, no networking works at all, oops”. And I don’t know any iptables debugging tools (though if you do, I’d like to know!) so so far I just stare at the rules until I understand them.

that’s all

I am a little less intimidated by iptables than I was last week so that’s good!