Today I put my new networking zine on the internet! I am very excited about that. If you are interested in networking but don’t quite know how it all fits together, you could take a look!

Today, let’s talk about packet sizes.

UDP

UDP is a simple protocol for sending information – you put information in a packet, send the packet to its destination, and it might get there or not.

One interesting thing about this is – when you send a UDP packet, you need to decide how much information to put in the packet.

This means that you need to know how much data can fit in a packet! Here’s a diagram showing what a UDP packet looks like.

The only thing I want you to pay attention to here is the size of the “length” section – 16 bits

maximum UDP packet size: 2^16 bits

So! The number for the length of a UDP packet is 16 bits wide. This means it can be between 0 and 2^16 - 1, or 0 to 65535.

So, I can have a UDP packet that is 65535 bytes, right?

Well, it turns out that if you send a UDP packet that is 30,000 bytes on the internet, it will probably not arrive. Why not? This is because of Ethernet!

Meet: Ethernet frame sizes and the MTU

When you send a packet to another computer on the internet, it needs to go through a bunch of cables. Some of them are Ethernet cables.

Every packet lives in an Ethernet frame. Ethernet frames can only contain 1500 bytes of data. This is called the “maximum transmission unit” or “MTU”!

So, if I try to send a 30,000 UDP packet using an Ethernet protocol with an MTU of 1500 bytes, what will happen? We can’t send 30,000 bytes in 1500 bytes. It doesn’t work.

So one of two things will happen: either the packet will get dropped (not sent at all) or fragmented.

Packet fragmentation

Sometimes networking implementations will split up packets into multiple pieces. So if you have a packet that is 15,000 bytes, you could split it up into 10 or so 1,500 byte packets.

I think this works okay with TCP packets (TCP packets are already a data stream split up into many pieces, so you can just split it up further and it’ll get reassembled at the end).

It seems weirder to me to split up a UDP packet – if I’m sending you a large UDP packet and it gets fragmented, how do I reassemble it? What if the parts that split up got out of order? Someone on Twitter told me you can reassemble UDP packets but I don’t know how it works yet.

I’m definitely still confused about how packet fragmentation works and in what cases it makes you lose packets. (is fragmentation worse for TCP packets or UDP packets?)

How do you figure out your MTU?

What if you want to know what the MTU is between two points?

It turns out that you can use a thing called Path MTU Discovery. I found this out by reading the wikipedia article as usual. Basically you

• send a large packet with a flag on it saying “never fragment this!”
• at the first point where that packet would be fragmented, it’ll notice and send back an ICMP packet saying that the packet got fragmented (ICMP packets are what routers use to send meta information like this)
• success! You now know the first point at which the packet would be fragmented!

It turns out that on Linux there is a tool that will do this called tracepath.

$ tracepath google.com
1?: [LOCALHOST]                                         pmtu 1500
1:  OpenWrt.lan                                           1.705ms 
1:  OpenWrt.lan                                           1.973ms 
2:  10.252.42.193                                         9.116ms 
3:  10.170.192.58                                         8.046ms
asymm  4 

I think the MTU on my local network is 1500 bytes. Which makes sense I guess! My local network is pretty normal.

David Murray gave a talk about Path MTU Discovery at Papers We Love Seattle which looks pretty interesting.

jumbo frames

Sometimes you can send huge packets though! On the internet in general, you can’t expect to be able to send packets above 1500 bytes, but if you own the whole datacenter and all of your network infrastructure? Why not!

Newer versions of Ethernet support “jumbo” frames, which means that you can send large packets.

This documentation about MTU inside AWs is really interesting if you run AWs instances and are wondering how all this applies to AWS.

that’s all

Probably I’ve gotten something wrong in here somewhere. I think the fact that there are these limits on packet sizes on the internet is super interesting! I found the Wikipedia articles on this topic to be pretty readable.