Twenty years ago this month, RFC 1883 was published: Internet Protocol, Version 6 (IPv6) Specification. So what’s an Internet Protocol, and what’s wrong with the previous five versions? And if version 6 is so great, why has it only been adopted by half a percent of the Internet’s users each year over the past two decades?
Google also keeps a map of the world with IPv6 deployment numbers per country, handily color-coded for our convenience. More and more countries are turning green, with the US at nearly 25 percent IPv6, and Belgium still leading the world at almost 43 percent. Many other countries in Europe and Latin America and even Canada have turned green in the past year or two, but a lot of others are still stubbornly staying white, with IPv6 deployment figures well below one percent. Some, including China and many African nations, are even turning red or orange, indicating that IPv6 users in those countries experience significantly worse performance than IPv4 users.
The past four years, IPv6 deployment increased by a factor 2.5 each year: from 0.4 percent by the end of 2011 to 1 percent in late 2012, 2.5 percent at the end of 2013, and 6 percent a year ago. Having 4 percent of the world’s population Google’s users gain IPv6 connectivity in a year is a huge number, but considering that outside Africa, there’s no more IPv4 addresses to be had, the remaining 90 percent IPv4 users are still in for a rough ride.
Of course existing IPv4 addresses aren’t going anywhere, but without a steady supply of fresh IP addresses, it’s hard to keep the Internet growing in the manner it’s accustomed to. For instance, when moving to a new place, you may discover your new ISP can’t give you your own IPv4 address anymore, and puts you behind a Carrier Grade NAT (CGN) that makes you share one with the rest of the neighborhood. Some applications, especially ones that want to receive incoming connections, such as video chat, may not work well through certain CGNs.
If a 67 percent increase per year is the new normal, it’ll take until summer 2020 until the entire world has IPv6 and we can all stop slicing and dicing our diminishing stashes of IPv4 addresses.
Why the delay?
IPv6 has been around for two decades now. So why are we still measuring IPv6 deployment in percentage points per year, while Wi-Fi didn’t exist yet in 1995 and has used up the entire alphabet to label its variants since then, with many users on IEEE 802.11ac today?
In order for a Web browser to show a video of a cat riding a roomba, a bunch of video data needs to be transferred. But that’s the easy part: we have more than enough cabling and radio transmitters in place to make that happen. The hard part is knowing what the light pulses and voltages in these cables and the radio waves going through the air actually mean so the receiver can make heads or tails out of it. For that, we have protocols and standards.
Most of the time, when a standard gets updated or replaced, only two devices need to care. For instance, with the explosive growth of the cat on roomba category, Youtube may have needed to upgrade its 10 Gigabit Ethernet connections to 100 Gigabit Ethernet. That’s a serious upgrade of the affected servers and routers, but the rest of the internet doesn’t care. We’ve added four zeros to the Ethernet and Wi-Fi speeds over the past decades. But Ethernet, FDDI, Wi-Fi, PPP, Packet over SONET, Resilient Packet Ring and other standards just add their own control data when a packet is transmitted and then remove it after the packet is received at the other end of the line (or radio link). So I have no way of knowing whether a packet I receive from Youtube was transmitted over 10 Gigabit Ethernet or 100 Gigabit Ethernet or something else completely: when I get it, the packet looks the same in each case. This makes upgrading the bandwidth of the internet very easy: just upgrade one connection at a time.
Other standards govern the interpretation of the data by the ultimate receiver. When Youtube upgraded its videos from Adobe Flash to HTML5, obviously the software on Youtube’s end had to be upgraded as well as our browsers so those knew how to properly request the videos using HTML5 and then play them as intended. However, the routers along the way don’t care. They just see lots of small data packets coming by. That the content of those packets is now formatted slightly differently doesn’t influence the way the data is carried through the wires and sent in the right direction. So upgrading applications that run over the internet is harder than increasing the bandwidth because there will always be people visiting Youtube with old browsers. But it’s still relatively straightforward: deploy the new standard, but fall back to the old one if necessary. This way, we were able to move from postage stamp sized Sorenson Video-compressed videos to 4k H.264 in about a decade and a half.
Unfortunately, the Internet Protocol is different. The sending system needs to create an IP packet. Then, all the routers along the way (and any firewalls and load balancers) need to look at that IP packet to be able to send it on its way. Finally, the receiving system needs to be able to understand the IP packet to get at the information contained in it. Even worse, the applications on the sending and receiving ends often have to look at IP addresses and thus know the difference between an IPv4 address and an IPv6 address. So we can’t just upgrade a server and a client application, or two systems on opposite ends of a cable. We need to upgrade all servers, all clients, all routers, allfirewalls, all load balancers, and all management systems to IPv6 before we can retire IPv4 and thus free ourselves of its limitations.
So even though all our operating systems and nearly all network equipment supports IPv6 today (and has for many years in most cases), as long as there’s just one device along the way that doesn’t understand the new protocol—or its administrator hasn’t gotten around to enabling it—we have to keep using IPv4. In that light, having ten percent of users communicate with Google over IPv6 isn’t such a bad result.
Forward compatibility or backward compatibility?
So a system running IPv6 can’t talk to a system running IPv4. But who’s fault is that? Should IPv6 have been more backward compatible? Or should IPv4 have been more forward compatible?
“Firstly, it’s important to debunk a common misunderstanding that goes something like this: ‘IPv6 is badly designed because it isn’t backwards compatible, so we need all these annoyances like dual stack, tunnels and even NAT64,'” says Brian Carpenter, professor at the Department of Computer Science of the University of Auckland, who was involved in the discussions that led to the creation of IPv6. Dual stack is simply running IPv4 and IPv6 side-by-side: IPv4 to reach systems that run IPv4 and IPv6 to reach systems that run IPv6. This is currently the usual way to deploy IPv6. When two systems both support IPv6 but the routers between them don’t, they can put their IPv6 packets inside IPv4 packets and thus “tunnel” them. And if all else fails, IPv6 can be translated to IPv4 using NAT64.
Carpenter continues: “The fact that people don’t understand: the design flaw is in IPv4, which isn’t forwards compatible. IPv4 makes no allowance for anything that isn’t a 32 bit address. That means that, whatever the design of IPng, an IPv4-only node cannot communicate directly with an IPng-only node. So if you want interoperation, you actually have only three choices: dual stack, tunnel, or translation. That is true for every possible design of IPng, including the one we chose. Even adding 1 bit (or more likely 8 bits) to the address and changing nothing else creates the same problem.”
So Carpenter makes a strong argument that it’s the limitations of the IPv4 protocol that are now making it harder to adopt the very successor that aims to remove those limitations. Vint Cerf, the father—or one of the fathers, anyway—of the internet and the IPv4 protocol, somewhat humorously posits that IPv4’s 32-bit address length is because IPv4 is the “test version” of the Internet Protocol, and the production version (IPv6) has a more useful 128-bit address length. (26 minutes in on this interview.)
It’s hard to avoid the conclusion that even in the early 1980s, 32 bits, allowing for about four billion addresses, wasn’t enough. And 128 bits, allowing for 340 billion billion billion billion addresses is more than we reasonably need. Wouldn’t variable length addresses have been the prudent choice?
Interestingly, the Internet Architecture Board (IAB) initially wanted to replace the Internet Protocol with CLNP, the Connectionless-mode Network Protocol that had been created by the International Organization for Standardization (ISO). CLNP has variable length addresses with a maximum length of 160 bits. However, the Internet Engineering Task Force constituency, a loose group of vendors, network operators, academics and more that work to keep the internet running and write internet standards in the form of archaically formatted “RFC” documents, rejected that proposal. I always blamed this on the “not invented here” syndrome that few engineers are immune to, but reading Carpenter’s book Network Geeks How They Built the Internet, I was struck by how much deeper the anti-OSI sentiments ran.
Despite pressure from governments to adopt “real” standards rather than the “unofficial” and “informal” TCP/IP, the latter had won out because it pragmatically networked all kinds of different computers together, often for free. The ISO’s Open Systems Interconnect (OSI) family of protocols, on the other hand, suffered from a bad case of “kitchen sink,” including any and all possible options in order to satisfy all possible stake holders, delaying robust implementations. And it was expensive to boot.
The dissatisfaction with the IAB proposal ran so deep that the way the IETF chooses its leadership was completely changed, and a bottom-up “IP next generation” (IPng) effort was started to come up with a replacement for IPv4. One proposal included variable length addresses. But, according to Carpenter: “Yes, we knew that variable length addresses were a Good Thing but people who were writing code for small mini-computers didn’t want to write that kind of code. So the decision went in the direction of conservatism.”
So what did the IPv6 designers come up with?
IP headers 101
The Internet Protocol is the protocol that allows packets to find their way from one end of the Internet to the other. Like other protocols lower in the protocol hierarchy, IP adds some control information to each data packet to help it do its job. This control information is the IP header, which, unsurprisingly, is added to the front of each packet.
This is what the IP header looks like, first the IP version 4 header, followed by the IP version 6 header. For unspecified reasons, version numbers 0 – 3 were never given out, or perhaps returned after IPv4 was finished. Number 5 went to the experimental Internet Stream Protocol back in 1990 and versions 7 – 9 were given to proposed replacements for IPv4 in the period leading up to the development of IPv6.
(People who’ve already looked at RFC 1883 may notice that the IPv6 header looks different above from the one in the RFC. There are two reasons for that. First, I’ve drawn the header as five 64-bitwords rather than ten 32-bits words, as IPv6 is optimized for 64-bit processing. Second, in RFC 1883, the Traffic Class field was called Priority and it was only four bits in size, with the Flow Label field taking up the extra four bits for a total of 24. This was updated to the above in RFC 2460 in 1998.
Also note that the Internet Engineering Task Force (IETF) counts its bits, starting at zero, from left to right, highest to lowest, rather than the more usual right to left, lowest to highest.)
At first blush, it may seem that the IPv4 and IPv6 headers are radically different, only having the Version and Source and Destination fields in common. But appearances can be deceiving.
Unsurprisingly, in IPv4 packets the Version field contains the value 4 and in IPv6 packets the value 6.
In RFC 791, the IPv4 protocol specification, there is a detailed discussion on how the Type of Service field can be used to give certain packets higher priority than other packets. This mechanism was later replaced with a different system. The IPv6 specification doesn’t take a position on how the field should be interpreted, but does rename it to Traffic Class.
With IPv4, the Total Length field contains… the total length of the IP packet, including both the size of the IP header (normally 20 bytes) and the size of the contents of the IP packet. With IPv6, thePayload Length only counts the size of the data carried in the IP packet; the size of the IPv6 header (40 bytes) isn’t included. Ethernet can carry packets up to 1500 bytes long. If that 1500-byte packet is an IPv4 packet, it would contain the IPv4 header (20 bytes) and 1480 bytes of data, with a value of 1500 in the Total Length Field. For an IPv6 packet, that would be 40 bytes of IPv6 header and 1460 bytes of data, with 1460 in the Payload Length field. So IPv6 avoids the situation that can happen with IPv4 where it says the total length of the IP packet is 19, while the IPv4 header alone is 20 bytes.
The original definition of the Time to Live must have been inspired on Mission: Impossible. The idea is that it would be bad for messages to keep circling the network forever if there’s a loop in the network, so the sender puts a number of seconds in the TTL field. As routers buffer packets before they can transmit them, the seconds count down and eventually the TTL field reaches zero and the packet is destroyed. However, the people building the first IP routers found keeping track of the age of individual packets way too much work, so they simply had their routers decrease the TTL by one when processing a packet—whether the packet had been buffered for ten seconds or ten milliseconds. The authors of the IPv6 specification had no problem with this pragmatic approach and changed the name of the Time to Live field to Hop Limit to better reflect the actual behavior.
With IPv4, the Protocol field indicates what type of protocol must be used to decode the contents of the IP packet, such as TCP used for the Web, mail and many other applications, or UDP used for the DNS. With IPv6, the Next Header field serves the exact same purpose, but again, under a new name.
IPv6 has no room for extra options. The IPv4 header shown above is the minimum size IPv4 header—it can be increased to make room for additional optional features. In that case, the Internet Header Length (IHL) field contains a larger number than the usual 5, which indicates 5 32-bit words = 20 bytes. The IPv6 header has a fixed length and thus doesn’t need a header length field. However, IPv6 allows for options using a different mechanism: when options are used, an extra header is inserted between the IPv6 header and the TCP/UDP/etc. header. For instance, if a packet is too large, it can be fragmented and that requires the presence the Identification, Flags andFragment Offset that are always carried in the IPv4 header but are not part of the IPv6 header. So in that case a Fragment Header is added which contains these fields. The IPv6 Next Header field then contains 44, the number assigned to the IPv6 Fragment Header. The Fragment Header has a Next Header field of its own, which then contains (for instance) 6 for TCP or 17 for UDP.
Last but not least, IPv6 does away with the IPv4 Header Checksum field and adds the Flow Label field. A simple checksum is computed over the IPv4 header before a packet is transmitted. Then if a field gets corrupted along the way, the receiver will detect this because it also computes a checksum, and the computed checksum will be different from the one in the Header Checksum field. The packet is then thrown away. All the routers along the way have to put a new checksum in the packet because they update the Time to Live field. IPv6 removes this overhead, assuming that the checksums used by lower layer protocols such as Ethernet and higher layer protocols such as TCP will detect the problem if the IPv6 header gets corrupted in transit.
I strongly suspect that the Flow Label field was included simply because there were some unused bits left in the IPv6 header. The purpose of the flow label is to make it easy for routers and firewalls to easily detect which packets belong to the same “flow” (usually a TCP session) so all these packets can be treated the same.
Obviously, the Source Address and Destination Address fields in the IPv4 and IPv6 headers serve the same purpose. The whole point of the upgrade to IPv6 was to create more room for additional addresses, which the upgrade from 32 bits (IPv4) to 128 bits (IPv6) does in spades. It’s a bit like going from 10-digit phone numbers to 40-digit phone numbers. The generals won’t be fighting that war for a third time.
Configuring IPv6 addresses
So we can conclude that the IPv6 header makes the addresses really, really, really, really large, adds a flow label, removes one or two IPv4 complications and then applies a fresh coat of paint without really changing anything fundamental.
Routers now have 128 bits to consider when deciding where to send IP packets. Turns out this really doesn’t make much of a difference: routing protocols were updated to handle the larger addresses but otherwise work just as well as they do with IPv4 (or just as poorly / insecurely).
Of course in order for packets to be routed across the internet, a computer or other networked device first needs to be configured with an IPv6 address. It would be nice if an IP address could be burned into a device at the factory the same way an Ethernet/Wi-Fi MAC address is put in a device at the factory. But our poor routers have a hard enough time keeping track of all the IPv4 addresses in the form of half a million large blocks—they wouldn’t be able to keep track of billions of individual IP addresses. This is similar to how each city has its own area code so the phone companies don’t (well, didn’t) have to know where each individual phone number goes. Of course this means you need to get a new number when you move to a new city. Same with IP addresses: you need to get a new one when you move to a different (sub-)network.
Back in 1995, it was still very common for IPv4 systems to be configured with an IPv4 address manually. After all, the Dynamic Host Configuration Protocol (DHCP) specification was published only two years earlier. On the other hand, in the 1990s, IP(v4) was far from the only game in town. Many networks used IPX, a protocol that has 80-bit addresses, where each Ethernet or other type of subnet has the same 32 bits in common (these are broadcast by routers) and the remaining 48 bits are filled in with the MAC address that’s burned into Ethernet cards in the factory. CLNP uses a similar procedure. AppleTalk, on the other hand, only uses 24 bits for its addresses, with16 bits for the (sub)network part and 8 bits to identify individual systems within each subnet. The contents of those 8 bits is randomly generated.
So which approach does IPv6 use to provide systems with their 128-bit IPv6 address? Manual configuration? DHCP? Bits broadcast by routers complemented with a MAC address or a randomly generated value?
Answer: all of the above.
Back in the early days of IPv6, manual configuration was always an option, and it usually still is. But in addition to that, early IPv6 implementations also already used stateless autoconfiguration. IPv6 routers broadcast the first 64 bits of IPv6 addresses that are used on a subnet. Actually, theymulticast this information, as IPv6 is no longer so crude as to broadcast its house keeping info to systems that aren’t interested in it. A broadcast wakes up all systems connected to the network, a multicast only those subscribed to the multicast group address in question.
Systems connected to a network receive the “router advertisements” from IPv6 routers and combine the 64 bits learned from those with their MAC address to create a full IPv6 address. The nice thing about this is that every system always gets the same address without the need to have a DHCP server keep track of who has which address. The downside is that when I connect to Google at work, Google sees my Mac’s MAC address (pun entirely intended) in its IPv6 address. Then when I get home and connect to my home network and visit Google again, I have a very different IPv6 address because the upper 64 bits now come from my ISP rather than from my work network, but the bottom 64 bits still contain the same MAC address so Google knows it’s still me. The privacy people didn’t like that one bit, so in 2001, they came up with privacy extensions. With these enabled, a system uses a random number to generate the bottom 64 bits of the 128-bit IPv6 address. A new random number is generated every time a new network connection is made or after 24 hours.
And in 2003 DHCPv6, the IPv6 version of DHCP, was published. Many education and enterprise system administrators aren’t big fans of stateless autoconfiguration with privacy addresses, as this makes it hard to trace back which device was used for a given communication session. Having a DHCP server give out IPv6 addresses solves that issue. Because stateless autoconfiguration was already out there and works fine in smaller networks, it took a relatively long time for DHCPv6 to be widely supported, and Android still doesn’t do so.
There’s even a specification for Cryptographically Generated Addresses.
And many more years to come
A decade or so ago, it was still quite common for people to complain about certain IPv6 features, and proclaim the protocol would never catch on. Although part of that can be blamed on the conservative nature of network administrators, it’s true that adopting IPv6 requires abandoning some long standing IPv4 practices. For instance, with IPv4, it’s common to use Network Address Translation (NAT) so multiple devices can share the use on an IPv4 address. IPv6 has more than enough addresses to give each device its own, so there’s no NAT in IPv6. The Internet is probably better off without NAT and the complications that it adds, but without NAT as a first but relatively porous line of defense against random packets coming in from the open Internet, it’s necessary to be much more deliberate about which types of packets to accept and which to reject.
If we were to run the IPng process again today, I’m sure some decisions would go in a different direction and the result would probably look more like the IPv4 of 2015, the same way that IPv6 was heavily influenced by the network protocol landscape of the first half of the 1990s. But as Charles Dudley Warner said: “Everybody complains about the weather, but nobody does anything about it.” There are no viable alternatives to IPv6 on the horizon, and we don’t have the time to build a new protocol from scratch and spend twenty years getting that protocol to ten percent deployment—there’s no plan B.
Just like a watched pot never boils, the future has a tendency to just happen when we’re not paying attention. 2016 won’t be the year we start turning off IPv4, but the year that we do is getting closer every day.
This post was originally published at: Arstechnica.