We've rolled out about 12 million nodes on IPv6, over 300 customer /48 networks, and about 6,000 individual /64 subnetworks. 99% of those have been RFC 4193 Unique Local IPv6 Unicast addresses.
IPv6 is like a breath of fresh air when it comes to setting up address spaces. For the past 10 years as a network engineer, three overriding issues have existed when setting up new networks. (1) Begging ARIN for a /19 or /20 when setting up a major new network, (2) Constantly having to have a CIDR calculator running in my head when trying to figure out where the usable hosts/broadcast/network/gw live for the various /29s, /28s, /27s/ /26s, (3) Dreading having to communicate with a new partner or customer also in RFC 1918 space (because of the odds of an address conflict), and/or adding NATing so that we can do a hand off in public space.
With IPV6 - all of those issues are gone. Every subnetwork is always a /64 - CIDR is basically gone. Also, because RFC 4193 is so large (FC::/7, though, for local assignment, it's always FD::/8) - we for all practical purposes never have to worry about negotiating with partners.
Our "Addressing Strategy" is as follows:
For a new customer, Randomly generate, following the guidance of RFC 4193, a /48. Then, for all of their subnetworks, just increment by one. I.E. A customer might get FDC2:D343:107F::/48, and their first subnetwork in that range would be, FDC2:D343:107F:1::/64
For our own IPv6 space, we occasionally play stupid aggregation games, just to keep our routing tables a little more dense - aggregate on ::/56 boundaries - I.E. If we have a new data center with a bunch of subnetworks in our FD1B:D2C5:6AB3::/48 space, we might assign it to be FD1B:D2C5:6AB3:0100::/56, and then we have 255 subnetworks, FD1B:D2C5:6AB3:0100::/64 - FD1B:D2C5:6AB3:01FF::/64) that we work with.
About the only problem with IPv6, is that they make it difficult for you to get your own provider independent globally routable address space that you can advertise out via BGP. So, for instance, a company can't just (easily) get some address space and advertise it out through the internet - the powers that be want you to get a subnetwork out of your ISPs /32. As a result, we ended up just doing everything in RFC 4193, and, using a relative handful of "globally routable" IPv6 hosts for communicating on the internet from our ISPs /32 space. Our ISP gave us a globally routable /48, and we use about 12 hosts addresses from its possible 2^80 possible addresses. Talk about sparse utilization!
Eventually, as more IPv6 hosts land on the internet, and we want to communicate to them from inside our network, we'll probably just add a IPv6-to-IPv6 Network Prefix Translation (NPTv6), or, more likely, an IPv6 NAT system.
NAT has become so embedded in the Enterprise, and provider independent globally routable addresses assigned to companies so difficult to get, that, the globally routable space will ironically be used for fewer IPv6 hosts in many cases, than it was for IPv4 hosts in many cases.
I realize that this is very controversial, and contrary to what every IPv6 IETF network designer/architect/RFC writer wants to hear, but, in talking with the dozen or so colleagues that I have who are also deploying IPv6, it's seems to be the consensus. NAT has been used for so long, by so many network engineers, that non-routability internal space has become our "goto" strategy for the second layer of defense. (With stateful firewalls, of course, always, always being the first layer)
Indeed, when we tried to move a bunch of our applications to Amazon, they indicated that they were only supporting IPv6 for limited interaction on the internet, and that they weren't supporting IPv6 for their internal networks - really embedding the Proxy/NAT concept for AWS applications. Which is a shame, because we'd love to move a lot of our IPv6 applications into AWS.
You've heard it before, but for the sake of balancing your comment on NAT... Sincerly and honestly: NAT does not solve anything that a firewall (6 lines of ip6tables rules) - or stopping that unecessary service - doesn't. All it does is create two classes of network devices (and netizens): those that can talk directly and those who depend on a third party. There's no reason not to have a global address anymore, even if you don't expect to use it. Even something as simple as putting a firewall appliance in the middle is the wrong approach. Logic needs to be near the edge, in the host IMHO; by putting any rules in the middle (firewall, NAT, QoS), other than the basic routing, one loses the intended ability to dial direct in unanticipated scenarios tomorrow (for example, to run a new service without touching every hop between A and B because of dozens of attempts to "make it better" at that hop). IMHO NAT really is an example of our eagerness to hang ourselves, given enough rope.
Good balance - and certainly it's a common refrain from those who are architecting and writing RFCs. And, keep in mind, 100% of all corporate networks I've seen with NAT, also have (typically very expensive) firewalls with highly restrictive rules. But corporate IT stooges (which I've certainly played the roll of) will say: "Here is what NAT will do for you with IPv6"
#1 - You don't expose any internal hosts addresses to the external world, everything comes from a single external host.
#2 - No route back to the hosts inside the lan. If they can't route to you, they can't attack you. (Not really true, but it does make it more challenging to mount an attack from the Outside -> In. )
#3 - You can change ISPs without any internal renumbering required. Unlike with IPv4, getting provider-independent IPv6 space can be challenging.
IPv6-to-IPv6 Network Prefix Translation (NPTv6) might be a happy middle ground that solves #3, but still leaves you mostly exposed on #1 and #2.
I'm still vehement that #1 & #2 are wishful thinking (security through obscurity... better to drop all packets, or simply not use a public address; ULAs are fine IMHO), but perhaps something as simple as a prefix-aliasing notation would allow easier internal referencing and address #3. For example, MY_NETWORK:1. That can be done with shell/environment variable substitutions before using them, but it would be nice to be able to count on IPv6 applications to accept something like that for networks in the routing table... (might require a prefix-naming RFC, so a name can come with a router/prefix advertisement and/or the ability to store name-to-network prefix mappings in DNS SRV, or similar, allowing internal or global users to find the right host on the right network...)
That said, truly global addresses with service registries like DNS/mDNS, etc, remove the need for immutable (staticly-assigned and hard-coded) addresses entirely, albeit at the expense of registration and lookup.
I'm very much surprised by this given the converse situation at my last^3 company; this was an organization that had been around in the old .com days and therefore owned three (I think) /16s. We used globally routeable addresses for all hosts including our individual desktops, and I understood this made life much easier, e.g. it made it easier to set up VPN connections because there was no risk of collisions between our networks and people's home networks. And when we were bought out by a large multinational conglomerate there was no danger of collision between our and their networks, so connecting them was as simple as, well, connecting them.
I'm certain there are many companies that are doing precisely that - but, your introduction sentence was important, "Owned three /16s" - your company had already developed practices and procedures that assumed everyone had a globally routable IP address, and so it was much more likely that when they transitioned to IPv6, that they would likewise use globally routable IPv6 addresses. I bet there is a good chance that they went to the effort of getting their own provider independent IPv6 space - which eliminates a lot (all?) of the hassle associated with using a /48 chunk from their ISPs /32. I got started late in the game (1998), so I've never worked for a company that had a lot of globally routable space (a /18 at most). We always started out inside in RFC 1918, and saved our globally routable space for internet-facing hosts.
The nice things about RFC 4193, is that while the routes aren't floating around on the internet, the addresses you assign are highly, highly unlikely to conflict with someone else's network, and, in the very rare situation that they do during a merger/acquisition - you are no worse off than if you had changed ISPs and had to renumber your internal network anyways. Per RFC 4193 (Section 3.2.3), if you connected with 100 other networks, the probability of collision is 4.54*10^-09 - so the vast majority of the time you won't have a renumbering exercise to worry about.
Note, though - I'm not suggesting that anybody else adopt this strategy of using RFC 4193, and I actually wish more of my colleagues would start using globally routable IPv6 for their internal networks - particularly so I can learn from them to find out what they do when they change ISPs.
Useful post, and +1 for deploying IPv6. Too few companies do it yet. Especially the "300 customer networks" sounds like a solid step forward!
One remark though: I'm not familiar with each and every RFC on the subject, do you mind using names instead of numbers? (I also discussed it before on the Security StackExchange: http://meta.security.stackexchange.com/q/1040/10863 ).
There are certain RFCs that become, "Common Parlance". In IPv4, RFC 1918 is very well known to be the 10/8, 172.16/12, and 192.168/16 private IP space. Likewise, in IPv6, RFC 4193 is the new "RFC 1918" - and, in common conversation, usually rolls off the (at least my tongue) more easily than, "Unique Local IPv6 Unicast Addresses" - which, often gets confused with (long deprecated) "Site-local unicast" addresses (FECO::/10, RFC 3513) anyways. By calling out the particularly RFC, we eliminate all the confusion, and save a lot of space.
I really did try my best to keep the number soup down (Notice I didn't call out RFC 6296 when mentioning IPv6-to-IPv6 Network Prefix Translation)- I only used two, (And also used the name when I introduced RFC 4193) - but RFC 4193 is so important, that I hope every IPv6 network engineer has it memorized as much as RFC 1918 is to every IPv4 network engineer.
With a technical audience (Which HN has its share of), though - Use of the RFC numbers is critical, as they point to specific implementations of the text description, and may contain nuance, or, indeed, very different implementation constraints, than the name alone suggests.
Fascinating... it seems easy enough, basically in that address space is abundant enough to not have to be concerned with much of anything in a local space.
But it reads juuuuust slightly more complex than IPv4 does... it's just that side of difficult to comprehend. I think this is the true human factor behind the lack of adoption.
It's huge, like looking at all the stars in the sky, but bigger. Just the switch from numbers between 1-255 to hex codes and the more of them... it's that much more of a barrier to use and adoption.
Any ideas to simplify my own understanding of IPv6? I took the HE.net certification courses, which are excellent, but it's hard to make it stick. How do we make this intelligible so that everyone can be comfortable using it—and ideally, more comfortable than IPv4, so that it becomes preferable? That's the challenge.
I think it is just the prejudice about how "easy" things that we already know seem. But I do remember, and have seen many people, struggling to get a grasp of IPv4 subnetting, routing, security, etc. But you practice and over time it becomes second nature. If you put in half the amount of effort into IPv6 that you did when you learned IPv4 you'd probably be set.
While I am really looking forward to IPv6, I am slightly dissapointed about global addresses. Granted, globally routable adresses is the main (only) thing that is getting me exited for IPv6, but there is something nice about being able to go into a new network and only need to remember "2.10" for an IP address. (Espessial when all the IPs I care about are set to the low "0.x").
IPv6 is like a breath of fresh air when it comes to setting up address spaces. For the past 10 years as a network engineer, three overriding issues have existed when setting up new networks. (1) Begging ARIN for a /19 or /20 when setting up a major new network, (2) Constantly having to have a CIDR calculator running in my head when trying to figure out where the usable hosts/broadcast/network/gw live for the various /29s, /28s, /27s/ /26s, (3) Dreading having to communicate with a new partner or customer also in RFC 1918 space (because of the odds of an address conflict), and/or adding NATing so that we can do a hand off in public space.
With IPV6 - all of those issues are gone. Every subnetwork is always a /64 - CIDR is basically gone. Also, because RFC 4193 is so large (FC::/7, though, for local assignment, it's always FD::/8) - we for all practical purposes never have to worry about negotiating with partners.
Our "Addressing Strategy" is as follows:
For a new customer, Randomly generate, following the guidance of RFC 4193, a /48. Then, for all of their subnetworks, just increment by one. I.E. A customer might get FDC2:D343:107F::/48, and their first subnetwork in that range would be, FDC2:D343:107F:1::/64
For our own IPv6 space, we occasionally play stupid aggregation games, just to keep our routing tables a little more dense - aggregate on ::/56 boundaries - I.E. If we have a new data center with a bunch of subnetworks in our FD1B:D2C5:6AB3::/48 space, we might assign it to be FD1B:D2C5:6AB3:0100::/56, and then we have 255 subnetworks, FD1B:D2C5:6AB3:0100::/64 - FD1B:D2C5:6AB3:01FF::/64) that we work with.
About the only problem with IPv6, is that they make it difficult for you to get your own provider independent globally routable address space that you can advertise out via BGP. So, for instance, a company can't just (easily) get some address space and advertise it out through the internet - the powers that be want you to get a subnetwork out of your ISPs /32. As a result, we ended up just doing everything in RFC 4193, and, using a relative handful of "globally routable" IPv6 hosts for communicating on the internet from our ISPs /32 space. Our ISP gave us a globally routable /48, and we use about 12 hosts addresses from its possible 2^80 possible addresses. Talk about sparse utilization!
Eventually, as more IPv6 hosts land on the internet, and we want to communicate to them from inside our network, we'll probably just add a IPv6-to-IPv6 Network Prefix Translation (NPTv6), or, more likely, an IPv6 NAT system.
NAT has become so embedded in the Enterprise, and provider independent globally routable addresses assigned to companies so difficult to get, that, the globally routable space will ironically be used for fewer IPv6 hosts in many cases, than it was for IPv4 hosts in many cases.
I realize that this is very controversial, and contrary to what every IPv6 IETF network designer/architect/RFC writer wants to hear, but, in talking with the dozen or so colleagues that I have who are also deploying IPv6, it's seems to be the consensus. NAT has been used for so long, by so many network engineers, that non-routability internal space has become our "goto" strategy for the second layer of defense. (With stateful firewalls, of course, always, always being the first layer)
Indeed, when we tried to move a bunch of our applications to Amazon, they indicated that they were only supporting IPv6 for limited interaction on the internet, and that they weren't supporting IPv6 for their internal networks - really embedding the Proxy/NAT concept for AWS applications. Which is a shame, because we'd love to move a lot of our IPv6 applications into AWS.