Hi, I've been posting these reports to the ipv6-ops list for some time now. Shane Kerr asked if I could post them here too, something which I don't mind doing at all, so here comes the one for March: -------- Opprinnelig melding -------- Emne: IPv6 brokenness in Norway, March 2010 Dato: Thu, 01 Apr 2010 01:35:01 +0200 Fra: Tore Anderson <tore.anderson@redpill-linpro.com> Organisasjon: Redpill Linpro AS Til: ipv6-ops@lists.cluenet.de Hi list, I'm happy to announce that the client loss is at an all-time low: 0.074% over the whole month, and it's been steadily decreasing. Buggy versions of Opera as well as Mac OS X appears to be responsible for about 95% of all measured client loss. Some noteworthy dates and events: 20100302: Opera 10.50 released, using getaddrinfo() on Windows. Gets an initial boost thanks to the Microsoft browser ballot. 20100322: Opera pushes 10.51 to their auto-update function. 20100324: A-Pressen Interaktiv re-joins the experiment. 20100327: First day of (extended) Easter holiday for many Norwegians. 20100331: The Gathering, a large data-party, kicks off. Provides native IPv6 connectivity to all participants. The numbers show that the release of Opera has helped quite a lot. Before the release of 10.50, the overall client loss was above 0.090%, but already right before Easter holiday it's below 0.065%. Easter holiday shows an interesting effect - both the measured amount of native IPv6 _and_ client loss drop sharply. The reason for this is that there's one particular network here in Norway that is the by far largest deployment of native IPv6 (about 80-85% of all requests over native IPv6 comes from it), but it is at the same time one of the largest causes of client loss, because it aggravates the Mac OS X/Opera problems significantly by filtering 6to4 traffic on ingress for a large number of end users (that have no native IPv6 service). The typical user on this network is also much more likely to take an extended Easter holiday than the the rest of Norway's population, I believe. I've been measuring the uptake of Opera 10.50+ on Windows Vista/7 in the whole period, and it has been steadily increasing even during the extended Easter holiday. About 72.5% of the Win Vista/7 Opera users were using 10.50+ at the end of the month. So while I do expect the client loss number to rebound somewhat when Easter holiday is over (on the 6th of April), I will be very surprised if it stabilises higher than 0.06-0.07%. The Mac OS X problem appears to occur when a client station has internet service using NAT/RFC 1918 numbering for IPv4, and 6to4 for IPv6. The Apple Airport Extreme SOHO router can operate in this way, and did so by default up until recently. In this case, OS X, or more precisely its getaddinfo() implementation, will prioritise 6to4 over IPv4. In addition, GNU libc (therefore all major Linux distributions) behave in the same way, but I'm not able to measure any impact, probably because there's so few Linux users to begin with. The issue is not actually a bug, but a shortcoming of RFC 3484. I've elaborated on it in reports to Apple and the glibc developers here: http://lists.apple.com/archives/Ipv6-dev/2010/Mar/msg00003.html http://sourceware.org/bugzilla/show_bug.cgi?id=11438 As an aside it's a bit fun to see how a relatively small deployment of IPv6 eyeballs like the data party (around 5000 participants I think) can dramatically change the amount of hits coming in from hosts with native IPv6. It more than doubled overnight. There's two reports attached, one generated from the readers of VG Multimedia - the online edition of Norways largest newspaper, and also Norway's largest web site, and the other from the readers of A-Pressen Interaktiv - an umbrella organisation that publishes the online editions of about 70 regional newspapers. When put together, they constitute Norway's fourth largest web site. In addition to the overall calculation, I've done five other calculations so you can see the individual impact of the different problems I've identified: - One that excludes hits from buggy versions of Opera on Windows Vista/7 (which auto-configures 6to4 and Teredo by default) excluded, - one that excludes hits from buggy versions of Opera on other versions of Windows (where a user must explicitly enable 6to4/Teredo), - one that excludes all hits from Mac OS X, - one that excludes all of the above, and - one that excludes all hits from the two significant end-user networks in Norway that filter 6to4 ingress traffic (one of them is the one I discussed above). This calculation does _not_ exclude Opera/OS X; when Opera and OS X are excluded, the two networks have about the same levels of client loss as rest of the internet. The seventh column shows approximately how much of the total client loss would have been avoided (in percentage points and percent), if the software in question didn't prefer 6to4 over IPv4, or, in the last case, if the two end-user networks mentioned didn't filter 6to4 on ingress. Best regards, -- Tore Anderson Redpill Linpro AS - http://www.redpill-linpro.com/ Tel: +47 21 54 41 27