Dear colleagues, Attached are the draft minutes of our sessions in Dublin. Many thanks to Mirjam for writing them all up. Please review them and if you have any comments or corrections please reply to this list before Friday July 12th. If by that date we haven't received any major corrections we will assume that there is consensus to declare these as final and publish them on the website. Thanks, Marco on behalf of the WG chairs.
----
RIPE 66 Scribe: Mirjam Kuehne WG chairs: David Kessens, Shane Kerr, Marco Hogewoning Date: 15 May 2013, 14:00 - 15:30 IPv6 WG Meeting (first session)
A. Welcome and Administrative Business
Marco opened the first IPv6 WG session. This session focused on implementing IPv6. The minutes of the RIPE 65 IPv6 WG meeting were circulated and are now approved.
B. Implementing Full IPv6 Support: More than Binding to an AF_INET6 Socket - Bert Hubert, PowerDNS/Netherlabs
The presentation is available for download at: https://ripe66.ripe.net/presentations/134-Making_an_application_fully_IPv6_c...
Andrew Yourtchenko, Cisco, pointed to RFC 6555 (Happy Eyeballs: Success with Dual-Stack Hosts) and suggested that people take a look at this document. He added that the "Update Specification of the IPv4 ID Field" is now also a standard (RFC 6864). He wondered if it would make sense to set up some sort of a library where people could store pieces of code.
Bert responded that instead of setting up a library, people could publish their code snippets and say "If you take this bit of code, it will help your IPv4-to-IPv6 migration." He said he would like to prevent people from having to double their source code, which is already too long.
Andrew added that there would be some code snippets in his presentation later in this WG session.
Benedikt Stockebrand, Stepladder IT Training+Consulting GmbH, commented that there are three viable options: v4-only, v6-only or both. There is a fourth option which is to leave it to whatever the default setting is in a particular operating system, but that can be problematic. One really needs to think about this beforehand and the biggest problem is explaining to people how to do this properly. Benedikt also commented that one should normally not touch link-local addresses, because as soon as one uses link-local addresses, the application forces one to put multiple machines on the same network, which should be avoided. Furthermore, multiple addresses in the DNS are essentially a necessity. That is independent of IPv6, but in IPv6 it is even more important. The last thing Benedikt mentioned was related to ACLs. He said that sometimes you can work around it with a small hack, using a local packet filter instead of an application-related ACL.
Bert responded that he agrees with the comment on link-local addresses, but in the “real world” there are people who say this is what we need to do.
Dan York, ISOC, thanked Bert for the presentation. He also wanted to second Andrew who pointed to RFC 6555. Furthermore, there are other related draft documents coming out of the IETF right now (“Happy Eyeballs for Foo”) that look at mechanisms of testing both and figuring out which one is fastest. Finally, there is a RFC 5952 that says that you should use the brackets with the colon, so choice A is now made by most people. Bert added that Dan York actually wrote a book on this topic that contains everything he presented and much more.
Ondrej Filip, CZ.NIC, commented on the IPv6-only behaviour and said that the RFC specifies that you get both IPv4 and IPv6. However, if you configure your operating system to something different, it might break, because some applications assume the default behaviour.
Bert agreed that it is a policy that can be set, but it seems to be confusing for people.
C. Non-Working Clients” – Andrew Yourtchenko (private contribution)
The presentation is available at: https://ripe66.ripe.net/presentations/114-RIPE-IPv6-only-Clients-RIPE.pdf
Jen Linkova, Google, said she liked the presentation and that it is good to let people know that IPv6-only doesn't work properly on Android.
Andrew added that one thing he forgot to mention is that on the non-WiFi, it actually works.
Marco Hogewoning, WG Co-Chair, asked the audience how many people wanted the RIPE NCC to provide an IPv6-only WiFi network during the RIPE Meetings. Many hands went up. He then asked how many people fear that the connectivity would not be good enough: fewer hands went up. Marco said he would pass this on to the RIPE Meeting Technical Team and see what can be done next time.
ACTION RIPE NCC: To look into the possibility to provide a IPv6-only WiFi network at RIPE 67.
D. The Latest Developments in DHCPv6 - Tomek Mrugalski, Internet Software Consortium
The presentation is available at: https://ripe66.ripe.net/presentations/158-latest-development-in-dhcpv6.pdf
Shane Kerr, WG Co-Chair, asked Tomek what he thinks the typical time is between a new technology getting standardised in ‘DHCP land’ and it being deployed to production.
Tomek responded that this really depends on the nature of the technology and also on how the project is managed and how many people are involved. For instance, the DS-Lite standard was deployed very quickly whereas other implementations took over a year.
Paul Ebersman, Infoblox, asked if the new relay option of getting the hardware address included in the ISC roadmap for getting into the server to be able to consume that information for getting it into the release? Tomek explained that this is up to the server's allocation policy. For instance, you can log it so that you have an answer if law enforcement want to know from which address some bad activity was originated (this is a legal requirement in some countries). Another use case is that you can assign addresses and prefixes based on the server configuration. And finally, you can perform access control where you can, for example, maintain a blacklist for specific addresses or a whitelist for known clients.
Paul agreed that there are many use cases and said that some of his customers are quite interested. He would be interested in two requirements, one being that we must have relay code that actually inserts that information (for that requirement, we need to push the vendors). But we also need the DHCP server to actually get the information in the packet and then into the lease, for instance. Are there any plans for ISC to do this? Tomek said he couldn’t confirm or deny this.
Benedikt Stockebrand was wondering why Tomek is so worried about scalability, because he thought the obvious approach would be to set up several DHCP servers and spread the subnets across them. But he would prefer the simple solution to set up a suitably sized single DHCP server that can handle the full set of subnets.
Tomek clarified that he wasn't saying that one needed to deploy the load balancer or fail-over in every single network. Provided the server is powerful enough to support all the traffic then that's also fine. Benedikt elaborated that the problem he sees is that of additional complexity.
Tomek said he agreed with this concern. The second draft that he presented actually discusses cases where the failover is not necessary. He said he would consider failover as the last resort.
Benedikt added that he believes that DHCP is one of those areas in which one doesn't need the multiple servers to share the load, and that one could solve the scalability issues without adding any complexity in the protocol. But if it is described like that in the specs, people think they will have to implement it that way.
Tomek said that, like with other DHCP extensions, this is not a mandatory feature. It is mostly an implementation issue and he hopes that every implementer will actually disable this by default.
Marco suggested that this be discussed further on the IETF mailing list (DHCP-WG).
E. Dave Täht: “Fixing Bufferbloat with cerowrt” https://ripe66.ripe.net/presentations/237-cerowrt_vs_an_uncaring_universe.pd...
Vesna Manojlovic, RIPE NCC, said he really likes this project and would like to help him expose it to many more women and children (this was related to a comment Dave Täht made in his presentation where he said that this is still a research project and "please do not expose this to your women and children.").
Ragnar Anfinsen congratulated Dave on his good work. He added that he is currently busy specifying CPEs for his company and is wondering if he should include the one Dave was presenting.
Dave responded that he would like people to look deeply into OpenWRT, which is only about 8 Linux kernels and 8 demons, and to let him know what else should be included.
Vaibhav Baijpai, Jacobs University Bremen, wondered if there were any plans to accept tools that perform active measurements tests?
Dave confirmed that this was the case. He was now at the point where he trusted the underlying stack and would be happy to add more tools.
The same speaker also asked if there are any plans to collaborate with the RIPE NCC's RIPE Atlas Team because RIPE Atlas was not WRT-compatible.
Dave said that adopting a different hardware platform right now would take a lot of time, but that he was happy to help. -----------
RIPE 66 Scribe: Mirjam Kuehne WG chairs: David Kessens, Shane Kerr, Marco Hogewoning Date: 15 May 2013, 09:00 - 10:30 IPv6 WG meeting (second session)
Marco Hogewoning opened the second session of the IPv6 WG. This session was focused on measuring IPv6.
A. Vaibhav Bajpai (RACI Talk): “Measuring Effectiveness of Happy Eyeballs”
The presentation is available at:
https://ripe66.ripe.net/presentations/263-ripe66-happy-slides.pdf
Jen Linkova, Google, commented on the statement that the happy eyeball winner is usually slower. This is still an action item for most ISPs, because currently the IPv6 experience is not in sync. Nobody wants to do it, but it has to be done.
Vaibhav added that he was doing these measurements for three months (February - April). In the first month he observed that Google services had higher means and standard deviations, but in the last month this has gone. It looked like something has been done at Google internally.
Jen said that some of the different behaviour in IPv4 vs. IPv6 could also be due to quality of service.
Carsten Strotmann, Men & Mice, asked what operating system was used for the measurements and if he had tested it using Windows.
Vaibhav responded that he used Sam Knows probes and OSX. The Teredo box was running Linux. Windows was not tested.
Carsten offered to work together on this with Vaibhav.
Martin Levy, Hurricane Electric, commented that it is quite common to have different routing for IPv4 and IPv6 (different peering partners and different propagation of routes) and suggested to find a particular measurement point from where you can at least guarantee the beginning of the route propagation. Then go back and look at the measurements to determine if you see differences caused by a tunnel or something else in IPv6.
Vaibhav responded that, in addition to doing ‘Happy Eyeball’ tests, they also did traceroutes over both IPv4 and IPv6. They then get the AS path information to see how different the IPv4 and IPv6 routes are. This provides some confidence in the measurements.
Andrew Yourtchenko, Cisco, suggested that Vaibah speak to the RIPE Atlas developers. The RIPE Atlas probes would provide a lot of good data for this project
Vaibhav said that his tool is based on a single file with 800 lines of code; if the RIPE Atlas software developers would like to review the code, he would be happy to send it to them. It would also be beneficial to look at the RIPE Atlas measurements in greater detail.
ACTION RIPE NCC: get in touch with Vaibhav and collaborate (RIPE Atlas).
B. Vesna Manojlovic, RIPE NCC: “IPv6 RIPEness Fifth Star” https://ripe66.ripe.net/presentations/271-ripeness-5th-star-v2.key
An audience speaker, SIDN, asked if the RIPE NCC was also checking the content of the web page listed on the Alex 1M list because, even if it accessible over IPv6, it could just be a parking page. He was also interested to find out if MX records were checked.
Vesna said it was an interesting idea and although this wasn’t being done at the moment, it could probably be implemented. The HTTP fetch measurements in RIPE Atlas were already enabled. She added that she didn't think MX records were checked at the moment, but it was a good idea.
Vaibhav suggested to check if the content is the same in IPv4 and in IPv6, and offered to use his tool for that.
Vesna asked the audience if they thought they should remove the pages that are only parking pages. Many people nodded in approval of this suggestion.
Niall O’Reilly, University College Dublin, suggested to look at the developments over time and possibly provide a graph at RIPE 67 that shows how many LIRs have gained and possibly lost their fifth star. Marco asked what the audience thinks about raising the bar (from currently 2% to doubling it each year). People seemed to like the idea and some suggested to raise the bar at an even sharper rate.
Daniel Karrenberg, RIPE NCC, clarified that most of the graphs Niall was asking for are already available on http://ipv6ripeness.ripe.net
C. IPv6 Measurements Panel Moderator: Marco Hogewoning, RIPE NCC Panellists: George Michaelson (APNIC), Andrew Yourtchenko (Cisco), Steve Nash (Arbor Networks), Heidi Kivekäs (FICORA),Martin Levy (Hurricane Electrics)
Marco briefly introduced the panellists. Some panellists showed slides that illustrate their involvement in IPv6 deployment measurements:
Steve Nash: https://ripe66.ripe.net/presentations/157-Arbor_IPv6.pdf
Martin Levy: https://ripe66.ripe.net/presentations/309-RIPE_66_Dublin_-_IPv6_Measurements...
George began by saying that Geoff Huston sent his apologies and that he really wanted to be at this panel, but was unfortunately unable to attend the RIPE Meeting this time. They were both really interested in the topic of “What is the correct question?” What should we ask about the transition to IPv6? How shall we measure it? There are traffic-related questions when we look at ISPs, IXPs, and others providing the underlying technology and infrastructure. There are also DNS-related questions that involve a different set of people in the industry. These people don't necessarily play with routers, but they are important for making things visible. And then there are questions about end devices and volume and scale, and questions about who consumes the information. The answers to all of these questions would provide an encompassing view. The aim is to get bits flying, and not only in transit, but to the end points. That led to the idea to measure how many people can actually get full IPv6 delivery in the system - to content and back again. And how do we get to all those millions of people? That's how the idea emerged to place these flash-based advertisements and use action scripts that fetch objects and invoke DNS functions.
As a result, it was then possible for George and Geoff to perform a whole series of tests and to measure dual stack capability in the user’s browser preference. George thanked the people at Google who have been extremely generous and allowed them to use their advertising channel. He also thanked ISC, who provided a host in the US, and the RIPE community who, through the RIPE NCC, provided a server in Europe. Cisco had also provided some assistance. George added that they noticed how people want to know how they are doing, similar to what the IPv6 RIPEness provides. It encourages the spirit of competition. Therefore, they have been compiling per-country and per-provider reports. Overall, the number of people who are already using IPv6 is really very large. In a global context, however, it's a low percentage. But in absolute terms that is a lot of people. If one starts to look at the breakdowns on a per-economy level, the dynamics are very different depending on the economy you are looking at. In Romania, for instance, there was one single, large AS that has achieved a certain level of saturation in the marketplace and since then there had been no more visible growth of IPv6 uptake. France is another example which is dominated by a single provider that has deployed 6RD. In Italy and Ireland the rate of uptake is very, very different. Graphing and understanding these individual scenarios can help with decision-making about IPv6. So, the main question behind this work was how can we inform the discussion - who can we engage with regarding the nature of IPv6 deployment?
Steve Nash explained that Arbor Networks' primary business is creating software and systems for operators to deploy and to deliver denial of service protection services. This involves collecting a lot of data- data directly retrieved from systems deployed in the network, and through an annual survey that is has been running for eight years to date already (the ‘Worldwide Infrastructure Security Report’). Steve thanked all in the room who participate in this survey. In the recent survey, one quarter of the respondents confirmed that they had deployed IPv6 in their network, one half said they would be done by the end of the year, and for the rest it was longer than that. The survey also asks about the main concerns people have about IPv6. The main concern is traffic floods through DDOS attacks. However, when compared to IPv4, the numbers are the same, meaning that most people now view IPv6 as a normal service just like IPv4 and have the same concerns about both IPv4 and IPv6. This indicates a level of maturity of IPv6.
Also, half of those that responded said that IPv6 will grow by 20% over the next year, and a quarter think growth will exceed 100%. In fact, IPv6 traffic more than doubled last year. Some measurements show that 0.2% of all traffic is over IPv6. That’s 45 terabits per second worldwide. Out of 265 operators worldwide, 80 are located in the RIPE NCC service region. However, from the operators in this region, one gigabit per second of native IPv6 traffic is produced.
In conclusion, there seem to be many people out there that have IPv6 in their network who haven't completely rolled it out, so we don't measure any traffic.
Andrew Yourtchenko presented a web site that aggregates a lot of data sources related to IPv6 deployment, such as number of prefixes allocated, number of prefixes announced, number of ASes that provide IPv6 transit, IPv6 accessible web pages (based on the Alexa 1Milllion list) and IPv6-enabled users (based on the APNIC Google Flash A data). Andrew mentioned that a big jump in allocated IPv6 prefixes was visible around May 2012 in Ireland, which could be attributed to preparations for World IPv6 Launch. On the other hand, a similar jump in announced IPv6 prefixes could not be observed. Another example Andrew presented was related to eyeballs - Germany, for instance, had very little eyeball penetration until autumn last year when steady growth began.
Next, Martin Levy from Hurricane Electrics showed a graphical representation of networks interconnecting with other ASes. It shows that, while we can see complete IPv4 routing, there is a lack of routing inside the IPv6 world. Martin and others have been trying to encourage network operators to turn on IPv6, and he said it is amazing to see that we still dealing with this lack of global connectivity. He added that what he likes about the IPv6 RIPEness measurements is that it goes beyond just measuring the existence of a route. He concluded that there is still a lot of work to do until we get parity between IPv4 and IPv6.
The last panel speaker was Heidi Kivecas from FICORA, the Finnish Communications Regulatory Authority. They are most interested in the users’ perspectives and conducted a survey amongst Finnish operators to see where things are with IPv6 deployment today (the last survey took place in 2008). However, Heidi said they mostly rely on other people's measurements. As a regulatory authority, FICORA is mostly interested in finding out why ISPs decide to deploy IPv6 and why others have decided not to, at least not for now. From her perspective, measurements are the most interesting aspect when helping ISPs solve their problems.
Marco summarised that there are various ways to measure IPv6 deployments, from specific measurements on the network level, like RIPE Atlas does, to more soft measurements like the survey results shown by Steve Nash. He asked the panellists what their specific measurements were actually targeting.
George responded that while he believes that dialogue with network providers, as enacted by Arbor and also by the measurement Martin mentioned, is absolutely necessary. At APNIC they felt that there is a level of strategic planning and decision-making that goes beyond technology. It is more about business investment and long-term national strategic decisions. For instance, if a vendor comes and says "We have this amazing CGN here, it’s a really cool device and if you buy it you don't have to do IPv6", there is a whole body of practice related to that path. He said that they are interested in the questions, "What kind of network do we want to have? Do we want to have an IPv6 network with end-to-end properties?” In order to answer these questions, one needs to find out where the gap is and how many people out there could do IPv6 if a capital investment decision would be made. Therefore, the work he is ultimately involved with is targeted to people like Heidi who are involved in industry and government planning decisions about what kind of network we want to have in the future.
Martin responded that he is mostly interested in measuring at the BGP and access level. He said that instead of preaching to the converted, they need to go out and encourage network operators to finish the work they sometimes have already started but not completed. Eventually, he would like to see that people run pure IPv6, but there is still a lot of work to be done before that.
Steve responded that they are gathering data for the operators on a day-to-day basis. He said that it is a little worrying that the statistics he showed are slightly lower than those shown by others. This indicates that the operators know less about what's going on at the end-points.
Andrew added that they are mostly aggregating measurements, but also perform some of their own. He stressed that, in the end, it is not just about the protocol itself, but about the structure of the network. The measurements that all parties take collectively can help to make decisions and understand how operators can build their networks to ensure the continuation of the Internet.
Heidi said that she is interested in both sides. As an engineer, her main responsibility is to maintain the technical quality inside Finnish networks. However, as a politician she is mostly interested in when the end users receive IPv6 access and services.
George added that there are still some gaps in what is being done in terms of measurements and wondered if there are questions that aren't asked. He said that there is a poor understanding of the mobile sector and also of emerging Internet economies. They can have a big impact because they are the future markets. Regardless, the industry as a whole has done a good job in sharing the information it’s got.
Marco wondered if the fact that we ran out of IPv4 should change the way measurements are done. Should we, for instance, look more at performance comparisons in IPv4 and IPv6?
Steve said that there are things that need to be fixed, even the measurement aspect itself. He thinks people are not paying perhaps as much attention to gathering data as they should.
Martin agreed and acknowledged that this is a problem. He observed a complete lack of instrumentation and differentiation between what's going on inside people's network in the IPv4 space, and the IPv6 space. He also commented on the business side that George mentioned. If an operator has no visibility in the money spent on CGN infrastructure vs. deploying IPv6, they cannot make the right decision. One has to look at the complete picture, including the internal network, in order to see what value IPv6 can bring. If operators only look at the low percentage of IPv6 traffic, it doesn't provide the full picture. And because of the happy eyeballs, users don't actually know and cannot complain. And that's what is both brilliant and annoying.
Andrew agrees that often the monitoring tools are not in place, and that, at this point, the users must also be included in the troubleshooting tools. Jen said that they should think about what data they needed. Sometimes they just look at the sets of data we are used to. She asked Steve about the concerns people who took the survey said they have such as “Is misconfiguration a higher concern in IPv6 than in IPv4?”
Steve said no, it is at the same level - which is good.
George added that this is good to know and that in fact a lot of the auto-config stuff is working pretty well so people don't have type in an IPv6 address manually.
Andrew agreed and said that sometimes, as professionals, we are so used to the fact that the systems don't work and that we need to fix them by hand, that we don't trust enough that they actually do work. George encouraged people to look at their own measurements. They might be surprised about how much IPv6 they have in their network. This might help to discover opportunities that they were not aware of. Measure to find out where you are, and use it to find out where you want to be. If we did a better job in understanding the mismatch between IPv4 and IPv6 routing, we would be in a better place.
Martin said that he sat down with a vendor’s marketing team and, out of 12 chapters on the plan and equipment, only chapter 7 was about IPv6. He wondered why IPv6 wasn't included in each chapter. Therefore, no special chapter about IPv6 would be needed. Unfortunately the mindset still needs to be “Bring up an IPv4 network and then add IPv6 to it.” This needs to change.
Marco wondered whom they needed to actively convince at this stage? The regulators? The operators? Should we use measures like IPv6 RIPEness and especially the fifth star to let people know that they are at the bottom of the rank and how to change that?
In response, George said that he thinks it is better to compete by talking to each other than by chasing the bottom rank. He suggested people talk to their competitors about they do and don't do and why. They should also talk to their regulators and encourage them to provide incentives for IPv6 deployment. Also, measurement data can help with their planning. Andrew suggested to use large measurements network like RIPE Atlas to measure to do some sort of background probing between the end points that have IPv6. We could check the round trip times between end points and see if it changes over time. We could then graph significant deviations in round trip times.
Heidi, referred back to what George said, and stated that the main reason she is attending this meeting is to hear from ISPs as to what the main issues are. She can then report back to people who are not necessarily following forums like this.
Steve concluded that he would keep watching as things change. Martin stressed once more that a lot of work needs to be done and that the mindset of operators needs to change. IPv6 needs to be seen as an integral part of the network, not just like a test.
Daniel Karrenberg said that some of the background measurements Andrew was suggesting were already being done in RIPE Atlas. For instance, measurements are done from every RIPE Atlas probe that does IPv6 back to the root name servers. The results can be found on the RIPE Atlas web site. The RIPE NCC also started to operate RIPE Atlas anchors at various places in the network that will also do IPv6 measurements. He encouraged people to get involved in RIPE Atlas and configure their own measurements.
AOB:
Wilhelm Boeddinghaus, Strato AG, suggested to keep the number of the RIPE document "Requirements for IPv6 in ICT Equipment (currently ripe-554) the same and add an errata section. This will be further discussed on the IPv6 WG mailing list.
X. Close