IPv6 extension headers support on RIPE Atlas
[ This is a outcome of some online&offline discussions we have had on IETF8 ] There are some ongoing efforts to measure IPv6 extension headers filtering in the Internet (in particular, see Fernando Gont's talk: http://www.iepg.org/2013-11-ietf88/fgont-iepg-ietf88-ipv6-frag-and-eh.pdf ) While individuals are running some experiments from their networks, I think it would be extremely useful to run distributed measurements using Atlas to understand the current situation and possible impact better. Therefore it would be great if Atlas supports: - inserting Ipv6 extension headers into probe packets. Ideally user should be allowed to define a chain of extension headers as specified in RFC2460. In particular, it would be nice to insert the following headers: -- Hop-by-Hop Options -- Fragment -- Destination Options - HTTP measurements (Tim, please add particular requirements if you have any). If anyone else would like to see such features supported by Atlas - please speak up! -- SY, Jen Linkova aka Furry
jen, what, if any, is the danger in doing this? randy
On Wed, Nov 6, 2013 at 2:25 AM, Randy Bush <randy@psg.com> wrote:
jen,
what, if any, is the danger in doing this?
I see your point, Randy. I totally agree that allowing users to craft an arbitrary packet(header) might potentially trigger some bugs in software/hardware of intermediate routers or a target system. To ensure that RIPE Atlas is not used (intentionally or not) as a distributed attack system we could consider: - limiting # of probes simultaneously running such measurements to a low number. - requiring explicit permission of the probe owner to run such UDM. In that case I could not not see much difference between UDM and individuals crafting packets on their own. -- SY, Jen Linkova aka Furry
what, if any, is the danger in doing this? I see your point, Randy.
actually, i did not have a point other than asking if i should worry.
To ensure that RIPE Atlas is not used (intentionally or not) as a distributed attack system we could consider: - limiting # of probes simultaneously running such measurements to a low number. - requiring explicit permission of the probe owner to run such UDM.
this seems prudent, though i worry that the last point does not scale well. randy
On 6 November 2013 15:56, Randy Bush <randy@psg.com> wrote:
To ensure that RIPE Atlas is not used (intentionally or not) as a distributed attack system we could consider: - limiting # of probes simultaneously running such measurements to a low number. - requiring explicit permission of the probe owner to run such UDM.
this seems prudent, though i worry that the last point does not scale well.
Thinking about how we could make this last point scale better... I can see there are situations where some probe owners just wouldn't want to get involved in such measurements at all, and cases where a receptive probe owner may have probes in many different networks (on which they have varying levels of privilege), and therefore while wanting to take part may need probe-by probe granularity to ensure that they don't, for instance, break a customer's network. Probe owners could be given an option to opt-out on an account-wide basis, or opt-in some or all of their probes, but this doesn't automatically mean they consent to every "special" measurement, just that they will consider requests for consent. Then requests asking for consent to use a probe for controversial/risky measurements - containing details of what the measurement is - are sent only to opted-in probe owners. This would avoid the issue of sending requests for consent into a black hole, it's only sent to receptive probe owners. Some probe owners will be very liberal, so they could have an option to a) opt-in all probes and b) auto-consent. Do folk see where I'm going with this? Mike
On Wed, 6 Nov 2013, Mike Hughes wrote:
Do folk see where I'm going with this?
Where in the standards does it say that a packet with any of these headers should/may/must not be used on the Internet? I don't understand the reasoning at all. My opinion is that yes, we perhaps should not create packet generators that send line-rate with small packets with these headers "because we can", but I see little reason not to generally create a test that perhaps does a few pps (or less) of sending these packets (all combinations of headers) to test what happens. There is nothing on the Internet today stopping anyone with IPv6 access to generate these packets, so if it breaks the equipment, it's probably better that someone like RIPE, running Atlas probes, discovers this rather than a script kiddie out there then posting it to full-disclosure. -- Mikael Abrahamsson email: swmike@swm.pp.se
UNSUBSCRIBE REMOVE -----Original message----- From: Mikael Abrahamsson <swmike@swm.pp.se> To: Mike Hughes <mike@smashing.net> Cc: mat-wg@ripe.net Sent: Wed, Nov 6, 2013 11:54:13 CST Subject: Re: [mat-wg] IPv6 extension headers support on RIPE Atlas On Wed, 6 Nov 2013, Mike Hughes wrote:
Do folk see where I'm going with this?
Where in the standards does it say that a packet with any of these headers should/may/must not be used on the Internet? I don't understand the reasoning at all. My opinion is that yes, we perhaps should not create packet generators that send line-rate with small packets with these headers "because we can", but I see little reason not to generally create a test that perhaps does a few pps (or less) of sending these packets (all combinations of headers) to test what happens. There is nothing on the Internet today stopping anyone with IPv6 access to generate these packets, so if it breaks the equipment, it's probably better that someone like RIPE, running Atlas probes, discovers this rather than a script kiddie out there then posting it to full-disclosure. -- Mikael Abrahamsson email: swmike@swm.pp.se
Do folk see where I'm going with this?
Where in the standards does it say that a packet with any of these headers should/may/must not be used on the Internet?
I don't understand the reasoning at all. My opinion is that yes, we perhaps should not create packet generators that send line-rate with small packets with these headers "because we can", but I see little reason not to generally create a test that perhaps does a few pps (or less) of sending these packets (all combinations of headers) to test what happens.
There is nothing on the Internet today stopping anyone with IPv6 access to generate these packets, so if it breaks the equipment, it's probably better that someone like RIPE, running Atlas probes, discovers this rather than a script kiddie out there then posting it to full-disclosure.
wouldn't it be sufficient to support the fragment header initially? hop-by-hop we expect to be severely rate limited anyway, and RH0 is probably blocked. let's be able to test the one that actually matters. cheers, Ole
On Wed, Nov 6, 2013 at 6:53 PM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
Where in the standards does it say that a packet with any of these headers should/may/must not be used on the Internet?
I don't understand the reasoning at all. My opinion is that yes, we perhaps should not create packet generators that send line-rate with small packets with these headers "because we can", but I see little reason not to generally create a test that perhaps does a few pps (or less) of sending these packets (all combinations of headers) to test what happens.
There is nothing on the Internet today stopping anyone with IPv6 access to generate these packets, so if it breaks the equipment, it's probably better that someone like RIPE, running Atlas probes, discovers this rather than a script kiddie out there then posting it to full-disclosure.
My concern is as follows: let's say I create UDM to send packets with some dodgy ext. headers combination, run it from a large number of probes and suddenly trigger a nasty bug crashing routers across the Internet. First of all, distributed nature of the measurement might lead to significant impact on Internet stability comparing to me running such experiment from my local machine at home. That's why I believe # of probes running the test simultaneously should be limited. Secondly, probe hosts might get into trouble for their probes sending "malicious" packets so I think hosts should be aware of potentially risky measurements running on their probes. A host might be willing to cooperate but nobody wants unpleasant surprises. -- SY, Jen Linkova aka Furry
On Thu, 7 Nov 2013, Jen Linkova wrote:
My concern is as follows: let's say I create UDM to send packets with some dodgy ext. headers combination, run it from a large number of probes and suddenly trigger a nasty bug crashing routers across the Internet. First of all, distributed nature of the measurement might lead to significant impact on Internet stability comparing to me running such experiment from my local machine at home. That's why I believe # of probes running the test simultaneously should be limited.
Sure, if this is easy to implement, we could do this initially. Otherwise my thinking was to use TCP or alike anyhow, so if there was a lot of packet loss, there wouldn't be much packets either. I mean, we're talking access to web servers anyhow, right?
Secondly, probe hosts might get into trouble for their probes sending "malicious" packets so I think hosts should be aware of potentially risky measurements running on their probes. A host might be willing to cooperate but nobody wants unpleasant surprises.
Let's start with the fragmentation header and see what happens then. We could do other tests that are destination options at a later date. -- Mikael Abrahamsson email: swmike@swm.pp.se
On 2013.11.06. 18:33, Mike Hughes wrote:
On 6 November 2013 15:56, Randy Bush <randy@psg.com <mailto:randy@psg.com>> wrote:
> To ensure that RIPE Atlas is not used (intentionally or not) as a > distributed attack system we could consider: > - limiting # of probes simultaneously running such measurements to a > low number. > - requiring explicit permission of the probe owner to run such UDM.
this seems prudent, though i worry that the last point does not scale well.
Thinking about how we could make this last point scale better...
I can see there are situations where some probe owners just wouldn't want to get involved in such measurements at all, and cases where a receptive probe owner may have probes in many different networks (on which they have varying levels of privilege), and therefore while wanting to take part may need probe-by probe granularity to ensure that they don't, for instance, break a customer's network.
Probe owners could be given an option to opt-out on an account-wide basis, or opt-in some or all of their probes, but this doesn't automatically mean they consent to every "special" measurement, just that they will consider requests for consent.
With these kind of things it's generally better to use an opt-in approach. At the moment we don't have good statistics about how many of our users (hosts) would read such a call for participation, let alone how many would be willing to opt in. However, we are working on a few bits and pieces that will let hosts describe their probe/installation better (for example marking probes as "home vs backbone", connection type, nat/no-nat etc. -- see the mail thread on the Atlas mailing list). This could be a good indicator about how many hosts we can mobilise for such a call. Regards, Robert
While i see the benefit of such measurements, i'm a little bit worried about turning Atlas into a possible "distributed attack" tool. I think it would be better, at least for the time being, to allow such "packet manipulation" measurements under strict conditions. This might be the very first step of turning Atlas into Scapy-Atlas, which sounds exciting and worrying at the same time. -- Tassos Jen Linkova wrote on 6/11/2013 02:59:
[ This is a outcome of some online&offline discussions we have had on IETF8 ]
There are some ongoing efforts to measure IPv6 extension headers filtering in the Internet (in particular, see Fernando Gont's talk:
http://www.iepg.org/2013-11-ietf88/fgont-iepg-ietf88-ipv6-frag-and-eh.pdf )
While individuals are running some experiments from their networks, I think it would be extremely useful to run distributed measurements using Atlas to understand the current situation and possible impact better.
Therefore it would be great if Atlas supports: - inserting Ipv6 extension headers into probe packets. Ideally user should be allowed to define a chain of extension headers as specified in RFC2460. In particular, it would be nice to insert the following headers: -- Hop-by-Hop Options -- Fragment -- Destination Options
- HTTP measurements (Tim, please add particular requirements if you have any).
If anyone else would like to see such features supported by Atlas - please speak up!
Hi, Just to rewind a bit, the concern over support on-path for IPv6 Extension Headers, esp. the Fragmenttation Header, has always been there, it’s just that with the recent measurements by Fernando and an MSc student here, the numbers are rather worse than many people expected, moreso the longer the EH used. Fernando did the test extending his own IPv6 toolset, our student used scapy (which supports the four main EHs) to craft HTTP requests. We both targeted the top Alexa sites. The Fragmentation Header success rate across the sites tested was at best under 70%. The interest with Atlas is that we could test between participating sites, rather than just to Alexa-ranked web servers, which is more interesting esp. bearing in mind the desire to build IPv6 p2p apps. As for HTTP tests - I asked a few weeks ago about using the probes to do general HTTP measurements, and was told they were only possible for a limited set of users. No problem with that, but I simply guessed from that response that expecting the probes to do IPv6 EHs without significant changes was optimistic. And I certainly understand the security-related concerns. I have a different student now working further on the tests, which may include adding extra EH capability to scapy, and possible ‘looking glass’ style tests. I think Fernando is also exending his tests as he was only checking with one EH type. So we should get more/better information soon regardless, and we may get better clues as to how Atlas could help. I think this is an important issue for IPv6, witness the current discussions and I-Ds on EH handling, header lengths, etc. Best wishes, Tim On 6 Nov 2013, at 07:08, Tassos Chatzithomaoglou <achatz@forthnet.gr> wrote:
While i see the benefit of such measurements, i'm a little bit worried about turning Atlas into a possible "distributed attack" tool. I think it would be better, at least for the time being, to allow such "packet manipulation" measurements under strict conditions. This might be the very first step of turning Atlas into Scapy-Atlas, which sounds exciting and worrying at the same time.
-- Tassos
Jen Linkova wrote on 6/11/2013 02:59:
[ This is a outcome of some online&offline discussions we have had on IETF8 ]
There are some ongoing efforts to measure IPv6 extension headers filtering in the Internet (in particular, see Fernando Gont's talk:
http://www.iepg.org/2013-11-ietf88/fgont-iepg-ietf88-ipv6-frag-and-eh.pdf )
While individuals are running some experiments from their networks, I think it would be extremely useful to run distributed measurements using Atlas to understand the current situation and possible impact better.
Therefore it would be great if Atlas supports: - inserting Ipv6 extension headers into probe packets. Ideally user should be allowed to define a chain of extension headers as specified in RFC2460. In particular, it would be nice to insert the following headers: -- Hop-by-Hop Options -- Fragment -- Destination Options
- HTTP measurements (Tim, please add particular requirements if you have any).
If anyone else would like to see such features supported by Atlas - please speak up!
On 2013/11/06 8:27 , Tim Chown wrote:
Just to rewind a bit, the concern over support on-path for IPv6 Extension Headers, esp. the Fragmenttation Header, has always been there, it’s just that with the recent measurements by Fernando and an MSc student here, the numbers are rather worse than many people expected, moreso the longer the EH used. Fernando did the test extending his own IPv6 toolset, our student used scapy (which supports the four main EHs) to craft HTTP requests. We both targeted the top Alexa sites. The Fragmentation Header success rate across the sites tested was at best under 70%.
The interest with Atlas is that we could test between participating sites, rather than just to Alexa-ranked web servers, which is more interesting esp. bearing in mind the desire to build IPv6 p2p apps.
As for HTTP tests - I asked a few weeks ago about using the probes to do general HTTP measurements, and was told they were only possible for a limited set of users. No problem with that, but I simply guessed from that response that expecting the probes to do IPv6 EHs without significant changes was optimistic. And I certainly understand the security-related concerns.
I have a different student now working further on the tests, which may include adding extra EH capability to scapy, and possible ‘looking glass’ style tests. I think Fernando is also exending his tests as he was only checking with one EH type. So we should get more/better information soon regardless, and we may get better clues as to how Atlas could help. I think this is an important issue for IPv6, witness the current discussions and I-Ds on EH handling, header lengths, etc.
A couple of remarks: NLNet Labs used RIPE Atlas to measure fragmentation issues in the context of DNS. Though the error rate is high (around 10%) it is nowhere near 30%. It might be worth investigation whether the filtering happens at the end site or not. I can imagine that adding fragmentation support to http load balancers is something you do when you have really nothing else to do. In that case measuring fragmentation of tcp packets might give a rather pessimistic result. There are various reasons why RIPE Atlas cannot provide an interface that allows the user to specify arbitrary packets going out and reporting everything that is received. The first one is a security issue and the second leads to storage problems on the older probes. For this reason it would be nice if feature requests can be both minimal and specific. I.e. what is the minimal amount of reporting you need to measure what you want to measure and what is a safe way to generate the packets you need. In this context, would it be sufficient to have hop-by-hop or destination options that just consist of pad options? Hopefully somebody from the operator community can guess whether unlimited use of hop-by-hop options filled with padding would be safe. (By the way, I have no idea how routers are supposed to support hop-by-hop options for unicast packets. Does that do anything useful other than trying to DoS the router?) Finally, RIPE Atlas is not a mesh network. A subset is (the Atlas Anchors, but that part is very small at the moment). So using Atlas gives you more sources of traffic, but not more destinations. Philip Homburg
op 06-11-13 03:01, Philip Homburg schreef:
On 2013/11/06 8:27 , Tim Chown wrote:
Just to rewind a bit, the concern over support on-path for IPv6 Extension Headers, esp. the Fragmenttation Header, has always been there, it’s just that with the recent measurements by Fernando and an MSc student here, the numbers are rather worse than many people expected, moreso the longer the EH used. Fernando did the test extending his own IPv6 toolset, our student used scapy (which supports the four main EHs) to craft HTTP requests. We both targeted the top Alexa sites. The Fragmentation Header success rate across the sites tested was at best under 70%. A couple of remarks:
NLNet Labs used RIPE Atlas to measure fragmentation issues in the context of DNS. Though the error rate is high (around 10%) it is nowhere near 30%. Things seems to get better over time. Our most recent measurements (Oct 2013) showed that 6.9% of all 1059 IPv6 probes were not able to receive fragments. See slide 21 of https://ripe67.ripe.net/presentations/230-pmtud4dns.pdf
We measured it by querying for a larger DNS record that will be answered fragmented at 1280 MTU. This is a more realistic setting than fragmented TCP which shouldn't occur in the wild, right? We measured the ability of probes (client side of DNS) to receive fragments over UDP. With Emile (10% IPv6 frag filters, https://labs.ripe.net/Members/emileaben/ripe-atlas-packet-size-matters ), the ability of the probe to sent out and**receive fragmented ICMP messages was measured . The direction of the traffic and protocols used seems to matter... Currently we have a student working on a framework that can track these network properties over time. It is our intention to make the framework open source so others can review and redo our measurements exactly as we did them ourselves, and can build upon it (and contribute back :).
What is RH0 and UDM?
On Thu, Nov 7, 2013 at 8:10 AM, Shawn Wilson <ag4ve.us@gmail.com> wrote:
What is RH0 and UDM?
RH0 is Type 0 Routing Header, deprecated by rfc5095. UDM - User-Defined Measurements -- SY, Jen Linkova aka Furry
Hi Philip, On Wed, Nov 6, 2013 at 12:01 PM, Philip Homburg <philip.homburg@ripe.net> wrote:
There are various reasons why RIPE Atlas cannot provide an interface that allows the user to specify arbitrary packets going out and reporting everything that is received. The first one is a security issue and the second leads to storage problems on the older probes. For this reason it would be nice if feature requests can be both minimal and specific. I.e. what is the minimal amount of reporting you need to measure what you want to measure and what is a safe way to generate the packets you need.
Sure, we can work on more detailed feature request indeed if we agree with the idea of extension header support in principle. Regarding the storage problem on the old probes: as it's being discussed, it might be a good idea to allow probe owners to opt-in for 'risky' UDMs so old probes could be excluded.
In this context, would it be sufficient to have hop-by-hop or destination options that just consist of pad options? Hopefully somebody from the operator community can guess whether unlimited use of hop-by-hop options filled with padding would be safe.
I'd say that using hop-by-hop header is potentially more risky than destination/fragmentation headers. IMHO the priority list looks like: - destination and fragment headers (with pad1/padN support for destination option); - hop-by-hop header - but we might need to consider how to minimize the risk.
(By the way, I have no idea how routers are supposed to support hop-by-hop options for unicast packets. Does that do anything useful other than trying to DoS the router?)
Off top of my head: RSVP. In addition, there are some other possible use cases being discussed currently.
Finally, RIPE Atlas is not a mesh network. A subset is (the Atlas Anchors, but that part is very small at the moment). So using Atlas gives you more sources of traffic, but not more destinations.
Even getting more sources is very useful indeed.. -- SY, Jen Linkova aka Furry
Hi, I support such kind of measurements. On 6/11/2013 2:59 πμ, Jen Linkova wrote:
[ This is a outcome of some online&offline discussions we have had on IETF8 ]
There are some ongoing efforts to measure IPv6 extension headers filtering in the Internet (in particular, see Fernando Gont's talk:
http://www.iepg.org/2013-11-ietf88/fgont-iepg-ietf88-ipv6-frag-and-eh.pdf )
While individuals are running some experiments from their networks, I think it would be extremely useful to run distributed measurements using Atlas to understand the current situation and possible impact better.
Therefore it would be great if Atlas supports: - inserting Ipv6 extension headers into probe packets. Ideally user should be allowed to define a chain of extension headers as specified in RFC2460. In particular, it would be nice to insert the following headers: -- Hop-by-Hop Options -- Fragment -- Destination Options
- HTTP measurements (Tim, please add particular requirements if you have any).
If anyone else would like to see such features supported by Atlas - please speak up!
-- Dimitrios K. Kalogeras Electrical Engineer Ph.D. Network Engineer NTUA/GR-Net Network Management Center _____________________________________ skype: aweboy voice: +30-210-772 1863 fax: +30-210-772 1866 e-mail: D.Kalogeras@noc.ntua.gr
Jen,
Therefore it would be great if Atlas supports: - inserting Ipv6 extension headers into probe packets. Ideally user should be allowed to define a chain of extension headers as specified in RFC2460. In particular, it would be nice to insert the following headers: -- Hop-by-Hop Options -- Fragment -- Destination Options
- HTTP measurements (Tim, please add particular requirements if you have any).
If anyone else would like to see such features supported by Atlas - please speak up!
This would be really useful. Any protocol that supports extensions is subject to being blocked somewhere. Your example mentions IPv6 extensions, but the same problem occurs with any TCP option or TCP extension that can be blocked or modified by middleboxes on a path. The only possibility to verify whether a new TCP extension (ECN, TFO, SACK, Multipath TCP, ...) can be safely deployed is to perform measurements from a large number of sources. One possibility to allow RIPE Atlas users to perform such measurements in a controlled environment would be to port a tool like tracebox on RIPE Atlas. tracebox works like traceroute but allows to send any type of packet (v4,v6, tcp, udp, ...) and observes the interference caused by middleboxes the path towards a given destination. The tool has been presented at IMC recently and you can find additional details on http://www.tracebox.org We would be interested in porting tracebox on the RIPE Atlas platform if this meets the needs from the Atlas community Olivier -- INL, ICTEAM, UCLouvain, Belgium, http://inl.info.ucl.ac.be
Dear RIPE Atlas users, Jen, we have followed the discussion about IPv6 extension headers, and this is our understanding of what you would want us to implement -- please send any comments or suggestions to the list! On 06-nov.-13 01:59, Jen Linkova wrote:
There are some ongoing efforts to measure IPv6 extension headers filtering in the Internet [...] Therefore it would be great if Atlas supports: - inserting Ipv6 extension headers into probe packets. Ideally user should be allowed to define a chain of extension headers as specified in RFC2460. In particular, it would be nice to insert the following headers: -- Fragment
This is already possible to do: - specify "ping6" measurement with the packet larger then 1500 - specify "traceroute6", with the desirable packet size, and with an optional "do not fragment" flag that sets "no fragmentation" (which you should skip if you want fragmentation) I am curious if you would require more capabilities, and which ones? Or more documentation? (Emile did some research about it: https://labs.ripe.net/Members/emileaben/ripe-atlas-packet-size-matters )
-- Destination Options
The way we want to implement this is: user can specify the size of the extension header, but we will fill it with padding, as specified in the RFC. We judge this option to be "safe", and we will start implementing this, in the new version of the probe firmware.
-- Hop-by-Hop Options
Althou there were some concerns that this option can have impact on the routers on the path of the packets with such headers, we consider it "safe" enough to be added to the measurements, and we will work on implementing it, the same way as with "Destination Options". Since these features need to be added in the firmware, it will take a bit longer till the new FW is released & propagated, and added to the API & user interface. Due to upcoming holiday seasons, please allow even longer time till you actually see it in production.
- HTTP measurements (Tim, please add particular requirements if you have any).
(This is a totally different topic, that requires its own thread, and I will come back to this one at the later date. ) Please let us know if you still have any concerns about our plans to deploy these IPv6 related features. Regards, Vesna
participants (15)
-
Brian Buckles -
Dimitris Kalogeras -
Jen Linkova -
Mikael Abrahamsson -
Mike Hughes -
Ole Troan -
Olivier Bonaventure -
Philip Homburg -
Randy Bush -
Robert Kisteleki -
Shawn Wilson -
Tassos Chatzithomaoglou -
Tim Chown -
Vesna Manojlovic -
Willem Toorop