[ipv6-wg@ripe.net] IXP networks routing
Hello, We hesitate to allow IXP's /48 in our routing table. http://www.ripe.net/ripe/docs/ipv6-policy-ixp.html => "Networks assigned under this policy may not be globally routable." What's the best practice ? Thanks Best Regards, -- Nicolas DEFFAYET, NDSoftware NDSoftware NOC: http://noc.ndsoftwarenet.com/ FNIX6 (French National Internet Exchange IPv6): http://www.fnix6.net/ EuroNOG: http://www.euronog.org/
Dear diary, on Tue, Feb 25, 2003 at 03:53:42PM CET, I got a letter, where Nicolas DEFFAYET <nicolas.deffayet@ndsoftware.net> told me, that...
Hello,
Hello,
We hesitate to allow IXP's /48 in our routing table.
http://www.ripe.net/ripe/docs/ipv6-policy-ixp.html => "Networks assigned under this policy may not be globally routable."
What's the best practice ?
AFAIK they should not be globally routable and they are only for internal usage of the exchange points. See also http://www.apnic.net/meetings/12/docs/proposal-ipv6-ixp.html which specifically discourages announcing of these prefixes. Kind regards, -- Petr "Pasky the Kicker" Baudis . "A computer is a state machine. Threads are for people who can't program state machines." -- Alan Cox . Crap: http://pasky.ji.cz/
On Tue, Feb 25, 2003 at 04:11:57PM +0100, Petr Baudis <pasky@pasky.ji.cz> wrote a message of 29 lines which said:
AFAIK they should not be globally routable and they are only for internal usage of the exchange points.
Very bad idea. 1) When you traceroute through an exchange point, the IP addresses you get should be routable, for easier debugging. 2) Path MTU discovery may depend on it. Having IXP addresses not globally routable is as wrong as having RFC 1918 (or FECO::) addresses at an IXP.
Dear diary, on Tue, Feb 25, 2003 at 04:20:35PM CET, I got a letter, where Stephane Bortzmeyer <bortzmeyer@gitoyen.net> told me, that...
On Tue, Feb 25, 2003 at 04:11:57PM +0100, Petr Baudis <pasky@pasky.ji.cz> wrote a message of 29 lines which said:
AFAIK they should not be globally routable and they are only for internal usage of the exchange points.
Very bad idea.
1) When you traceroute through an exchange point, the IP addresses you get should be routable, for easier debugging.
2) Path MTU discovery may depend on it.
You should not drop the packets, noone says that. You should route them to the destination, they do not have to be routable back to the originator. Ad (1), as also some people said at IRC, you shoud not depend on the intermediate addresses having same path as your target address. Also the border routers should have probably also assigned addresses from the respective site they belong to. Ad (2), I can't see how it could harm the MTU discovery --- you get ICMP Can't Fragment from some node, but it doesn't matter that the node's address is not routable, does it?
Having IXP addresses not globally routable is as wrong as having RFC 1918 (or FECO::) addresses at an IXP.
There are heated discussions about site-local addresses and so on, the main problem is the danger of several such "sites" meeting at one point and the resulting mess ;-) (also, if you are using site-local addresses in your network, you could confuse the originating node of the packet when receiving something from the IX router). Having the address space unique has obvious advantages even if it's not globally routable. Kind regards, -- Petr "Pasky" Baudis . "A computer is a state machine. Threads are for people who can't program state machines." -- Alan Cox . Crap: http://pasky.ji.cz/
On Tue, 2003-02-25 at 16:20, Stephane Bortzmeyer wrote:
On Tue, Feb 25, 2003 at 04:11:57PM +0100, Petr Baudis <pasky@pasky.ji.cz> wrote a message of 29 lines which said:
AFAIK they should not be globally routable and they are only for internal usage of the exchange points.
Very bad idea.
1) When you traceroute through an exchange point, the IP addresses you get should be routable, for easier debugging.
2) Path MTU discovery may depend on it.
Having IXP addresses not globally routable is as wrong as having RFC 1918 (or FECO::) addresses at an IXP.
No, very GOOD idea. We block all inbound route announcements of IXP prefixes we are connected to on IPv4 for a very good reason, and I don't see what has changed from IPv4 to IPv6 that should make our routing policy any different for IPv6 IXP-space. Hint - a lot of networks don't use "next-hop-self" on their border routers toward iBGP peers and instead carry a route for the IXP block in their IGP so other BGP speakers have next-hop reachability. Some networks, unfortunately, also do NOT make sure to filter said IXP blocks properly. What do you think happens when suddenly a route for that IXP block leaks in on some peering session (on a different router than the one actually connected to the IXP of course), and eBGP has a lower administrative distance (preference) than the IGP - which is usually the default? Can you guess what happens to the traffic that should be going to a next-hop from that IXP-block? Yes, I've seen this very effect on several networks I've had the joy of troubleshooting, and could share some interesting stories on that topic. As for your specific objections: 1) Why does this make for easier debugging? You get a perfectly valid ICMP reply from a valid IP address that you should be able to resolve in DNS without any problems as it is unique (global unicast). If you mean you would like to use the actual IXP IP-address as a destination for a traceroute, well - don't. You shouldn't need to anyway. And unless you are one of the peers I am speaking to over said IXP, you have no valid reason in my book to send any traffic to my border-router on that IP-address, and I don't see why I should let you. Hey, services like echo and chargen were once considered very useful debugging tools too - that doesn't mean anyone would recommend having said services open to the world nowadays. 2) How would lack of routability of this space break path MTU discovery? Please show me where in RFC1191 or RFC1981 it says that you would need to send any packets addressed directly to the IP who sent you a "Datagram Too Big/Packet Too Big" ICMP message in either IPv4 or IPv6, respectively, for pMTUd to work. /leg
Hi, as an IXP-Operator I would like to comment this in three ways: 1. Requesting an Address-space be it v4 or v6 for an IXP-Infrastructure and not making it globally reachable somehow misses the point. It should be clearly reachable - ideally through *all* connected peers of the infrastructure. (they certainly can decide on the security for these destinations) 2. Address-space differs from IXP to ISP substiantially. ISPs hand out IP-addresses to customers and IXPs assign single (or *very* few) addresses to ISPs. That means that address consumption and renewals are very rare. Even the default allocations from the IRRs for IXPs is - to my opinion - far too large. 3. Same with the members fee. Ok, I am speaking for a small IXP, but a RIPE membership cannot be afforded right now. There is in contradiction the need for 1 single AS-number and one small prefix the cost which is normally calculated for the untrained new LIR ISP, who needs training, hostmaster-help, etc. Why not add a special categoy for IXP demands. There is a small number of them (50 in Europe?) and basicall NO effort after giving them their numbers for work. Best regards, Kurt Kayser Lars Erik Gullerud wrote:
On Tue, 2003-02-25 at 16:20, Stephane Bortzmeyer wrote:
On Tue, Feb 25, 2003 at 04:11:57PM +0100, Petr Baudis <pasky@pasky.ji.cz> wrote a message of 29 lines which said:
AFAIK they should not be globally routable and they are only for internal usage of the exchange points.
Very bad idea.
1) When you traceroute through an exchange point, the IP addresses you get should be routable, for easier debugging.
2) Path MTU discovery may depend on it.
Having IXP addresses not globally routable is as wrong as having RFC 1918 (or FECO::) addresses at an IXP.
No, very GOOD idea.
We block all inbound route announcements of IXP prefixes we are connected to on IPv4 for a very good reason, and I don't see what has changed from IPv4 to IPv6 that should make our routing policy any different for IPv6 IXP-space.
Hint - a lot of networks don't use "next-hop-self" on their border routers toward iBGP peers and instead carry a route for the IXP block in their IGP so other BGP speakers have next-hop reachability. Some networks, unfortunately, also do NOT make sure to filter said IXP blocks properly. What do you think happens when suddenly a route for that IXP block leaks in on some peering session (on a different router than the one actually connected to the IXP of course), and eBGP has a lower administrative distance (preference) than the IGP - which is usually the default? Can you guess what happens to the traffic that should be going to a next-hop from that IXP-block? Yes, I've seen this very effect on several networks I've had the joy of troubleshooting, and could share some interesting stories on that topic.
As for your specific objections:
1) Why does this make for easier debugging? You get a perfectly valid ICMP reply from a valid IP address that you should be able to resolve in DNS without any problems as it is unique (global unicast). If you mean you would like to use the actual IXP IP-address as a destination for a traceroute, well - don't. You shouldn't need to anyway. And unless you are one of the peers I am speaking to over said IXP, you have no valid reason in my book to send any traffic to my border-router on that IP-address, and I don't see why I should let you. Hey, services like echo and chargen were once considered very useful debugging tools too - that doesn't mean anyone would recommend having said services open to the world nowadays.
2) How would lack of routability of this space break path MTU discovery? Please show me where in RFC1191 or RFC1981 it says that you would need to send any packets addressed directly to the IP who sent you a "Datagram Too Big/Packet Too Big" ICMP message in either IPv4 or IPv6, respectively, for pMTUd to work.
/leg
-- +++ Kurt Kayser Consulting - ISP & Carrier Netzwerkdesign, Planung, Schulungen *** Heinrich-M�ller-Str. 1c, 90530 R�thenbach b.St. Wolfgang, Germany *** Tel: +49 (0) 9129 289315, Fax: +49 (0) 9129 289316, Mobil: +49 (0) 160 5810284
On Wed, Feb 26, 2003 at 02:10:41PM +0100, Kurt Kayser wrote: | Hi, | | as an IXP-Operator I would like to comment this in three ways: Hoi Kurt, | 1. Requesting an Address-space be it v4 or v6 for an IXP-Infrastructure | and not making it globally reachable somehow misses the point. It should | be clearly reachable - ideally through *all* connected peers of the | infrastructure. | (they certainly can decide on the security for these destinations) You probably are using this /48 out of 2001:7f8::/32 with the wrong reasons. It should be used for peering meshes and not for services at your IXP, and therefor it should not have to be routable or globally visible. If you want PI space for your IXP to run services in, you can join the long list of people that would like to have PI space in IPv6, which is simply not possible at this point in time. | 2. Address-space differs from IXP to ISP substiantially. ISPs hand out | IP-addresses | to customers and IXPs assign single (or *very* few) addresses to ISPs. That | means | that address consumption and renewals are very rare. Even the default | allocations | from the IRRs for IXPs is - to my opinion - far too large. The standard allocation size as per common practice at this point is either /64 or /48. More types of sizes are being debated all the time, but to this day, no other sizes have been established. A /64 might be too small for IXPs with more than one peering mesh, so the next step up is /48. | 3. Same with the members fee. Ok, I am speaking for a small IXP, but a RIPE | membership | cannot be afforded right now. There is in contradiction the need for 1 | single AS-number | and one small prefix the cost which is normally calculated for the | untrained new LIR | ISP, who needs training, hostmaster-help, etc. Why not add a special | categoy for IXP | demands. There is a small number of them (50 in Europe?) and basicall NO | effort after | giving them their numbers for work. You cannot identify your IXP as a special pig in the race of pigs. Therefor, no exception should be made for you, or RIRs, or any other enterprise. What would happen if every enterprise started an IXP and claimed a right to their own PI space ? It would become a mess! To put it frank: go to an upstream and request a block of 'PA' from their space to run your services in, and let them aggregate your traffic. If you want independability, go to multiple upstreams. groet, Pim -- __________________ Met vriendelijke groet, /\ ___/ Pim van Pelt /- \ _/ Business Internet Trends BV PBVP1-RIPE /--- \/ __________________
Hi Pim (and mailing-list observers, of course :), please see my inline comments. Pim van Pelt wrote:
On Wed, Feb 26, 2003 at 02:10:41PM +0100, Kurt Kayser wrote: | 1. Requesting an Address-space be it v4 or v6 for an IXP-Infrastructure | and not making it globally reachable somehow misses the point. It should | be clearly reachable - ideally through *all* connected peers of the | infrastructure. | (they certainly can decide on the security for these destinations) You probably are using this /48 out of 2001:7f8::/32 with the wrong reasons. It should be used for peering meshes and not for services at your IXP, and therefor it should not have to be routable or globally visible.
We're currently only running v4 with PA-space, which is blocked by the LIR from whom we borrowed it. So w.r.t. v6, there are just link-local tests ongoing.
If you want PI space for your IXP to run services in, you can join the long list of people that would like to have PI space in IPv6, which is simply not possible at this point in time.
Agreed, that would be the way to go for me as well.
| 2. Address-space differs from IXP to ISP substiantially. ISPs hand out | IP-addresses | to customers and IXPs assign single (or *very* few) addresses to ISPs. That | means | that address consumption and renewals are very rare. Even the default | allocations | from the IRRs for IXPs is - to my opinion - far too large. The standard allocation size as per common practice at this point is either /64 or /48. More types of sizes are being debated all the time, but to this day, no other sizes have been established. A /64 might be too small for IXPs with more than one peering mesh, so the next step up is /48.
I'm again referring currently more to v4, since there are /19s or /20s default for new LIRs.
| 3. Same with the members fee. Ok, I am speaking for a small IXP, but a RIPE | membership | cannot be afforded right now. There is in contradiction the need for 1 | single AS-number | and one small prefix the cost which is normally calculated for the | untrained new LIR | ISP, who needs training, hostmaster-help, etc. Why not add a special | categoy for IXP | demands. There is a small number of them (50 in Europe?) and basicall NO | effort after | giving them their numbers for work. You cannot identify your IXP as a special pig in the race of pigs. Therefor, no exception should be made for you, or RIRs, or any other enterprise. What would happen if every enterprise started an IXP and claimed a right to their own PI space ? It would become a mess!
I do not consider myself a pig, so please let's keep this discussion on a serious level, ok? I sincerely believe in NEW IXP-members (not for free of course!), but with special conditions (no support, just one AS and one prefix) for the IXP-setup. I also believe in the RIPE-NCC to be able to distinguish between enterprises that might request a status of an IXP. It all depends of the policy and conditions.
To put it frank: go to an upstream and request a block of 'PA' from their space to run your services in, and let them aggregate your traffic. If you want independability, go to multiple upstreams.
First part has happened so far, but traffic for the network is a problem (basically who pays the upstream-ISP for the traffic) and connectivity is just not there. So it's a very bad hack for local connectivity, which could be greatly improved, if there are mechanisms in place that would allow this. .kurt -- +++ Kurt Kayser Consulting - ISP & Carrier Netzwerkdesign, Planung, Schulungen *** Heinrich-M�ller-Str. 1c, 90530 R�thenbach b.St. Wolfgang, Germany *** Tel: +49 (0) 9129 289315, Fax: +49 (0) 9129 289316, Mobil: +49 (0) 160 5810284
Hi Kurt, Hi Pim, and everybody who listens, I would like to support Kurt�s request for inventing an IXP membership. Today the ixp�s are a vital part of the global infrastructure, but they are not represented as a legal member of ripe. Sure we can come to every meeting and can do nearly all the things the LIR�s are doing, but it is always some kind of guest status.
I also believe in the RIPE-NCC to be able to distinguish between enterprises that might request a status of an IXP. It all depends of the policy and conditions. This is very simple. Have a lock at the peering matrix and you can distinguish between a enterprise and an IXP (or am i wrong here?).
greetings - Stefan Wahl
On 02-03-2003 16:12PM, "Stefan Wahl" <swahl@netsign.de> wrote:
I also believe in the RIPE-NCC to be able to distinguish between enterprises that might request a status of an IXP. It all depends of the policy and conditions. This is very simple. Have a lock at the peering matrix and you can distinguish between a enterprise and an IXP (or am i wrong here?).
Well the fears are probably not concerning existing IXPs. The fear is that setting up an (fake) IXP becomes a way to get independent address space. Naturally, we do not want to create a chicken and egg situation when you limit "IXP-space" to well established IXPs only. Arien
Hi, On Tue, Mar 04, 2003 at 10:37:27AM +0100, Arien Vijn wrote:
Well the fears are probably not concerning existing IXPs. The fear is that setting up an (fake) IXP becomes a way to get independent address space.
Yep. This is what I tried to point out. I'm not fundamentally opposing anything here, I just point out "difficult issues". Gert Doering -- NetMaster -- Total number of prefixes smaller than registry allocations: 57021 (57147) SpaceNet AG Mail: netmaster@Space.Net Joseph-Dollinger-Bogen 14 Tel : +49-89-32356-0 80807 Muenchen Fax : +49-89-32356-299
I would like to support Kurt´s request for inventing an IXP membership. Today the ixp´s are a vital part of the global infrastructure, but they are not represented as a legal member of ripe. Sure we can come to every meeting and can do nearly all the things the LIR´s are doing, but it is always some kind of guest status.
Uhm, we are a IX and we are a member? All you need to do is pay the membership fee. - kurtis -
Hi, On Sun, Mar 02, 2003 at 04:12:58PM +0100, Stefan Wahl wrote:
I would like to support Kurt�s request for inventing an IXP membership. Today the ixp�s are a vital part of the global infrastructure, but they are not represented as a legal member of ripe. Sure we can come to every meeting and can do nearly all the things the LIR�s are doing, but it is always some kind of guest status.
I don't understand this. The RIPE meetings, and the policy making working group (lir-wg) are open to everybody. This is not tied to being a LIR. What kind of disadvantage to you see for the IXPs here? [..]
I also believe in the RIPE-NCC to be able to distinguish between enterprises that might request a status of an IXP. It all depends of the policy and conditions. This is very simple. Have a lock at the peering matrix and you can distinguish between a enterprise and an IXP (or am i wrong here?).
So is C&W (just to pick one example) an enterprise or an IXP? Yes, I am nitpicking, but if you want to introduce special categories, you should be able to specify the criteria pretty well. Gert Doering -- NetMaster -- Total number of prefixes smaller than registry allocations: 57021 (57147) SpaceNet AG Mail: netmaster@Space.Net Joseph-Dollinger-Bogen 14 Tel : +49-89-32356-0 80807 Muenchen Fax : +49-89-32356-299
[...]
[..]
I also believe in the RIPE-NCC to be able to distinguish between enterprises that might request a status of an IXP. It all depends of the policy and conditions. This is very simple. Have a lock at the peering matrix and you can distinguish between a enterprise and an IXP (or am i wrong here?).
So is C&W (just to pick one example) an enterprise or an IXP?
C&W is a enterprise. Period. They operate at least 2 IXP's, but if Enterprises operate a IXP or anyone who operates / will operate an IXP should taken care of. We should work on policies for the IXP-IPV6-SPACE ... (anyhow, gert, isn't SPACE in SpaceNet *SCNR*)
Yes, I am nitpicking, but if you want to introduce special categories, you should be able to specify the criteria pretty well.
But it's better that we do currently nitpicking than filling holes later on :-) --jan -- Jan Ahrent Czmok - Senior Network Engineer - Access Networks Global Access Telecommunications, Inc. - Stephanstr. 3 - 60313 Frankfurt voice: +49 69 299896-35 - fax: +49 69 299896-66 - email: czmok@gatel.de
Kurt Kayser (kurt_kayser@gmx.de) wrote:
Hi Pim (and mailing-list observers, of course :),
please see my inline comments.
Pim van Pelt wrote:
On Wed, Feb 26, 2003 at 02:10:41PM +0100, Kurt Kayser wrote: | 1. Requesting an Address-space be it v4 or v6 for an IXP-Infrastructure | and not making it globally reachable somehow misses the point. It should | be clearly reachable - ideally through *all* connected peers of the | infrastructure. | (they certainly can decide on the security for these destinations) You probably are using this /48 out of 2001:7f8::/32 with the wrong reasons. It should be used for peering meshes and not for services at your IXP, and therefor it should not have to be routable or globally visible.
We're currently only running v4 with PA-space, which is blocked by the LIR from whom we borrowed it. So w.r.t. v6, there are just link-local tests ongoing.
Hi Kurt, Hi Pim, Isn't it always these PA / PI Discussion about the same issues ? In regards to the IPv6, what is a IXP offering ? (in technical sense): - Classic Services (e.g. providing Interconnect possibilities) - (sometimes) colocation space - New Services: (e.g. Tunnel Broker (?), Gateway (v4/v6) Multicast/Anycast ?, DiffServ, Billing(?) ... Therefore i would like to see that we are really doing back to a technical level to discuss possibilities and further development in the EIX stuff, not just bashing around with pigs and irons and politics. Many people tend to do this times, but we all should stick our head together and create services / ideas for the future.
If you want PI space for your IXP to run services in, you can join the long list of people that would like to have PI space in IPv6, which is simply not possible at this point in time.
I would not rather call it PI space.. why not defining and using IXP-space (with cooperation from the RIR (like RIPE/APNIC/ARIN etc.)
| 2. Address-space differs from IXP to ISP substiantially. ISPs hand out | IP-addresses | to customers and IXPs assign single (or *very* few) addresses to ISPs. That | means | that address consumption and renewals are very rare. Even the default | allocations | from the IRRs for IXPs is - to my opinion - far too large. The standard allocation size as per common practice at this point is either /64 or /48. More types of sizes are being debated all the time, but to this day, no other sizes have been established. A /64 might be too small for IXPs with more than one peering mesh, so the next step up is /48.
I'm again referring currently more to v4, since there are /19s or /20s default for new LIRs.
LIR != ISP != IXP !!!
| 3. Same with the members fee. Ok, I am speaking for a small IXP, but a RIPE | membership | cannot be afforded right now. There is in contradiction the need for 1 | single AS-number | and one small prefix the cost which is normally calculated for the | untrained new LIR | ISP, who needs training, hostmaster-help, etc. Why not add a special | categoy for IXP | demands. There is a small number of them (50 in Europe?) and basicall NO | effort after | giving them their numbers for work. You cannot identify your IXP as a special pig in the race of pigs. Therefor, no exception should be made for you, or RIRs, or any other enterprise. What would happen if every enterprise started an IXP and claimed a right to their own PI space ? It would become a mess!
Come on. Calm down. Think! Ripe is surely educated enough to differentiate between a Enterprise, ISP or IXP. To see real _further_ development, we should really propose to RIPE to offer a "IXP Membership" - these would help small IXPs to grow and to attract. And they are independant of ANY ISP or CARRIER, which is not good IMHO... Also we should see that we come to a more technical level on the RIPE Meetings. If compared to the NANOG Meetings (which are AFAIK different) i see more output and help from the talks at NANOG than at RIPE. At RIPE it has come to a tooo political level, which is really stopping things at the moment. But it's just my 0.2 EuroCent... --jan -- Jan Ahrent Czmok - Senior Network Engineer - Access Networks Global Access Telecommunications, Inc. - Stephanstr. 3 - 60313 Frankfurt voice: +49 69 299896-35 - fax: +49 69 299896-66 - email: czmok@gatel.de
On 25-02-2003 15:53PM, "Nicolas DEFFAYET" <nicolas.deffayet@ndsoftware.net> wrote:
We hesitate to allow IXP's /48 in our routing table.
http://www.ripe.net/ripe/docs/ipv6-policy-ixp.html => "Networks assigned under this policy may not be globally routable."
What's the best practice ?
We announce it with the no-export community string. Don't know if it's best practice but it does seem to work all-right. We do it the same way in IPv4. Arien -- Arien Vijn Amsterdam Internet Exchange http://www.ams-ix.net
participants (11)
-
Arien Vijn -
Gert Doering -
Jan Czmok -
Kurt Erik Lindqvist -
Kurt Kayser -
Lars Erik Gullerud -
Nicolas DEFFAYET -
Petr Baudis -
Pim van Pelt -
Stefan Wahl -
Stephane Bortzmeyer