Dear EIXers, I thought I'd expand on my comments in the EIX session earlier this week. In a network that peers over an exchange, you'll see routes like this in the BGP table: Network Next Hop Metric LocPrf Weight Path *>i62.3.192.0/18 193.148.15.112 200 110 0 6728 i *>i62.8.128.0/17 193.148.15.209 110 0 9132 ? *>i62.12.128.0/17 193.148.15.224 1100 110 0 15623 i Note that the next hop points to an address that falls within the (old) AMS-IX peering LAN prefix. Since this router isn't directly connected to the AMS-IX, it needs to resolve this next hop: Routing entry for 193.148.15.0/24 Known via "ospf 1", distance 110, metric 20, type extern 2, forward metric 1 Last update from 62.212.94.61 on GigabitEthernet1/0.1, 00:00:41 ago Routing Descriptor Blocks: * 62.212.94.61, from 62.212.80.68, 00:00:41 ago, via GigabitEthernet1/0.1 Route metric is 20, traffic share count is 1 So in this case the peering LAN prefix is announced over OSPF, so the next hop resolves correctly and all the traffic is sent to the next router, which is the one directly connected to the AMS-IX here. Now consider what happens when someone elsewhere announces the peering LAN prefix over EBGP (this is taken from route-views.oregon-ix.net): BGP routing table entry for 193.148.15.0/24, version 426250 Paths: (1 available, best #1, table Default-IP-Routing-Table) Not advertised to any peer 3277 8482 28968 3246 1200 194.85.4.249 from 194.85.4.249 (194.85.4.249) Origin IGP, localpref 100, valid, external, best And: Routing entry for 193.148.15.0/24 Known via "bgp 6447", distance 20, metric 0 Tag 3277, type external Last update from 194.85.4.249 03:30:30 ago Routing Descriptor Blocks: * 194.85.4.249, from 194.85.4.249, 03:30:30 ago Route metric is 0, traffic share count is 1 AS Hops 5 The important part here is that the 193.148.15.0/24 prefix in OSPF has an administrative distance of 110, while the EBGP route shown here has a much lower administrative distance of 20. This means that the next hop address for the AMS-IX would resolve to an EBGP neighbor and all traffic to exchange peers is redirected over this neighbor! Now fortunately this only happens for routers that learn the peering LAN prefix over EBGP, so the impact is usually limited, but it's well worth the effort to filter out the peering LAN prefixes you connect to yourself. What happened this week at AMS-IX was even more fun: the new prefix for the peering LAN is a /23, but someone started to announce a /24 within that /24. (Probably typed ... 255.255.255.0 rather than ... 255.255.254.0, easy mistake.) The rule that kicked in now wasn't the relatively minor "EBGP has a lower administrative distance than IGPs" but the much heavier "longest match first" one. So in this case even the next hop resolved to an external route on the AMS-IX connected routers themselves, breaking the peering sessions. Iljitsch van Beijnum