Hi all, Here are the minutes from the EIX-WG meeting at RIPE 44 in Amsterdam. Please let me know if you have any corrections or are able to fill in names of any missing speakers. Sorry for the delay, these have been festering in my Inbox for a while. Cheers, Mike -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- RIPE 44 / EIX WG / Grand Ballroom / Wed 29/01/2003 DRAFT MINUTES Chairman: Mike Hughes <mike@linx.net> LINX Scribe: Dave Knight <dknight@ripe.net> RIPE NCC Administrivia: Fearghas can't make it, Mike Hughes chair this time - Agenda Bashing Change of order to fit everyone in as AMSIX need to leave early - Scribe David Knight, RIPE NCC - Attendee list Duly passed out - Apologies Feaghas MacKay Arnold Nipper Session is being webcast, use the microphones for questions! Euro-IX Presentation: Christian Panigl - Member status: 24 members TOPIX, LIPEX and NIX-CZ have joined since RIPE43 - Website www.euro-ix.net Getting a lot of hits Has a DB of connected customers Has a members only area Peering details Switch DB, which switch/os/modules members are running, etc Creatign a regulatory issues repository - Projects 3 projects ongoing Network monitoring, driven primarily by LINX Network Security, driven primarily by DECIX Member application process, 24 members, 24 ways, trying to harmonise this - Forum Last year the first one was held in Barcelona, decided to do more, members will be contacted shortly in regard to one in Mar/Apr The next general meeting will be at the forum - Questions ? None Packet Clearinghouse IP Phone Project: Bill Woodcock - www.pch.net/inoc-dba 24 members TOPIX, LIPEX and NIX-CZ have joined since RIPE43 - Website www.euro-ix.net Getting a lot of hits Has a DB of connected customers Has a members only area Peering details Switch DB, which switch/os/modules members are running, etc Creatign a regulatory issues repository - Projects 3 projects ongoing Network monitoring, driven primarily by LINX Network Security, driven primarily by DECIX Member application process, 24 members, 24 ways, trying to harmonise this - Forum Last year the first one was held in Barcelona, decided to do more, members will be contacted shortly in regard to one in Mar/Apr The next general meeting will be at the forum - Questions ? None Packet Clearinghouse IP Phone Project: Bill Woodcock http://www.pch.net/inoc-dba Putting up SIP telephones in NOCs in many ISPs Dial using AS numbers, want to complain about a problem at Genuity just pick up the phone and dial 1, speak to bill dial 42 Many large ISPs participating, LINX has all their offices online already, many exchanges are getting involved Netnod just ordered 60 phones, every member ISP will get one It's simple, any SIP device will work, mostly using Cisco 7960s Cisco can be persuaded to throw them in free with a large order Started with 100 phones which were given out for free, trying to get more for those who can't get their own - Questions ? ? How is it funded It's not, it's kind of the open source alternative, soon it will be the only phone network in history with service on 7 continents SIP is just no that difficult, you can run your own phone company in your spare time IXP Interconnect Case Study: Keith Mitchell, James Rice -Keith Mitchell- A year ago Keith gave a presentation on connecting exchange points A lot of overview coverered there, explains background From that the conclusions are that this can be good, but be very careful At Xchange point they have done this, with BT UK6X informally Have executed formal agreement with LoNAP Layer2 interconnect is very much in vogue, especially in the UK Seeing a lot of pressure to glue layer2 together, it's so much cheapear than doing it with routers It saves money, but is it smart ? A good idea for cost saving, not such a good idea with multiple exchanges on one metro Lets not connect everyone together witout routers XchangePoint and LoNAP been informally cooperating for a year Made a formal agreement about a week ago Have identified multiple connections modes, have implemented the first, a simple p2p vlan In the agreement there is a commitment to entering the modes if it works out, taking it a step at a time A block VLAN IDs unique to both parties asigned Gets scary doing this with multiple parties, only 12bit space Using 100 IDs, 40 XchangePoint, 40 LoNAP, 20 reserved All p2p, no public peering exchange addresses in use, 2 peers assign a /30 for a peering They don't do any settltment across the exchange, like a peering agreement Do people pay ? The terms are betwen the customer and operator they are connected to Resiliency You can't do spanning tree Could run different VLANS across the links Using manual configuration AUP Started writing, ended up with a very long list Decided to take the simple approach Any traffic across the connect must comply with the AUP at either end All traffic tagged All parties must know the rules at either end Service levels Layer2 stuff, you can only see the failure end to end, contact your operator for resolution Keep each other informed about tickets, etc Is this a legal agreement ? Not really, they have found a way to make it binding Termination notice period Have limits in place for costs sharing, etc Open Issues Do you extend your address space across the other exchange? -James Rice- Cisco use default VLAN 1, it can't be filtered and cant be disabled You decided therefore not to use it, your customers connect using it, this is not good .. Cisco, do something! LoNAP has a collector, ExchangePoint has a reflector! Wether or not to have a multilateral peering router is a good idea ? Should they participate in each others ? So far they have heard no rumblings of losing a customer who no longer needs to be at both exchanges Both exchanges have fairly different marketing strategies -Keith Mitchell- Problem with scaling broadcasts Switch vendors should provide a better solution for this Have been going for about a week, it's going OK, 10 VLANs up in the first week The agreement is in the public domain, please check first with Keith or James first though - Questions ? -Randy Bush- ? Complex layer 2 networks have reliability problems, you seem to be trying to reach a compromise for many reasons which are hidden. We know STP, etc have bugs, these things fail, you are trying to reach a compromise to solve a business need and it would be nice if next time you could reveal what is under the covers here, why you made some of the decisions, what were you trying to avoid, why you chose not to try those things? A real hell lies here and you are working hard to avoid it, others are walking into it and it would be nice to get explicit about this -Keith Mitchell- Some of this is covered in the presentation given last time, if we let this run for a while there will be more to tell next time -?- ? Are you running a seprete vlan for ipv6 -Keith Mitchell- At the moment LoNAP doesnt do it XchangePoint is putting it on seperate vlans Anyone wanting to do dual stack on a router can support them on different interfaces -Bill Woodcock- This is the same methodology that has been used between Seattle IXP and the Seattle PAX, it works there -Keith Mitchell- I know its been done in the US -Bill Woodcock- Yeah, it works fine -Mike Hughes- Would you be interested in putting the do's and don'ts into BCP, share your gotchas, etc, this is new -Keith Mitchell- Yes, sure IPv6 Policy Issues Update: Mike Hughes Mike and Netnod will write a document on getting address space for independent IXP services nets, so they are independent of a particular upstream You turn the classic relationship on its head as an IXP, asking customers for addresses Questions ? -Randy Bush- Why is that upside down ? -Mike Hughes- Kurtis asserted it this way, if you want to have the same fight as in the IPv6-WG please do it later ;) -Kurtis Lindqvist- Just a reflection, IXPs in several locations only get 0ne /48, not a problem space wise, but this is only one AS num they would rather have seperate blocks .. -Mike Hughes- RIPE NCC! We need a monitor on the stage for audio! IPv6 Exchange Point Survey: Alexander Koch Been spending some spare time thinking about rolling out v6, has Juniper kit, it just works 1 Cisco working in v6 BGP full mesh, a great surprise on this old box As they rolled out v6 in Europe and the US, many exchanges have v6 assigments but nothing is happening, they support it but to what level? They see issues with exchanges in europe using eui64 adressing, this is not what we want to see, other exhanges running on 6bone space other exchanges doing it on a sep vlan What he wants to say is some small things about exchanges and how he would ask them to do it, make it work and make it work easier At many IXPs they use eui64, he has no problem with hex, but decimal is so much easier Seen that many IXPs put v6 on a seperate switch, you would have to use a specific connection, whats the point? XchangePoint doing VLANS all the way through, why ? Others doing it to stop interference in operational security, yet to hear of any problems with this though anywhere in the world! Many IXPs already doing this Some IXPs just do it the wy he would like everyone to do it, they offer ipv6 parallel to v4 An argument we still hear is that some providers cannot do it, they have to do this with a test, these guys need anpther port anyway Cisco IOS versions lately are good, far more stable than previuosly try it out All exchanges consider offering it on the same VLAN! Those that need a new port will need it anyway More and more people will use the same router for v4 and v6 Way more people will use v6 at the AMSIX, at DECIX it is getting bigger, dont put restrictions on this Push v6, put it in the same LAN - Questions ? -Bernard Tuy- ? Next time please have some slides, it is boring listening to 10 minutes like this, this is interesting for us all, we like to see more information -Mike Hughes- Alexander can write up his thoughts and get them out on the list While we are here, of the EURO IX members, who have their details up ? About half of us then of those who are here -Bernard Tuy- Is it in the public area ? -Mike Hughes- Yes -Christian Panigl- ? Is there a benefit for the IXP in having them in the same VLAN -Alexander Koch- You can use a smaller router without VLAN trunking, this needs a different feature set with more bugs MCAST On a seperate VLAN, v6 should be on the same! -?- That made prove to people afraid of v6 that its not harmful -Bernard Tuy- It's not a matter of afraid, but about monitoiring, if you want to seperate monitoring you should use seperate monitoring -Christian Panigl- As an IXP operator there is therefore a benefit to seperate VLANS, it's a trade off We use a different VLAN mainly so we offer diferent AUPS on the diferent lans, we allow transit only in v6 Eventually though the policies will converge IXP Presentations: o AMS-IX Henk Steenman Still growing in ports, members, traffic, etc Been suffering many problems which will be discussed at the meeting friday - Questions ? -Bernard Tuy- If everyone can say how many memebers they have and how much traffic that would be nice. IPv6 too. -Arien Vijn- 21 IP addresses, some have 2 routers connected, 17 AS numbers o INEX Dave Wilson Incorporated in 96, operational in 97, now 10 members Not presented in a while Hit a peak 5 min avg of 100meg/s Moving premises from Ballycoulin? to the other side of Dublin Some people taking advantage of this to upgrade connections HEAnet on site but connecting shortly Couple of members on STM1, a couple will soon be on GigE Investing in a 3508 soon to take new GIG member connects Recorded 100% uptime this year New pricing structure, copied ripes small/med/large Investigating new protocols About 1.5 years ago outsourced to a pair of consultants they are responsible now - Questions ? -Mike Hughes- ? v6 -Dave Wilson- One of the new investigations o INXS Udo Steineger Not a rock band ;) Most of you know our presentation, this time shortened with just news Presentation: http://www.inxs.de/ History and Overview Joining Requirements: AS, LIR, Router at INXS, No Layer2 links! (yet) Services / Fees Web Server: www.inxs.de, Lists: tech-list, org-list MCAST Trial: Regular members only, no fees in test, dedicated VLAN, general rules apply, exchange via PIM-SM, MBGP, MSDP, IPv6 Trial: Same as MCAST, using C&W v6 allocation Contact: udo@cw.net - Questions ? -Gert Doering- Please use this opportunity to get v6 reverse for your peering mesh! -Bruce Campbell- We can fix that tonight! o LINX Chris Fletcher ! Delayed due to laptop problems o LoNAP James Rice LoNAP James Rice Pretty before/after pictures of LoNAP racks, nice cabling News New site at RedBus, more nice pictures Private Gig interconnect New website with useful information, bandwidth stats, peering matrix, etc A recent question was the last time a non scheduled LoNAP happened, nobody can remember First IXP interconnect in Europe Until the end of February new members will not have to pay the joinup fee, cables run by LoNAP 80% peering right now Looking glass on the website 8291 routes available at LoNAP Most of that comes from MfN New customers: Marula, Associated Networks, Aeximous Connection policy, dont need a router housed with LoNAP, you can have a private circuit into one of the colo facilities v6 is on the todo list, none so far, want an indepedent allocation but dont currently qualify, getting that fixed this week, If you are interested in joining fill in the form on the website o LINX Chris Fletcher You know who we are, heres an update Usual stats, graphs, what we're doing with engineering, new network map, news on changes to billing doing some training work Traffic Last week peaked at 25Gbps http://stats.linx.net Members 136 Engineering Upgrade of Extreme switchesd to 10Gig Creating a 1010Gbps ring at the cire Started 1t6th Jan New blades needed 6 reloads did this on spare blades so downtime was very small Network Map 2 seperate networks Billing Quite a big change Started usage billing in 2002 Cost of inter site links depends on usage IX monthly income = variable charges Dont need to redict growth over coming year Fixed monthly inomne make cashflow work easily Required income / total usage = pounds per meg, for monthly charges Apply this to each port multiplied by the avg megs per second This guarantees the required income per month and fairly splits the cost among the members There is also a flat charge per port Higher charges > 75% usage on a port, ie 750Mb on a Gig port this discourages members filling a port and adding latency 100Mbps at 50 Mbps = 752 pouns It is hard to say that a port costs a certain amount They have plans for a calculator Choice of monthly in arrears by direct debit, or quarterly in advance Advanced billing is based on an estimate Correction based on actual usage is applid next quarter Training Had this around for a while Came up with a framework of thisngs internet engineers need to know Spoke to universities and providers, universitiess interested, taken it on as a basis for courses LAIT LINX Accredited Internet Technician System and netwrk training with three LEvels Contacts Training www.s-nt.co.uk v6 Today 3 members peering, on shared vlan, hex as number addressing - Questions ? o NAMEX IXP Maurizio Goretti Located in Rome near the main train station First University of Rome Est. 1995 as Nautilus, hosted by the university supercomputer centre "CASPUR" Non profit Oct 2001, NaMex 21 members Plain Layer2 backbone, Cabletron Enterassu SSR 8000 100/1000 New site, computer centre is moving, IXP is moving too, have 60m2 5 carriers with fiber AS24796 PI allocation BGPv4 IPv6 Public Network 2001:7f8:10::/48 BGP+ peering Before next RIPE meeting New site 1 Gig peak MCAST start Join EuroIX Permanent staff needs hiring o NETNOD Kurtis Lindqvist News GigE to be installed in Malmo and Goteburg Plan to be operational in Q103 110 k / year Singel switch per location Will close down DPT-622 as soon as GIgE as operational at all locations It will likely take a year before everyone moves due to contracts DPT-2.5 in Stockholm will stay in Fast Ethernet at all GigE platforms All customers get an INOC-DBA configured phone, just apply It stays NETNOD property Planning to hold a customer meeting in end Feb early March IPv6 is coming up, finally got a block from RIPE NCC MTU Made an annoucnement to change MTU on one switch, one switch is bridged to old FDDI setup Many providers canot do MTU4470 Some claim it is not needed Operations Swedish Military outsoured, they were expensive, got someone else cheaper will hopefully give better service Migrated from mountain cave A to mountain cave C, locations are not disclosed Did a study with the Swedish government to see what happens when you take down an exchange, not a lot - Questions ? No o NIX.CZ Prague Non profit Est 1996 35 members Price: setup 1600 - year 5200 Offer from redbus for free hosting, stopped by floods 3rd pop - Sitel Telehouse Sucesfully started a week ago, Nad elektranou, Praha 10 4th pop at Pragonet Topology Map Using Cisco 2950, 3550, 6006 Operation Intranet for easy member coms Reached 1 gig in last few das Monitoring of ports 2003 development Joined EuroIX Its good for them to comunicate with others IPv6 support First 3 ISP want to peer pure IPv6 networks Getting an aloc from RIPE NCC Will use the same vlan for v4 and v6 if it causes probems they will change this later Plan to improve NOC Contact info Peering Matrix Monitoring, each end user can see switches and see all ports - Questions ? No o XchangePoint Keith Mitchell Going for 2 years 70 customers: 5 major ptts, US OLSP, 20 od Euro ISP, 30 odd UK ISPs Choice of transit from 10 major ISPs, Ethernet circuit providers, ADSL Aggregation providers Architechture is a little different, DWDM kit to sell private p2p to customers At several London sites Traffic http://traffic.xchangepoint.net Steady at 250/300 Mbps .. Growth in inter site DWDM Developments LoNAP Interconnect .uk Secondary name server All switches upgraded to ExtremeWare 6.2.2 Build 56 less painful than 2, half of the network running each Gotchas going from 6.1.9 to 6.2.2 more info from Extreme on that would have been helpful Amending AUP to allow controlled connection of 3rd party switched Transit Marketplace highly successful, next event in London on Feb 27th Feb Close to securing funding for additional European locations Contacts AOB: -James Rice- LoNAP membership 2000 pounds, not 2000 Euros -Keith Mitchell- XchangePoint has 6 people on v6 -Mike Hughes- It's Paul Thorntons birthday, he's 30! Now, to the bar!