The word GIX has been mentioned several times in the recent time. The name is realy not what it should be, but the idea is to have THE Exchange, sort of a grounding-point for a flat addressing architecture. It is described as 'Level-2', to be able to handle all special cases, that will arise when 'general infrastructure' and 'mission oriented' networks are connected across the same medium. I image the routing implemenation for the 'general infrastructure' networks to be a handfull of route-servers run by the routing authoritys for each continent, to avoid the n-squared peering problem, and to get stability. In Ebone context, we can live with the 'Level-3' implemenation we have today to solve the problem of getting maximal connectivity, from a user in one corner of Europe to another user somwhere in the entire universe. I envision that the Ebone open infrastructure, is a Level-3 extension from this 'GIX' and that any European network/network-providor shoul have equal access to the Ebone core. We have the technology, the needed people but, as all networking in Europe is about the network-POLITICS not the network-SERVICES it might not be possible to have this pice of usefull infrastructure. If we fail this time, please take your wathces and adjust them backwards five years! I include the full 'GIX-proposal' for your information, remember, this documnet is 'politicly' nothing, as it is a proposal from a small group people. And do remember, time is critical now, the whole CIDR model is based on that large chunks of networks, like continents are fully interconnected. -Peter ------------------------------------------------------------------------ Proposal for Global Internet Connectivity IEPG Working Document Guy Almes, Peter Ford, and Peter Lothberg 12 June 1992 I. Introduction A significant subset of the IEPG membership has advocated a common forum for the coordination of world-wide research and non- research networking. At the November 1991 meeting in Santa Fe, the idea of a global internet exchange emerged in basic form. At the March 1992 informal meeting, the idea was discussed in a day-long meeting at which goals and basic concepts were shared and at which a consensus began to emerge. At the June 1992 meeting in Tokyo, a detailed proposal was discussed and refined. This paper documents the ideas that may allow us to move the ideas from discussion to plan to implemented reality. Key technical goals are to maximise connectivity, promote appropriate routing, promote effective sharing of resources such as links, and enable a variety of transit options. Key non-technical goals are to ensure the participation by all segments of the Internet community -- both the traditional CCIRN constituency and others -- in working toward a high-quality world-wide Internet. The reader should consider that the substantive ideas of the paper reflect discussions with several people. The helpful suggestions of Tony Hain, Geoff Huston, Bernhard Stockman and Claudio Topolcic are especially appreciated. II. Goals The proposal addresses the following technical and non- technical goals: <> Maximal Connectivity: Enhance access and connectivity of the global Internet. Interconnectivity of networks should be enhanced, including those networks which are not directly funded by traditional CCIRN constituencies. <> Cost-effective Transit: Make available to all our networks the best available possibilities for worldwide transit. To the extent possible, promote the sharing of global transit resources and the sharing of costs. <> High-quality Routing: Work toward routing management that supports the connectivity and transit goals while being technically optimal. Each of these goals is expanded in turn. In addressing maximal connectivity, we debated on whether 'universal' or 'maximal' was the right term. There was no question, however, but that, in the words of one member "Connectiv- ity is the key goal". There are several possible limitations on connectivity. Some, such as connectivity limits imposed on a particular community of interest, are unavoidable and even desirable. Thus, a term such as ~selective connectivity~ might be appropriate. Others, such as any limits imposed by usage con- straints on intercontinental transit networks, are undesirable in the long run. Transit networks and interconnection structures open to both research, education, and other traffic will aid the broad growth of the Internet. To summarise, we want to promote, enable, and maximise universal connectivity among transit networks, so that the only limits on universal connectivity are imposed by user communities. In addressing cost-effective transit, we noted that there is a diversity of transit needs: some networks need universal affordable transit, for example, while others need higher-perfor- mance transit even if costs are not minimised. Similarly, there is a diversity of transit possibilities: there are at least five national providers in the US, for example, and we could contemplate new transit structures. To summarise, we should work to harness the variety of transit options to serve the diversity of transit needs. At the same time, we should ensure that issues surrounding transit do not limit interconnectivity, defeat our routing goals, or adversely effect existing networks. In addressing high-quality routing, we emphasised routing as a means to supporting the connectivity and transit goals mentioned above, but also noted technical symmetry, stability, and manage- ability as explicit goals. Though it might go without saying, we also agreed that we should use the best of currently available routing techniques, while not requiring what does not exist. At the same time, it is desirable to provide a platform for the early deployment of advanced routing tools. To summarise, we should work together to build worldwide routing and routing management in support of maximal connectivity and cost-effective transit, while using the best available techniques to support technical qualities such as stability and symmetry. Along with these goals, we also agreed on the following qualities that must apply to our approaches to them: -- Scalability: We understand that the size of the Internet, measured either in traffic or number of connected networks, is growing exponentially, and that our engineering and operations must scale. -- Manageability: We understand that the worldwide Internet must be as well-managed as one under a single administration, and yet recognise the autonomy of each constituent network. -- Accountability: We understand that any successful worldwide connectivity structure must be accountable. -- Timeliness: We understand that the cost of staying with the status quo, or the cost of delay, is very high. Maintaining the integrity of the Internet requires prompt action. In approaching solutions to these goals, we often had to deal with tensions among competing goals and the limits imposed on our technology. III. Our Vision -- Technical <briefly, a set of global interconnect points at which cooperative advanced inter-AS routing is supported and at which transit and regional networks meet. Universal connectivity, good routing, and several transit options are supported.> At several places around the world, we envision global interconnection points. Each such point would consist of a managed facility with 24-by-7 coverage and excellent environmental support. At each point there would be a high-speed broadcast LAN freely available for all kinds of traffic. Each participant would be free to, at its expense, bring a circuit to this facility and place a router on the LAN. It would then be free to exchange routes and traffic with (the routers of) other participants at that LAN. Each participant would also pay for a pro rata share of the cost of the floor space, environmental support, and administrative support required for the interconnection points. At this level of detail, we are simply describing the engineering of the current US FIX structure. From a technical point of view, we consider that it is the best approach among current models. There are, however, several non-technical requirements needed:
An Open Forum for Coordination. This forum would provide full participation for non-US and for non-research-and-education networks. Very Broad Usage. A global interconnection point must be able to pass research, education, and other traffic. Some connect- ing networks may still have research/education AUPs, but the interconnection structure should not. In addition, we need solutions to several technical shortcomings of the traditional FIX engineering structure: N-squared Routing. All routing is done on the basis of pair- wise peering among pairs of routers on the LAN. We consider that, as the use of the interconnection points grows, this may not scale. Anarchic Routing. When a router on the LAN advertises a route to a given destination network, its peer has no basis for knowing whether that router is authorised by the destination network to advertise it. Destination-based Routing. If we were able to route across the interconnection points on the basis of both source and destination, then we could better use the variety of transit possibilities, including federally funded ones, without limiting connectivity due to policy issues. Our inability to do so limits these choices, and also leads to undesirable asymmetric routes. In order to achieve our technical vision, each of these shortcom- ings will need to be dealt with. In many cases, these problems will be dealt with by means of improved routers of participants and not by means of new technology in the interconnection points themselves.
N-squared Routing. We anticipate that the number of routers at the interconnection points and the number of network numbers advertised by each router will both increase dramatically. In order to cope with this, we will provide for the deployment and management of a set of Route Servers on each of the interconnection points. In simple terms, Route Servers use inter-AS routing protocols such as BGP to learn and advertise routes, but do not themselves participate in the forwarding of packets. Networks that attach will have the option of using these Route Servers, of using traditional pair-wise peering, or of using some combination. There may, in fact, be multiple different Route Servers at a given connection point used by different sets of participants. The intention of Route Servers is not to impose policy, but to implement the dissemination of routes in a manner that scales and can be well managed. Such Route Servers might also help solve some of the so-called ROAD problems. For example, CIDR support in a Route Server could help even in the period before CIDR is supported in the routers used within a participating network. Similar examples could be given with respect to Policy Routing and Source Demand Routing. Route Servers will be a subject of Section V (Research Efforts) below, and its deployment and management will be a subject of the cooperative routing efforts discussed throughout the paper. Anarchic Routing. One of the major successes of the NSFnet Backbone Service has been the Policy Routing Database, a database designed and administered by MERIT which records the ASes author- ised to advertise certain IP network numbers to the Backbone. By correctly and faithfully administering this database, MERIT provides a communication link, as it were, between network managers who set and agree on policies and the routers which implement routing exchanges. For each IP network number, for example, the database records the primary, secondary, and (sometimes) tertiary ASes that can advertise that network number. We anticipate a similar need at the global interconnection points. Rice University's network number (128.42), for example, could be authorised for the NSFnet Backbone Service as primary and SprintLink as secondary (and used only for backup). A global routing Registry would be designed and administered, and this Registry would be instantiated at each interconnection point. Registry management would be selected to ensure the neutrality of the Registry with respect to possibly competing networks that connect at the Interconnection Point. Tools could be written to produce derivatives of the Registry for particular engineering purposes, such as Cisco access lists for the control of route filtering. If we can judge from experience on the NSFnet Backbone Service, such a Registry would provide a framework in which misunderstandings and conflicts in routing could be avoided and/or resolved before they could cause undesired asymmetric routes or other problems. Destination-based Routing. There are currently many circum- stances in which several transit networks must be traversed from source to destination, in which several alternate paths are possible, and in which constraints (such as AUPs) or preferences (based on cost or performance issues) will influence the selection of the most appropriate path. Further, not all of the constraints and preferences can be cast in terms of traditional routing which considers only the destination IP address. Examples of needed improvements include: -- TOS Support. Some traffic may be tagged to require low latency (e.g., telnet) or high bandwidth (e.g., image trans- fer) or low cost (e.g., email for high schools). To some extent the TOS/QOS concept may support this, but operational support for this on a worldwide basis is beyond current art. -- Source-based Routing. Some traffic may be appropriate to carry over special-purpose networks due to identification of both source and destination. For example, ESnet might be willing to serve as a transit network, but only for traffic from one energy-community site to another. Similarly, consideration of both source and destination might be useful in avoiding asymmetric routes. For example, in the case of traffic exchanges between a university and a commercial site, destination-based routing often leads to the use of research transit networks in one direction and commer- cial transit networks in the other, even when the research transit network can support the full exchange. -- Flow Support. Some traffic may be part of a flow that requires a form of bandwidth reservation supported by one special transit network, and that transit might be configured to support that flow. Support for global conferencing might require this flow to be carried over general infrastructure networks for part of the path and special transit networks for other parts. Routers in common use on the Internet do not support any of these abilities, but experience with DARTnet and with wider experiments with IDPR suggests that some of them could be deployed in the near future. An Open Forum for Coordination. We anticipate the need for the full participation of engineers from all connecting networks in the coordination of routing and management across the global interconnection points. The global routing Registry would provide one kind of communication path to allow for this coordination to be natural and convenient in normal cases. The famous RIPE coffee breaks offer another needed tool, but can only happen less frequently. Advanced collaboration support tools, including conferencing, shared blackboard, and email-based tools, could provide some technical support tools for this effort. The FEPG within the US agencies and RIPE within the European IP community provide useful precedents. The key is effective and full participation by networks in issues of concern to those networks. This may require a formally incorporated organisation as a structural umbrella. Very Broad Usage. There will be no AUP constraints on the interconnection points themselves. Further, by enabling effective routing and by bringing several transit alternatives to the interconnection points, we will enable participating networks to cope responsibly with the AUP issues that remain. IV. Our Vision -- Non-technical <briefly, a structure to do collectively what must be done collectively to accomplish the stated goals. This collective action will probably require some organisa- tional structure, and this structure should respect autonomy and be accountable.> Although the IEPG can manage the design of technical solutions to our goals and although we perceive very broad agreement with the goals and technical structures we propose, our proposal cannot succeed without some cooperative structure for managing the global interconnection points and setting the policies that govern it. At a minimum, we need a small organisational structure that can do this necessary management and governance. The structure needs to secure the fullest possible participa- tion of the Internet community. This must include networks from North America, Europe, and the Pacific Rim. It must include government agencies, university consortia, and for-profit and not- for-profit companies. It must include networks that support research, education, industrial, and mixed traffic. The structure needs to be representative of and accountable to all its participants. Thus, what policies the structure does set must earn the kind of authority that comes from such accountabili- ty. The structure needs to address what must be done collectively without itself becoming a network in competition with its partici- pants. Thus, for example, the cooperative structure would not provide connectivity directly to any user site. Similarly, the cooperative structure would be careful not to favor on competing participant over another. The structure must avoid deciding issues best left to the participants themselves. Thus, for example, the structure should avoid deciding which networks should use which other networks for transit, or how various pairs of networks should do settlements. It is important to remember that the purpose of our efforts here is to cooperatively harness the strengths of our various networks in support of the Internet community -- not to replace our networks with a homogeneous Internet. Among the organisations that could help us find the right organisational structure to meet these objectives are the CCIRN, the Internet Society, EDUCOM, RARE, FARnet, and PACCOM. Fortunately, there is near-universal recognition of the mutual benefit involved in full connectivity. V. Research Efforts Needed to Achieve the Vision <briefly, we need specific research and development efforts to improve on the routing techniques and network management techniques available to us> Several research and development efforts need to be conceived, funded, executed, and deployed in order to achieve all the aspects of our technical vision. Among them are:
Advanced Route Servers. Very urgent. While pair-wise routing works today, scaling for the future will require successful design and deployment of Route Servers. Initial experience could take place using third-party BGP techniques with current routers, and CIDR routers could become available in the near future. Source-based Routing. Early experience with source-based inter-AS routing should be evolved into operational routers that could pass packets among participant routers at the interconnection points. These routers could allow us to avoid asymmetric routes, but would have to be both sophisticated in the policies supported and very fast to avoid becoming a performance bottleneck. TOS/QOS-based Routing. In order to support high-performance applications on a worldwide basis, we will need to support TOS/QOS at the interconnection points. This will need to be supported in a way that interoperates with other emerging approaches. Flow-based Routing. As the concept of flows becomes better defined, support for it at the interconnection points would add significant value to the worldwide Internet. Routers that recognise and expedite the forwarding of flows would be of great utility at the interconnection points. Some of these projects involve multi-year research and development. Others, such as simpler versions of Route Servers and CIDR, may be deployable within a year.
VI. Immediate Non-technical Agenda <briefly, work with the Internet Society and the CCIRN to set up an accountable cooperative structure to accomplish the technical agenda. This should be in place by summer 1992 > Under the leadership of such collective organisations as the Internet Society and the CCIRN, we should immediately set up a structure that can allow the first step at our technical agenda. Efforts should be made to accomplish this by summer 1992. We will consider several possibilities. One very specific possibility, which we would view positively, follows. The CCIRN and the Internet Society would charge a task force of the IEPG to develop a more detailed design, mindful of the very similar agenda before the US Interim Interagency NREN project. Considering the urgency of providing an initial global interconnection point during calendar 1992, and continuing this specific example, this task force will attempt to liaise very closely with the NSF. Within this strategy, we hope to leverage activities already planned at FIX-E. There are other possibilities that we will explore, but we view this as the most hopeful. VII. Immediate Technical Agenda <briefly, set up one interconnect point using existing routing and network management technology. It is likely that the NSFnet Implementation Plan and the FEPG Interim Interagency NREN Architecture, both of which are converg- ing during summer 1992, will influence the resulting design. In NSFnet/NREN terms, this section calls for a NAP on the east coast of the United States.> At one place on the east coast of the United States, we envision a global interconnection point. This point would consist of a managed facility with 24-by-7 coverage and excellent environ- mental support. At this point there would be an FDDI ring freely available for all kinds of traffic. Each participant will be free to, at its own expense, bring a circuit to this facility and place a router on the ring. It would then be free to exchange routes and traffic with (the routers of) other participants at the ring. N-squared routing would be addressed by the provisioning of route servers. This would take some time to perfect, but it would be taken on immediately both for experimentation with the route server idea and to permit scaling of the rapidly growing route tables we all keep. Anarchic routing would be addressed by an interim Routing Registry. One possible approach to this would be to adapt for our use the registry the NSF will be setting up for the NSFnet Backbone Service. We would live with destination-based routing and forwarding, but would immediately start some experiments with enhanced forwarding. These experiments would not be allowed to compromise the operational stability of the interconnection points. The IEPG task force discussed in Section VII would meet quarterly to work out problems and evolve the structure. Even between meetings, this task force would serve an ombudsman role and would oversee the interconnection point to ensure that the interests of the international community are being well served. The meetings should be coordinated with existing meetings such as RIPE, IETF, and the Internet Society. There would be no AUP constraints on the interconnection points, themselves. VIII. Evolution to the Vision <briefly, the cooperative structure guides the immediate technical structure toward the vision, making use of the results of the stated research efforts> Under the leadership of the CCIRN and the Internet Society, the interconnection points and their management would evolve. This will eventually lead to the need for us to procure the services needed to support and manage multiple interconnection points. This will require organisational maturity in order to manage such a procurement, and in order to have an authority structure that combines credibility, broad trust, and accountability to the international community. Similarly, we will need to ensure progress on the research agenda required for high-quality interconnection points for the future. This will pay off enormously when the time comes to increase the number of interconnection points. Similarly, after the initial single interconnection point is established, a set of design tasks will be required prior to our being able to field a second site. Examples of issues are the coordination of the multiple instantiations of the Routing Registry and the patterns by which the various options of transit among the multiple sites are handled. Later, we will also need to reconsider the technical basis of the interconnection point as Internet technology evolves. During the era of fractional T1 intercontinental links, our current choice of an FDDI ring seems more than adequate. As more advanced transmission media such as 150 Mb/s ATM with support for isochro- nous traffic become deployed in the Internet, we will surely need to reconsider this technical basis. On all these areas of work, technical service, non-technical management, and research management, we will need to ensure that the broad needs of the international Internet community are being addressed. IX. Conclusion <briefly, the Internet is saved and the IEPG wanders off to the bar> This paper has proposed a serious program for improving the global interconnectivity of the Internet. The goals of maximal connectivity subject to community-of-interest constraints, cost- effective transit, and high-quality routing are addressed, and attention to short- and long-term efforts, technical and non- technical efforts, and research agenda is paid. There is solid hope for success provided that these efforts are begun very soon.