- db-wg - mailman.ripe.net

draft minutes of DB-WG meeting at RIPE 25
by Wilfried Woeber, UniVie/ACOnet 10 Oct '96

10 Oct '96

This is the 1st draft of the minutes for the DB WG meeting at RIPE 25. I expect to add a couple of URLs to point to presentations as soon as they might become available from the RIPE FTP-Server. The Action List probably needs renumbering, to get it aligned with the summary notes for the plenary. As always, all comments, input and improvements welcome. Wilfried. -------------------------------------------------------------------------- Wilfried Woeber : e-mail: Woeber(a)CC.UniVie.ac.at Computer Center - ACOnet : Vienna University : Tel: +43 1 4065822 355 Universitaetsstrasse 7 : Fax: +43 1 4065822 170 A-1010 Vienna, Austria, Europe : NIC: WW144 -------------------------------------------------------------------------- Minutes for the DB WG meeting RIPE 25, Amsterdam, NL **1st DRAFT**, 9-Oct-1996 Chair: Wilfried Woeber Scribe: John Crain Participants: 51 (according to the sign-up list) A. Administrative stuff John Crain offered to take notes. Thanks a lot, John! The proposed agenda was agreed, including the following additional items: - Input from the DNS-WG and - Input from the Local-IR WG both for agenda item Y. General input from other WGs and - advisories: in aut-num: objects for Z. AOB B. DB-SW status report by the RIPE NCC Carol Orange offered the report on the state of the RIPE Database and the DB-SW. Version 2.0 of the software was released recently and put into production at the NCC. Important new features are: - inverse lookup (whois -i) - the package is now fully classless - support for near real-time-shadowing (As an aside, a DB-Workshop was offered on Monday morning.) More details about the growth of the DB and the increase in the number of objects, etc. were shown on the slides. [ These slides are expected to become available soon ] [ at ftp://ftp.ripe.net/ripe/presentations/ ] Activities for the short term future are - produce documentation - fix any version 2.0 bugs - try to improve consistency In general, version 2.0 is found to run very well. Sabine Dolderer reported on problems with the handling of old (invalid) handles. This is one of the known problems. Sabine was asked to report the problem to the ripe-dbm(a)ripe.net mailbox. C. DB consistency and sanity checks Is it worthwhile doing more sanity checking? There was general consensus that the quality and consistency of the data in the RIPE-DB should be improved. The group formally supports this activity which is also part of the RIPE NCC Activity Plan for 1997. A project has already been set up with the Free University of Amsterdam to look at the consistency issues in general. Students may wish to interview selected users. The Local-IR staff and the users of the Routing Registry are expected to contribute to this activity. The group then identified a couple of obvious inconsistencies which should be checked, either at the time of a transaction or on a regular basis. E.g. - person/role objects being refered to no longer exists - persons (multiple) with and without nic-hdls. - person objects which are no longer referenced by any other object (proposal to eventually remove these objects, considering the age of the object) - multiple route objects with different origin, where there is obviously no trasnition and/or multi homing. The various checks, when they get implemented eventually, should at least generate warnings, both at the time of an update and on a regular basis. The logistics (frequency, identifying the proper recipients for the warnings, etc.) need more work. The discussion of a potential reservation of some NIC-Handle types (like AUTOxx-RIPE, ROLExx-RIPE, etc.) led to the conclusion - that there is no need for special handling of role: objects - and that AUTOxx-RIPE should be reserved to avoid confusion when requesting automatic handle assignment with the AUTO-xx mechanism. Niall O'Reilly suggested that the various registries should help with the transistion process by changing old person objects that are (mis-)used to define a role into correct role objects. D. Inverse lookup discussion Is automatic recursion still useful? Just observed as a fact, quite a few people have aliases defined to disable the automatic recursion for their environment. However, for compatibility reasons and because most of the casual users are expected to need the information provided by the recursion, the decision was to stick with the current default (recursion is on by default, use -r flag to disable it). However, the behaviour of the software when doing an inverse lookup ("-i" flag) is to be changed to imply "-r" (recursion disabled). Discussion then moved on to find out wether inverse lookup should be allowed for all attributes. Common feeling was that it is not necessary for all attributes, but it might be useful to implement for a couple of additional attributes. In order to find out about the real need, a discussion on the DB-WG list should be started and the NCC was asked to look at the logs to find out about "popular" failed queries. Kurt Kayser asked about the software behaviour and the timing involved when a "server running at low priority" message is issued. This message is issued for general queries that (potentially) produce a large number of hits. As long as the TCP connection is up, it's still working. E. Documentation Ambrose Magee was then introduced as the new primary maintainer of the DB-SW. He reported on the proposed structure of the Documentation for the RIPE Database objects, the usage and the software. The group endorsed the concept with some minor proposals for improvement and stressed the importance of the documentation, and in general, the importance of consolidation vs. new functionality. In particular, the proposal is to have a main document that is stable and a set of appendices with things that may change or require more freequent updates. [ The slides are expected to become available soon ] [ at ftp://ftp.ripe.net/ripe/presentations/ ] One of the suggestins was to seperate the advanced sections (e.g. on how to obtain the database etc.) from the general user documentation. Nigel Titley proposed to start working with the sections on the whois client and utilities and move that information to the beginning of the documentation Bernhard Stockman asked for a list of all active objects (Index), including definition of the syntax and the also the semantics. Daniel proposed to add that functionality toi the software itself, probably by adding a whois -T with -v and possibly whois -T to report all objects. This is is already available for listing the schema for an individual object. F. New functionality Even thouogh there was some discussion on new functionality, like a URL: attribute and amendments to a cgi script to provide access to the data via html, it was stressed that for the time being consolidation should be the priority instead of new functions. The maintainers of the DB-SW were asked to maintain and publish the "consolidated wish-list" and the "known bug list". Y. General input from other WGs DNS: There was concern over potential conflicts between information in RIPE Database and the TLD's internal databases. Becuase of that potential conflict, some TLD administrators do not register their secondaries in the RIPE DB. Referal to their database would be useful. Daniel Karrenberg reported that this referal mechanism is already on the wish list. No design work has been done yet. There is a couple of potential implications, e.g. the idea to implement the referral for other (all?) objects. Guy Davis observed that it might be useful to do a more specific search on hosts/domains for DNS. Daniel Karrenberg was in favour of more specific, but has mixed feelings on referral because of all the different whois initiatives within the Internet. Thus he suggested to implement a quick fix for the top level domains only. Janos Zsako thought that the referral should be implemented in the server not in the client! Wilfried Woeber observed that the referral approach can only be useful if most TLD's are willing to supply an interface for whois. There was an indication of considerable interest to offer access to the local TLD data. Local-IR: Mandatory attributes (as described in the regsistry procedures) should be made mandatory in the database (enforced by the software). In particular the status: attribute in inetnum: objects and the nic-hdl: attribute in general should be made mandatory. Of course, this implies a major clean-up of the database by the LIR's! Daniel Karrenberg observed that initially this will only be applicable for new objects. The clean-up will come later. Kevin Hoadley asked for some time to implement the necessary changes to local scripts and procedures. The following discussion led to a proposal to start enforcing this on the 1st of January 1997. A. Blasco Bonito asked about potential implications of making the 'status:' attribute mandatory. Carol Orange suggested to define an additional value for the status: (USED?) to distinguish between address space allocations, assignments and usage. A. Blasco Bonito's comment that "The general attitude of LIR's who are large providers is to assign to other smaller providers. These then distribute to their customers." prompted Daniel Karrenberg to state that "If this is the case then they are not following the registration policies." A mandatory status: attribute has implications that have to be discussed in more detail. Z. AOB Brian Renaud proposed to obsolete the advisory: attribute for aut-num: objects. For 'route:' objects, these atrtributes should remain valid. Ther was general agreement and the NCC was asked to implement that change. Following up on a mail received from David Kessens, ther was a review of the various mailing lists that dal with different aspects of the database business. These are: DB-WG DB-BETA RRIMPL Daniel Karrenberg described the scope of these lists and observed that all three have their own specific use: - DB-BETA is for people running new code. - DB-WG is for architecture discussions - RRIMPL has a wider scope Bugs in the released version of the RIPE DB-SW should be submitted to ripe-dbm(a)ripe.net discussions about details of running new code should go to db-beta@ripe-net. Sabine Dolderer observed that if there are two maintainers then the "weakest" one is used. Daniel Karrenberg pointed out that there is no hierarchy involved. The software just scans all entries and tries to find a match. The first match found is honored. This is in line with the current concept and design. This is the correct behaviour as all maintainers are equally authorized. To allow specific maintainer functions is at the moment beyond the functionality of the DB. Appendix: New actions [The relative numbering is going to change to match the full meeting minutes] Action (1) on Wilfried Woeber and Carol Orange, To come up with a technical proposal for better DB consistency checking. Action (2) on all WG members, To think about implications of the proposed DB consistency checking and to provide input. Action (3) on RIPE NCC (Carol Orange, Daniel Karrenberg), To report about the FUA findings (on DB consistency) at next ripe meeting. Action (4) on RIPE NCC (Carol Orange, Daniel Karrenberg), To document the approach to find and change references to person objects in order to support migration to mandatory handles (using inverse look up and "search and replace") and send it to the list. Action (5) RIPE NCC (Ambrose Magee) (time allowing), To implement the change that "whois -i ..." implies "whois -i -r ..." Action (6a) on Wilfried Woeber, To ask on the DB-WG mailing list about the need of improved inverse lookup functionality (more attributes allowed for -i). Action (6b) on Daniel Karrenberg, To look at query logs for "failed" -i attempts. Action (7) on Ripe NCC (Ambrose Magee), Produce the RIPE-DB Documentation a.s.a.p. and according to the agreed structure. Action (8a) on RIPE NCC (Carol Orange), Publish the "known bugs" list, on the Web or on the FTP-Server. Action (8b) on RIPE NCC (Carol Orange), Publish the "open issues" list (consolidated wishlist) on the Web or on the FTP server. Action (9) on RIPE NCC (Ambrose Magee, Carol Orange, Daniel Karrenberg), To propose and implement (a.s.ap.) a referral mechanism for TLD domain: objects (only). Action (10) on Wilfried Woeber and the RIPE NCC, To make a proposal on the technical and procedural requirements of making the status: attribute for inetnum: objects mandatory. Action (11) on RIPE NCC (Ambrose Magee), To change the DB-SW to obsolete the advisory: attribute in inetnum: objects. Action (12) on RIPE NCC (Ambrose Magee), To add mailing lists with their descriptions to the database documents. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -WW, 9-OCT-1996 20:05:57

3 2

2nd draft, minutes of DB-WG meeting at RIPE 25
by Wilfried Woeber, UniVie/ACOnet 10 Oct '96

10 Oct '96

This is the 2nd draft of the minutes for the DB WG meeting at RIPE 25. It incoporates comments by David Kessens. I expect to add a couple of URLs to point to presentations as soon as they might become available from the RIPE FTP-Server. The Action List probably needs renumbering, to get it aligned with the summary notes for the plenary. As always, all comments, input and improvements welcome. Wilfried. -------------------------------------------------------------------------- Wilfried Woeber : e-mail: Woeber(a)CC.UniVie.ac.at Computer Center - ACOnet : Vienna University : Tel: +43 1 4065822 355 Universitaetsstrasse 7 : Fax: +43 1 4065822 170 A-1010 Vienna, Austria, Europe : NIC: WW144 -------------------------------------------------------------------------- Minutes for the DB WG meeting RIPE 25, Amsterdam, NL **1st DRAFT**, 9-Oct-1996 Chair: Wilfried Woeber Scribe: John Crain Participants: 51 (according to the sign-up list) A. Administrative stuff John Crain offered to take notes. Thanks a lot, John! The proposed agenda was agreed, including the following additional items: - Input from the DNS-WG and - Input from the Local-IR WG both for agenda item Y. General input from other WGs and - advisories: in aut-num: objects for Z. AOB B. DB-SW status report by the RIPE NCC Carol Orange offered the report on the state of the RIPE Database and the DB-SW. Version 2.0 of the software was released recently and put into production at the NCC. Important new features are: - inverse lookup (whois -i) - the package is now fully classless - support for near real-time-shadowing (As an aside, a DB-Workshop was offered on Monday morning.) More details about the growth of the DB and the increase in the number of objects, etc. were shown on the slides. [ These slides are expected to become available soon ] [ at ftp://ftp.ripe.net/ripe/presentations/ ] Activities for the short term future are - produce documentation - fix any version 2.0 bugs - try to improve consistency In general, version 2.0 is found to run very well. Sabine Dolderer reported on problems with the handling of old (invalid) handles. This is one of the known problems. Sabine was asked to report the problem to the ripe-dbm(a)ripe.net mailbox. C. DB consistency and sanity checks Is it worthwhile doing more sanity checking? There was general consensus that the quality and consistency of the data in the RIPE-DB should be improved. The group formally supports this activity which is also part of the RIPE NCC Activity Plan for 1997. A project has already been set up with the Free University of Amsterdam to look at the consistency issues in general. Students may wish to interview selected users. The Local-IR staff and the users of the Routing Registry are expected to contribute to this activity. The group then identified a couple of obvious inconsistencies which should be checked, either at the time of a transaction or on a regular basis. E.g. - person/role objects being refered to no longer exists - persons (multiple) with and without nic-hdls. - person objects which are no longer referenced by any other object (proposal to eventually remove these objects, considering the age of the object) - multiple route objects with different origin, where there is obviously no trasnition and/or multi homing. The various checks, when they get implemented eventually, should at least generate warnings, both at the time of an update and on a regular basis. The logistics (frequency, identifying the proper recipients for the warnings, etc.) need more work. The discussion of a potential reservation of some NIC-Handle types (like AUTOxx-RIPE, ROLExx-RIPE, etc.) led to the conclusion - that there is no need for special handling of role: objects - and that AUTOxx-RIPE should be reserved to avoid confusion when requesting automatic handle assignment with the AUTO-xx mechanism. Niall O'Reilly suggested that the various registries should help with the transistion process by changing old person objects that are (mis-)used to define a role into correct role objects. D. Inverse lookup discussion Is automatic recursion still useful? Just observed as a fact, quite a few people have aliases defined to disable the automatic recursion for their environment. However, for compatibility reasons and because most of the casual users are expected to need the information provided by the recursion, the decision was to stick with the current default (recursion is on by default, use -r flag to disable it). However, the behaviour of the software when doing an inverse lookup ("-i" flag) is to be changed to imply "-r" (recursion disabled). Discussion then moved on to find out wether inverse lookup should be allowed for all attributes. Common feeling was that it is not necessary for all attributes, but it might be useful to implement for a couple of additional attributes. In order to find out about the real need, a discussion on the DB-WG list should be started and the NCC was asked to look at the logs to find out about "popular" failed queries. Kurt Kayser asked about the software behaviour and the timing involved when a "server running at low priority" message is issued. This message is issued for general queries that (potentially) produce a large number of hits. As long as the TCP connection is up, it's still working. E. Documentation Ambrose Magee was then introduced as the new primary maintainer of the DB-SW. He reported on the proposed structure of the Documentation for the RIPE Database objects, the usage and the software. The group endorsed the concept with some minor proposals for improvement and stressed the importance of the documentation, and in general, the importance of consolidation vs. new functionality. In particular, the proposal is to have a main document that is stable and a set of appendices with things that may change or require more freequent updates. [ The slides are expected to become available soon ] [ at ftp://ftp.ripe.net/ripe/presentations/ ] One of the suggestins was to seperate the advanced sections (e.g. on how to obtain the database etc.) from the general user documentation. Nigel Titley proposed to start working with the sections on the whois client and utilities and move that information to the beginning of the documentation Bernhard Stockman asked for a list of all active objects (Index), including definition of the syntax and the also the semantics. Daniel again proposed to add that functionality to the software itself. This is already on the consolidated wish-list, and to be implemented by adding a whois -T with -v and possibly whois -T to report all objects. whois -T <type> is already available for listing the schema for an individual object. F. New functionality Even thouogh there was some discussion on new functionality, like a URL: attribute and amendments to a cgi script to provide access to the data via html, it was stressed that for the time being consolidation should be the priority instead of new functions. The maintainers of the DB-SW were asked to maintain and publish the "consolidated wish-list" and the "known bug list". Y. General input from other WGs DNS: There was concern over potential conflicts between information in RIPE Database and the TLD's internal databases. Becuase of that potential conflict, some TLD administrators do not register their secondaries in the RIPE DB. Referal to their database would be useful. Daniel Karrenberg reported that this referal mechanism is already on the wish list. No design work has been done yet. There is a couple of potential implications, e.g. the idea to implement the referral for other (all?) objects. Guy Davis observed that it might be useful to do a more specific search on hosts/domains for DNS. Daniel Karrenberg was in favour of more specific, but has mixed feelings on referral because of all the different whois initiatives within the Internet. Thus he suggested to implement a quick fix for the top level domains only. Janos Zsako thought that the referral should be implemented in the server not in the client! Wilfried Woeber observed that the referral approach can only be useful if most TLD's are willing to supply an interface for whois. There was an indication of considerable interest to offer access to the local TLD data. Local-IR: Mandatory attributes (as described in the regsistry procedures) should be made mandatory in the database (enforced by the software). In particular the status: attribute in inetnum: objects and the nic-hdl: attribute in general should be made mandatory. Of course, this implies a major clean-up of the database by the LIR's! Daniel Karrenberg observed that initially this will only be applicable for new objects. The clean-up will come later. Kevin Hoadley asked for some time to implement the necessary changes to local scripts and procedures. The following discussion led to a proposal to start enforcing this on the 1st of January 1997. A. Blasco Bonito asked about potential implications of making the 'status:' attribute mandatory. Carol Orange suggested to define an additional value for the status: (USED?) to distinguish between address space allocations, assignments and usage. A. Blasco Bonito's comment that "The general attitude of LIR's who are large providers is to assign to other smaller providers. These then distribute to their customers." prompted Daniel Karrenberg to state that "If this is the case then they are not following the registration policies." A mandatory status: attribute has implications that have to be discussed in more detail. Z. AOB Brian Renaud proposed to obsolete the advisory: attribute for aut-num: objects. For 'route:' objects, these atrtributes should remain valid. Ther was general agreement and the NCC was asked to implement that change. Following up on a mail received from David Kessens, ther was a review of the various mailing lists that dal with different aspects of the database business. These are: DB-WG DB-BETA RRIMPL Daniel Karrenberg described the scope of these lists and observed that all three have their own specific use: - DB-BETA is for people running new code. - DB-WG is for architecture discussions - RRIMPL has a wider scope Bugs in the released version of the RIPE DB-SW should be submitted to ripe-dbm(a)ripe.net discussions about details of running new code should go to db-beta@ripe-net. Sabine Dolderer observed that if there are two maintainers then the "weakest" one is used. Daniel Karrenberg pointed out that there is no hierarchy involved. The software just scans all entries and tries to find a match. The first match found is honored. This is in line with the current concept and design. This is the correct behaviour as all maintainers are equally authorized. To allow specific maintainer functions is at the moment beyond the functionality of the DB. Appendix: New actions [The relative numbering is going to change to match the full meeting minutes] Action (1) on Wilfried Woeber and Carol Orange, To come up with a technical proposal for better DB consistency checking. Action (2) on all WG members, To think about implications of the proposed DB consistency checking and to provide input. Action (3) on RIPE NCC (Carol Orange, Daniel Karrenberg), To report about the FUA findings (on DB consistency) at next ripe meeting. Action (4) on RIPE NCC (Carol Orange, Daniel Karrenberg), To document the approach to find and change references to person objects in order to support migration to mandatory handles (using inverse look up and "search and replace") and send it to the list. Action (5) RIPE NCC (Ambrose Magee) (time allowing), To implement the change that "whois -i ..." implies "whois -i -r ..." Action (6a) on Wilfried Woeber, To ask on the DB-WG mailing list about the need of improved inverse lookup functionality (more attributes allowed for -i). Action (6b) on Daniel Karrenberg, To look at query logs for "failed" -i attempts. Action (7) on Ripe NCC (Ambrose Magee), Produce the RIPE-DB Documentation a.s.a.p. and according to the agreed structure. Action (8a) on RIPE NCC (Carol Orange), Publish the "known bugs" list, on the Web or on the FTP-Server. Action (8b) on RIPE NCC (Carol Orange), Publish the "open issues" list (consolidated wishlist) on the Web or on the FTP server. Action (9) on RIPE NCC (Ambrose Magee, Carol Orange, Daniel Karrenberg), To propose and implement (a.s.ap.) a referral mechanism for TLD domain: objects (only). Action (10) on Wilfried Woeber and the RIPE NCC, To make a proposal on the technical and procedural requirements of making the status: attribute for inetnum: objects mandatory. Action (11) on RIPE NCC (Ambrose Magee), To change the DB-SW to obsolete the advisory: attribute in aut-num: objects. Action (12) on RIPE NCC (Ambrose Magee), To add mailing lists with their descriptions to the database documents. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -WW, 9-OCT-1996 20:05:57

2 1

Slides of RIPE Database 2.0 Workshop
by RIPE Database Administration 10 Oct '96

10 Oct '96

Dear Colleagues, The slides for the RIPE Database 2.0 Workshop (held during the recent RIPE meeting) are available at: ftp://ftp.ripe.net/ripe/presentations/ripe-m25-orange-DB-Workshop.ps.tar.gz Please send any comments to <ripe-dbm(a)ripe.net>. Regards, Ambrose Magee ____________________________ RIPE Database Administration.

1 0

Re: draft minutes of DB-WG meeting at RIPE 25
by Wilfried Woeber, UniVie/ACOnet 10 Oct '96

10 Oct '96

=> The various checks, when they get implemented eventually, should at least => generate warnings, both at the time of an update and on a regular basis. = =One note on this one: = =The new software doesn't create a new object anymore when adding a NIC =handle to an existing object. It will only do this when there are other =differences between the new and old object then the 'changed:' and the =added NIC handle attribute. Thanks for the note. => Daniel proposed to add that functionality toi the software itself, probably => by adding a whois -T with -v and possibly whois -T to report all objects. => This is is already available for listing the schema for an individual object. = =Wasn't this proposal on the wish list for a long, long time (and even =accepted by the database working group as a good thing) ?!? = =davidk> List of current proposals: =davidk> =davidk> 5) whois -tv Yes it was. I'll re-word that to properly reflect history. From the technical point of view, it doesn't hurt to propose it again for implementation :-) => Action (11) on RIPE NCC (Ambrose Magee), => To change the DB-SW to obsolete the advisory: attribute in inetnum: objects. = ^^^^^^^^ = =No further comments ... ;-) That's the fate of lazy people (like me) who do a cut-n-paste from the local *broken* copy. Yours, standing at the corner of the room, and facing the walls, Wilfried.

1 0

Re: Fwd: Re: Short summaries of WG sessions [DataBase]
by Wilfried Woeber, UniVie/ACOnet 09 Oct '96

09 Oct '96

Hi David ! => Z. AOB => - clarification abou mailing lists => - obsolete advisories in inetnum objects => => Action (11) on RIPE NCC (Ambrose Magee), => To change the DB-SW to obsolete the advisory: attribute in inetnum: objects. = =Just to avoid even more confusion on this trickey advisory topic: = =I assume you are talking about the 'aut-num:' object since the 'inetnum:' =object never had support for this ... You're right, of course. I was mixed up, and John's notes are correct anyway! Could the minute folks at the NCC (Roderik?) please fix the text ? Thanks and apologies, Wilfried.

1 0

Fwd: Re: Short summaries of WG sessions [DataBase]
by Wilfried Woeber, UniVie/ACOnet 08 Oct '96

08 Oct '96

The attached text fragment for the RIPE 25 Minutes was submitted yesterday. You might want to have that as an interim summary while the full DB-WG minutes are being worked on... Cheers, Wilfreid. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here's the Summary for the DB-WG Meeting, RIPE 25, Amsterdam I might again be too verbose, I don't know. Please feel free to fix my typos and any broken sentences. Roderick, apologies for the one-day delay over the promise... Wilfried. Chair: Wilfried Woeber Scribe: John LeRoy Crain Participants: 51 (according to the sign-up list) The following agenda was agreed: A. Administrative stuff - volunteering of the scribe - WG-agenda bashing B. DB-SW status report by the RIPE NCC - recent changes, V2.0 - new features: experience, feedback C. DB consistency and sanity checks - at time of transaction (create/modify) - at regular intervals - input from other WGs - input from the NCC (address space handling) - RIPE/(NIC)-Handle space, special cases, like ROLExxx-RIPE, RIPExxx-yyy, AUTOxxx-RIPE D. Inverse lookup discussion - recursive lookup on by default, should this change? - extending the list of attributes that can be use in -i lookup? E. Documentation - structure and contents as proposed by the NCC - user input F. New functionality - things on the consolidated wish-list - url handling - inet-rtr object extension Y. General input from other WGs - DNS WG: registration of 2nd level (and lower) domains - Local-IR WG: proposal to have the SW enforce mandatory attributes (status, NIC-handles) Z. AOB - clarification abou mailing lists - obsolete advisories in inetnum objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [ The following summary is not necessarily following ] [ the actual agenda and sequence of discussions, for ] [ a complete report refer to the WG minutes, please! ] Results and decisions The usual report about the status of the DB and the DB software was offered by the RIPE-NCC (Carol Orange) and well received by the group. There is still a couple of minor bugs around (as can be expected anyway when implementing a major new softeware release), these bugs are being fixed asap. The behaviour of the software when doing an inverse lookup ("-i" flag) is to be changed to imply "-r" (recursion disabled). NIC-Handles (in general) and the status: attribute for inetnum: objects should be enforced by the DB software. This is to bring the software in line with the requirements in the address allocation guidelines documentation. The proposed flag day for these changes is 1.1.1997. In order to avoid confusion, the string AUTO (which is used to request and control automatic handle assignment) shall be reserved (not to be used for a real handle). There was general consensus that the quality and consistency of the data in the RIPE-DB should be improved. The group formally supports this activity which is also part of the RIPE NCC Activity Plan for 1997. A project has been set up with the Free University of Amsterdam to look at the consistency issues. Students may wish to interview selected users. Current thinking is that there shall be more sanity checking at the time of a transaction (each time) and that we need some background checks that can be performed at regular intervals. These should then catch inconsistencies which can not be properly detected by the transaction tests. Making this process succeed is expected to require collaboration and some effort from Local-IR folks as well as from the routing folks. Ambrose Magee was introduced as the new primary maintainer of the DB-SW. He then reported on the proposed structure of the Documentation for the RIPE Database objects, the usage and the software. The group endorsed the concept with some minor proposals for improvement and stressed the importance of the documentation, and in general, the importance of consolidation vs. new functionality. Input received from the DNS-WG asked for a referral mechanism in the RIPE-DB to support registering 2nd and lower level domain objects at the site of a TLD Administration, but still offering access with whois by way of the RIPE-DB. There was a reasonable number of TLDs represented that promised support on their end for implementing that approach. The decision reached calls for an interim solution to add a referral mechanism to domain: objects. This would only be used in (TLD) domain: objects that describe the TLD and point to the authoritative DB that holds the 2nd and any lower level data. During the discussion of the DB-SW behaviour of handling multiple maintainer entries, it was clarified that there is no hierarchy involved. The software just scans all entries and tries to find a match. The first match found is honored. This is in line with the current concept and design. New actions: Action (1) on Wilfried Woeber and Carol Orange, To come up with a technical proposal for better DB consistency checking. Action (2) on all WG members, To think about implications of the proposed DB consistency checking and to provide input. Action (3) on RIPE NCC (Carol Orange, Daniel Karrenberg), To report about the FUA findings (on DB consistency) at next ripe meeting. Action (4) on RIPE NCC (Carol Orange, Daniel Karrenberg), To document the approach to find and change references to person objects in order to support migration to mandatory handles (using inverse look up and "search and replace") and send it to the list. Action (5) RIPE NCC (Ambrose Magee) (time allowing), To implement the change that "whois -i ..." implies "whois -i -r ..." Action (6a) on Wilfried Woeber, To ask on the DB-WG mailing list about the need of improved inverse lookup functionality (more attributes allowed for -i). Action (6b) on Daniel Karrenberg, To look at query logs for "failed" -i attempts. Action (7) on Ripe NCC (Ambrose Magee), Produce the RIPE-DB Documentation a.s.a.p. and according to the agreed structure. Action (8a) on RIPE NCC (Carol Orange), Publish the "known bugs" list, on the Web or on the FTP-Server. Action (8b) on RIPE NCC (Carol Orange), Publish the "open issues" list (consolidated wishlist) on the Web or on the FTP server. Action (9) on RIPE NCC (Ambrose Magee, Carol Orange, Daniel Karrenberg), To propose and implement (a.s.ap.) a referral mechanism for TLD domain: objects (only). Action (10) on Wilfried Woeber and the RIPE NCC, To make a proposal on the technical and procedural requirements of making the status: attribute for inetnum: objects mandatory. Action (11) on RIPE NCC (Ambrose Magee), To change the DB-SW to obsolete the advisory: attribute in inetnum: objects. Action (12) on RIPE NCC (Ambrose Magee), To add mailing lists with their descriptions to the database documents. WW, 7-OCT-1996. -------------------------------------------------------------------------- Wilfried Woeber : e-mail: Woeber(a)CC.UniVie.ac.at Computer Center - ACOnet : Vienna University : Tel: +43 1 4065822 355 Universitaetsstrasse 7 : Fax: +43 1 4065822 170 A-1010 Vienna, Austria, Europe : NIC: WW144 --------------------------------------------------------------------------

2 1

TEST database.
by RIPE Database Administration 04 Oct '96

04 Oct '96

Dear Colleagues, The test version of the RIPE database is back on-line. Requests for maintainer objects in the test database should be sent to <ripe-dbm(a)ripe.net>. This is to be changed in the near future. Participants in the RIPE Database 2.0 Workshop during the recent RIPE meeting had test objects and maintainers created for them in the test database. If you wish these objects to remain in the test database, please send a message to <ripe-dbm(a)ripe.net> as soon as possible. Regards, Ambrose Magee ____________________________ RIPE Database Administration.

1 0

Re: mnt-lower attribute not allowed?
by Wilfried Woeber, UniVie/ACOnet 04 Oct '96

04 Oct '96

Hi Steven et.al. ! => => The AS object that matches the origin AS of a route object could be the => parent object of the route object, but then people are still able to put => route objects in with another origin AS which might not be desired by the => owner of the non-portable CIDR address space. Furthermore, a second => hierarchy exists: the IP prefix tree. => => You might want to try to formulate a proposal that is not too => complicated and also addresses the concerns of most people ... = =How about making the inetnum object the parent of the route object? =I think it would make sense to allow the "owner" of the inetnum to create =the route objects.. I'm not convinced that this is the way to go. For two major reasons: - in many environments the routing is handled differently from the address allocation. In particular, where filters or other things are (semi-)automatically derived from objects in the DB, it's certainly not feasable to have the customers ("owner" of the address space) change things without coordination. If they do, it might not reflect reality at all or it might interfere with local logistics. And there's the issue of porting PA address space for a limited amount of time to a different ISP. - I think we should come up with a concept that works as well outside the RIPE-coordinated address space and the central registry concept. I think we should . come up with a solution that works for routing-registries that do not include an address registry in the same DB, . find a solution that works uniformly across all parts of the address space (A, B, C-19[3-5], TWD, et.al.) = You can also keep the owner in the "notify:" field of =the route object, so once you delegate maintenance of the route object to =someone else, you can still keep track of what's happening with it. = =Or does this sound ludicrous? No, it made me think, thus it was a very useful approach :-) I'd favour an approach that is (conceptually) similar to the handling of domain objects - something like an upstream search based on the AS #. Although I haven't a clear picture of how it could work in reality, because the "tree" is certainly less well-defined than in the DNS namespace. Cheers, Wilfried. -------------------------------------------------------------------------- Wilfried Woeber : e-mail: Woeber(a)CC.UniVie.ac.at Computer Center - ACOnet : Vienna University : Tel: +43 1 4065822 355 Universitaetsstrasse 7 : Fax: +43 1 4065822 170 A-1010 Vienna, Austria, Europe : NIC: WW144 --------------------------------------------------------------------------

1 0

mnt-lower attribute not allowed?
by Schmitz＠rus.uni-stuttgart.de 04 Oct '96

04 Oct '96

Dear colleagues, yesterday I tried to apply the hierarchical authorisation scheme on route objects. However, I did not succeed. As an example, have a look at this: route: 193.174.0.0/15 descr: DFN-AGG-1 origin: AS1275 mnt-by: DFN-MNT mnt-lower: DFN-MNT changed: poldi(a)dfn.de 950907 changed: noc(a)noc.dfn.de 961001 source: RIPE password: ... resulted in: Update FAILED: [route] 193.174.0.0/15 route: 193.174.0.0/15 descr: DFN-AGG-1 origin: AS1275 mnt-by: DFN-MNT changed: poldi(a)dfn.de 950907 changed: noc(a)noc.dfn.de 961001 source: RIPE *ERROR*: attribute "mnt-lower" unknown in route object Is this just an error in typing (I took the same spelling as given in the RIPE database 2.0 workshop), or is hierarchical authorisation not done for route objects (in the workshop there was only an example for inetnum objects), or is it just not implemented yet? If it was implemented only for inetnum objects I do not see any reason why it should not be also possible for route objects. Could anybody please give a hint? Regards Joachim _____________________________________________________________________________ DFN Network Operation Center, Dr. Joachim Schmitz, (finger help(a)noc.dfn.de) >>>>>> mailto: noc(a)noc.dfn.de <<<<<< Rechenzentrum Universitaet Stuttgart, Allmandring 30, D-70550 Stuttgart Phone: 0711-685-5810, 0711-685-5576, FAX +711 6788363 (business hours) EMERGENCY phone +711 1319 139 with answering machine and pager _____________________________________________________________________________

4 3

Hierarchical Authorisation.
by RIPE Database Administration 02 Oct '96

02 Oct '96

Hi, Joachim ! Hierarchical authorisation is currently available for inetnum and domain objects. >From the HELP & HOWTO document: - You can protect against people *creating* (only creating) objects direct (one level) below in the hierarchy of an object type (only for 'inetnum:/domain:' objects) by using your maintainer in a 'mnt-lower:' attribute. The authorization method of this maintainer object will then be used upon creation of any object direct below the object that contains the 'mnt-lower:' attribute. This is also mentioned in the Release Notes of the new software. Hope this helps. Regards, Ambrose Magee ____________________________ RIPE Database Administration.

1 0

More attributes for indexing?
by Steven Bakker 02 Oct '96

02 Oct '96

Hi folks, At the last RIPE meeting there was some discussion about which attributes would be useful to keep indices on. Well, since our office move last weekend, I came up with a (rather predictable) one: $ whois -i ad "Lockton House" % ignored attribute "address" since it is not indexed as a reference % no attributes specified for -i query $ _ Dang! It would be nice to have an index on address, although such an index might become veeerrrry large, due to the fact that you can specify multiple "address" attributes per object; dunno. Any thoughts? Cheers, Steven

4 3

Re: interesting person entry...
by Wilfried Woeber, UniVie/ACOnet 27 Sep '96

27 Sep '96

=> I personally feel that this should be done through the config file: give => every attribute a property of allowed empty lines or not. For example => 'remarks:' lines can be read easier when they may contain empty lines => while empty rev-srv: lines are useless. => I roughly agree about the approach, but see below. > Wilfried: I suggest a collection of all attributes should be analyzed > to see where empty lines are useful and where not. I currently do not > have an overview of all attributes but I feel that > - single ones > - attributes used as reference (e.g. tech-c) > - attributes necessary for administrative database purposes (e.g. changed > or source) > must not be empty (just as it came to my mind). Agreed. > I wonder how many > attributes might be left over which could carry empty lines. I feel that > the address attribute should not have empty lines. However, empty lines > could be useful as David noted on the remark attribute. I think all the lines where syntax checking is being done should *not* allow empty lines. To some degree I can see the usefulness of *single* empty lines within a *block* of attributes that allow free format text. Currently remark:, trouble:, and address: comes to my mind - and maybe a few more. But even then, the optical effect that is described can easily be obtained by e.g. putting a "." or "-" as the only character into the value field. I suppose that would remove some complexity from the software to support this "cosmetic sugar". Thus it would be my personal proposal to disallow empty lines altogether (empty in the sense of an attribute having no value supplied). Your input is very welcome, and yes, that could be put onto the draft agenda for RIPE-26 if there is no clear consensus or any strong feelings. Wilfried. -------------------------------------------------------------------------- Wilfried Woeber : e-mail: Woeber(a)CC.UniVie.ac.at Computer Center - ACOnet : Vienna University : Tel: +43 1 4065822 355 Universitaetsstrasse 7 : Fax: +43 1 4065822 170 A-1010 Vienna, Austria, Europe : NIC: WW144 --------------------------------------------------------------------------

2 1

interesting person entry...
by Schmitz＠rus.uni-stuttgart.de 26 Sep '96

26 Sep '96

Dear colleagues, dear Ambrose, by accident I discovered a person entry which in its form is definitely not wanted: *pn: Stefan Seiz *ad: Herdweg 73 *ad: \ ... > 128 empty address lines!!! *ad: / *ad: 70174 Stuttgart *ad: Germany *ph: +49 711 18748-0 *fx: +49 711 18748-32 *em: index(a)schwaben.com *nh: SSE2-RIPE *ny: noc(a)ecrc.de *ch: wer(a)ecrc.de 951115 *so: RIPE I am in no position to decide whether the entry was done including these empty lines or whether this is an error of the database software. I think a check should be implemented during acceptance of new database objects (or updates) which - removes empty input lines (or fields which do contain nothing but white space) - afterwards checks whether all mandatory fields for the object are filled in should be done (actually I expect such a test to be performed anyway...) Could anybody please check what is wrong here? Thank you! Joachim _____________________________________________________________________________ DFN Network Operation Center, Dr. Joachim Schmitz, (finger help(a)noc.dfn.de) >>>>>> mailto: noc(a)noc.dfn.de <<<<<< Rechenzentrum Universitaet Stuttgart, Allmandring 30, D-70550 Stuttgart Phone: 0711-685-5810, 0711-685-5576, FAX +711 6788363 (business hours) EMERGENCY phone +711 1319 139 with answering machine and pager _____________________________________________________________________________

3 3

interesting person entry... (followup)
by Schmitz＠rus.uni-stuttgart.de 26 Sep '96

26 Sep '96

Dear colleagues, regarding empty input lines in the RIPE database: rubin$ egrep '^\*ad: *$' ripe.db.pn | wc -l 13442 rubin$ egrep '^\*..: *$' ripe.db.* | wc -l 15522 (with the database as of this morning around 2:10 am) I estimate this really should be changed... Regards Joachim _____________________________________________________________________________ DFN Network Operation Center, Dr. Joachim Schmitz, (finger help(a)noc.dfn.de) >>>>>> mailto: noc(a)noc.dfn.de <<<<<< Rechenzentrum Universitaet Stuttgart, Allmandring 30, D-70550 Stuttgart Phone: 0711-685-5810, 0711-685-5576, FAX +711 6788363 (business hours) EMERGENCY phone +711 1319 139 with answering machine and pager _____________________________________________________________________________

1 0

RE: RE: HELP! 'networkupdate', diskussion on short names of attributes
by Markus Bock 20 Sep '96

20 Sep '96

Hi all ! Contents: --------- 1. Thanksgiving for quick response 2. More about the problem with 'networkupdate' 3. Comments on short names of attributes 4. Comments on whois -i option 5. Question on destination for emails in 'RIPE DB' context 1. Thanksgiving for quick response ================================== At first I would like to thank - Wilfried Woeber, UniVie/ACOnet - Geert Jan de Groot, RIPE NCC - Carol Orange, RIPE NCC and specially - David Kessens who gave me quick response to my 'cry for help'. Thank you very much for your suggestions and for taking time to think about that problem with the 'networkupdate' client programm. Below you can read more about the progress in that case. And no, there is no reason to say sorry for a 'delayed response'. I'm pleased about your quick and helpful reactions. Thank you again! 2. More about the problem with 'networkupdate' ============================================= Below i repeat parts of David Kessens answer ... > Hi Markus, > > Markus Bock writes : > > > > For writing database front ends which manipulate and visualisize > > the stored data we want to use the 'networkupdate' program to > > have access to the database from different workstations via LAN. > This function is indeed present for this purpose. Yes, great. I think with the direct network update-method one has many potentialities to develope client-server based applications with a two task architecture like you see in the sketch below (just an example). ----------------- client-area | application | ----------- ----------------- | networkupdate | ----------------- | ^ v | communication drivers & protokolls // // network ======================================== \\ \\ // || VV communication drivers & protokolls | ^ v | ----------------- server-area | DB-server | ----------- | whoisd | ----------------- | | ----------------- / \ / \ ________ ___________ (________) (___________) | DB | | DB | | source | | source | | XXXX | | OWN | -------- ----------- > Maybe I make something wrong or this effect has something to do with > > running under Slowaris 2.5 ... ? > May be, may be not. Could you try the C version of networkupdate > (available from ftp.ripe.net/tools/whois-tools*) and see if the same > problem happens ? Yes I did it. The same problem happens NOT. I performed several tests with the C version and everything is working fine giving the estimated results. Just for information: --------------------- I had to do the following changes inside the makefile of the C versions of the ripe whois tools to compile it successful at our site under Solaris 2.5 with gcc: - define '-DSYSV' added - additional LIBS 'LIBS= -lresolv -lsocket -lnsl' added - changing the syntax to call install # install: ${PROG} # $(INSTALL) -c ${DESTDIR} -s -m 755 ${PROG} networkupdate # $(INSTALL) -c ${MANDIR}/whois.1 -m 444 whois.man > Could you try 'networkupdate -h whois.ripe.net'. You should get a message > back that tells you that you are not authorized to network updates. The > problem is problably in the client side of the software if you don't get > this message. Yes, I did it. - With the perl version I didn't received the message that tells me that I'm not authorized to do network updates. There is an analogical problem like I described in my 'cry for help'. 'networkupdate' terminates immediately. - With the C version I received the estimated message % ERROR: ***You are not authorized to do network updates***. With this results I agree with you that the problem is problably in the client side of the software. I tried to find out what is potentially wrong in the software by taking the examples for a client and a server from L. Walls camel book and comparing it with the RIPE DB SW (I assume that the same mechanisms are used by the RIPE DB SW). But I was still not able to change it in that way that these examples are able to handle the input of multiple lines (I'm not familar with 'command buffering or not' stuff) or to produce an analogical failure like decribed above. So I have no further result with this case. Has anyone tested the perl version of the networkupdate programm succesfully at his site ? If yes, in which enviroment (OS, version, HW etc.) ? > Please contact <ripe-dbm(a)ripe.net> and see if they can help you with > extra debugging code in the whoisd daemon/whois client to do a more > detailed trace of what is happening in the server software. Yes, thank you. I'll do it. > I hope this helps, > David K. > PS Please don't expect any fast responses right now since I am still busy > with moving & am not working yet. I have, however, already built a new > computer & got a ppp account at ISI so I am not completely > disconnected anymore ;-). Yes, thank you again. Your help enables me to go sucessfully on with my work. Also this gives me the opportunity to wish you success and good luck for your future at ISI. 3. Comments on short names of attributes ======================================== First: ------ While testing the RIPE DB for use to store own data objects I examined the following cases: Assume that you have to store 30 or more new database objects with about 300 (or more) new attributes in a (SPLIT) RIPE database (I already did it). Because you will have problems to find sensefull short names for attributes you use a numbering scheme like this: # DB-object dummy A1 ATTR A1 dummy_longname_1 ATTR A2 dummy_longname_2 ATTR A3 dummy_longname_3 # DB-object dummy A1 OBJ A1 ATSQ A1 A2 A3 OBJ A1 MAND A1 OBJ A1 MULT A2 A3 OBJ A1 UNIQ A1 OBJ A1 KEYS A1 Seems ok. *** But be careful. Don't use uppercase letters *** for naming database objects and attributes *** in a SPLIT database. The side effects are that at first everything is working fine (indexing, dbupdate updates via email etc.). The data will be stored correctly. But if you query the database via whois and whoisd you ever get the message that will be displayed if no match was found (have a look in your ripedb-config file). What happened ? After I had a look at the source code of the whoisd I think one can find the explanation in this section # sub findsplitdbs { # local(*keys, @preselection)=@_; # local(@searchdb)=(); # # print STDERR join(" ", keys %CLASSLESSDBS), "\n"; # @preselection=grep($CLASSLESSDBS{$_}, @preselection) if (scalar(grep(/^$VALIDPREFIXKEY$/o, @keys))==scalar(@keys)); # # # # and now we are going to do something really smart... # # and trickey things # local(@upperkeys)=@keys; # grep(tr/a-z/A-Z/, @upperkeys); # # # # see comment below why we need this # local(@tmpkeys,@tmpupperkeys); # local($tmp); and so on. I think that with this the software is not able to find the datafiles, i.e. which are named by this konvention /whereever/it/is/SOURCE.db.OBJECT /whereever/it/is/SOURCE.db.OBJECT.pag /whereever/it/is/SOURCE.db.OBJECT.dir where OBJECT is in uppercase letters. Does somebody agree with me ? Second: ------- Above I wrote about a problem to find sensefull short names if you have to store many new database objects with many (many) attributes. What you have at least for this is a two letter code with all alphabetic characters (in lowercase) and all digits. So you have (26+10) * (26+10) = 1296 (I hope it is correct...) possibilities for short names. This seems to be enough. But you have still the problem to find sensefull short names for a large amount of database objects and attributes. Maybe I see a problem 'where no problem is' but I can assume that in future even the RIPE database (if it will growth by objects and attributes) will reache the border finding sensefull names for short attributes. So my questions are: - Is it possible to use in future only the long names of attributes and objects or - needs this change too much manpower and time ? >From a view of data modelling I think the short names are redundant even if I know that from a view of implementing the database and writing code they are useful (i.e. for doing syntax checks). 4. Comments on whois -i option ============================== Wow ! I really like this option. If you have a somewhat clever data modelling you can realize something like a 'where clause' which is known from SQL. Another thing is that you can realize something like a 'join' over two database objects. You just have to play a little bit around with the 'KEY' and 'REC' terms inside the definition of your database objects. 5. Question on destination for emails ===================================== in 'RIPE DB' context ==================== I don't know if the database working group is the right address to report everthing in context RIPE database. Should one better report only for i.e. problems with implementation and porting the software directly to the database maintainer at RIPE NCC <ripe-dbm(a)ripe.net> ? and on the other side Should one better report only i.e. issues about the database concepts directly to the database working group ? Is there somewhat like a rule for this ? Thanks again ! have a nice day, Markus

2 1

FYI: List of current proposals
by davidk＠ISI.EDU 20 Sep '96

20 Sep '96

Dear all, This is my latest (and last) list of active proposals. There is still appears to be enough work for the new database maintainer(s) ;-). List of current proposals: 1) AUTO NIC handles (included in 2.00 release) 2) Role object (included in 2.00 release) 3) AS macro syntax 4) inverse lookup (included in 2.00 release) 5) whois -tv 6) referral mechanism 7) INADDR 8) NEW (included in 2.00 release) 9) stronger authorization 10) extended as-in 11) stored/processed attribute 12) name tag 13) url attribute 14) syntactic cleanup 15) WWW-WAIS-database interface 16) whois -H, cgi-bin behavior and HTML formatted output 17) reference exist check 18) don't delete referenced objects You can find a more detailed description below. Please let the new database maintainer(s) know if you know of any other proposals that I forgot about, Kind regards, David K. ---- 1) AUTO NIC handles: Auto assignment of NIC handles Current state: DONE Time estimation: long Syntax: nic-hdl: AUTO-#[Initials] where: # - is an arbritary identification number Initials - optional, advises the database software to use initials for the generation of the NIC handle. will cause NIC handles to be generated automatically. The number is needed to give the possibility for autogenerating and referencing more then one person object (that doesn't have a NIC handle yet) under every condition in one RIPE database update. AUTO is used as a prefix to avoid any confusion between real NIC handles and requests for auto generating NIC handles. Example: person: NIC Handle nic-hdl: AUTO-45NH [... stuff deleted ... ] person: Nick Hansson nic-hdl: AUTO-3NH [... stuff deleted ... ] inetnum: x.y.z.0 - x.y.z.255 admin-c: AUTO-45NH tech-c: AUTO-3NH [... stuff deleted ... ] Will assign a NIC handle with Initials NH and the first free number to 'NIC Handle' and NH and the next free number to 'Nick Hansson'. It will also fill these NIC handles in the right attributes of the objects. 2) New role objects Current state: DONE Time estimation: medium Syntax: The definition of the object: - name will be 'role:' - will have the same attributes as a person object - will have a 'tech-c:' and 'admin-c:' attribute - A NIC handle will be generated just as with person objects How to reference the object: - The object can be referenced from 'tech-c' and 'zone-c:' attributes only - recursive lookups go only one level deep 3) AS macro syntax New syntax for community and AS macros Current state: nothing done, need approval of the RIPE minutes first Time estimation: very short New syntax: community: Format: <string> The <stringmust satisfy the following constraints and the constaints for as macros (see below): + It must not start with the any upper or lower case permutations of the character "A" followed by the character "S". That is, the pattern "[Aa][Ss]" is not legal at the start of a community name. + It must not be the same as any of the routing policy expression keywords. (See RIPE-181, Appendix A for a complete list of these keywords.) It can expected that the keywords that the IETF Routing Policy System (RPS) working group defines for RPSL will be added to this list of keywords. as-macro: Format: AS-<string> The <stringmust satisfy the following constraints: + It must be at least two characters long. + It must contain only upper case letters, lower case letters, digits, or dashes (underscores (_) were dropped to avoid confusion between dashes and underscores) + It must start with a letter + It must end with a letter or digit All comparisons will be case insensitive. 4) inverse lookup Current state: DONE Time estimation: medium What is it? Inverse lookup makes it possible to find objects that match a certain key in a specified attribute. Example: whois -i admin-c,tech-c,zone-c DK13-RIPE gives all objects that have DK13-RIPE as their admin-c, tech-c or zone-c. 5) verbose templates/whois -tv Current state: data collection (from the RIPE documents) started Time estimation: short/medium, see note below Notes: - This will probably be a joint project with Merit. - The data collection part takes most time but is needed for some other projects anyway: We need documentation on all our objects for our own documentation, the RPSL IETF working group and future work to get the InterNIC and RIPE syntax synchronized. What is it? Verbose templates are meant to give more information on object templates then we give now. It is intended that 'whois -v -t inetnum' will give the syntax of the attribute (as with 'whois -t'), a short description of the attribute and a 'minimal' regular expression to give a better idea on how the attribute should look like. 6) referral mechanism Current state: No detailed proposal, nothing done Time estimation: short/medium Notes: This is not a agreed on proposal but is interesting to implement for experimenting with the topic, although it is of use for the community even in an experimental phase. More sophistacted designs might come up that will need much more time to implement and can be regarded as separate projects. What is it? We want to have an automatic referral mechanism that will make it possible that RIPE database objects refer to other whois servers and your client can then search that servers automatically. Example: domain: NL refer: WHOIS domain-registry.nl 7) INADDR reverse delegation integration with the RIPE database Current state: nothing done Time estimation: short/medium This proposal will allow for sending your reverse delegation requests directly to the RIPE database. The updated objects are sent to the reverse delegation robot after the update succeeded if a special keyword (INADDR?) is present in the Subject: line of your message. 8) NEW Current state: DONE Time estimation: short substitute the <auto-assign(a)ripe.netmail box by the keyword NEW in the update to the RIPE database. Only new objects will then be accepted for addition to the RIPE daatabase. 9) stronger authorization Current state: No proposals, nothing done, see note Time estimation: ??? Note: - Merit has a mechanism, but it is unclear if it is general enough and might need too much human intervention when dealing with a database as big as RIPE has. Also, import restrictions might disallow us to use Merit's code. No proposals on how to do this exactly are available right now. 10) extended as-in extended as-in/as-out attribute Current state: Proposal circulated, nothing done at RIPE, see note Time estimation: ???, but probably not too long if Brian Renaud's implementation is good. Note: - Brian Renaud (Merit) has made an implementation and has sent it in. I haven't had time to really look at it. It looks promising. Previous concerns that I had because this implementation required certain external tools seems like they have been addressed. What is it? See proposal by Cengiz Alaettinoglu <cengiz(a)ISI.EDUthat is circulated to the RPSL IETF working group mail list. 11) stored/processed attribute Current state: Proposal approved. Nothing implemented yet. Time estimation: short/medium What is it: Attributes added to all objects that contain a time-stamp and serial number. This allows for better knowledge on the history and age of objects. The attributes will also make the near real time mirroring more robust then it is now. In the future features like rolling back and forth to certain database states can be supported but need to be addressed in a separate project. A proposal has been circulated and appoved to the RIPE database working group. 12) name tag handle (Full name of person) data in tech-c/admin-c/zone-c attributes Current state: No detailed proposal has been made yet. Time estimation: short/medium (might be more complicated then expected right now) The proposal makes trouble shooting with disappeared person objects easier (esspecially when person might be located in a different database). The proposal als allows for non-recursive lookups in many cases when the user has enough info when seeing the name only. 13) url attribute Current state: No proposal yet Time estimation: short What is it? Attribute that can be used in any database object for references to any other site by using an url. No exact proposal has been made yet, but the idea seems to be very useful. 14) Syntactic cleanup Current state: Nothing done yet Time estimation: short/medium It is perceived by the user community as well as by the software developers for the routing registry that some of the syntax is unecessary complicated and/or makes more confusion then it solves. Therefore it was decided on the last RIPE meeting to remove the following features: - the ripe-181 line continuation will no longer be supported. People that used it will be asked to update their objects. - The syntactic sugar words will always be mandatory to avoid all kind of interesting problems (for example with deletions). The raw database file will also contain the syntactic sugar. We will announce this change long in advance to the proper lists to give people the chance to make changes to their software if necessary. 15) WWW-WAIS-database interface Current state: running implemention at GARR-NIS available Time estimation: medium What is it? Check out http://www.nis.garr.it/services/whois.html and select: 'Accesso via WAIS al servizio WHOIS' 16) whois -H, cgi-bin behavior and HTML formatted output Current state: design needs to be made Time estimation: short/medium What is it? There are already several whois client CGI scripts available, however many people believe that the HTML formatting could be done at the server side which allows for (more) consistent ouput for the users and immediate availability of new features. 17) reference exist check Current state: hooks are available in the code Time estimation: short/medium, see note below What is it? Checks if the references that are used in an object exist when doing a creation/update of an object. This is a missing feature that is mandatory in any decent database system. Note: a 100% safe check is very difficult to implement (for split databases) in the current code and will take considerable more time then the above estimate. However, the simple test will come very close to 100% and is considered to be enough for now. 18) don't delete referenced objects Current state: hooks are available in the code Time estimation: short/medium, see note below What is it? Checks if an object is referenced by other objects before deleting it. This is a missing feature that is mandatory in any decent database system. Note: a 100% safe check is very difficult to implement (for split databases) in the current code and will take considerable more time then the above estimate. However, the simple test will come very close to 100% and is considered to be enough for now. ========

1 0

proposal for inet-rtr obj extensions, for multicast, etc.
by Wilfried Woeber, UniVie/ACOnet 18 Sep '96

18 Sep '96

Dear all, (to complete a long-standing RIPE action and to make progress) please find below a modified version of a document (originally by Erik-Jan Bos) which then was describing a multicast router in a routing registry. This version proposes a more generalized approach and tries to discuss some implications, benefits and compatibility issues. It is up for discussion in the relevant RIPE working groups - in particular Routing, Mbone, (IPv6??) and DataBase. Of course any input and suggestions welcome! I would like to have your comments before and at the RIPE 25 Meeting, as well as on the mailing lists. Apologies for any rough edges, inconsistencies, omissins and typos. I'm sure we're going to change and improve this text (if this is at all perceived as a useful exercise) and before it could eventually become a RIPE Document. Best regards, Wilfried. -------------------------------------------------------------------------- Wilfried Woeber : e-mail: Woeber(a)CC.UniVie.ac.at Computer Center - ACOnet : Vienna University : Tel: +43 1 4065822 355 Universitaetsstrasse 7 : Fax: +43 1 4065822 170 A-1010 Vienna, Austria, Europe : NIC: WW144 -------------------------------------------------------------------------- --- Start of document Generalized Description of Peering Relations in a Routing Registry Erik-Jan Bos, Wilfried Woeber Document ID: ripe-nnn September, 1996 [ adapted and extended from a previous proposal titled: ] [ ] [ A Multicast Router in the Routing Registry ] [ Erik-Jan Bos ] [ Document ID: ripe-nnn ] [ May, 1995 ] **** DRAFT ** WORK IN PROGRESS ** DRAFT **** ABSTRACT This document describes an extension and generalization of concepts as documented in ripe-122 (Specifying an "Internet Router" in the Routing Registry) by which it becomes possible to define peering relations in a very general way. This approach should in particular be understood to support documentation and debugging of "overlay networks" and regular routing protocol peerings at the same time. Originally this work was started to acommodate the description of a multicast capable Internet router within a routing registry. Comments and suggestions received at RIPE DB-WG sessions and private input led to the proposal to generalize the functionality. In addition to the inet-rtr specific attributes we propose the inclusion of a url: attribute, which is already proposed as an extension to all defined objects. 1. Introduction With the growing of the Mbone worldwide, managing the Mbone becomes a complex task. The multicast-bone is already suffering from problems that used to be present in the unicast-bone several years ago, such as routing coordination (and the lack of scoping of announcements). While the Mbone is currently a focus of interest, activity, growth and coordination efforts, we expect the general concept of an overlay network that needs routing coordination to become a general issue (e.g. IPv6 tunnels). Today, registries are available for administering policies and routes and routers [1]. Tools are availble [2,3] for checking and analyzing the real world against what is registered. The Mbone in particular, and other "overlay networks" in general, could and should benefit from these technologies coming from the unicast world. The features of the specification described in this document will be usable e.g. in the RIPE Data Base, after review by the RIPE Data Base WG. After implementation tools could be developed for mapping and analyzing the various different peering environments and/or overlay meshes. Furthermore, administrative details (such as contact person) and operational information (e.g. NOC info as described in role objects) are easily administered by well known procedures and become easily retrievable. Further discussion of the basic concepts led to the idea of generalizing this approach to cater for registering all sorts of peerings, with the dynamic exchange of routing information being a secondary issue. A long-term goal should be to provide the level of configuration and debugging support that is already available for maintaining external unicast routing to a broader range of routing domains. 2. Multicast routers versus unicast routers An "Internet Router" or a "gateway" in the current version of IP (IPv4) can be defined as follows [4]: "Gateways in the Internet form a cooperative, interconnected structure. Datagrams pass from gateway to gateway until they reach a gateway that can deliver the datagram directly." This definition can be applied to both unicast and multicast routers. Since both a unicast and multicast router are fulfilling the "Internet Router" function for different natures of traffic, there is no need for a special routing registry data base object. Moreover, both functions could be present in the same box. 3. A Router specification a multi-protocol router in the Internet Because an Internet Router - can perform different types of routing at the same time, e.g. . IP unicast routing . multicast routing using DVMRP tunnels or PIM . IPv4/IPv6 or "general purpose" tunnels - in many situations there are defined or potential interactions between the routing domains documentation of these facts should be possible in the routing registry. 3.1 A Simple Router Object, IPv4-only, Current Definiton Each router is uniquely defined by by its object name, also known as the fully qualified domain name. Below is an example of a internet router object, specifying a general purpose unicast router: inet-rtr: broodjeham.surfnet.nl localas: AS1103 ifaddr: 192.87.106.102 255.255.255.192 ifaddr: 192.87.4.99 255.255.255.0 peer: 161.53.2.1 AS2108 BGP4 admin-c: Erik-Jan Bos tech-c: Henk Steenman notify: netmaster(a)surfnet.nl changed: Erik-Jan.Bos(a)surfnet.nl 950505 source: RIPE Several pieces of important information are present in this object as described in ripe-122. First of all there is the FQDN of the router, acting as object name. The Home Autonomous System Number is shown, which (currently) is only meaningful in the context of external IP routing. Furthermore, the ifaddr:-lines show that this router has two interfaces, connected to two different networks, and the masks used on these networks. Finally the peer attribute records the IP (Version 4) address for an external BGP peering and lists peering specific information, eg. the localas: of the peer router (AS2108) and the type and version of the EGP used (BGP4). 3.2 A Router for IPv4 and DVMRP, extending the current structure For a moment, let us assume that this box is at the same time used as a multicast router and has a DVMRP tunnel configured. One of the approaches would be to define a dedicated attribute to describe this configuration, say mpeer: for multicast peering. The router object would then look like the following example: inet-rtr: broodjeham.surfnet.nl localas: AS1103 ifaddr: 192.87.106.102 255.255.255.192 ifaddr: 192.87.4.99 255.255.255.0 peer: 161.53.2.1 AS2108 BGP4 mpeer: 192.87.108.13 16 1 admin-c: Erik-Jan Bos tech-c: Henk Steenman notify: netmaster(a)surfnet.nl remarks: SURFnet Multicast router changed: Erik-Jan.Bos(a)surfnet.nl 950505 source: RIPE While all the previously defined and well-known attributes would remain unchanged, the newly invented mpeer: line would be used to describe a DVMRP peering and showing it's peer's IP address, the treshold and metric. Again, the format and interpretation of the entries beyond the IP (Version 4) address would be specific for that attribute. The advantage of that approach is that there is no need to change the definition of exiting attributes, which in turn preserves the functionality of any tools written to understand the specific information. However, there is a couple of major disadvantages when proceeding in that direction - the most important ones being that - the object's definition and schema has to be changed whenever a new attribute for a differnet kind of peering is proposed - frequent changes of the objects schema would lead to incompatible versions of the software - eventually having a sizeable number of different attributes for the various peerings leeds to congestion of the namespace for attribute tags and (probably) to confusion and errors on the human part of the system - and (maybe the most important thing) there is currently no way to describe peerings that do not use IPv4 addresses to identify the peer, e.g. IPv6 and some other protocols 3.3 The Generalization of the Peer Description Concept Because of these reasons given in 3.2 we propose to change the concept of the peering description to become more general and extensible by moving the peering protocol identifier into the first part on the line and deriving the format and meaning from that. Thus the example object used here would look like the following: inet-rtr: broodjeham.surfnet.nl localas: AS1103 ifaddr: 192.87.106.102 255.255.255.192 ifaddr: 192.87.4.99 255.255.255.0 g-peer: BGP4 161.53.2.1 AS2108 g-peer DVMRP 192.87.108.13 16 1 admin-c: Erik-Jan Bos tech-c: Henk Steenman notify: netmaster(a)surfnet.nl remarks: SURFnet Multicast router changed: Erik-Jan.Bos(a)surfnet.nl 950505 source: RIPE Changing the object's schema in that way raises the issue of backwards compatibility versus "cleanliness" (tools) and compatibility of possibly different versions of the software deploied at different registries. 3.4 Backwards Compatibility Discussion For the backwards compatibility we propose to define a new attribute g-peer, "generalized peering description", and eventually make the peer: attribute obsolete - as soon as the tools have been converted to understand the new format. The disadvantage is the coexistence and/or duplicate registration of BGP4 specific information in peer: and/or g-peer: lines. An alternate approach would be to stick with the peer: attribute for the BGP4 specific information and to specify all other types of peering in g-peer: attributes. While this may be an advantage in the short term, it might add complexity for the tools in the long run by requiring them to understand different attribute formats. For the different software version issue we propose to have the implementation treat any unknown peering type (and the rest of the line) as a remark. This situation should generate a warning at the time of registration or update, and there should probably a switch to instruct the software to suppress the warning or do more stringent tests. We specifically ask the maintainers of tools which are currently relying on the peer: attribute, and the RIPE- and Internet-Community in general, for guidance about these issues! All other attributes would remain as already defined for a routing registry [1]. See next chapter for the syntax definition and concepts for extension. 4. Multiprotocol Router Syntax Definition An object for a router capable of maintaining different routing protocol (or tunnel) peerings in parallel is defined here. Each of the possible attributes are shown, on a single line. The second column defines whether the attribute must be present [mandatory] of could be present [optional]. The third column defines whether exactly one line is allowed [single] or that more than one line of the same attribute is allowed [multiple]. In case an attribute has more lines than one, the name of the attribute must be repeated on each line. inet-rtr: [mandatory] [single] localas: [mandatory] [single] ifaddr: [mandatory] [multiple] peer: [obsolete] [multiple] g-peer: [mandatory] [multiple] admin-c: [mandatory] [multiple] tech-c: [mandatory] [multiple] notify: [optional] [multiple] remarks: [optional] [multiple] changed: [mandatory] [multiple] source: [mandatory] [single] Each attribute has the following syntax: inet-rtr: The fully qualified domain name of the router. localas: The Autonomous System to which the router belongs. [Note: For the time being this is specific for the IPv4 external routing domain. It might even be useful to move that information to a g-peer: BGP4 entry?!] ifaddr: An interface IPv4 address of the router. [Note: the current draft of this proposal does not deal with necessary or desirable changes for IPv6.] peer: obsolete, potentially useful until RR tools are changed! g-peer: A peering of the router with another router, indicating the protocol and any protocol-specific information. [Note: please refer to the next section for the protocols being proposed in this draft and the proposed mechanisms for extension] Format: <protocol-ID> <peer address> <additional information> Examples: g-peer: DVMRP 192.87.108.13 16 1 g-peer: BGP4 161.53.2.1 AS2108 g-peer: IPv6-TUN IPv6-Address IPv6-specific Status: mandatory, multiple instances allowed. admin-c: Full name or uniquely assigned NIC-handle of a contact person for administrative matters. tech-c: Full name or uniquely assigned NIC-handle of a contact person for technical matters. notify: An e-mail address to which notifications of changes to this object should be send. remarks: Remarks or comments, for clarification purposes only. changed: The e-mail address and date of a change. source: Source of the information. 4.1 Discussion of Application Use of this approach has already been discussed for BGP4: g-peer: BGP4 <remote-peer address> <peer AS> example: BGP4 192.87.4.6 AS2122 localAS AS2121 or even g-peer: BGP4 <remote-peer address> <peer AS> [localAS <ASnumber>] DVMRP g-peer: DVMRP <remote-peer address> <metric> <thresh> example: DVMRP 192.87.4.6 192.87.108.13 16 1 IP-over-IP g-peer: IPv4-TUN <remote-peer address> example: IPv4-TUN 192.87.4.6 or g-peer: IPv6-TUN <remote-peer address> example: IPv6-TUN IPv6-Address 4.2 Extensions * t.b.s. * [ The gist of it should be that future extensions could be incorporated by simply editing a software configuratin file to define additional protocol types and supplying the syntax checking rules. To be discussed in more detail. ] 5. Acknowledgements This specification, being an addition to ripe-122, leans on the specification done by the author of ripe-122. Credits go to Tony Bates. Furthermore, the RIPE Mbone-WG has already done some reviewing of a previous version of this document and they receive credits as well. We are grateful for detailed suggestions received from David Kessens (ISI, then with the RIPE-NCC) and Steve Richardson (Merit). 6. Security considerations Security is not considered in this document. 7. References [1] Bates, Tony, "Specifying an Internet Router in the Routing Registry", ripe-122, October 1994. On-line at URL: ftp://ftp.ripe.net/ripe/docs/ripe-122.txt [2] PRIDE Tools Release 1. On-line at URL: ftp://ftp.ripe.net/pride/tools/pride-tools-1.tar.Z [3] Merit Inc. RRDB Tools. On-line at URL: ftp://rrdb.merit.edu/pub/meritrr [4] Douglas Comer, "Internetworking with TCP/IP", Prentice-Hall, 1988. ISBN: 0-13-470188-7. --- End of document --------------------------------------------------------------------------------

2 1

Fwd: Re: About db-wg minutes
by Wilfried Woeber, UniVie/ACOnet 18 Sep '96

18 Sep '96

Dear all ! As you might know, we lost the notes for the Berlin DB-WG meeting due to a nasty act of theft. So much for the hospitality of a university... I think (hope) that the notes provided by David Kessens have already been circulated. Unfortunately I'm under too much pressure these days to do a thorough investigation. Thus, just to make sure, here it is (again?). Thanks David ! See you next week in Amsterdam, Wilfried. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I include below my draft for the minutes. Feel free to change them according to your personal needs (we talk later about the bribes), See you, David K. ---- Database Working Group Chair: Wilfried Woeber Scribe: Hans Petter Holen 1. Administrative stuff Hans Petter Holen volunteered to take notes on his notebook. However his notebook computer was stolen right before he was able to E-mail the minutes to the chairman. Therefore the minutes are reengineered as best as possible by Wilfried Woeber and David Kessens. The proposed WG-agenda was accepted. 2. DB-SW report David Kessens offered the traditional database software report. The slides of the presentation can be obtained from: ftp://ftp.ripe.net/ripe/presentations/ ripe-m24-david-DB-REPORT.ps.gz The database has grown by 25 % in 3 months time. A new more powerfull 1.1 version is ready although not published yet since it still lacks some small pieces before it can be released as a non-beta version. Most interesting new features in addition to the already known new features are: - Hierarchical authorization for the creation of 'domain:' and 'inetnum:' objects - the extended country attribute syntax is implemented - inetnum objects can finally be sent in prefix notation - line continuation with a '\' is supported - many changes to ease installation - many bug fixes The hierarchical authorization needed some decisions to be taken by the community on how the hierarchy of the objects would be defined: The following was decided: For 'domain:' objects: - creation of TLD domain objects need intervention of the RIPE NCC - the parent object is the biggest domain that still completely matches the domain of the object to be added. For 'inetnum:' objects: - objects can be created automatically if no parent object could be found For 'route:' objects: [ Wilfried, I need some help here, my notes are not clear ... ] No final decisions were made since all proposals had disadvantages - Do the authorization as specified in the 'mnt-by:' attribute in the origin AS of the route object. Problem: People can still create route objects with other origin ASs It seems that we need some more thinking before we can come up with a nice solution. Automatic NIC handle generation: A proposal was done with special keywords AUTO and a different syntax for more experienced users that would want to have a bit more control. It was perceived that this was unnecessary complicated. We finally found a solution that provides a single mechanism for dealing with all easy and more complicated cases: nic-hdl: AUTO-#[Initials] where: # - is a random number Initials - optional, advises the database software to use initials for the generation of the NIC handle. will cause NIC handles to be generated automatically. The number is needed to give the possibility for autogenerating and referencing more then one person object under every condition in one RIPE database update. AUTO is used as a prefix to avoid any confusion between real NIC handles and requests for auto generating NIC handles. Role objects We finally reached agreement on the role object. The definition of the object: - name will be 'role:' - will have the same attributes as a person object - will have a 'tech-c:' and 'admin-c:' attribute - A NIC handle will be generated just as with person objects How to reference the object: - The object can be referenced from 'tech-c' and 'zone-c:' attributes only - recursive lookups go only one level deep Finally we decided on a proposal by Cengiz to clean up some syntactic problems in the routing registry part of the software: - the ripe-181 line continuation will no longer be supported. People that used it will be asked to update their objects. - The syntactic sugar words will always be mandatory to avoid all kind of interesting problems (for example with deletions). The raw database file will also contain the syntactic sugar. We will announce this change long in advance to the proper lists to give people the chance to make changes to their software if necessary. 3. RADBServer report by Gerald Winters [ for wilfried or Gerald Winters <gerald(a)merit.edu>, his slides were not on the Merit ftp server ] 4. Extended syntax for communities and as macros [ Wilfried: please check the conditions, I was not sure if we decided on a minimal length of the string ] Recently Brian Renaud posted a proposal to the RIPE database working group on an extension of the possible characters that could be used for communties and as macros. Gerald Winters presented the proposal to the RIPE database working group and asked for input of the working group on some unresolved minor issues. The working group found consensus on the following new syntax: community: Format: <string> The <string> must satisfy the following constraints and the constaints for as macros (see below): + It must not start with the any upper or lower case permutations of the character "A" followed by the character "S". That is, the pattern "[Aa][Ss]" is not legal at the start of a community name. + It must not be the same as any of the routing policy expression keywords. (See RIPE-181, Appendix A for a complete list of these keywords.) It can expected that the keywords that the IETF Routing Policy System (RPS) working group defines for RPSL will be added to this list of keywords. as-macro: Format: AS-<string> The <string> must satisfy the following constraints: + It must be at least two characters long. + It must contain only upper case letters, lower case letters, digits, or dashes (underscores (_) were dropped to avoid confusion between dashes and underscores) + It must start with a letter + It must end with a letter or digit All comparisons will be case insensitive. The slides of the presentation can be obtained from: ftp://ftp.ripe.net/ripe/presentations/ ripe-m24-renaud-DB-syntax.ps.gz 5. www interface based on WAIS by Antonio Blasco Bonito Blasco presented a new user interface for the world wide web developed at GARR-NIS. The underlying technology was based on WAIS. The interface provides interesting features that are not possible with the current 'whois' or WAIS interface like looking up arbitrary strings but with a higher level of granularity then we are used to; per object type and OR'ed and AND'ed searches are possible. Furthermore, the software automatically detects URL's in the objects and generates a pointer to the site in the URL. The slides can be obtained from: ftp://ftp.ripe.net/ripe/presentations/ ripe-m24-bonito-DB-waisinterface.ps.gz The community expressed their interest in the interface and asked the RIPE NCC to implement this software as soon as possible even when the interface would not always be consistent with the layout of the new RIPE web interface. Carol Orange told that the RIPE NCC would try to integrate the interface with the new web site, but that the resources are very limited and that the interface would need to wait until the launch of the new RIPE website. 6. Review of active proposals and priorities David Kessens showed a slide on the proposals that were done in the past. Since the meeting was running out of time, it was then decided that David would send a list of active proposals to the mailing list and to provide a mechanism for the community to decide on the priorities. 7. AOB None. Meeting closed. List of new actions: Action 24.? on David Kessens Implement the automatic NIC handle assignment in the RIPE database with the syntax as decided as decided on during this working group session. Action 24.? on David Kessens Send a list of active proposals to the mailing list and provide a mechanism for the community to decide on the priorities --------------------------------------------------------------------------------

1 0

Draft/Proposed Agenda for the DB-WG Meeting at RIPE 25
by Wilfried Woeber, UniVie/ACOnet 17 Sep '96

17 Sep '96

This is the 1st (rough) proposal of a draft agenda for the Database-WG at RIPE 25 in Amsterdam. There is a considerable amount of contention for slot-time, so we're not going to have more than 90 minutes this time. Thus I propose to put the effort on structural things, this time and leave the details for the DB-WG list after the meeting. Like: - approaches for better data consistency (if any :-) - documentation - priorities Apologies if I missed something obvious... As usual, comments, additions, deletions welcome. See you, Wilfried. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Proposed/Draft Agenda: Database-WG, RIPE 25, Amsterdam, NL ---------------------------------------------------------- A. Administrative stuff - volunteering of the scribe - WG-agenda bashing B. DB-SW status report by the RIPE NCC - recent changes, V2.0 - new features: experience, feedback C. DB consistency and sanity checks - at time of transaction (create/modify) - at regular intervals - input from other WGs - input from the NCC (address space handling) - RIPE/(NIC)-Handle space, special cases, like ROLExxx-RIPE, RIPExxx-yyy, AUTOxxx-RIPE D. Inverse lookup discussion - recursive lookup on by default, should this change? - extending the list of attributes that can be use in -i lookup? E. Documentation - structure and contents as proposed by the NCC - user input F. New functionality - things on the consolidated wish-list - url handling - inet-rtr object extension Y. General input from other WGs Z. AOB Draft 1.0, 17-SEP-1996, WW. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 0

HELP! 'networkupdate' fails
by Markus Bock 11 Sep '96

11 Sep '96

Hello ! Actually I'm testing the ripe-dbase-software (release 2.03). In near future we want to use the ripe-dbase-software to store internal data (i.e. customer data, infrastructure data ...) and to run a secondary of the ripe database at our site. We're running under Solaris 2.5 with the special perl4.036-hack (from here many thanks to Peter Haag) which is avaiable on your ftp-server. Everything is running fine, i.e - all kinds of queries - handling of new, own database objects - running with different sources in split-mode - set-up of the database. For writing database front ends which manipulate and visualisize the stored data we want to use the 'networkupdate' program to have access to the database from different workstations via LAN. But it is not working at our side, it is not possible to forward objects via STDIN to the whoisd after calling the networkupdate programm. 'whoisd -V -D' gives me the following trace: (europa is the server-machine, x11 is the client-machine) # europa: whoisd -V -D # whoisd daemon (6827) - running in debug mode # whoisd daemon (6827) - will connect to port: 43 with protocol: 6 # whoisd daemon (6827) - changing (UID,GID) from: (0,1 14 12 9 8 7 6 5 4 3 2 0 1) to: (500,200) # whoisd daemon (6827) - changed (UID,GID) to: (500,200 14 12 9 8 7 6 5 4 3 2 0 1) # (6832) child connection x11.NIC.DTAG.DE (194.25.1.107) # (6832) got in parse # (6832) left parse, searchstring: # (6832) whoisd called with flags:U:Vnp2.0 # (6832) whoisd lookup: # (6832) whoisupdate(name:'x11.NIC.DTAG.DE',ip:'194.25.1.107',log:'mb localhost') called # (6832) checking 194.25.1.107=~ /194.25.1.107/ # (6832) update from: 194.25.1.107 accepted, regular: 194.25.1.107 # (6832) whoisupdate accepted # (6832) exit connection [194.25.1.107] After calling 'networkupdate - h europa' on the client-machine it terminates immediately! After accepting the whoisupdate the connection is terminated by the whoisd! I checked our ripedb-config file more than one time and I thing that everything inside is correct. Maybe I make something wrong or this effect has something to do with running under Slowaris 2.5 ... ? Or is there a tricky way to setup the possibility for networkupdates ? Is somebody able to help me ? Kind regards Markus

3 2

RE: HELP! 'networkupdate' fails
by Wilfried Woeber, UniVie/ACOnet 09 Sep '96

09 Sep '96

>But it is not working at our side, it is not possible to forward >objects via STDIN to the whoisd after calling the networkupdate >programm. I think there was a brief note somewhere in the release-notes (might have been somewhere else) which hinted at some special actions to enable that functionality. I think there is some address filters somewhere ? I might be barking at the wrong tree, of course. Wilfried. -------------------------------------------------------------------------- Wilfried Woeber : e-mail: Woeber(a)CC.UniVie.ac.at Computer Center - ACOnet : Vienna University : Tel: +43 1 4065822 355 Universitaetsstrasse 7 : Fax: +43 1 4065822 170 A-1010 Vienna, Austria, Europe : NIC: WW144 --------------------------------------------------------------------------

2 1

RIPE Database Workshop
by Carol Orange 20 Aug '96

20 Aug '96

W O R K S H O P A N N O U N C E M E N T R I P E D A T A B A S E 2.0 ------------------------------ MONDAY, SEPTEMBER 23, 1996 9.00 - 12.00 In conjunction with the 25th RIPE Meeting to be held at the Nikhef Institute in Amsterdam, the RIPE NCC will hold a workshop in which the features of the new release of the RIPE Database package will be introduced. The workshop will be held at the RIPE meeting location. Topics to be covered in the workshop include: + Hierarchical authorization + Automatic NIC-handle mechanism + Role objects + Classless update and lookup mechanisms + Reverse lookup of objects + Near real time mirroring Where applicable, examples will be used to illustrate how the features above can best be employed. Common questions regarding the use of each feature will also be discussed. Time will be reserved to address the questions of workshop attendees. The test database will be set up during the meeting with the examples covered in the workshop. Attendees can try things out during the RIPE meeting, and address any questions that may arise to RIPE NCC database support staff. Note that the content of the workshop is distinct from that of the Database WG meeting. The workshop will be of a tutorial nature describing how to work with the new database features. Priorities for further developments will be addressed in the WG as always. REGISTRATION The workshop is open to all members of the RIPE community. Attendees from RIPE NCC contributors will however receive priority. If you plan to attend, please register, using the form at the end of this message, as soon as possible, but before September 16th. This will allow proper planning of workshop logistics. Those who do not register in advance may be turned away if space restrictions arise. Upon completion, send the form should to: <meeting(a)ripe.net> %START PART 1 - Registration %MTG [ dbws1 ] 1) Your name Enter First name, Last name in FULL e.g. John Doe Mary-Beth Walton %NAME [ ] 2) Your Organisation/Institution %ORG [ ] 3) Your e-mail address %EMAIL [ ] 4) Registtry ID The registry ID string of the local IR (if any) e.g. fr.lereseau de.dasnetz uk.thenet. Contributing local IRs will receive priority registration. %REGID [ ] %END

1 0

Newest patches: ripe-dbase.2.0.3 available
by David.Kessens＠ripe.net 19 Aug '96

19 Aug '96

Dear all, The latest release unexpectedly contained a serious flaw in the classless database opening code. The problem was not found before putting things in production due to a non-complete copy of one file into the test database directory (my fault). The new database staff did some emergency measures and consulted me on the phone (The moving company was just packing my computers & modems ;-(). An older version was put back in production which caused the reappearing of some (non fatal) bugs. I want to apologize for the inconvenience this has caused although I hope that you also have some understanding: We were the last three weeks in the inconvenient position that we had to put new code in production while not having the time for an appropriate test period. Postponing the release was not a viable option since I was leaving the RIPE NCC (In fact I already did two weeks ago...), the new people first need some time to get up to speed, and I saw clear signs (in the internals of the database) that we rather should handle fast then slow with introducing the new software. I have prepared a new bug fixed version that also fixes a few other problems (including some older bugs that were already present in the previous release). This version runs now since friday afternoon on our production machine (note the subtle change in the config file: the 'av' attribute added in the 'OBJ an ATSQ' line). The code is available from: ftp://ftp.ripe.net/ripe/dbase/software/ripe-dbase.2.0.3.tar.gz Kind regards, David K. ---

1 0

bug fixes & upgraded functionality for 2.0 database release
by David.Kessens＠ripe.net 15 Aug '96

15 Aug '96

Dear all, I have fixed a few small bugs. I also added functionality for automatically adding the current date to the last 'changed:' attribute field in your update when no date is present. Apply the included patches or get a new distribution from: ftp://ftp.ripe.net/ripe/dbase/software/ripe-dbase-2.0.2.tar.gz Kind regards, David K. --- *** encmp.old Wed Aug 14 17:38:53 1996 --- encmp.pl Wed Aug 14 17:45:34 1996 *************** *** 1,9 **** # encmp - compare two associative arrays # # $RCSfile: encmp.pl,v $ ! # $Revision: 2.0 $ # $Author: ripe-dbm $ ! # $Date: 1996/08/08 10:47:30 $ # # This routine takes two objects and compares them case insensitive # Returns 1 if objects are equal, else returns 0; --- 1,11 ---- + # encmp - compare two associative arrays + # 'syntax sugar' is removed before comparing! # # $RCSfile: encmp.pl,v $ ! # $Revision: 2.1 $ # $Author: ripe-dbm $ ! # $Date: 1996/08/14 15:45:19 $ # # This routine takes two objects and compares them case insensitive # Returns 1 if objects are equal, else returns 0; *************** *** 88,103 **** # we do a compare after the syntactic sugar has been removed. if ($key eq "ae") { ! $a=~ s/(^|\n)exclude\s+(AS\d+)\s+to\s+/$1$2 /g; ! $b=~ s/(^|\n)exclude\s+(AS\d+)\s+to\s+/$1$2 /g; } elsif ($key eq "ai") { ! $a=~ s/(^|\n)from\s+(AS\d+\s+\d+)\s+accept\s+/$1$2 /g; ! $b=~ s/(^|\n)from\s+(AS\d+\s+\d+)\s+accept\s+/$1$2 /g; } elsif ($key eq "ao") { ! $a=~ s/(^|\n)to\s+(AS\d+)\s+announce\s+/$1$2 /g; ! $b=~ s/(^|\n)to\s+(AS\d+)\s+announce\s+/$1$2 /g; } elsif ($key eq "it") { $a=~ s/(^|\n)from\s+($match)\s+($\s*pref[\=\s\S]+$)?accept\s+/$1$2 $3/go; --- 90,105 ---- # we do a compare after the syntactic sugar has been removed. if ($key eq "ae") { ! $a=~ s/(^|\n)exclude\s+(as\d+)\s+to\s+/$1$2 /g; ! $b=~ s/(^|\n)exclude\s+(as\d+)\s+to\s+/$1$2 /g; } elsif ($key eq "ai") { ! $a=~ s/(^|\n)from\s+(as\d+\s+\d+)\s+accept\s+/$1$2 /g; ! $b=~ s/(^|\n)from\s+(as\d+\s+\d+)\s+accept\s+/$1$2 /g; } elsif ($key eq "ao") { ! $a=~ s/(^|\n)to\s+(as\d+)\s+announce\s+/$1$2 /g; ! $b=~ s/(^|\n)to\s+(as\d+)\s+announce\s+/$1$2 /g; } elsif ($key eq "it") { $a=~ s/(^|\n)from\s+($match)\s+($\s*pref[\=\s\S]+$)?accept\s+/$1$2 $3/go; *************** *** 117,119 **** --- 119,122 ---- } 1; + *** syntax.old Wed Aug 14 17:39:05 1996 --- syntax.pl Wed Aug 14 17:45:59 1996 *************** *** 1,8 **** ! # # $RCSfile: syntax.pl,v $ ! # $Revision: 2.2 $ # $Author: ripe-dbm $ ! # $Date: 1996/08/09 12:55:04 $ # # ARGUMENTS: *ASSOC object # RETURNS: INTEGER object_status --- 1,8 ---- ! # $RCSfile: syntax.pl,v $ ! # $Revision: 2.3 $ # $Author: ripe-dbm $ ! # $Date: 1996/08/14 15:45:49 $ # # ARGUMENTS: *ASSOC object # RETURNS: INTEGER object_status *************** *** 311,317 **** # } else { ! if ($itmp=~ /^ac|ah|tc|zc|cy|mb|ml$/) { local($val, $msg) = &dosyntax($itmp, $object{$itmp}, *object); # print STDERR "checksyntax - before attr: $itmp rtcode: $rtcode val: $val msg: $msg\n" if ($opt_V); if (defined($val)) { --- 311,317 ---- # } else { ! if ($itmp=~ /^ac|ah|ch|tc|zc|cy|mb|ml$/) { local($val, $msg) = &dosyntax($itmp, $object{$itmp}, *object); # print STDERR "checksyntax - before attr: $itmp rtcode: $rtcode val: $val msg: $msg\n" if ($opt_V); if (defined($val)) { *************** *** 711,759 **** # Try to do something clever with the changed field # # ch - changed # ! if ($key eq "ch") { ! print STDERR "dosyntax - checking ch\n" if $opt_V; ! local($date)=""; ! if ($value=~ / /) { ! $value=~ s/ +(\S+)$//; ! $date=$1; ! } ! # print STDERR "*", $date, "*", $value, "\n"; ! if (!(&isemail($value))) { ! return $O_ERROR, "syntax error in e-mail part of \"$ATTL{$key}\""; } ! if (!$date) { ! return $O_ERROR, ! "No date specified in \"$ATTL{$key}\" attribute"; ! } ! if ($date!~ /^(\d\d)(\d\d)(\d\d)$/) { ! return $O_ERROR, "date part of \"$ATTL{$key}\" not in YYMMDD format"; ! } ! # 1988 is the start of the world. This is where we test for proper ! # date values of YYMMDD ! if (($1 < 88) || ($2 > 12) || ($3 > 31)) { ! return $O_ERROR, "date part of \"$ATTL{$key}\" is not a valid YYMMDD value"; ! } ! if ($date>$DATE) { ! $object{$key} =~ s/$date/$DATE/g; ! return $O_WARNING, ! "date in \"$ATTL{$key}\" ($date) is in the ". ! "future - changed to $DATE"; ! } ! $object{$key}=~ s/ +/ /g; - return $O_OK; - } # --- 711,804 ---- # Try to do something clever with the changed field # # ch - changed + # wd - withdrawn # ! if ($key=~ /^ch|wd$/) { ! print STDERR "dosyntax - checking ch,wd ($key)\n" if $opt_V; ! local($returncode)=$O_OK; ! local($date); ! local(@newchanged)=(); ! local(@changedvalues)=split(/\n+/, $value); ! # ! # add current date if needed to last specified attribute ! if ($changedvalues[$#changedvalues]!~ / \d{6,6}$/) { ! ! $changedvalues[$#changedvalues].=" ".$DATE; ! ! &addwarning(*object, "added current date to ".$ATTL{$key}." field"); ! ! $returncode=$O_WARNING; ! } + + # + # sort values on date + + @changedvalues=sort sortchangedfield @changedvalues; + + #print STDERR join("\n", "values:", @changedvalues); + + foreach $value (@changedvalues) { + + $date=""; + + if ($value=~ / /) { + $value=~ s/ +(\S+)$//; + $date=$1; + } + + # print STDERR "*", $date, "*", $value, "\n"; + + if (!(&isemail($value))) { + return $O_ERROR, "syntax error in e-mail part of \"$ATTL{$key}\""; + } ! if (!$date) { ! return $O_ERROR, "No date specified in \"$ATTL{$key}\" attribute"; ! } ! ! if ($date=~ /^(\d\d)(\d\d)(\d\d)$/) { ! ! # ! # 1988 is the start of the world. This is where we test for proper ! # date values of YYMMDD ! if ($1<88) { ! return $O_ERROR, "date ($date) part of \"$ATTL{$key}\" is older then the database itself!"; ! } ! ! if (($2<1) || ($2>12) || ($3>31) || ($3<1)) { ! return $O_ERROR, "date ($date) part of \"$ATTL{$key}\" is not a valid YYMMDD value"; ! } ! ! } ! else { ! ! return $O_ERROR, "date part of \"$ATTL{$key}\" not in YYMMDD format"; ! } ! ! if ($date>$DATE) { ! ! &addwarning(*object, "date in \"$ATTL{$key}\" ($date) is in the future - changed to $DATE"); ! $date=$DATE; ! $returncode=$O_WARNING; ! ! } ! ! push(@newchanged, $value." ".$date); ! } ! ! $object{$key}=join("\n", @newchanged); ! ! return $returncode; } # *************** *** 1752,1758 **** # # check if the ID is not already used ! return $O_ERROR, "error in \"$ATTL{$key}\", auto NIC handle assignment\nidentification code ($1.$2) already used in other object" if ($NICID{$object{"so"}, $1.$2}); # # find initials if not already specified --- 1797,1803 ---- # # check if the ID is not already used ! return $O_ERROR, "error in \"$ATTL{$key}\", auto NIC handle assignment\nidentification code ($1.$2) already used in other object" if ($NICID{$object{"so"}, $prefix}); # # find initials if not already specified *************** *** 1768,1776 **** $object{$key}=$prefix.$initials; $prefix=~ /\-(\d+)$/; - $NICID{$object{"so"}, $1}=1; return $O_OK; } --- 1813,1827 ---- $object{$key}=$prefix.$initials; + # + # check if the ID is not already used + $prefix=~ /\-(\d+)$/; + return $O_ERROR, "error in \"$ATTL{$key}\", auto NIC handle assignment identification code ($1) already used in other object\nplease use another number then $1" if ($NICID{$object{"so"}, $prefix}); + + $NICID{$object{"so"}, $prefix}=1; + return $O_OK; } *************** *** 2170,2204 **** return $O_OK; } - - # - # wd - withdrawn - # - if ($key eq "wd") { - - print STDERR "dosyntax - checking wd ($key)\n" if $opt_V; - - if ($value !~ /^(\d\d)(\d\d)(\d\d)$/) { - return $O_ERROR, - "date part of \"$ATTL{$key}\" not in YYMMDD format"; - } - - # 1988 is the start of the world. This is where we test for proper - # date values of YYMMDD - - if (($1 < 88) || ($2 > 12) || ($3 > 31)) { - return $O_ERROR, - "date part of \"$ATTL{$key}\" is not a valid YYMMDD value"; - } - - if ($value gt $DATE) { - return $O_ERROR, - "date in \"$ATTL{$key}\" ($1$2$3) is in the future"; - } - - return $O_OK; - } - # # These are not checked and not used, just in here for clarity # --- 2221,2226 ---- *** misc.old Wed Aug 14 17:38:41 1996 --- misc.pl Wed Aug 14 17:46:14 1996 *************** *** 2,13 **** # misc - miscellaneaous functions # # $RCSfile: misc.pl,v $ ! # $Revision: 2.0 $ # $Author: ripe-dbm $ ! # $Date: 1996/08/08 10:47:30 $ # require "defines.pl"; sub ReplaceGlobalVars { local(*text)=@_; --- 2,45 ---- # misc - miscellaneaous functions # # $RCSfile: misc.pl,v $ ! # $Revision: 2.1 $ # $Author: ripe-dbm $ ! # $Date: 1996/08/14 15:46:06 $ # require "defines.pl"; + + sub sortchangedfield { + + $a=~ /(\d{6,6})$/; + local($date1)=$1; + + $b=~ /(\d{6,6})$/; + + # print STDERR "-$a-$b-$date1-$1-\n"; + + if ((!$date1) && (!$1)) { + + return 0; + + } + elsif (!$date1) { + + return -1; + + } + elsif (!$1) { + + return 1; + + } + else { + + return $date1 <=> $1; + + } + + } sub ReplaceGlobalVars { local(*text)=@_;

2 1

bug fix for 2.00 release
by RIPE Database Manager 10 Aug '96

10 Aug '96

Dear all, I found this morning a bug that could case the software not to remove the lockfile when doing a database cleanup when indexing files with a full path name. Please substitute your 'donetdbm.pl' file by the included one or pick up a new distribution from: ftp://ftp.ripe.net/ripe/dbase/software/ripe-dbase-2.0.1.tar.gz Kind regards, David K. -------- # $RCSfile: donetdbm.pl,v $ # $Revision: 2.1 $ # $Author: david $ # $Date: 1996/08/10 15:53:32 $ require "defines.pl"; require "dbopen.pl"; require "dbclose.pl"; require "enread.pl"; require "enwrite.pl"; require "addkey.pl"; require "enkeys.pl"; require "enukey.pl"; # # exit cleanly when asked to ... sub quitdonetdbm { unlink(@DONETDBMFILES); print "\n$PROGRAMNAME terminated by signal.\n"; exit; } sub flushkeys { local(*database, *normalkeys)=@_; # print STDERR "do flush to disk for normal keys (happens every $MAXLISTLENGTH keys) ..."; local($oldoffsets,$key,$value,$length); # # old overflow files local(@toobig)=(); # # new overflow files local(@toobignewkeys)=(); local(@toobignewvalues)=(); # # normal keys local(@addkeys)=(); local(@addvalues)=(); while (($key,$value)=each(%normalkeys)) { $length=length($key); $key=~ s/^.{$OVERFLOWSIZE-1,$OVERFLOWSIZE-1}.*$/$1/ if ($length>=$OVERFLOWSIZE); # # check if the key already existed, that makes life # more complicated if (($oldoffsets=$database{$key})) { # # add to overflow file if ($oldoffsets=~ s/^$OVERFLOWPREFIXREGULAR//o) { push(@toobig, $oldoffsets, $value); next; } $value=join("\,", $oldoffsets, $value); } # # do a value addition $length+=length($value); if ($length>=$OVERFLOWSIZE) { push(@toobignewkeys, $key); push(@toobignewvalues, $value); } else { push(@addkeys, $key); push(@addvalues, $value); } } # # clean up the memory %normalkeys=(); # # add data to old overflow files local($dbfile)=$database[1].$OVERFLOWEXTENSION; if (@toobig) { while (@toobig) { if (open(OVERFLOWADD, ">>".$dbfile.shift(@toobig))) { print OVERFLOWADD "\,", shift(@toobig); close(OVERFLOWADD); } else { &fatalerror("in donetdbm flushkeys: cannot open for addition $dbfile.$_ value: ".shift(@toobig)." code: $!"); } } } # # create new overflow files if (@toobignewkeys) { local($filenumber)=$database{$OVERFLOWKEY}; local(@overflowvalues)=(++$filenumber .. ($filenumber+scalar(@toobignewkeys)-1)); push(@addkeys, $OVERFLOWKEY); push(@addvalues, $overflowvalues[$#overflowvalues]); #print STDERR join(" ", @toobignewkeys), "\n"; #print STDERR join(" ", @toobignewvalues), "\n"; #print STDERR join(" ", @overflowvalues), "\n"; foreach (@overflowvalues) { if (open(OVERFLOWNEW, ">".$dbfile.$_)) { print OVERFLOWNEW shift(@toobignewvalues); close(OVERFLOWNEW); } else { &fatalerror("in donetdbm flushkeys: cannot open for creation $dbfile.$_ value: ".shift(@toobignewvalues)." code: $!"); } } push(@addkeys, @toobignewkeys); push(@addvalues, (@overflowvalues=grep($_=$OVERFLOWPREFIX.$_, @overflowvalues))); } # # add all keys to the physical database in one single command !!! @database{@addkeys}=@addvalues; # print STDERR " done\n"; return 0; } sub flushclkeys { local($basename, $i, *list, *notexistingprefixes)=@_; #print STDERR "do flush to disk prefix \/$i (happens every $MAXSTRINGCOMPONENTS prefixes) ..."; local($file)=$basename.$i; push(@DONETDBMFILES, $file) if (grep($_ ne $file, @DONETDBMFILES)==scalar(@DONETDBMFILES)); open(FLUSHCLKEYS, ">>".$file); print FLUSHCLKEYS $list[$i]; close(FLUSHCLKEYS); #print STDERR " done\n"; $list[$i]=""; $notexistingprefixes[$i]=0; return 0; } sub donetdbm { local($filename, $todir, $options)=@_; local($type,$uniquekey,$offset,$prefix); local($NEWDIR, $newfilename); local($classless)=$options & $CLASSLESSOPTION; local($cleaning)=$options & $CLEANOPTION; local(%entry); local($objseen) = 0; local($processed)=0; local(@clalist)=(); local(@classlesslist)=(); local(@nrofentries)=(); local(%normalkeys)=(); local(@notexistingprefixes)=(1) x 129; local(@keys,@otherkeys,@pointsto,@otherpointsto,@classless); local($basename); local($newdb)='newdb'; local(%newdb,@newdb); local(%nothing)=(); local($lockfile); # print STDERR "donetdbm - file: $filename classless: $classless cleaning: $cleaning\n"; if ($cleaning) { ($NEWDIR, $newfilename)= $filename=~ /$SPLITFILENAME/o; $lockfile=$CLEANLOCK.$newfilename; # We will create a temporary directory to build the new database and # indexes. $NEWDIR.=$newfilename."-newindex.".$$; $newfilename=$NEWDIR."\/".$newfilename; &fatalerror("Temporary directory \"$NEWDIR\" already exists!") if (-d $NEWDIR); if (!mkdir($NEWDIR, 0750)) { &fatalerror("Failed to create temporary directory ($!): $NEWDIR"); } &dbopen(*newdb, *nothing, 1, $newfilename ) || &fatalerror("Cannot open $newfilename"); seek($newdb,0,2); } else { &delormoveindices($filename, "", $classless); } local(%dummyobject)=(); local($db)='db'; local(%db,@db); &dbopen(*db,*dummyobject,1,$filename) || &fatalerror("$0: error opening database: $filename"); seek($db,0,0); if ($cleaning) { &fatalerror("lockfile: $lockfile already exists") if (-e $lockfile); push(@DONETDBMFILES, $lockfile); open(LOCKFILE, ">".$lockfile) || &fatalerror("cannot create $lockfile"); print LOCKFILE "$$\n"; close(LOCKFILE); } if ($classless) { local($prefixfilename); if ($filename=~ /^.*\/([^\s\/]+)\s*$/) { $prefixfilename=$2; } else { $prefixfilename=$filename; } $basename=$TMPDIR."/prefixdata.".$$.".".$prefixfilename; # # remove any possible left over files... local(@files)=(); for $i (0..128) { push(@files, $basename.$i) if (-f $basename.$i); } unlink(@files) if (@files); } # # exit clean on a signal (except for the cleaning files, I am sorry # no time for the real fancy stuff) $SIG{'INT'}='quitdonetdbm'; $SIG{'KILL'}='quitdonetdbm'; $SIG{'TERM'}='quitdonetdbm'; @OPENNETDBMFILES=(); # # start gathering the keys... while ($type=&enread($db, *entry, -1)) { next if (!$OBJATSQ{$type}); # print STDERR "type: $type\n"; # $objseen++; # if ($objseen % 400 == 0) { # print STDERR "donetdbm - indexed $objseen\n"; # } # print STDERR "entry ($type): ", $entry{$type}, "\n"; # print STDERR "\noffset: ", $entry{"offset"}; # print STDERR "\nuniquekey: $uniquekey\n"; if ($cleaning) { $offset=&enwrite($newdb,*entry); } else { $offset=$entry{"offset"}; } &enkeys(*entry, $OBJKEYS{$type}, *keys, *otherkeys, *pointsto, *otherpointsto, *classless, $classless); $processed+=grep((($normalkeys{$_}) && ($normalkeys{$_}=join("\,", $normalkeys{$_}, $offset))) || ($normalkeys{$_}=$offset), ($uniquekey=&enukey(*entry, $type)), @keys, @otherkeys, @pointsto, @otherpointsto); if ($cleaning) { $processed=&flushkeys(*newdb, *normalkeys) if ($processed>$MAXLISTLENGTH); } else { $processed=&flushkeys(*db, *normalkeys) if ($processed>$MAXLISTLENGTH); } if ($classless) { foreach (@classless) { /^(.+)\/(\d+)$/; $classlesslist[$2]=join("", $classlesslist[$2], $1, "\%", $uniquekey, "\%"); $nrofentries[$2]=&flushclkeys($basename,$2,*classlesslist,*notexistingprefixes) if (++$nrofentries[$2]>$MAXSTRINGCOMPONENTS); } } } # # note: we do the wait for a child flush *after* the classless indexing # it might save us some extra time! if ($cleaning) { &flushkeys(*newdb, *normalkeys); } else { &flushkeys(*db, *normalkeys); } # # now we start with the classless indices if ($classless) { undef %normalkeys; print STDERR "docldbm - start inserting prefixes\n" if $opt_V; if ($cleaning) { &dbclopen(*dummyobject,1,$newfilename); } else { &delormoveindices($filename.$CLASSLESSEXT, "", $classless); &dbclopen(*dummyobject,1,$filename); } $objseen = 0; local($entry); local($p,$i,$cla,$newcla); # # we need to read entries separated by '%' characters # # trickey: we don't need to reset it for: # # flushclkeys, getvalues, addkey since # they are not affected. local($/)="\%"; for $i (1..128) { print STDERR "try $basename$i\n" if ($opt_V); # # flush first the remaining prefixes... &flushclkeys($basename, $i, *classlesslist, *notexistingprefixes) if ($nrofentries[$i]); next if ($notexistingprefixes[$i]); undef $nrofentries[$i]; if (open(FLUSHCLKEYS, "<".$basename.$i)) { print STDERR "opened $basename$i\n" if ($opt_V); $entry=1; while ($entry) { print STDERR "read from $basename$i\n" if ($opt_V); $processed=0; undef @clalist; while (($processed<$MAXLISTLENGTH) && ($entry=<FLUSHCLKEYS>)) { $key=<FLUSHCLKEYS>; chop($entry); chop($key); push(@clalist,$entry,$key); $processed++; } while ($prefix=shift(@clalist)) { # print STDERR $prefix, "\n"; if (&addkey(*mspnxl, ($cla=join("\/", $prefix, $i)), shift(@clalist))) { for ($p=$i-1; ($p>0) && (($notexistingprefixes[$p]) || (!defined($mspnxl{($newcla=join("\/", &iprightzeromask($prefix,$p), $p))}))); $p--) {}; # print STDERR $p, " ",$i, " ", $newcla, "\n"; &addkey(*mspnxl, $p>0?$newcla:"0\/0", $cla); } } } close(FLUSHCLKEYS); push(@DONETDBMFILES, $basename.$i); } else { &fatalerror("Couldn\'t open prefix file: $basename$i ($!)"); } } unlink(@DONETDBMFILES) if (@DONETDBMFILES); # # make the not existing prefixes array $mspnxl{$NOTEXISTINGPREFIXESKEY}=join(" ", @notexistingprefixes); &dbclclose(); } # # we might still have a child process for the normal keys... # wait; # print STDERR "closing\n"; if ($todir) { ($NEWDIR, $newfilename)= $filename=~ /$SPLITFILENAME/o; $lockfile=$CLEANLOCK.$newfilename; &fatalerror("lockfile: $lockfile already exists") if (-e $lockfile); # # we don't accept termination of the program from here @DONETDBMFILES=(); $SIG{'INT'}='IGNORE'; $SIG{'KILL'}='IGNORE'; $SIG{'TERM'}='IGNORE'; open(LOCKFILE, ">".$lockfile) || &fatalerror("cannot create $lockfile"); print LOCKFILE "$$\n"; close(LOCKFILE); $todir.="\/" if ($todir!~ /\/\s*$/); &dbclose(*db); &delormoveindices($filename.$CLASSLESSEXT, $todir.$newfilename.$CLASSLESSEXT, $classless) if ($classless); &delormoveindices($filename, $todir.$newfilename, 0); unlink($lockfile); } elsif ($cleaning) { &dbclose(*newdb); &dbclose(*db); # # we don't accept termination of the program from here @DONETDBMFILES=(); $SIG{'INT'}='IGNORE'; $SIG{'KILL'}='IGNORE'; $SIG{'TERM'}='IGNORE'; &delormoveindices($newfilename.$CLASSLESSEXT, $filename.$CLASSLESSEXT, $classless) if ($classless); &delormoveindices($newfilename, $filename, 0); unlink($lockfile); warn "could\'t remove directory ($NEWDIR) errorcode: $!\n" if (!rmdir($NEWDIR)); } else { # print STDERR "close db\n"; &dbclose(*db); # print STDERR "done db\n"; } # print STDERR "done -$/- db\n"; return; #print STDERR "done db\n"; } 1; --------

1 0