November 2001 - db-wg - mailman.ripe.net

<auto-dbm@ripe.net> is slow this afternoon.
by RIPE Database Administration 17 May '18

17 May '18

Dear Colleagues, Because of a very high level of use this afternoon, the response time of <auto-dbm(a)ripe.net> is longer than normal. I have made contact with those who are responsible and I expect the situation to improve as the machine works through the queue. Please be patient. Obviously we at the RIPE NCC Database Group are continuously working to make the performance of the database as good as possible. However, we cannot make improvements in zero time. We thank you for your co-operation up to now and we ask for your continued co-operation so that everyone receives the best possible service. If you have anymore questions, please contact <ripe-dbm(a)ripe.net>. Regards, Ambrose Magee ____________________________ RIPE Database Administration.

2 2

Re: Object Metadata Proposal
by Janos Zsako 15 Nov '01

15 Nov '01

> From owner-db-wg(a)ripe.net Thu Nov 8 17:53:37 2001 > From: Andrei Robachevsky <andrei(a)ripe.net> Dear Andrei, I think both proposals (this one and the garbage collection) are very good and very useful. I just have a further suggestion and a question (I apologize if this has already been discussed). > Metadata Contents > ------------------ > Other meta-attributes may be added as needed. My suggestion would be to add a meta-attribute "creation-time:" which would store the date and time the object was created (as opposed to "update-time:", which only shows when it was last updated). This attribute would never change. The rationale behind having this attribute is that one would have a proof of the existence of the object in the database from (at least) creation-date (until at least query time). This may help for example auditing LIR assignments to the same end-user (which is limited to one AW worth in any 12 months). My question would be about the generation of the initial values (of update-time and ref-time) for the _existing_ objects. One aproach would be to use the "changed:" value for "update-time", and for "ref-time" the most recent "changed" value of the objects referencing the given object. The only problem is that we know "changed" is not (very) reliable. An other approach would be to use the time of the (initial) creation of the meta-attributes as initial value both for "update-time" and "ref-time". This is probably misleading. A better approach would probably be to leave these values empty, but this would not meet the current spec. The most exact approach would be to retrieve the exact values from the update logs, but this is probably too expensive. My question is which approach do you plan to take (one of the above or one I did not think about)? Thanks and regards, Janos

2 1

automatic DB cleanup proposal (v2)
by Andrei Robachevsky 08 Nov '01

08 Nov '01

Dear Colleagues, This is a slightly modified version of Ulf Kieber's proposal based now on metadata concept (please see "Object Metadata" proposal on the list next to this one). Your comments and suggestions are welcome. Thanks, Andrei Robachevsky RIPE NCC ----------------------------------------------------- Automatic DB Cleanup proposal ============================= Motivation ---------- In the light of data privacy issues raised by governmental organizations and in RIPE Database Working Group sessions during RIPE-39 and RIPE-40, as well as following the Great Dangling Person Objects Deletion after de.* ccTLD move-out it is necessary to implement an automized garbage collection procedure to remove personal data that is unreferenced for a certain amount of time. Proposed Change --------------- Four meta-attibutes "update-time:", "ref-time:", "ref-count:" and "expire-time:" should be maintained and used for this purpose (*). The values of these meta-attributes would be a modification timestamp, reference counter modification timestamp, reference counter and object expiration time (if applicable) respectively. The reference counter would contain the number of objects still refering to this object, and thus be similar to the link counter of an inode in a UNIX file system. The modification timestamp would be updated whenever the object is updated. The reference counter modification timestamp would be updated whenever link counter changes. The analogy in the UNIX file system would be sort of mtime and ctime. The expiration time will be updated for person and role object only. For other object types it will be emplty meaning "not set". The expiration time will be updated whenever link counter reaches zero or changes from zero to a positive value. In the first case the expiration time will be calculated by adding the expiration period to the time when this event occured. In the second case the expiration time will be unset (empty value). A garbage collection process running on a periodic basis should then generate a deletion update (transaction) overriding authorization checks (**) for all person and role objects that reached their expiration time. The expiration period is to be discussed and agreed upon in the community. The initial proposal is 90 days. Explicit generation of an update (transaction) is needed for tracking/debugging purposes. (*) Please see the "Object Metadata" proposal. (**) Overriding authorization allows deletion of a maintained object to be performed. Respective maintainers will be notified. Overriding authorization still doesn't allow deletion of a referenced object.

1 0

Object Metadata Proposal
by Andrei Robachevsky 08 Nov '01

08 Nov '01

Dear Colleagues, Please review the proposal below. This proposal is supposed to provide means for reliable timestamping, and also support garbage collection procedure (please see "Automatic DB cleanup proposal" following this one). But of course the functionality can be expanded when needed. Your comments are welcome as always, Best regards, Andrei Robachevsky RIPE NCC ------------------------------------------------------------- Object metadata proposal ======================== Motivation ---------- Several proposals already exist that require reliable status information about an object in the database, such as last update time, number of references, etc. These are requests for reliable timestamping and automatic cleanup procedure. In order to be reliable this information should be automatically generated by the database software and be outside users' control. Storing this information as part of the object data was generally considered not a good idea since this requires modifications to the object not solicited by the user, and breaks some basic principles of the RIPE Database. But in fact these attributes/facilities do not represent an object, but rather status information about the object. Extending Ulf Kieber's analogy with the Unix FS, they are part of the inode, not the data blocks. Object metadata --------------- Status information, or object metadata, will be automatically generated by the database, but stored and accessed separately from the object itself. This will not affect any object template, preserve users' data and add flexibility to the RIPE Database. Metadata Contents ------------------ Metadata contents will be presented as metadata attribute-value pairs, or meta-attributes. The initial set of meta-attributes is required by the above mentioned proposals and will include: update-time: <YYYY-MM-DD> <HH:MM:SS> This meta-attribute will store the time when the update (object creation or subsequent updates) was processed by the database software (when modification was made to the Database). ref-count: <number> The number of references to this object from other objects in the RIPE Database. Zero means the object is not referenced. ref-time: <YYYY-MM-DD> <HH:MM:SS> Last time the "ref-count:" was updated. It is empty if ref-count has never been updated. expire-time: <YYYY-MM-DD> <HH:MM:SS> This meta-attribute displays the time after which the object will be automatically deleted from the database overriding authorisation checks. Empty value of the meta-attribute means "not set". Please see "Automatic DB cleanup" proposal for more information. Other meta-attributes may be added as needed. Metadata will not propagate through NRTM. It is up to a NRTM client to generate the metadata as updates are processed. Querying Metadata ------------------ Object metatdata may be accessed by using "-q meta" flag when interrogating the database. This flag is can be used together with other queries, except other "-q" flags. Metadata of all objects returned will be displayed. An object metadata will be displayed after the object with a blank like between them. It is displayed as meta-attributes preceded by a '%' sign. According to RIPE whois definition a line starting with '%' sign should be generally considered as a comment and should be parsed at client's own risk. For example: $ whois -q meta RC1234-RIPE ... person: Referenced Contact nic-hdl: RC1234-RIPE ... changed: refcon(a)somedom.dom 20011005 source: RIPE %metainfo: RC1234-RIPE %update-time: 2001-10-05 16:50:07 %ref-time: 2001-10-06 09:17:33 %expire-time: %ref-count: 3 Using Metadata -------------- The metadata may be consulted by humans or metadata-aware clients for different purposes. The database software may use metadata for its garbage collection routine, or other internal processes.

1 0