automatic DB cleanup proposal
Re everybody, following up to some personal talks during the RIPE meeting I propose an automatic cleanup of orphaned person and role objects in the RIPE DB. Motivation ---------- In the light of data privacy issues raised by governmental organizations and in RIPE Database Working Group sessions during RIPE-39 and RIPE-40, as well as following the Great Dangling Person Objects Deletion after de.* ccTLD move-out I propose an automized garbage collection procedure to remove personal data that is unreferenced for a certain amount of time. Proposed Change --------------- An additional attibute ``expire:'' with two values should be added to all person and role objects. The values to this attribute would be a reference counter and a modification timestamp. The reference counter would contain the number of objects still refering to this object, and thus be analog to the link counter of an inode in a UNIX file system. The modification timestamp would be updated whenever the object is updated or the link counter changes. The analogy in the UNIX file system would be sort of a mix of ctime and mtime. Both values could be combined into a single value, but each value of it's own may be of interest to some people. Combining both values would leave the resulting value useless to human interpretation. A garbage collection process running on a periodic basis should then delete all objects with a reference counter of ``0'' and a modification timestamp of more than e.g. 8000000 seconds (approx. 3 month) ago. The expiration period is to be discussed and agreed upon in the community. An additional switch to the whois client should be added to make the otherwise invisible attribute visible, the DB should consequently not accept ``expire:'' attributes in updates. (The DB could certainly be made more forgiving about this, but since the move to v.3 tightened the syntax we should probably not weaken it again.) Database Objects affected ------------------------- An additional attribute would be added to all person and role objects. role: [mandatory] [single] [lookup key] address: [mandatory] [multiple] [ ] phone: [optional] [multiple] [ ] fax-no: [optional] [multiple] [ ] e-mail: [mandatory] [multiple] [lookup key] trouble: [optional] [multiple] [ ] admin-c: [mandatory] [multiple] [inverse key] tech-c: [mandatory] [multiple] [inverse key] nic-hdl: [mandatory] [single] [primary/look-up key] remarks: [optional] [multiple] [ ] notify: [optional] [multiple] [inverse key] mnt-by: [optional] [multiple] [inverse key] expire: [generated] [single] [ ] changed: [mandatory] [multiple] [ ] source: [mandatory] [single] [ ] person: [mandatory] [single] [lookup key] address: [mandatory] [multiple] [ ] phone: [mandatory] [multiple] [ ] fax-no: [optional] [multiple] [ ] e-mail: [optional] [multiple] [lookup key] nic-hdl: [mandatory] [single] [primary/look-up key] remarks: [optional] [multiple] [ ] notify: [optional] [multiple] [inverse key] mnt-by: [optional] [multiple] [inverse key] expire: [generated] [single] [ ] changed: [mandatory] [multiple] [ ] source: [mandatory] [single] [ ] Example ------- inetnum: 10.0.0.0 - 10.0.255.255 netname: XMAS descr: Santa's Workshop Inc. Christmas Toys Manufacturing Facility Northern Nowhere country: NN admin-c: SC12-RIPE tech-c: SWT95-RIPE status: ASSIGNED PA mnt-by: SANTA-SECURITY-MNT changed: jbgood@antarctic.nn 19960412 source: RIPE role: Santa Workshop Team address: Santa's Workshop Inc. Jingle Bell Lane 12 1224CH Christmastown Northern Nowhere phone: +12 12 122 3333 fax-no: +12 12 221 4444 e-mail: team@antarctic.nn admin-c: SC12-RIPE tech-c: SC12-RIPE tech-c: RD212-RIPE nic-hdl: SWT95-RIPE mnt-by: SANTA-SECURITY-MNT expire: 1 849700000 changed: jbgood@antarctic.nn 19960412 source: RIPE person: Santa A. Claus address: Santa's Workshop Inc. Jingle Bell Lane 12 1224CH Christmastown Northern Nowhere phone: +12 12 122 2121 +12 12 122 2211 ext. 1221 fax-no: +12 12 221 1212 e-mail: santa@antarctic.nn nic-hdl: SC12-RIPE mnt-by: SANTA-SECURITY-MNT expire: 3 849700000 changed: jbgood@antarctic.nn 19960412 source: RIPE person: Rudolf R N Reindeer address: Santa's Workshop Inc. Jingle Bell Lane 21 1224CH Christmastown Northern Nowhere phone: +12 12 122 1111 fax-no: +12 12 122 2222 nic-hdl: RD212-RIPE e-mail: rudolf@xmas.nn mnt-by: SANTA-SECURITY-MNT expire: 1 849700000 changed: jbgood@antarctic.nn 19960412 source: RIPE -- Ulf Kieber email: kieber@gatel.net Senior Network Engineer voice: +49-69-299896-21 Global Access Telecommunications, Inc. fax : +49-69-299896-40 internet solutions for business www : www.gatel.net The Database Toothbrush Proposal
I think the base principles behind cleanups are great, and I intend proposing APNIC explore a similar course of action to the manual sweep. I am worried that there may be *external* referencees to objects, and while we have no 'contractual' obligation it might not be nice to do this to people without some engagement but I expect most people would welcome losing a path to spam. As to automatic sweeping, I think there are problems with link counting which are similar to those we've faced in discussing cross-database linkage and references. Too many risks of the link count getting out of whack. I think you could use it as a guide to a sweep, but not as a determinant without some review and other checks. Eg to trigger more exhaustive checks to see if there are are references in twisty ways. cheers -George
George Michaelson writes:
I am worried that there may be *external* referencees to objects, and while we have no 'contractual' obligation it might not be nice to do this to people without some engagement but I expect most people would welcome losing a path to spam.
external references to DB objects are never going to work reliably. I don't see why we should try ever so hard in making something work when we can't succeed. Also, we would have to argue with data privacy agencies all over Europe as to why we need to leave an object in the DB that's referenced only from an external source. ``They should create their own object in their own DB'' would be an almost certain response from the data privacy guys.
As to automatic sweeping, I think there are problems with link counting which are similar to those we've faced in discussing cross-database linkage and references. Too many risks of the link count getting out of whack.
Nope. The RIPE DB already has a mechanism that prevents objects from being deleted if there are still references to it. So, a double check is already in place. :) The generation of the reference counter should be done by counting references in the DB anyway, instead of mere incrementing or decrementing the existent counter. This way the DB resyncs every time a person or role object is touched.
I think you could use it as a guide to a sweep, but not as a determinant without some review and other checks. Eg to trigger more exhaustive checks to see if there are are references in twisty ways.
The only other precaution that should be taken IMHO is to honor any applicable mnt-nfy and notify attribute. -- Ulf Kieber email: kieber@gatel.net Senior Network Engineer voice: +49-69-299896-21 Global Access Telecommunications, Inc. fax : +49-69-299896-40 internet solutions for business www : www.gatel.net
I think the base principles behind cleanups are great, and I intend proposing APNIC explore a similar course of action to the manual sweep.
Indeed, I also applaud the initiative.
I am worried that there may be *external* referencees to objects, and while we have no 'contractual' obligation it might not be nice to do this to people without some engagement but I expect most people would welcome losing a path to spam.
I seem to recall statements to the effect that "the RIPE database is not a phone directory" from quite a while ago when the issue of the "dangling person" objects was discussed, indicating that the RIPE database should not be used in this manner. Regards, - Håvard
Hi, On Mon, Oct 08, 2001 at 08:02:38PM +0200, Ulf Kieber wrote:
following up to some personal talks during the RIPE meeting I propose an automatic cleanup of orphaned person and role objects in the RIPE DB.
Proposal seconded, with the change discussed already (use the same time format as in "changed:"). Gert Doering -- NetMaster -- SpaceNet AG Mail: netmaster@Space.Net Joseph-Dollinger-Bogen 14 Tel : +49-89-32356-0 80807 Muenchen Fax : +49-89-32356-299
participants (4)
-
George Michaelson -
Gert Doering -
Havard Eidnes -
kieber＠gatel.net