Suggestion further validity-checking
Dear DB-WG, I am newly subscribed to this mailing-list although I am already active in RIPE-community for quite some time. Some of you might know me in person. To be honest, I am not sure if this is the right place to send my mail to and if it's not I'd be happy to be told who to address better. For debugging our prefix-list-generator I increased verbosity of the tool a little. As side-effekt I was also pointed to a route-object in the ripe-database which contains a "hole"-definition where the netmask doesn't match (X.Y.226.0/21) and actually is the same size as the entire route-object. I already contacted the maintainer of that object but think it might be a good idea to add a validity-check on adding new objects to prevent this kind of faulty objects in the DB. Best regards -- Jens Ott Geschäftsführer Opteamax GmbH Simrockstr. 4b 53619 Rheinbreitbach Tel.: +49 2224 969500 Fax: +49 2224 97691059 Email: jo@opteamax.de HRB: 23144, Amtsgericht Montabaur Geschäftsführer: Jens Ott Umsatzsteuer-ID.: DE264133989
Hi Jens,
On 28 May 2019, at 12:07, Jens Ott - Opteamax GmbH via db-wg <db-wg@ripe.net> wrote:
Dear DB-WG,
I am newly subscribed to this mailing-list although I am already active in RIPE-community for quite some time. Some of you might know me in person.
To be honest, I am not sure if this is the right place to send my mail to and if it's not I'd be happy to be told who to address better.
Seems to be very relevant
For debugging our prefix-list-generator I increased verbosity of the tool a little. As side-effekt I was also pointed to a route-object in the ripe-database which contains a "hole"-definition where the netmask doesn't match (X.Y.226.0/21) and actually is the same size as the entire route-object.
I found a few of these ('invalid route range' for the holes attribute) in route objects: - holes attribute 195.214.188.0/23 on route 195.214.184.0/22AS8705 - holes attribute 193.99.144.0/23 on route 62.191.180.0/22AS702 - holes attribute 194.174.168.0/21 on route 62.191.180.0/22AS702 - holes attribute 193.99.144.0/23 on route 158.116.242.0/24AS702 - holes attribute 194.174.168.0/21 on route 158.116.242.0/24AS702 All route6 objects appear to be ok.
I already contacted the maintainer of that object but think it might be a good idea to add a validity-check on adding new objects to prevent this kind of faulty objects in the DB.
Whois does validate the holes attribute when creating or updating a route/route6 object, but these invalid values were added before this rule was added (in 2009 or earlier). Unfortunately, no cleanup was done when this rule was implemented, but in recent times we try to do this. I will also contact the maintainers of these route objects and ask them to fix the holes attribute(s). In general, if you are parsing RIPE database objects, be sure to check for invalid values. Thanks for your feedback. Regards Ed Shryane RIPE NCC
Best regards -- Jens Ott Geschäftsführer
Opteamax GmbH
Simrockstr. 4b 53619 Rheinbreitbach
Tel.: +49 2224 969500 Fax: +49 2224 97691059 Email: jo@opteamax.de
HRB: 23144, Amtsgericht Montabaur Geschäftsführer: Jens Ott Umsatzsteuer-ID.: DE264133989
Edward Shryane via db-wg wrote on 28/05/2019 12:12:
Unfortunately, no cleanup was done when this rule was implemented, but in recent times we try to do this. I will also contact the maintainers of these route objects and ask them to fix the holes attribute(s). I wonder if this key should be formally deprecated. It's used for 643 out of 302354 route: objects and 40 out of 28803 route6: objects, i.e. ~0.2% and 0.1% respectively. The complexity associated with handling it is substantial and most tools simply ignore it.
Nick
On Tue, May 28, 2019 at 1:31 PM Nick Hilliard via db-wg <db-wg@ripe.net> wrote:
Edward Shryane via db-wg wrote on 28/05/2019 12:12:
Unfortunately, no cleanup was done when this rule was implemented, but in recent times we try to do this. I will also contact the maintainers of these route objects and ask them to fix the holes attribute(s). I wonder if this key should be formally deprecated. It's used for 643 out of 302354 route: objects and 40 out of 28803 route6: objects, i.e. ~0.2% and 0.1% respectively. The complexity associated with handling it is substantial and most tools simply ignore it.
+1 for deprecation of "holes:" key. Kind regards, Job
On Tue, May 28, 2019 at 01:36:45PM +0200, Job Snijders via db-wg wrote:
On Tue, May 28, 2019 at 1:31 PM Nick Hilliard via db-wg <db-wg@ripe.net> wrote:
Edward Shryane via db-wg wrote on 28/05/2019 12:12:
Unfortunately, no cleanup was done when this rule was implemented, but in recent times we try to do this. I will also contact the maintainers of these route objects and ask them to fix the holes attribute(s). I wonder if this key should be formally deprecated. It's used for 643 out of 302354 route: objects and 40 out of 28803 route6: objects, i.e. ~0.2% and 0.1% respectively. The complexity associated with handling it is substantial and most tools simply ignore it.
+1 for deprecation of "holes:" key.
+1 Piotr -- Piotr Strzyżewski Silesian University of Technology, Computer Centre Gliwice, Poland
Hi Ed,
Whois does validate the holes attribute when creating or updating a route/route6 object, but these invalid values were added before this rule was added (in 2009 or earlier).
The attributes of the object I observed, say it was created in 2011 and modified in 2017.
Unfortunately, no cleanup was done when this rule was implemented, but in recent times we try to do this. I will also contact the maintainers of these route objects and ask them to fix the holes attribute(s).
In general, if you are parsing RIPE database objects, be sure to check for invalid values.
That's what we do ... we silently drop any object containing invalid values and do not accept them at all. This one only came to my attention as it was just parsed while I increased log-level to debug and live-watched the logfile.
Thanks for your feedback.
It's in my own interest ;) .... and yes ... I also give a +1 for removing the holes attribute completely ;) Best regards Jens -- Jens Ott Geschäftsführer Opteamax GmbH Simrockstr. 4b 53619 Rheinbreitbach Tel.: +49 2224 969500 Fax: +49 2224 97691059 Email: jo@opteamax.de HRB: 23144, Amtsgericht Montabaur Geschäftsführer: Jens Ott Umsatzsteuer-ID.: DE264133989
+1 for removing "holes" attribute. On Tue, May 28, 2019 at 1:59 PM Jens Ott - Opteamax GmbH via db-wg < db-wg@ripe.net> wrote:
Hi Ed,
Whois does validate the holes attribute when creating or updating a
route/route6 object, but these invalid values were added before this rule was added (in 2009 or earlier).
The attributes of the object I observed, say it was created in 2011 and modified in 2017.
Unfortunately, no cleanup was done when this rule was implemented, but
in recent times we try to do this. I will also contact the maintainers of these route objects and ask them to fix the holes attribute(s).
In general, if you are parsing RIPE database objects, be sure to check
for invalid values.
That's what we do ... we silently drop any object containing invalid values and do not accept them at all. This one only came to my attention as it was just parsed while I increased log-level to debug and live-watched the logfile.
Thanks for your feedback.
It's in my own interest ;)
.... and yes ... I also give a +1 for removing the holes attribute completely ;)
Best regards Jens
-- Jens Ott Geschäftsführer
Opteamax GmbH
Simrockstr. 4b 53619 Rheinbreitbach
Tel.: +49 2224 969500 Fax: +49 2224 97691059 Email: jo@opteamax.de
HRB: 23144, Amtsgericht Montabaur Geschäftsführer: Jens Ott Umsatzsteuer-ID.: DE264133989
Hi Jens,
On 28 May 2019, at 13:59, Jens Ott - Opteamax GmbH <jo@opteamax.de> wrote:
Hi Ed,
Whois does validate the holes attribute when creating or updating a route/route6 object, but these invalid values were added before this rule was added (in 2009 or earlier).
The attributes of the object I observed, say it was created in 2011 and modified in 2017.
In this case, for the holes: attribute value 176.67.226.0/21, the prefix doesn't match the prefix length. Validation for this scenario was added after the object was modified in early 2017 (and existing data was not cleaned up when the validation was added). Regards Ed
Hi Jens, Can you share the exact example? I'd like to understand what type of data validity error your are pointing to. Kind regards, Job On Tue, May 28, 2019 at 12:08 PM Jens Ott - Opteamax GmbH via db-wg <db-wg@ripe.net> wrote:
Dear DB-WG,
I am newly subscribed to this mailing-list although I am already active in RIPE-community for quite some time. Some of you might know me in person.
To be honest, I am not sure if this is the right place to send my mail to and if it's not I'd be happy to be told who to address better.
For debugging our prefix-list-generator I increased verbosity of the tool a little. As side-effekt I was also pointed to a route-object in the ripe-database which contains a "hole"-definition where the netmask doesn't match (X.Y.226.0/21) and actually is the same size as the entire route-object.
I already contacted the maintainer of that object but think it might be a good idea to add a validity-check on adding new objects to prevent this kind of faulty objects in the DB.
Best regards -- Jens Ott Geschäftsführer
Opteamax GmbH
Simrockstr. 4b 53619 Rheinbreitbach
Tel.: +49 2224 969500 Fax: +49 2224 97691059 Email: jo@opteamax.de
HRB: 23144, Amtsgericht Montabaur Geschäftsführer: Jens Ott Umsatzsteuer-ID.: DE264133989
Hi Job, sure I can ;) route: 176.67.224.0/21 descr: VITSOL ASN route origin: AS56973 holes: 176.67.226.0/21 mnt-by: JKRATOS mnt-by: OHUBALEK-MNT mnt-by: PS32766-MNT created: 2011-06-29T13:16:32Z last-modified: 2017-01-30T12:27:51Z source: RIPE The holes network could have a maximum netmask of /23 Kind regards Jens Am 28.05.19 um 13:12 schrieb Job Snijders:
Hi Jens,
Can you share the exact example? I'd like to understand what type of data validity error your are pointing to.
Kind regards,
Job
On Tue, May 28, 2019 at 12:08 PM Jens Ott - Opteamax GmbH via db-wg <db-wg@ripe.net> wrote:
Dear DB-WG,
I am newly subscribed to this mailing-list although I am already active in RIPE-community for quite some time. Some of you might know me in person.
To be honest, I am not sure if this is the right place to send my mail to and if it's not I'd be happy to be told who to address better.
For debugging our prefix-list-generator I increased verbosity of the tool a little. As side-effekt I was also pointed to a route-object in the ripe-database which contains a "hole"-definition where the netmask doesn't match (X.Y.226.0/21) and actually is the same size as the entire route-object.
I already contacted the maintainer of that object but think it might be a good idea to add a validity-check on adding new objects to prevent this kind of faulty objects in the DB.
Best regards -- Jens Ott Geschäftsführer
Opteamax GmbH
Simrockstr. 4b 53619 Rheinbreitbach
Tel.: +49 2224 969500 Fax: +49 2224 97691059 Email: jo@opteamax.de
HRB: 23144, Amtsgericht Montabaur Geschäftsführer: Jens Ott Umsatzsteuer-ID.: DE264133989
-- Jens Ott Geschäftsführer Opteamax GmbH Simrockstr. 4b 53619 Rheinbreitbach Tel.: +49 2224 969500 Fax: +49 2224 97691059 Email: jo@opteamax.de HRB: 23144, Amtsgericht Montabaur Geschäftsführer: Jens Ott Umsatzsteuer-ID.: DE264133989
Hi Jens,
On 28 May 2019, at 13:21, Jens Ott - Opteamax GmbH via db-wg <db-wg@ripe.net> wrote:
Hi Job,
sure I can ;)
route: 176.67.224.0/21 descr: VITSOL ASN route origin: AS56973 holes: 176.67.226.0/21 mnt-by: JKRATOS mnt-by: OHUBALEK-MNT mnt-by: PS32766-MNT created: 2011-06-29T13:16:32Z last-modified: 2017-01-30T12:27:51Z source: RIPE
The holes network could have a maximum netmask of /23
Whois returns a different error in this case: "prefix length 21 is invalid on 176.67.226.0/21". This is the only holes attribute where the prefix length doesn't match the prefix (as you mentioned, /23 is ok).
Kind regards Jens
Regards Ed Shryane RIPE NCC
participants (6)
-
Cynthia Revström -
Edward Shryane -
Jens Ott - Opteamax GmbH -
Job Snijders -
Nick Hilliard -
Piotr Strzyzewski