Hi Ronald,
On 11 Jun 2021, at 10:19, Ronald F. Guilmette via db-wg <db-wg@ripe.net> wrote:
In message <032287F0-AD5A-48B9-AFEB-079D70F76726@ripe.net>, Edward Shryane <eshryane@ripe.net> wrote:
The DB team is implementing the NONAUTH cleanup job according to the proposal: https://www.ripe.net/ripe/mail/archives/db-wg/2021-March/006876.html <https://www.ripe.net/ripe/mail/archives/db-wg/2021-March/006876.html>
I see. That message is very interesting. There are a couple of parts of it that I have trouble interpreting however, to wit:
There are approximately 64 routes and 37 route6's with a prefix not listed in any RIR's delegated stats. These will not be affected.
Can you explain the meaning of that, in small words?
The cleanup job will use all of the RIRs delegated stats. If the route prefix is not defined in any delegated stats, then it's skipped. The count above didn't consider prefixes split between different RIRs, but now all stats are combined. I now found 28 routes and 37 route6 objects with a prefix not listed in any RIRs delegated stats (which I downloaded today). These prefixes were not listed in any delegated stats: 192.31.196.0/24 192.88.99.0/24 2001::/32 2001:4:112::/48 2002::/16 2011:4188::/48
Also, there's this:
Only route(6) prefixes that are "allocated" or "assigned" in any RIRs delegated stats should remain...
I'm just looking at the nro-extended-stats file. In that one, the only things marked as "allocated" are all just ASNs and they are all just "allocated" to IANA. So I'm not clear why you said ""allocated" or "assigned". The former set seems to be effectively a null set.
I'm not using the NRO extended stats, but rather I'm combining each RIRs delegated stats. Each RIRs delegated stats are more specific about the prefix status, including "assigned".
Last but not least, there is this very perplexing line:
The origin AS status is not considered, only the IPv4/IPv6 prefix.
Why?
The origin AS is not validated either in the RIPE database when creating or updating a route(6) object (apart from excluding AS numbers reserved by IANA). Also, this cleanup job was created out of discussion about deregistered address space: "196.52.0.0/14 revoked, cleanup efforts needed" in January.
Maybe it's just me, but I don't think that RIPE should be effectively condoning the use of unregistered bogon AS numbers. (And by the way, I just did some work on that. There are 82 route objects in the "normal" (non-RIPE-NONAUTH) data base at present that refer to bogon ASNs.
As long as there is a cleanup occurring, why not clean these up also? (See list at end.)
If the DB-WG agrees that we should, and can define *which* AS numbers should be cleaned up, the job can be extended to include AS numbers. If so, should *both* the prefix and origin be considered, or separately (i.e. should we cleanup if either is unregistered)?
I ran a script using those rules (against the latest split files and delegated stats from each RIR) and found the following unregistered prefixes: https://pastebin.com/aeyhUKeH <https://pastebin.com/aeyhUKeH>
Well, so I guess I need to get all those separate per-RIR delegated stats files, because I guess what the NRO has isn't actually accurate (?)
I used the per-RIR delegated stats as I'm more familiar with them (they are already used in Whois for the mirror databases).
My list appears to agree with your list (I didn't de-duplicate prefixes).
Ummm... no. As I had noted, I found 857 (non-deduped) bogon routes using my methodology. Your list appears to contain only 820.
After de-duping, the differences seem to come down to just these CIDRs:
41.217.144.0/21
From the AFRINIC delegated stats, 41.217.144.0/21 is composed of: afrinic|ZZ|ipv4|41.217.144.0|1024||reserved| afrinic|CM|ipv4|41.217.148.0|1024|20090529|allocated|F367BA6A From the cleanup proposal: If a prefix is partially "available" or "reserved" then it is also considered to be unregistered. So this prefix *is* included.
192.31.196.0/24
192.31.196.0/24 is not listed in any RIRs delegated stats, according to the NRO's delegated stats it's reserved for the IETF. I see this prefix is listed as an anycast address for DNAME: https://datatracker.ietf.org/doc/html/rfc7534 https://www.as112.net/well-known.html So I believe this prefix is eligible for cleanup, but it's not listed in the RIR delegated stats so it will be skipped.
192.88.99.0/24
This prefix is also listed in the NRO delegated stats as reserved for the IETF, and not in the RIR delegated stats.
192.175.48.0/24
That last one appears to be an error on my part... and I will be trying to figure out how & why that one got into my set.
This prefix is listed in ARIN's delegated stats arin|US|ipv4|192.175.48.0|256|19960105|assigned|9064d089391b6dcc310a2188da60c437 But in the NRO delegated stats it's listed as IETF reserved: iana|ZZ|ipv4|192.175.48.0|256|20150501|reserved|ietf|iana I don't know why.
The other three however seems to be ones that you missed.
The first can be explained by the "reserved" status, and the other two don't appear in the RIR delegated stats.
Hopefully our two lists will convergeat some point.
I think there will always be some differences between the NRO delegated stats and the RIR delegated stats (e.g. the IETF reserved prefixes).
Also for unregistered IPv6 prefixes referenced by NONAUTH route6 objects: https://pastebin.com/unK958cE <https://pastebin.com/unK958cE>
I'm not even looking at any IPv6 stuff myself.
IPv6 will be included in the cleanup job (about 94 route6 objects will be affected), the methodology should be the same as for IPv4.
We will announce to the DB-WG when our implementation is complete.
Any ETA for that?
The DB team are currently working on it, I don't have an ETA but I expect it will be ready in the coming weeks. Also thanks to you for checking NONAUTH for bogon route objects, your questions and comments are very welcome. Regards Ed Shryane RIPE NCC
Regards, rfg
P.S. Route objects in the regular data base that refer to bogon ASNs:
194.165.87.0/24 AS21389 83.216.160.0/19 AS31419 213.228.244.0/24 AS31453 86.110.128.0/19 AS31419 83.136.64.0/21 AS8871 83.216.160.0/20 AS31419 83.216.176.0/20 AS31419 86.110.128.0/20 AS31419 86.110.144.0/20 AS31419 194.99.204.0/22 AS36895 83.216.160.0/21 AS31419 83.216.168.0/21 AS31419 83.216.176.0/21 AS31419 83.216.184.0/21 AS31419 91.102.168.0/21 AS8295 91.102.168.0/24 AS8295 91.102.169.0/24 AS8295 91.102.170.0/24 AS8295 91.102.171.0/24 AS8295 91.102.172.0/24 AS8295 91.102.173.0/24 AS8295 91.102.174.0/24 AS8295 91.102.175.0/24 AS8295 193.33.110.0/23 AS42796 193.201.204.0/24 AS14880 193.201.205.0/24 AS14880 86.110.128.0/21 AS31419 86.110.136.0/21 AS31419 86.110.144.0/21 AS31419 86.110.152.0/21 AS31419 91.220.83.0/24 AS46414 5.1.112.0/21 AS198738 5.1.112.0/24 AS198738 5.1.113.0/24 AS198738 81.52.165.0/24 AS36879 81.52.166.0/24 AS36879 81.52.167.0/24 AS36879 81.52.168.0/24 AS36879 81.52.169.0/24 AS36879 185.48.220.0/22 AS199724 194.201.253.0/24 AS25568 5.160.196.0/24 AS62280 5.160.197.0/24 AS62280 5.160.240.0/24 AS62280 5.160.241.0/24 AS62280 77.246.74.0/24 AS37631 185.106.44.0/24 AS200968 158.255.59.0/24 AS203655 185.148.138.0/23 AS198738 82.98.78.0/24 AS394935 185.164.160.0/22 AS14076 185.200.32.0/22 AS33431 185.225.136.0/22 AS33431 185.244.144.0/22 AS204634 185.194.4.0/22 AS205740 93.92.104.0/22 AS1975505 185.62.220.0/22 AS1975505 185.62.221.0/24 AS1975505 217.68.70.0/24 AS214359 45.14.212.0/22 AS2209013 91.107.84.0/24 AS202532 185.148.136.0/22 AS198738 185.250.112.0/22 AS20180320 147.28.44.0/24 AS558363 147.28.45.0/24 AS558363 147.28.46.0/24 AS558363 147.28.47.0/24 AS558363 45.8.92.0/24 AS46723 45.8.95.0/24 AS46723 45.8.94.0/24 AS46723 45.8.93.0/24 AS46723 45.8.92.0/24 AS397796 45.8.95.0/24 AS397796 45.8.94.0/24 AS397796 141.98.12.0/24 AS208469 188.92.121.0/24 AS40178 193.135.119.0/24 AS46723 45.95.0.0/24 AS46723 45.95.1.0/24 AS46723 45.95.2.0/24 AS46723 45.95.3.0/24 AS46723 193.110.94.0/24 AS16712