April 2022 - db-wg - mailman.ripe.net

Decision on NWI-2 Historical queries
by denis walker 18 May '22

18 May '22

Colleagues When I wrote the spec for historical queries, almost 10 years ago, I included an arbitrary constraint to only show history back to the most recent deletion point. This was to get something in production quickly and see how useful it was. Over the years several people have asked for this arbitrary constraint to be removed. No one has objected to removing it. The co-chairs therefore recommend that this arbitrary constraint be removed. The co-chairs now ask the RIPE NCC to produce an impact analysis and implementation plan to remove it. We will then seek a final approval from the community on the plan. cheers denis co-chair DB-WG

5 10

Whois Release 1.103
by Edward Shryane 09 May '22

09 May '22

Dear Colleagues, RIPE Database release 1.103 has been deployed to the Release Candidate environment. We plan to deploy to production on Monday 9th May. The database in the Release Candidate environment has been reloaded with dummified production data from 20th April. This release includes the following main changes: • Fixed "goefeed:" validation when reading object containing multiple remarks: attributes (#975) • Removed WHITEPAGES org type. (#1003) • Better Error Message when Changing "mnt-by:" on Allocations (#988) • Allow user "mnt-by:" when creating inet6num allocations (already allowed for inetnums) (#982) • Disallow weak keys and hash algorithms (#642, #974) • Abuse-c is Required for End User Organisations With Resources (#955, #969) • Domain overlap validator (#956) • Increase HTTP idle timeout from 30s to 60s (#998) The release notes are on the website: https://www.ripe.net/manage-ips-and-asns/db/release-notes/ripe-database-rel… More information on the Release Candidate environment can be found on our website: https://www.ripe.net/manage-ips-and-asns/db/release-notes/rc-release-candid… Please let us know if you find any issues with this release in the RC environment. Regards Ed Shryane RIPE NCC

1 1

Missing org: fields for direct allocations from RIPE
by Ronald F. Guilmette 30 Apr '22

30 Apr '22

Some considerable time ago I previously noted here the fact that there exists in the RIPE WHOIS data base some WHOIS records for what I personally would refer to as "top level" IP address block allocations (i.e. ones which were assinged directly by RIPE NCC to some specific RIPE member organization) and that contain no org: field which would otherwise associate these blocks with a specific organisation having its own organisation: record in the data base. At that time, I was basically told (by denis, as I recall) that this was a historical artifact in the data base, that it would absolutely NOT be rectified, and that I should go pound sand. Having long ago become resigned to this level of respectful helpfulness when raising legitimate issues relevant to the RIPE data base here, I let it go, and I have not raised the issue again since that itme. Now however I feel that I have some clear and concrete basis for doing so. I call everyone's attention to the following post that I just made to the mailing list of the RIPE Anti-Abuse Working Group: https://www.ripe.net/ripe/mail/archives/anti-abuse-wg/2022-April/006346.html I should say that it is my (naive?) belief that (a) every operator of an inbound mailbox or an inbound mail server should have the right to decide for themselves when any given network is not worthy of having its inbound emails accepted, and also (b) RIPE and the RIPE WHOIS data base should not make the task of locally filtering inbound packets or emails from a given network any harder than it needs to be. As noted in the mailing list post linked to above, it is my desire to locally block all further inbound emails from the entire set of IPv4 CIDRs currently associated with the Dutch network which is alternatively known as Signet B.V. and/or TransIP B.V. (These people are clearly lack the requsite minimal level of competence to run a network from which outbound emails emmanate.) In any universe guided by either reason or simplicity it should be easily possible to find the ORG handle for any given (offending) network and then to use that handle as a "reverse" lookup key in a query to the relevant RIR data base and thus automatically derive the full set of IP address blocks currently assigned to the given organization. Unfortunately, and for reasons that have yet to be adequately explained, we do not live in that simple or rational universe, at least not when it comes to the RIPE WHOIS�data base. For quite some time now I have had a relatively trivial Perl script that does this exact job. I call it "org2cidrs". It works flawlessly when provided an ORG handle known to any one of the five Regional Internet Registries, except in the case of RIPE where, as I have noted above, some IP block allocations have no org: field and thus no connection to any particular organisation... at least none that could be automatically determined by a reverse WHOIS query based on some ORG handle. The bottom line is that although I have found it quite easy to deduce that the handle ORG-SI6-RIPE represents the offending organization that I would now like to block, a reverse (org) WHOIS query against the RIPE WHOIS server using that handle yields only a subset of the IP block allocations currently assigned to that organisation. More specifically, although the vast majority of IP blocks assigned to this organization do in fact have corresponding WHOIS records in the data base that -do- contain an org: field designating ORG-SI6-RIPE as the registrant organization, the following two blocks, both marked as "status: LEGACY", have no org: field present in their respective RIPE WHOIS records, and thus these allocation records are effectively invisible to any reverse WHOIS query using -any- ORG handle (e.g. ORG-SI6-RIPE) as lookup key: 136.144.128.0/17 157.97.168.0/22 I don't know how to say this delicately, so I will just say it. It is, in my opinion, fundamentally idiotic that every other one of the five Regional Internet Registries make it trivially easy to find ALL of the IP allocations associated with a given ORG handle, but that RIPE, for reasons that are apparently shrouded in mystery and/or which have long ago been lost to the sands of time, continues to insist on making what should be simple difficult. The last time I checked there were only on the order of about 122 "top level" RIPE IP block allocations whose corresponding WHOIS records failed to include an org: field. In each of these cases it would be trivially possible for RIPE NCC to manufacture an appropriate organization: record out of the information already in the data bases and/or information which is readily available on the web or from the actual organizations themselves. Once these new organisation: records were created it would also be trivial for RIPE NCC to insure that every single IP block allocation in the data base is associated with some ORG handle, thus simplifying automated processing of the data base -and- bringing its functionality into line with that of the other four RIR data bases. The fact that this has not already been done, especially in a case like the legacy blocks which clearly belong to ORG-SI6-RIPE, is, I think, indicative of some narrow-minded insistance on the preservation of past practices, even when those practices are, as in this case, provably detrimental to reasonable and serious users of the data base. Regards, rfg P.S. I have always and will always argue in favor of anything that simplifies the parsing and/or automated processing of WHOIS data base records. Having *every* IP block allocation record include an org: field would do just that. Furthermore, quite frankly I am both flummoxed and flabberghasted that a call was made here, some time ago, for suggestions on how to make it simpler to parse the data base, EVEN AS my simple, straightforward, and (I would have thought) obvious suggestion to represent all handles in the data base only and consistantly in upper case garnered -no- response from this working group whatsoever.

2 3

NWI-8 LIR´s SSO Authentication Groups
by denis walker 25 Apr '22

25 Apr '22

Colleagues This work item had two defined stages. Stage 1 was to introduce a default authentication group for LIRs. This has been implemented and is in widespread use now. Stage 2 was to introduce user defined authentication groups. This has had no discussion with little support or objection. If anyone feels this would be a useful extension please let the working group know. Otherwise the co-chairs will recommend that stage 2 is cancelled and NWI-8 is marked as Finished. cheers denis co-chair DB-WG

1 0

NWI-4 status for an assigned allocation
by denis walker 25 Apr '22

25 Apr '22

Colleagues This has been discussed over many years. The co-chairs recently asked the RIPE NCC to produce an impact/implementation report on the suggestion to introduce a new status value 'ALLOCATED-ASSIGNED PA'. This had one supporter and one objection. The objection was basically that 'something' will break. As I have since pointed out, the status values and structure has been changed several times over the years and nothing significant broke. Do we have any final comments before the RIPE NCC produces an impact/implementation report on this suggestion? cheers denis co-chair DB-WG

1 0

NWI-2 historical queries
by denis walker 25 Apr '22

25 Apr '22

Colleagues Over the last few years several community members have asked for the arbitrary restriction on deleted objects in historical queries of operational data to be removed. The DB-WG co-chairs recently recommended that the RIPE NCC goes ahead with producing an impact/implementation report, including a new legal review. Since then we have had one objection based on privacy of data. I believe I have addressed that issue in some detail. The co-chairs recommend that the RIPE NCC goes ahead with publishing a new impact/implementation report o this issue. cheers denis co-chair DB-WG

1 0

New NWI for geofeed?
by denis walker 14 Apr '22

14 Apr '22

Colleagues [Apologies for the length of this email...] The chairs would like to suggest creating a new NWI for the "geofeed:" attribute and suggest the following draft Problem statement and Solution definition. If there is agreement on this the RIPE NCC will do an impact assessment including legal review and summary of what the other RIRs are doing. There has already been quite a discussion in the WG on this issue with a lot of support. So hopefully we can reach a consensus quickly, at least on setting up the NWI and starting the impact assessment. Having read the latest draft IETF docs, there are some outstanding questions. Comments and changes are welcome... cheers denis co-chair DB-WG Problem statement Associating an approximate physical location with an IP address has proven to be a challenge to solve within the current constraints of the RIPE Database. Over the years the community has chosen to consider addresses in the RIPE Database to relate to entities in the assignment process itself, not the subsequent actual use of IP addresses after assignment. The working group is asked to consider whether the RIPE Database can be used as a springboard for parties wishing to correlate geographical information with IP addresses by allowing structured references in the RIPE Database towards information outside the RIPE Database which potentially helps answer Geo IP Location queries The IETF is currently discussing an update to RPSL to add a new attribute "geofeed: url". The url will reference a csv file containing location data. Some users have already started to make use of this feature via the "remarks: geofeed: url". It is never a good idea to try to overload structured data into the free format "remarks:" attribute. This has been done in the past, for example with abuse contact details before we introduced the "abuse-c:" attribute. There is no way to regulate what database users put into "remarks:" attributes. So even if the new "geofeed:" attribute is not agreed, the url data will still be included in the RIPE Database. Currently there are 24,408 INETNUM and 516,354 INET6NUM objects containing a "remarks: geofeed: url" attribute in the database. These have 7,731 distinct values in the INETNUMs and 1,045 distinct values in the INET6NUMs. Solution definition Implement a new "geofeed:" attribute according to the IETF's definition. Although the IETF has not yet concluded discussions on this attribute we can still implement it in the RIPE Database RPSL data definition. The RIPE Database already has many local differences to the RPSL standard. As expressed in the Problem statement, users are already using the geofeed data by overloading the "remarks:" attribute. That is a dirty hack which should be avoided. An invalid formated url will be a syntax error. The RIPE NCC will perform a one time conversion of the existing data to convert "remarks: geofeed: url" to "geofeed: url". If an update then contains a "remarks: geofeed: url" attribute, the update will be successful and the response should include an appropriate Warning message. At some point in the (near) future, this could be changed to an update failure as a syntax error. An update containing a "geofeed:" and a "remarks: geofeed:" attribute or more than one "remarks: geofeed:" will be a syntax error. The resource holder should be able to create, modify, delete the "geofeed:" attribute in allocation objects. Questions: -Should the database software do any checks on the existence/reachability of the url as part of the update with an error if the check fails? -Should the RIPE NCC do any periodic repeat checks on the continued existence/reachability of the url? -Should the RIPE NCC do any periodic checks on the content structure of the csv file referenced by the url? -Should the Solution definition define how this will be adopted into RDAP or should we simply ask the RIPE NCC to define this in their impact assessment? -The RIPE Database contains hierarchical address space objects. Should it be acceptable for "geofeed:" attributes to exist at multiple levels within a specific hierarchy? -Suppose a geofeed csv file referenced by a /16 INETNUM object contains location data for the whole /16. Then a more specific /24 INETNUM object references another geofeed csv file that contains conflicting location data for this /24. Should this be a concern for the RIPE Database? -Should geofeed data be inherited? If you query for a /24 that does not contain a "geofeed:" attribute, but a less specific /16 does contain a "geofeed:" attribute, should this data be returned? In other words could it be used in a similar way to "abuse-c:"? -Thinking ahead to how people will actually deploy this data and what short cuts they could make. It is said that when reading a geofeed csv file, consumers of the data should ignore all data within that file not directly concerning the address space queried in the RIPE Database. Could you therefore create a single csv file with location data for all your address space and reference the same file in all your RIPE Database address objects? The address space owner could rely on the data consumer to pick out the correct piece of data for the relevant address space. The manager of the csv file then only has to work with one file. If this is possible and does happen (which the IETF doc 'Finding geofeeds' seems to suggest is possible for unsigned geofeed data), would it therefore make sense to apply "geofeed:" hierarchically as with "abuse-c:"? Allow a single, default "geofeed:" attribute in the ORGANISATION object to be applied to all that organisations address space, with the option of specific localised "geofeed:" attributes in address space objects. That could be a neater solution, and easier to setup, than applying thousands of references to the same geofeed file at a more specific level in the database. -Relating to the above 3 questions, should geofeed data only be considered applicable if returned by a specific geofeed locater application which takes into account the hierarchical nature of address space in the database? Otherwise do the standard database query mechanisms have to take into account this hierarchy and locate the most specific "geofeed:" attribute from the less specific objects? -Should/could the RIPE Database return the csv file as part of the query? If so should the file be cached (for how long?) to avoid too many downloads? -Should we only allow HTTPS urls? (Which the IETF doc 'Finding geofeeds' seems to suggest) -Should the RIPE NCC go ahead and implement this now, with our own set of RIPE rules? Or should we try to coordinate this and agree a set of common rules between all the RIRs before any deployment in the RIPE Database? -For the legal review, there are 2 statements in the IETF doc 'Finding geofeeds' which may be of concern: *[RFC8805] geofeed data may reveal the approximate location of an IP address, which might in turn reveal the approximate location of an individual user. Unfortunately, [RFC8805] provides no privacy guidance on avoiding or ameliorating possible damage due to this exposure of the user. In publishing pointers to geofeed files as described in this document the operator should be aware of this exposure in geofeed data and be cautious. All the privacy considerations of [RFC8805] Section 4 apply to this document. *It is significant that geofeed data may have finer granularity than the inetnum: which refers to them. It is clear that the RIPE NCC cannot prevent this data being referenced by objects in the RIPE Database. It is already being referenced from "remarks:" attributes. Perhaps the RIPE NCC should require (as part of their service agreement) that it's members obtain written consent from their customers to publish this location data, or at least inform the customers in writing that it will be published. Also, although RFC8805 says postcode is deprecated it is still provided for in the csv files. So anyone can still enter location data to this detail. -The IETF doc 'Finding geofeeds' suggests that geofeed information 'will be' available in bulk accessed whois data. In view of the privacy concerns above, is this likely? -The IETF doc 'Finding geofeeds' says "To minimize the load on RIR whois [RFC3912] services, use of the RIR's FTP [RFC0959] services SHOULD be the preferred access." Is the RIPE NCC expected to download all the geofeed files and make them available through their FTP service? -The IETF doc 'Finding geofeeds' states that consumers of the geofeed data MUST NOT access this data in real time via the RPSL servers 'too frequently' or at 'magic times like midnight'. Some users will do whatever they want to do if they are able to do, regardless of any statements to the contrary. Should the RIPE NCC enforce such access rules by some means? References The IETF doc 'Finding geofeeds': https://datatracker.ietf.org/doc/draft-ietf-opsawg-finding-geofeeds/?includ… geofeed file format: https://www.rfc-editor.org/rfc/rfc8805.html

12 53

Future of "geoloc:"
by denis walker 13 Apr '22

13 Apr '22

Colleagues Now we have the "geofeed:" attribute, what should we do with "geoloc:"? Right now far more people are using "geoloc:" than "geofeed:". denis$ zgrep "^geofeed:" ~/Desktop/ripe.db.inetnum.gz | wc -l 289 denis$ zgrep "^remarks:\s*geofeed" ~/Desktop/ripe.db.inetnum.gz | wc -l 13 Although from the 289 there are only 161 unique organisations. denis$ zgrep "^geoloc:" ~/Desktop/ripe.db.inetnum.gz | wc -l 35453 cheers denis co-chair DB-WG

4 3

Decision on NWI-4 INETNUM status values
by denis walker 11 Apr '22

11 Apr '22

Colleagues For many years people have been asking for a way to assign a whole allocation without having to create 2 smaller assignments. The situation is more relevant now with so many /24 allocations. After the RIPE NCC rejected the idea of a dual primary key, for technical reasons, there were two options on the table. Making the "status:" attribute multiple or creating a new status value 'ALLOCATED-ASSIGNED PA'. No one has objected to either of these options. The co-chairs therefore recommend the option put forward by the RIPE NCC to add a new status value 'ALLOCATED-ASSIGNED PA'. Of the two options, this is probably the simplest and likely to cause less problems with user's scripts that parse objects. If there are no objections to this, the co-chairs now ask the RIPE NCC to produce an impact/implementation report to add this new status value and include the business rules to restrict it's use. We will then seek a final approval from the community on the report. cheers denis co-chair DB-WG

6 20