- db-wg - mailman.ripe.net

Re: [db-wg] Proposal to allow non-ASCII characters in "org-name:", "person:" and "role:" attributes
by Ángel González Berdasco 18 Dec '23

18 Dec '23

It's nice to see you here, Marco. I don't think it will be possible to change the default without potentially causing problems for clients not expecting that. Adding that -C option to the server seems appropriate, and unless you expect that at one point there will be no clients not providing that, you will face the issue that if the default changes from iso-8859-1 to utf-8, old clients will be wrong, as you can't get every client to be updated at the same time. The most similar situation would be if you had a predefined epoch at which both server and clients would change the encoding. But it still requires all clients to have been updated, and the date having been agreed far in advance (and not posponed later!). A situation that could help would be if the server marked the encoding so that the client could recognise the output is no longer iso-8859-1, but utf-8, such as including a BOM (although it may be preferable to include that inside a comment rather than at the beginning, I see benefits both ways), and thus old-but-not-ancient clients could autodetect the switch. Most likely, the server default will not change, though. Best regards -------- Original message -------- From: Marco d'Itri via db-wg <db-wg(a)ripe.net> Date: 12/18/23 00:44 (GMT+01:00) To: Piotr Strzyzewski <piotr(a)internetsailor.net> Cc: db-wg <db-wg(a)ripe.net> Subject: Re: [db-wg] Proposal to allow non-ASCII characters in "org-name:", "person:" and "role:" attributes On Dec 03, Piotr Strzyzewski via db-wg <db-wg(a)ripe.net> wrote: > As the UTF-8 topic was briefly discussed during DB-WG session at RIPE87 > in Rome, I would like to propose moving forward with it. If that means a > topic for first (?) interim meeting, let it be. Let me know please if > this works for you. Thanks in advance. In Rome I talked a bit with Edward about this. Background: I am the author of the whois client used by all Linux distributions. I fully agree that switching to UTF-8 is desirable, but we cannot just change the encoding of port 43 without major side effects. Since version 5.5.4 (december 2019), the client assumes that the output of whois.ripe.net is Latin 1 and then transcodes it to the system encoding. Receiving unexpected UTF-8 would cause mojibake. My suggestion is to add a new query "command line" option to specify the desired encoding (limiting it to either ISO-8859-1 or UTF-8), as supported by other whois servers. -C is the most common choice, but maybe it would be better to use --charset to not waste a single letter option. See https://github.com/rfc1036/whois/blob/next/servers_charset_list . In a few years then it will be much easier to switch the default from Latin 1 to UTF-8. -- ciao, Marco

1 0

Whois Release 1.109
by Edward Shryane 05 Dec '23

05 Dec '23

Dear colleagues, RIPE Database release 1.109 has been deployed to the Release Candidate environment for testing. We plan to deploy to production on Monday 4th December. The data in the RC environment is a dummified copy of production from 16th November. This release has the following main changes: * RDAP redaction (#1317) * Removed organisation "e-mail" from RDAP to match Whois and added to redaction list (#1333) * Removed geofeed prefix validation (#1326) * Added HTTPS Support to Whois Backend (#1329) * Support HTTP Basic authentication (#1342) * Support HTTPS Client Certificate Authentication (#1343) * Initialize NRTMv4 from command-line (JMX) (#1310) * Use parallel stream to generate NRTMv4 snapshots more quickly (#1309) * Include AS-SET in RIPE-NONAUTH dump (#1352) The release notes are on the website: https://apps.db.ripe.net/docs/Release-Notes/#ripe-database-release-1-109 More information on the Release Candidate environment can be found on our website: https://apps.db.ripe.net/docs/Release-Notes/ Please let us know if you find any issues with this release in the RC environment. Regards Ed Shryane RIPE NCC

1 1

Impact Analysis for NWI-4: using the inetnum and status tuple as a primary key
by Edward Shryane 28 Nov '23

28 Nov '23

Dear colleagues, Following is the impact analysis for NWI-4 "Role of status: field in multivalued status context" based on the use of the "inetnum:" and "status:" tuple as a primary key. Please let me know any questions, corrections or suggestions, in particular if there is impact to any functionality that I haven't considered. I will update the impact analysis as needed. Regards Ed Shryane RIPE NCC Summary ------- Firstly, refer to the Problem Statement for NWI-4: https://www.ripe.net/ripe/mail/archives/db-wg/2016-May/005242.html "Some believe that the main underlying issue here is that it is currently not possible to create an assignment that is the same size as an allocation in the RIPE Database. And resource holders are of course supposed to create an assignment for the address space in an allocation that is in use, by address policy." We investigated the impact of allowing a Provider Aggregatable ("PA") assignment to be the same size as its parent allocation to satisfy NWI-4, using a tuple of the prefix and status as the primary key to differentiate between them. This change will have a major impact on querying and updating inetnum objects in the RIPE database. A previous impact analysis of a new status value "ALLOCATED-ASSIGNED PA" was proposed in December 2022: https://www.ripe.net/ripe/mail/archives/db-wg/2022-December/007722.html but has a number of drawbacks, in particular: * The LIR and end-user organisations cannot both be added to the inetnum * The LIR and End User cannot both co-maintain the inetnum So we do not consider this as a feasible solution for NWI-4. General ------- The inetnum primary key tuple will consist of an "inetnum:" attribute value in prefix *or* range notation, immediately followed by the "status:" attribute value. Using the status as part of the inetnum primary key is *optional* and only necessary if there are multiple inetnums with the same prefix. There are space characters in the inetnum "status:" values "ALLOCATED PA" and "ASSIGNED PA", meaning the tuple (primary key) will have a space in it. Sorting inetnums in a hiearchy will take *both* the prefix and the status into account, e.g. an inetnum with "ALLOCATED PA" status will be ordered before an "ASSIGNED PA" status. Only "ALLOCATED PA" and "ASSIGNED PA" status values can be used in the primary key tuple. Only a single "ASSIGNED PA" inetnum can have the same prefix as the "ALLOCATED PA" inetnum. Whois (Port 43) Queries ----------------------- Support will be added for an inetnum tuple in Whois queries. Adding the status to an inetnum prefix or range will be optional. If an exact match (i.e. returning a single inetnum) is needed in the query response, a client may have to specify the tuple (i.e. both the inetnum prefix and status) in the query. If both an allocation and assignment are returned with the same prefix or range, then the client needs to match which inetnum it was querying for based on the status. Querying for only an inetnum prefix or range without any hierarchical flags may return multiple inetnum objects in the response, where previously only one was returned. The allocation is returned first, followed by the assignment. If using the exact match (-x) flag and only specifying the inetnum prefix or range, multiple inetnum objects may be returned in the response, where previously only one was returned. The allocation is returned first, followed by the assignment. If using the less specific (-L, -l) flag, if both an assignment and allocation with the same prefix or range are matched, then first the less-specific assignment is returned, followed by the allocation. If using the more specific (-M, -m) flag, if both an assignment and allocation with the same prefix or range are matched, then first the more-specific allocation is returned, followed by the assignment. RDAP API -------- Querying RDAP for an "ip" resource in the RDAP protocol can only be done by prefix, not by prefix and status. Hence using a tuple as the primary key is incompatible with the RDAP protocol. If there are multiple matches in RDAP for the same inetnum prefix or range key, then an error is returned. To support querying RDAP for a single "ip" resource, we will need to extend the protocol to include the inetnum prefix and (optional) status tuple. REST API -------- The REST API will be updated to allow a tuple to be optionally used in the URL for an inetnum primary key. The status must immediately follow the inetnum prefix or range in the URL path and all spaces are URL-encoded. REST API requests may have to use a tuple as the inetnum primary key, if there are multiple matches on the prefix alone. REST API responses containing an inetnum may contain a tuple as the primary key. REST API Create --------------- There is no change when creating an allocation via the REST API, as it is assumed that no assignment will exist yet. Creating an allocation is only allowed by the RIPE NCC. Creating an assignment the same size as an existing allocation will be allowed. The primary key is not included in the request URL when creating an object. For example: (POST) https://rest.db.ripe.net/ripe/inetnum REST API Lookup --------------- Performing a REST API lookup for an inetnum object requires the tuple to be specified only if both an allocation and assignment exist with the same prefix. For example: (GET) https://rest.db.ripe.net/ripe/inetnum/192.0.2.0/24ASSIGNED%20PA (GET) https://rest.db.ripe.net/ripe/inetnum/192.0.2.0%20-%20192.0.2.255ASSIGNED%2… (GET) https://rest.db.ripe.net/ripe/inetnum/192.0.2.0/24ALLOCATED%20PA Performing a lookup for a single inetnum prefix or range will fail if the status is not specified and there are multiple matches (i.e. a REST API GET request must not match multiple objects). REST API Query (Search) ----------------------- Refer to the Whois Query (port 43) section above, as the query behaviour is the same for REST API search. Searching for an inetnum prefix or range alone may return multiple inetnum matches (i.e. both an allocation and an assignment) where previously only a single inetnum was returned. For example: (GET) https://rest.db.ripe.net/search?query-string=192.0.2.0 The client will need to match which inetnum it was querying for using the status. Searching for an inetnum prefix or range and status tuple will be supported and will return a single inetnum match. For example: (GET) https://rest.db.ripe.net/search?query-string=192.0.2.0/24ASSIGNED%20PA (GET) https://rest.db.ripe.net/search?query-string=192.0.2.0%20-%20192.0.2.255ASS… REST API Update and Delete -------------------------- Specifying the tuple as a primary key in the URL when updating or deleting an inetnum is optional (using PUT or DELETE methods). Updating or deleting an inetnum will only require the tuple to be specified if there are multiple objects matching the prefix. Syncupdates ----------- There is no impact on Syncupdates. Objects are submitted in RPSL format and a tuple is not used. Mailupdates ----------- There is no impact on Mailupdates. Objects are submitted in RPSL format and a tuple is not used. Nightly Dump and Split Files ---------------------------- In the nightly dump and split files (made available on the FTP site), multiple inetnum objects may have the same "inetnum:" prefix but with different "status:" values. Clients parsing these files may need to be updated to match an inetnum based on the tuple of prefix and status. Delegated Stats File -------------------- In the RIPE delegated stats file (made available on the FTP site), multiple inetnum objects may share the same prefix but with different "status:" values. Clients parsing the delegated stats file may need to be updated to match an inetnum based on the tuple of prefix and status. Refer to the delegated stats format specification: https://ftp.ripe.net/ripe/stats/RIR-Statistics-Exchange-Format.txt An IPv4 resource is specified including the start (address), value (size) and status. Authorisation rules for creating route objects ---------------------------------------------- To create a route object in the RIPE database, you must authenticate against the address space you are referring to. First an existing exactly matching route object is checked. If there is none, then any less specific route is checked. If there is none, then an exactly matching or less specific inetnum is checked. If the exactly matching or less specific inetnum is an allocation and assignment with the same prefix, then (both) the assignment is checked first followed by the allocation. Ref. https://apps.db.ripe.net/docs/Authorisation/Protection-of-Route-Object-Spac… Authorisation rules for creating domain objects ----------------------------------------------- The creation of domain objects for "in-addr.arpa" domains must satisfy a hierarchical authorisation against inetnum address space. First an existing exactly matching inetnum object is checked. If there is none, then a less specific inetnum is checked. If the exactly matching or less specific inetnum is an allocation and assignment with the same prefix, then (both) the assignment is checked first followed by the allocation. Ref. https://apps.db.ripe.net/docs/Authorisation/Protection-of-Reverse-Delegatio… Force deleting inetnum objects ------------------------------ Force delete allows you to delete inet(6)num, route(6) and domain objects by using the maintainer of a covering address space object, instead of the maintainer of the object itself. This means the "mnt-lower:" of an allocation or the "mnt-by:" of a provider independent (PI) or anycast assignment or legacy under contract, each have the authority to force delete any more specific or related object. There is no impact on force delete. If an assignment exists with the same prefix as an allocation, only the allocation is checked for authentication during force delete. An assignment with the same prefix as an allocation can be deleted using force delete. Ref. https://apps.db.ripe.net/docs/Authorisation/Force-Delete-Functionality/ Force deleting route objects ---------------------------- If this route object has a maintainer that you do not have the credentials for, it can block you from creating a new route(6) object. In this case you, as the resource holder, can simply delete the blocking route(6) object. When creating a route object, if an existing exactly matching or less specific route object has a maintainer that you do not have credentials for, then it can block you from creating the route object. In this case you, as the resource holder, can simply delete the blocking route(6) object. The force delete functionality will look for an exact matching, encompassing or less specific inetnum object that is co-maintained by the RIPE NCC in the hierarchy of the object that is to be deleted. If the exactly matching or less specific inetnum is an allocation and assignment with the same prefix, then (both) the assignment is checked first followed by the allocation. Ref. https://apps.db.ripe.net/docs/Authorisation/Protection-of-Route-Object-Spac…

4 5

Revised Agenda RIPE87 - Database Working Group
by William Sylvester 28 Nov '23

28 Nov '23

All, Here is the proposed agenda for the Database Working Group for RIPE87. If you would like to present, have questions, comments, or requested changes please let the chairs know. Best regards, William, David, & Denis DB-WG Chairs A. Introduction - 10 min * Welcome * Scribe * Agenda B. NWI Update – 10 mins David Tatlisu, Co-Chair C. Operational Update – 15 mins Edward Shryane, RIPE NCC C. Geolocation within RDAP – 15 mins Mark Kosters, ARIN D. NWI-4 Tuple Impact Analysis – 10 mins Edward Shryane, RIPE NCC Z. AOB (open discussion)

1 0

The RIPE Database Web Application is now Open Source
by Edward Shryane 23 Nov '23

23 Nov '23

Dear colleagues, The RIPE Database Web Application is now open source and available on GitHub: https://github.com/RIPE-NCC/db-web-ui This is the software running the RIPE database website: https://apps.db.ripe.net/ The software is provided under the BSD license (the same as for Whois): https://github.com/RIPE-NCC/db-web-ui/blob/master/LICENSE.md We intend to publish all improvements made to the website in this repository. If you find a bug, or want to request a technical improvement please open an issue via https://github.com/RIPE-NCC/db-web-ui/issues If you have a functional issue with the website, please continue to contact support (via the left-hand menu) or email ripe-dbm(a)ripe.net. Regards Ed Shryane RIPE NCC

1 0

Agenda RIPE87 - Database Working Group
by William Sylvester 20 Nov '23

20 Nov '23

All, Here is the proposed agenda for the Database Working Group for RIPE87. If you would like to present, have questions, comments, or requested changes please let the chairs know. Best regards, William, David, & Denis DB-WG Chairs A. Introduction - 10 min * Welcome * Scribe * Agenda B. Operational Update– 15 mins Edward Shryane, RIPE NCC C. Geolocation within RDAP– 15 mins Mark Kosters, ARIN D. NWI-4 Tuple Impact Analysis– 10 mins Edward Shryane, RIPE NCC Z. AOB (open discussion)

1 0

geolocation purpose
by denis walker 03 Nov '23

03 Nov '23

Colleagues The discussion and agreement of the text for a new purpose for the RIPE Database covering geolocation was done by the RIPE NCC and it's executive board. They refused to publish the text in advance, didn't announce that the board had agreed to it or that the T&C document has been updated. I still don't understand why this had to be done so secretly. Anyway this is the text of the new purpose from the T&C, https://www.ripe.net/manage-ips-and-asns/db/support/documentation/terms "Facilitating network operations by publishing geolocation information about the usage of Internet number resources (including country, region, city of the party to which the resources have been distributed by the Registrants)" Does this cover what you all wanted from the new "geofeed:" attribute? "Facilitating network operations" suggests this information is only for use by network operators in the management of their networks. Any use of this geolocation information by the wider public or other geolocation information providers is therefore not covered by this purpose. AFAIK this change to the T&C has not yet been announced by the RIPE NCC as required by Article 8.2. So the new purpose is not yet in effect. cheers denis co-chair DB-WG

3 2

Using the RIPE Database for IPAM
by denis walker 04 Oct '23

04 Oct '23

Colleagues Do you use the RIPE Database as your IPAM solution? If so, read on.... The RIPE NCC has done a brief review of the recommendations made by the DB Task Force (DBTF). You can find it here: https://www.ripe.net/participate/ripe/tf/rdb-requirements-tf/initial-analys… There is one point in the summary that I disagree with: One recommendation can be implemented immediately by the RIPE NCC: - 3. Using the RIPE Database as an IPAM solution The recommendation from the DBTF was: "The task force recommends limiting and discouraging the use of the RIPE Database as an enterprise IPAM solution." This has not yet been discussed by the community and there is no consensus on this recommendation. So the RIPE NCC should not be doing anything to implement this recommendation. There are a number of points relating to this. The first BoF held by the DBTF had 46 participants. They had some polls about IPAM: Where do you hold your canonical IP address assignment details? (13 answers) Internal database or IPAM system (5 - 38%) RIPE Database (5 - 38%) Internal database for detailed assignments and RIPE database for allocations (3 - 23%) What should we do about the IPAM use of the RIPE Database? (12 answers) We should discourage it (3-25%) We should tolerate it but provide restrictions and best practices (5-42%) We should promote it and facilitate it (4-33%) During the BoF some comments were also made on this: Shane's comment "IPAM seems to be an important thing" Nathalie said about 50% of people attending training courses over a number of years said they use the RIPE DB for IPAM. Also trainees said open source IPAM solutions often pull data out of the RIPE DB but have no functionality to push data to the RIPE DB. During the presentation by the DBTF in the DB-WG session at RIPE 83 Shane said (on IPAM): "if you had very light requirements for IPAM then maybe it is OK, but it's not really what it's for" "unable to find out if this [response] was based on misunderstanding of the question" "we think there are probably good ways to go forward with that, for example making hooks so that external IPAM solutions can hook into it...that makes sense, there are certainly many commercial vendors that make IPAM products who would be happy to use hooks, there are many ways that third parties could use this but probably not a requirement" So it looks like there are very mixed views on the use of the RIPE Database for IPAM. Some people also expressed surprise that so many people use the database for IPAM. It is clear that we don't actually understand who, how and why people use the database for IPAM. We need a better understanding of the how and why and the number of users doing this before we can make any decision about discouraging or encouraging this use of the database. Even to consider if IPAM should be defined as a purpose of the database. As we are talking about it, maybe now is the time to start this discussion. If you use the RIPE Database as your IPAM solution maybe you can offer some more details. cheers denis co-chair DB-WG

5 5

Question about the status of the implementation of NWI-4
by Erik Bais 29 Sep '23

29 Sep '23

Hi, I was wondering how long it is going to take, to finish the implementation of NWI-4. And what the current status is on the topic. Regards, Erik Bais

2 2

RIPE Database Quarterly Planning Q4 2023
by Edward Shryane 29 Sep '23

29 Sep '23

Dear colleagues, The next quarterly plans for the RIPE Database are now available on our website: https://www.ripe.net/manage-ips-and-asns/db/quarterly-planning In this quarter, we will continue to work on publishing the source code to our web application and move the RIPE Database documentation to a new interface. By sharing these plans with you, we want to inform you of the ongoing developments on the RIPE Database and get your feedback on other work items you would like us to focus on. If you have comments or questions around this, we hope you'll let us know on the list or you can always contact the DB team directly at ripe-dbm(a)ripe.net . Regards Ed Shryane RIPE NCC

1 0