Implementation plan and dates for NWI5 - Out-of-region ROUTE(6) objects and removal of ASN authorisation for ROUTE(6) object creation.
Dear colleagues, In February 2018, the RIPE Database Working Group reached consensus on NWI-5 - creating out of region ROUTE(6) objects in the RIPE Database. We will soon be making changes to the way out-of-region objects are handled in the RIPE Database. It's important that database users are aware of these changes and what they mean: 1. The RIPE Routing Registry will no longer support the creation of out-of-region ROUTE(6) and AUT-NUM objects 2. Existing out-of-region objects will have their "source:” attribute changed to "RIPE-NONAUTH” 3. The creation of ROUTE(6) objects will no longer need authorisation from the ASN holder We will implement these changes in the Release Candidate environment on Thursday, 2 August and go to full production on Tuesday, 4 September. Our implementation plan can be found at: https://www.ripe.net/manage-ips-and-asns/db/impact-analysis-for-nwi-5-implem... <https://www.ripe.net/manage-ips-and-asns/db/impact-analysis-for-nwi-5-implementation> The RIPE Database Working Group chairs also wrote a RIPE Labs article that describes the background of NWI-5: https://labs.ripe.net/Members/denis/out-of-region-route-6-and-aut-num-object... <https://labs.ripe.net/Members/denis/out-of-region-route-6-and-aut-num-objects-in-the-ripe-database> <https://labs.ripe.net/Members/denis/out-of-region-route-6-and-aut-num-object... <https://labs.ripe.net/Members/denis/out-of-region-route-6-and-aut-num-objects-in-the-ripe-database>> Please contact ,ripe-dbm@ripe.net <mailto:ripe-dbm@ripe.net>. if you still have questions after reading the implementation plan. Best regards, Nathalie Trenaman Product Manager RIPE NCC
Dear Nathalie, On Thu, Jul 19, 2018 at 03:18:37PM +0200, Nathalie Trenaman via db-wg wrote:
We will implement these changes in the Release Candidate environment on Thursday, 2 August and go to full production on Tuesday, 4 September.
Our implementation plan can be found at: https://www.ripe.net/manage-ips-and-asns/db/impact-analysis-for-nwi-5-implem...
In section 5, the following is stated: """For NRTM, we don't plan to separate out the stream by source. Updates from both the RIPE and RIPE-NONAUTH sources will be included. The source is specified in the request (e.g., -g RIPE:3:11012700-LAST), but updates from both RIPE and RIPE-NONAUTH will be returned. The client will need to filter on the source attribute as needed.""" I'd like to ask that this aspect is reconsidered. I'd prefer to mirror the two data sources as two separate data streams. So "-g RIPE:3:11012700-LAST" should only return proper 'RIPE' objects, and additionally I can track RIPE-NONAUTH through the use of "-g RIPE-NONAUTH:3:1102700-LAST". Presenting objects to NRTM clients where the objects have a different source than was requested by the client is undefined behaviour. Therefor I'd prefer Using separate streams for the two separate sources, that way it is business as usual for the NRTM clients. To illustrate further: my current irrd.conf (irrd v3) is as following: job@eng0 ~$ grep ripe /etc/irrd.conf ! RIPE - ripe-dbm@ripe.net irr_database ripe mirror_host 193.0.6.135 4444 irr_database ripe mirror_protocol 3 irr_database ripe mirror-access 27 irr_database ripe filter routing-registry-objects|route6 And on September 4th I'd reconfigure it as: ! RIPE - ripe-dbm@ripe.net irr_database ripe mirror_host 193.0.6.135 4444 irr_database ripe mirror_protocol 3 irr_database ripe mirror-access 27 irr_database ripe filter routing-registry-objects|route6 irr_database ripe-nonauth mirror_host 193.0.6.135 4444 irr_database ripe-nonauth mirror_protocol 3 irr_database ripe-nonauth mirror-access 27 irr_database ripe-nonauth filter routing-registry-objects|route6 Similarly I'd expect that the RIPE and RIPE-NONAUTH sources are provided as separate dumps at https://ftp.ripe.net/ripe/dbase/ (where I'd expect ripe-nonauth.db.gz) and same for https://ftp.ripe.net/ripe/dbase/split/ Kind regards, Job
Dear Job, DB-WG, we plan to return objects from both the RIPE and RIPE-NONAUTH sources for NRTM, as it will be less disruptive when NWI-5 goes into production. - If clients do nothing, they still receive all updates. - If we do not return all updates by default, then out of region object updates will not appear (even though they will occur). - Receiving all updates will require separate queries in separate connections, which need to be merged on the client side. - The default behaviour for Whois queries (on port 43, REST API and Web) will be to return objects from both sources. On the other hand, to separate NRTM streams by source: - The NRTM query does specify the source, this implies updates from only that source. - It is straightforward to support separate sources on the server side. - There are relatively few active NRTM clients that would need to be updated. We also do not plan to separate the database dumps by source. This can be done, but any consumers need to make adjustments or will no longer find out of region objects. I welcome discussion on these points, we will implement as the DB-WG prefers. Regards Ed Shryane RIPE NCC
On 19 Jul 2018, at 15:50, Job Snijders via db-wg <db-wg@ripe.net> wrote:
Dear Nathalie,
On Thu, Jul 19, 2018 at 03:18:37PM +0200, Nathalie Trenaman via db-wg wrote:
We will implement these changes in the Release Candidate environment on Thursday, 2 August and go to full production on Tuesday, 4 September.
Our implementation plan can be found at: https://www.ripe.net/manage-ips-and-asns/db/impact-analysis-for-nwi-5-implem...
In section 5, the following is stated:
"""For NRTM, we don't plan to separate out the stream by source. Updates from both the RIPE and RIPE-NONAUTH sources will be included.
The source is specified in the request (e.g., -g RIPE:3:11012700-LAST), but updates from both RIPE and RIPE-NONAUTH will be returned. The client will need to filter on the source attribute as needed."""
I'd like to ask that this aspect is reconsidered. I'd prefer to mirror the two data sources as two separate data streams. So "-g RIPE:3:11012700-LAST" should only return proper 'RIPE' objects, and additionally I can track RIPE-NONAUTH through the use of "-g RIPE-NONAUTH:3:1102700-LAST".
Presenting objects to NRTM clients where the objects have a different source than was requested by the client is undefined behaviour. Therefor I'd prefer Using separate streams for the two separate sources, that way it is business as usual for the NRTM clients.
To illustrate further: my current irrd.conf (irrd v3) is as following:
job@eng0 ~$ grep ripe /etc/irrd.conf ! RIPE - ripe-dbm@ripe.net irr_database ripe mirror_host 193.0.6.135 4444 irr_database ripe mirror_protocol 3 irr_database ripe mirror-access 27 irr_database ripe filter routing-registry-objects|route6
And on September 4th I'd reconfigure it as:
! RIPE - ripe-dbm@ripe.net irr_database ripe mirror_host 193.0.6.135 4444 irr_database ripe mirror_protocol 3 irr_database ripe mirror-access 27 irr_database ripe filter routing-registry-objects|route6 irr_database ripe-nonauth mirror_host 193.0.6.135 4444 irr_database ripe-nonauth mirror_protocol 3 irr_database ripe-nonauth mirror-access 27 irr_database ripe-nonauth filter routing-registry-objects|route6
Similarly I'd expect that the RIPE and RIPE-NONAUTH sources are provided as separate dumps at https://ftp.ripe.net/ripe/dbase/ (where I'd expect ripe-nonauth.db.gz) and same for https://ftp.ripe.net/ripe/dbase/split/
Kind regards,
Job
Hi Edward, On Thu, Jul 19, 2018 at 09:27:23PM +0200, Edward Shryane wrote:
we plan to return objects from both the RIPE and RIPE-NONAUTH sources for NRTM, as it will be less disruptive when NWI-5 goes into production.
I am not convinced anyone can authoritatively make that statement. As far as I know the RIPE NCC IRR would be the first to publish multiple sources in a single stream. There is an industry convention related to IRR sources, I see no immediate reason to deviate from this standard.
- If clients do nothing, they still receive all updates.
And perhaps consider them corrupted, because they requested source X and get back source Y.
- If we do not return all updates by default, then out of region object updates will not appear (even though they will occur).
The 20 NRTM clients (of which we probably know most personally) just have to configure a second source and all will be good.
- Receiving all updates will require separate queries in separate connections, which need to be merged on the client side.
The whole point is to not merge them into a single source, so that we can programmatically differentiate between the two sources. If there is merging, it happens at a different point in the pipeline, not at 'ingress' (from the point of view of a mirror).
- The default behaviour for Whois queries (on port 43, REST API and Web) will be to return objects from both sources.
I have no problem with that.
On the other hand, to separate NRTM streams by source:
- The NRTM query does specify the source, this implies updates from only that source.
exactly. This is a foundational assumption in all NRTM software I am aware of.
- It is straightforward to support separate sources on the server side. - There are relatively few active NRTM clients that would need to be updated.
I am happy to hear that separate sources are straight forward to implement. On the IRRd v2, v3 and v4 side of things it is not straight forward to implement.
We also do not plan to separate the database dumps by source. This can be done, but any consumers need to make adjustments or will no longer find out of region objects.
yes, that is the point. We make a change, and people need to update if they rely on this. By splitting the NRTM and database dumps, most implementations only need to add an additional source, rather than reprogram their client software to understand that what used to be a single file containing a single source, now contains multiple sources.
I welcome discussion on these points, we will implement as the DB-WG prefers.
Because of NWI-5 the RIPE IRR is splitting into two, and we should plan accordingly. To me that means separate database dumps and separate NRTM streams. Kind regards, Job
I would just like to concur with Job's comments and that including both sources in the same NRTM stream is not something that existing mirroring implementations expect to encounter. Separate mirror streams and database dumps will be consistent with existing practice and produce expected results in response to user queries. Note that several IRRd "!" query commands and the RIPE "-K" flag do no include the source attribute which means the client software is unable to distinguish sources on it's end when using these queries. -Larry Blunk Merit Network On Thu, Jul 19, 2018 at 3:48 PM Job Snijders via db-wg <db-wg@ripe.net> wrote:
Hi Edward,
On Thu, Jul 19, 2018 at 09:27:23PM +0200, Edward Shryane wrote:
we plan to return objects from both the RIPE and RIPE-NONAUTH sources for NRTM, as it will be less disruptive when NWI-5 goes into production.
I am not convinced anyone can authoritatively make that statement. As far as I know the RIPE NCC IRR would be the first to publish multiple sources in a single stream. There is an industry convention related to IRR sources, I see no immediate reason to deviate from this standard.
- If clients do nothing, they still receive all updates.
And perhaps consider them corrupted, because they requested source X and get back source Y.
- If we do not return all updates by default, then out of region object updates will not appear (even though they will occur).
The 20 NRTM clients (of which we probably know most personally) just have to configure a second source and all will be good.
- Receiving all updates will require separate queries in separate connections, which need to be merged on the client side.
The whole point is to not merge them into a single source, so that we can programmatically differentiate between the two sources. If there is merging, it happens at a different point in the pipeline, not at 'ingress' (from the point of view of a mirror).
- The default behaviour for Whois queries (on port 43, REST API and Web) will be to return objects from both sources.
I have no problem with that.
On the other hand, to separate NRTM streams by source:
- The NRTM query does specify the source, this implies updates from only that source.
exactly. This is a foundational assumption in all NRTM software I am aware of.
- It is straightforward to support separate sources on the server side. - There are relatively few active NRTM clients that would need to be updated.
I am happy to hear that separate sources are straight forward to implement. On the IRRd v2, v3 and v4 side of things it is not straight forward to implement.
We also do not plan to separate the database dumps by source. This can be done, but any consumers need to make adjustments or will no longer find out of region objects.
yes, that is the point. We make a change, and people need to update if they rely on this. By splitting the NRTM and database dumps, most implementations only need to add an additional source, rather than reprogram their client software to understand that what used to be a single file containing a single source, now contains multiple sources.
I welcome discussion on these points, we will implement as the DB-WG prefers.
Because of NWI-5 the RIPE IRR is splitting into two, and we should plan accordingly. To me that means separate database dumps and separate NRTM streams.
Kind regards,
Job
-- *Larry Blunk* Senior Network Engineer [image: Merit.edu] <https://www.merit.edu> ljb@merit.edu | 734.527.5725 p | 734.395.4363 c | www.merit.edu 1000 Oakbrook Drive, Suite 200 | Ann Arbor, MI 48104 [image: Like Merit on Facebook] <http://www.facebook.com/meritnetwork> [image: Follow Merit on Twitter] <http://www.twitter.com/meritnetwork> [image: Connect with Merit on LinkedIn] <http://www.linkedin.com/company/merit-network> *Upcoming Courses and Events* <https://www.merit.edu/professional-development/>
On Thu, Jul 19, 2018 at 07:40:33PM +0000, Job Snijders via db-wg wrote: Dear All,
We also do not plan to separate the database dumps by source. This can be done, but any consumers need to make adjustments or will no longer find out of region objects.
yes, that is the point. We make a change, and people need to update if they rely on this. By splitting the NRTM and database dumps, most implementations only need to add an additional source, rather than reprogram their client software to understand that what used to be a single file containing a single source, now contains multiple sources.
I welcome discussion on these points, we will implement as the DB-WG prefers.
Because of NWI-5 the RIPE IRR is splitting into two, and we should plan accordingly. To me that means separate database dumps and separate NRTM streams.
Although I do not run NRTM, I fully support Job's point of view. It is consistent with current practice. Piotr -- Piotr Strzyżewski Silesian University of Technology, Computer Centre Gliwice, Poland
Dear Colleagues, thank you all for your feedback. We will implement the following for NWI-5, and also deploy to production on 4th September: - Separate NRTM streams by source. Updates to out of region objects (route, route6, aut-num) will appear on the RIPE-NONAUTH stream only. In order to be notified of all changes, NRTM clients will need to query both streams. After NWI-5 is deployed to production in September, the updates to change the source on out of region objects will appear on the RIPE-NONAUTH stream. - Move out of region objects into a separate database dump file, and split files. We will deploy these changes separately to the Release Candidate environment, and will notify the current NRTM users individually. Regards Ed Shryane RIPE NCC
On 23 Jul 2018, at 11:46, Piotr Strzyzewski <Piotr.Strzyzewski@polsl.pl> wrote:
On Thu, Jul 19, 2018 at 07:40:33PM +0000, Job Snijders via db-wg wrote:
Dear All,
We also do not plan to separate the database dumps by source. This can be done, but any consumers need to make adjustments or will no longer find out of region objects.
yes, that is the point. We make a change, and people need to update if they rely on this. By splitting the NRTM and database dumps, most implementations only need to add an additional source, rather than reprogram their client software to understand that what used to be a single file containing a single source, now contains multiple sources.
I welcome discussion on these points, we will implement as the DB-WG prefers.
Because of NWI-5 the RIPE IRR is splitting into two, and we should plan accordingly. To me that means separate database dumps and separate NRTM streams.
Although I do not run NRTM, I fully support Job's point of view. It is consistent with current practice.
Piotr
-- Piotr Strzyżewski Silesian University of Technology, Computer Centre Gliwice, Poland
Thanks Ed I think this is a sensible plan. cheersdenisco-chair DB-WG From: Edward Shryane via db-wg <db-wg@ripe.net> To: db-wg <db-wg@ripe.net> Cc: Piotr Strzyzewski <Piotr.Strzyzewski@polsl.pl>; ljb@merit.edu; Job Snijders <job@ntt.net> Sent: Tuesday, 24 July 2018, 12:18 Subject: Re: [db-wg] Implementation plan and dates for NWI5 - Out-of-region ROUTE(6) objects and removal of ASN authorisation for ROUTE(6) object creation. Dear Colleagues, thank you all for your feedback. We will implement the following for NWI-5, and also deploy to production on 4th September: - Separate NRTM streams by source. Updates to out of region objects (route, route6, aut-num) will appear on the RIPE-NONAUTH stream only. In order to be notified of all changes, NRTM clients will need to query both streams. After NWI-5 is deployed to production in September, the updates to change the source on out of region objects will appear on the RIPE-NONAUTH stream. - Move out of region objects into a separate database dump file, and split files. We will deploy these changes separately to the Release Candidate environment, and will notify the current NRTM users individually. Regards Ed Shryane RIPE NCC
On 23 Jul 2018, at 11:46, Piotr Strzyzewski <Piotr.Strzyzewski@polsl.pl> wrote:
On Thu, Jul 19, 2018 at 07:40:33PM +0000, Job Snijders via db-wg wrote:
Dear All,
We also do not plan to separate the database dumps by source. This can be done, but any consumers need to make adjustments or will no longer find out of region objects.
yes, that is the point. We make a change, and people need to update if they rely on this. By splitting the NRTM and database dumps, most implementations only need to add an additional source, rather than reprogram their client software to understand that what used to be a single file containing a single source, now contains multiple sources.
I welcome discussion on these points, we will implement as the DB-WG prefers.
Because of NWI-5 the RIPE IRR is splitting into two, and we should plan accordingly. To me that means separate database dumps and separate NRTM streams.
Although I do not run NRTM, I fully support Job's point of view. It is consistent with current practice.
Piotr
-- Piotr Strzyżewski Silesian University of Technology, Computer Centre Gliwice, Poland
participants (6)
-
denis walker -
Edward Shryane -
Job Snijders -
Larry Blunk -
Nathalie Trenaman -
Piotr Strzyzewski