Job, Ben So while I 100% agree with the problem statement here. (if not clear by my email-address, I do work for Amazon, and me in particular has been fighting both RIPE and the adversaries about this illicit Entry) Ideally I don't want to maintain my data in more than one place (right now, that place is RADB). With this suggestion and the hieararchical "protected namespace" this would, if I understand this correctly, not be a problem? I have no problem in moving over to use AS16509:AS-AMAZON as my official AS-SET going forward, if this can't be recreated elsewhere and therefor create these problems, which has the potential of being fairly large if an ISP generates empty prefix lists for us. Another alternative we thought about was to be able to have "invisible" objects or "protected objects" or whatever you want to call them, so we can create for example AS-AMAZON in every IRR-source but have it be invisible to avoid maintaining it, to protect ourselves from automatic prefix-list generators that would use the RIPE-object (empty) infront of the RADB-object (millions of routes). We touch our IRR-data basically daily (RPKI data every minute...) so scaling out here is not very nice. And on this topic, we tried to get this object deleted through regular support, but it was not possible since there is technically nothing that stops anyone from registering anything and calling it whatever. So there is really no jurisdiction from a database point of view to address it. However one thing that IS interesting is that the name itself, is a registered and protected trademark in essentially every single country in the world, so that’s an avenue one could argue should carry some weight in database-cleanliness? Either way, before we torch IRR to the ground there is some incremental steps we could take to make it better it seems like All for. /FK PS: Perhaps RIPE could charge me 8$/mo for a verified blue checkmark on our as-set objects? __ .DS On 2022-11-14, 19:30, "db-wg on behalf of Ben Cartwright-Cox via db-wg" <db-wg-bounces@ripe.net on behalf of db-wg@ripe.net> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. I also support this proposal. I've assisted some of the mentioned networks in the Original Post to solve the overlapping AS-SETS and it has been a real operational pain. It's clear that the actors that are doing this are motivated either by amusement at causing networks downtime or pain, or as a way to ransom the impacted networks. While those actors could move to another RIR or non-authenticated database to do this, I believe that solving this here would help shift networks to using a more secure by default AS-SET convention, and protect networks that have yet to move. I see two ways out of this problem, RIPE comes up with a policy for getting overlapping AS-SET objects removed from the database (that are causing problems, either accidentally or in these cases deliberately) or deprecate (as discussed in the OP) short AS-SETs. On Mon, Nov 14, 2022 at 6:03 PM Job Snijders via db-wg <db-wg@ripe.net> wrote: > > Dear DB-WG, > > Speaking in individual capacity. > > In RFC 2622 section 5 specifies the naming convention for AS-SET > objects. https://www.rfc-editor.org/rfc/rfc2622#section-5.1 > There basically are two styles: > > * "short" (example: AS-SNIJDERS) > * "hierarchical" (example: AS15562:AS-SNIJDERS) > > Problem statement > ================= > In recent weeks a number of hypergiant cloud providers have faced the > thorny effects of adversarial AS-SET object naming collisions between > IRR databases. > > An example of this phenomenon is the existence of AS-AMAZON in both RADB > and RIPE. According to https://www.peeringdb.com/net/1418 the RADB copy > of the object is the the correct one and populated with a number of > members entries. The RIPE one is empty, and not under control of Amazon. > > The existence of the AS-AMAZON object in the RIPE database might cause > some operators to inadvertently apply empty prefix-filters to EBGP > sessions which in turn causes various problems. > > It seems Amazon has no recourse to get the AS-AMAZON object removed from > the RIPE database; because the existence of that object in the RIPE > database does not violate any policies (as far as I know). But perhaps, > going forward, this community can do a little bit more to help prevent > similar situations from happening to others. > > Solution proposal > ================= > I think the solution is to - GOING FORWARD - disallow creation of new > AS-SET objects which follow the 'short' naming style. > > The advantage of hierarchical naming is that the existing authorization > rules as applied by the RIPE Whois Server database engine do a decent > job of protecting/separating namespaces. 'Grandfathering' existing > short-named objects ensures that implementation of this solution > proposal causes minimal (if any) disruption to existing workflows. > > The RIPE database engine blocking creation of short-named AS-SETs might > help nudge the industry towards making hierarchical naming the norm. > > Related work > ============ > Related work throughout the registry industry: IRRd version 4 forces new > AS-SET objects to be structured hierarchically: > https://github.com/irrdnet/irrd/issues/408 > > Kind regards, > > Job > > -- > > To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://lists.ripe.net/mailman/listinfo/db-wg -- To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://lists.ripe.net/mailman/listinfo/db-wg

Hi Yang On Wed, 16 Nov 2022 at 11:06, Yang Yu <yang.yu.list@gmail.com> wrote:

I support this proposal.

...

It seems Amazon has no recourse to get the AS-AMAZON object removed from the RIPE database; because the existence of that object in the RIPE database does not violate any policies (as far as I know).

Also ran into this issue and would like to see policy support to handle this kind of abuse.

On Mon, Nov 14, 2022 at 3:08 PM denis walker via db-wg <db-wg@ripe.net> wrote:

...

Interesting timing. I was about to make the same suggestion but for a different reason...accountability. Currently ANYONE can create a set object in the RIPE Database. You can be completely anonymous, not a member or LIR, hold no resources. All you need to do is create a ROLE, MNTNER and set object.

Anyone with an email can make a RIPE account and start creating objects in RIPE database. In other registries there are usually some safeguards on user / mntner object creation. Limiting database updates to only accounts associated with LIR sounds reasonable.

It is not quite as open as you think. Someone with nothing at all in the database can create a ROLE/MNTNER pair of objects. If these are not referenced by any operational data within 90 days they will be automatically deleted. So if you don't hold any internet resources, or more specifics, there is not much you can do in the database. The one exception to this is with the set objects. Anyone can create any of the set object types and reference their ROLE and MNTNER objects. This group of objects is then protected from any automated cleanup. The set object can be almost empty with only the mandatory, basic attributes. This does not only apply to the AS-SET but any of the set object types. I did ask a question and although the answer is implied from some of the comments made, from a database perspective, if you want the syntax rules changed someone needs to positively confirm or deny my question. Is there any legitimate reason for anyone to create a (short named) set object who does not hold an ASN resource? Only if the answer is no, can we enforce hierarchical naming. cheers denis co-chair DB-WG

Yang

I support the proposal of moving away from the short naming. And I also agree with Gert on his remark .. about the pain for using RADB __ Regards, Erik Bais On 14/11/2022, 22:36, "db-wg on behalf of Gert Doering via db-wg" <db-wg-bounces@ripe.net on behalf of db-wg@ripe.net> wrote: Hi, On Mon, Nov 14, 2022 at 05:41:16PM +0000, Job Snijders via db-wg wrote: > Solution proposal > ================= > I think the solution is to - GOING FORWARD - disallow creation of new > AS-SET objects which follow the 'short' naming style. Support. (Though I think that ASes that use RADB as their primary documentation store deserve some amount of pain, but that's a different crusade) gert -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard, Michael Emmer Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279

Hi Denis, While I agree with your goal of trying to clean up the DB more generally, I think all of the more general issues have to be left for another time as I don't think that is appropriate for a NWI and should be done through the PDP. I think we should purely tackle the issue of as-sets for now as I think there is enough unanimity in it and it is small enough in scope that it makes sense to do it through the NWI process. (I think the main person raising any objections was myself) I have a question though, do we know if all current hierarchical as-sets were authorized by the ASN holder when they were created? I wrote the above question and then I looked into it, but as it is a bit off-topic and some other things that popped up, I decided to put it in a separate thread. The TL;DR of it though is that, I am pretty sure that the answer is no. There are at least some that weren't authorized. Another issue is how to deal with the short as-sets that already exist and are potentially causing issues. Just as an example, I can see that there is currently an AS-AMAZON in the RIPE DB that I assume was not authorized by Amazon and looks to not have any good reason to exist (no members). I think having a way for trademark holders and well-known entities to deal with dubious as-sets would be good. Maybe just during a limited time period to prevent any future uncertainty. -Cynthia On Sat, Nov 19, 2022 at 4:01 PM denis walker via db-wg <db-wg@ripe.net> wrote:

Colleagues

I know it is a weekend but there does seem to be some urgency on this matter. The chairs have been following this discussion and have a couple of questions before we move on.

The chairs believe we can consider this to be a feature request for the RIPE Database and handle it through the Numbered Work Item (NWI) mechanism. This will address the matter much faster, unless anyone believes it should be based on a formal policy.

To assist the RIPE NCC with their impact analysis can we be clear on how you want to change the syntax. My understanding is you want rules along these lines:

-An AS-SET name must be hierarchical -There must be at least one colon (:) character in the name -The first element of the name must be an ASN -The second element of the name must be an AS-SET name starting with 'AS-' -Any further elements can be either ASNs or AS-SET names -Any other existing syntax rules that don't conflict with this change -These rules to only apply to creating new AS-SET objects -Existing non-hierarchical AS-SET objects can still be updated

This discussion has focused on the AS-SET object and the authorisation problems they can cause. Should we make this change to all set object types? The benefits of doing this include:

-Consistent rules applied to all set object types -Accountability for all (new) set objects in the database -Close the one exception where anyone can create a cluster of objects in the database and link them to a (operationally empty) set object to protect them from automated deletion -Objects can then only be created in the database by holders of Internet resources or more specifics

If we apply it to all set objects then my original question still stands, is there any legitimate reason for someone to create any set object if they don't hold an ASN resource?

If we can address these issues then the chairs can move this process on quickly at the start of next week.

cheers denis co-chair DB-WG

--

To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://lists.ripe.net/mailman/listinfo/db-wg

Job Snijders via db-wg wrote on 20/11/2022 13:07:

I'd argue that the rules for what constitute valid hierarchical names should not be changed; so the second component of the name doesn't need to start with 'AS-'.

you mean "does need to start with 'AS-'"? I don't see how rfc2622 allows naked terms, or how that would allow rpsl consumers to determine what type of set a specific named item was. Nick

Hi Denis, Colleagues, Following is the impact analysis of NWI-19. Low impact on Whois. * The DB team will disallow the creation of AS-SET objects which follow the 'short' naming style (i.e. starting with 'AS-' and without a colon). * Only AS-SET objects which follow the 'hierarchical' naming style (i.e. including one or more colons) can be created. * Existing AS-SET objects are not affected and can be updated or deleted as before. However, if an existing AS-SET object with short naming style is deleted, it cannot be re-created with the same name. No impact on the LIR Portal or internal registry software. No impact on the RPKI service. Client software needs to be checked so that AS-SET objects with 'short' naming style will not be created in the RIPE database. It will still be possible to create AS-SET objects with a 'short' naming style in other RIR databases and other IRR databases such as RADB. Clients may not be able to use the same name in the RIPE database as in other databases. Implementing and deploying this change will take some time. In the meantime, there may be an increase in AS-SET object creation with a 'short' naming style. Regards Ed Shryane RIPE NCC

On 21 Nov 2022, at 15:00, Edward Shryane via db-wg <db-wg@ripe.net> wrote:

Hi Denis, Colleagues,

...

On 21 Nov 2022, at 13:54, denis walker via db-wg <db-wg@ripe.net> wrote:

Colleagues

The chairs discussed this issue over the weekend and agreed that we see a consensus to change the syntax rules for AS-SET object names in the RIPE Database. We also agreed that we believe this feature request can be managed through the Numbered Work Item process. We therefore ask the RIPE NCC to document 'NWI-19 Change to AS-SET object naming rules'.

I've updated the NWI page: https://www.ripe.net/manage-ips-and-asns/db/numbered-work-items

This change will go live shortly after an internal review.

...

The chairs would like the RIPE NCC to prepare an impact analysis.

The DB team will prepare an impact analysis now.

Regards Ed Shryane RIPE NCC

--

To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://lists.ripe.net/mailman/listinfo/db-wg

Hi, On Tue, Nov 22, 2022 at 08:00:47PM +0100, denis walker via db-wg wrote:

Another suggestion. There are 1361 short named AS-SET objects that don't have any 'members' or 'mbrs-by-ref' attributes. In other words they are operationally empty objects. (This includes AS-AMAZON.) We could introduce an automated cleanup process similar to the way we remove unreferenced PERSON and ROLE objects. If an AS-SET object remains operationally empty for 90 days it will be automatically deleted. This would include hierarchical objects. The hierarchical objects can easily be recreated by the ASN maintainers at any time if they are needed later. This gets around the problem of who has the authority to remove rogue objects. It becomes a database cleanup operation. Any thoughts?

I would support that. Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard, Michael Emmer Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279

On Tue, Nov 22, 2022 at 07:11:17PM +0000, Nick Hilliard via db-wg wrote:

Careful with this, e.g. AS-NULL. There are some situations where referencing an empty set can be useful in RPSL.

For 'AS-NULL' back history see this thread: https://www.ripe.net/ripe/mail/archives/db-wg/2014-December/thread.html#4461 Kind regards, Job

Colleagues Any thoughts on these 'RIPE-NONAUTH' objects? On Tue, 22 Nov 2022 at 21:17, denis walker <ripedenis@gmail.com> wrote:

Hi Nick

On Tue, 22 Nov 2022 at 20:11, Nick Hilliard <nick@foobar.org> wrote:

...

denis walker via db-wg wrote on 22/11/2022 19:00:

...

Any thoughts on this? There are 2128 AUT-NUM objects with source RIPE-NONAUTH. Do we want these to be able to authorise the creation of hierarchical AS-SET objects when we don't know who maintains the AUT-NUM objects?

I don't see a particular reason to prevent holders of existing NON-AUTH ASNs from defining a hierarchical AS-SET object associated with their ASN. The as-set object would be no more or less authoritative than the aut-num object.

Then another option could be to only allow such objects to also have the source NONAUTH

...

These ASNs have 'source: RIPE-NONAUTH' because we don't know who created the AUT-NUM objects or who now maintains them in the RIPE Database. They were created when anyone could create an AUT-NUM object in the RIPE Database for non RIPE issued ASNs. Authorisation was bypassed to allow them to be created. The 'NONAUTH' tag makes it clear they are not authoritative. Consumers of this data can then make an informed decision about whether or not they trust these objects. If we allow these objects to authorise hierarchical AS-SET objects with 'source: RIPE' we have in effect turned non authoritative data back into authoritative data. If we give the related AS-SET objects 'source: RIPE-NONAUTH' we make it clear that these objects are also not authoritative. Consumers of the data should make their own informed decisions about the content of these AS-SET objects. cheers denis co-chair DB-WG

I agree with Nick. However there are currently as-set objects in RIPE based on aut-num objects in RIPE-NONAUTH. I think it might be worth considering if these should be cleaned up. I posted about this about a week ago in a separate thread on db-wg. -Cynthia On Tue, Nov 29, 2022 at 7:30 PM Nick Hilliard via db-wg <db-wg@ripe.net> wrote:

denis walker wrote on 29/11/2022 13:00:

...

If we allow these objects to authorise hierarchical AS-SET objects with 'source: RIPE' we have in effect turned non authoritative data back into authoritative data. If we give the related AS-SET objects 'source: RIPE-NONAUTH' we make it clear that these objects are also not authoritative. Consumers of the data should make their own informed decisions about the content of these AS-SET objects. if ASnnnnn has source: RIPE-NONAUTH, then the corresponding as-set would be ASnnnnn:FOOBAR, so this would also need to be RIPE-NONAUTH.

I don't see an issue with RIPE-NONAUTH aut-nums being able to create scoped RIPE-NONAUTH as-sets. This also gives a clear route for garbage collection: if the aut-num becomes invalid, then the corresponding as-sets also become invalid.

IIRC it's not possible to create new aut-num objects with source: RIPE-NONAUTH?

Nick

--

To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://lists.ripe.net/mailman/listinfo/db-wg

On Tue, 29 Nov 2022 at 20:44, Nick Hilliard <nick@foobar.org> wrote:

Cynthia Revström wrote on 29/11/2022 18:56:

...

I agree with Nick. However there are currently as-set objects in RIPE based on aut-num objects in RIPE-NONAUTH. I think it might be worth considering if these should be cleaned up. I posted about this about a week ago in a separate thread on db-wg.

right, yes, you're correct. I've included a list below.

The RIPE NCC can't stand over the authenticity of these objects because the AS in question isn't a RIPE-authenticated ASN. So RIPE-NONAUTH would be an appropriate place for them.

Changing this should not be service affecting, because the AS in question is already RIPE-NONAUTH. Having said that, "should not" is not the same as "will not".

Some of these objects are stale.

Some of them are legacy resources, but can be subject to the same approach as defined in ripe-731.

Do we need a policy change to move these, e.g. similar to ripe-731, or simply including them in the scope of ripe-731?

The policy ripe-731 is all about deleting objects that conflict with RPKI. So I don't see this issue being a part of the same scope. However, RIPE-NONAUTH is considered to be a separate IRR registry, even if the objects are physically in the same database. As a community we can argue that it is logical that if an ASN in the registry RIPE-NONAUTH authorises the creation of an AS-SET object, that new object must be located in the same registry. So putting these previously created AS-SET objects in the RIPE registry was a bug. We can therefore fix this bug and move these objects to the correct registry. I don't see that needing a policy. We can add it to NWI-19. Thoughts??? cheers denis co-chair DB-WG

Nick

Niall O'Reilly via db-wg wrote on 30/11/2022 16:15:

If we indeed don't need a policy, I think it would be cleaner to make this a new NWI, as adding to an NWI after implementation work has begun seems, IMESHO, likely to cause confusion.

agreed. In terms of breakage, we're carrying a 25yo basket of eggs. Any shift in position will is likely to cause a crack here or there. It's unlikely that any shift in position will cause more breakage than introducing RIPE-NONAUTH in the first place. Nick

On 22 Nov 2022, at 19:11, Nick Hilliard via db-wg wrote:

Careful with this, e.g. AS-NULL. There are some situations where referencing an empty set can be useful in RPSL.

I'm not sure whether Nick meant this as a single exception, or rather as an example of a category of useful empty sets. I can well imagine that most "operationally empty objects" (as Denis put it) are either useless or troublesome. As Nick says, "Careful with this." Niall

On 23 Nov 2022, at 16:56, denis walker wrote:

One empty set is the same as any other empty set. We only need one clearly defined and easily recognisable empty set and we have that, AS-NULL. If I create an empty set, AS-WALKER it doesn't do anything that AS-NULL can't do.

Sure. What I have in mind is AS-NIALLSPECIAL, which I populate with a list of AS numbers which I want to advertise to let others know that these are to be handled in some special way, unlike those in AS-NIALLNORMAL. According to operational circumstances, there might be periods, even long ones, with nothing special going on; AS-NIALLSPECIAL would then be empty, but only for as long as this continued to be the case. Something I don't know is whether this hypothetical scenario is operationally realistic or simply a product of my certainly ill-informed, and perhaps over-active, imagination. If it were the latter, then a special case just for AS-NULL would be sufficient. Otherwise, inferring equivalence to AS-NULL of any set which happens to have been empty for some specified period seems unsafe. I'll be happy to learn that I'm imagining trouble that can never arise. Niall

Based on what has been said in this thread so far, I cannot support automatic clean-up of AS-SETs even if they are empty. There is simply way too little to be gained compared to the issues it could cause. Also if there is someone who maliciously created a short AS-SET, they could simply just add an ASN into it to exclude it from automatic clean-up. It might be complicated but I really think the better way to go about it is to handle each of the individual cases in which this is an issue as I suspect that there are not many and they are probably pretty clear cases. I know of AS-AMAZON but are we aware of any other potentially problematic AS-SETs in the RIPE DB currently? Also I don't even know if it is currently causing issues for amazon, but maybe Fredrik could answer that? -Cynthia On Thu, Nov 24, 2022 at 4:14 PM Nick Hilliard via db-wg <db-wg@ripe.net> wrote:

Niall O'Reilly via db-wg wrote on 23/11/2022 18:17:

...

What I have in mind is AS-NIALLSPECIAL, which I populate with a list of AS numbers which I want to advertise to let others know that these are to be handled in some special way, unlike those in AS-NIALLNORMAL.

According to operational circumstances, there might be periods, even long ones, with nothing special going on; AS-NIALLSPECIAL would then be empty, but only for as long as this continued to be the case.

this ^^^ is one of the failure modes.

It would not be safe to assume that empty as-sets named in RPSL policies are unused. It would be less unsafe to assume that unreferenced as-sets are unused. A reasonable middle ground might be - after the proposed new unqualified as-set block has been implemented - to check out all unreferenced as-sets older than a specific period of time and flag those for deletion.

It would also be worthwhile inspecting rpsl in other IRRDBs to see if there are any references. The reason for this is that lots of people use tools like bgpq3 / peval / etc, and query aggregate IRRDBs, e.g. RADB, NTT, etc.

So if you have RPSL in another IRRDB and this references an empty as-set in the RIPE IRRDB, that definition may be picked up in preference to other as-set definitions. I.e. by removing an as-set definition in the RIPE IRRDB, it could unexpectedly influence routing policies elsewhere.

These are corner cases, but they should show why some care will be needed to figure out whether deleting these objects is a good idea.

Nick

--

To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://lists.ripe.net/mailman/listinfo/db-wg

Hi Cynthia "Problems" is a wide definition. We know that some ISPs have built filters using the malicious AS-AMAZON record in RIPE DB, instead of the real record in RADB. This generates an empty prefix-list, and therefor completely filtered the inbound routes from us to that ISP. However, luckily in all of our cases, there has been enough capacity on upstream links and therefor all traffic rerouted without end-user really noticing, however the path is probably not the most optimal, diminishing the value of peering (not good). And since this is affecting our inbound-traffic, we have to spend significant time vetting all of our interconnects currently to look for suspicious drops in inbound-traffic, based upon heuristic models. The blatant trolling in the RIPE db has forced us to craft alarms based upon this peculiar behavior, where inbound flows moves around without any sensible explanation (such as linkdown, port-errors, etc etc). We have hundreds of thousands of BGP-sessions in the world so this is no small task. I would like to point out though, that despite all of this, I think the other affected companies, have had bigger outages, and I guess we are just waiting for when that day comes to us, which is also why we are fairly invested in fixing this. In almost all cases "we" (as in the routing security community) have been able to found the original creators of said objects (seems to all come from a tunnelbroker discord server) and pulling the historic chatlogs when it was created, it was all for fun & games, and most creators decided to just delete them when people from the community reached out and explained that the risk involved in this behavior, is probably larger than what most people believe, given it's also potentially squatting on trademarks. /FK On 2022-11-25, 00:47, "Cynthia Revström" <me@cynthia.re> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. I have just been informed that AS-AMAZON in the RIPE DB is indeed causing issues for Amazon currently, so please disregard that question. -Cynthia On Fri, Nov 25, 2022 at 12:20 AM Cynthia Revström <me@cynthia.re> wrote: > > Based on what has been said in this thread so far, I cannot support > automatic clean-up of AS-SETs even if they are empty. > There is simply way too little to be gained compared to the issues it > could cause. > Also if there is someone who maliciously created a short AS-SET, they > could simply just add an ASN into it to exclude it from automatic > clean-up. > > It might be complicated but I really think the better way to go about > it is to handle each of the individual cases in which this is an issue > as I suspect that there are not many and they are probably pretty > clear cases. > I know of AS-AMAZON but are we aware of any other potentially > problematic AS-SETs in the RIPE DB currently? > Also I don't even know if it is currently causing issues for amazon, > but maybe Fredrik could answer that? > > -Cynthia > > On Thu, Nov 24, 2022 at 4:14 PM Nick Hilliard via db-wg <db-wg@ripe.net> wrote: > > > > Niall O'Reilly via db-wg wrote on 23/11/2022 18:17: > > > What I have in mind is AS-NIALLSPECIAL, which I populate with a list of > > > AS numbers which I want to advertise to let others know that these > > > are to be handled in some special way, unlike those in AS-NIALLNORMAL. > > > > > > According to operational circumstances, there might be periods, even > > > long ones, with nothing special going on; AS-NIALLSPECIAL would then be > > > empty, but only for as long as this continued to be the case. > > > > this ^^^ is one of the failure modes. > > > > It would not be safe to assume that empty as-sets named in RPSL policies > > are unused. It would be less unsafe to assume that unreferenced as-sets > > are unused. A reasonable middle ground might be - after the proposed > > new unqualified as-set block has been implemented - to check out all > > unreferenced as-sets older than a specific period of time and flag those > > for deletion. > > > > It would also be worthwhile inspecting rpsl in other IRRDBs to see if > > there are any references. The reason for this is that lots of people > > use tools like bgpq3 / peval / etc, and query aggregate IRRDBs, e.g. > > RADB, NTT, etc. > > > > So if you have RPSL in another IRRDB and this references an empty as-set > > in the RIPE IRRDB, that definition may be picked up in preference to > > other as-set definitions. I.e. by removing an as-set definition in the > > RIPE IRRDB, it could unexpectedly influence routing policies elsewhere. > > > > These are corner cases, but they should show why some care will be > > needed to figure out whether deleting these objects is a good idea. > > > > Nick > > > > -- > > > > To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://lists.ripe.net/mailman/listinfo/db-wg

denis walker wrote on 30/11/2022 14:26:

This is an interesting point. I know nothing about bgpq3 or peval (even though it has been around since the beginning of time). So I just read the documentation on GitHub for them. I never realised this feature existed. What you have described here Nick is in effect a cross registry dependency. A feature that, as far as the RIPE Database is concerned, is undocumented and unsupported. If anyone actually uses this feature we have probably just broken it with NWI-19.

Well, heh. This assumes the feature worked to start with. Cross-registry queries are a fact of life. It is advisable to specify source lists explicitly because otherwise you can end up with all sorts of junk in a query (e.g. from RADB which mirrors several well known registries). No-one uses the "-f" option in peval because it's very inefficeint, although there are several query frameworks out there which do the same sort of thing, namely to pull down a copy of the objects they are interested in into a local database, and then run a query from there (IXP Manager does this, but it's ixp/route server specific). Long term, the fix is to use fully qualified object names, i.e. source::ASxxxxx:localtoken. This means updating RPSL, because that syntax is currently not legitimate. Nick

Hi denis, I am not sure if this feature is used or not however I think this is a very good reason to not go forward with a clean-up (at least until we have properly evaluated things). We will probably have to figure out some other way to deal with objects that are currently causing issues I think. -Cynthia On Wed, Nov 30, 2022 at 3:27 PM denis walker via db-wg <db-wg@ripe.net> wrote:

On Thu, 24 Nov 2022 at 16:13, Nick Hilliard via db-wg <db-wg@ripe.net> wrote:

...

Niall O'Reilly via db-wg wrote on 23/11/2022 18:17:

...

What I have in mind is AS-NIALLSPECIAL, which I populate with a list of AS numbers which I want to advertise to let others know that these are to be handled in some special way, unlike those in AS-NIALLNORMAL.

According to operational circumstances, there might be periods, even long ones, with nothing special going on; AS-NIALLSPECIAL would then be empty, but only for as long as this continued to be the case.

this ^^^ is one of the failure modes.

It would not be safe to assume that empty as-sets named in RPSL policies are unused. It would be less unsafe to assume that unreferenced as-sets are unused. A reasonable middle ground might be - after the proposed new unqualified as-set block has been implemented - to check out all unreferenced as-sets older than a specific period of time and flag those for deletion.

It would also be worthwhile inspecting rpsl in other IRRDBs to see if there are any references. The reason for this is that lots of people use tools like bgpq3 / peval / etc, and query aggregate IRRDBs, e.g. RADB, NTT, etc.

So if you have RPSL in another IRRDB and this references an empty as-set in the RIPE IRRDB, that definition may be picked up in preference to other as-set definitions. I.e. by removing an as-set definition in the RIPE IRRDB, it could unexpectedly influence routing policies elsewhere.

These are corner cases, but they should show why some care will be needed to figure out whether deleting these objects is a good idea.

This is an interesting point. I know nothing about bgpq3 or peval (even though it has been around since the beginning of time). So I just read the documentation on GitHub for them. I never realised this feature existed. What you have described here Nick is in effect a cross registry dependency. A feature that, as far as the RIPE Database is concerned, is undocumented and unsupported. If anyone actually uses this feature we have probably just broken it with NWI-19.

These two tools (and there may be others) work differently. Let's look first at peval. It says in the docs:

"\-s <source-list>" Consider the sources specified in the comma separated <source-list>. If an object is defined in multiple sources in <source-list>, peval uses the definition first encountered in <source-list> from left to right.

Suppose we have '-s RIPE,RADB'. If RADB contains AS-AMAZON and we create an AS-AMAZON in the RIPE Database, then the object in RIPE DB will be used instead of the one in RADB. I guess that is the problem we are trying to solve. But suppose RADB contains AS-WALKER and you don't trust this object. By creating an empty AS-WALKER object (or one with different content) in the RIPE DB you can effectively override the object in RADB in your aggregation. This could be an intentional and legitimate action. The question is, does anyone actually do this? If so we may have just broken this. You can no longer create a short name AS-SET object in the RIPE DB to override one in RADB.

However for peval you can get around this using:

"\-f <file-name>" IRR cache file. You can have any RPSL object in this file, except route objects. They will override these objects in IRR. This option is intended for private objects, or to test new public objects before publishing. You can specify more than one cache file by specifying this option repeatedly.

So you could put a short name AS-SET object in a cache file to override the object you want to ignore in RADB.

The bgpq3 tool also has a source flag (-S) and sources with identical objects are handled the same way as with peval. This tool doesn't have the cache file option of peval. So with bgpq3 you can no longer intentionally override an object in RADB using this feature, if anyone ever does do this.

This is an odd feature as it creates interdependencies between IRR databases that are not defined in the database documentation. It also means that, if anyone uses this feature, we cannot safely delete any AS-SET based on being empty or un-referenced (within its own registry database). It also breaks the self referential integrity of the RIPE Database (and others) because data may be entered into the RIPE Database for use by external tools to impact on data in another registry. Even though peval has been around since the beginning of time, this feature is not even covered by the historical purposes of the RIPE Database. Yet again showing that we don't understand how the RIPE Database is used.

So can anyone say, with any certainty, if this feature is being used?

cheers denis co-chair DB-WG

...

Nick

--

To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://lists.ripe.net/mailman/listinfo/db-wg

Hi Denis, I support the idea of having empty AS-SETs deleted automatically and keeping only AS-NULL in the DB. @Edward Shryane question to the DB team: would it be possible to have some functionality in the LIR portal that converts the short-named AS-SETs to the hierarchical ones with a click of a button? That would help people transition much easier and faster to the hierarchical AS-SETs. Perhaps a button that can be triggered by user and 1) creates a new hierarchical AS-SET 2) copies all elements from old AS-SET to the new one 3) Deletes the old AS-SET 4) Updates the corresponding import/export rules. Kind Regards Stavros On 23/11/2022, 17:58, "db-wg on behalf of denis walker via db-wg" <db-wg-bounces@ripe.net on behalf of db-wg@ripe.net> wrote: Hi Niall On Wed, 23 Nov 2022 at 00:30, Niall O'Reilly via db-wg <db-wg@ripe.net> wrote: > > On 22 Nov 2022, at 19:11, Nick Hilliard via db-wg wrote: > > > Careful with this, e.g. AS-NULL. There are some situations where > > referencing an empty set can be useful in RPSL. > > I'm not sure whether Nick meant this as a single exception, > or rather as an example of a category of useful empty sets. One empty set is the same as any other empty set. We only need one clearly defined and easily recognisable empty set and we have that, AS-NULL. If I create an empty set, AS-WALKER it doesn't do anything that AS-NULL can't do. But AS-WALKER isn't instantly recognisable as an empty set. You have to query it to see that. If we have 100 empty sets in the database, including AS-NULL, that are all being used for valid reasons, 99 of them are redundant, duplicated objects and should be deleted...unless someone can tell me the value of a duplicated empty set. cheers denis co-chair DB-WG > > I can well imagine that most "operationally empty objects" > (as Denis put it) are either useless or troublesome. > > As Nick says, "Careful with this." > > Niall > > -- > > To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ripe... -- To unsubscribe from this mailing list, get a password reminder, or change your subscription options, please visit: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ripe...