NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
Dear Routing WG, Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI based BGP Origin Validation on virtually all EBGP sessions, both customer and peering edge. This change positively impacts the Internet routing system. The use of RPKI technology is a critical component in our efforts to improve Internet routing stability and reduce the negative impact of misconfigurations or malicious attacks. RPKI Invalid route announcements are now rejected in NTT EBGP ingress policies. A nice side effect: peerlock AS_PATH filters are incredibly effective when combined with RPKI OV. For NTT, this is the result of a multiyear project, which included outreach, education, collaboration with industry partners, and production of open source software shared among colleagues in the industry. Shout out to Louis & team (Cloudflare) for the open source GoRTR software and the OpenBSD project for rpki-client(8). I hope some take this news as encouragement to consider RPKI OV "invalid == reject"-policies as safe to deploy in their own BGP environments too. :-) Kind regards, Job
congrats! it's now been a year of folk rollout reject invalids. the word seems to be that pretty much all issues are on the vendors' laps. randy
Congratulations, Job! You and NTT have been a voice for routing security and a source for useful tools and techniques for a very long time. The routing infrastructure is the better for your support. (And after this expression of gratitude, here’s the ask: when do we get to hear the lessons-learned?) —Sandy
On Mar 25, 2020, at 9:09 PM, Job Snijders <job@ntt.net> wrote:
Dear Routing WG,
Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI based BGP Origin Validation on virtually all EBGP sessions, both customer and peering edge. This change positively impacts the Internet routing system.
The use of RPKI technology is a critical component in our efforts to improve Internet routing stability and reduce the negative impact of misconfigurations or malicious attacks. RPKI Invalid route announcements are now rejected in NTT EBGP ingress policies. A nice side effect: peerlock AS_PATH filters are incredibly effective when combined with RPKI OV.
For NTT, this is the result of a multiyear project, which included outreach, education, collaboration with industry partners, and production of open source software shared among colleagues in the industry.
Shout out to Louis & team (Cloudflare) for the open source GoRTR software and the OpenBSD project for rpki-client(8).
I hope some take this news as encouragement to consider RPKI OV "invalid == reject"-policies as safe to deploy in their own BGP environments too. :-)
Kind regards,
Job
On 2020-03-26 02:09, Job Snijders wrote:
Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI based BGP Origin Validation on virtually all EBGP sessions, both customer and peering edge. This change positively impacts the Internet routing system.
Hello Job, It is the word "virtually" that triggers me :), because in my mind it translates to "not all of them". Why haven't you enabled it on all our EBGP sessions? And doesn't this make enabling it on the rest of the validation less useful? Because if an invalid announcements is received on an EBGP session without RPKI validation, doesn't it propagate trough the rest of the network via iBGP, and thus make the hijack reachable for all of NTT? I'm sure you guys thought about this, but I'm just wondering what you did to prevent the scenario I just described :). Thanks for making the world a safer place! Kind regards, Tijn Buijs
On Sat, Apr 04, 2020 at 10:20:38AM +0200, Tijn Buijs wrote:
On 2020-03-26 02:09, Job Snijders wrote:
Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI based BGP Origin Validation on virtually all EBGP sessions, both customer and peering edge. This change positively impacts the Internet routing system.
It is the word "virtually" that triggers me :), because in my mind it translates to "not all of them". Why haven't you enabled it on all our EBGP sessions?
Currently, a number of technical caveats affects our ability for 100% deployment. As a result a limited number of RPKI invalid route announcements are still propagated to EBGP peers. We are developing technical workarounds and engaging vendors in order to enable us to gradually decrease this number in the coming months. We remain committed to global RPKI deployment, and we will provide updates on our progress from time to time as we work to reduce this number to zero. Kind regards, Job
Dear Tijn, I didn't fully answer your email, more below. On Sat, Apr 04, 2020 at 10:20:38AM +0200, Tijn Buijs wrote:
And doesn't this make enabling it on the rest of the validation less useful? Because if an invalid announcements is received on an EBGP session without RPKI validation, doesn't it propagate trough the rest of the network via iBGP, and thus make the hijack reachable for all of NTT?
Certainly good questions. When a RPKI Invalid announcement comes in via a EBGP session where RPKI is not enabled, the route cannot be marked as RPKI invalid because the Origin Validation is not applied in the routing policy associated with that specific EBGP ingress attachment point. NTT performs Origin Validation only in "ebgp-in" policies. Other protection mechanisms still apply: maximum prefix limits, routes that have bogon ASNs anywhere in the AS_PATH are rejected, if a Peerlock protected ASN shows up anywhere in the AS_PATH the route is blocked, and of course prefix-list filters generated from IRR data are still in use.
I'm sure you guys thought about this, but I'm just wondering what you did to prevent the scenario I just described :).
In the general case we can assume your BGP peers have a backbone and will announce all their routes to you at every interconnection point. One way to look at RPKI deployment progress is to view it as incremental reduction of your risk surface. For each peer ASN where the operator applies Origin Validation on every individual BGP session with that specific ASN, the needle is moved a little bit. I observed one interesting situation that might be good to share with the group: there was one specific BGP session (with one of our larger peering partners) on which we were not anble to enable RPKI Origin Validation for technical reasons. I reached out to the peer to discuss what could be done because NTT was receiving a substantial amount of routing information via this session (all of valid, invalid, and not-found). It would be suboptimal if we manage to block a hijack on 29 out of the 30 BGP sessions yet still end up carrying the incorrect route because of that 30th session. The peering partner isn't there yet for full scale global deployment in their whole network, but they did already have some experience with RPKI. The solution they came up with was to enable RPKI OV on that one session in the /outbound/ direction (their 'ebgp-out' policy, the announcements from that peer towards NTT). This closed that tiny gap through which invalids could still propagate from this peer's customer cone into NTT's network. Peers scratching each other's back :-) Kind regards, Job
A very big step forward, Congrats. On Sat, Apr 4, 2020 at 9:05 PM Job Snijders <job@ntt.net> wrote:
Dear Tijn,
I didn't fully answer your email, more below.
On Sat, Apr 04, 2020 at 10:20:38AM +0200, Tijn Buijs wrote:
And doesn't this make enabling it on the rest of the validation less useful? Because if an invalid announcements is received on an EBGP session without RPKI validation, doesn't it propagate trough the rest of the network via iBGP, and thus make the hijack reachable for all of NTT?
Certainly good questions. When a RPKI Invalid announcement comes in via a EBGP session where RPKI is not enabled, the route cannot be marked as RPKI invalid because the Origin Validation is not applied in the routing policy associated with that specific EBGP ingress attachment point. NTT performs Origin Validation only in "ebgp-in" policies.
Other protection mechanisms still apply: maximum prefix limits, routes that have bogon ASNs anywhere in the AS_PATH are rejected, if a Peerlock protected ASN shows up anywhere in the AS_PATH the route is blocked, and of course prefix-list filters generated from IRR data are still in use.
I'm sure you guys thought about this, but I'm just wondering what you did to prevent the scenario I just described :).
In the general case we can assume your BGP peers have a backbone and will announce all their routes to you at every interconnection point. One way to look at RPKI deployment progress is to view it as incremental reduction of your risk surface. For each peer ASN where the operator applies Origin Validation on every individual BGP session with that specific ASN, the needle is moved a little bit.
I observed one interesting situation that might be good to share with the group: there was one specific BGP session (with one of our larger peering partners) on which we were not anble to enable RPKI Origin Validation for technical reasons.
I reached out to the peer to discuss what could be done because NTT was receiving a substantial amount of routing information via this session (all of valid, invalid, and not-found). It would be suboptimal if we manage to block a hijack on 29 out of the 30 BGP sessions yet still end up carrying the incorrect route because of that 30th session.
The peering partner isn't there yet for full scale global deployment in their whole network, but they did already have some experience with RPKI. The solution they came up with was to enable RPKI OV on that one session in the /outbound/ direction (their 'ebgp-out' policy, the announcements from that peer towards NTT). This closed that tiny gap through which invalids could still propagate from this peer's customer cone into NTT's network. Peers scratching each other's back :-)
Kind regards,
Job
participants (5)
-
Ehsan Ghazizadeh -
Job Snijders -
Randy Bush -
Sandra Murphy -
Tijn Buijs