- ripe-atlas - mailman.ripe.net

Rate limiting on the APIs
by Robert Kisteleki 06 Feb '23

06 Feb '23

Dear all, We have seen in the past that even with the best intentions, every now and then some users are using excessive amounts of requests to the RIPE Atlas APIs, in some cases resulting in degraded performance to all users. An example for the curious: some AWS account asking the probe API for live queries once a day at 8am like clockwork; over a million queries for probe statuses (note: we don't have a million probes - every query is repeated hundreds or thousands of times) [1] We will apply rate limiting to protect from (some of) these. In particular: * The API in general will impose a limit of no more than 300 queries/second (if done over multiple connections) * The measurement API in particular will impose a limit of 150 queries/second (if done over multiple connections) * When reaching the limit, an HTTP response code of 429 (Too Many Requests; https://www.rfc-editor.org/rfc/rfc6585#section-4) will be given. According to our logs these limits will in general not be visible to users. Users going over these limits will see the 429 response code; client libraries should be able to handle this. We will observe the effect of these measures to determine if we need to tighten further - or perhaps relax them. We plan to apply these changes sometime next week; I'll let you know once they are in place. Regards, Robert [1] a much better way of doing this is to use the probe archive (https://ftp.ripe.net/ripe/atlas/probes/archive/); this is one-time fetch of 2MB or so and probe statuses are pretty stable in general

1 1

Fwd: Deprecating failed prefixes in the host
by Brian E Carpenter 02 Feb '23

02 Feb '23

FYI -------- Forwarded Message -------- Subject: Re: Deprecating failed prefixes in the host Date: Thu, 2 Feb 2023 17:14:22 +1300 From: Brian E Carpenter <brian.e.carpenter(a)gmail.com> To: IPv6 Operations <v6ops(a)ietf.org> Answering some of the comments made, the new version now at https://github.com/becarpenter/misc/blob/main/deprecator.py picks a random RIPE Atlas anchor probe as its ping target (unless the user supplies their own ping target). Thanks to the RIPE Atlas community for not shooting me down in flames. (An even better approach would be choose a new random ping target every hour or so, it's easy enough to code that too.) On my Windows host, I have noticed some odd behaviour of the source selection logic if a ULA is active. No such problem on Linux. For personal reasons I have no time to track that down right now. Again, there's no pretence that this is an operational solution; it's just a proof of half-baked concept. Regards Brian On 24-Jan-23 15:20, Brian E Carpenter wrote: > Hi, > > We keeping falling over the same issue with multi-provider multi-homing: routing magic can keep traffic away from the link to/from a failing provider, but hosts don't know about this and keep sending traffic *from* addresses within the failing provider's prefix. > > As Eduard Vasilenko has pointed out, source address selection at the host is the root cause of this, and RFC6724 doesn't specify a way to change source address selection automatically. Also, we don't currently have a mechanism for rapidly withdrawing a failing prefix. > > To be clear, a typical user program acting as a client uses getaddrinfo() to choose a destination address, but does not use bind() to set a source address - instead it uses connect() and the system chooses a source address. So the need is to avoid choosing a source address that lies within the prefix of a failing provider. > > This message suggests a host-only mechanism to achieve this. If you want a name for it, > consider "unhappy eyeballs". I'm not claiming originality, because it seems obvious to me. > > The mechanism is brutal: have a program running continuously on the host that regularly tests the liveness of each assigned source address, e.g., once a minute. The liveness test is simply a ping to some target address somewhere in the Internet, sourced from the address being tested (i.e., it *does* use bind()). If the ping fails several times in a row, deprecate the source address in question, i.e. set its preferred lifetime to zero. RFC6724 will then ignore it, so no new sessions will use it. Keep the deprecated address in the liveness test, and if it starts to work again, un-deprecate it (i.e. restore its lifetime). > > This is brutal because it does nothing for sessions that are already in progress when a source address fails. They will just time out. I'm sure that solutions such as TAPS is proposing will be much more elegant. It's also brutal because it doesn't know anything about prefixes, it just detects failing /128 addresses. (I don't know if there is a userland mechanism for deprecating a whole prefix.) But this works for legacy client code that simply uses connect(). > > Sounds horrible? Yes. It's also very hard to test; I don't have a setup that allows a proper test. But simulating a ping failure isn't so hard. > > You can find a quick and nasty Python version for Windows 10 and Linux at > https://github.com/becarpenter/misc/blob/main/deprecator.py > This is experimental software that might disturb network access; if you run it, it is entirely at your own risk. It needs sudo or Administrator privilege. > > There's an associated and harmless test program at > https://github.com/becarpenter/misc/blob/main/deptest.py > > (By the way, there seems to be an unexpected interaction between the RFC6724 source-selection algorithm and the presence of a ULA on Windows 10: under investigation.) > > Regards > Brian Carpenter

1 0

Encouraging people to upgrade software probe versions
by Seth David Schoen 01 Feb '23

01 Feb '23

Hi, I recently posted two threads ultimately related to the same question, where it turned out that a feature I wanted to use was present in the current software probe release, but not in the old version that I had installed. It was indeed relatively easy to upgrade (as a reply on this list predicted, the upgrade didn't break anything or require any reconfiguration). But I'd still say "relatively easy", because, on the other hand, the upgrade wasn't managed through my OS package manager like the rest of the software on my VPS server. It appears to me that a lot of software probes (like mine was until recently!) are still running whichever software probe version was current when they were first deployed. After all, I don't believe there's automatic software upgrade functionality in the methods that most people are using to deploy these. Is that true from the point of view of the rest of the Atlas community? Is there anything that might usefully be done to encourage people to upgrade periodically? (For example, either sending some kind of e-mail reminder, or distributing probe software via OS package managers, or even including a mechanism in the probe client itself by which it could upgrade itself -- even if as an opt-in feature?)

10 28

Notification of brief downtime for anchor - ie-dub-as1213
by Mick O'Donovan 31 Jan '23

31 Jan '23

Hi all, I just wanted to give you a brief proactive notification that we, HEAnet, shall be taking our RIPE Atlas anchor called ie-dub-as1213 offline for a brief period as per the below to perform a network maintenance on the edge device it connects to. Date - 8th February 2023 Time - UTC 08:00 - 09:00 Regards, Mick -- Mick O’Donovan Senior Network Engineer HEAnet CLG Ireland's National Education and Research Network 1st Floor | 5 George's Dock | IFSC | Dublin D01 X8N7 | Ireland +353 1 6609040 | mick.odonovan(a)heanet.ie | www.heanet.ie Registered in Ireland, No. 275301 | CRA No. 20036270

1 0

Notification of brief downtime for anchor - ie-dub-as1213
by Mick O'Donovan 31 Jan '23

31 Jan '23

Hi all, I just wanted to give you a brief proactive notification that we, HEAnet, shall be taking our RIPE Atlas anchor called ie-dub-as1213 offline for a brief period as per the below to perform a network maintenance on the edge device it connects to. Date - 8th February 2023 Time - UTC 08:00 - 09:00 Regards, Mick -- Mick O’Donovan Senior Network Engineer HEAnet CLG Ireland's National Education and Research Network 1st Floor | 5 George's Dock | IFSC | Dublin D01 X8N7 | Ireland +353 1 6609040 | mick.odonovan(a)heanet.ie | www.heanet.ie Registered in Ireland, No. 275301 | CRA No. 20036270

1 0

Atlas anchors as ping targets
by Brian E Carpenter 30 Jan '23

30 Jan '23

Hi, I have a question about using Atlas anchors for a special purpose. Is the following acceptable, or would people have objections to the following idea? TL;DR: Is it OK to use Atlas anchors as random ping targets? Over in IETF-land we are (again) debating how to make IPv6 multi-provider multihoming work properly. There's a scenario description and picture at https://github.com/becarpenter/book6/blob/main/6.%20Management%20and%20Oper… One unsolved problem is that if one provider fails, many client applications may continue to use a source address belonging to that provider, instead of switching to a source address belonging to the other provider. One approach to solving this without any changes except in the host is to magically deprecate failing source addresses until they work again. One approach to that is a pure hack - a program that runs in the background pinging a remote target from various source addresses, to deprecate and undeprecate them as appropriate. There's code at https://github.com/becarpenter/misc/blob/main/deprecator.py. This is experimental software that might disturb network access. Would it be OK to choose the probe target by picking an Atlas anchor at random? (I have running code for that, but will not post it to github if there are objections.) I am quite sure that this code is only of interest to a few IPv6 geeks. It could of course be instrumented for measurement purposes, but multihomed sites of this kind are so rare that I don't think there would be much to measure. Regards Brian Carpenter

3 4

Access to probes and documents
by Mccherry, Paul (Postgraduate Researcher) 30 Jan '23

30 Jan '23

Althiough I am logged in when trying to access various probes I do not seem to have the rights, probes such as 91,353,365,3094.... Additionally I noticed I cannot access any documents on RIPE labs such as 'RIPE IPmap: geolocating routes across the internet' or 'Five proposals for a better RIPE Atlas.' Can anyone resolve these access rights please ? Regards Paul McCherry Phd Researcher Lancaster University

2 2

TLS Certificate probes fail ("handshake failure") against Vercel servers
by Daniel Kahn Gillmor 25 Jan '23

25 Jan '23

Hi RIPE Atlas folks-- I noticed today that using the "SSL" probe (gathering X.509 certificates from a TLS endpoint) appears to fail against servers operated by Vercel. the failure is a uniform "handshake failure" according to the web interface. I think this might be a bug in the probe's implementation of the client side of TLS. Where could i find the specific implementation of that probe type so that i can debug it further? Or, if this is a known issue, where can i read up on it in more detail? I've demonstrated this by probing Vercel's own servers, since they appear to be eating their own dogfood. You can see this with measurement 49131334: https://atlas.ripe.net/measurements/49131334/#probes here is the definition of the query i made: { "definitions": [ { "target": "vercel.com", "af": 4, "port": 443, "hostname": "vercel.com", "description": "SSL measurement to vercel.com", "resolve_on_probe": true, "skip_dns_check": false, "type": "sslcert" } ], "probes": [ { "type": "area", "value": "WW", "requested": 10 } ], "is_oneoff": true, } This is pretty odd, since normal TLS clients appear to be able to connect to them fine. Here's me running a manual handshake debugger against vercel.com's servers: $ gnutls-cli-debug vercel.com GnuTLS debug client 3.7.8 Checking vercel.com:443 whether the server accepts default record size (512 bytes)... yes whether %ALLOW_SMALL_RECORDS is required... no whether we need to disable TLS 1.2... no whether we need to disable TLS 1.1... no whether we need to disable TLS 1.0... no whether %NO_EXTENSIONS is required... no for TLS 1.0 (RFC2246) support... no for TLS 1.0 (RFC2246) support with TLS 1.0 record version... no for TLS 1.1 (RFC4346) support... no fallback from TLS 1.1 to... failed for TLS 1.2 (RFC5246) support... yes for TLS 1.3 (RFC8446) support... yes for known TLS or SSL protocols support... yes TLS1.2 neg fallback from TLS 1.6 to... failed (server requires fallback dance) for HTTPS server name... Vercel for certificate chain order... sorted for safe renegotiation (RFC5746) support... yes for encrypt-then-MAC (RFC7366) support... no for ext master secret (RFC7627) support... no for heartbeat (RFC6520) support... no for version rollback bug in RSA PMS... yes whether the server ignores the RSA PMS version... yes whether small records (512 bytes) are tolerated on handshake... yes whether cipher suites not in SSL 3.0 spec are accepted... yes whether a bogus TLS record version in the client hello is accepted... yes whether the server understands TLS closure alerts... yes whether the server supports session resumption... yes for anonymous authentication support... no for RSA key exchange support... no |<1>| FFDHE groups advertised, but server didn't support it; falling back to server's choice for ephemeral Diffie-Hellman support... yes |<1>| FFDHE groups advertised, but server didn't support it; falling back to server's choice for RFC7919 Diffie-Hellman support... no for ephemeral EC Diffie-Hellman support... yes for VKO GOST-2012 (draft-smyshlyaev-tls12-gost-suites) support... no for curve SECP256r1 (RFC4492)... yes for curve SECP384r1 (RFC4492)... yes for curve SECP521r1 (RFC4492)... yes for curve X25519 (RFC8422)... yes for AES-GCM cipher (RFC5288) support... yes for AES-CCM cipher (RFC6655) support... no for AES-CCM-8 cipher (RFC6655) support... no for AES-CBC cipher (RFC3268) support... no for CAMELLIA-GCM cipher (RFC6367) support... no for CAMELLIA-CBC cipher (RFC5932) support... no for 3DES-CBC cipher (RFC2246) support... no for ARCFOUR 128 cipher (RFC2246) support... no for CHACHA20-POLY1305 cipher (RFC7905) support... yes for GOST28147-CNT cipher (draft-smyshlyaev-tls12-gost-suites) support... no for MD5 MAC support... no for SHA1 MAC support... no for SHA256 MAC support... no for GOST28147-IMIT MAC (draft-smyshlyaev-tls12-gost-suites) support... no for max record size (RFC6066) support... no for OCSP status response (RFC6066) support... yes $ Thanks for any pointers, --dkg

2 2

Duplicate packet detection in traceroute results
by Ana Custura 20 Jan '23

20 Jan '23

Hi everyone, Just a question on how Atlas detects duplicate ('dup') packets in a traceroute. Is the 'dup' field added to a result when two received packets are exactly the same, regardless of whether they were received for a different initial TTL? Or are any packets that were unexpectedly received for a specific TTL marked as duplicates (e.g. if you receive 6 replies from 2 different IP addresses to the 3 initial packets sent with TTL 1, will 3 of these replies be marked as duplicates)? Thanks for your help! Regards, Ana

2 2

Re: [atlas] ripe-atlas Digest, Vol 137, Issue 8
by Geert Jan de Groot 19 Jan '23

19 Jan '23

On 09/01/2023 11:32, ripe-atlas-request(a)ripe.net wrote: > The hardware probes are fully managed by RIPE NCC, and their only purpose is to execute measurements. I agree here, whether to update is up to RIPE NCC. > On software probes, RIPE NCC is not in control of the unit. The software probe may run other services and updating is up to its local sysadmin. Consider operating system updates, which are updated by RIPE NCC on the hardware probes, but cannot be updated on software probes. > > To automatically update, the solution here is a simple CRON job which executes yum -q update atlasswprobe periodically (say once per day or so), which was the solution that was part of the RPM up to 5080. Would it help if we add this to the instructions? One question perhaps is if a probe running very obsoleted probe code, is useful or perhaps at some time probes older than version 1234 should not be used for pool measurements? For instance, release 2345 enables measurements on 240/12 while older releases, including release 2344, 240/12 is broken. Then, having probes with software 2344 or lower will give bias on measurements for 240/12. And for that reason I wonder if it makes sense to have a policy to "phase out" probes that are running obsoleted code, however well intended? Geert Jan

2 1