- ripe-atlas - mailman.ripe.net

Proposal: Measure well-known CDNs,[CDN-HTTP]
by Dirk-Jan van Helmond 20 Dec '22

20 Dec '22

Hi RIPE Atlas users, I stumbled upon this thread and I wanted to chime in. I work for Cloudflare and we of course use a wide set of external visibility and performance management tools to track our visibility. But on occasions I also use the RIPE Atlas network to perform some specific reachability tests. I believe the distribution of probes and the fact that they're als in eyeball ASN is a really valuable factor that RIPE Atlas has. At the moment I'm running into a limitation around HTTP tests. I would like to use the RIPE Atlas network to curl some pages to retrieve and evaluate the HTTP headers. This is currently not possible with Atlas as it is only allowed to probe Anchors for safety reasons, which I fully understand and support (you don't want to turn Atlas into a botnet). The HTTP targets that I would like to probe would be all on our IP space, so I don't see a direct risk from the outbound part of the request. I can see there is some risk with the return part, where the content from an "unknown" system gets returned to a probe in someone's private network, with possible ramifications. So I was pondering about two possible options that can potentially help me, but not risk the Atlas project or demonstrate favoritism towards particular companies. 1) One of the options would be to open up some of our IP space to the probes for HTTP probes. 2) The other option would be for us to host an Anchor and put our CDN in front of it. That way RIPE has complete control over the returned data. I would love to hear the general consensus on such approach and the viability of it Best Regards, -- Dirk-Jan van Helmond On 17/12/2022 00:19, Barry Raveendran Greene wrote: > > >> On Dec 16, 2022, at 23:29, Stephane Bortzmeyer <bortzmeyer at nic.fr> wrote: >> >> There is a larger problem here, a more strategic one: such a feature >> would contribute to the centralisation of the Internet, which is >> already too important. Tagging some targets are "important" and >> "worthy of measurements" would mean that we consider some HTTP servers >> to be more useful than others. That would be a bad message from RIPE. > > We’ve come full circle - we started with centralized PTTs - moved to a decentralized ASN/Paul Baran model - now Re-centralized based on marketing domination. > > +1 with Stephane’s observation. The selection of who to measure is a statement. +1 Also, while the data would be useful, I don't think the role of the ripe ncc is to grade commercial services. Let other companies or individual researchers do that. -- Dirk-Jan van Helmond Leader Solutions Engineering EMEA mobile: +31611645622 email: dirkjan(a)cloudflare.com www.cloudflare.com <https://www.cloudflare.com/>

1 0

Re: [atlas] Proposal: Remove support for non-public measurements [ONLY-PUBLIC] (Robert Kisteleki)
by Renny Leal 16 Dec '22

16 Dec '22

May the non-public measurements cost more in terms of credits or even money to support the infrastructure? El 16 dic. 2022 11:28 a. m., ripe-atlas-request(a)ripe.net escribió: Send ripe-atlas mailing list submissions to ripe-atlas(a)ripe.net To subscribe or unsubscribe via the World Wide Web, visit https://lists.ripe.net/mailman/listinfo/ripe-atlas or, via email, send a message with subject or body 'help' to ripe-atlas-request(a)ripe.net You can reach the person managing the list at ripe-atlas-owner(a)ripe.net When replying, please edit your Subject line so it is more specific than "Re: Contents of ripe-atlas digest..." Today's Topics: 1. Re: Proposal: Remove support for non-public measurements [ONLY-PUBLIC] (Petr ?pa?ek) 2. RIPE Atlas Quarterly Planning Q1 2023 (Robert Kisteleki) 3. Re: Proposal: Remove support for non-public measurements [ONLY-PUBLIC] (Robert Kisteleki) 4. Re: Proposal: Remove support for non-public measurements [ONLY-PUBLIC] (Magnus Sandberg) 5. Re: Proposal: Generic HTTP measurements [GENERIC-HTTP] (Hugo Salgado) 6. New RIPE Atlas stream release (Chris Amin) 7. Re: Proposal: Measure well-known CDNs,[CDN-HTTP] (Stephane Bortzmeyer) ---------------------------------------------------------------------- Message: 1 Date: Fri, 16 Dec 2022 12:21:03 +0100 From: Petr ?pa?ek <pspacek(a)isc.org> To: ripe-atlas(a)ripe.net Subject: Re: [atlas] Proposal: Remove support for non-public measurements [ONLY-PUBLIC] Message-ID: <06d70394-ea13-9be7-486e-a9017a77ba39(a)isc.org> Content-Type: text/plain; charset=UTF-8; format=flowed On 15. 12. 22 19:41, Steve Gibbard wrote: > I worry that that would have a ?chilling effect? on use of the service. I hear your concerns, and have a proposal how to quantify this concern: - First, amend the page to say that all the data are public. (Possibly also switch the flag, but that can be a separate step.) - Second, observe what has changed in the usage pattern. - Third, evaluate. That way we don't need to stay in limbo over hypothetical situations but get real data. Side note about usefulness of one-off measurement history: I think it _might_ be is interesting for anyone doing study on any given outage, or even study about optimization practices over time. For example, the DNS community has service called DNSViz which does just one-off measurements, and yet, researchers come and write papers based on data from DNSViz. HTH. -- Petr ?pa?ek ------------------------------ Message: 2 Date: Fri, 16 Dec 2022 12:32:11 +0100 From: Robert Kisteleki <robert(a)ripe.net> To: "ripe-atlas(a)ripe.net" <ripe-atlas(a)ripe.net> Subject: [atlas] RIPE Atlas Quarterly Planning Q1 2023 Message-ID: <d4aacacc-9691-d6f4-3cbb-ae0e9ad2d18f(a)ripe.net> Content-Type: text/plain; charset=UTF-8; format=flowed Dear colleagues, The quarterly plans for RIPE Atlas can now be found at: https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas In the last quarter, we worked on several proposals to change the RIPE Atlas behaviour, including implementing general availability of HTTP measurements and removing the ability to schedule ?non-public? measurements. This quarter, we?ll continue with the distribution of the v5 RIPE Atlas hardware probes to hosts, ambassadors and sponsors as well as work on other items proposed by the community. If you have any input on our plans or want to let us know what you would like to see us work on, please let us know. Kind regards, Robert Kisteleki Research and Development Manager RIPE NCC ------------------------------ Message: 3 Date: Fri, 16 Dec 2022 12:56:45 +0100 From: Robert Kisteleki <robert(a)ripe.net> To: Magnus Sandberg <mem(a)fallback.netnod.se>, ripe-atlas(a)ripe.net Subject: Re: [atlas] Proposal: Remove support for non-public measurements [ONLY-PUBLIC] Message-ID: <36a79d58-83e1-4e93-a707-aa47a653ac2e(a)ripe.net> Content-Type: text/plain; charset=UTF-8; format=flowed Hi, > The definition of a "small" test could be something like; > - maximum 20 probes > - runs for maximum 60 minutes > - is repeated maximum 10 times > ? (if you run every 300sec you have to limit the endtime to 50 minutes) What would you propose the API to do if this rule is violated? Should it outright refuse the measurement, or should it silently turn on the public flag (silently, because even if we emit a warning, it is probably never seen by a human...)? > With such limits you can troubleshoot things (non-publicly) but you > can't build your monitoring system on top of that. I guess this hinges on what level of support (legal, technical or other) we're aiming for when it comes to others building services on top of RIPE Atlas. Regards, Robert ------------------------------ Message: 4 Date: Fri, 16 Dec 2022 14:13:47 +0100 From: Magnus Sandberg <mem(a)fallback.netnod.se> To: Robert Kisteleki <robert(a)ripe.net>, ripe-atlas(a)ripe.net Subject: Re: [atlas] Proposal: Remove support for non-public measurements [ONLY-PUBLIC] Message-ID: <2ed5e29a-9590-f20c-fd6f-f21e58846d40(a)fallback.netnod.se> Content-Type: text/plain; charset=UTF-8; format=flowed On 2022-12-16 at 12:56, Robert Kisteleki wrote: > Hi, > >> The definition of a "small" test could be something like; >> - maximum 20 probes >> - runs for maximum 60 minutes >> - is repeated maximum 10 times >> ?? (if you run every 300sec you have to limit the endtime to 50 minutes) > > What would you propose the API to do if this rule is violated? Should it > outright refuse the measurement, or should it silently turn on the > public flag (silently, because even if we emit a warning, it is probably > never seen by a human...)? I think the safest thing would be to refuse with an error message. Just override a setting with just a warning is probably not enough, as you wrote. >> With such limits you can troubleshoot things (non-publicly) but you >> can't build your monitoring system on top of that. > > I guess this hinges on what level of support (legal, technical or other) > we're aiming for when it comes to others building services on top of > RIPE Atlas. I really love RIPE Atlas and all the possibilities that come with an open and mature platform. I think as much as possible should be open and reusable by others. However, I understand that some people in some cases have other wishes. Regards, // mem ------------------------------ Message: 5 Date: Fri, 16 Dec 2022 10:15:09 -0300 From: Hugo Salgado <hsalgado(a)nic.cl> To: Robert Kisteleki <robert(a)ripe.net> Cc: "ripe-atlas(a)ripe.net" <ripe-atlas(a)ripe.net> Subject: Re: [atlas] Proposal: Generic HTTP measurements [GENERIC-HTTP] Message-ID: <Y5xvXQ1ICmRBpPEv@pepino> Content-Type: text/plain; charset="us-ascii" Dear Robert, As I have expressed at other times, I think we have to be very careful with enabling HTTP probes due to traffic increase issues. I have personally delivered quite a few probes to remote locations in regions with very poor connectivity, and the host's first concern is how much bandwidth the measurements take up. I have always reassured them that these are only network level connectivity measurements, and no web traffic. It seems to me that in the first world it may be difficult to imagine places where a few megabytes per month of traffic matters, but that is the case in remote regions, and I think the benefit of having an Atlas network with worldwide coverage is greater than being able to perform higher layer measurements or monitoring. It seems to me that the idea of making it "opt-in" with a tag on the probe is the right one, even if it makes its deployment and usability slow at first. I sincerely believe we have a responsibility to hosts that currently help in complex regions. Thanks and regards, Hugo Salgado NIC Chile - .CL On 15:03 14/12, Robert Kisteleki wrote: > Dear RIPE Atlas users, > > We recently published a RIPE Labs article containing a few proposals: > https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. > We'd like to encourage you to express your comments about this proposal (if > you'd like to share them) here. > > Regards, > Robert Kisteleki > For the RIPE Atlas team > > -- > ripe-atlas mailing list > ripe-atlas(a)ripe.net > https://lists.ripe.net/mailman/listinfo/ripe-atlas > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <https://lists.ripe.net/ripe/mail/archives/ripe-atlas/attachments/20221216/e…> ------------------------------ Message: 6 Date: Fri, 16 Dec 2022 15:34:04 +0100 From: Chris Amin <camin(a)ripe.net> To: ripe-atlas(a)ripe.net Subject: [atlas] New RIPE Atlas stream release Message-ID: <d278c506-e467-5f3f-040b-296efd878b05(a)ripe.net> Content-Type: text/plain; charset=UTF-8; format=flowed Dear colleagues, The new version of the RIPE Atlas streaming API is now available. As mentioned in the previous e-mail, the API can now be accessed using either plain WebSockets or plain GET requests. Socket.IO support is being maintained for now, so you don't have to immediately change your installations, but the new interfaces should be preferred wherever possible. The documentation can be found at: https://atlas.ripe.net/docs/apis/streaming-api/ Kind regards, Chris Amin RIPE NCC On 30/11/2022 10:31, Chris Amin wrote: > In around three weeks we will be releasing a new version of the RIPE > Atlas stream (https://atlas-stream.ripe.net) with an API based on > WebSockets and plain HTTP requests. > > The current Socket.IO interface will be maintained for backwards > compatibility for the near future, but some subscription parameters will > be dropped due to extremely low or no usage. These are: > > * Historic playback (startTime, endTime) ? note that this does not > include sendBacklog, which will still work as now > * Arbitrary server-side filtering of fields (equalsTo, greaterThan, > lessThan) ? note that filtering by "prb", "msm", "type" etc will work as > now > > We can see from the service logs that each of these parameters has > either not been used at all or used by a single IP address in the past > month, so the impact should be minimal. However, if you happen to be > using one of these parameters then please get in touch with me. > > Aside from these changes, existing use cases should continue to work as > they do now, and we will be monitoring closely for any issues. > > There will be more information on the new API, including complete > documentation, closer to the time. ------------------------------ Message: 7 Date: Fri, 16 Dec 2022 16:28:07 +0100 From: Stephane Bortzmeyer <bortzmeyer(a)nic.fr> To: Robert Kisteleki <robert(a)ripe.net> Cc: "ripe-atlas(a)ripe.net" <ripe-atlas(a)ripe.net> Subject: Re: [atlas] Proposal: Measure well-known CDNs,[CDN-HTTP] Message-ID: <Y5yOh9hvnmGG0PCV(a)nic.fr> Content-Type: text/plain; charset=us-ascii On Wed, Dec 14, 2022 at 03:02:46PM +0100, Robert Kisteleki <robert(a)ripe.net> wrote a message of 15 lines which said: > We recently published a RIPE Labs article containing a few proposals: > https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. > We'd like to encourage you to express your comments about this proposal (if > you'd like to share them) here. (In the Cons) "Possible arguments about which provider to include in this set and which to refuse." There is a larger problem here, a more strategic one: such a feature would contribute to the centralisation of the Internet, which is already too important. Tagging some targets are "important" and "worthy of measurements" would mean that we consider some HTTP servers to be more useful than others. That would be a bad message from RIPE. ------------------------------ Subject: Digest Footer -- ripe-atlas mailing list ripe-atlas(a)ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas ------------------------------ End of ripe-atlas Digest, Vol 136, Issue 15 *******************************************

2 1

Upcoming RIPE Atlas stream release in December
by Chris Amin 16 Dec '22

16 Dec '22

Dear colleagues, In around three weeks we will be releasing a new version of the RIPE Atlas stream (https://atlas-stream.ripe.net) with an API based on WebSockets and plain HTTP requests. The current Socket.IO interface will be maintained for backwards compatibility for the near future, but some subscription parameters will be dropped due to extremely low or no usage. These are: * Historic playback (startTime, endTime) — note that this does not include sendBacklog, which will still work as now * Arbitrary server-side filtering of fields (equalsTo, greaterThan, lessThan) — note that filtering by "prb", "msm", "type" etc will work as now We can see from the service logs that each of these parameters has either not been used at all or used by a single IP address in the past month, so the impact should be minimal. However, if you happen to be using one of these parameters then please get in touch with me. Aside from these changes, existing use cases should continue to work as they do now, and we will be monitoring closely for any issues. There will be more information on the new API, including complete documentation, closer to the time. Kind regards, Chris Amin RIPE NCC

1 1

RIPE Atlas Quarterly Planning Q1 2023
by Robert Kisteleki 16 Dec '22

16 Dec '22

Dear colleagues, The quarterly plans for RIPE Atlas can now be found at: https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas In the last quarter, we worked on several proposals to change the RIPE Atlas behaviour, including implementing general availability of HTTP measurements and removing the ability to schedule “non-public” measurements. This quarter, we’ll continue with the distribution of the v5 RIPE Atlas hardware probes to hosts, ambassadors and sponsors as well as work on other items proposed by the community. If you have any input on our plans or want to let us know what you would like to see us work on, please let us know. Kind regards, Robert Kisteleki Research and Development Manager RIPE NCC

1 0

Request for Atlas credits for teaching and research
by Nitinder Mohan 16 Dec '22

16 Dec '22

Dear RIPE Atlas community, I am a postdoctoral researcher in Technical University of Munich, Germany. I am coming to you with a request to transfer some of your surplus Atlas credits to my account nitinder.mohan(a)tum.de if possible. I work on several Internet measurements research problems which rely on Atlas platform for measurements. Additionally, I am also planning to give out a cloud connectivity measurment assignment to my course students to give them hands-on experience with Atlas -- which exceeds my local credit generation rate. Your help will be highly appreciated! Thanks and Regards Nitinder Mohan Technical University Munich (TUM) https://www.nitindermohan.com/

10 11

Re: [atlas] Non-public probes?
by Ernst J. Oud 15 Dec '22

15 Dec '22

Chris, Weird. That probe is on AS15435 (just like me) which is an ISP that not yet offer any ipv6 functionality. So how come ipv4-works is not tagged? But thanks for the clarification that this tag is the reason I can’t add it. I can’t do any ipv6 measurements to try. Regards Ernst J. Oud

1 0

Re: [atlas] Non-public probes?
by Ernst J. Oud 15 Dec '22

15 Dec '22

But what is going on then with: https://atlas.ripe.net/frames/probes/1003690 This probe is within AS15435 and when I list all probes in that AS it is shown, but I cannot add it to a measurement, it is rejected. Using a curl request to: stat.ripe.net/data/atlas-probes/data.json?resouce=15435 shows this probe, with a tag “is_public” : false and I cannot access this probe’s info via the link above, I cannot add it to a measurement and I cannot access any data it collects. How does this rhyme to that non-public “probes are therefore contributing very nearly as much to the network as everybody else”? Regards, Ernst J. Oud

4 4

Non-public probes?
by Ernst J. Oud 15 Dec '22

15 Dec '22

Hi, Just recently I realized that a probe can be labeled as “non-public”. Why is this allowed? Atlas is a community effort. Why are probe “owners” allowed to have benefits from using Atlas without sharing data? Regards, Ernst J. Oud

2 1

Proposal: remove per-hop “late” packets from traceroute [LATE-PACKETS]
by Robert Kisteleki 15 Dec '22

15 Dec '22

Dear RIPE Atlas users, We recently published a RIPE Labs article containing a few proposals: https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. We'd like to encourage you to express your comments about this proposal (if you'd like to share them) here. Regards, Robert Kisteleki For the RIPE Atlas team

2 1

Atlas api down
by Ernst J. Oud 14 Dec '22

14 Dec '22

Hi, As of around 17:30 UTC on December 13th, all measurements other than built-ins, from the website or via Magellan, report no results. The website just hangs and Magellan times out. I believe all access via the api to be broken. I read that maintenance is pending, but that is scheduled the 14th… What’s up? Regards, Ernst J. Oud

3 6