On Thu, Jan 24, 2019 at 3:41 AM Robert Kisteleki <robert@ripe.net> wrote:

On 2019-01-24 06:39, Christopher Morrow wrote:

...

howdy! So, if I have a dns server somewhere and I want to make a bunch of measurements: "how does NYC see my dns?" (pick N probes near NYC) "How does DEN see my dns?" (pick N probes near DEN) "How does LHR see my dns?" (pick N probes near LHR) "how does ...."

...you get the idea... I'm trying to characterize my service from a 'user' perspective by large Metropolis, well within N km of that metropolis anyway :) but across ~70 or so metros, in v4 and v6 and with authoritative and non-authoritative queries with each test running for 4 days so i can see some daily cycles in traffic patterns/etc. The number of measurements is large in total, breaking up by 3-4 metropolis chunks is super tedious :(

Can I request an API key (or some other thing) which can make that sort of request happen for all my measurements in one go? The thing I'm probing is very able to handle a few extra thousand queries per second.. and I'd only be hurting myself i get my estimate wrong :)

thanks! -chris

Hi,

Note taht we have a large amount of measurements already against 8.8.8.8, 1.1.1.1 and such, perhaps reusing data from these is an option?

perhaps. I can go digging. i was/am particularly interested in performance for v4/v6 cached/not from the same vantage point at as nearly the same time. I didn't think that was going to be fruitful in searching existing measurements :( I should have taken a bit longer to dig though.

(I'd be particularly happy if it was, since one of the very reasons we wanted to make this whole network really open is to be able to reuse data from others' measurements.)

If that's not possible, then I'd advise you to run a small script that issues a measurement request and parametrises it with cantage point / target pairs, and use an API key for it. Perhaps also ask for a quota increase first :-)

I have this script, but I need to have something that runs over many days (or maintains state) and issues the measurements at a pace which maximizes my measurement speed and minimizes my inolvement :) It wasn't apparent that the restriction I'm seeing is 'per api key', I thought it was 'per target'.

Cheers, Robert

On Thu, Jan 24, 2019 at 3:52 AM Shane Kerr <shane@time-travellers.org> wrote:

Chris,

On 24/01/2019 06.39, Christopher Morrow wrote:

...

howdy! So, if I have a dns server somewhere and I want to make a bunch of measurements: "how does NYC see my dns?" (pick N probes near NYC) "How does DEN see my dns?" (pick N probes near DEN) "How does LHR see my dns?" (pick N probes near LHR) "how does ...."

...you get the idea... I'm trying to characterize my service from a 'user' perspective by large Metropolis, well within N km of that metropolis anyway :) but across ~70 or so metros, in v4 and v6 and with authoritative and non-authoritative queries with each test running for 4 days so i can see some daily cycles in traffic patterns/etc. The number of measurements is large in total, breaking up by 3-4 metropolis chunks is super tedious :(

Can I request an API key (or some other thing) which can make that sort of request happen for all my measurements in one go? The thing I'm probing is very able to handle a few extra thousand queries per second.. and I'd only be hurting myself i get my estimate wrong :)

I was part of a team on a hackathon a while back which needed to compute distances between points on the globe, including Atlas probes:

https://github.com/shane-kerr/ripe-atlas-anycast-work

I see that the openflights project has since moved to GitHub, and so you can get the IATA airport data here:

https://github.com/jpatokal/openflights/blob/master/data/airports.dat

The geo data wasn't problematic for me, once I found: github.com/kelvins/geocoder which apparently uses: https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.... oh hai! It looks like I didn't do it the 'insane way', mostly :) (I should make the repository of code I wrote available, oops) I guess I should update the README.md...

The repository is basically a last-point-in-time snapshot of the hackathon work, so it a bit rough. 😬 Anyway, this code is probably the closest to what you want:

https://github.com/shane-kerr/ripe-atlas-anycast-work/blob/master/add-dist.p...

i think you are just sorting through measurement response data to find things near/around/at your points of interest here, right? that seems useful.

Turning the great_circle_dist() function into a filter across the set of all probes (meta-probes.json, which you can download in advance) should get you something like what you want.

looks neat ;)

Note that this was all done 4 years ago according to GitHub, so maybe Atlas provides better tools for this nowadays. 😊

they do have a method to request probes based on geo data and a radius from that point, which is what I am doing. They also have filters (which I've apparently sucked at reading completely!) to limit based on type/ip-version/activity/etc. mostly you can query for: "probes near here with these sets of attributes" which works pretty well ;) my main problem was/is (I think) that I want to say: "For these M metros, please find 10 (or so) probes with working address family (v4||v6) and probe dns server (V4||V6 address) with a (non-)authoritative request" my M number is ~70+. I didn't want (for some reason?) to try and gather the probes across the globe into less measurements but more probes... huh, I could try that I guess, then I'd only have 4 measurements not M*4. Cheers,

-- Shane