On Thu, Jan 24, 2019 at 3:52 AM Shane Kerr <shane@time-travellers.org> wrote:
Chris,

On 24/01/2019 06.39, Christopher Morrow wrote:
> howdy!
> So, if I have a dns server somewhere and I want to make a bunch of
> measurements:
>    "how does NYC see my dns?" (pick N probes near NYC)
>    "How does DEN see my dns?" (pick N probes near DEN)
>    "How does LHR see my dns?" (pick N probes near LHR)
>    "how does ...."
>
> ...you get the idea... I'm trying to characterize my service from a
> 'user' perspective by large Metropolis, well within N km of that
> metropolis anyway :) but across ~70 or so metros, in v4 and v6 and with
> authoritative and non-authoritative queries with each test running for 4
> days so i can see some daily cycles in traffic patterns/etc. The number
> of measurements is large in total, breaking up by 3-4 metropolis chunks
> is super tedious :(
>
> Can I request an API key (or some other thing) which can make that sort
> of request happen for all my measurements in one go? The thing I'm
> probing is very able to handle a few extra thousand queries per second..
> and I'd only be hurting myself i get my estimate wrong :)

I was part of a team on a hackathon a while back which needed to compute
distances between points on the globe, including Atlas probes:

https://github.com/shane-kerr/ripe-atlas-anycast-work

I see that the openflights project has since moved to GitHub, and so you
can get the IATA airport data here:

https://github.com/jpatokal/openflights/blob/master/data/airports.dat


The geo data wasn't problematic for me, once I found:
  github.com/kelvins/geocoder

which apparently uses:
  https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat
 
oh hai! It looks like I didn't do it the 'insane way', mostly :)
(I should make the repository of code I wrote available, oops)

I guess I should update the README.md...

The repository is basically a last-point-in-time snapshot of the
hackathon work, so it a bit rough. 😬 Anyway, this code is probably the
closest to what you want:

https://github.com/shane-kerr/ripe-atlas-anycast-work/blob/master/add-dist.py


i think you are just sorting through measurement response data to find things near/around/at your points of interest here, right?
that seems useful.
 
Turning the great_circle_dist() function into a filter across the set of
all probes (meta-probes.json, which you can download in advance) should
get you something like what you want.


looks neat ;)
 
Note that this was all done 4 years ago according to GitHub, so maybe
Atlas provides better tools for this nowadays. 😊


they do have a method to request probes based on geo data and a radius from that point, which is what I am doing.
They also have filters (which I've apparently sucked at reading completely!) to limit based on type/ip-version/activity/etc.
mostly you can query for: "probes near here with these sets of attributes" which works pretty well ;)

my main problem was/is (I think) that I want to say:
  "For these M metros, please find 10 (or so) probes with working address family (v4||v6) and probe dns server (V4||V6 address) with a (non-)authoritative request"
 
my M number is ~70+.
I didn't want (for some reason?) to try and gather the probes across the globe into less measurements but more probes... huh, I could try that I guess, then I'd only have 4 measurements not M*4.

Cheers,

--
Shane