RIPE Atlas / UDM as a replacement for TTM
Hi, things have been strangely silent on the topic of "the end of the TTM boxes" after the last RIPE meeting - so let me try to stir this somewhat. I've been told that UDMs in Atlas "can do what I need" (use RIPE measurements to prove to our customers that our Internet reachability is good). It might, but I find it surprising painful, as I can only request pings from "10 probes at a time" in the UDM - and for reasonable accuracy, we'd need something of at least the size of TTM (100+ probes), or *better*, as we know that individual Atlas probes are not expected to have the same network stability as a TTM box ("always up. ALWAYS!") - setting up 30 or more individual UDMs to get 150 clients each to ping our two TTM boxes is... not the way to go. Second, I'm a bit confused how to automatedly download the .json-format UDM results - while I'm logged in, clicking on the link gives me a nice .json download, but when using the same link with wget, I get a login page. I'm not overly willing to download 30 UDM result files manually every day - so there must be some way to use "wget" etc. to get these data files... any hints on that? Example link: https://udm01.atlas.ripe.net/atlas/udm_download.json?attach=1&msm_id=1002000&y=2012&w=25 Third, we still need reasonable ping targets in our network, that are sort of "not under our control" (to meet the goal of "the RIPE NCC does these measurements, they are neutral, so we're not faking anything here") - and that are not part of the normal network churn that www.space.net or one of our routers might have. So far we use the TTM boxes for that - but they might die of old age some day... Which brings back the topic of the Atlas Anchor boxes... any news on that? Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (89) 32356-444 USt-IdNr.: DE813185279
In your letter dated Mon, 18 Jun 2012 15:53:26 +0200 you wrote:
I've been told that UDMs in Atlas "can do what I need" (use RIPE measurements to prove to our customers that our Internet reachability is good). It might, but I find it surprising painful, as I can only request pings from "10 probes at a time" in the UDM - and for reasonable accuracy, we'd need something of at least the size of TTM (100+ probes), or *better*, as we know that individual Atlas probes are not expected to have the same network stability as a TTM box ("always up. ALWAYS!") - setting up 30 or more individual UDMs to get 150 clients each to ping our two TTM boxes is... not the way to go.
These limits are supposed to be there temporary. They are there mainly to avoid a rush from probe hosts with losts of credits to overload the system. Ultimately, the idea is that should be able to spend the credits you have. If you want your limits raised, just ask.
Second, I'm a bit confused how to automatedly download the .json-format UDM results - while I'm logged in, clicking on the link gives me a nice .json download, but when using the same link with wget, I get a login page. I'm not overly willing to download 30 UDM result files manually every day - so there must be some way to use "wget" etc. to get these data files... any hints on that?
Example link: https://udm01.atlas.ripe.net/atlas/udm_download.json?attach=1&ms m_id=1002000&y=2012&w=25
There is an example at the bottom of <https://atlas.ripe.net/doc/api> that explains how to go past ripe-access in python.
Third, we still need reasonable ping targets in our network, that are sort of "not under our control" (to meet the goal of "the RIPE NCC does these measurements, they are neutral, so we're not faking anything here")
It is not clear to me how a ping target in your network can fake anything. A probe in your network is a different story, but a target? Probes just report how quickly and reliably they can access the target. If have no idea how you could fake that by controlling the target.
- and that are not part of the normal network churn that www.space.net or one of our routers might have. So far we use the TTM boxes for that - but they might die of old age some day... Which brings back the topic of the Atlas Anchor boxes... any news on that?
I leave it to somebody else to comment on the anchor boxes. They will come, but when and where, I have no idea.
Second, I'm a bit confused how to automatedly download the .json-format UDM results - while I'm logged in, clicking on the link gives me a nice .json download, but when using the same link with wget, I get a login page. I'm not overly willing to download 30 UDM result files manually every day - so there must be some way to use "wget" etc. to get these data files... any hints on that?
Example link: https://udm01.atlas.ripe.net/atlas/udm_download.json?attach=1&ms m_id=1002000&y=2012&w=25
There is an example at the bottom of <https://atlas.ripe.net/doc/api> that explains how to go past ripe-access in python.
FWIW, the same design pattern works with Perl's WWW::Mechanize <http://search.cpan.org/dist/WWW-Mechanize/lib/WWW/Mechanize.pm>
On 18.06.2012, at 16:23, Philip Homburg wrote:
It is not clear to me how a ping target in your network can fake anything. A probe in your network is a different story, but a target? Probes just report how quickly and reliably they can access the target. If have no idea how you could fake that by controlling the target.
- and that are not part of the normal network churn that www.space.net or one of our routers might have. So far we use the TTM boxes for that - but they might die of old age some day... Which brings back the topic of the Atlas Anchor boxes... any news on that?
I leave it to somebody else to comment on the anchor boxes. They will come, but when and where, I have no idea.
I fully understand the value of being able to say "All the measurement infrastructure including the targets is managed by a neutral and impartial third party". Coming back to the anchor boxes: the idea of RIPE Atlas anchors is to have a bigger probe for deployment inside network infrastructure. This will complement the standard probes that are typically deployed nearer to the edge of the network. Atlas Anchors will have a higher capacity and more bandwidth, so they will be able to do more measurements than the standard probe. In addition to being high capacity probes Atlas Anchors will also serve as targets for active measurements: they will be well-known, willing and cooperating targets for active measurements from the standard probes. The active measurement traffic from the standard probes can be monitored at the Anchors. One possible use of that is to diagnose asymmetries in the path. We will not get one-way delay back because of the lack of GPS time, but some of the benefits of one-way will be possible. We are currently working out the details and a plan for an initial deployment. In particular we want to avoid the mistakes we made with TTM, specifically in the hardware/software life-cycle area. The straw-man currently looks like this: - RIPE NCC members only - hardware bought and owned by the host - initially a very specific but widely available configuration including OOB access etc. - hardware capable of supporting additional services, like f.i. a local K-root instance - hardware installed and network configured by host - once hardware installed and connected, RIPE NCC takes over operations (like Gert wants ;-) - no GPS clock, but as accurate NTP as possible - formal agreement about hosting and operations - agreed hardware replacement cycle We are evaluating widely available hardware for this and developing the probe software. We will ask for beta testers after the summer. In the meantime all input about this is welcome! Does this answer your questions Gert? Daniel
Hi, On Tue, Jun 19, 2012 at 02:44:35PM +0200, Daniel Karrenberg wrote:
Coming back to the anchor boxes: the idea of RIPE Atlas anchors is to have a bigger probe for deployment inside network infrastructure. This will complement the standard probes that are typically deployed nearer to the edge of the network. Atlas Anchors will have a higher capacity and more bandwidth, so they will be able to do more measurements than the standard probe.
This was part of the idea, which I didn't write down in my initial mail. We're also prepared to shell out some money for the Atlas Anchor boxes - after all, we have budget for two TTM boxes, and this can be re-used here (read between the lines: two Anchor boxes should not require a significantly bigger yearly budget than two TTM boxes :-) ). [..]
We are currently working out the details and a plan for an initial deployment. In particular we want to avoid the mistakes we made with TTM, specifically in the hardware/software life-cycle area. The straw-man currently looks like this:
- RIPE NCC members only - hardware bought and owned by the host - initially a very specific but widely available configuration including OOB access etc. - hardware capable of supporting additional services, like f.i. a local K-root instance - hardware installed and network configured by host - once hardware installed and connected, RIPE NCC takes over operations (like Gert wants ;-) - no GPS clock, but as accurate NTP as possible - formal agreement about hosting and operations - agreed hardware replacement cycle
Fine with me :-) I think what is additionally needed as part of the formal agreement is how much bandwidth the host is willing to invest (call it "sponsor" or call it "trade for Atlas credits" :-) ).
Does this answer your questions Gert?
Yes, thanks! (And I'll now go and entertain myself with HTTP::Mechanize...) Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (89) 32356-444 USt-IdNr.: DE813185279
Hi, On Mon, Jun 18, 2012 at 04:23:19PM +0200, Philip Homburg wrote:
In your letter dated Mon, 18 Jun 2012 15:53:26 +0200 you wrote:
I've been told that UDMs in Atlas "can do what I need" (use RIPE measurements to prove to our customers that our Internet reachability is good). It might, but I find it surprising painful, as I can only request pings from "10 probes at a time" in the UDM - and for reasonable accuracy, we'd need something of at least the size of TTM (100+ probes), or *better*, as we know that individual Atlas probes are not expected to have the same network stability as a TTM box ("always up. ALWAYS!") - setting up 30 or more individual UDMs to get 150 clients each to ping our two TTM boxes is... not the way to go.
These limits are supposed to be there temporary. They are there mainly to avoid a rush from probe hosts with losts of credits to overload the system.
Ultimately, the idea is that should be able to spend the credits you have. If you want your limits raised, just ask.
Well, the paragraph above could have been seen as a question, but indeed it did not have question mark. So I ask: what's the process to install UDMs with a higher probe count? Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (89) 32356-444 USt-IdNr.: DE813185279
In your letter dated Tue, 19 Jun 2012 15:54:10 +0200 you wrote:
Ultimately, the idea is that should be able to spend the credits you have. If you want your limits raised, just ask.
Well, the paragraph above could have been seen as a question, but indeed it did not have question mark.
So I ask: what's the process to install UDMs with a higher probe count?
Get in touch with the Atlas group in one of many ways (mail somebody you know, create a ticket, etc.) and specify for which account, which limits you want raised to what values and optionally give a description of what you are trying to do. Once the limits are raised you can start the UDM yourself.
participants (4)
-
Daniel Karrenberg -
Gert Doering -
Philip Homburg -
Richard L. Barnes