New on RIPE Labs: Another Look at RIPE Atlas Probe Lifetimes
Dear colleagues, We took another, more detailed look at probe lifetimes and the dynamics of probes connecting and disconnecting from the RIPE Atlas infrastructure to try to understand how to keep the network growing in the long term. https://labs.ripe.net/Members/wilhelm/another-look-at-ripe-atlas-probe-lifetimes?pk_campaign=labs&pk_kwd=list-mat Kind regards, Mirjam Kuehne RIPE NCC
maybe virtual vm probes might be useful to investigate and also a split file system for v3 probs with /var/log on a different file system ? Colin
On 25 Aug 2016, at 15:30, Mirjam Kuehne <mir@ripe.net> wrote:
Dear colleagues,
We took another, more detailed look at probe lifetimes and the dynamics of probes connecting and disconnecting from the RIPE Atlas infrastructure to try to understand how to keep the network growing in the long term.
Kind regards, Mirjam Kuehne RIPE NCC
Hi, i noticed another problem besides the hardware, the "controllers" are not really highly-available? (E.g. ctr-ams07, NL) Sometimes the probes are disconnected from a RIPE controller for some time even though there is no absolutely no network issue at the probes site. I suppose this could happen from time to time but it's too often. (Maybe it's just me, maybe everyone should check the connection history at RIPE atlas). Also if a controller is not working sometimes it seems to take forever until the probe will use a failover controller and in the meanwhile its "disconnected"... Any chance to improve the availability? Best Regards Max M. On 25.08.2016 16:37, Colin Johnston wrote:
maybe virtual vm probes might be useful to investigate and also a split file system for v3 probs with /var/log on a different file system ?
Colin
On 25 Aug 2016, at 15:30, Mirjam Kuehne <mir@ripe.net> wrote:
Dear colleagues,
We took another, more detailed look at probe lifetimes and the dynamics of probes connecting and disconnecting from the RIPE Atlas infrastructure to try to understand how to keep the network growing in the long term.
Kind regards, Mirjam Kuehne RIPE NCC
Hello, Some clarifications below. On 2016-08-25 16:51, Max Mühlbronner wrote:
Hi,
i noticed another problem besides the hardware, the "controllers" are not really highly-available? (E.g. ctr-ams07, NL)
They are not individually highly-available, but we have many of them for redundancy. So if some of them go down, only a portion of the network is affected.
Sometimes the probes are disconnected from a RIPE controller for some time even though there is no absolutely no network issue at the probes site. I suppose this could happen from time to time but it's too often. (Maybe it's just me, maybe everyone should check the connection history at RIPE atlas).
Every now and then, the NCC network has a hickup, or scheduled network maintenance. This can break the connections of the probes currently connected to us. We have controllers in other networks too (in Germany, US east/west and Singapore) which of course can have similar issues. As you probably know, probes keep an open TCP (SSH) connection to the controller. If any part of the network between the probe and the controller fails hard enough, there is a disconnection. In other words, probe disconnection can be caused by issues in the probe's network, in ours, or in anything in between :)
Also if a controller is not working sometimes it seems to take forever until the probe will use a failover controller and in the meanwhile its "disconnected"... Any chance to improve the availability?
Generally speaking, probes try hard to get back to the same controller they were connected to before disconnection. If this does not succeed for two hours, then they connect to the reg.servers ("trust anchors" if you will) to ask for a new controller. In the meantime they continue to measure and buffer results -- though they indeed show up as disconnected and therefore can't participate in the newest measurements. We have this mechanism in place to avoid probes jumping around from controller to controller in case there's a glitch somewhere. Hope this clarifies, Robert
Best Regards
Max M.
On 25.08.2016 16:37, Colin Johnston wrote:
maybe virtual vm probes might be useful to investigate and also a split file system for v3 probs with /var/log on a different file system ?
Colin
On 25 Aug 2016, at 15:30, Mirjam Kuehne <mir@ripe.net> wrote:
Dear colleagues,
We took another, more detailed look at probe lifetimes and the dynamics of probes connecting and disconnecting from the RIPE Atlas infrastructure to try to understand how to keep the network growing in the long term.
Kind regards, Mirjam Kuehne RIPE NCC
Hi, On Thu, Aug 25, 2016 at 03:37:06PM +0100, Colin Johnston wrote:
maybe virtual vm probes might be useful
You're not giving up on this, are you? Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
On Aug 25, 2016, at 4:46 PM, Gert Doering <gert@space.net> wrote:
On Thu, Aug 25, 2016 at 03:37:06PM +0100, Colin Johnston wrote:
maybe virtual vm probes might be useful
You're not giving up on this, are you?
It’s way easier for me to spin up 25 virtual probes than 25 physical ones :) Hoping that virtual probe is making progress :) - Jared
It’s way easier for me to spin up 25 virtual probes than 25 physical ones :)
it is even easier to spin up 1,000 prngs
participants (7)
-
Colin Johnston -
Gert Doering -
Jared Mauch -
Max Mühlbronner -
Mirjam Kuehne -
Randy Bush -
Robert Kisteleki