Turris Omnia as probe Platform?
Dear all, some of you are probably aware of the Turris Omnia project by CZ.NIC ( https://www.indiegogo.com/projects/turris-omnia-hi-performance-open-source-r... ). This might be an interesting platform for running an VM probe?! Just thought it might be worth brining it up here for discussion. Cheers Thomas
Hi, On Fri, Dec 04, 2015 at 09:09:14PM +0700, Kessler, Thomas wrote:
This might be an interesting platform for running an VM probe?!
I think there is no lack of "interesting platforms to host a VM", but the point of the discussion was "virtualizing measurements leads to unreliable results as you do not control the hardware well enough" - and for that reason, I'd strongly discourage *any* sort of VM solution. gert -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
Hello, I don't fully agree here. There're always so many things, which may involve measurement realiability just on first from the probe. You always believe, that "hardware" probe is more reliable. Not, it isn't - current probe itself is quite cheap piece of not-so-powerfull hardware. And if someone want's to trick measurement, he can - there're so many ways. You don't have any kind of control on network behind the probe - and that's more important part in terms of reliable measurements... Many thing are going virtual around us (even on high-end routing platforms). Virtualised probe can be more reliable in terms of computing power compared to small TP-Links & so-on. Turris itself is very powerfull hardware (and it's open, both HW & SW). Some kind of integration between Atlas & Turris is good idea in my oppinion. Requirement of hardware-probe is in 'old-school' point of view. And at least with v3 of probes, it's harder to keep probe alive (dying "low-cost" USB flash is big issue...). With regards, Daniel On 4.12.2015 17:39, Gert Doering wrote:
Hi,
On Fri, Dec 04, 2015 at 09:09:14PM +0700, Kessler, Thomas wrote:
This might be an interesting platform for running an VM probe?!
I think there is no lack of "interesting platforms to host a VM", but the point of the discussion was "virtualizing measurements leads to unreliable results as you do not control the hardware well enough" - and for that reason, I'd strongly discourage *any* sort of VM solution.
gert
On Fri, Dec 04, 2015 at 11:30:34PM +0100, Daniel Suchy wrote:
I don't fully agree here. There're always so many things, which may involve measurement realiability just on first from the probe. You always believe, that "hardware" probe is more reliable. Not, it isn't - current probe itself is quite cheap piece of not-so-powerfull hardware. And if someone want's to trick measurement, he can - there're so many ways. You don't have any kind of control on network behind the probe - and that's more important part in terms of reliable measurements...
The thing is "we perfectly well know the characteristics of the measuring device". Everything deviating from the norm will be caused by the network, not "something in the VM environment".
Many thing are going virtual around us (even on high-end routing platforms). Virtualised probe can be more reliable in terms of computing power compared to small TP-Links & so-on.
Computing power is totally unimportant as compared to reproduceability of a given measurement on a given probe. Of course a VM is more powerful than either of these probes, but does this make it more suitable to be part of Atlas? I say no, because for a measurement platform, "raw power" is not the key issue. Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
What you want from the probe is predictability in the form of minimal jitter and possibly some calibration (to equate latency). IF the Turris people can run the Atlas code with some sort of real-time scheduling then it is going to be OK as a probe. Can they? If it is a process competing with whatever else the router might be doing at the time then I agree, it wouldn’t be suitable as a probe for things that involve precise time measurements, although they could still run the set of measurements for which precise timing is not necessary (routing, DNS, etc). In the Atlas probe all software is under control of one party and *I guess* doesn’t interfere with the rest of the software running on them. The criteria should be reproducibility of the experiment, not whether it is a VM or something else. Just my opinion Joao
On 05 Dec 2015, at 17:27, Gert Doering <gert@space.net> wrote:
On Fri, Dec 04, 2015 at 11:30:34PM +0100, Daniel Suchy wrote:
I don't fully agree here. There're always so many things, which may involve measurement realiability just on first from the probe. You always believe, that "hardware" probe is more reliable. Not, it isn't - current probe itself is quite cheap piece of not-so-powerfull hardware. And if someone want's to trick measurement, he can - there're so many ways. You don't have any kind of control on network behind the probe - and that's more important part in terms of reliable measurements...
The thing is "we perfectly well know the characteristics of the measuring device". Everything deviating from the norm will be caused by the network, not "something in the VM environment".
Many thing are going virtual around us (even on high-end routing platforms). Virtualised probe can be more reliable in terms of computing power compared to small TP-Links & so-on.
Computing power is totally unimportant as compared to reproduceability of a given measurement on a given probe.
Of course a VM is more powerful than either of these probes, but does this make it more suitable to be part of Atlas? I say no, because for a measurement platform, "raw power" is not the key issue.
Gert Doering -- NetMaster -- have you enabled IPv6 on something today...?
SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
Hi! On Sat, Dec 5, 2015 at 6:02 PM, João Damas <joao@bondis.org> wrote:
What you want from the probe is predictability in the form of minimal jitter and possibly some calibration (to equate latency). IF the Turris people can run the Atlas code with some sort of real-time scheduling then it is going to be OK as a probe. Can they?
It depends on what level of control you need. The board will be supported by vanilla Linux kernel (we are almost there). So you/we can compile in whatever is needed. The hardware should be more than enough for routing (=processing input IRQs from NICs and forwarding) and almost everything else can be prioritized. But I suppose we need to discuss that into more detail to address all possible interactions. Is there some documentation of kernel tuning in the current probes? And the probe could perform self-tests periodically and disable itself either temporarily or permanently if the router fails the test.
If it is a process competing with whatever else the router might be doing at the time then I agree, it wouldn’t be suitable as a probe for things that involve precise time measurements, although they could still run the set of measurements for which precise timing is not necessary (routing, DNS, etc). In the Atlas probe all software is under control of one party and *I guess* doesn’t interfere with the rest of the software running on them.
I personally (though I am the Turris kernel guy) think that we can cooperate on determining interactions and tune the basic system for the probe, if needed. But I suppose that we won't be shipping the routers with the probe turned on by default anyway. From my point of view it would make sense to do more general work in this direction. It boils down to the question whether we can have the probe in form of a daemon? Of course there have to be some requirements for reliable operation, that can be enforced on the OS side and checked on the daemon side. But it does not make sense to think of them if the software probe is still deemed undesirable. Is it? Tomas
On 05 Dec 2015, at 19:29, tomas.hlavacek@nic.cz wrote:
Hi!
On Sat, Dec 5, 2015 at 6:02 PM, João Damas <joao@bondis.org> wrote:
What you want from the probe is predictability in the form of minimal jitter and possibly some calibration (to equate latency). IF the Turris people can run the Atlas code with some sort of real-time scheduling then it is going to be OK as a probe. Can they?
It depends on what level of control you need. The board will be supported by vanilla Linux kernel (we are almost there). So you/we can compile in whatever is needed.
Schedule determinism (some sort of real time kernel extension) : to send probe traffic in a predictable manner Accurate time-of-arrival stamping for incoming packets, so you can trust the measurement. Joao
I personally (though I am the Turris kernel guy) think that we can cooperate on determining interactions and tune the basic system for the probe, if needed. But I suppose that we won't be shipping the routers with the probe turned on by default anyway.
The probe's measurement source code is published on the website. We're not using any special kernel tweaks. Cheers, Robert
Computing power is totally unimportant as compared to reproduceability of a given measurement on a given probe.
100% with you there. but, for example, is fetching a dane tlsa or a browser cert gonna be bothered by the kinds of things which vms do to us? the problem with this path is that we fragment what measurements can be done on which probes, a major pain. randy
I think there is no lack of "interesting platforms to host a VM", but the point of the discussion was "virtualizing measurements leads to unreliable results as you do not control the hardware well enough" - and for that reason, I'd strongly discourage *any* sort of VM solution.
i strongly agree that vms risk reducing the precision of results. i am less sure this makes them useless or dangerous if clearly marked, as in probe version, perhaps. heck, if i want precision below ~15ms, i would be leery of v1 and v2 probes. otoh, nlring is mostly virtual and has its uses. but probes are so small and sufficiently cheap that i question the utility of maintaining a vm-based infrastructure. ymmv. randy, a researcher who uses atlas data, sometimes for timing sometimes not
participants (7)
-
Daniel Suchy -
Gert Doering -
João Damas -
Kessler, Thomas -
Randy Bush -
Robert Kisteleki -
tomas.hlavacek＠nic.cz