Hello Philip. Ping relies heavily on the system's CPU. That is why it is considered unreliable for measuring when you are pinging CPU loaded machines/devices. The CPU on the probe is a n ARM7TDMI running at 55MHz, based on information from this link: http://www.digi.com/products/wireless-wired-embedded-solutions/solutions-on-... Regards, Stelios Mersinas On Wed, Jun 20, 2012 at 11:48 AM, Philip Homburg <philip.homburg@ripe.net>wrote:

On 6/20/12 9:11 , Randy Bush wrote:

...

we want to use atlas probes in an experiment. being prudent (you can tell it was not i), we decided to try to get some basic calibration. one run was just on a local LAN.

three hosts on the same gige switch o probe 2285 o psg.com, a not very fancy or fast freebsd 9 box with intel/pro1000 gige ports o bbgp.psg.com, a funky older freebsd 9 box with bge gige

probe 2285 pinging bbgp.psg.com, average RTT: 1.5606994382 [0], number of pings: 356*3

psg.com pinging bbgp.psg.com, average RTT: 0.253424332344 [0], number of pings: 674

has anyone done similar probe calibration experiments? does anyone have any clue as to why the difference?

Just to confirm my suspicion, I tried to other way around:

This is an old AMD64 running FreeBSD pinging an Atlas probe on the same LAN: $ ping 130.37.15.50 PING 130.37.15.50 (130.37.15.50): 56 data bytes 64 bytes from 130.37.15.50: icmp_seq=0 ttl=64 time=2.515 ms 64 bytes from 130.37.15.50: icmp_seq=1 ttl=64 time=0.913 ms 64 bytes from 130.37.15.50: icmp_seq=2 ttl=64 time=0.915 ms 64 bytes from 130.37.15.50: icmp_seq=3 ttl=64 time=0.929 ms

And this is the same FreeBSD box pinging a Celeron 766 MHz, running a micro kernel operating system, also on the same LAN: $ ping prism PING prism.hq.phicoh.net (130.37.15.36): 56 data bytes 64 bytes from 130.37.15.36: icmp_seq=0 ttl=96 time=0.364 ms 64 bytes from 130.37.15.36: icmp_seq=1 ttl=96 time=0.210 ms 64 bytes from 130.37.15.36: icmp_seq=2 ttl=96 time=0.211 ms 64 bytes from 130.37.15.36: icmp_seq=3 ttl=96 time=0.214 ms

This does not involve any of the Atlas software, just the ucLinux kernel running on the probe.

My conclusion is: probes are just very slow.

They are fine for measuring multi millisecond delays on WAN links but not for sub-millisecond delays on local links.

On Wed, Jun 20, 2012 at 11:29 AM, Philip Homburg <philip.homburg@ripe.net> wrote:

I have to admit that we never investigated the extent of the jitter problem.

Maybe the RIPE NCC could investigate the idea of creating a worldwide network of high accuracy measurement nodes. Time stamping could happen at (network) hardware level (or alternatively at kernel level) and the time source could be GPS. Oh wait :-)

On 6/20/12 13:01 , Randy Bush wrote:

...

They are fine for measuring multi millisecond delays on WAN links but not for sub-millisecond delays on local links. some experiments care about jitter. we are seeing variance noticeably greater than bsd boxen.

Obviously the current Atlas probes will be worse than just about anything running on older Intel boxes. At the moment we don't have engineering targets for jitter. But it is certainly worth putting on the wish list. That requires some kind of project description, i.e. what kind of variance are you trying to measure and what kind of jitter is the probe allowed to have before the measurement is spoiled. Keeping jitter low conflicts with doing many experiments on a single underpowered probe. So it won't be easy.

@stelios: yes icmp goes the slow path. but atlas has a very constrained measurment model, and it seems to be pretty much based on icmp.

I could be wrong, but as far as I know, routers don't have any kind of 'ping' feature in the fast path. But if you have suggestions for protocols that work better than ICMP ECHO then please let us know. As far as I know, a dedicated (x86) box that just acts as a ping target is best for round trip latency measurements.