September 2015 - ripe-atlas - mailman.ripe.net

NIX.CZ (Czech Republic) has joined RIPE Atlas anchors
by Michela Galante 24 Sep '15

24 Sep '15

Dear RIPE Atlas users, We're happy to announce that another RIPE Atlas anchor has joined the network and is now available as a target for probe hosts conducting their own user-defined measurements. The new anchor is called cz-prg-as6881 and it is hosted by NIX.CZ in Prague, Czech Republic. Anchoring measurements (ping, ping6, traceroute and traceroute6) are scheduled towards this anchor from all the other anchors as well as several hundred probes, providing a continual overview of regional reachability. Results of these measurements are available as raw data in JSON format, and visualised on the map and "seismograph" with many configurable features. You can access the anchoring measurements for all anchors at: https://atlas.ripe.net/anchors/list/ Learn more about RIPE Atlas anchors at: https://atlas.ripe.net/about/anchors/ Thank you for participating in RIPE Atlas! Kind Regards, RIPE Atlas Team atlas(a)ripe.net

1 0

Downloading ad-hoc Atlas measurements as JSON
by Jonathan Brewer 24 Sep '15

24 Sep '15

Hi All, One of my current frustrations is that when downloading ad-hoc Atlas measurements, the file name is always the same: "RIPE-Atlas-measurement.json". This is unfortunate. It would be great if the web server would by default call this measurement by its ID. Anyone else experiencing this problem? Regards, Jon

5 6

Maintenance works today
by Robert Kisteleki 24 Sep '15

24 Sep '15

Dear All, We'll carry out maintenance work on the core RIPE Atlas infrastructure today, approximately between 10-11h Amsterdam time (8-9 UTC). This will also affect the API/UI, most likely resulting in failed API calls or web page loads. Excuses for the inconvenience. Regards, Robert Kisteleki

1 0

RIPE Atlas Tools and Python versions
by Daniel Quinn 24 Sep '15

24 Sep '15

As some of you may already know, we’ve been working on a new command-line toolkit that will let you do nifty things like: * List measurement and probe metadata * Generate reports from measurement results * Stream measurement results It’s coming along quite nicely, but I’m running into just too many cases where support for Python 2.6 is (a) making my job harder, and (b) making my code uglier. Given that (a) means I spend more time fixing compatibility rather than adding features and (b) just makes me sad, I wanted to ask the community how important Python 2.6 is to you. So here’s the question: Of those of you who would find such a tool interesting/useful, how many of you *only* have access to a computer running Python 2.6? Such systems include CentOS distributions earlier than 7, and corresponding RedHat Enterprise systems. If you’re using Debian, or Ubuntu, you should already have 2.7 by default, and if you’re on Gentoo or Arch, you probably have both 2.7 and 3.5 already. I understand that Apple’s OSX is running 2.7 these days as well. If I don’t get a wave of messages about how very important 2.6 is to your collective health and wellbeing, I’m going to consider this a non-issue and drop support for 2.6. Thanks for your input everyone :-)

5 5

Friday's events on RIPE Atlas
by Romeo Zwart 23 Sep '15

23 Sep '15

Dear all, Here is a description of the events that took place last Friday and some 'lessons learned' that we took away from it. The high level summary of events is that an Atlas user was authorised to create an extreme number of measurements involving a large number of probes. This effectively overloaded the back-end machinery in various different ways. Even though this event could only happen because of the exception in resource use limits, we have implemented workarounds and countermeasures to avoid a repetition in future, and we will be investigating some of the more fundamental issues in the coming period. More information is available below for those interested. Observations: - Problems started shortly before 11am when one of the Atlas users created a large number of new measurements each involving all available probes. The user had been given an exceptional amount of credits as part of a special experiment. Therefore the normal limitations on the impact any individual user can have on the system were not active when the measurements were created and activated. - The results of the newly created measurements put a lot of strain on the measurement scheduler, which triggered our interest. After some investigation the cause of the overload was identified and the related measurements were ended. - However, by this time the majority of results, up to that moment, had already reached our queuing servers and the consumers were already ingesting the results into our Hadoop storage platform. - At this phase we discovered a capacity problem with the process that consumes the Atlas results, so we doubled the capacity of that component on the fly. - This exposed the next bottleneck in our platform in the form of an accumulation of the created results on a very small number of processing nodes. Normally the incoming measurement results are distributed over several storage nodes, so this strongly reduced the consumption rate of new data. - A third factor that contributed was the fact that, in attempt to curb growth of the Atlas data, we have migrated the Atlas data sets to a more efficient compression algorithm earlier in the year. This saved us some 40-50% of storage space for the Atlas data, at the expense of some compute power. Under normal circumstances, even at high loads, this compute power is abundantly available on the storage cluster. Under the specific circumstances of last Friday's events, it turned out that the change of the compression algorithm had increased processing time for some Hadoop system tasks by up to a factor of 8, which had a direct impact on the data consumption speed. Immediate actions taken: - Removed special privileges of the end-user in question - Added capacity to the Atlas consumer processes - Returned (temporarily) to less efficient compression on the Atlas data sets. Lessons learned and further planned action: - Granting special privileges for some of the Atlas users needs (even) more attention than it already receives. - We need to better communicate "best practices" to these power users so they can use their extra allowances responsibly. - Improved compression of Atlas data has decreased our storage demands but also decreased our processing capacity. This needs further investigation to find the optimum configuration. - Investigate possibilities to better spread incoming results over more worker nodes (reduce hotspots). - Investigate and quantify reasonable boundaries of scalability of the whole system, to guide the limits for granting credits to end users. Kind regards, Romeo Zwart

10 15

Booking.com IT Services, Singapore has joined RIPE Atlas anchors
by Valentino Dijkstra 22 Sep '15

22 Sep '15

Dear RIPE Atlas users, We're happy to announce that another RIPE Atlas anchor has joined the network and is now available as a target for probe hosts conducting their own user-defined measurements. The new anchor is called sg-sin-as43996 and is hosted by Booking.com <http://booking.com/> IT Services, Singapore. Anchoring measurements (ping, ping6, traceroute and traceroute6) are scheduled towards this anchor from all the other anchors as well as several hundred probes, providing a continual overview of regional reachability. Results of these measurements are available as raw data in JSON format, and visualised on the map and "seismograph" with many configurable features. You can access the anchoring measurements for all anchors at: https://atlas.ripe.net/anchors/list/ <https://atlas.ripe.net/anchors/list/> Learn more about RIPE Atlas anchors at: https://atlas.ripe.net/about/anchors/ <https://atlas.ripe.net/about/anchors/> Thank you for participating in RIPE Atlas! Kind regards, RIPE Atlas Team RIPE NCC atlas(a)ripe.net <mailto:atlas@ripe.net>

1 0

Booking.com IT Services, Singapore, as joined RIPE Atlas anchors
by Valentino Dijkstra 22 Sep '15

22 Sep '15

Dear RIPE Atlas users, We're happy to announce that another RIPE Atlas anchor has joined the network and is now available as a target for probe hosts conducting their own user-defined measurements. The new anchor is called nz-lyj-as17746 and is hosted by Booking.com IT Services, Singapore. Anchoring measurements (ping, ping6, traceroute and traceroute6) are scheduled towards this anchor from all the other anchors as well as several hundred probes, providing a continual overview of regional reachability. Results of these measurements are available as raw data in JSON format, and visualised on the map and "seismograph" with many configurable features. You can access the anchoring measurements for all anchors at: https://atlas.ripe.net/anchors/list/ <https://atlas.ripe.net/anchors/list/> Learn more about RIPE Atlas anchors at: https://atlas.ripe.net/about/anchors/ <https://atlas.ripe.net/about/anchors/> Thank you for participating in RIPE Atlas! Kind regards, RIPE Atlas Team RIPE NCC atlas(a)ripe.net <mailto:atlas@ripe.net>

1 0

[rfc-editor@rfc-editor.org: [lmap] RFC 7594 on A Framework for Large-Scale Measurement of Broadband Performance (LMAP)]
by Stephane Bortzmeyer 21 Sep '15

21 Sep '15

We just need "millions" of Atlas to implement this vision :-)

1 0

Data delays on RIPE Atlas
by Robert Kisteleki 19 Sep '15

19 Sep '15

Dear all, The data backend behind RIPE Atlas is experiencing some issues at the moment. This causes delays in processing and serving the data, meaning all results are delivered late. We're working on a fix. Excuses for the inconvenience. Regards, Robert

3 4

Strange 500 submission error
by Stephane Bortzmeyer 18 Sep '15

18 Sep '15

Atlas' API sends me back HTML pages :-) RIPEAtlas.RequestSubmissionError: Status 500, reason "<!DOCTYPE html> <html> <head> <meta charset="UTF-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge" /> <meta content="RIPE Atlas, RIPE, Atlas" name="description" /> <link rel="stylesheet" type="text/css" media="screen" href="https://www-static.ripe.net/static/common/fonts/open-sans.css" /> <link rel="stylesheet" type="text/css" media="print" href="https://www.ripe.net/portal_css/Ripe/print.css" /> <link href="https://www.ripe.net/++resource++collective.js.prettify.stylesheets/prettif…" media="screen" type="text/css" rel="stylesheet" /> <link rel="stylesheet" type="text/css" media="screen" href="https://www.ripe.net/portal_css/Ripe/++resource++ripe.plonetheme.stylesheet…" /> <link rel="stylesheet" type="text/css" media="screen" href="https://www.ripe.net/portal_css/Ripe/++resource++ripe.plonetheme.stylesheet…" /> <link rel="stylesheet" type="text/css" media="screen" href="https://www.ripe.net/portal_css/Ripe/++resource++ripe.plonetheme.stylesheet…" />  <title> - RIPE Atlas — RIPE Network Coordination Centre</title> <link rel="shortcut icon" type="image/x-icon" href="https://www.ripe.net/favicon.ico" /> <link rel="apple-touch-icon" href="https://www.ripe.net/touch_icon.png" /> <link rel="search" href="https://www.ripe.net/search" title="Search this site" /> ...

5 7