- ripe-atlas - mailman.ripe.net

RIPE Atlas Tools and Python versions
by Daniel Quinn 24 Sep '15

24 Sep '15

As some of you may already know, we’ve been working on a new command-line toolkit that will let you do nifty things like: * List measurement and probe metadata * Generate reports from measurement results * Stream measurement results It’s coming along quite nicely, but I’m running into just too many cases where support for Python 2.6 is (a) making my job harder, and (b) making my code uglier. Given that (a) means I spend more time fixing compatibility rather than adding features and (b) just makes me sad, I wanted to ask the community how important Python 2.6 is to you. So here’s the question: Of those of you who would find such a tool interesting/useful, how many of you *only* have access to a computer running Python 2.6? Such systems include CentOS distributions earlier than 7, and corresponding RedHat Enterprise systems. If you’re using Debian, or Ubuntu, you should already have 2.7 by default, and if you’re on Gentoo or Arch, you probably have both 2.7 and 3.5 already. I understand that Apple’s OSX is running 2.7 these days as well. If I don’t get a wave of messages about how very important 2.6 is to your collective health and wellbeing, I’m going to consider this a non-issue and drop support for 2.6. Thanks for your input everyone :-)

5 5

Friday's events on RIPE Atlas
by Romeo Zwart 23 Sep '15

23 Sep '15

Dear all, Here is a description of the events that took place last Friday and some 'lessons learned' that we took away from it. The high level summary of events is that an Atlas user was authorised to create an extreme number of measurements involving a large number of probes. This effectively overloaded the back-end machinery in various different ways. Even though this event could only happen because of the exception in resource use limits, we have implemented workarounds and countermeasures to avoid a repetition in future, and we will be investigating some of the more fundamental issues in the coming period. More information is available below for those interested. Observations: - Problems started shortly before 11am when one of the Atlas users created a large number of new measurements each involving all available probes. The user had been given an exceptional amount of credits as part of a special experiment. Therefore the normal limitations on the impact any individual user can have on the system were not active when the measurements were created and activated. - The results of the newly created measurements put a lot of strain on the measurement scheduler, which triggered our interest. After some investigation the cause of the overload was identified and the related measurements were ended. - However, by this time the majority of results, up to that moment, had already reached our queuing servers and the consumers were already ingesting the results into our Hadoop storage platform. - At this phase we discovered a capacity problem with the process that consumes the Atlas results, so we doubled the capacity of that component on the fly. - This exposed the next bottleneck in our platform in the form of an accumulation of the created results on a very small number of processing nodes. Normally the incoming measurement results are distributed over several storage nodes, so this strongly reduced the consumption rate of new data. - A third factor that contributed was the fact that, in attempt to curb growth of the Atlas data, we have migrated the Atlas data sets to a more efficient compression algorithm earlier in the year. This saved us some 40-50% of storage space for the Atlas data, at the expense of some compute power. Under normal circumstances, even at high loads, this compute power is abundantly available on the storage cluster. Under the specific circumstances of last Friday's events, it turned out that the change of the compression algorithm had increased processing time for some Hadoop system tasks by up to a factor of 8, which had a direct impact on the data consumption speed. Immediate actions taken: - Removed special privileges of the end-user in question - Added capacity to the Atlas consumer processes - Returned (temporarily) to less efficient compression on the Atlas data sets. Lessons learned and further planned action: - Granting special privileges for some of the Atlas users needs (even) more attention than it already receives. - We need to better communicate "best practices" to these power users so they can use their extra allowances responsibly. - Improved compression of Atlas data has decreased our storage demands but also decreased our processing capacity. This needs further investigation to find the optimum configuration. - Investigate possibilities to better spread incoming results over more worker nodes (reduce hotspots). - Investigate and quantify reasonable boundaries of scalability of the whole system, to guide the limits for granting credits to end users. Kind regards, Romeo Zwart

10 15

Booking.com IT Services, Singapore has joined RIPE Atlas anchors
by Valentino Dijkstra 22 Sep '15

22 Sep '15

Dear RIPE Atlas users, We're happy to announce that another RIPE Atlas anchor has joined the network and is now available as a target for probe hosts conducting their own user-defined measurements. The new anchor is called sg-sin-as43996 and is hosted by Booking.com <http://booking.com/> IT Services, Singapore. Anchoring measurements (ping, ping6, traceroute and traceroute6) are scheduled towards this anchor from all the other anchors as well as several hundred probes, providing a continual overview of regional reachability. Results of these measurements are available as raw data in JSON format, and visualised on the map and "seismograph" with many configurable features. You can access the anchoring measurements for all anchors at: https://atlas.ripe.net/anchors/list/ <https://atlas.ripe.net/anchors/list/> Learn more about RIPE Atlas anchors at: https://atlas.ripe.net/about/anchors/ <https://atlas.ripe.net/about/anchors/> Thank you for participating in RIPE Atlas! Kind regards, RIPE Atlas Team RIPE NCC atlas(a)ripe.net <mailto:atlas@ripe.net>

1 0

Booking.com IT Services, Singapore, as joined RIPE Atlas anchors
by Valentino Dijkstra 22 Sep '15

22 Sep '15

Dear RIPE Atlas users, We're happy to announce that another RIPE Atlas anchor has joined the network and is now available as a target for probe hosts conducting their own user-defined measurements. The new anchor is called nz-lyj-as17746 and is hosted by Booking.com IT Services, Singapore. Anchoring measurements (ping, ping6, traceroute and traceroute6) are scheduled towards this anchor from all the other anchors as well as several hundred probes, providing a continual overview of regional reachability. Results of these measurements are available as raw data in JSON format, and visualised on the map and "seismograph" with many configurable features. You can access the anchoring measurements for all anchors at: https://atlas.ripe.net/anchors/list/ <https://atlas.ripe.net/anchors/list/> Learn more about RIPE Atlas anchors at: https://atlas.ripe.net/about/anchors/ <https://atlas.ripe.net/about/anchors/> Thank you for participating in RIPE Atlas! Kind regards, RIPE Atlas Team RIPE NCC atlas(a)ripe.net <mailto:atlas@ripe.net>

1 0

[rfc-editor@rfc-editor.org: [lmap] RFC 7594 on A Framework for Large-Scale Measurement of Broadband Performance (LMAP)]
by Stephane Bortzmeyer 21 Sep '15

21 Sep '15

We just need "millions" of Atlas to implement this vision :-)

1 0

Data delays on RIPE Atlas
by Robert Kisteleki 19 Sep '15

19 Sep '15

Dear all, The data backend behind RIPE Atlas is experiencing some issues at the moment. This causes delays in processing and serving the data, meaning all results are delivered late. We're working on a fix. Excuses for the inconvenience. Regards, Robert

3 4

Strange 500 submission error
by Stephane Bortzmeyer 18 Sep '15

18 Sep '15

Atlas' API sends me back HTML pages :-) RIPEAtlas.RequestSubmissionError: Status 500, reason "<!DOCTYPE html> <html> <head> <meta charset="UTF-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge" /> <meta content="RIPE Atlas, RIPE, Atlas" name="description" /> <link rel="stylesheet" type="text/css" media="screen" href="https://www-static.ripe.net/static/common/fonts/open-sans.css" /> <link rel="stylesheet" type="text/css" media="print" href="https://www.ripe.net/portal_css/Ripe/print.css" /> <link href="https://www.ripe.net/++resource++collective.js.prettify.stylesheets/prettif…" media="screen" type="text/css" rel="stylesheet" /> <link rel="stylesheet" type="text/css" media="screen" href="https://www.ripe.net/portal_css/Ripe/++resource++ripe.plonetheme.stylesheet…" /> <link rel="stylesheet" type="text/css" media="screen" href="https://www.ripe.net/portal_css/Ripe/++resource++ripe.plonetheme.stylesheet…" /> <link rel="stylesheet" type="text/css" media="screen" href="https://www.ripe.net/portal_css/Ripe/++resource++ripe.plonetheme.stylesheet…" />  <title> - RIPE Atlas — RIPE Network Coordination Centre</title> <link rel="shortcut icon" type="image/x-icon" href="https://www.ripe.net/favicon.ico" /> <link rel="apple-touch-icon" href="https://www.ripe.net/touch_icon.png" /> <link rel="search" href="https://www.ripe.net/search" title="Search this site" /> ...

5 7

Ripe-Atlas 500 Internal Server Error ?
by Max Mühlbronner 17 Sep '15

17 Sep '15

Hi, Atlas is not working right now. (500 Internal Server Eror) Same from different locations/networks. But i guess you already noticed? Best Regards -- Max Mühlbronner 42com Telecommunication GmbH Straße der Pariser Kommune 12-16 10243 Berlin E-Mail: mm(a)42com.com Web: www.42com.com Firmenangaben/Company information: Handelsregister/Commercial register: Amtsgericht Berlin HRB 99071 B Umsatzsteuer-ID/VAT-ID: DE223812306 Geschäftsführer/CEO: Thomas Reinig, Alexander Reinig Diese E-Mail enthält Informationen von 42com Telecommunication GmbH. Diese sind möglicherweise vertraulich und ausschließlich für den Adressaten bestimmt. Sollten Sie diese elektronische Nachricht irrtümlicherweise erhalten haben, so informieren Sie uns bitte unverzüglich telefonisch oder per E-Mail. This message is intended only for the use of the individual or entity to which it is addressed. If you have received this message by mistake, please notify us immediately.

2 1

Join our new webinar on "Advanced RIPE Atlas Usage"
by Vesna Manojlovic 17 Sep '15

17 Sep '15

Dear colleagues, We are pleased to announce our new webinar: Advanced RIPE Atlas Usage, aiming to teach LIRs how to use RIPE Atlas measurements for network monitoring and troubleshooting. https://www.ripe.net/support/training/learn-online/webinars/advanced-ripe-a… It's part of the series of interactive on-line learning sessions, that give you the opportunity to learn and interact with trainers without leaving your desk, as well as for hands-on practice, and to get your questions answered by developers. Learn how to: - Use RIPE Atlas measurements for network monitoring and troubleshooting - Use API calls for measurement creation - Integrate RIPE Atlas with existing monitoring systems There are three dates scheduled: 2nd July, 27th August & 17th September. Webinars start at 11:30 UTC, and take about one hour. Register now at: https://www.ripe.net/support/training/learn-online/webinars We look forward to seeing you online. Kind regards, Vesna Manojlovic RIPE NCC

4 6

TELUS in Montreal, Canada has joined RIPE Atlas anchors
by Valentino Dijkstra 16 Sep '15

16 Sep '15

Dear RIPE Atlas users, We're happy to announce that another RIPE Atlas anchor has joined the network and is now available as a target for probe hosts conducting their own user-defined measurements. The new anchor is called ca-mtr-as852 and is hosted by TELUS in Montreal, Canada. Anchoring measurements (ping, ping6, traceroute and traceroute6) are scheduled towards this anchor from all the other anchors as well as several hundred probes, providing a continual overview of regional reachability. Results of these measurements are available as raw data in JSON format, and visualised on the map and "seismograph" with many configurable features. You can access the anchoring measurements for all anchors at: https://atlas.ripe.net/anchors/list/ <https://atlas.ripe.net/anchors/list/> Learn more about RIPE Atlas anchors at: https://atlas.ripe.net/about/anchors/ <https://atlas.ripe.net/about/anchors/> Thank you for participating in RIPE Atlas! Kind regards, RIPE Atlas Team RIPE NCC atlas(a)ripe.net <mailto:atlas@ripe.net>

1 0