Categorize probes (feature request)
Hi! Depending on the measurement, sometimes I want a realistic "user's view" which means using probes in residential areas, and sometimes I want a "backbone" view, which mean using probes in the back-bone (good connected) e.g. to measure my access network or my servers without the delay introduced by the probes access network. Thus, it would be cool if the probe selection algorithm allows also to choose "residential" probes or "backbone" probes. Maybe this categorization can be done automatically, or probe owners can be asked to specify the probes connectivity manually. Thanks, Klaus
On 8/11/13, 18:03 , Klaus Darilion wrote:
Hi!
Depending on the measurement, sometimes I want a realistic "user's view" which means using probes in residential areas, and sometimes I want a "backbone" view, which mean using probes in the back-bone (good connected) e.g. to measure my access network or my servers without the delay introduced by the probes access network.
A kind of rough classification is asked when you register a probe, but apparently it cannot be changed later on, or used for filtering. It would be an interesting feature though. If it becomes an option, please allow multiple choices: currently the choices are exclusive. regards, Gilles
Hi Klaus, Gilles, thank you for describing the feature that you would like to see. We already have this on the roadmap, listed under "More control over probe selection - Enable choice of probes based on the type of connection": http://roadmap.ripe.net/ripe-atlas/ However, as you can see on the roadmap, we have many features that we are working on, and few more in the planning; also, during RIPE meeting we received few more that will make it to the "requested" column... Posting your wish on the mailing list will raise the priority of your requested feature, so, everyone, please keep on doing that! You can expect that we will get to work on this "soon", but I can't give you any time estimates yet on when will this be implemented. Kind regards, Vesna On 08-nov.-13 20:26, Gilles Massen wrote:
On 8/11/13, 18:03 , Klaus Darilion wrote:
Hi!
Depending on the measurement, sometimes I want a realistic "user's view" which means using probes in residential areas, and sometimes I want a "backbone" view, which mean using probes in the back-bone (good connected) e.g. to measure my access network or my servers without the delay introduced by the probes access network.
A kind of rough classification is asked when you register a probe, but apparently it cannot be changed later on, or used for filtering. It would be an interesting feature though.
If it becomes an option, please allow multiple choices: currently the choices are exclusive.
regards, Gilles
Hi, On Mon, Nov 11, 2013 at 11:49:57AM +0100, Vesna Manojlovic wrote:
Posting your wish on the mailing list will raise the priority of your requested feature, so, everyone, please keep on doing that!
- restart an UDM after it has stopped - change parameters of an UDM and restart it, keeping the same ID (usage case: test run an UDM with just a few probes, develop the software to fetch and evaluate the json result data, then set the number of probes / number of packets / frequency / ... to what the final measurement should be, without having to change the ID in the grabbing software - especially if tweaking parameters for a couple of rounds this gets inconvenient) Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
On 11.11.2013 12:12, Gert Doering wrote: > Hi, > > On Mon, Nov 11, 2013 at 11:49:57AM +0100, Vesna Manojlovic wrote: >> Posting your wish on the mailing list will raise the priority of your >> requested feature, so, everyone, please keep on doing that! > - restart an UDM after it has stopped > > - change parameters of an UDM and restart it, keeping the same ID > > (usage case: test run an UDM with just a few probes, develop the > software to fetch and evaluate the json result data, then set > the number of probes / number of packets / frequency / ... to > what the final measurement should be, without having to change the > ID in the grabbing software - especially if tweaking parameters for > a couple of rounds this gets inconvenient) That's one reason why I switched to the REST API regards Klaus
On 11.11.2013 12:21, Klaus Darilion wrote: > > On 11.11.2013 12:12, Gert Doering wrote: >> Hi, >> >> On Mon, Nov 11, 2013 at 11:49:57AM +0100, Vesna Manojlovic wrote: >>> Posting your wish on the mailing list will raise the priority of your >>> requested feature, so, everyone, please keep on doing that! >> - restart an UDM after it has stopped >> >> - change parameters of an UDM and restart it, keeping the same ID >> >> (usage case: test run an UDM with just a few probes, develop the >> software to fetch and evaluate the json result data, then set >> the number of probes / number of packets / frequency / ... to >> what the final measurement should be, without having to change the >> ID in the grabbing software - especially if tweaking parameters for >> a couple of rounds this gets inconvenient) > That's one reason why I switched to the REST API To be more verbose: My "add measurement" script writes the ID to a file, and my "analyze" script reads the ID from this file. regards Klaus
Hi, On Mon, Nov 11, 2013 at 12:22:59PM +0100, Klaus Darilion wrote:
To be more verbose: My "add measurement" script writes the ID to a file, and my "analyze" script reads the ID from this file.
I could do that, but I don't want that - I'm not adding new measurements often enough to make it worthwile to add a script for that (the web ui is nice enough). Almost all I need should be there today, it's just a matter of priorization - and as Vesna said "voice your wishes", here I am :-) Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
Hi, I do have some wishes. - configurable ping timeout up to 30+ seconds. Or possibly 5 at least. - contrains for selecting probes so that one can configure pings from 2 probes on every AS. - tcp ping to check connectivity to places that do not accept pings - longer time-out for the atlas website. Is it 2 hours now? Could it be 12 please? - take firewalled probes out of the pool for new UDMs. I often get 2 probes out of 10 that can not reach anything. If probe can nog reach most root servers, it's pretty useless. - stop - modify - start. Of UDMs Thanks, Dave.
Hi, My one wish (for now :) )... Allow me to disable the processing of RA's on IPv6, so that I can configure IPv6 statically and not have it overwritten by an RA. Thanks, Mike.
Hi, On 2013/11/11 19:32 , Mike. wrote:
Allow me to disable the processing of RA's on IPv6, so that I can configure IPv6 statically and not have it overwritten by an RA.
This feature has been requested by a handful of people. However, this introduces more complexity in the already way too complex network configuration of the probes. So we like to avoid implementing this feature. Of course if there is wide spread demand for it, then we will look into it. I should try it sometime, but in theory one way to give the static address priority is to set all prefixes in the RA to deprecated. On a 'normal' network where you one prefix, it should not make a difference for any of the other hosts. Philip
On Fri, 22 Nov 2013 11:08:39 +0100 Philip Homburg <philip.homburg@ripe.net> wrote:
I should try it sometime, but in theory one way to give the static address priority is to set all prefixes in the RA to deprecated. On a 'normal' network where you one prefix, it should not make a difference for any of the other hosts.
It will make them prefer IPv4 over IPv6 for dual-stack destinations. # curl -vI google.com * About to connect() to google.com port 80 (#0) * Trying 2a00:1450:4001:808::1004... ... # ip addr change 2001:470:xxxxxxxx preferred_lft 0 dev wlan0 # curl -vI google.com * About to connect() to google.com port 80 (#0) * Trying 173.194.112.131.. ... -- With respect, Roman
I've said this before, although perhaps not in the right venue: I second (third, etc) the request for configurable (longer) ping timeouts. -Rusty On Mon, Nov 11, 2013 at 1:24 PM, dave <gboonie@gmail.com> wrote:
Hi,
I do have some wishes. - configurable ping timeout up to 30+ seconds. Or possibly 5 at least. - contrains for selecting probes so that one can configure pings from 2 probes on every AS. - tcp ping to check connectivity to places that do not accept pings - longer time-out for the atlas website. Is it 2 hours now? Could it be 12 please? - take firewalled probes out of the pool for new UDMs. I often get 2 probes out of 10 that can not reach anything. If probe can nog reach most root servers, it's pretty useless. - stop - modify - start. Of UDMs
Thanks,
Dave.
Hey, good to see the recent flurry of feature requests. I take it as an indication of active usage. Having become somewhat of a power user myself recently I have a request too. Let me suggest a little more structured way to make these requests. This would help everyone, including the developers, to understand them better. Also a snazzy title will make it easier to identify requests in the road map. Here is my first one: Title: Regularly Publish Metadata Description: Regularly publish a snapshot of all metadata, such as https://atlas.ripe.net/contrib/active_probes.json and https://atlas.ripe.net/atlas/publicprobesgrid.json?start=XX&limit=XX&sort=XX&dir=XX A period of once every 24 hours would be sufficient. Use: When analysing past measurements it is often required to refer to the state of RIPE Atlas at the time of the measurement. Referring to the current state will be less and less useful as the data analysed is more and more in the past. Changes in the meta data will make the results confusing at best and misleading at worst. Assumed complexity: Low, just dump it once a day and make it available in a structured way thru the API. Assumed cost: A little storage, not significant in the great scheme of things.
Hi, On 11/11/2013 12:12 PM, Gert Doering wrote:
- restart an UDM after it has stopped
- change parameters of an UDM and restart it, keeping the same ID
I'd second those. For the second point: if keeping the same ID is not possible, allow to create a new measurement as an (editable) copy of a previous query (which would be a desirable feature on its own). Best, Gilles -- Fondation RESTENA - DNS-LU 6, rue Coudenhove-Kalergi L-1359 Luxembourg tel: (+352) 424409 fax: (+352) 422473
On 13.11.2013, at 15:59 , Gilles Massen <gilles.massen@restena.lu> wrote:
For the second point: if keeping the same ID is not possible, allow to create a new measurement as an (editable) copy of a previous query (which would be a desirable feature on its own).
I prefer creating a new editable measurement over changing an already defined measurement in any other aspect than ending time. Changing other parameters of a measurement will create confusion in diagnosing problems and will complicate any study of how Atlas is used. Changing the definition of something that is normally intended to be immutable looks like bad style/architecture to me in general. Measurement IDs are not scarce and I am sure Gert can write his scripts in a way that does not require him to keep track of IDs manually. ;-) Daniel
Hi, On Wed, Nov 13, 2013 at 06:01:54PM +0100, Daniel Karrenberg wrote:
Measurement IDs are not scarce and I am sure Gert can write his scripts in a way that does not require him to keep track of IDs manually. ;-)
Sure I could. But I don't *want* that. Results are received by asking for an ID and a timeline. This is what I want to use, not "oh, jump through some 5 extra hoops to figure out what ID my measurement had at some point in the past". Doing this manually is tedious, doing this in a script is brittle. If your internal model doesn't allow this, add a "user-visible ID" that can be left unchanged but is sufficient to query the results. (And I don't want to set up a new UDM every time I want to test how a parameter change affects what I see. For me, this is *one* UDM that evolves over time) Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
On 13.11.2013, at 19:49 , Gert Doering <gert@space.net> wrote:
Hi,
On Wed, Nov 13, 2013 at 06:01:54PM +0100, Daniel Karrenberg wrote:
Measurement IDs are not scarce and I am sure Gert can write his scripts in a way that does not require him to keep track of IDs manually. ;-)
Sure I could. But I don't *want* that....
We disagree. I am sure the Atlas developers can implement anything that can be done with an algorithm that runs on the platforms they have at their disposal. My point is specifically that not anything and everything should be done in any specific system. In the early Unix days we called this "creeping featurism". Good design and style avoids this. Someone else said it much better than I can possibly do: La perfection est enfin atteinte, non pas lorsqu'il n'y a plus rien à ajouter, mais lorsqu'il n'y a plus rien à enlever. Antoine de Saint Exupéry My amateur translation: Perfection will finally be achieved, not when there is nothing left to add, but when there is nothing left to take away. So I still argue that what you want to do is best done in the form of a shell around multiple measurements. Of course if more people want such a shell, it should be put on the roadmap and supplied by the NCC. Daniel
Hi, On Thu, Nov 14, 2013 at 08:17:45AM +0100, Daniel Karrenberg wrote:
On 13.11.2013, at 19:49 , Gert Doering <gert@space.net> wrote:
On Wed, Nov 13, 2013 at 06:01:54PM +0100, Daniel Karrenberg wrote:
Measurement IDs are not scarce and I am sure Gert can write his scripts in a way that does not require him to keep track of IDs manually. ;-)
Sure I could. But I don't *want* that....
We disagree.
I'm not sure we can disagree on what *I* *want*. :-)
I am sure the Atlas developers can implement anything that can be done with an algorithm that runs on the platforms they have at their disposal. My point is specifically that not anything and everything should be done in any specific system. In the early Unix days we called this "creeping featurism". Good design and style avoids this. Someone else said it much better than I can possibly do:
Nice quote, and I agree with the underlying idea. OTOH, if you want to build a *successful* platform, it certainly helps to make it nice-to-use to those that are trying to *use* it. I can't find any intuitive reasons why I would find it useful to have to add new UDMs all the time when all I want is "restart a given UDM" or "add more probes to a given UDM, but keep the rest of the parameters the same", etc. But anyway. My wishes have been stated a few times now, do what you want with it. Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
Hello,
But anyway. My wishes have been stated a few times now, do what you want with it.
Since this thread is also about quotes: "There's no problem in Computer Science that can't be solved by adding another level of indirection to it" I propose a middle ground here (between "UDMs are immutable" and "I need a stable ID anyway") -- labels. The user specifying a UDM would be able to attach one (ore more?) labels to it, like "Ping to Gert's network". Then this label can be used in access functions, like data downloads, where instead of an ID, you ask for the label; the result is a union of all the measurements with that label. In other words, a label is a kind of ID that stays the same across multiple measurements. Would this work for you? Regards, Robert
On 22.11.2013, at 11:30 , Robert Kisteleki <robert@ripe.net> wrote:
Hello,
But anyway. My wishes have been stated a few times now, do what you want with it.
Since this thread is also about quotes: "There's no problem in Computer Science that can't be solved by adding another level of indirection to it"
I propose a middle ground here (between "UDMs are immutable" and "I need a stable ID anyway") -- labels.
The user specifying a UDM would be able to attach one (ore more?) labels to it, like "Ping to Gert's network". Then this label can be used in access functions, like data downloads, where instead of an ID, you ask for the label; the result is a union of all the measurements with that label. In other words, a label is a kind of ID that stays the same across multiple measurements.
Would this work for you?
Regards, Robert
What is needed from my point of view: - make it possible to define a new measurement copying any parameter that makes remote sense from an existing measurement - int API and GUI, API more important - allow filtering in the API based on 'owner' and 'description' This way one can create a series of related measurements and find them back by a common description tag. No new field is needed. Just make existing ones searchable in the API as they are in the GUI. What would be nice to have is a forward/backward link chain of these measurements, e.g. enumerate measurements based on this msmid enumerate measurements this msmid is based on. Daniel
Hi, On Fri, Nov 22, 2013 at 11:30:04AM +0100, Robert Kisteleki wrote:
The user specifying a UDM would be able to attach one (ore more?) labels to it, like "Ping to Gert's network". Then this label can be used in access functions, like data downloads, where instead of an ID, you ask for the label; the result is a union of all the measurements with that label. In other words, a label is a kind of ID that stays the same across multiple measurements.
That would work for me, yes (I think it was mentioned in the thread already). To make it fully useful, it would need to go hand in hand with "take this UDM, clone it, change a few parameters, and keep the rest" to make modifying just single parameters easier than "remember everything and type it all in again" :-) Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
On 2013.11.11. 11:49, Vesna Manojlovic wrote:
You can expect that we will get to work on this "soon", but I can't give you any time estimates yet on when will this be implemented.
Update: this request is closely related to a set of changes we're already working on, related to the layout of the probe list and probe status pages. So I can confirm that this is no longer "requested" but "in progress". Regards, Robert
On 08.11.2013, at 18:03 , Klaus Darilion <klaus.darilion@nic.at> wrote:
Hi!
Depending on the measurement, sometimes I want a realistic "user's view" which means using probes in residential areas, and sometimes I want a "backbone" view, which mean using probes in the back-bone (good connected) e.g. to measure my access network or my servers without the delay introduced by the probes access network.
Thus, it would be cool if the probe selection algorithm allows also to choose "residential" probes or "backbone" probes. Maybe this categorization can be done automatically, or probe owners can be asked to specify the probes connectivity manually.
Thanks, Klaus
I understand the question. I am not sure what the answer is, e.g. which categories we should use and how to do the categorisation. At the moment the Anchors provide backbone views. Most small probes are at the edge of the network. Daniel
Hi Daniel! On 12.11.2013 10:26, Daniel Karrenberg wrote:
On 08.11.2013, at 18:03 , Klaus Darilion <klaus.darilion@nic.at> wrote:
Hi!
Depending on the measurement, sometimes I want a realistic "user's view" which means using probes in residential areas, and sometimes I want a "backbone" view, which mean using probes in the back-bone (good connected) e.g. to measure my access network or my servers without the delay introduced by the probes access network.
Thus, it would be cool if the probe selection algorithm allows also to choose "residential" probes or "backbone" probes. Maybe this categorization can be done automatically, or probe owners can be asked to specify the probes connectivity manually.
Thanks, Klaus
I understand the question. I am not sure what the answer is, e.g. which categories we should use and how to do the categorisation. At the moment the Anchors provide backbone views. Most small probes are at the edge of the network. I do want to measure the connectivity of the anchors node as this actually measures the connectivity of the probes when I assume that the anchor nodes are well connected.
I want to test the connectivity from the Internet backbone to my servers (similar like the RIPE DNSMON). So for some UDM I want to use "backbone nodes" to see if there are problems with my servers, and for other UDMs I want to use "residental nodes" to see the end user view. regards Klaus
On 12.11.2013 12:00, Klaus Darilion wrote:
Hi Daniel!
On 08.11.2013, at 18:03 , Klaus Darilion <klaus.darilion@nic.at> wrote:
Hi!
Depending on the measurement, sometimes I want a realistic "user's view" which means using probes in residential areas, and sometimes I want a "backbone" view, which mean using probes in the back-bone (good connected) e.g. to measure my access network or my servers without the delay introduced by the probes access network.
Thus, it would be cool if the probe selection algorithm allows also to choose "residential" probes or "backbone" probes. Maybe this categorization can be done automatically, or probe owners can be asked to specify the probes connectivity manually.
Thanks, Klaus
I understand the question. I am not sure what the answer is, e.g. which categories we should use and how to do the categorisation. At the moment the Anchors provide backbone views. Most small probes are at the edge of the network. I do want to measure the connectivity of the anchors node as this actually measures the connectivity of the probes when I assume that
On 12.11.2013 10:26, Daniel Karrenberg wrote: the anchor nodes are well connected. ^^^^^^ Ups, typo. Should read: "I do NOT want" ...
I want to test the connectivity from the Internet backbone to MY servers (similar like the RIPE DNSMON). So for some UDM I want to use "backbone nodes" to see if there are problems with my servers, and for other UDMs I want to use "residental nodes" to see the end user view.
regards Klaus
On 12.11.2013, at 12:00 , Klaus Darilion <klaus.darilion@nic.at> wrote:
I want to test the connectivity from the Internet backbone to my servers (similar like the RIPE DNSMON). So for some UDM I want to use "backbone nodes" to see if there are problems with my servers, and for other UDMs I want to use "residental nodes" to see the end user view.
Why not set up a UDM using only anchors for this, e.g from the anchors to your servers? Anchors have probe ids between 6000 and 7000. Daniel
On 14.11.2013, at 9:27 , Daniel Karrenberg <daniel.karrenberg@ripe.net> wrote:
Why not set up a UDM using only anchors for this, e.g from the anchors to your servers?
See measurement 1039778 for an example. It nicely explores the connectivity of a server: Daniel
participants (11)
-
Daniel Karrenberg -
dave -
Gert Doering -
Gilles Massen -
Klaus Darilion -
Mike. -
Philip Homburg -
Robert Kisteleki -
Roman Mamedov -
Rusty Dekema -
Vesna Manojlovic