Reasons to celebrate - passed 1K active probes :)
Hi everyone, *you* are part of our big success: more then 1024 probes before the end of this year! We've published an article on Labs about that yesterday: https://labs.ripe.net/Members/becha/ripe-atlas-has-reasons-to-celebrate Thank you very much for participating in this exciting project, and Happy New Year! Vesna
On Fri, Dec 23, 2011 at 10:11:08AM +0100, Vesna Manojlovic <BECHA@ripe.net> wrote a message of 12 lines which said:
*you* are part of our big success: more then 1024 probes before the end of this year!
I hope that, in 2012, we'll celebrate the release of the source code, with instructions on how to build your own probe and/or to add measurements.
Dear Stephane, On 12/23/11 10:17 AM, Stephane Bortzmeyer wrote:
On Fri, Dec 23, 2011 at 10:11:08AM +0100, Vesna Manojlovic <BECHA@ripe.net> wrote a message of 12 lines which said:
*you* are part of our big success: more then 1024 probes before the end of this year!
I hope that, in 2012, we'll celebrate the release of the source code,
I have noted your interest in opening the source code.
with instructions on how to build your own probe and/or to add measurements.
If you join the beta-testers of User Defined Measurements (UDM), you can already try out adding some of your measurements, and then let us know what is the further functionality you would like to have. Please write to atlas-dev at ripe.net if you are interested. Regards, Vesna
fre 2011-12-23 klockan 11:13 +0100 skrev Vesna Manojlovic:
Dear Stephane,
On 12/23/11 10:17 AM, Stephane Bortzmeyer wrote:
On Fri, Dec 23, 2011 at 10:11:08AM +0100, Vesna Manojlovic <BECHA@ripe.net> wrote a message of 12 lines which said:
*you* are part of our big success: more then 1024 probes before the end of this year!
I hope that, in 2012, we'll celebrate the release of the source code,
I have noted your interest in opening the source code.
A +1 to that. Merry Christmas, Simon
On 23/12/11 11:24, Simon Josefsson wrote:
fre 2011-12-23 klockan 11:13 +0100 skrev Vesna Manojlovic:
Dear Stephane,
On 12/23/11 10:17 AM, Stephane Bortzmeyer wrote:
On Fri, Dec 23, 2011 at 10:11:08AM +0100, Vesna Manojlovic <BECHA@ripe.net> wrote a message of 12 lines which said:
*you* are part of our big success: more then 1024 probes before the end of this year!
I hope that, in 2012, we'll celebrate the release of the source code,
I have noted your interest in opening the source code.
A +1 to that.
+1 :-)
Merry Christmas,
+1 too. Merry Christmas.
Simon
-- ||// //\\// Rodolfo "kix" Garcia ||\\// //\\ http://www.kix.es/
It seems like +1 is slightly too few dimensions in this poll, so I wrote a quick Google poll to capture a few more dimension: <http://goo.gl/pOnXB> On Dec 23, 2011, at 11:14 AM, Rodolfo kix Garcia wrote:
On 23/12/11 11:24, Simon Josefsson wrote:
fre 2011-12-23 klockan 11:13 +0100 skrev Vesna Manojlovic:
Dear Stephane,
On 12/23/11 10:17 AM, Stephane Bortzmeyer wrote:
On Fri, Dec 23, 2011 at 10:11:08AM +0100, Vesna Manojlovic <BECHA@ripe.net> wrote a message of 12 lines which said:
*you* are part of our big success: more then 1024 probes before the end of this year!
I hope that, in 2012, we'll celebrate the release of the source code,
I have noted your interest in opening the source code.
A +1 to that.
+1 :-)
Merry Christmas,
+1 too. Merry Christmas.
Simon
-- ||// //\\// Rodolfo "kix" Garcia ||\\// //\\ http://www.kix.es/
On 28/12/11 18:55, Richard L. Barnes wrote:
It seems like +1 is slightly too few dimensions in this poll, so I wrote a quick Google poll to capture a few more dimension: <http://goo.gl/pOnXB>
Nice! Thanks a lot.
On Dec 23, 2011, at 11:14 AM, Rodolfo kix Garcia wrote:
On 23/12/11 11:24, Simon Josefsson wrote:
fre 2011-12-23 klockan 11:13 +0100 skrev Vesna Manojlovic:
Dear Stephane,
On 12/23/11 10:17 AM, Stephane Bortzmeyer wrote:
On Fri, Dec 23, 2011 at 10:11:08AM +0100, Vesna Manojlovic <BECHA@ripe.net> wrote a message of 12 lines which said:
*you* are part of our big success: more then 1024 probes before the end of this year!
I hope that, in 2012, we'll celebrate the release of the source code,
I have noted your interest in opening the source code.
A +1 to that.
+1 :-)
Merry Christmas,
+1 too. Merry Christmas.
Simon
-- ||// //\\// Rodolfo "kix" Garcia ||\\// //\\ http://www.kix.es/
-- ||// //\\// Rodolfo "kix" Garcia ||\\// //\\ http://www.kix.es/
I hope that, in 2012, we'll celebrate the release of the source code, with instructions on how to build your own probe and/or to add measurements. Looking at it purely from a technical point of view, releasing the
On 12/23/11 10:17 , Stephane Bortzmeyer wrote: source code should be doable. Of course it will require time to clean it up a bit and make it run on a generic Linux system instead of just on Lantronix modules. But I have no idea how we could allow probes with unknown firmware connect to the atlas infrastructure. At moment we can rely on the probes to faithfully report what they see. And we know exactly what software the probes are running. When probes can run arbitrary firmware, analyzing the results may become much harder.
On Fri, Dec 23, 2011 at 12:12:51PM +0100, Philip Homburg <philip.homburg@ripe.net> wrote a message of 16 lines which said:
releasing the source code [...] allow probes with unknown firmware connect to the atlas infrastructure.
That's two completely different things. I mentioned only the first one. In my vision, modified probes were intended for another C&C, not for the Atlas service.
On 12/23/11 12:23 , Stephane Bortzmeyer wrote:
On Fri, Dec 23, 2011 at 12:12:51PM +0100, Philip Homburg<philip.homburg@ripe.net> wrote a message of 16 lines which said:
releasing the source code [...] allow probes with unknown firmware connect to the atlas infrastructure. That's two completely different things. I mentioned only the first one. In my vision, modified probes were intended for another C&C, not for the Atlas service.
Okay. I assumed that with 'with instructions on how to build your own probe' you meant how to connect it to the Atlas infrastructure. But if you want to connect the probe to another C&C then you may also need the source code for the back-end. The software that runs on the probe is quite simple. The back-end is a completely different story.
On 23/12/11 12:12, Philip Homburg wrote:
I hope that, in 2012, we'll celebrate the release of the source code, with instructions on how to build your own probe and/or to add measurements. Looking at it purely from a technical point of view, releasing the
On 12/23/11 10:17 , Stephane Bortzmeyer wrote: source code should be doable. Of course it will require time to clean it up a bit and make it run on a generic Linux system instead of just on Lantronix modules.
But I have no idea how we could allow probes with unknown firmware connect to the atlas infrastructure. At moment we can rely on the probes to faithfully report what they see. And we know exactly what software the probes are running.
You can release the source code in git/mercurial/... and the probes can run the same (open) source code. People can help to get a better source code. This is a good point because, if the source code is available, some people can implement the probe in other devices, for example linux-based DSL routers (open-wrt,...). If some people want to run different code, they need the interface to communicate with the atlas infrastructure (commands, arguments,...), and probably this info is in the source code ;-) therefore, you won't have problems.
When probes can run arbitrary firmware, analyzing the results may become much harder.
Best Regards, kix -- ||// //\\// Rodolfo "kix" Garcia ||\\// //\\ http://www.kix.es/
If some people want to run different code, they need the interface to communicate with the atlas infrastructure (commands, arguments,...), and probably this info is in the source code ;-) therefore, you won't have problems. I'm not worried about people getting their modified firmware to talk to
On 12/23/11 17:27 , Rodolfo kix Garcia wrote: the atlas infrastructure. A lot of that stuff is even documented :-) The problem is, do you trust those results to be correct? And it is not just faking results. That can be done today, to some extent. But more, what if somebody decides to improve traceroute and it works just slightly different and gives also just slightly different results?
fre 2011-12-23 klockan 17:36 +0100 skrev Philip Homburg:
If some people want to run different code, they need the interface to communicate with the atlas infrastructure (commands, arguments,...), and probably this info is in the source code ;-) therefore, you won't have problems. I'm not worried about people getting their modified firmware to talk to
On 12/23/11 17:27 , Rodolfo kix Garcia wrote: the atlas infrastructure. A lot of that stuff is even documented :-)
The problem is, do you trust those results to be correct? And it is not just faking results. That can be done today, to some extent. But more, what if somebody decides to improve traceroute and it works just slightly different and gives also just slightly different results?
One way to deal with that is to let the probes send in the hash value of its firmware or something similar, which can be used to detect problems like that. And you could prepare "official" reports based only on the probes running "official" firmware. I really think this is orthogonal to releasing source code though. If you haven't built in any security mechanism, people can already do what you appear to be afraid of today. /Simon
On 12/23/11 19:10 , Simon Josefsson wrote:
One way to deal with that is to let the probes send in the hash value of its firmware or something similar, which can be used to detect problems like that. And you could prepare "official" reports based only on the probes running "official" firmware. You want untrusted firmware to send a hash value of itself? How do you know it is not lying?
I really think this is orthogonal to releasing source code though. If you haven't built in any security mechanism, people can already do what you appear to be afraid of today.
(From a technical point of view) releasing the source is not an issue. The probes come with key material that allows them to connect to the Atlas infrastructure. In theory you can get that out of the probe. But, you would violate the agreement as a probe host and it would be quite tricky to do. And, you can take over only one probe at a time which has to be in your physical possession. If we would allow 'third party' probes to connect to the Atlas infrastructure then all of that changes. No need to physically obtain a probe. Just download the source, request a key. And start hacking away.
fre 2011-12-23 klockan 19:33 +0100 skrev Philip Homburg:
On 12/23/11 19:10 , Simon Josefsson wrote:
One way to deal with that is to let the probes send in the hash value of its firmware or something similar, which can be used to detect problems like that. And you could prepare "official" reports based only on the probes running "official" firmware. You want untrusted firmware to send a hash value of itself? How do you know it is not lying?
I really think this is orthogonal to releasing source code though. If you haven't built in any security mechanism, people can already do what you appear to be afraid of today.
(From a technical point of view) releasing the source is not an issue. The probes come with key material that allows them to connect to the Atlas infrastructure. In theory you can get that out of the probe. But, you would violate the agreement as a probe host and it would be quite tricky to do. And, you can take over only one probe at a time which has to be in your physical possession.
If we would allow 'third party' probes to connect to the Atlas infrastructure then all of that changes. No need to physically obtain a probe. Just download the source, request a key. And start hacking away.
How does this keying work today? I haven't seen this documented anywhere. If you embed a symmetric or asymmetric key in the probes, which sounds like what you are suggesting (and is more advanced than what I expected), there shouldn't be any threat to publish source code for the firmware: people will not have access to any private key that you will trust. I was assuming that your current threat model was that you aren't concerned about fake probes, but if you have embedded keys in the probes, that suggests a different threat model. Solving that is relatively complicated, and describing your setup would be a useful contribution. My proposed solution to send a hash of the firmware was to be able to diagnose on the server side which firmware sent what information and to do larger-scale data mining. It is not a solution to malicious probes. Sorry if anything I said implied that. Cheers, /Simon
On 12/25/11 20:45 , Simon Josefsson wrote:
fre 2011-12-23 klockan 19:33 +0100 skrev Philip Homburg:
(From a technical point of view) releasing the source is not an issue. The probes come with key material that allows them to connect to the Atlas infrastructure. In theory you can get that out of the probe. But, you would violate the agreement as a probe host and it would be quite tricky to do. And, you can take over only one probe at a time which has to be in your physical possession.
If we would allow 'third party' probes to connect to the Atlas infrastructure then all of that changes. No need to physically obtain a probe. Just download the source, request a key. And start hacking away. How does this keying work today? I haven't seen this documented anywhere. The slides should give you a general idea of how it works: <http://ripe61.ripe.net/presentations/269-20101118-RIPE61-MAT-Robert.pdf>
If you embed a symmetric or asymmetric key in the probes, which sounds like what you are suggesting (and is more advanced than what I expected), there shouldn't be any threat to publish source code for the firmware: people will not have access to any private key that you will trust.
That's right.
My proposed solution to send a hash of the firmware was to be able to diagnose on the server side which firmware sent what information and to do larger-scale data mining. It is not a solution to malicious probes. Sorry if anything I said implied that.
So, assuming for a moment that we cannot let 'third party' probes connect to the Atlas infrastructure, because we cannot trust the results, what would be the point of releasing the source? One is that somebody may want to run his own private copy of the whole Atlas system. But that is going to to be a lot of work setting it all up. If we would allow third party probes to connect, but it ignore their results and not schedule any UDMs on those probes. Just publish the raw results somewhere. Would that be a net benefit to the community, or just a PR disaster waiting to happen?
Hi Philip, On this point:
So, assuming for a moment that we cannot let 'third party' probes connect to the Atlas infrastructure, because we cannot trust the results, what would be the point of releasing the source? One is that somebody may want to run his own private copy of the whole Atlas system. But that is going to to be a lot of work setting it all up.
...
If we would allow third party probes to connect, but it ignore their results and not schedule any UDMs on those probes. Just publish the raw results somewhere. Would that be a net benefit to the community, or just a PR disaster waiting to happen?
I think you may be skipping a potential middle point here. Thinking back to RIPE 61, there were some folks who were offering to manufacture probes if the firmware were open-source. That seems to indicate that a "partnership" model might be viable, in which anyone can get the source code, but if a probe maker signs a contract with RIPE, then their probes can feed information into the real RIPE Atlas system. Having contracts would provide a way for RIPE NCC to get guarantees related to security, fraud, etc. A more open model might not be useless, either. After all, one can make much richer decisions based on data sources than "accept/ignore". As long as the data is source-tagged, then different sources can be compared for consistency, which would provide some validation that the probes aren't providing completely bogus data. Putting these two ideas together, it seems like you could enable two "classes" of data, "high assurance" from probes made by RIPE NCC and its partners, and "low assurance" from anyone else. Think of it as a "freemium" model -- it's cheap and easy for small-scale projects to feed into the "low assurance" class, but if you want to be a real contributor, do the work to get into the "high assurance" class. You could even provide additional features to "high assurance" sources (UDM access / management, say) as an incentive. --Richard
Hi Philip,
On this point:
So, assuming for a moment that we cannot let 'third party' probes connect to the Atlas infrastructure, because we cannot trust the results, what would be the point of releasing the source? One is that somebody may want to run his own private copy of the whole Atlas system. But that is going to to be a lot of work setting it all up. ...
If we would allow third party probes to connect, but it ignore their results and not schedule any UDMs on those probes. Just publish the raw results somewhere. Would that be a net benefit to the community, or just a PR disaster waiting to happen? I think you may be skipping a potential middle point here. Thinking back to RIPE 61, there were some folks who were offering to manufacture probes if the firmware were open-source. That seems to indicate that a "partnership" model might be viable, in which anyone can get the source code, but if a probe maker signs a contract with RIPE, then their probes can feed information into the real RIPE Atlas system. Having contracts would provide a way for RIPE NCC to get guarantees related to security, fraud, etc. Let me once again emphasize that I'm looking at this from a technical
A more open model might not be useless, either. After all, one can make much richer decisions based on data sources than "accept/ignore". As long as the data is source-tagged, then different sources can be compared for consistency, which would provide some validation that the probes aren't providing completely bogus data. At the moment, analysis is already hard. The Internet is an extremely
On 12/28/11 18:52 , Richard L. Barnes wrote: point of view. There are many other issues to consider, which I'm ignoring here. I'd say if a party were to manufacture probes under supervision of RIPE NCC then that is independent of the issue of open source. That's just a normal business relationship. It is possible that an organization would agree to sponsor probes in return for Atlas to become open source. But then the source would effectively be payment for something. That would be similar to the Android model: you can download the source, but there is no guarantee that you can actually install it on your own phone and some functionality, like connecting the Android market, may be unavailable. From a technical point of view, just releasing the probe sources is the easiest, because it is a relatively small amount of code. Relatively easy to compile and install. The back-end systems are way more complex. diverse landscape. It is very much a research question whether adding untrusted data is good or not. I certainly don't know the answer to that question.
Putting these two ideas together, it seems like you could enable two "classes" of data, "high assurance" from probes made by RIPE NCC and its partners, and "low assurance" from anyone else. Think of it as a "freemium" model -- it's cheap and easy for small-scale projects to feed into the "low assurance" class, but if you want to be a real contributor, do the work to get into the "high assurance" class. You could even provide additional features to "high assurance" sources (UDM access / management, say) as an incentive.
One issue is what do you do with the data in the low assurance class. At the moment, probe hosts have access to data of their own probe and RIPE NCC researchers have access to all data. If researchers want to have that low assurance data, then great. If not, then it may be a waste of resources.
On Tue, Dec 27, 2011 at 11:58:24AM +0100, Philip Homburg wrote:
So, assuming for a moment that we cannot let 'third party' probes connect to the Atlas infrastructure, because we cannot trust the results, what would be the point of releasing the source? One is that somebody may want to run his own private copy of the whole Atlas system. But that is going to to be a lot of work setting it all up.
Open source means that everybody can search for potential bugs, problems, etc. On the other hand - everybody can submit some patches with improvements and help develop ATLAS. -- gucio -> Piotr Strzyżewski E-mail: Piotr.Strzyzewski@polsl.pl
Congrats ! And note that, in pure geek logic, 1K equals 2^10 here, and not one thousand :-) Congrats again. And a very Happy New Year, too. Patrick Vande Walle On 23/12/11 10:11, Vesna Manojlovic wrote:
Hi everyone,
*you* are part of our big success: more then 1024 probes before the end of this year!
We've published an article on Labs about that yesterday: https://labs.ripe.net/Members/becha/ripe-atlas-has-reasons-to-celebrate
Thank you very much for participating in this exciting project, and Happy New Year!
Vesna
participants (8)
-
Patrick Vande Walle -
Philip Homburg -
Piotr Strzyzewski -
Richard L. Barnes -
Rodolfo kix Garcia -
Simon Josefsson -
Stephane Bortzmeyer -
Vesna Manojlovic