We've recently experienced a storm of customer complaints after starting to use IP addresses from a new allocation/assignment. The IP block was earlier in use by a German company. The result is that services like Google and Yahoo assume users are from Germany and presents content accordingly. Some users have also reported that they are excluded from certain services. A quick web search reveals that this is a pretty common problem. While not blaming RIPE or the other RIRs, we believe this problem should be addressed, and that the initiative has to come from the RIRs. I would like to propose that RIPE NCC works together with other RIRs to see if it's possible to implement procedures/routines to notify providers and users of IP geolocation services of new, relocated and deleted allocations. Also, one should probably consider if it's a need for a (distributed?) database with more fine-grained location data than the whois database currently provides (also, I've been told that licensing issues prohibits geolocation providers from using the RIR DBs directly, but I've not been able to verify this). Apologies in advance if this has been discussed before -- I searched the archive, but got no hits. -- -o) Vegard Svanberg, CTO - Monsternett (www.monsternett.no) /\\ Violgata 3A, N-1776 HALDEN, NORWAY _\_v Phone: (+47) 69701802 | Fax: (+47) 69701801
With my DB-WG Chair hat on: Trying to - with reasonable accuracy - deduct geo-location-info for a particular end-system using a particualr address, at a particular point in time, from a block registered in a particular RIR's DB as an assignment is very bad idea to begin with. At least for the RIPE-DB, there are no agreed or definend semantics for the interpretation of the country: attribute. The data is a hint, at best. Actually, the country codes entered could point to the home country of a particular LIR's administrative location, to the location of the responsible NOC (operating from abroad) or something altogether, completely, arbitrary. Also, there are the codes "ZZ" and "EU", neither of which is clearly defined regarding its meaning. Just to fill in some background: a while ago we had the discussion in the DB-WG proposing to completely remove the country attribute for the reasons given. However, we received input that requested the country data to be kept, but *purely* for statistical reasons! It was understood that the accuracy and quality of that sort of data is just good enough exclusively for *that* pupose - but not really for something else and least for basing operational decisions on! Now, taking off my DB-WG hat.... My personal feeling is that the RIRs' databases are definitely the wrong place to maintain such (volatile) data (in many cases for subranges of registry entries!) and the RIRs are the wrong organisatins to get involved with such a "service". And, to top it off, with mobile users and tunnels, the whole concept of managing access to data or services based on an IP address is going to brake more often than it does already ;-) Wilfried Vegard Svanberg wrote:
We've recently experienced a storm of customer complaints after starting to use IP addresses from a new allocation/assignment. The IP block was earlier in use by a German company. The result is that services like Google and Yahoo assume users are from Germany and presents content accordingly. Some users have also reported that they are excluded from certain services.
A quick web search reveals that this is a pretty common problem.
While not blaming RIPE or the other RIRs, we believe this problem should be addressed, and that the initiative has to come from the RIRs.
I would like to propose that RIPE NCC works together with other RIRs to see if it's possible to implement procedures/routines to notify providers and users of IP geolocation services of new, relocated and deleted allocations.
Also, one should probably consider if it's a need for a (distributed?) database with more fine-grained location data than the whois database currently provides (also, I've been told that licensing issues prohibits geolocation providers from using the RIR DBs directly, but I've not been able to verify this).
Apologies in advance if this has been discussed before -- I searched the archive, but got no hits.
Wilfried,
particular end-system using a particualr address, at a particular point in time, from a block registered in a particular RIR's DB as an assignment is very bad idea to begin with.
At least for the RIPE-DB, there are no agreed or definend semantics for the interpretation of the country: attribute. The data is a hint, at best.
agreed, but the problem in question is not necessarily related to the RIPE DB being used as source of the geolocation information. What seems to be desirable is some notification of changes when a certain address block is unassigned or, more importantly, reassigned. This doesn't have to provide the new location information, but should initiate a new run of the location magic for this address range, so the GeoLoc provider knows when to update/refresh which information. Of course, we know similar demands from the domain business and these requests aren't all free of concern, but the situation might be different here in the addressing world. -Peter
On 27 Oct 2009, at 16:36, Peter Koch wrote:
What seems to be desirable is some notification of changes when a certain address block is unassigned or, more importantly, reassigned. This doesn't have to provide the new location information, but should initiate a new run of the location magic for this address range, so the GeoLoc provider knows when to update/refresh which information.
This presumes that GeoLoc providers will always take the trouble to check for these changes and act on them, neither of which seems likely.
Hi Peter! Peter Koch wrote:
Wilfried,
particular end-system using a particualr address, at a particular point in time, from a block registered in a particular RIR's DB as an assignment is very bad idea to begin with.
At least for the RIPE-DB, there are no agreed or definend semantics for the interpretation of the country: attribute. The data is a hint, at best.
agreed, but the problem in question is not necessarily related to the RIPE DB being used as source of the geolocation information. What seems to be desirable is some notification of changes when a certain address block is unassigned or, more importantly, reassigned.
Well, we actually do have that mechanism in place: the registry database. It offers a granularity of (legacy, plus) allocations to LIRs and the assignments from within the PA blocks. In principle, this data should be reasonably accurate and should be maintained regularly. I presume at least the allocations, being maintained by the RIR should be pretty reliable?
From within the blocks, that's a different story maybe... But it should remain a scan and pull-technology by those who want to read and consume the data. I am not convinced that a notification service for everyone and her dog would be scalable?
This doesn't have to provide the new location information, but should initiate a new run of the location magic for this address range, so the GeoLoc provider knows when to update/refresh which information.
For some situations this could be a couple of times per hour, for mobile devices ;-) Other than that I'd suggest a frequency of once per day to pick up the new information.
Of course, we know similar demands from the domain business and these requests aren't all free of concern, but the situation might be different here in the addressing world.
Yep, I think the discussion is useful, at least to start to understand what the expectations, the reality and the boundary conditions are! I always get back to see that resource distribution for the IP-world, for the Internet was never taking (volatile) national boundaries into account, neither on the administrative nor on the operational level. Maybe the parties trying to use geo-location to control access to information and (licensed) services should also start to accept that fact for their business model?
-Peter
Wilfried.
On Oct 27, 2009, at 5:36 PM, Peter Koch wrote:
Wilfried,
particular end-system using a particualr address, at a particular point in time, from a block registered in a particular RIR's DB as an assignment is very bad idea to begin with.
At least for the RIPE-DB, there are no agreed or definend semantics for the interpretation of the country: attribute. The data is a hint, at best.
agreed, but the problem in question is not necessarily related to the RIPE DB being used as source of the geolocation information. What seems to be desirable is some notification of changes when a certain address block is unassigned or, more importantly, reassigned. This doesn't have to provide the new location information, but should initiate a new run of the location magic for this address range, so the GeoLoc provider knows when to update/refresh which information.
Of course, we know similar demands from the domain business and these requests aren't all free of concern, but the situation might be different here in the addressing world.
Address blocks are quarantined for a while when they are returned, so I guess that anybody who regularly checks the database should notice they went missing and this could be a nice hint that those addresses aren't in use anymore and any data associated with it should be removed. Or are you proposing more of a push system in which the RIR tries to find and notify those people who collect these data to notify them there have been changes. A third solution would be to publish a sort of BOGON list which contains all unallocated blocks in s machine parseable format, this might even be usefull for other purposes as well, but I can aleady think of two problems: First of all there is a risk that people built filters based on this list and those might not get updated again, so we try and debogonize space forever instead just once when the /8 comes down from IANA, secondly if people are starting to use this list as a way to generate filters we again create some form of 'off button' for the internet and that might give people the wrng idea that the NCC in fact can do something about abuse or other "illegal" activities (I use quotes here because it's hard to define the term in an international situation, what may be illegal in your eyes migth be perfectly legal in other parts of the world). I can think of option 1 happening, in fact that mechanism is there, option 2 (push) is almost impossible to achieve and I don't like option 3 at all. Grtx, Marco
On 28.10 09:29, Marco Hogewoning wrote: ...
I can think of option 1 happening, in fact that mechanism is there,
Vergard (and others) found out this is not happening.
option 2 (push) is almost impossible to achieve
agree
and I don't like option 3 at all.
The RIRs are currently discussing an extended 'stats' file format that will include the "unallocated" address space. This waay ftp://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-latest and similar files from other RIRs would include the unallocated address space as well. This way we aim to publish once a day a onsistent view about *all* address space. Once implemented this couldbe used as option 3. If it was useful we could add a "quarantined" flag to help. What do people think? Daniel
On 28 Oct 2009, at 17:03, Daniel Karrenberg wrote:
The RIRs are currently discussing an extended 'stats' file format that will include the "unallocated" address space. This waay ftp://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-latest and similar files from other RIRs would include the unallocated address space as well. This way we aim to publish once a day a onsistent view about *all* address space. Once implemented this couldbe used as option 3. If it was useful we could add a "quarantined" flag to help.
What do people think?
that would be really user(tm), including the quarantine flag, or whatever name you end up giving it Joao
On Oct 28, 2009, at 5:03 PM, Daniel Karrenberg wrote:
I can think of option 1 happening, in fact that mechanism is there,
Vergard (and others) found out this is not happening.
I noticed this, but then there is always the case of getting it idiot proof and finding a bigger idiot.
option 2 (push) is almost impossible to achieve
agree
and I don't like option 3 at all.
The RIRs are currently discussing an extended 'stats' file format that will include the "unallocated" address space. This waay ftp://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-latest and similar files from other RIRs would include the unallocated address space as well. This way we aim to publish once a day a onsistent view about *all* address space. Once implemented this couldbe used as option 3. If it was useful we could add a "quarantined" flag to help.
What do people think?
I think I already gave most of my concern and that is that we end up with a new bogon list and might have to go to extensive efforts do debogonize address space at very small blocks. Now that's not that different from the current issues people are having with the GEO-ip so I wonder if this would really solve the problem. Marco
Daniel Karrenberg wrote:
On 28.10 09:29, Marco Hogewoning wrote:
...
I can think of option 1 happening, in fact that mechanism is there,
Vergard (and others) found out this is not happening.
option 2 (push) is almost impossible to achieve
agree
and I don't like option 3 at all.
The RIRs are currently discussing an extended 'stats' file format that will include the "unallocated" address space. This waay ftp://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-latest and similar files from other RIRs would include the unallocated address space as well. This way we aim to publish once a day a onsistent view about *all* address space. Once implemented this couldbe used as option 3. If it was useful we could add a "quarantined" flag to help.
What do people think?
I think that is a very laudable goal to achieve - in itself! It would conribute to the vision of having a unique or coordinated source for information about the status of resources on a global level; by just consulting one source/file/feed/list/you name it at an arbitrary point in time. One of the problems that I perceive as fundamental in the curent way that the set of Registries manages the global resource pool is the "regional concept". This starts at whois and maybe ends at geolocation. Btw, I am not saying that this should be changed, quite to the contrary. But it adds complexity to the system and makes it difficult to understand for the uninitiated ;-) That said, this list would offer just another means for the GL service providers to try and provide a better (more accurate, more up-to-date) service. At the same time it would not help in getting around the facts (of differnet kinds) that the management, the operations and logistics of the Internet are agnostic - by design - of national or regional geographic boundaries or numbering trees.
Daniel
Wilfried
On Wed, Oct 28, 2009 at 09:29:30AM +0100, Marco Hogewoning wrote:
Address blocks are quarantined for a while when they are returned, so I guess that anybody who regularly checks the database should notice they went missing and this could be a nice hint that those addresses aren't in use anymore and any data associated with it should be removed.
would that affect intra LIR assignment changes or just returns to the RIR?
Or are you proposing more of a push system in which the RIR tries to find and notify those people who collect these data to notify them there have been changes.
I'd not say that the RIR should actively approach every GeoLoc provider but the RIR, or the RIPE community, to pick on the difference, could try to engage into dialogue, which I believe has been started already. This is not to end up with the location information in the RIPE DB which is unlikely to happen due to both technical (granularity) and business ("our heuristics'r'us") limitations(*). But getting the words out how address-to-user-bindings change might be a worthwhile goal. If possible, this would be an interesting presentation+discussion at an EOF plenary.
I can think of option 1 happening, in fact that mechanism is there, option 2 (push) is almost impossible to achieve and I don't like option 3 at all.
Well, yes, share your concerns re:the "off" button. -Peter (*) There are services that use the RIR DBs, though. See for example the perl IP::Country module, which is quite useful. It's not so much that _all_ uses of DB data were bad, but the application needs to carefully select and understand the GeoLoc service's properties. An update once a year is OK if you just want to map log file excerpts to some map.
Peter Koch wrote: [...]
If possible, this would be an interesting presentation+discussion at an EOF plenary.
Indeed! João, could you please make a note somewhere for the planning phase for RIPE60? Wilfried
* Wilfried Woeber, UniVie/ACOnet <Woeber@CC.UniVie.ac.at> [2009-10-27 17:18]:
My personal feeling is that the RIRs' databases are definitely the wrong place to maintain such (volatile) data (in many cases for subranges of registry entries!) and the RIRs are the wrong organisatins to get involved with such a "service".
While I would probably agree that the RIRs databases would be the wrong place to maintain the data, and that the RIRs probably shouldn't provide this service directly, I do think it's important that the RIRs try to establish some standards regarding notifications and updates. This seems to be a pretty common problem today, and most probably even more so in the future, as the usage of geo-location services for sure will increase, and the number of ISPs/organizations who will either have to renumber and/or receive additional IP blocks previously allocated to someone else, will increase. -- -o) Vegard Svanberg, CTO - Monsternett (www.monsternett.no) /\\ Violgata 3A, N-1776 HALDEN, NORWAY _\_v Phone: (+47) 69701802 | Fax: (+47) 69701801
Vegard Svanberg wrote:
While I would probably agree that the RIRs databases would be the wrong place to maintain the data, and that the RIRs probably shouldn't provide this service directly, I do think it's important that the RIRs try to establish some standards regarding notifications and updates.
This is something which could be applied to any third party service that classifies IP addresses in some way - including, say, abuse blacklists. I assume that the problem will also ramp up in intensity after IPv4 depletion, if the address space does indeed fragment due to transfers. Acknowledging that the RIRs can't solve the problem and we need to stimulate third parties to take action, I think that if we can make this work then you have a very strong incentive to use RIR-approved transfer procedures instead of the black market. (In fact, it's a great example of the usefulness of RIRs as the authoritative source of this information.) If we really believe that the tools to do this already exist, perhaps we have a need for a BCP document on detecting updates? All the best, Dave -- Dave Wilson, Senior Network Engineer HEAnet Limited, Ireland's Education and Research Network 1st Floor, 5 George's Dock, IFSC, Dublin 1 Registered in Ireland, no 275301 tel: +353-1-660 9040 fax: +353-1-660 3666 web: http://www.heanet.ie/ H323 GDS:0035301101738 PGP: 1024D/C757ADA9 HEAnet National Networking Conference http://www.heanet.ie/conferences/2009/
Dear Vegard, Vegard Svanberg wrote:
I would like to propose that RIPE NCC works together with other RIRs to see if it's possible to implement procedures/routines to notify providers and users of IP geolocation services of new, relocated and deleted allocations.
The RIPE NCC whois database is available as a daily dump over FTP in a format suitable for bulk text processing. This should serve as a sufficient "notification mechanism" that meets your requirements, and I would expect that any organization that makes it its business to provide GeoIP information to paying customers would be aware of its existence and actively utilizing it to this end. One of the GeoIP providers (MaxMind) provides a free low-resolution GeoIP database called CeoLiteCountry which happens to also be packaged by many *nix distributions due to its liberal license. Being free, it is much more widely used by content providers than commercial offerings. Unfortunately, being a static database rather than a service, the accuracy of the information contained in it depends on a fairly long chain of custody which includes MaxMind (that presently updates it once a month, vs. their commercial offering which is updated weekly), the various distribution packagers (the geoip-database package in the current Ubuntu release, for example, contains data which dates to March 2002), and individual system administrators which make use of it. A problem no notification mechanism can address.
Apologies in advance if this has been discussed before -- I searched the archive, but got no hits.
I'm not sure if this is something the RIPE NCC or this work group should, or even could, solve. -- Respectfully yours, David Monosov
On 28.10 08:55, David Monosov wrote:
One of the GeoIP providers (MaxMind) provides a free low-resolution GeoIP database called CeoLiteCountry ...
Acually MaxMind are also providing city. You can see this in REX when using the "Geolocation" tab. This tool provides a year of history too, so you can check what Maxmind thinks o any block you may be about to start using. http://labs.ripe.net/content/rex-resource-explainer Daniel
In message <20091028160953.GN58385@reiftel.karrenberg.net>, at 17:09:53 local time, on Wed, 28 Oct 2009, Daniel Karrenberg <daniel.karrenberg@ripe.net> remarked:
Acually MaxMind are also providing city. You can see this in REX when using the "Geolocation" tab. This tool provides a year of history too, so you can check what Maxmind thinks o any block you may be about to start using.
That gives an interesting view of my home IP address (within 83.67.0.0/17) - currently "London", although for four months this summer they apparently split it into sixteen different cities. None of which is closer than 100Km to where I really live... But they got the country right (which is what intellectual-property-owning users of geolocation services seem most worried about). -- Roland Perry Public Affairs Officer, RIPE NCC
participants (10)
-
Daniel Karrenberg -
Dave Wilson -
David Monosov -
Jim Reid -
João Damas -
Marco Hogewoning -
Peter Koch -
Roland Perry -
Vegard Svanberg -
Wilfried Woeber, UniVie/ACOnet