Question regarding delta between "delegated-ripencc-extended-latest" and Whois output
Dear RIPE DB mailing list people, we, the IPFire development team, just bumped across an IP address somewhere in the 51.15.0.0/17 network (no abuse, we were just curious :-) ). While the RIPE Whois server returned "NL" as the country of that subnet [1], the "delegated-ripencc-extended-latest" file available at ftp.ripe.net [2] only contains 51.15.0.0/16, which points to "FR". Since the first one result contains the correct country code of that IP network, I would like to ask why "delegated-ripencc-extended-latest" does not include subnets like this, and where the Whois server fetches his answer from. Did we miss something? Is there another or a better way of fetching precise subnet data from RIPE? It would be great if someone could point us into the right direction. :-) Thanks, and best regards, Peter Müller [1] Relevant snippet of the Whois server output:
inetnum: 51.15.0.0 - 51.15.63.255 [...] country: NL [...]
[2] https://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-extended-latest, relevant snippet of this file:
ripencc|FR|ipv4|51.15.0.0|65536|19930901|assigned|925547da-35fb-4346-b464-aadff095c6ea
On Mon, Jun 29, 2020 at 10:31 AM Peter Müller via db-wg <db-wg@ripe.net> wrote:
Dear RIPE DB mailing list people,
we, the IPFire development team, just bumped across an IP address somewhere in the 51.15.0.0/17 network (no abuse, we were just curious :-) ).
While the RIPE Whois server returned "NL" as the country of that subnet [1], the "delegated-ripencc-extended-latest" file available at ftp.ripe.net [2] only contains 51.15.0.0/16, which points to "FR".
Since the first one result contains the correct country code of that IP network, I would like to ask why "delegated-ripencc-extended-latest" does not include subnets like this, and where the Whois server fetches his answer from.
If you run a query for an individual address you will be given a response for the most specific network containing that address. In contrast, the stats file will give you the aggregate, along with things like original registration date and a country code for the organization that controls the whole block. It is possible for a controlling entity in country A to have a branch office or customer in country B. That's why more specific registrations sometimes have a different country code from the parent. The RIPE Database, queried by Whois or RDAP, serves a different purpose from the stats file. The former enables communication between operators and supports the routing registry. The latter is to provide high level statistical data to researchers. Kind regards, Leo Vegoda
Peter Müller via db-wg wrote on 29/06/2020 18:30:
While the RIPE Whois server returned "NL" as the country of that subnet [1], the "delegated-ripencc-extended-latest" file available at ftp.ripe.net [2] only contains 51.15.0.0/16, which points to "FR".
Since the first one result contains the correct country code of that IP network, I would like to ask why "delegated-ripencc-extended-latest" does not include subnets like this, and where the Whois server fetches his answer from.
The entirety of this /16 is assigned to a single organisation (Online.net), which is why there's a /16 in delegated-ripencc-extended-latest. This file contains only supernets and nothing more granular. Online.net created an inetnum object for 51.15.0.0/17 which sets the country to be NL. This is why the RIPE DB returns NL for that particular block. Incidentally, IP address allocation queries in the RIPE Database now return the country where the holder of the block is legally resident, not where the IP address is actually used. The situation with 51.15.0.0/17 and 51.15.0.0/16 is only the tip of a gigantic iceberg of quirks and anomalies, not just because you have inconsistent statements about the country, but also because this is legacy (i.e. pre-RIPE) address space and there can be multiple inetnum entries in the ripedb which may be fully inconsistent with each other and where all, some or none of the inetnum records may be an accurate reflection of reality. I.e. if you're planning to use either the RIPE DB or any other whois db for the ipfire geoip blocking mechanism, you may want to consider putting up very large and obtrusive warnings about how thoroughly inaccurate geoblocking may end up being in practice. Nick
Hello Nick, hello Leo, thanks for your fast and informative replies. I take the liberty to reply to both of you at the same time.
The RIPE Database, queried by Whois or RDAP, serves a different purpose from the stats file. The former enables communication between operators and supports the routing registry. The latter is to provide high level statistical data to researchers.
I see, it did not became clear to us that those file mainly exists for statistical purposes - we were glad to find something that has a unique syntax across all RIRs, instead of reintroducing the wheel for each one. Oh well... :-)
Incidentally, IP address allocation queries in the RIPE Database now return the country where the holder of the block is legally resident, not where the IP address is actually used.
Yes. This seems to be a common source of misunderstandings when trying to assign a country code to an IP network, as most people/applications/databases do not specify if they are trying to show the jurisdiction of that IP network, or the country where its machines are physically located. In my humble opinion, the latter is hard - if any - to determine.
The situation with 51.15.0.0/17 and 51.15.0.0/16 is only the tip of a gigantic iceberg of quirks and anomalies, not just because you have inconsistent statements about the country, but also because this is legacy (i.e. pre-RIPE) address space and there can be multiple inetnum entries in the ripedb which may be fully inconsistent with each other and where all, some or none of the inetnum records may be an accurate reflection of reality.
Although we have already sensed something in that direction, it is a bit frustrating to receive that message from someone in the RIPE DB working group... What would you do if you need to build a database to determine the country assigned to an IP address - let's take the one assigned to it by it's owner, regardless of jurisdiction or physical location -? Would you collect the most specific inetnums first, and proceed with the less specific ones? (Basically, this is what we trying to do as we need to replace the GeoIP database provided by MaxMind due to license changes - some other open-source projects are in need of an alternative as well -, and instead of just using the next commercial provider, we are trying to build an open-source solution. In case of further interest, please refer to: https://blog.ipfire.org/post/on-retiring-the-maxmind-geoip-database)
I.e. if you're planning to use either the RIPE DB or any other whois db for the ipfire geoip blocking mechanism, you may want to consider putting up very large and obtrusive warnings about how thoroughly inaccurate geoblocking may end up being in practice.
I believe we are more or less aware of that inaccuracies. Personally, I consider firewall rules based on Autonomous Systems Numbers to be _way_ more precise and practical - countries like US or NL are hard to block in productive environments. We are keen to raise awareness of those issues in our user community as well, but unfortunately are unable to drop the ability of selecting countries as source or destination of a firewall rule completely. For a quick look, it also might be interesting to show some geographical information so very unusual connections are easier to identify. Anyway: Thanks for your replies so far and thanks in advance for the upcoming ones. :-) Thanks, and best regards, Peter Müller
Peter Müller wrote on 29/06/2020 20:54:
What would you do if you need to build a database to determine the country assigned to an IP address - let's take the one assigned to it by it's owner, regardless of jurisdiction or physical location -?
There's no short or easy answer to this - if there were, there would be lots of geo-ip databases available on the internet. You would need to distil the information from a lot of separate sources (e.g. whois databases, detailed dfz routing dumps from multiple sources, pings from beacons to test rtt, possibly dns as a fuzzy input, etc). It would be a highly labour intensive process and you would be left with large numbers of question marks. Nick
On Mon, Jun 29, 2020 at 1:22 PM Nick Hilliard via db-wg <db-wg@ripe.net> wrote:
Peter Müller wrote on 29/06/2020 20:54:
What would you do if you need to build a database to determine the country assigned to an IP address - let's take the one assigned to it by it's owner, regardless of jurisdiction or physical location -?
There's no short or easy answer to this - if there were, there would be lots of geo-ip databases available on the internet.
Also, GeoIP data is collected to do different things. Some people care about jurisdiction, other care about territorial rights for media distribution, other care about language, and others care about latency. No single database is going to be good at doing all those things.
participants (3)
-
Leo Vegoda -
Nick Hilliard -
Peter Müller