RIPE Database Proxy Service Issues
[Apologies for duplicate emails] Dear colleagues, There has been discussion on various mailing lists regarding the status of the RIPE Database Proxy Service. Before I address the issues that arose, I'd like to give you some background information on the service itself that may help with the discussions. Technical Background -------------------- To prevent the automatic harvesting of personal information (real names, email addresses, phone numbers) from the RIPE Database, there are PERSON and ROLE object query limits defined in the RIPE Database Acceptable Use Policy. This is set at 1,000 PERSON or ROLE objects per IP address per day. Queries that result in more than 1,000 objects with personal data being returned result in that IP address being blocked from carrying out queries for that day. Users of the RIPE Database have unlimited access to Network Information Centre (NIC)-related objects. They can use the -r flag in order to filter out personal objects and query NIC objects without any limitations. The RIPE Database Proxy Service allows websites to provide a third party interface to the RIPE Database. Without the proxy service, the third parties would quickly run into the limits set on RIPE Database queries. With the proxy service, we whitelist the third party IP address and ask them to pass their user's IP address to us, so limits are only set on the user's IP address, not the third party's. There is no technical way to ensure that the user IP addresses passed to us by the third party are valid. Potentially, third party users of the proxy service could harvest all personal data in the RIPE Database (approximately 2 million objects) in a matter of hours. To ensure that the RIPE NCC's Terms and Conditions are followed, we require a contract between the third party and the RIPE NCC. Users of the Proxy Service -------------------------- In the past ten years, the RIPE NCC has had 31 requests for the proxy service and over the past year, there have been only four active users of the service. Of these four, one is already a RIPE NCC member. NIC Information --------------- All NIC information is still available without access to the proxy service. In the normal presentation of whois data, there is a redirect system that allows users with a normal whois client to deal directly with the RIPE Database whois service. There is no need for a proxy service in this scenario. The proxy service is only necessary if the data needs to be presented in alternative forms, such as on a third party's website. The limits imposed on RIPE Database queries only apply to personal data. Users can always access NIC data in any form they like if they are happy not to receive personal data. On 6 March 2012, the RIPE NCC proposed to change the default behaviour of the query system to instead return only "ALLOWED" results if a user had reached their daily personal data query limit, but there was disagreement over this on the mailing list so the change was not implemented. The proposal is available at: http://www.ripe.net/ripe/mail/archives/db-wg/2012-March/003885.html Legal Considerations -------------------- The RIPE NCC operates under European Data Protection laws, so to avoid risk in this area we insist on having a contract with third parties who wish to use the proxy service. The RIPE NCC and its Executive Board believes that the proxy service should become a member service because it tightens the contractual relationship between the RIPE NCC and third parties. Currently, no such agreement that meets the EU Data Protection legislation is in place between the RIPE NCC and the proxy service users. In order to tighten the contractual relationship between the RIPE NCC and the Proxy service users, taking into account the recent approval of the Charging Scheme 2013 that caused a simplification of the contractual agreements between the RIPE NCC and its service users, the RIPE NCC offered to conclude the membership agreement for continuation of the service. Next Steps? ------------ The Executive Board approved changes to the draft version of the Activity Plan and Budget 2013, and the RIPE NCC published the final version on 13 December 2012: http://www.ripe.net/internet-coordination/news/announcements/ripe-ncc-activi... We do apologise, however, that the changes regarding the proxy service were not more explicitly communicated to the members and the RIPE community in advance of the final publication of the Activity Plan. The RIPE NCC asks that non-RIPE NCC member proxy service users become members but we propose to waive their membership fee until the discussion of the RIPE NCC Charging Scheme 2014 takes place. This will give the membership and community the opportunity to discuss the best way forward for the proxy service in the coming months while ensuring a strong contractual bond between the RIPE NCC and users of this service. In the meantime, there will be no changes to the proxy service and no loss of functionality for the community. The RIPE NCC and its Executive Board will return to its members with proposals for ways to ensure that their wishes are met with regard to service developments while allowing the RIPE NCC to be operate efficiently and responsively. If you have any comments on this issue, please direct them to the RIPE NCC Services Working Group mailing list <ncc-services-wg@ripe.net>. Best regards, Axel Pawlik Managing Director RIPE NCC
* Axel Pawlik:
Users of the RIPE Database have unlimited access to Network Information Centre (NIC)-related objects. They can use the -r flag in order to filter out personal objects and query NIC objects without any limitations.
Interesting. This does not work for me: fw@deneb:~$ printf '%s\r\n' "-r 193.0.14.129" | nc whois.ripe.net 43 % This is the RIPE Database query service. % The objects are in RPSL format. % % The RIPE Database is subject to Terms and Conditions. % See http://www.ripe.net/db/support/db-terms-conditions.pdf %ERROR:201: access denied for 85.183.209.94 % % Queries from your IP address have passed the daily limit of controlled objects. % Access from your host has been temporarily denied. % For more information, see % http://www.ripe.net/data-tools/db/faq/faq-db/why-did-you-receive-the-error-2... % This query was served by the RIPE Database Query Service version 1.47.5 (WHOIS1) fw@deneb:~$ Before that, I managed to blow my entire daily quota with a single query, namely "-r r" (typo for "?", requesting help).
-----Original Message----- From: anti-abuse-wg-bounces@ripe.net [mailto:anti-abuse-wg- bounces@ripe.net] On Behalf Of Florian Weimer Sent: Wednesday, January 02, 2013 8:50 PM To: ripencc-management@ripe.net Cc: anti-abuse-wg@ripe.net
* Axel Pawlik:
Users of the RIPE Database have unlimited access to Network Information Centre (NIC)-related objects. They can use the -r flag in order to filter out personal objects and query NIC objects without any limitations.
Interesting. This does not work for me:
fw@deneb:~$ printf '%s\r\n' "-r 193.0.14.129" | nc whois.ripe.net 43
%ERROR:201: access denied for 85.183.209.94 % % Queries from your IP address have passed the daily limit of controlled objects. % Access from your host has been temporarily denied.
Before that, I managed to blow my entire daily quota with a single query, namely "-r r" (typo for "?", requesting help).
But Axel Pawlik also wrote:
On 6 March 2012, the RIPE NCC proposed to change the default behaviour of the query system to instead return only "ALLOWED" results if a user had reached their daily personal data query limit, but there was disagreement over this on the mailing list so the change was not implemented.
He also referred to http://www.ripe.net/ripe/mail/archives/db-wg/2012-March/003885.html, wherein it is stated: 'The current blocking mechanism works on the basis of all or nothing. If you are not blocked, you can successfully query for any data in the RIPE Database. If you are blocked (either temporarily or permanently), any query from you is rejected with a "Denied access" error message.' I assume that the above explains your predicament. -- Thor Kottelin http://www.anta.net/
* Thor Kottelin:
But Axel Pawlik also wrote:
On 6 March 2012, the RIPE NCC proposed to change the default behaviour of the query system to instead return only "ALLOWED" results if a user had reached their daily personal data query limit, but there was disagreement over this on the mailing list so the change was not implemented.
He also referred to http://www.ripe.net/ripe/mail/archives/db-wg/2012-March/003885.html, wherein it is stated: 'The current blocking mechanism works on the basis of all or nothing. If you are not blocked, you can successfully query for any data in the RIPE Database. If you are blocked (either temporarily or permanently), any query from you is rejected with a "Denied access" error message.'
I assume that the above explains your predicament.
Oh, it seems so. I guess it would be nice to have an option which says, "please process this query in a way which does not cause blocking". I mistakenly assumed that "-r" would be that option, but it actually isn't.
-----Original Message----- From: anti-abuse-wg-bounces@ripe.net [mailto:anti-abuse-wg- bounces@ripe.net] On Behalf Of Florian Weimer Sent: Wednesday, January 02, 2013 9:44 PM To: Thor Kottelin Cc: anti-abuse-wg@ripe.net
I guess it would be nice to have an option which says, "please process this query in a way which does not cause blocking". I mistakenly assumed that "-r" would be that option, but it actually isn't.
I'm not sure why you were blocked initially despite using the -r switch. I was able to reproduce the problem: the IP address I used became blocked as well. Perhaps this was some kind of overload protection rather than the privacy-related restriction we have been discussing. -- Thor Kottelin http://www.anta.net/
On Jan 2, 2013, at 10:00 PM, Thor Kottelin <thor@anta.net> wrote:
I'm not sure why you were blocked initially despite using the -r switch. I was able to reproduce the problem: the IP address I used became blocked as well. Perhaps this was some kind of overload protection rather than the privacy-related restriction we have been discussing.
Hello Thor, This should not happen, could you please kindly contact us through ripe-dbm@ripe.net and give us your client IP address so we can investigate the cause. You should not get blocked when using "-r" flag with a RESOURCE query and there should be no overload issue at all. Please note "-r" flag will turn off recursion for contact information after retrieving the objects that match a query. If the query is directly for personal data (like a person's name) using "-r' would have no effect as the direct query results will be objects with personal data. Kind Regards, Kaveh. --- Kaveh Ranjbar, RIPE NCC Database Group Manager
-----Original Message----- From: anti-abuse-wg-bounces@ripe.net [mailto:anti-abuse-wg- bounces@ripe.net] On Behalf Of Kaveh Ranjbar Sent: Thursday, January 03, 2013 12:11 AM To: Thor Kottelin Cc: anti-abuse-wg@ripe.net
On Jan 2, 2013, at 10:00 PM, Thor Kottelin <thor@anta.net> wrote:
I'm not sure why you were blocked initially despite using the -r
was able to reproduce the problem: the IP address I used became blocked as well. Perhaps this was some kind of overload protection rather
switch. I than the
privacy-related restriction we have been discussing.
Please note "-r" flag will turn off recursion for contact information after retrieving the objects that match a query. If the query is directly for personal data (like a person's name) using "- r' would have no effect as the direct query results will be objects with personal data.
Thanks. This explains the phenomenon. -- Thor Kottelin http://www.anta.net/
Dear Florian As Kaveh pointed out, the -r flag simply says don't recusively search through the objects returned in response to the query and find referenced personal data. But if the query itself is for a personal data object the -r flag will have no effect. Your query "-r r" is looking for all objects in the RIPE Database where the primary key containes the letter 'r'. As most nic-handles end in '-ripe' you queried for all persoanl data directly. This is why your single query blocked you. Your idea for a 'non blocking' query option is along the lines of our technical suggestion from last year, which Axel referred to. Perhaps the community would like to take another look at that as there was no consensus last year. The current blocking mechanism is not optimal and could be improved. Regards Denis Walker Business Analyst RIPE NCC Database Group
* Thor Kottelin:
But Axel Pawlik also wrote:
On 6 March 2012, the RIPE NCC proposed to change the default behaviour of the query system to instead return only "ALLOWED" results if a user had reached their daily personal data query limit, but there was disagreement over this on the mailing list so the change was not implemented.
He also referred to http://www.ripe.net/ripe/mail/archives/db-wg/2012-March/003885.html, wherein it is stated: 'The current blocking mechanism works on the basis of all or nothing. If you are not blocked, you can successfully query for any data in the RIPE Database. If you are blocked (either temporarily or permanently), any query from you is rejected with a "Denied access" error message.'
I assume that the above explains your predicament.
Oh, it seems so. I guess it would be nice to have an option which says, "please process this query in a way which does not cause blocking". I mistakenly assumed that "-r" would be that option, but it actually isn't.
participants (5)
-
Axel Pawlik
-
denis@ripe.net
-
Florian Weimer
-
Kaveh Ranjbar
-
Thor Kottelin