RRC12: no DB insertion since mid April?
Hi, Asking RIS DB for more detailed info regarding 2001:440:1880::/48 I was surprised to see only two entries returned vs. the three peers reported by RISwhois. Comparing the results I saw rrc12/DE-CIX was the one missing out; a further check of http://www.ris.ripe.net/cgi-bin/rrcstatus.cgi showed there's not been any data inserted from rrc12 for almost two months now! Are you aware of this? is there some specific, hard to fix, problem with the box? -- Rene
Hi Rene, I have noticed that it was not inserted for a while. The reason is the data we collect over there getting quite data, and DBinsert can not handle it anymore. I do not know what the quick solution would be. I will leave this ticket open and we will see how we can improve it. -- Arife Vural SED, RIPE NCC On Wed, 07 Jun 2006 19:00:54 +0200, Rene Wilhelm wrote: * Hi, * * Asking RIS DB for more detailed info regarding 2001:440:1880::/48 * I was surprised to see only two entries returned vs. the three * peers reported by RISwhois. * * Comparing the results I saw rrc12/DE-CIX was the one missing out; * a further check of http://www.ris.ripe.net/cgi-bin/rrcstatus.cgi * showed there's not been any data inserted from rrc12 for almost * two months now! * * Are you aware of this? is there some specific, hard to fix, * problem with the box? * * -- Rene
Hi Arife,
I have noticed that it was not inserted for a while. The reason is the data we collect over there getting quite data, and DBinsert can not handle it anymore.
Ah, I see.
I do not know what the quick solution would be.
I will leave this ticket open and we will see how we can improve it.
The raw data on lama suggests the activity on rrc12 has come down again to more "normal" levels, at least disk usage is of the same order, even a bit lower, than that of rrc03 at ams-ix: rrc00 rrc03 rrc12 Jan 430240 4312288 2009136 Feb 1174536 3503580 1822364 Mar 460220 3702528 2219576 Apr 730940 3544852 5987960 May 1997620 3488828 8392800 Jun 526332 851528 783260 Perhaps it is possible to switch DBinsert back on again while continuing investigation on how to deal with future hurricane size update storms? Cheers, -- Rene
Rene Wilhelm wrote:
Hi Arife,
I have noticed that it was not inserted for a while. The reason is the data we collect over there getting quite data, and DBinsert can not handle it anymore.
Ah, I see.
I do not know what the quick solution would be.
I will leave this ticket open and we will see how we can improve it.
The raw data on lama suggests the activity on rrc12 has come down again to more "normal" levels, at least disk usage is of the same order, even a bit lower, than that of rrc03 at ams-ix:
rrc00 rrc03 rrc12 Jan 430240 4312288 2009136 Feb 1174536 3503580 1822364 Mar 460220 3702528 2219576 Apr 730940 3544852 5987960 May 1997620 3488828 8392800 Jun 526332 851528 783260
Perhaps it is possible to switch DBinsert back on again while continuing investigation on how to deal with future hurricane size update storms?
It's not that dbinsert is not running, just that it's still processing data from April. To avoid this problem (and have current data inserted at the expense of delaying insertion of old data) dbinsert used to insert from the most recent file first ... but that then caused problems if an application assumed that database entries were in time-order. It might be worth flagging some of these old files as 'AI' or 'RI' in the updtlist table so that dbinsert will skip them and then gradually move them back to 'A'/'R' as processing time permits. James
Hi,
It might be worth flagging some of these old files as 'AI' or 'RI' in the updtlist table so that dbinsert will skip them and then gradually move them back to 'A'/'R' as processing time permits.
I think this is a good idea. Recent data with a hole in April/May is probably more useful than no data from June. Henk
James
------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------ 1160438400. Watch this space...
I skipped the data for April and May, and restarted Dbinsert from the begining of this month. Also run few optimize/reapir on mostly used SQL tables. It looks SQL queries are running in reasonable speed. Will keep an eye on it. On Mon, Jun 12, 2006 at 02:54:31PM +0200, Henk Uijterwaal wrote:
Hi,
It might be worth flagging some of these old files as 'AI' or 'RI' in the updtlist table so that dbinsert will skip them and then gradually move them back to 'A'/'R' as processing time permits.
I think this is a good idea. Recent data with a hole in April/May is probably more useful than no data from June.
Henk
James
------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------
1160438400. Watch this space...
-- Arife Vural SED, RIPE NCC
Hi Arife, DB status page shows RRC12 insertion reached middle of June 3d, so it's finally processing the old data faster than they originally came in (i.e. less than 24 hours work for 24 hours worth of updates). This gives hopes the rrc12 DBinsert will one day catch up with real time again :) Cheers, -- Rene On Mon, 12 Jun 2006, Arife Vural wrote:
I skipped the data for April and May, and restarted Dbinsert from the begining of this month.
Also run few optimize/reapir on mostly used SQL tables. It looks SQL queries are running in reasonable speed.
Will keep an eye on it.
On Mon, Jun 12, 2006 at 02:54:31PM +0200, Henk Uijterwaal wrote:
Hi,
It might be worth flagging some of these old files as 'AI' or 'RI' in the updtlist table so that dbinsert will skip them and then gradually move them back to 'A'/'R' as processing time permits.
I think this is a good idea. Recent data with a hole in April/May is probably more useful than no data from June.
Henk
participants (5)
-
arife＠ripe.net -
Henk Uijterwaal -
James Aldridge -
Rene Wilhelm -
RIPE NCC RIS