Postmortem for whois incident on 16th and 24th June
Dear colleagues, Please find below the summary of incident for whois updates: When: Between approximately 20:00 UTC on Sunday, 16 June and 01:00 UTC on Monday 17 June, there was an intermittent interruption to the Whois Update service. On 24 June, between 08:03 and 08:21 UTC there was a subsequent incident. Why: The interruption occurred due to several unusual update messages containing many maintainers. Aside from this, there were a number of spurious update notification emails sent to uninvolved maintainers. Bounced mail messages from those maintainers were processed as updates which caused a lot more emails to be generated. Action: At the time of the incidents, we took measures to mitigate the impact to our users. On 24 June, we deployed the 1.112.1 release to production to mitigate this issue (validate max references). On 1 July, another release of whois 1.113.2 was made which improved our handling of multipart/mixed bounced and auto-submitted messages. The release notes are on the website: https://apps.db.ripe.net/docs/Release-Notes/#ripe-database-release-1-113-2 In the following weeks, we continued to monitor the whois updates and we did not observe any further instances since 24 June. We apologise to our users for the inconvenience this interruption may have caused. Regards, Mahesh
participants (1)
-
Mahesh Aggarwal