Dear colleagues,

As we shared back in May, we have been working to reduce our data centre footprint by managing the data for RIS and RIPEstat more efficiently[1]. We have experienced some problems which mean that parts of the historical RIPEstat data will not be available for an extended period.

To reduce our data centre usage, we have been exporting RIS-related datasets from the database that contains our largest datasets (HBase) to a new environment, which allows us to decommission our old Hadoop cluster. During the migration, we encountered timeouts and memory limitations with old servers and software that held historical data.

Here we faced a difficult choice: we could either extend our data centre contracts at considerable additional cost, or we could accept that certain historical data would not be available to users for an extended period. We decided on the second approach.

It is important to be clear that we haven’t lost anything here. However, this does mean that we will not be able to provide aggregated historical data for a number of API calls and their corresponding widgets for some time after we complete our migration. These include: as-path-length, as-routing-consistency, asn-neighbours, asn-neighbours-history, bgp-update-activity, bgp-state, bgp-updates, bgplay, ris-peerings, routing-status, visbility.

As mentioned, this has been a difficult process. We may still encounter other issues when we make the switch to the new environment tomorrow (24 September). My team is doing their utmost to keep any disruption to a minimum, but there are no guarantees everything will go to plan.

We will be looking at the most efficient way to rebuild access to the historical data in the coming months, and we will share more details in an article on RIPE Labs soon.

Kind regards,

Felipe Victolla Silveira

Chief Technology Officer

RIPE NCC

[1] For more on this, refer to the following:
- Reducing our Data Centre Footprint (RIPE Labs):
https://labs.ripe.net/author/felipe_victolla_silveira/reducing-the-ripe-nccs-data-centre-footprint/
- RIPE 88 Technology Update (slides 10-20):
https://ripe88.ripe.net/wp-content/uploads/presentations/89-RIPE-88-RIPE-NCC-Technology-Update-Final.pdf 

-----
To unsubscribe from this mailing list or change your subscription options, please visit: https://mailman.ripe.net/mailman3/lists/mat-wg.ripe.net/
As we have migrated to Mailman 3, you will need to create an account with the email matching your subscription before you can change your settings.
More details at: https://www.ripe.net/membership/mail/mailman-3-migration/