Changes to the rpki archives on ftp.ripe.net
Dear colleagues, As you may know, the RIPE NCC publishes daily RPKI dumps for all trust anchors on ftp.ripe.net/rpki. Today, we have made some changes to this longitudinal dataset. We have added: * The compressed JSON output of a validator (output.json.xz). We added this because it contains validated ASPA and BGPsec payloads. * A compressed version of the CSV file of ROAs (roas.csv.xz) * A xz compressed archive of the validated and unvalidated objects (repo.tar.xz) We made this change because it significantly reduces the size of the files. At this point, we do not plan to re-compress historic files. However, to realise the savings, we must stop producing the larger version at some point and clean up the "duplicate" files from the transition period. For now, we will create the old (roas.csv, repo.tar.gz) and new (roas.csv.xz, repo.tar.xz) side by side during a transition period. We plan to stop creating the old format after a transition period of at least a month, taking into account the feedback we receive. We will inform you when this happens. Ties de Kock Specialist Software Engineer RIPE NCC
Dear colleagues, A while ago we announced changes to the RPKI archives on our FTP server [0]. Based on your feedback, we have compressed all historic roa.csv files (to roa.csv.xz) and repo.tar.gz files (to repo.tar.xz). This results in a uniform set of files that is easier to process than a mix of compressed and uncompressed files. As announced earlier, we will stop creating the roa.csv and repo.tar.gz files. We will do so after the 12th of February. Afterwards, we will clean up the old, now recompressed files (roas.csv, repo.tar.gz). Kind regards, Ties de Kock RIPE NCC [0]: https://ftp.ripe.net/rpki/
On 11 Oct 2023, at 08:59, Ties de Kock <tdekock@ripe.net> wrote:
Dear colleagues,
As you may know, the RIPE NCC publishes daily RPKI dumps for all trust anchors on ftp.ripe.net/rpki. Today, we have made some changes to this longitudinal dataset.
We have added: * The compressed JSON output of a validator (output.json.xz). We added this because it contains validated ASPA and BGPsec payloads. * A compressed version of the CSV file of ROAs (roas.csv.xz) * A xz compressed archive of the validated and unvalidated objects (repo.tar.xz)
We made this change because it significantly reduces the size of the files. At this point, we do not plan to re-compress historic files. However, to realise the savings, we must stop producing the larger version at some point and clean up the "duplicate" files from the transition period.
For now, we will create the old (roas.csv, repo.tar.gz) and new (roas.csv.xz, repo.tar.xz) side by side during a transition period. We plan to stop creating the old format after a transition period of at least a month, taking into account the feedback we receive. We will inform you when this happens.
Ties de Kock Specialist Software Engineer RIPE NCC
participants (1)
-
Ties de Kock