James Aldridge wrote:
Take a look at http://www.ris.ripe.net/ipv6-updates/all/ (or any of the individual RRC's plots) and you'll see a certain periodicity about the graph. It seems that the RRD is only getting about half the data:
5 minutes and 35 minutes past each hour:
weesp runs the "collect" script and pushes the raw files to halfweg.
0 minutes past each hour:
halfweg runs the "ipv6-updates/update.sh" script looking for files from the "previous hour" (date -d "1 hour ago" +"%Y%m%d.%H"). However, at this time, only files up to the last run of "collect" (25 minutes earlier) are present - YYYYMMDD.HH00 to YYYYMMDD.HH35.
Next time "update.sh" runs, an hour later, it again only looks at files from the previous hour (and not the missing files from the hour before that). Data from the last 25 minutes of each hour are never inserted into the RRD.
A solution to this would be for risops@halfweg's crontab to run the "ipv6-updates/update.sh" sufficiently long after 5 minutes past the hour for the "collect" run on weesp to have pushed all the previous hour's files to halfweg.
Thoughts?
In general I think that modifying timing of various jobs is the wrong tactic for dealing with timing problems. If you have a dependency between two tasks, best to make it explicit. In the world of Unix and cron, the way to do this is to make a simple script that runs the two commands in order, one after the other. If the commands are running on separate boxes, then the easiest solution is to set up a provider/consumer system where one box touches a file when the data is ready, and the other deletes it when used. In the halfweg/weesp situation, "collect" would touch a file when the data has arrived, and "update.sh" would check for this. We can run "update.sh" every minute (or every 5 minutes if we're squeemish) in this case, reducing the latency to the absolute minimum. -- Shane Kerr RIPE NCC