the rpki-validator.h2.mv.db file is already >22GB in size, I assume there is no way to reduce the disk usage requirements via a configuration change?
How big will the db file become?
There is a config setting regulating how often the validator will remove the "garbage", i.e. unused objects that it still stores but they are not really used but kept just in case (no connectivity to download updated records, broken manifests, CRLs, etc.). The setting is called "rpki.validator.rpki.object.cleanup.grace.duration" and by default it is set to "P7D", i.e. a period of 7 days, which is pretty conservative. You can set it to much smaller periods, 1-2 day or 12 hours and still have reasonable behaviour. There is another parameter “rpki.validator.validation.run.cleanup.grace.duration" with the similar meaning, but it's less crucial when it comes to storage space. Also Validator version 3 is lighter than 2 in a sense that it consumes less memory, but it can only do it by storing more data on the disk.
I hope this helps.
thanks for your reply. I changed it to: rpki.validator.rpki.object.cleanup.grace.duration=P1D in conf/application.properties, restarted the validator and expected some reduction in disk usage after a day or so, but the db folder uses almost 50GB now.. What disk usage should I expect / prepare for? There are also some errors shown on stdout (details see bellow). Since there doesn't appear much activity on the RPKI validator 3 issue tracker a general question: What is the general maintenance status of RPKI Validator 3? thanks, nusenu ERROR pool-1-thread-1 o.h.i.ExceptionMapperStandardImpl HHH000346: Error during managed flush [org.hibernate.exception.ConstraintViolationException: could not execute statement] ERROR pool-1-thread-1 o.s.s.s.TaskUtils$LoggingErrorHandler Unexpected error occurred in scheduled task. org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint ["RPKI_OBJECT__SHA256_IDX ON PUBLIC.RPKI_OBJECT(SHA256) VALUES (X'e17d74b2082b15701d300e24d97a3d7b0eb081d59034c6d026b9555b0509bc0e', 3443950)"; SQL statement: /* insert net.ripe.rpki.validator3.domain.RpkiObject */ insert into rpki_object (created_at, updated_at, version, authority_key_identifier, last_marked_reachable_at, serial_number, sha256, signing_time, type, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [23505-197]]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:259) at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:225) at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:540) at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746) at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714) at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:532) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:304) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at net.ripe.rpki.validator3.domain.validation.RpkiRepositoryValidationService$$EnhancerBySpringCGLIB$$3380f99b.validateRsyncRepositories(<generated>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112) at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:178) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3032) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3547) at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:89) at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:600) at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:474) at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337) at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39) at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1437) at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:494) at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3245) at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2451) at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:473) at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:156) at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38) at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:231) at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:68) at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:536) ... 21 common frames omitted -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu