Skip to content

Timestamping troubleshooting

This page covers a common instance-level issue: timestamping fails after restoring or migrating a LabID database, usually because the active TSA certificate files are missing from storage/.

Symptoms

  • Cloning an ELN experiment template fails when the template contains a timestamped attachment.
  • Uploading or embedding a file into an Experiment fails with a TimeStampError.
  • Logs mention a missing cacert.pem file for the active TSA, for example freeTSA.org.

Typical error messages look like this:

TSA 'freeTSA.org' root_certificate not found: .../storage/certificates/freeTSA.org_2026-04-20/cacert.pem
Failed to start timestamp process for: .../storage/attachments/.../Screenshot.png

Cause

LabID stores TSA certificate material in the instance storage directory. If a database is moved to a new server, or the storage volume is not restored together with the database, the TSA row can still point to a certificate path that no longer exists on disk.

This usually happens when:

  • the database was restored without the matching storage/certificates/... files
  • the TSA fixture was loaded after the migration that moved certificates
  • the storage directory or volume is not mounted where Django expects it
  • the certificate files were removed from storage/

Fix

  1. Verify that the expected certificate path exists on the server.
ls -l /opt/labid/labid/storage/certificates/
  1. Reload the TSA fixture so the certificate files are copied back into storage.
python scripts/management/load.py \
    -i core/fixtures/timestampauthorities.yml \
    -u <admin_username> \
    --update \
    --lookup-fields name
  1. Refresh the CRL data once the TSA files are back in place.
python manage.py shell -c "from core.tasks import auto_update_crl_lists; auto_update_crl_lists()"
  1. Restart the Celery workers and RabbitMQ if timestamping still fails after the files are restored.
sudo systemctl restart rabbitmq-server
sudo systemctl restart labid-celery-worker labid-celery-worker_archive labid-celery-worker_ingestion labid-celery-beat

Notes

  • If the error appears directly in the web request logs, check whether the deployment is running Celery in eager mode. In production, timestamping should normally be handled asynchronously.
  • This issue can affect more than template cloning. Any feature that timestamps attachments, PDFs, or archives can fail until the TSA certificate files are restored.