Rotation and Retention

Offline NDJSON output and application logs both rotate with the same circular, index-based scheme.

How Rotation Works

When rotation is enabled, the writer rolls over before the next write would exceed max_size_mb.

Instead of timestamped files, the runtime uses indexed files such as:

connections.0.ndjson
connections.1.ndjson
connections.2.ndjson

With max_files: 3, the index wraps and overwrites the oldest numbered file:

connections.0.ndjson
connections.1.ndjson
connections.2.ndjson
connections.0.ndjson

Configuration

offline_output:
  output_file: "/var/log/pqc-sniffer/"
  rotation_enabled: true
  max_size_mb: 100
  max_files: 10

What to Expect

• retention is bounded by max_files * max_size_mb
• old content is overwritten when the index wraps
• this is not timestamp rotation
• directory targets produce indexed connections.N.ndjson files during rotation

Info

If you need long-term retention, ship rotated files elsewhere before the circular index wraps.

Application Logs

Application logs use the same circular, index-based rotation as NDJSON output, so the same bound applies. Files are named by index and reused in place:

pqc-sniffer.0.log
pqc-sniffer.1.log
pqc-sniffer.2.log

logging:
  level: info
  file:
    path: /var/log/pqc-sniffer/pqc-sniffer.log
    rotation_enabled: true
    max_size_mb: 50
    max_files: 5
    min_free_disk_mb: 100   # stop logging if the log filesystem drops below this (0 disables)
    rate_limit_per_sec: 0   # cap verbose records/sec (warn+ always pass; 0 disables)
    archive:                # retention-aware archiving (default off)
      enabled: false
      dir: ""               # empty => "<log dir>/archive"
      max_total_mb: 1000    # prune oldest archives past this total (0 = unlimited)
      max_age_days: 0       # also prune archives older than N days (0 = off)

When the index wraps, the reused log file is truncated before new records are written to it — the old generation is discarded, not appended to. Total log disk use is therefore bounded by max_files * max_size_mb (here, 5 × 50 MB = 250 MB), and no single file grows past max_size_mb.

Free-space safeguard

Circular truncation bounds the log files themselves, but the log filesystem can still fill from other sources (packet captures, the database, other services). As an independent backstop, min_free_disk_mb sets a floor: when free space on the log filesystem falls below it, the sink stops writing log records rather than help drive the disk to exhaustion, and logs a one-line notice to standard error. Logging resumes automatically once space is reclaimed. Set it to 0 to disable the guard (default: 100).

Rate limiting verbose logs

Because log records can be triggered by network traffic, an attacker generating traffic — especially with debug/info logging enabled — can inflate log volume and accelerate disk consumption. rate_limit_per_sec caps how many verbose (trace/debug/info) records are written to the log file each second using a token bucket. warn, error, and critical records are never limited, so real problems are always recorded, and any suppressed records are summarized in-band ([log-rate-limiter] suppressed N record(s)…) so nothing disappears silently. Set it to 0 to disable (default). This bounds the rate of growth; the truncating rotation and free-space guard bound the total.

Retention-aware archiving

By default, circular rotation discards the old generation when it reuses an index — bounded, but not retention-aware. Enabling archive turns reuse into archival without changing the circular rotation of the live files: immediately before a rotated index is truncated, its contents are gzip-compressed into dir as pqc-sniffer.<index>.<UTC-timestamp>.log.gz. A standard log shipper (filebeat/rsyslog/fluentd) pointed at that directory forwards the archives to central storage, so records survive beyond the local circular window.

Archiving does not reintroduce unbounded growth: max_total_mb and max_age_days prune the oldest archives back under budget. For an active investigation, place a .forensic_hold file in the archive directory — while it exists, pruning is suspended and no archive is deleted, regardless of budget. Remove the marker to resume normal pruning.

Note

Archiving preserves records; it does not push them off-host by itself. Point a log shipper at the archive directory (and the live indexed files) to forward to a central, access-controlled platform. Compression widens the local retention window for the same disk budget.

Warning

With archiving off, a reused index is overwritten on wraparound, so the local log window is finite. Enable archive (and/or ship logs centrally) if you need retention beyond max_files * max_size_mb for incident response or forensics.