PulseBeta

Archive and grace hold

Pulse keeps telemetry for 7 days (UTC days, today included). Then it’s yours: every day at 00:30 UTC, Pulse copies the day that just turned 8 days old to your own bucket, checks the copy, and deletes its own.

Your bucket

Set it in Settings → Storage (org owners and admins): any S3-compatible endpoint, such as AWS S3, Cloudflare R2, Backblaze B2 or Google Cloud Storage (interoperability keys). Pulse needs to put, read and delete objects; it checks this when you save. With a key prefix set there, everything below goes under that prefix. The archive is on your storage bill, not Pulse’s.

What lands there

pulse/logs/dt=2026-10-04/<id>.parquet
pulse/metrics/dt=2026-10-04/<id>.parquet
pulse/manifests/2026-10-04.json
  • Parquet files (Snappy), one schema per signal, in Hive-style dt= folders. Logs: time, observed, sev, sev_text, body, service, trace_id, span_id, scope, resource, attrs. Metrics: time, start, name, unit, service, kind, monotonic, temporality, value, count, min, max, buckets, resource, attrs. Times are UTC milliseconds; resource and attrs are JSON. Columns are only ever added at the end.
  • The manifest is written last, so if it’s there the day is complete: {org, day, schema: 1, files: [{key, signal, rows, bytes, md5, t0, t1}], written_at}.

Read the files with anything that reads Parquet, for example DuckDB:

SELECT service, count(*) FROM read_parquet('s3://my-bucket/pulse/logs/dt=2026-10-*/*.parquet') WHERE sev >= 17 GROUP BY 1;

The schema is Iceberg-compatible, so PyIceberg’s add_files can register the files in an Iceberg catalog without rewriting them.

Grace hold

If no bucket is set, or Pulse can’t write to it, the day goes on grace hold instead of being deleted:

  • Pulse emails the org’s owners and admins and sends an archive alert.
  • Every day it tries again, and emails a digest of the days held and when they’ll be dropped. It warns 48 hours before a drop.
  • At the end of the grace period the day is dropped: deleted for good. The email carries the day’s manifest.

The grace period is 7 days by default; set it from 0 to 30 days on the Archive page or with PUT /api/v1/settings {"grace_days": 14}. Zero drops a day at once. Held days aren’t queryable, and they’re billed per GB-day (see pricing).

Fixed the bucket? Press Retry now on the Archive page (POST /api/v1/archive/retry) rather than waiting for 00:30. GET /api/v1/archive and the MCP tool archive_status show each day’s state: archived, grace, or dropped, with the last error.

Next: MCP · This page as Markdown