Delete, migrate, back up and restore data

Target reader: project developers and local operators.

This is the current-format 0.4 Alpha lifecycle. It upgrades the G4 dataset in place; do not repeat the old Alpha reset. It does not restore old accounts, old JWTs or SDK v1/v2 queues. archived is still a feedback processing status.

Developer actions

An effective project owner/admin can delete a feedback report or close its project. Only an organization owner can close the organization and all its projects. In the Console, open the feedback detail, project Overview/Settings, or organization Settings, choose the action, and enter the exact displayed target ID. The server verifies the current account, session and membership. A Project Key cannot delete, read or export tenant data; system administrators have no tenant bypass.

The confirmation stops new access, uploads and download signatures. Closure revokes the affected Keys and removes scope memberships. Other projects/organizations and G4 anonymous historical user references remain. The cleanup receipt distinguishes pending, failed and completed; only the requesting account can view it. Keep its authenticated Console link and refresh it after maintenance. A timeout is unconfirmed; retry the same target. Actual object deletion must succeed before its metadata is removed. Copies already downloaded cannot be recalled, and earlier signed URLs may remain usable until their original expiry or object deletion.

Any current project member may choose Export project data. The complete ZIP contains visible feedback, comments, actual attachments and manifest.json with byte counts and SHA-256. Viewer contact fields remain trimmed by the server. There are no account records, passwords/hashes, tokens, Project Keys, invitations or signed URLs. Limits are 1000 reports and 128 MiB uncompressed; oversized or incomplete exports fail as a whole. Keep the downloaded copy private, verify every listed file and remove it when no longer needed. Deletion cannot recall that copy.

Finite retention and maintenance

Feedback, comments and attachments expire exactly 90 days after the report's received_at. New comments, attachments and retries do not renew this clock. Complete backups expire after 7 days; sanitized operational audit data and completed receipt requester associations expire after 180 days. The SDK local queue keeps its separate 7-day rule. Reads enforce expiry even before physical maintenance runs.

From the Infra repository, scripts/data-maintenance.sh --action inspect reports the deployment ID. Supply that exact ID to the following commands; --apply is required to mutate:

Minimal example

./scripts/data-maintenance.sh --action cleanup --deployment-id "$deployment_id" --batch 50
./scripts/data-maintenance.sh --action cleanup --deployment-id "$deployment_id" --batch 50 --apply
./scripts/data-maintenance.sh --action reconcile --deployment-id "$deployment_id"

The object/report budget is 1–100 per invocation, with separate finite expiry/auxiliary-record batches; repeat while pending remains. Failure has a nonzero exit code and preserves references for retry. An unlinked upload intent waits one hour before cleanup, including an already-absent compensated upload. Unknown objects are reported by digest and never removed by prefix guesses. Buckets must be private and unversioned, and their unique store marker must match the database. Minimal opaque scope/identity and client-ID digest tombstones remain for the deployment's lifetime to prevent late recreation; no report body or contact is retained in them. Expired sessions/codes are cleared without resetting issuance budgets early.

Operator backup and isolated recovery

Normal dev-apps-up.sh initializes/verifies the separate recovery-journal and recovery-witness volumes after migration. Every acknowledged restrictive change is independently durable. Neither authority volume belongs inside a database/object backup. Missing, stale, corrupted or unfinished authority blocks both APIs. Only the original authoritative database can reconcile an interrupted mutation with --action reconcile-recovery --deployment-id "$deployment_id" --apply.

Create a private backup root and run the coordinated helper from Infra:

python3 scripts/backup-local.py --root "$PWD/.backups"
python3 scripts/backup-local.py --root "$PWD/.backups" --apply

It fences the source, runs real pg_dump, captures registered object bytes, validates all checksums and records the manifest in the separate authority catalog before resuming the source. Recent unresolved uploads, unknown objects or missing accepted attachment bytes prevent completion. A partial directory is not a backup. Backup files contain sensitive database and object contents; keep their directory mode 0700 and files 0600. The helper reports the backup directory and manifest hash. Prune only the initialized root:

export DATA_BACKUP_ROOT="$PWD/.backups"
./scripts/data-maintenance.sh --action prune-backups --deployment-id "$deployment_id" --output /backup-data --batch 20
./scripts/data-maintenance.sh --action prune-backups --deployment-id "$deployment_id" --output /backup-data --batch 20 --apply

For recovery, select a complete unexpired backup_name from that root. Keep the current source project/environment selected. Verify it before running any pg_restore, then obtain a current source-fenced cutover seal:

./scripts/data-maintenance.sh --action verify-backup --deployment-id "$deployment_id" --output "/backup-data/$backup_name"
umask 077
mkdir -p .restore-control
export DATA_CONTROL_ROOT="$PWD/.restore-control"
./scripts/data-maintenance.sh --action begin-cutover --deployment-id "$deployment_id" --apply > "$DATA_CONTROL_ROOT/cutover.json"

Prepare a separate, empty local Compose project and private bucket using its own environment file, non-conflicting ports and credentials. Do not run migrations, seeds or APIs there before restoring the complete dump. Never restore into the source or an existing business database. The following variables identify that explicitly prepared destination; the source authority project is the still-fenced original:

docker compose --file docker-compose.yml --env-file "$restore_env" --project-name "$restore_project" up -d postgres minio
docker compose --file docker-compose.yml --env-file "$restore_env" --project-name "$restore_project" run --rm --no-deps minio-init
export RECOVERY_AUTHORITY_PROJECT="$source_project"
python3 scripts/restore-local.py --backup "$DATA_BACKUP_ROOT/$backup_name" --seal "$DATA_CONTROL_ROOT/cutover.json" --target-project "$restore_project" --target-env "$restore_env" --authority-project "$source_project"
python3 scripts/restore-local.py --backup "$DATA_BACKUP_ROOT/$backup_name" --seal "$DATA_CONTROL_ROOT/cutover.json" --target-project "$restore_project" --target-env "$restore_env" --authority-project "$source_project" --apply

The restore copies verified bytes into an empty destination store, replays the latest deletion/account/Key/member/SDK-gate facts and physically cleans deleted/expired objects before opening. Any failure keeps the destination isolated; retry the same backup and seal. Start its APIs only after the command reports completed, keeping RECOVERY_AUTHORITY_PROJECT set so dev-apps-up.sh uses the external authority override. Do not run down -v on the former source: it would remove authority still used by the replacement. Keep the old source stopped; retiring its data and relocating authority require the operator's reviewed scope.

All restored sessions, codes and pending invitations are invalid. Surviving accounts must reset passwords through the existing captured-email flow before logging in; deleted/disabled identities remain blocked. Shared active Keys survive personal account deletion, while revoked Keys and closed scopes remain unusable. Data/receipt metadata created after the backup point may be absent; only the explicitly journaled correction facts are retained independently. A missing necessary owner identity cannot be invented. Use a sufficiently recent backup.

python3 scripts/check-g5-data.py --recovery reproduces the owned PG/MinIO round trip and failure gates with synthetic data. This proves local controlled recovery, not production RPO/RTO, disaster survivability or an SLA. G6 must select actual independent storage, scheduling, monitoring, credentials, mail delivery and recovery targets.

The independent catalog reserves each incomplete backup before dump bytes are written. prune-backups also removes these reservations at their original 7-day expiry; retrying never renews it. To discard one immediately after resuming its source, use --action discard-backup --output "/backup-data/$backup_name" with the exact deployment and --apply. Approved backups and unknown files are refused. If the helper is killed, inspect recovery-status and resume only its exact source fence with resume-source --fence-id "$fence_id" --apply; keep it closed when ownership is uncertain.

The restore helper validates the independently approved archive before pg_restore, rejects active target APIs and nonempty unowned databases, and stores a private checkpoint in the seal directory. A failed import can repeat only for that same initially empty database and unchanged backup/seal/environment. Once correction begins, it never imports the old dump again. Target environment and control files must be 0600 and their private control directory 0700. After successful correction it leaves APIs stopped for the operator to start with the external authority binding.

Common mistakes

A backup containing only PostgreSQL is incomplete. Never copy an old journal together with the database and treat it as the latest authority. Never mark cleanup complete after an S3 failure, reuse a source bucket as a restore target, or clear a restored fence by hand.

Next step

Review account recovery, then run the isolated verification commands before adopting this procedure for real data. Production operational commitments belong to G6.