Path-Scoped CI/CD

LakePilot computes affected workspace members from changed files and the uv dependency graph instead of hand-maintained path filters.

Service descriptors

Each deployable service owns services/<service>/service.toml containing identifiers and repository paths only (never commands):

[service]
name = "icepack"
workspace_member = "services/icepack"
dockerfile = "services/icepack/Dockerfile"
ecr_repository = "icepack-api"
terraform_root = "services/icepack/terraform/icepack-api/env/{environment}"
owned_paths = ["ui/**", "tools/iceberg-inspector/**"]

The detector (scripts/ci/affected.py) parses the root pyproject.toml workspace, member dependencies, and descriptors, then emits test and deploy matrices. A changed shared library selects every transitive consumer. A path listed in owned_paths by multiple descriptors selects every owner — e.g. tools/iceberg-inspector/** is owned by both Icepack and IceLens, so a change there rebuilds and redeploys both. A path under services/ or libs/ that is not a registered member fails CI.

Workflows

Workflow	Trigger	Behavior
service-ci.yml	pull request	Detect affected members; focused ruff/mypy/pytest; build affected images (no push); render Helm charts; Terraform fmt/validate; dev Terraform plan
deploy-affected-services.yml	push to main	Deploy each affected service to dev (fail-fast: false, per-service concurrency)
reusable-service-deploy.yml	workflow_call	Shared OIDC auth, ECR build/push with short-SHA tags, Terraform plan/apply, prod approval via main-deploy
deploy-icepack.yml / deploy-icelens.yml	manual dispatch	Independent per-service promotion to any environment
publish-grafana-monitoring.yml	push to main (monitoring paths)	Publish Grafana Git Sync dashboards for Icepack and IceLens
publish-oaasis-monitoring.yml	push to main (monitoring paths)	Upload OaaSis alert manifests for Icepack and IceLens

Pull requests never apply Terraform. Merges apply dev only. Stage, preprod, and prod are explicit dispatches; prod requires the main-deploy protected environment. Promotions rebuild the selected revision in the target account with the same short git SHA tag, or deploy an existing ECR tag when image_tag is supplied.

Local usage

git diff --name-only --no-renames main HEAD > /tmp/changed.txt
uv run --no-project python -m scripts.ci.affected --changed-paths-file /tmp/changed.txt