Warehouse CRD

A Warehouse declares an upstream source system — the system of record pipelines extract rows from, plus the verified credential to reach it. Data in Layer is derived from a warehouse and reconstructible from it. The serving side is the VectorStore; the two sit on opposite sides of the gateway.

apiVersion: hevlayer.com/v1alpha1
kind: Warehouse
metadata:
  name: prod-snowflake
  namespace: layer
spec:
  kind: snowflake
  snowflake:
    account: acme-xy12345
    user: SVC_LAYER
    role: SVC_LAYER_ROLE
    warehouse: EXTRACT_WH
    keyPairSecretRef:
      name: snowflake-rsa
    pool:
      size: 5
      timeout: 30s
  verifyInterval: 1h

Connection

Field	Purpose
kind	snowflake. databricks and iceberg are reserved by the schema but rejected by the operator until implemented.
snowflake.account	Snowflake account identifier.
snowflake.user	Service user the key pair authenticates.
snowflake.role	Optional role assumed on connect.
snowflake.warehouse	Snowflake compute warehouse extraction queries run on.
snowflake.keyPairSecretRef	Secret in the same namespace holding private-key.pem and optional passphrase. The credential is never stored in the CRD.
snowflake.pool	Connection pool tuning: size, timeout.
verifyInterval	Probe cadence. Defaults to 1h.

A warehouse is account, credential, and compute — not a catalog. Which database, schema, or table to read belongs to the pipeline source; one credential reaches many databases.

Verification

The operator probes the warehouse on apply, whenever the referenced Secret’s content changes, and every verifyInterval. For snowflake the probe opens a key-pair session, runs SELECT 1 on the declared compute warehouse, and closes.

Phase	Meaning
Pending	Not yet probed.
Verified	Last probe succeeded; status.verifiedAt is the probe time.
Failed	Last probe failed; status.failureReason says why.

Failed is a loud signal, not an outage: in-flight pipeline runs keep their connections, new runs refuse to start, and the condition surfaces in kubectl get warehouse and the dashboard. Pipelines start only against a Verified warehouse.

Rotation

Swap the referenced Secret’s content. The operator re-verifies and status.verifiedAt advances; consumers resolve credentials through the warehouse at connection-build time, so new connections pick up the new key with no redeploy. Pointing keyPairSecretRef at a different Secret name is a spec edit with the same flow.

Pipeline source

A pipeline extracting from a warehouse names it in spec.sourceRef. The source block owns the what — database, schema, query, cursor — and the warehouse owns the where and who:

spec:
  sourceRef:
    kind: snowflake
    warehouseRef: prod-snowflake
    database: ANALYTICS
    query: >-
      SELECT ID, TITLE, BODY, REFRESH_ID FROM PUBLIC.NOTES
      WHERE REFRESH_ID > :cursor
    cursor:
      column: REFRESH_ID

When sourceRef.kind is snowflake, the operator requires warehouseRef to name a Verified warehouse in the same namespace. It mounts the warehouse’s key-pair Secret into the worker pod at /var/run/hevlayer/warehouse/ and injects HEVLAYER_WAREHOUSE — connection JSON resolved from the warehouse spec (account, user, role, compute warehouse, pool), no credential material. The worker builds its own connection from the two; HEVLAYER_SOURCE_REF carries the source block verbatim as for any other source.

Keys

An ApiKey binds to a warehouse with a warehouse.<name> entitlement carrying a list of opaque claims strings. Layer stores and echoes the strings; the application routes on them. No client route reaches a source system — clients touch indexes, not warehouses — so the entitlement grants nothing in Layer, and it inerts when the warehouse is deleted.

Deletion

Deleting a warehouse fences everything drawing from it. A finalizer blocks deletion while status.consumers is non-zero — pipelines extracting from it or keys entitled to it — annotate with hevlayer.com/force-delete: "true" to override.

Status

status:
  phase: Verified
  verifiedAt: "2026-06-10T00:00:00Z"
  failureReason: null
  consumers:
    pipelines: 2
    apiKeys: 1

The operator emits Kubernetes Events on phase transitions and counts observed references in status.consumers.