API

Backend

Embed

Add an embed block to a string attribute in your schema and Layer computes that attribute’s vector on every write. Put ["Embed", "..."] in rank_by and Layer computes the query vector with the same model. Your application never loads a model or sends a vector of its own.

Example

This write creates the articles namespace with an embedded text attribute and upserts two rows. Layer embeds each row’s text with MiniLM, one of the CPU models, before storing it.

await client.write_namespace("articles", {
    "upsert_rows": [
        {"id": "planet-1", "title": "Planet",
         "text": "Jupiter is the biggest planet in the Solar System."},
        {"id": "photo-1", "title": "Photosynthesis",
         "text": "Plants turn sunlight, water, and carbon dioxide into food."},
    ],
    "distance_metric": "cosine_distance",
    "schema": {
        "text": {
            "type": "string",
            "embed": {
                "model": "sentence-transformers/all-MiniLM-L6-v2",
                "dims": 384,
                "serving": {"prefer": "local"},
            },
        },
    },
})

_, err := client.WriteNamespace(ctx, "articles", hevlayer.TurbopufferWriteRequest{
    "upsert_rows": []map[string]any{
        {"id": "planet-1", "title": "Planet",
            "text": "Jupiter is the biggest planet in the Solar System."},
        {"id": "photo-1", "title": "Photosynthesis",
            "text": "Plants turn sunlight, water, and carbon dioxide into food."},
    },
    "distance_metric": "cosine_distance",
    "schema": map[string]any{
        "text": map[string]any{
            "type": "string",
            "embed": map[string]any{
                "model":   "sentence-transformers/all-MiniLM-L6-v2",
                "dims":    384,
                "serving": map[string]any{"prefer": "local"},
            },
        },
    },
})

await client.writeNamespace("articles", {
  upsert_rows: [
    { id: "planet-1", title: "Planet",
      text: "Jupiter is the biggest planet in the Solar System." },
    { id: "photo-1", title: "Photosynthesis",
      text: "Plants turn sunlight, water, and carbon dioxide into food." },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: {
      type: "string",
      embed: {
        model: "sentence-transformers/all-MiniLM-L6-v2",
        dims: 384,
        serving: { prefer: "local" },
      },
    },
  },
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "planet-1", "title": "Planet",
       "text": "Jupiter is the biggest planet in the Solar System."},
      {"id": "photo-1", "title": "Photosynthesis",
       "text": "Plants turn sunlight, water, and carbon dioxide into food."}
    ],
    "distance_metric": "cosine_distance",
    "schema": {
      "text": {
        "type": "string",
        "embed": {
          "model": "sentence-transformers/all-MiniLM-L6-v2",
          "dims": 384,
          "serving": { "prefer": "local" }
        }
      }
    }
  }'

Query by meaning with Embed. Layer reads the model from the text schema and embeds the query string with it:

response = await client.query_namespace("articles", {
    "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]],
    "top_k": 3,
    "include_attributes": ["title", "text"],
})
print(response.rows, response.performance)

response, err := client.QueryNamespace(ctx, "articles", &hevlayer.QueryRequest{
    RankBy:            []any{"text", "ANN", []any{"Embed", "largest planet in the solar system"}},
    TopK:              3,
    IncludeAttributes: []string{"title", "text"},
})

const response = await client.queryNamespace("articles", {
  rank_by: ["text", "ANN", ["Embed", "largest planet in the solar system"]],
  top_k: 3,
  include_attributes: ["title", "text"],
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]],
    "top_k": 3,
    "include_attributes": ["title", "text"]
  }'

{
  "rows": [
    { "id": "planet-1", "$dist": 0.137, "title": "Planet",
      "text": "Jupiter is the biggest planet in the Solar System." }
  ],
  "performance": {
    "embedding_tokens": 7,
    "embedding_ms": 9
  }
}

Where the vector comes from

embed.serving.prefer picks the provider that runs the model:

`embed.serving.prefer`	Who computes the vector
`local`	The locally configured embedder: the `layer-embed` CPU service the gateway reaches at `LAYER_EMBED_URL`, or an in-process Lattice or CLIP artifact. No key, no GPU, no account.
`worker`	A GPU worker in your own cluster, serving an open-weights checkpoint you declared.
`turbopuffer`	Turbopuffer’s hosted embedding service, from its managed model menu.

The provider is independent of the store the vector is written to, so any store that accepts embed can use any configured provider.

Layer never substitutes one provider for another. A profile that selects a provider the deployment has not configured fails validation at write time, and a provider that fails at request time returns an error instead of falling back to a different model.

native is accepted as an alias for turbopuffer, autoscaler for worker, and lattice for local on a Lattice profile.

All three providers are available. Layer validates a turbopuffer profile and forwards it unchanged to the upstream POST /v2/namespaces/{ns} and POST /v2/namespaces/{ns}/query, so Turbopuffer computes the vector and it never passes through Layer. local and worker compute the vector in Layer and write an ordinary ANN column.

Embedding expressions / schema: supported

Per-store resolution

Omitting serving selects turbopuffer (native). Only Turbopuffer can compute that vector itself. On a store that cannot embed, Layer computes the vector and the store receives an ordinary vector column. The rule depends on the store, never on the request:

Store	`turbopuffer` (the default)	`local`, `worker`
Turbopuffer	Native: forwarded unchanged, and Turbopuffer embeds.	Layer embeds and writes an ANN column.
hev search	Gateway-resolved through the configured Turbopuffer embedding provider.	Layer embeds.
Postgres	Gateway-resolved: the `local` embedder if it lists the model, otherwise the configured Turbopuffer embedding provider.	Layer embeds.

Layer resolves a gateway-resolved profile once, on the write that declares the attribute, and the namespace keeps that provider for good. If neither provider serves the model, the write returns 503 and nothing is stored.

Postgres takes the same embed declaration as Turbopuffer. On Community Edition Compose, which runs the CPU service and has no Turbopuffer key, a profile naming one of the CPU models needs no serving block:

{
  "schema": {
    "text": {"type": "string", "full_text_search": true,
             "embed": {"model": "BAAI/bge-small-en-v1.5"}}
  },
  "upsert_rows": [
    {"id": "planet-1", "text": "Jupiter is the biggest planet in the Solar System."}
  ]
}

Layer embeds each row before the write’s transaction, so if the provider fails, no rows or schema are written. The vector lands in embed_<attribute> (or embed.attribute), declared [N]f32 on the first write that carries rows; a schema-only write declares no vector column yet. distance_metric is optional and defaults to cosine_distance. rank_by: [attr, "ANN", ["Embed", text]] and Auto both embed the query text. Multi-query bodies are not supported on Postgres; use Auto or HybridText.

The profile is stored in Postgres next to the namespace schema, so it survives a gateway restart with no object store configured, and GET .../schema returns the embed block you wrote. These return 422 UnsupportedByStore, and nothing is written:

Request	`feature`
A second attribute declares `embed`	`schema.embed`
A later write changes `model`, `dims` or the provider	`schema.embed`
A client `vector` in the same namespace	`max_vector_fields`
`embed.chunk`	`embed.chunk`

A Postgres namespace holds either gateway-embedded vectors or client vectors, never both. To change models, write to a new namespace.

Embedding expressions / schema: approximate — Gateway-resolved embedding only; one embedded attribute per namespace; chunked embedding returns 422.

CPU models

Layer serves three CPU embedding models, small enough to run on a laptop under Docker. They use the same schema as the larger GPU and hosted models, with lower retrieval quality. Community Edition Compose starts the text service next to the gateway, and the Helm chart runs it as a gateway sidecar; neither needs a key, GPU or download after the image pull, and the service has no route out of the machine.

Model	Dims	Modality	Use it for
`sentence-transformers/all-MiniLM-L6-v2`	384	text	Start here. 22M parameters, symmetric, no prefixes.
`BAAI/bge-small-en-v1.5`	384	text	Better retrieval for 50% more parameters. Try it when MiniLM’s results are weak.
`openai/clip-vit-base-patch32`	512	image	Text-to-image search. Both towers run on CPU.

The two text models are served by layer-embed, a separate CPU process whose image contains both sets of weights. The gateway reaches it over a private network at LAYER_EMBED_URL (for example http://embed:8081); nothing is downloaded at startup, and the gateway image carries no model weights. Set the variable and prefer: local resolves any model the service advertises. Leave it unset and a prefer: local write that names a text model returns 422. CLIP runs inside the gateway process from the directory named by LAYER_LOCAL_CLIP_MODEL_PATH; see Images.

There is no default model: a prefer: local profile always names one. Omit dims and Layer uses the dimension the service advertises for that model. Every model here pins a fixed artifact; revision, instructions, and chunk are not accepted on a local profile.

Query embedding has a 10 second budget and each write 60 seconds, shared by every batch of that write. Set LAYER_EMBED_QUERY_TIMEOUT_MS and LAYER_EMBED_WRITE_TIMEOUT_MS to change them. A service that is still loading returns 503 for local profiles only; native and GPU profiles are unaffected.

Text

The two text models are embedded differently, and Layer handles both; the example at the top of this page uses MiniLM. bge expects an instruction prefix on queries and MiniLM expects none, so the gateway adds the prefix exactly once, only when it embeds a query, and never on the text it stores. bge uses CLS pooling and MiniLM uses mean pooling. Both outputs are L2-normalized, so cosine_distance scores are comparable across rows. Switching a namespace between them requires a full re-index.

Images

With modality: image on a CLIP model, Layer embeds each attribute value with the image tower at write time and embeds Embed query text with the text tower, against the same vector column. Both towers run on CPU beside the gateway.

await client.write_namespace("photos", {
    "upsert_rows": [
        {"id": "commons-131", "title": "Sunset at Porto Covo",
         "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"},
    ],
    "distance_metric": "cosine_distance",
    "schema": {
        "image_url": {
            "type": "string",
            "embed": {
                "model": "openai/clip-vit-base-patch32",
                "dims": 512,
                "modality": "image",
                "serving": {"prefer": "local"},
            },
        },
    },
})

_, err := client.WriteNamespace(ctx, "photos", hevlayer.TurbopufferWriteRequest{
    "upsert_rows": []map[string]any{
        {"id": "commons-131", "title": "Sunset at Porto Covo",
            "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"},
    },
    "distance_metric": "cosine_distance",
    "schema": map[string]any{
        "image_url": map[string]any{
            "type": "string",
            "embed": map[string]any{
                "model":    "openai/clip-vit-base-patch32",
                "dims":     512,
                "modality": "image",
                "serving":  map[string]any{"prefer": "local"},
            },
        },
    },
})

await client.writeNamespace("photos", {
  upsert_rows: [
    { id: "commons-131", title: "Sunset at Porto Covo",
      image_url: "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" },
  ],
  distance_metric: "cosine_distance",
  schema: {
    image_url: {
      type: "string",
      embed: {
        model: "openai/clip-vit-base-patch32",
        dims: 512,
        modality: "image",
        serving: { prefer: "local" },
      },
    },
  },
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "commons-131", "title": "Sunset at Porto Covo",
       "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"}
    ],
    "distance_metric": "cosine_distance",
    "schema": {
      "image_url": {
        "type": "string",
        "embed": {
          "model": "openai/clip-vit-base-patch32",
          "dims": 512,
          "modality": "image",
          "serving": { "prefer": "local" }
        }
      }
    }
  }'

Query the image column with text:

response = await client.query_namespace("photos", {
    "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]],
    "top_k": 16,
    "include_attributes": ["title", "image_url"],
})

response, err := client.QueryNamespace(ctx, "photos", &hevlayer.QueryRequest{
    RankBy:            []any{"image_url", "ANN", []any{"Embed", "sunset over water"}},
    TopK:              16,
    IncludeAttributes: []string{"title", "image_url"},
})

const response = await client.queryNamespace("photos", {
  rank_by: ["image_url", "ANN", ["Embed", "sunset over water"]],
  top_k: 16,
  include_attributes: ["title", "image_url"],
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]],
    "top_k": 16,
    "include_attributes": ["title", "image_url"]
  }'

{
  "rows": [
    { "id": "commons-131", "$dist": 0.412, "title": "Sunset at Porto Covo",
      "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" }
  ],
  "performance": {
    "embedding_tokens": 5,
    "embedding_ms": 134
  }
}

Write responses report embedding_images instead of embedding_tokens. CPU CLIP handles query-time text embeds and write-time image embeds for small and medium corpora. A bulk image backfill needs a GPU. The provider is set per namespace, so a backfill can run on a GPU while queries embed on CPU.

The lens demo runs this setup: Wikimedia Commons Quality images embedded and searched on CPU, with the gateway’s performance echo shown beside each result. Source at github.com/hev/lens.

Image input

An image profile accepts an HTTP(S) URL or a base64 image string and fetches at most 20 MiB per URL. A rate-limited image host returns HTTP 429 with error upstream_error; a server error from the image host returns HTTP 503 with error service_unavailable. Layer preserves the host’s Retry-After response header for both. Malformed URLs or base64, non-retryable 4xx responses, oversize content, and undecodable images return 422 validation_error.

Changing models requires a re-index

The two text models both emit 384 dimensions, so switching between them leaves the namespace schema valid. It still requires re-embedding every row.

Different models place vectors in unrelated spaces. If documents were embedded with MiniLM and queries are embedded with bge, the distances compare incompatible vectors and the ranking is meaningless. The same applies when moving from a CPU model to a hosted or GPU model, and when the weights, tokenizer, pooling, or prefixes behind a model id change: equal dimensions never make two vector spaces compatible.

Layer records the artifact fingerprint a local profile was declared with. A write that names a different model for an attribute that already has rows, or a write or Embed query after the service starts serving a different artifact for the pinned model, returns 422 and names the pinned model. Re-index from your source data into a fresh namespace so every vector comes from the new model, verify it, then switch your application to it.

Bringing your own weights

You can mount a model directory into the layer-embed service to add or override text checkpoints. Set LAYER_EMBED_MODELS_DIR on that service to a read-only bundle in the same manifest format as its baked models: a directory with a manifest.json listing each model’s files with their sizes and SHA-256 hashes. Mounted entries override baked entries by model id. The service verifies every listed file, loads and warms the whole menu, and only then reports ready; the gateway discovers the merged menu at startup. Only BERT text models with 384-dimensional CLS or mean pooling are accepted. The mount is optional and the bundled models need none. Overriding a model id with different weights changes its artifact fingerprint, so existing namespaces on that id need a full re-index after the restart.

Under Compose, mount the directory into the embed service read-only and set LAYER_EMBED_MODELS_DIR to its path in a Compose override file. The stock Compose file mounts nothing.

For a single static retriever that trades retrieval quality for microsecond embeds and a few megabytes of deployment size, see Lattice.

Query with `Embed`

["Embed", text] computes a query vector from text using the model settings of the attribute being ranked. When rank_by names the source attribute, Layer reads the model from its schema:

response = await client.query_namespace("clinical-notes", {
    "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
    "top_k": 10,
})
print(response.rows)

response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{
    RankBy: []any{"text", "ANN", []any{"Embed", "chest pain radiating to left arm"}},
    TopK:   10,
})

const response = await client.queryNamespace("clinical-notes", {
  rank_by: ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
    "top_k": 10
  }'

When rank_by names the derived vector attribute embed_<attr>, pass the model explicitly. Omitting it returns 422 with a model name must be provided.

response = await client.query_namespace("clinical-notes", {
    "rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
        "model": "acme/clinical-retrieval-v3",
    }]],
    "top_k": 10,
})

response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{
    RankBy: []any{"embed_text", "ANN", []any{"Embed", "chest pain radiating to left arm",
        map[string]any{"model": "acme/clinical-retrieval-v3"}}},
    TopK: 10,
})

const response = await client.queryNamespace("clinical-notes", {
  rank_by: ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
    model: "acme/clinical-retrieval-v3",
  }]],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
      "model": "acme/clinical-retrieval-v3"
    }]],
    "top_k": 10
  }'

Query vectors are cached for 60 seconds by default. Set LAYER_EMBED_CACHE_TTL_MS to change the TTL. Instruction prefixes are part of the cache key. A missing provider returns 503 service_unavailable.

`Embed` with `Auto`

An inline Embed lets query routing run a semantic or fused query in one request:

response = await client.query_namespace("articles", {
    "rank_by": ["title", "Auto", "how plants turn sunlight into food", {
        "vector": ["Embed", "how plants turn sunlight into food", {"field": "text"}],
    }],
    "top_k": 10,
})

response, err := client.QueryNamespace(ctx, "articles", &hevlayer.QueryRequest{
    RankBy: []any{"title", "Auto", "how plants turn sunlight into food", map[string]any{
        "vector": []any{"Embed", "how plants turn sunlight into food",
            map[string]any{"field": "text"}},
    }},
    TopK: 10,
})

const response = await client.queryNamespace("articles", {
  rank_by: ["title", "Auto", "how plants turn sunlight into food", {
    vector: ["Embed", "how plants turn sunlight into food", { field: "text" }],
  }],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["title", "Auto", "how plants turn sunlight into food", {
      "vector": ["Embed", "how plants turn sunlight into food", {"field": "text"}]
    }],
    "top_k": 10
  }'

The first tuple field (title) is the attribute for the lexical legs. field inside Embed names the attribute whose schema supplies the embedding profile and vector column (text above). Omit field when both are the same attribute.

Layer chooses the route before it resolves Embed, so a short input routed to hybrid_text never calls the embedding provider. A semantic or fused response reports the routing decision (routing.policy: "v1", routing.executed: true) and merges embedding measurements into performance. Without a vector or an inline Embed, the router returns routing.executed: false and leaves embedding to the caller.

Model settings

The full embed block:

Field	Meaning
`model`	A provider-namespaced Hugging Face repo id.
`dims`	The vector dimension. Must match what the model emits.
`attribute`	Where the vector is stored. Defaults to `embed_<attr>`, and may not be the source attribute.
`serving.prefer`	`local`, `worker`, or `turbopuffer`. See where the vector comes from.
`modality`	`text` (the default) or `image`. `image` requires a CLIP-family model.
`revision`	Pins a checkpoint revision, including a finetuned one.
`instructions.document` / `instructions.query`	Prefixes for asymmetric retrieval models. Both affect the query-cache key.
`chunk`	Splits the attribute into one row per chunk on write. See chunking long documents.

The first write that establishes an embedded schema must also set distance_metric.

revision, instructions, and chunk are Layer extensions. Layer never forwards them upstream, and they require a GPU-served profile, except that a CLIP image profile may use local.

Chunking long documents

A single vector for a long document matches queries poorly, so long sources should be split into chunks and each chunk embedded as its own row.

Split documents in your indexer and write one row per chunk. Store the parent document’s id as an ordinary attribute so each hit can be traced back to its source.

Performance accounting

Write and query responses report embedding work under performance:

{
  "rows": [ /* ... */ ],
  "performance": {
    "embedding_tokens": 8,
    "embedding_ms": 42
  }
}

Queries omit embedding_tokens on a cache hit. Layer merges provider measurements into the same object and exports the work as hevlayer_embed_tokens_total and hevlayer_embed_compute_seconds_total, labeled by namespace, store kind, model, and serving mode.