PKMS Storage & E2EE Architecture

The Personal Knowledge Management System (PKMS) architecture stores workspace and document artifacts in isolated object-storage prefixes while exposing a gateway API through the Omni edge proxy.

This design is intended to preserve zero-knowledge client-side encryption boundaries while giving web and desktop clients a stable HTTP contract.

1. Storage Folder Structure

Workspace and document elements are organized under workspace-scoped object keys:

workspaces/
└── {workspaceId}/
    ├── manifest.om
    ├── notebooks.om.enc
    ├── vector_manifest.om
    ├── search/
    │   └── block-index.om
    ├── graph/
    │   └── block-edges.om
    └── documents/
        └── {documentId}/
            ├── document.om or document.om.enc
            ├── blocks.om or blocks.om.enc
            ├── markdown.md
            └── meta.om

Grounded to the storage key helpers in omni-edge-proxy/src/routes/pkms.ts:251-285 and omni-edge-proxy/src/routes/pkms.ts:757-785.

Key components

• manifest.om: workspace manifest used by /v1/workspaces routes.
• notebooks.om.enc: legacy encrypted notebook blob used by compatibility notebook routes.
• search/block-index.om: stored keyword/block index used for fallback search.
• vector_manifest.om: manifest metadata for vector/indexing state.
• graph/block-edges.om: stored graph edge data.
• document.om or document.om.enc: stored document record.
• blocks.om or blocks.om.enc: stored block payloads.
• markdown.md: stored markdown text.
• meta.om: plaintext sidecar metadata used for previews and lightweight listing.

2. Encryption and response transformation

The proxy supports optional response-side decryption, but only when a caller opts in.

Relevant headers:

Authorization: Bearer <token>
X-Proxy-Decrypt: true
X-Client-Key: <client-key>

Current behavior from omni-edge-proxy/src/routes/pkms.ts:122-233:

1. Without X-Proxy-Decrypt: true, stored encrypted values are returned unchanged.
2. With X-Proxy-Decrypt: true, the proxy tries to resolve a client key.
3. X-Client-Key takes precedence over any fallback key resolver.
4. If no client key is available, the response is returned unchanged.

The response transformer can handle:

• encrypted envelope objects shaped like { format: "omni.om.encrypted", ciphertext }
• encrypted blob strings shaped like iv:tag:ciphertext
• nested arrays and objects containing those values

Search indexing boundary

The current implementation now maintains both the stored block index and semantic vector synchronization from the backend mutation layer.

• Unified document create/update routes trigger backend-owned vector sync after storage writes: omni-edge-proxy/src/routes/pkms.ts:1161-1365
• Background sync is attached to the Cloudflare Workers lifecycle with waitUntil(...): omni-edge-proxy/src/routes/pkms.ts:801-835
• The sync service reads stored artifacts, rebuilds search/block-index.om, updates Cloudflare Vectorize, and refreshes vector_manifest.om: omni-edge-proxy/src/pkms/noteIndexService.ts:176-412
• Semantic retrieval still depends on both local block index content and Vectorize matches in the workspace namespace: omni-edge-proxy/src/contextBuilder.ts:129-225

For implementation history and debugging details, see:

• core-console/content/docs/developer-guide/vector-sync-plan.mdx:1-109
• core-console/content/docs/developer-guide/vector-debugging-handoff.mdx:1-174

3. Current API contract shape

Clients should interact with PKMS through the edge proxy routes rather than writing storage adapters directly.

Workspace actions

• GET /v1/workspaces
• POST /v1/workspaces
• GET /v1/workspaces/:workspaceId

Grounded to omni-edge-proxy/src/routes/pkms.ts:1151-1188.

Notebook compatibility actions

• GET /v1/workspaces/:workspaceId/notebooks
• POST /v1/workspaces/:workspaceId/notebooks
• GET /v1/workspaces/:workspaceId/notebooks/:notebookId

Grounded to omni-edge-proxy/src/routes/pkms.ts:1191-1225.

Unified document actions

Canonical document routes live under /v1/documents and /v1/workspaces/:workspaceId/documents.

• POST /v1/documents
• PATCH /v1/documents/:id
• DELETE /v1/documents/:id?workspace_id=...
• POST /v1/documents/query
• GET /v1/workspaces/:workspaceId/documents
• POST /v1/workspaces/:workspaceId/documents
• GET /v1/workspaces/:workspaceId/documents/:documentId
• PUT /v1/workspaces/:workspaceId/documents/:documentId
• DELETE /v1/workspaces/:workspaceId/documents/:documentId

Grounded to omni-edge-proxy/src/routes/pkms.ts:1227-1365.

Legacy compatibility actions

Legacy note routes remain active and are still used by the current desktop client for CRUD.

• GET /v1/workspaces/:workspaceId/notes
• POST /v1/workspaces/:workspaceId/notes
• GET /v1/workspaces/:workspaceId/notes/:noteId
• PATCH /v1/workspaces/:workspaceId/notes/:noteId
• DELETE /v1/workspaces/:workspaceId/notes/:noteId

Important implementation detail:

• these compatibility handlers currently perform direct raw artifact writes and deletes
• they are not backed by createUnifiedDocument(...), patchUnifiedDocument(...), or removeDocumentIndex(...)

Grounded to omni-edge-proxy/src/routes/pkms.ts:1378-1508.

4. Integration guidelines for developers

When implementing features or refactoring modules:

1. Use proxy routes, not direct storage writes: clients should integrate through the edge proxy contract documented in core-console/content/docs/api-reference/endpoints.mdx:1-725.
2. Treat legacy note routes as compatibility-only: they are still used by the current desktop client, but new clients should prefer unified document routes. See web-omni/src/features/desktop-client/lib/documentRepository.ts:311-472.
3. Maintain plaintext sidecars intentionally: meta.om is used for lightweight preview metadata and notebook association. See omni-edge-proxy/src/routes/pkms.ts:728-755 and omni-edge-proxy/src/routes/pkms.ts:769-775.
4. Do not assume automatic semantic sync: the current implementation maintains stored block index data, but async vector synchronization is still a gap. See omni-edge-proxy/src/routes/pkms.ts:857-924.
5. Use unified delete routes when you need derived-index cleanup: unified delete handlers call removeDocumentIndex(...), while legacy delete does not. See omni-edge-proxy/src/routes/pkms.ts:1250-1259 and omni-edge-proxy/src/routes/pkms.ts:1500-1508.
6. Expect mixed-mode clients today: the desktop client already mixes modern workspace/search APIs with legacy note CRUD. See web-omni/src/features/desktop-client/lib/documentRepository.ts:311-420.