AGENTS.md

Backend architecture and cross-service invariants for server/core, server/auth-realtime and the repo-root datagen/. Shared principles: root AGENTS.md. Go standards: core/AGENTS.md. Core domain rules sit beside their packages: core/internal/document/, core/internal/datasource/, core/internal/assistant/. auth-realtime: auth-realtime/AGENTS.md.

Stack

server/core/: one binary, cmd/core, on :8080. Owns Postgres, the search index, Valkey, object storage, the GitHub/Slack apps, the assistant and outbound data-source connections.
server/auth-realtime/: :8081, Better Auth (organization plugin) and Hocuspocus in one Hono process; forwards non-auth /api/... to core (OXYNOTE_AUTH_REALTIME_BACKEND_URL).
datagen/: separate Go module synthesising demo Postgres/MariaDB content.

Go builds go through goreleaser (make build in server/core and datagen), producing bin/ and the ghcr.io/oxynote/{core,datagen}:dev images the dev compose stack consumes. datagen shares core's test/lint workflow with its own copy of the golangci-lint profile; its pgdemo/mariademo suites start throwaway containers.

Caddy / ports

docker/Caddyfile: :8080 front door (/core/* → core:8080, /auth-realtime/* → auth-realtime:8081, else web:3000), :8082 changedetection.io, :8083 Grafana (direct). /api/x/... is not exposed; GitHub and Slack callback URLs point at :8080/core/api/apps/....

Core request surface

internal/server/router.go is the canonical map.

/api/...: session-authed via auth-realtime's /api/auth/get-session (SERVER_AUTH_BETTER_AUTH_URL). GET /api/capabilities reports one boolean per optional service, snapshotted at boot, plus aiAssistant (status + model).
/api/x/...: no auth; auth-realtime fetches/stores branch content here (/x/documents/{id}/branches, /x/documents/{id}/branch/{branchId}), triggers emails and initializes or tears down orgs.
/api/apps/...: sessionless; GitHub/Slack webhooks gated by request signature, GET /apps/slack/install completes the OAuth exchange. Must stay outside /api/x, since third parties reach core through the front door.
/api/mcp: streamable HTTP MCP server (internal/server/internal/mcp) over the assistant's ungated tool registry plus documents as resources. JWT bearer tokens come from auth-realtime's @better-auth/mcp; each request is validated against GET /api/internal/mcp/session (JWKS + consent row, so revocation 401s immediately) and scoped by documents:read/documents:write/data-sources:read.
/api/ws topics (wetsocks/wsserver) are registered in router.go as <event>@<domain.path>; binders are Handler.BindXxx under internal/server/internal/....

The README lists the public routes; update it when changing handlers.

Document storage / Hocuspocus

Documents have branches (document_branches); every document has at least one, and the main one is flagged default (see the document package's AGENTS.md for the branch rules).
Hocuspocus documentName is "<documentId>-<branchIdentifier>"; "default" is resolved by resolveBranchId (auth-realtime/src/hocuspocus.ts). Split on the first - only.
Branch content is stored twice: ProseMirror JSON in content (JSONB) and the Yjs binary in raw_content (base64 on the wire). raw_content is authoritative for CRDT continuity.
onLoadDocument/onStoreDocument round-trip through /api/x/documents/{id}/branch/{branchId}. Never seed a doc from another with Y.applyUpdate (clientIDs CRDT-merge and duplicate content); use replaceYdocContent (auth-realtime/src/ydocument.ts), which deep-clones Y.XmlElements preserving the non-string attrs clone() drops. Read the comments on onLoadDocument and on applyMergeToOpenDocument (routes.ts) first.
Fork, branch update, merge and branch delete go through auth-realtime, which flushes the affected branch's pending store (both sides for a merge, the -default alias too), refuses the operation if the flush fails, then proxies to core's session-authed /api/x group with the caller's headers. After a protection change or a delete it drops every client socket on the document (resetConnections; hocuspocus's closeConnections is not that), since read-only is decided per connection in onAuthenticate. A merge mutates the in-memory target Y.Doc via replaceYdocContent and persists rawContent immediately.
The ProseMirror schema is auth-realtime/src/schema/; its Go mirror is core/internal/document/node.go.

Database

Postgres 18. Migrations are embedded from core/internal/db/migrations/ and applied by db.New (rubenv/sql-migrate). The system is live: every schema change is a new NNN_<name>.sql; an applied migration is never edited. A dev volume behind on migrations can be reset with docker compose -p oxynote -f docker/docker-compose.dev.yaml down -v.

Core migrations own the Better Auth tables (snake_case columns matching the fields mappings in auth-realtime/src/auth.ts). auth-realtime/sql/better_auth_schema.sql is reference output, never applied. Regenerate it to diff after changing auth.ts:

# from server/auth-realtime/
source ../../docker/env/auth-realtime.local.env && npx @better-auth/cli generate --output ./sql/better_auth_schema.sql

Env vars

docker/env/*.example.env lists every variable each service reads, empty ones included; the dev compose reads only the gitignored *.local.env copies. Core reads OXYNOTE_CORE_* through buildinfo.Getenv("FOO").

Every integration is optional and keyed on one variable (GITHUB_APP_ID, SLACK_CLIENT_ID, ASSISTANT_PROVIDER, CHANGEDETECTION_API_URL, VALKEY_ADDRESS, S3_URL, EMAIL_SMTP_HOST). Key set with the rest of its group missing is a boot error; key empty disables the feature (routes answer <domain>.not_configured, background work skipped). Two degrade instead: empty S3_URL stores objects under S3_LOCAL_PATH (no default; unset is a boot error), empty VALKEY_ADDRESS keeps assistant conversations in-process (pkg/memkit). A model too weak for the assistant disables it with a warning. Search: SEARCH_INDEX_PATH (no default) is a bleve index, a Postgres cache rebuilt at boot when missing or outdated; it pins a deployment to one core instance.

6.4 KiB Raw Permalink Blame History