178 lines
10 KiB
Markdown
178 lines
10 KiB
Markdown
# enkly — DDD microservices
|
|
|
|
This is the monolithic **enkly-ai** codebase (a vLLM-backed agentic
|
|
coding assistant with an HTMX chat UI, billing, and a RAG index)
|
|
restructured into eight independently deployable,
|
|
Domain-Driven-Design-oriented microservices. No product feature was
|
|
removed; see [ARCHITECTURE.md](./ARCHITECTURE.md) for the bounded-context
|
|
map, the handful of behavioral trade-offs the split required, and what
|
|
to review before a production rollout.
|
|
|
|
Inference runs on two GPU-backed [vLLM](https://docs.vllm.ai)
|
|
OpenAI-compatible servers — one serving the chat model (Qwen, with
|
|
tool calling and extended thinking enabled), one serving the embedding
|
|
model — rather than a single local Ollama daemon. See `pkg/vllmkit`
|
|
for the client, `docker-compose.yml` / `k8s/11-vllm.yaml` for how the
|
|
two servers are deployed, and `CHANGES.md`'s most recent entry for the
|
|
full writeup of what the migration off Ollama changed and why.
|
|
|
|
## Services
|
|
|
|
| Service | Port | Owns |
|
|
|---|---|---|
|
|
| `gateway` | 8080 (HTTP) | The HTMX web UI; the only service with a public HTTP surface |
|
|
| `iam` | 9001 (gRPC) | Accounts, sessions, email verification, password reset |
|
|
| `billing` | 9002 (gRPC) | Plans, Stripe checkout/portal/webhooks, daily usage quota |
|
|
| `conversation` | 9003 (gRPC) | Conversations, messages, tool-call audit log |
|
|
| `workspace` | 9004 (gRPC) | The agent's sandboxed filesystem/bash tools; file upload/download |
|
|
| `knowledge` | 9005 (gRPC) | pgvector-backed RAG index/search |
|
|
| `agent` | 9006 (gRPC) | The LLM run loop; orchestrates billing/conversation/workspace/knowledge |
|
|
| `notification` | — (RabbitMQ only) | Sends email in reaction to iam's events |
|
|
|
|
Communication: **gRPC** for every synchronous, request/response call
|
|
between services (see `api/proto/`); **RabbitMQ** for asynchronous,
|
|
no-reply-needed side effects — today, exactly the email-sending that
|
|
`iam` triggers (see `pkg/eventbus`).
|
|
|
|
## Running it
|
|
|
|
```sh
|
|
cp .env.example .env # fill in STRIPE_* / ENKLY_SMTP_* if you want those features
|
|
docker compose up --build
|
|
```
|
|
|
|
`vllm-chat` and `vllm-embed` download their own model weights from
|
|
Hugging Face on first start — no separate `pull` step — so the first
|
|
`up` can take several minutes before `agent`/`knowledge` actually come
|
|
online (they wait on both vLLM servers' `/health` check via
|
|
`depends_on`). `docker compose logs -f vllm-chat` to watch progress.
|
|
This requires a host with an NVIDIA GPU and the NVIDIA Container
|
|
Toolkit installed (`docker info | grep -i nvidia` to check) — vLLM's
|
|
production performance depends on real GPU acceleration; see
|
|
`docker-compose.yml`'s `vllm-chat`/`vllm-embed` services for the
|
|
details (pinned image tag, `--gpu-memory-utilization`, `/dev/shm`
|
|
sizing).
|
|
|
|
Then open <http://localhost:8080>.
|
|
|
|
To index a codebase for the `search_codebase` tool:
|
|
|
|
```sh
|
|
docker compose run --rm -v "$(pwd)":/src indexer -path /src
|
|
```
|
|
|
|
## Claude-Code-inspired agent features
|
|
|
|
Two additions to the agent service, beyond the tool-for-tool monolith port, modeled on
|
|
Claude Code:
|
|
|
|
- **Project memory (`ENKLY.md`)** — drop an `ENKLY.md` file in the sandboxed workspace root
|
|
(same mechanism as `write_file`/the file explorer) and the agent folds it into its system
|
|
prompt on every turn, under a "Project memory" heading — conventions, gotchas, how to run
|
|
things in *this* repo. No file means no change in behavior. See
|
|
`services/agent/internal/infrastructure/agentrun/agent.go`'s `MemoryLoader`.
|
|
- **`dispatch_subagent` (a Task tool)** — the model can hand off a broad, open-ended
|
|
investigation ("find every place X happens", "map out how Y flows through this service")
|
|
to a nested, read-only sub-agent that explores on its own and reports back in one message.
|
|
The sub-agent's own tool calls never enter the main conversation's context — only its
|
|
final report does — and it structurally cannot write a file, edit a file, or run a
|
|
command (its tool list excludes them, and its executor refuses them even if asked). See
|
|
`services/agent/internal/infrastructure/toolexec/subagent.go` and
|
|
`toolcatalog.SubagentDefs()`.
|
|
- **Adaptive, interleaved thinking** — the agent already streamed Qwen's reasoning trace
|
|
back per round (`ChatEvent.Thinking`); two refinements bring that closer to how Claude's
|
|
interleaved thinking actually behaves. First, thinking effort now varies by round instead
|
|
of being one static setting for the whole run: round 0 (pure planning, no tool results
|
|
yet) asks the chat model for `"high"` effort, later rounds defer to whatever default you've
|
|
configured, and a template-parse retry asks for `"low"` so recovery stays fast — see
|
|
`agentrun.ThinkPolicy` / `DefaultThinkPolicy`. Second, a round's reasoning trace is no
|
|
longer resent to the model on every subsequent round for the rest of the turn: once a
|
|
newer round's thinking exists, the older one is dropped from the resent history (only the
|
|
most recent round's trace is ever "in flight," matching the Claude API's own behavior of
|
|
only needing the thinking block immediately preceding the current tool use) — see
|
|
`agentrun.compactThinkingHistory`, called alongside the existing `compactToolHistory` on
|
|
every round. A new optional `Callbacks.OnThinkingRound(round int, thinking string)` also
|
|
fires once per round with just that round's own trace, for a caller that wants to render
|
|
reasoning as discrete per-step blocks rather than one running blob. The gateway now does
|
|
exactly that: `domain.Stream.AddThinkingRound` records each round, `agent.proto`'s new
|
|
`TurnEvent.thinking_round` (a `ThinkingRound{round, thinking}` oneof case) carries it over
|
|
gRPC, and `StreamChat` emits it as a `thinking_round` SSE event rendered as its own
|
|
collapsible card (`message.html`'s `thinking_round`/`thinking_round_oob`, styled via
|
|
`.tool-card.is-thinking-round` in `style.css`) inserted above the reply in execution
|
|
order — the same `beforebegin` OOB trick tool-call cards already use. This sits alongside,
|
|
not in place of, the existing single auto-collapsing `.thinking-card`: that one shows the
|
|
round currently in flight streaming live; the per-round cards are the settled record of
|
|
every round once done, positioned right before the tool call each one led to.
|
|
|
|
## Dynamic workflows, effort control, and long-context recovery
|
|
|
|
A further round of additions, modeled on Claude's agentic-coding feature set and scoped
|
|
to what's honestly deliverable against a vLLM-served Qwen backend — see `CHANGES.md`
|
|
§7 for the full writeup (written against the Ollama backend these features originally
|
|
shipped on; see `CHANGES.md`'s latest entry for what the later vLLM migration changed).
|
|
The short version:
|
|
|
|
- **`dispatch_workflow`** — the large-scale sibling of `dispatch_subagent`: give it a
|
|
batch of steps (one per file/service/call site in a migration) and it runs them as
|
|
parallel, read-only sub-agents, honoring any `depends_on` ordering, verifying each
|
|
report against its own task, and retrying once on a weak one — automatic verification
|
|
and plan adjustment, without a second LLM call per step. Gated by plan
|
|
(`pkg/entitlements`) and bounded by `ENKLY_WORKFLOW_MAX_PARALLEL`.
|
|
- **Effort control** (`pkg/effort`) — five levels, Low through Max, picked per turn via a
|
|
leading `/effort <level>` line in the chat message (see `pkg/turnopts` for why a
|
|
directive line rather than a new proto field, for now) or `ENKLY_DEFAULT_EFFORT` for the
|
|
deployment default. Each level sizes max rounds, context budget, thinking-effort floor,
|
|
and `dispatch_workflow`'s own parallelism cap.
|
|
- **Adaptive thinking** — escalates one round's thinking effort above its level's floor on
|
|
a concrete difficulty signal (the prior tool call errored, produced an unusually large
|
|
result, or the turn is running long), instead of holding every round at one fixed
|
|
setting. On by default (`ENKLY_ADAPTIVE_THINKING`).
|
|
- **Compaction ledger** — `compactToolHistory` now keeps a running, auto-maintained note
|
|
of every tool result it's trimmed away, so a long turn that's compacted its 30th result
|
|
still has a stable map of what happened earlier instead of silently losing the thread.
|
|
- **Live system entries** — `/system <instruction>` (and `permission:`/`budget:` variants)
|
|
fold additional guidance into the system prompt mid-conversation, always appended after
|
|
the stable base prompt rather than edited in place.
|
|
- **Fast mode** — `/fast` runs the turn on a second, smaller/quantized model served by
|
|
its own vLLM deployment (`ENKLY_FAST_CHAT_MODEL` + `ENKLY_FAST_VLLM_CHAT_URL`) when
|
|
one's configured; otherwise it's a no-op, so it's always safe to request.
|
|
|
|
## First build
|
|
|
|
No `go.sum` is committed. Every Dockerfile runs `go mod tidy` during
|
|
the build to resolve and lock the full dependency graph (including
|
|
what grpc/protobuf/etc. pull in transitively) — this needs normal
|
|
outbound network access the first time, same as any other Go project.
|
|
After the first successful `docker compose up --build`, consider
|
|
running `go mod tidy` once at the repo root and committing the
|
|
resulting `go.sum`, then switching each Dockerfile's `RUN go mod tidy`
|
|
back to `RUN go mod download` for faster, better-cached rebuilds (see
|
|
the comment in `services/iam/Dockerfile`).
|
|
|
|
## Regenerating gRPC code
|
|
|
|
`gen/` is committed so the repo is buildable without protoc installed,
|
|
but if you edit anything under `api/proto/`, regenerate it:
|
|
|
|
```sh
|
|
make proto # requires protoc, protoc-gen-go, protoc-gen-go-grpc on PATH
|
|
```
|
|
|
|
## Repository layout
|
|
|
|
```
|
|
api/proto/<context>/v1/*.proto gRPC contracts — the API surface between services
|
|
gen/ generated *.pb.go / *_grpc.pb.go (do not hand-edit)
|
|
pkg/ shared libraries (eventbus, auth helpers, render, vLLM client, Postgres helper, ...)
|
|
services/<name>/
|
|
cmd/server/main.go entrypoint: wiring, not logic
|
|
internal/domain/ entities, repository interfaces — no framework imports
|
|
internal/application/ use cases — transplanted business logic, transport-agnostic
|
|
internal/infrastructure/ Postgres/Stripe/vLLM/filesystem adapters implementing domain interfaces
|
|
internal/interfaces/grpc/ the only files that import the generated gRPC package
|
|
migrations/0001_init.sql this service's own schema (database-per-service)
|
|
tools/indexer/ standalone CLI, a knowledge-service gRPC client
|
|
docker-compose.yml full local stack
|
|
```
|
|
|
|
See [ARCHITECTURE.md](./ARCHITECTURE.md) for the design rationale. |