This commit is contained in:
enkly
178
README.md
178
README.md
@@ -1,2 +1,178 @@
|
||||
# enkly-ai
|
||||
# enkly — DDD microservices
|
||||
|
||||
This is the monolithic **enkly-ai** codebase (a vLLM-backed agentic
|
||||
coding assistant with an HTMX chat UI, billing, and a RAG index)
|
||||
restructured into eight independently deployable,
|
||||
Domain-Driven-Design-oriented microservices. No product feature was
|
||||
removed; see [ARCHITECTURE.md](./ARCHITECTURE.md) for the bounded-context
|
||||
map, the handful of behavioral trade-offs the split required, and what
|
||||
to review before a production rollout.
|
||||
|
||||
Inference runs on two GPU-backed [vLLM](https://docs.vllm.ai)
|
||||
OpenAI-compatible servers — one serving the chat model (Qwen, with
|
||||
tool calling and extended thinking enabled), one serving the embedding
|
||||
model — rather than a single local Ollama daemon. See `pkg/vllmkit`
|
||||
for the client, `docker-compose.yml` / `k8s/11-vllm.yaml` for how the
|
||||
two servers are deployed, and `CHANGES.md`'s most recent entry for the
|
||||
full writeup of what the migration off Ollama changed and why.
|
||||
|
||||
## Services
|
||||
|
||||
| Service | Port | Owns |
|
||||
|---|---|---|
|
||||
| `gateway` | 8080 (HTTP) | The HTMX web UI; the only service with a public HTTP surface |
|
||||
| `iam` | 9001 (gRPC) | Accounts, sessions, email verification, password reset |
|
||||
| `billing` | 9002 (gRPC) | Plans, Stripe checkout/portal/webhooks, daily usage quota |
|
||||
| `conversation` | 9003 (gRPC) | Conversations, messages, tool-call audit log |
|
||||
| `workspace` | 9004 (gRPC) | The agent's sandboxed filesystem/bash tools; file upload/download |
|
||||
| `knowledge` | 9005 (gRPC) | pgvector-backed RAG index/search |
|
||||
| `agent` | 9006 (gRPC) | The LLM run loop; orchestrates billing/conversation/workspace/knowledge |
|
||||
| `notification` | — (RabbitMQ only) | Sends email in reaction to iam's events |
|
||||
|
||||
Communication: **gRPC** for every synchronous, request/response call
|
||||
between services (see `api/proto/`); **RabbitMQ** for asynchronous,
|
||||
no-reply-needed side effects — today, exactly the email-sending that
|
||||
`iam` triggers (see `pkg/eventbus`).
|
||||
|
||||
## Running it
|
||||
|
||||
```sh
|
||||
cp .env.example .env # fill in STRIPE_* / ENKLY_SMTP_* if you want those features
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
`vllm-chat` and `vllm-embed` download their own model weights from
|
||||
Hugging Face on first start — no separate `pull` step — so the first
|
||||
`up` can take several minutes before `agent`/`knowledge` actually come
|
||||
online (they wait on both vLLM servers' `/health` check via
|
||||
`depends_on`). `docker compose logs -f vllm-chat` to watch progress.
|
||||
This requires a host with an NVIDIA GPU and the NVIDIA Container
|
||||
Toolkit installed (`docker info | grep -i nvidia` to check) — vLLM's
|
||||
production performance depends on real GPU acceleration; see
|
||||
`docker-compose.yml`'s `vllm-chat`/`vllm-embed` services for the
|
||||
details (pinned image tag, `--gpu-memory-utilization`, `/dev/shm`
|
||||
sizing).
|
||||
|
||||
Then open <http://localhost:8080>.
|
||||
|
||||
To index a codebase for the `search_codebase` tool:
|
||||
|
||||
```sh
|
||||
docker compose run --rm -v "$(pwd)":/src indexer -path /src
|
||||
```
|
||||
|
||||
## Claude-Code-inspired agent features
|
||||
|
||||
Two additions to the agent service, beyond the tool-for-tool monolith port, modeled on
|
||||
Claude Code:
|
||||
|
||||
- **Project memory (`ENKLY.md`)** — drop an `ENKLY.md` file in the sandboxed workspace root
|
||||
(same mechanism as `write_file`/the file explorer) and the agent folds it into its system
|
||||
prompt on every turn, under a "Project memory" heading — conventions, gotchas, how to run
|
||||
things in *this* repo. No file means no change in behavior. See
|
||||
`services/agent/internal/infrastructure/agentrun/agent.go`'s `MemoryLoader`.
|
||||
- **`dispatch_subagent` (a Task tool)** — the model can hand off a broad, open-ended
|
||||
investigation ("find every place X happens", "map out how Y flows through this service")
|
||||
to a nested, read-only sub-agent that explores on its own and reports back in one message.
|
||||
The sub-agent's own tool calls never enter the main conversation's context — only its
|
||||
final report does — and it structurally cannot write a file, edit a file, or run a
|
||||
command (its tool list excludes them, and its executor refuses them even if asked). See
|
||||
`services/agent/internal/infrastructure/toolexec/subagent.go` and
|
||||
`toolcatalog.SubagentDefs()`.
|
||||
- **Adaptive, interleaved thinking** — the agent already streamed Qwen's reasoning trace
|
||||
back per round (`ChatEvent.Thinking`); two refinements bring that closer to how Claude's
|
||||
interleaved thinking actually behaves. First, thinking effort now varies by round instead
|
||||
of being one static setting for the whole run: round 0 (pure planning, no tool results
|
||||
yet) asks the chat model for `"high"` effort, later rounds defer to whatever default you've
|
||||
configured, and a template-parse retry asks for `"low"` so recovery stays fast — see
|
||||
`agentrun.ThinkPolicy` / `DefaultThinkPolicy`. Second, a round's reasoning trace is no
|
||||
longer resent to the model on every subsequent round for the rest of the turn: once a
|
||||
newer round's thinking exists, the older one is dropped from the resent history (only the
|
||||
most recent round's trace is ever "in flight," matching the Claude API's own behavior of
|
||||
only needing the thinking block immediately preceding the current tool use) — see
|
||||
`agentrun.compactThinkingHistory`, called alongside the existing `compactToolHistory` on
|
||||
every round. A new optional `Callbacks.OnThinkingRound(round int, thinking string)` also
|
||||
fires once per round with just that round's own trace, for a caller that wants to render
|
||||
reasoning as discrete per-step blocks rather than one running blob. The gateway now does
|
||||
exactly that: `domain.Stream.AddThinkingRound` records each round, `agent.proto`'s new
|
||||
`TurnEvent.thinking_round` (a `ThinkingRound{round, thinking}` oneof case) carries it over
|
||||
gRPC, and `StreamChat` emits it as a `thinking_round` SSE event rendered as its own
|
||||
collapsible card (`message.html`'s `thinking_round`/`thinking_round_oob`, styled via
|
||||
`.tool-card.is-thinking-round` in `style.css`) inserted above the reply in execution
|
||||
order — the same `beforebegin` OOB trick tool-call cards already use. This sits alongside,
|
||||
not in place of, the existing single auto-collapsing `.thinking-card`: that one shows the
|
||||
round currently in flight streaming live; the per-round cards are the settled record of
|
||||
every round once done, positioned right before the tool call each one led to.
|
||||
|
||||
## Dynamic workflows, effort control, and long-context recovery
|
||||
|
||||
A further round of additions, modeled on Claude's agentic-coding feature set and scoped
|
||||
to what's honestly deliverable against a vLLM-served Qwen backend — see `CHANGES.md`
|
||||
§7 for the full writeup (written against the Ollama backend these features originally
|
||||
shipped on; see `CHANGES.md`'s latest entry for what the later vLLM migration changed).
|
||||
The short version:
|
||||
|
||||
- **`dispatch_workflow`** — the large-scale sibling of `dispatch_subagent`: give it a
|
||||
batch of steps (one per file/service/call site in a migration) and it runs them as
|
||||
parallel, read-only sub-agents, honoring any `depends_on` ordering, verifying each
|
||||
report against its own task, and retrying once on a weak one — automatic verification
|
||||
and plan adjustment, without a second LLM call per step. Gated by plan
|
||||
(`pkg/entitlements`) and bounded by `ENKLY_WORKFLOW_MAX_PARALLEL`.
|
||||
- **Effort control** (`pkg/effort`) — five levels, Low through Max, picked per turn via a
|
||||
leading `/effort <level>` line in the chat message (see `pkg/turnopts` for why a
|
||||
directive line rather than a new proto field, for now) or `ENKLY_DEFAULT_EFFORT` for the
|
||||
deployment default. Each level sizes max rounds, context budget, thinking-effort floor,
|
||||
and `dispatch_workflow`'s own parallelism cap.
|
||||
- **Adaptive thinking** — escalates one round's thinking effort above its level's floor on
|
||||
a concrete difficulty signal (the prior tool call errored, produced an unusually large
|
||||
result, or the turn is running long), instead of holding every round at one fixed
|
||||
setting. On by default (`ENKLY_ADAPTIVE_THINKING`).
|
||||
- **Compaction ledger** — `compactToolHistory` now keeps a running, auto-maintained note
|
||||
of every tool result it's trimmed away, so a long turn that's compacted its 30th result
|
||||
still has a stable map of what happened earlier instead of silently losing the thread.
|
||||
- **Live system entries** — `/system <instruction>` (and `permission:`/`budget:` variants)
|
||||
fold additional guidance into the system prompt mid-conversation, always appended after
|
||||
the stable base prompt rather than edited in place.
|
||||
- **Fast mode** — `/fast` runs the turn on a second, smaller/quantized model served by
|
||||
its own vLLM deployment (`ENKLY_FAST_CHAT_MODEL` + `ENKLY_FAST_VLLM_CHAT_URL`) when
|
||||
one's configured; otherwise it's a no-op, so it's always safe to request.
|
||||
|
||||
## First build
|
||||
|
||||
No `go.sum` is committed. Every Dockerfile runs `go mod tidy` during
|
||||
the build to resolve and lock the full dependency graph (including
|
||||
what grpc/protobuf/etc. pull in transitively) — this needs normal
|
||||
outbound network access the first time, same as any other Go project.
|
||||
After the first successful `docker compose up --build`, consider
|
||||
running `go mod tidy` once at the repo root and committing the
|
||||
resulting `go.sum`, then switching each Dockerfile's `RUN go mod tidy`
|
||||
back to `RUN go mod download` for faster, better-cached rebuilds (see
|
||||
the comment in `services/iam/Dockerfile`).
|
||||
|
||||
## Regenerating gRPC code
|
||||
|
||||
`gen/` is committed so the repo is buildable without protoc installed,
|
||||
but if you edit anything under `api/proto/`, regenerate it:
|
||||
|
||||
```sh
|
||||
make proto # requires protoc, protoc-gen-go, protoc-gen-go-grpc on PATH
|
||||
```
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
api/proto/<context>/v1/*.proto gRPC contracts — the API surface between services
|
||||
gen/ generated *.pb.go / *_grpc.pb.go (do not hand-edit)
|
||||
pkg/ shared libraries (eventbus, auth helpers, render, vLLM client, Postgres helper, ...)
|
||||
services/<name>/
|
||||
cmd/server/main.go entrypoint: wiring, not logic
|
||||
internal/domain/ entities, repository interfaces — no framework imports
|
||||
internal/application/ use cases — transplanted business logic, transport-agnostic
|
||||
internal/infrastructure/ Postgres/Stripe/vLLM/filesystem adapters implementing domain interfaces
|
||||
internal/interfaces/grpc/ the only files that import the generated gRPC package
|
||||
migrations/0001_init.sql this service's own schema (database-per-service)
|
||||
tools/indexer/ standalone CLI, a knowledge-service gRPC client
|
||||
docker-compose.yml full local stack
|
||||
```
|
||||
|
||||
See [ARCHITECTURE.md](./ARCHITECTURE.md) for the design rationale.
|
||||
Reference in New Issue
Block a user