gratefulagents

Architecture guide

Why self-host your coding agents?

A coding agent reads your repository, runs commands against it, and writes code back. Where that happens, who holds the model credentials, and what audit trail it leaves matter as much as the diff it produces. This guide covers what self-hosting actually changes, the six reasons teams do it, and what running the infrastructure yourself involves.

Try GratefulAgents locally → What’s different

What self-hosting actually means

A coding agent is an AI system that autonomously reads code, reasons about a task, calls tools — shell, editor, browser, APIs — and produces commits or pull requests. Unlike a copilot that completes a line at a time, it manages a multi-step plan and iterates until the task is done or a human steps in.

Self-hosted means every part of that loop runs on infrastructure you control. The agent process, the sandbox it executes in, and the credentials it uses all sit inside your boundary. On a cloud-managed harness you grant the vendor access to your repositories and the agent runs on their machines instead. Both produce pull requests; they differ in who you have to trust.

Six reasons teams do it

Data residency and regulatory compliance

GDPR, financial-services regulation, government contracting, and plain internal policy all draw a line around where source code may travel. A cloud harness run moves your files, your prompts, and every intermediate output across the vendor's network. Self-hosting takes that vendor out of the path: the checkout, the sandbox, the tool calls, and the run history stay on infrastructure you control.

Be precise about the inference boundary. Self-hosting the harness is not the same as keeping every byte on your own hardware. Unless you point GratefulAgents at a model endpoint you host yourself, prompts and the repository context inside them still go to whichever provider you configure — Anthropic, OpenAI, OpenRouter, xAI, or Copilot — under that provider's terms. What you gain is control of that boundary: you choose the provider, you hold the contract and the key, and each run's trace shows exactly what was sent. If your obligations forbid sending code to any third party, you need a self-hosted inference endpoint too.

Intellectual property protection

Regulation is not the only reason. Proprietary algorithms, unreleased codebases, and trade-secret infrastructure code are high-value assets, and routing them through a third-party harness creates a trust dependency many security teams will not sign off on. Self-hosting narrows that dependency to a single model provider you pick yourself.

Private dependencies and internal registries

Real codebases pull from private package registries, internal APIs, and services that are not routable from the public internet. A cloud agent needs tunnelling or firewall exceptions to reach any of it. An agent running inside your network reaches them the same way a developer's laptop does.

Model cost under your own contract

Cloud harnesses bundle inference into the subscription, so you pay their price with no view of the underlying calls. Bringing your own key means your existing Anthropic, OpenAI, Bedrock, or OpenRouter agreement applies, volume discount included. Per-run cost and token counts then show you where the spend actually goes.

An audit trail you own

When an autonomous agent changes code, you need to answer: what did it read, which tools did it call, which model responded, and what led to this commit? Cloud harnesses surface some of this, usually as a summary feed. A harness that emits a full distributed trace gives you the whole record — every step, prompt, response, and cost — to replay, audit, and compare across runs.

The runtime becomes infrastructure code

Self-hosting on Kubernetes makes the agent runtime a Helm chart and a set of CRD manifests. You can stand up an identical environment in staging, promote the same configuration to production, and keep all of it in version control. A SaaS subscription cannot be diffed or rolled back.

What it takes to run one

Running the harness yourself is more work than signing up for a SaaS. Six problems have to be solved, and they are worth understanding before you commit — whether you deploy GratefulAgents or build something in-house. Each one below states the problem, then how GratefulAgents answers it.

Isolated sandboxes

Every run needs somewhere to execute code and call tools that is isolated from other runs and from your cluster's core services — ephemeral, with controlled network and filesystem access.

In GratefulAgents Runs execute in pods provided by kubernetes-sigs/agent-sandbox, a Kubernetes SIG project for isolated agent execution. Each run gets a fresh pod and scoped secrets, with no cross-contamination between runs.

Credential handling

The harness needs model API keys, GitHub tokens, and registry credentials. They have to be stored securely, scoped per project, and injected into the sandbox without leaking into logs or traces.

In GratefulAgents Keys are Kubernetes secrets in your own cluster, scoped per project. Model calls are never proxied — agents reach Claude, OpenAI, OpenRouter, Grok, or Copilot directly with your key, so no usage data leaves your cluster on the way.

Scheduling and lifecycle

Runs have to be queued, dispatched, watched, retried on failure, and cleaned up. Doing that reliably, and surviving a node restart mid-run, is most of the operational work.

In GratefulAgents A Kubernetes controller manager reconciles custom resources — AgentRun for a single task, Project for shared settings and defaults — so starting a run means applying a manifest. Scheduling, dispatch, and cleanup are the controller's problem, not yours.

Trigger integrations

Agents earn their keep when real events start them: a GitHub issue, a review comment, a Linear ticket, a Slack message, a nightly schedule. Every source needs a webhook receiver, event normalisation, and signature validation.

In GratefulAgents GitHubRepository, LinearProject, SlackAgent, and Cron are CRDs. Triggers are declared next to everything else in your cluster, with no external orchestration service to run.

An observability path

Without a structured record of each run, debugging a regression in agent behaviour is guesswork. You want steps, tool calls, model exchanges, token counts, costs, and errors — not a summary feed.

In GratefulAgents Every run emits a distributed trace to a Jaeger instance the Helm chart deploys. The Agent Ops console shows per-run cost, tokens, tool calls, subagent graphs, compaction events, and errors, and lets you compare the same task across models.

A human-in-the-loop surface

Autonomous agents make mistakes. You need to watch a run in progress, steer it, stop it, and resume from a checkpoint — and that rarely happens while you are sat at your desk.

In GratefulAgents Runs operate in interactive, plan, or autopilot mode, switchable mid-run. Steer, stop, and retry from the web dashboard, the desktop client (macOS Apple Silicon, Linux), or iOS and Android.

Your first run, in fifteen minutes

The fastest way to judge any of this is to run it. A local Kind cluster needs no cloud account and no server — just Docker.

  1. Install prerequisites Install Docker Desktop, or Docker Engine on Linux. The installer downloads Kind and the rest of the cluster tooling for you. Full requirements are in the getting-started guide.
  2. Clone and run the installer Clone the repository, then run make kind-install. This provisions a single-node Kind cluster and deploys the GratefulAgents chart, including PostgreSQL/pgvector, MinIO for object storage, and a Jaeger instance for traces. It prints the local dashboard URL and the command to retrieve your admin password.
  3. Add your model key Open the dashboard, go to Settings → Credentials, and add your Anthropic Claude or OpenAI key. It is stored as a Kubernetes secret in your own cluster and is used only to authenticate your calls to that provider.
  4. Connect a repository Create a GitHubRepository resource pointing at a repository you own, and configure a GitHub App webhook so issues and pull requests can trigger runs.
  5. Start a run and watch the trace Open a GitHub issue or start a run from the dashboard. Follow the live activity in the Agent Ops console, then read the Jaeger trace once it finishes.
Note on current status: GratefulAgents is in early development. Bugs are expected. It is not yet suitable for production-critical workloads, has no documented high-availability configuration, and there is no managed hosting or support SLA. Treat it as early-adopter software and test it thoroughly before relying on it.

Where GratefulAgents sits

The best-known coding agents — Devin and GitHub Copilot's coding agent — are cloud services: the checkout, the agent process, and the model credentials all live on the vendor's infrastructure. GratefulAgents puts all three inside a cluster you run. The differences page lays the architecture out side by side.