Concepts on AgentOps

Agents

Mon, 01 Jan 0001 00:00:00 +0000

An Agent is the central CRD in AgentOps. It declares everything the operator needs to run an AI agent as a Kubernetes workload: which model to use, what tools are available, how memory works, who can delegate to it, and what safety guardrails to enforce.

Agent CRD spec

The Agent CRD (agents.agents.agentops.io, short name ag) carries the full agent definition. Here is a representative example:

apiVersion: agents.agentops.io/v1alpha1
kind: Agent
metadata:
 name: platform-engineer
 namespace: agents
spec:
 # ── Lifecycle ──
 mode: daemon  # daemon (always-on) or task (one-shot)

 # ── Model ──
 model: anthropic/claude-sonnet-4-20250514
 fallbackModels:
 - openai/gpt-4o
 titleModel: openai/gpt-4o-mini  # cheap model for auto-titling sessions
 providerRefs:
 - name: anthropic
 - name: openai

 # ── Identity ──
 systemPrompt: |
 You are a senior platform engineer specializing in Kubernetes,
 GitOps, and infrastructure automation.
 contextFiles:
 - configMapRef:
 name: platform-context
 key: AGENTS.md

 # ── Built-in tools ──
 builtinTools:
 - bash
 - read
 - edit
 - write
 - grep
 - glob
 - fetch

 # ── External tools (AgentTool CRs) ──
 tools:
 - name: kubectl
 permissions:
 mode: deny
 rules: ["kubectl_delete", "kubectl_scale"]
 - name: flux
 - name: git

 # ── Memory ──
 memory:
 serverRef: agentops-memory  # service name or AgentTool CR
 project: platform-engineer
 contextLimit: 5 # observations injected per turn
 autoSummarize: true
 autoSave: true
 autoSearch: true

 # ── Discovery & delegation ──
 discovery:
 description: "Kubernetes and GitOps specialist. Handles cluster operations, Helm releases, Flux reconciliation, and infrastructure debugging."
 tags: ["kubernetes", "gitops", "infrastructure"]
 scope: namespace  # namespace | explicit | hidden
 # allowedCallers: [...] # only used with scope: explicit

 # ── Safety ──
 toolHooks:
 blockedCommands:
 - "rm -rf /"
 - "kubectl delete namespace"
 allowedPaths:
 - "/workspace"
 - "/tmp"
 maxSteps: 100 # safety limit on agent loop iterations
 timeout: "10m" # per-prompt timeout

 # ── Infrastructure ──
 image: ghcr.io/samyn92/agent-runtime-fantasy:latest
 resources:
 requests:
 memory: "256Mi"
 cpu: "100m"
 limits:
 memory: "1Gi"
 storage:
 size: "10Gi"
 env:
 LOG_LEVEL: "info"
 secrets:
 - name: GITHUB_TOKEN
 secretRef:
 name: agent-tokens
 key: github-token

Spec reference

Field	Type	Default	Description
`mode`	`daemon` / `task`	required	Lifecycle mode. Daemon = Deployment, task = Job template.
`model`	string	required	Primary model in `provider/model` format.
`fallbackModels`	[]string	`[]`	Tried in order if the primary model fails.
`titleModel`	string	primary model	Cheap model for auto-titling daemon sessions.
`providers`	–	–	Removed. Use `providerRefs` to reference Provider CRs.
`providerRefs`	[]ProviderBinding	required (min 1)	References to Provider CRs with optional per-agent call default overrides.
`systemPrompt`	string	`""`	Injected at the start of every session.
`contextFiles`	[]ContextFileRef	`[]`	ConfigMap-backed context files (e.g. AGENTS.md).
`builtinTools`	[]string	all defaults	Built-in Fantasy tools: `bash`, `read`, `edit`, `write`, `grep`, `ls`, `glob`, `fetch`. Set to `[]` to disable all.
`tools`	[]AgentToolBinding	`[]`	References to AgentTool CRs with optional permission overrides.
`permissionTools`	[]string	`[]`	Tools requiring user approval before execution.
`enableQuestionTool`	bool	`false`	Allow the agent to ask interactive questions.
`memory`	MemorySpec	nil	Memory configuration (see Memory).
`discovery`	DiscoverySpec	nil	Controls delegation visibility and access.
`toolHooks`	ToolHooksSpec	nil	Blocked commands, allowed paths, audit tools, memory save rules.
`maxSteps`	int	`100`	Maximum agent loop iterations (prevents infinite loops).
`timeout`	string	`"10m"`	Per-prompt timeout for daemons, job timeout for tasks.
`image`	string	`ghcr.io/samyn92/agent-runtime-fantasy:latest`	Runtime container image.
`resources`	ResourceRequirements	nil	CPU/memory requests and limits.
`storage`	StorageSpec	nil	PVC config for daemon agents (ignored for tasks).
`concurrency`	ConcurrencySpec	nil	Controls parallel AgentRun execution.
`networkPolicy`	NetworkPolicySpec	nil	Whether to create a NetworkPolicy for this agent.
`schedule`	string	`""`	Cron expression for periodic AgentRuns.
`schedulePrompt`	string	`""`	Prompt used when schedule triggers a run.
`resourceBindings`	[]AgentResourceBinding	`[]`	External resources (repos, buckets) bound to this agent.

Lifecycle modes

Daemon mode

A daemon agent is a long-running Deployment with a Service and optional PVC. It stays up, accepts prompts via HTTP POST on port 4096, and maintains a conversation session with working memory.

Memory

Mon, 01 Jan 0001 00:00:00 +0000

Memory is the key differentiator of the AgentOps platform. Instead of treating each agent session as stateless, AgentOps implements a three-layer memory architecture that gives agents persistent, searchable, relevance-ranked context across sessions and restarts.

The system is split between the Fantasy runtime (in-process working memory) and agentops-memory (persistent short-term and long-term storage). No LLM calls are involved in memory management — summarization is deterministic, retrieval is BM25-ranked full-text search.

Three-layer model

Layer 1: Working memory (ephemeral)

Working memory is an unbounded list of fantasy.Message objects held in Go runtime memory. It represents the current conversation — every user prompt, assistant response, tool call, and tool result.

Providers

Mon, 01 Jan 0001 00:00:00 +0000

A Provider extracts LLM provider configuration from individual Agent CRs into a shared, reusable Kubernetes resource. Instead of repeating provider type, credentials, and endpoint settings across every agent, you define them once in a Provider CR and reference it from any number of agents via spec.providerRefs.

Provider CRD spec

The Provider CRD (providers.agents.agentops.io, short name prov) maps directly to the Fantasy SDK’s provider surface. Here is a representative example:

apiVersion: agents.agentops.io/v1alpha1
kind: Provider
metadata:
 name: anthropic
 namespace: agents
spec:
 # ── Backend ──
 type: anthropic  # Fantasy SDK provider backend

 # ── Credentials ──
 apiKeySecret:
 name: llm-api-keys
 key: ANTHROPIC_API_KEY

 # ── Per-call defaults (optional) ──
 defaults:
 anthropic:
 effort: high  # extended thinking effort
 thinkingBudgetTokens: 8192

Spec reference

Field	Type	Required	Default	Description
`type`	`anthropic` \| `openai` \| `google` \| `azure` \| `bedrock` \| `openrouter` \| `openaicompat`	Yes	–	Fantasy SDK backend type.
`apiKeySecret`	SecretKeyRef	Yes	–	Secret containing the API key. Validated on reconcile.
`endpoint`	ProviderEndpoint	No	–	API endpoint overrides (base URL, custom headers).
`config`	ProviderConfig	No	–	Type-specific configuration (org, project, Vertex, Bedrock, Responses API).
`defaults`	ProviderCallDefaults	No	–	Per-call options applied to all agents using this provider.

See the CRD Reference for the full field-level specification.

Tools

Mon, 01 Jan 0001 00:00:00 +0000

Tools extend what an agent can do beyond generating text. AgentOps uses the Model Context Protocol (MCP) for tool integration. MCP tools are compiled Go binaries that communicate over stdio — no HTTP servers, no network hops for tool calls within the pod.

The AgentTool CRD (agents.agents.agentops.io/agenttools, short name agtool) defines the tool catalog. The operator provisions tools based on their source type, and the Fantasy runtime spawns them at startup.

Agent Delegation and Orchestration

Mon, 01 Jan 0001 00:00:00 +0000

Agent delegation lets one agent spawn work on other agents and collect structured results — without polling, without consuming LLM tokens while waiting, and with full trace propagation across the delegation chain.

Discovery

Before an agent can delegate, it needs to know what’s available. The built-in list_task_agents tool queries the Kubernetes API for Agent CRs in the same namespace and returns a filtered list based on:

Visibility scope — namespace (default, visible to all), explicit (visible only to allowedCallers), or hidden (never listed).
Tags — categorization labels like kubernetes, security, frontend that agents use to find the right specialist.
Allowed callers — when scope is explicit, only agents named in allowedCallers can discover and delegate.

The runtime knows its own identity (AGENT_NAME env var) and uses it to filter the discovery response.

Observability

Mon, 01 Jan 0001 00:00:00 +0000

Every component in AgentOps emits OpenTelemetry traces. The runtime follows the GenAI semantic conventions for LLM-specific attributes. The console proxies traces from Tempo and enriches them with delegation metadata from AgentRun CRDs.

Runtime tracing

The agent runtime produces rich OTEL spans for every operation in the agent loop:

Span hierarchy

GenAI semantic convention attributes

Every LLM span carries standard attributes:

Attribute	Example
`gen_ai.operation.name`	`chat`, `invoke_agent`
`gen_ai.provider.name`	`anthropic`, `openai`, `google`
`gen_ai.request.model`	`claude-sonnet-4-20250514`
`gen_ai.response.model`	`claude-sonnet-4-20250514`
`gen_ai.usage.input_tokens`	`12847`
`gen_ai.usage.output_tokens`	`3291`
`gen_ai.usage.reasoning_tokens`	`1024`
`gen_ai.usage.cache_read_tokens`	`8192`
`gen_ai.request.temperature`	`0.3`
`gen_ai.response.finish_reasons`	`stop`

Tool call events

Tool executions are recorded as tool.call events on the gen_ai.generate span. Each event includes:

The Web Console

Mon, 01 Jan 0001 00:00:00 +0000

The AgentOps console is a progressive web app backed by a Go BFF (Backend-for-Frontend). It connects to agent runtimes via the Fantasy Event Protocol (FEP) over Server-Sent Events for real-time streaming of responses, tool calls, delegation events, and interactive controls.

Architecture

Tech stack

Layer	Technology
Backend	Go, chi v5 router, controller-runtime informers for CRD access
Frontend	SolidJS 1.9, Vite 7, Tailwind 4
Transport	REST (60+ endpoints) + SSE (FEP multiplexer)
PWA	Workbox service worker, offline-capable static assets, NetworkFirst API caching

API surface

The BFF exposes 60+ endpoints under /api/v1, organized by domain:

Agent Resources

Mon, 01 Jan 0001 00:00:00 +0000

An AgentResource is a Kubernetes CRD that represents an external resource agents can work with. It provides a declarative catalog entry for things like GitHub repositories, GitLab projects, S3 buckets, and documentation — along with the credentials needed to access them.

Resources are separate from agents by design. You declare a resource once, then bind it to any number of agents.

Why a separate CRD?

Without AgentResource, every agent that needs access to a repository would duplicate the owner, repo name, branch, and credential reference in its own spec. AgentResource centralizes that: