Building Agent Board: A Local-First AI Ops Cockpit

Why Build This?

Most AI tools today require sending your data to external APIs. That's a non-starter for sensitive work. I wanted a cockpit that runs entirely locally — your models, your data, your infrastructure — with enterprise-grade safety rails built in.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Agent Board                            │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │   Ollama    │  │ Docker MR   │  │   NemoClaw          │  │
│  │  (local)    │  │  (local)    │  │  (sandboxed exec)   │  │
│  └──────┬──────┘  └──────┬──────┘  └──────────┬──────────┘  │
│         │                │                     │             │
│         └────────────────┼─────────────────────┘             │
│                          ▼                                   │
│              ┌─────────────────────┐                         │
│              │   Safety Rails      │                         │
│              │  • PII Redaction    │                         │
│              │  • Prompt Injection │                         │
│              │  • Content Filter   │                         │
│              └──────────┬──────────┘                         │
│                         │                                    │
│              ┌──────────▼──────────┐                         │
│              │  OpenTelemetry      │                         │
│              │  • Traces           │                         │
│              │  • Metrics          │                         │
│              │  • Logs             │                         │
│              └─────────────────────┘                         │
└─────────────────────────────────────────────────────────────┘

Core Components

1. Multi-Model Chat Interface

The UI supports switching between models on the fly:

// src/lib/models.ts
export const MODELS = {
  'llama3.2': { provider: 'ollama', context: 128000 },
  'mistral': { provider: 'ollama', context: 32000 },
  'codellama': { provider: 'ollama', context: 16000 },
  'docker-model-runner': { provider: 'docker', context: 128000 },
};

2. Safety Rails Pipeline

Every prompt passes through a validation pipeline before reaching the model:

// src/lib/safety/rails.ts
export async function validatePrompt(prompt: string): Promise<ValidationResult> {
  const checks = [
    redactPII,
    detectPromptInjection,
    filterContent,
    enforceLengthLimits,
  ];
  
  for (const check of checks) {
    const result = await check(prompt);
    if (!result.pass) {
      return { pass: false, reason: result.reason, stage: check.name };
    }
  }
  return { pass: true };
}

PII Redaction uses regex patterns + NER to catch:

• API keys, tokens, secrets
• Email addresses, phone numbers
• Credit card numbers, SSNs
• Custom patterns via config

Prompt Injection Detection checks for:

• Instruction override attempts ("ignore previous instructions")
• Role confusion ("you are now admin")
• Data exfiltration requests
• Encoded/obfuscated payloads

3. NemoClaw Sandboxed Execution

Code execution runs in isolated containers:

# docker-compose.nemoclaw.yml
services:
  nemoclaw:
    image: nemoclaw/sandbox:latest
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    read_only: true
    tmpfs:
      - /tmp:noexec,nosuid,size=100m
    network_mode: none
    mem_limit: 512m
    cpus: '0.5'

4. OpenTelemetry Observability

Full tracing without external dependencies:

// src/lib/otel.ts
import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
import { JaegerExporter } from '@opentelemetry/exporter-jaeger';

const provider = new NodeTracerProvider();
provider.addSpanProcessor(
  new BatchSpanProcessor(
    new JaegerExporter({ endpoint: 'http://localhost:14268/api/traces' })
  )
);
provider.register();

Key Features

| Feature | Implementation | |---------|----------------| | Multi-model support | Ollama, Docker Model Runner, NemoClaw | | PII redaction | Regex + spaCy NER, configurable patterns | | Prompt injection defense | Heuristic + ML classifier | | Code execution | NemoClaw sandboxed containers | | Observability | OpenTelemetry → Jaeger/Zipkin | | Persistence | Local SQLite + vector embeddings | | UI | Next.js 16 + React 19 + Tailwind |

Deployment

# One-command startup
docker compose -f config/docker-compose.yml up -d

# Services started:
# - agent-board:3000 (UI)
# - ollama:11434 (models)
# - jaeger:16686 (traces)
# - nemoclaw:8080 (sandbox)

Key Takeaways

• Local-first AI is viable for production workloads with the right tooling
• Safety rails must be layered: regex → heuristic → ML → sandbox
• OpenTelemetry gives you enterprise observability without vendor lock-in
• NemoClaw makes untrusted code execution practical
• Docker Model Runner simplifies GPU-accelerated local inference

Code References

• agent-board Repo
• Docker Compose

Further Reading

• Ollama Documentation
• Docker Model Runner
• NemoClaw Sandbox
• OpenTelemetry JS
• Prompt Injection Research

Conclusion

Agent Board proves you don't need cloud APIs for serious AI work. The combination of local models, safety rails, sandboxed execution, and full observability creates a platform that's both powerful and trustworthy. It's the foundation I use daily for coding, research, and experimentation — all without a single byte leaving my machine.