AI that never phones home.

Every model runs on your hardware: Gemma 3 / Phi-4 language models through a llama.cpp sidecar, SAM2 for imagery segmentation, and YOLO-E for object detection in FMV and still imagery. Natural-language queries, segmentation masks, and georeferenced detections — with zero cloud AI calls, because the enclaves this platform serves don't have a cloud.

Gemma 3 / Phi-4llama.cpp sidecarNL→SQL (DuckDB) SAM2 segmentationYOLO-E detection100% local inference

Local-first architecture

A Python sidecar, spawned on demand

The C#/WPF host launches an embedded Python orchestrator (FastAPI) which in turn drives a llama.cpp server loading Gemma GGUF weights from local disk. Lazy-start by default: the sidecar only spins up when you first use AI.

No cloud dependency — chat, NL→SQL, embeddings, RAG, and segmentation all execute against localhost services.
Local model weights — GGUF and checkpoint files live in %APPDATA%/VoltGrid/models/; an in-app downloader with progress fetches them once, or you copy them in by hand.
Quantized and lean — Gemma 3 4B Instruct at Q4 quantization keeps the LLM footprint around 2.5 GB.
Fails safe — every AI endpoint returns 503 when the sidecar or LLM isn't running; the rest of the platform is unaffected.

AI orchestrator · localhost only

VoltGrid (C#/WPF)
  └─ AiOrchestratorHost → sidecar :8004
       ├─ /v1/chat    → llama.cpp :8080 (Gemma GGUF)
       ├─ /nl2sql     → NL → DuckDB SQL
       ├─ /embed      → text → vector
       ├─ /rag/query  → vector similarity search
       ├─ /segment    → SAM2 on COGs
       └─ /health     → liveness probe

● all inference local · no cloud AI calls

Model lineup

Three models, one operating picture

Language, segmentation, and detection each feed results back into the same map and DuckDB hub.

💬

Geospatial chatbot

Gemma 3 / Phi-4 via llama.cpp answer geospatial questions in the AI Assistant panel — grounded in your local data, not a hosted API.

🗃️

NL→SQL for DuckDB

Type "show all tracks in the Cyan team" and the LLM drafts the DuckDB SQL with schema context — spatial predicates like ST_DWithin included.

✂️

SAM2 segmentation

Load a COG, mark an area, and SAM2 produces polygon boundaries rendered as GeoJSON overlays — building extraction, flood extent, vegetation mapping.

🎯

YOLO-E on imagery

Zero-shot detection over the ArcGIS REST ImageServer: COG window → PNG → detect → geo-reference → FeatureSet, consumable by ArcGIS Maps SDK clients.

🎥

YOLO-E on FMV

Frame-level detection in the FMV player, with each detection georeferenced via KLV telemetry, published as a CoT event, drawn on the blue-force overlay, and broadcast to TAK peers.

🧠

Vector search & RAG

Build a vector index on any DuckDB text column, query for semantically similar rows, and enrich the chatbot's answers with retrieved context.

Human-in-the-loop

AI suggests. The analyst approves.

NL→SQL never executes blind: the generated DuckDB SQL fills the Query Builder editor where you review it, edit it, and click Run yourself. The model drafts; the human decides.

In the flow of work — the "Ask AI" bar sits inside the Query Builder; the AI Assistant lives in the tools drawer.
Detection on your terms — set the class labels and confidence threshold before any detection runs, in the Image Exploitation viewer or the AI panel.
Optional by design — FMV AI detection ships disabled by default; with it off, the detect toggle stays hidden and no sidecar is ever contacted.
Transparent status — capability endpoints report exactly which models are resolved, downloaded, and running.

/api/ai

# status + per-feature availability
GET  /api/ai
GET  /api/ai/capabilities
GET  /api/ai/models          → installed status
POST /api/ai/models/download → SSE progress

# inference
POST /api/ai/chat
POST /api/ai/nl2sql
POST /api/ai/segment         → SAM2 on a COG
POST /api/ai/detect/cog      → YOLO-E FeatureSet

# sidecar down → 503 · app unaffected

The YOLO-E detection service runs as a separate container; NVIDIA GPU (CUDA 11.8+) is strongly recommended for usable FMV latency — CPU works but slowly. LLM answers and generated SQL are drafts and should be reviewed before use, which is exactly how the workflow is built.