Skip to content
Guides & References

Self-hosted LLM for network config analysis: air-gapped Iris with Ollama or vLLM

Run Iris on a self-hosted LLM with Ollama or vLLM so device configs never leave your network. Design, traffic flow, pros, cons and sizing.

Iris: AI CenterReference designUpdated
On this page

Self-hosted LLM for network config analysis: air-gapped Iris with Ollama or vLLM

In this design, Iris’s features run on a model you host. rConfig sends configs, diffs and compliance results to a server on your own network, and nothing goes to a cloud provider. It’s the design for regulated environments, sites with no internet access, and anyone whose security team says configs don’t leave the building.

What it looks like

flowchart LR
    U["Engineer in rConfig<br/>Ask AI / Analyze / Narrate"] --> R["rConfig server"]
    R -- "HTTP on your network<br/>(port 11434 for Ollama)" --> M["Model server<br/>Ollama or vLLM, with GPU"]
    M --> R
    R -.-> L[("Audit trail<br/>metadata only")]
    X["Internet"] -. "not needed for inference" .- M
Component Role
rConfig Builds each request: config or diff, background knowledge, your conventions
Model server Ollama (native provider in rConfig) or vLLM, LM Studio or LiteLLM through OpenAI-Compatible
GPU Holds the model and its context window; the main sizing decision

rConfig’s Privacy tab shows the provider as On-Premises when its address is localhost, a private range or a .local or .internal host.

Ollama or vLLM

Ollama vLLM
Setup One binary or container, pull a model, done A Python serving stack, more tuning
Concurrency Fine for one team Built for many simultaneous requests
rConfig provider Ollama OpenAI-Compatible
Context window Server setting; default depends on GPU memory Server setting (maximum model length)
Best for Pilots, small teams, a single site Larger teams, shared AI platform

Pros

  • Configs, diffs and compliance results never leave your network.
  • No per-token cost and no provider rate limits.
  • Works on sites with no internet access at all.

Cons

  • Answer quality depends on the model you can afford to run. Long, complex configs are where self-hosted models fall behind the largest cloud models.
  • You own the GPU, the model updates and the uptime.
  • The context window needs deliberate sizing. Ollama’s default is 4k tokens on GPUs with under 24 GiB of memory; see Sizing a local model.

Getting it right

  • Bind the model server to the network, not just localhost. Ollama listens on 127.0.0.1 by default; start it with OLLAMA_HOST=0.0.0.0 and allow port 11434 from the rConfig server only.
  • Test from the rConfig server: curl http://<model-host>:11434/api/tags should list your models.
  • Use the exact model name from ollama list, including the tag.
  • Size the context window before judging the model.

Frequently asked questions

Can rConfig's AI features work in an air-gapped network?

Yes. Point Iris at Ollama or an OpenAI-compatible server such as vLLM on your network. Inference needs no internet access, and the optimisation assistant's documentation index is stored in rConfig's own database.

Ollama or vLLM?

Ollama is the quickest to stand up and suits one team and modest load. vLLM serves many concurrent requests more efficiently on GPU servers and suits larger teams. rConfig talks to Ollama natively and to vLLM through the OpenAI-Compatible provider.

Do agents like Claude Code also stay on site?

No. This design covers Iris inside rConfig. Cloud-model agents connected over MCP send data to their own provider. For a fully on-site setup including agents, see the air-gapped NOC design.

Type to search.