Self-Hosted & OpenAI-Compatible Endpoints
Route governed agent traffic to models you host yourself — vLLM, Ollama, or any OpenAI-compatible inference server — through a Gateway connection (Team and above). Clevername still governs every call: CleverGuard scans content, and guardrails, drift detection, and audit logging apply the same as with cloud providers.
localhost. That relay is not currently available. What is available is the OpenAI-Compatible gateway connection described below, which requires an endpoint Clevername can reach over the public internet.What a Gateway connection is
Alongside direct provider keys, Settings → Keys → Gateways lets you connect customer-managed AI infrastructure: a LiteLLM proxy, Azure OpenAI / Azure AI Foundry, AWS Bedrock, Vertex AI, or any OpenAI-Compatible endpoint. The OpenAI-Compatible type is the one to use for self-hosted models served by vLLM, Ollama, Together AI, or your own inference server. Clevername forwards governed calls to {base_url}/v1/chat/completions on your endpoint, so inference runs on your hardware while governance stays in Clevername.
Requirements
- Team plan or above — the Gateways tab is not shown on Free or Pro
- A publicly reachable HTTPS base URL — Clevername calls your endpoint from its own infrastructure, so
localhost, private-network, and link-local addresses are rejected - An OpenAI-compatible API — the endpoint must serve
GET /v1/modelsandPOST /v1/chat/completions - An API key, if your server requires one — optional; leave it blank for unauthenticated servers
Expose your server over HTTPS
Serve your model with vLLM, Ollama, or another OpenAI-compatible server and put it behind a public HTTPS hostname (a reverse proxy or tunnel in front of the server is fine). Confirm GET https://your-host/v1/models returns your model list.
Add the gateway in Clevername
Go to Settings → Keys → Gateways, choose OpenAI-Compatible, and enter a Label, the Base URL, and an optional API Key. Clevername validates the credentials and fetches the model list from your endpoint when you save.
Assign a discovered model to an agent
Model IDs are taken exactly as your endpoint reports them. In AI Company → Agents, open an agent and pick one of those IDs. Every governed call from that agent — through its cn-live-* gateway token — is then routed to your endpoint, while still passing through CleverGuard scanning, the agent's guardrail profile, and audit logging.
What gets governed
Self-hosted models receive the same governance as cloud models. Clevername enforces your policies regardless of where the model runs.