Opper | PrivateGPT

Opper is an EU-hosted AI gateway with an OpenAI-compatible endpoint at https://api.opper.ai/v3/compat. One API key gives access to 700+ models from 50+ providers, including OpenAI, Anthropic, Google, DeepSeek and Mistral. Token rates are the providers’ rates with no markup.

Capabilities with PrivateGPT

Capability	Status
Model discovery (`/models`)	✅
Tokenizer endpoint (`/tokenize`)	❌
Embeddings	✅
Tool / function calling	✅ model-dependent
Streaming	✅
Vision / image input	✅ model-dependent

The same endpoint serves chat and embeddings, so no second server is needed.

Setup

Get an API key

Sign up at platform.opper.ai.
Create an API key under API Keys.

Run PrivateGPT

Package install

Docker

uv (local)

OPENAI_API_BASE=https://api.opper.ai/v3/compat \
  OPENAI_API_KEY=your-opper-api-key \
  PGPT_LLM_DEFAULT=claude-sonnet-4-6 \
  PGPT_EMBEDDING_DEFAULT=text-embedding-3-small \
  private-gpt serve

Store the API key in an .env file or use OPENAI_API_KEY as an environment variable to avoid exposing it in shell history.

Notes

Bare model ids such as claude-sonnet-4-6 or gpt-5.5 route each request across the providers that serve that model. An id with a provider prefix, such as anthropic/claude-sonnet-4-6, pins one route.
/models returns context_length for each model, so PrivateGPT sets the context window automatically.
/models lists chat and embedding models together. Set PGPT_LLM_DEFAULT to a chat model and PGPT_EMBEDDING_DEFAULT to an embedding model.
The model catalogue is at opper.ai/models.