Skip to navigation
Inference Providers

Standard Compute

Standard Compute is a hosted, OpenAI-compatible LLM gateway at https://api.stdcmpt.com/v1. PrivateGPT can use it through the existing OpenAI-compatible integration.

Standard Compute serves chat models. This setup sends chat requests to Standard Compute and keeps embeddings on your existing embedding setup, for example a local Ollama server. It does not require a new PrivateGPT backend.

Setup

1

Create a key and choose a chat model

  1. Sign up at standardcompute.com and create an API key in the dashboard.
  2. Set STANDARDCOMPUTE_API_KEY in your environment or .env file. Keep the secret out of committed files.
export STANDARDCOMPUTE_API_KEY="your-standardcompute-api-key"
  1. Choose a model. StandardCompute is the routed model: it sends each request to the cheapest enabled model that can handle it, with automatic fallback. To pin a specific model instead, list the public catalog and use an exact ID from the response:
curl https://api.stdcmpt.com/v1/models
export STANDARDCOMPUTE_MODEL="StandardCompute"

See the Standard Compute docs for the API reference.

2

Start a local embeddings server

Skip this step if you already have an embeddings server. Otherwise, start Ollama and pull the embeddings model:

ollama pull mxbai-embed-large

These examples use Ollama’s OpenAI-compatible API at http://localhost:11434/v1, with the separate placeholder key ollama.

3

Run PrivateGPT

OPENAI_API_BASE=https://api.stdcmpt.com/v1 \
  OPENAI_API_KEY="$STANDARDCOMPUTE_API_KEY" \
  OPENAI_EMBEDDING_API_BASE=http://localhost:11434/v1 \
  OPENAI_EMBEDDING_API_KEY=ollama \
  PGPT_LLM_DEFAULT="$STANDARDCOMPUTE_MODEL" \
  PGPT_EMBEDDING_DEFAULT=mxbai-embed-large \
  private-gpt serve

Follow the corresponding installation guide first, including PrivateGPT’s other required services.

Model limits and capabilities

  • Use the model ID returned by /v1/models unchanged.
  • /v1/models returns context_length for each model, so PrivateGPT can set the context window from discovery.
  • Tools, streaming, reasoning, and image input depend on the selected model. Check supported_parameters in the /v1/models response.
  • The examples keep embeddings on a separate server. Retrieved document excerpts included in a chat prompt are still sent to the hosted API.