Troubleshooting

Models not discovered at startup

Symptom: PrivateGPT starts but shows no models, or API calls return a “no models available” error.

Causes and fixes:

1. LLM server not running. Make sure your LLM server is up before starting PrivateGPT.

   $
   curl http://localhost:11434/v1/models   # Ollama
   $
   curl http://localhost:8000/v1/models    # vLLM / LlamaCPP

2. Wrong OPENAI_API_BASE. Verify the URL matches your server’s address and includes /v1.

   $
   OPENAI_API_BASE=http://localhost:11434/v1 private-gpt serve

3. Auto-discovery disabled. Check that PGPT_LLM_AUTO_DISCOVER_MODELS is not set to false.

Connection refused inside Docker

When running PrivateGPT in Docker and pointing at a server on the host machine, localhost refers to the container, not the host.

macOS / Windows: use host.docker.internal:

$
docker run -p 8080:8080 \
>
  -e OPENAI_API_BASE=http://host.docker.internal:11434/v1 \
>
  zylonai/private-gpt:latest

Linux: use --network host instead:

$
docker run --network host \
>
  -e OPENAI_API_BASE=http://localhost:11434/v1 \
>
  zylonai/private-gpt:latest

Context window overflow

Symptom: Long conversations or large documents produce truncated responses or errors from the LLM server.

Cause: Without a tokenizer endpoint (Ollama), PrivateGPT estimates token count at 4 chars = 1 token, which can be inaccurate.

Fix: Set context_window explicitly in a detailed model profile to a value below the model’s actual limit:

1
models:
2
  - name: qwen3.5:35b
3
    type: llm
4
    mode: openai
5
    context_window: 28000   # conservative — leaves headroom

Embedding dimensions mismatch

Symptom: Error message containing Embedding dimensions mismatch during ingestion or retrieval.

Cause: The vector store was initialized with a different embedding model (different output dimensions) than the one currently configured.

Fix: Either:

• Switch back to the original embedding model, or
• Wipe the vector store and re-ingest your documents:

  $
  make wipe

Ensure embed_dim in your profile matches the model’s output dimension (e.g. 1024 for mxbai-embed-large):

1
vectorstore:
2
  embed_dim: 1024

Tokenizer download fails (HuggingFace gated models)

Symptom: Startup fails with an authentication error when downloading a tokenizer from HuggingFace.

Cause: The model’s tokenizer repository is gated and requires a HuggingFace token.

Fix:

1. Request access to the model on HuggingFace.
2. Generate an access token.
3. Pass the token via environment variable:

   $
   PGPT_HUGGINGFACE_TOKEN=hf_... private-gpt serve
   Or set it in your profile:

   1
   huggingface:
   2
     access_token: hf_...

Port already in use

Symptom: [Errno 48] Address already in use on startup.

Fix: Change the port with the --port flag:

$
private-gpt serve --port 8081

Or for Docker:

$
docker run -p 8081:8080 -e OPENAI_API_BASE=... zylonai/private-gpt:latest

Profile file not found

Symptom: FileNotFoundError: Settings file not found for profile 'foo'.

Cause: PGPT_PROFILES=foo was set but settings-foo.yaml does not exist in the settings folder.

Fix: Make sure the file exists and is in the correct location. For Docker, it must be mounted:

$
-v ./settings-foo.yaml:/home/worker/app/settings-foo.yaml