Odysseus AI GPU Not Detected

Last updated: June 6, 2026

If Odysseus AI starts but local model inference does not use your GPU, fix the runtime in this order: host driver, Docker GPU access, compose overlay, then model backend settings.

Quick answer

Odysseus itself can run without a GPU. The GPU is usually needed by the model backend, not by the web app. First prove the host can see the GPU, then prove Docker can see it, then restart Odysseus with the correct GPU compose option.

1. Identify Which Layer Is Failing

Host driver

Your operating system must see the GPU before Docker or Odysseus can use it.

Docker runtime

Docker needs GPU access through NVIDIA Container Toolkit, ROCm device mapping, or the platform-specific runtime.

Model backend

Ollama, vLLM, llama.cpp, or another backend must load a model that fits your available memory.

2. Check the Host GPU First

Run the command for your GPU type on the host machine, not inside the Odysseus container.

NVIDIA

Copy command

nvidia-smi

AMD ROCm

Copy command

rocminfo
ls -l /dev/dri

If this fails, install or repair the host GPU driver before changing Odysseus.

3. Verify Docker GPU Access

A working host driver is not enough. Docker must be able to see the same device from a container.

Copy command

docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

If the Odysseus repository includes GPU helper scripts, use them before editing compose files by hand:

Copy command

scripts/check-docker-gpu.sh
scripts/check-docker-gpu.sh --print-install-commands
scripts/check-docker-gpu.sh --enable-nvidia-overlay

For AMD ROCm setups, check device permissions and the render group:

Copy command

scripts/check-docker-amd-gpu.sh
# Then set COMPOSE_FILE and RENDER_GID in .env as reported by the script.

4. Recreate the Odysseus Stack

After changing GPU runtime or compose settings, restart the stack from the Odysseus repository directory.

Copy command

docker compose down
docker compose up -d --build
docker compose logs --tail 120 odysseus

For NVIDIA Docker installs, this command should work inside the app container if GPU passthrough is active:

Copy command

docker compose exec odysseus nvidia-smi -L

5. Apple Silicon and macOS Notes

Docker Desktop on macOS does not expose the Apple Silicon Metal GPU to Linux containers for local model inference. The practical setup is to run Ollama natively on macOS, then connect Odysseus to the native Ollama endpoint.

Copy command

ollama list
ollama pull llama3.1:8b
OLLAMA_HOST=0.0.0.0:11434 ollama serve

Then use http://host.docker.internal:11434/v1 inside Odysseus if Odysseus itself is running in Docker Desktop.

6. If the GPU Works but Models Are Still Slow

- Use a smaller quantized model before testing a large model.
- Confirm the model backend logs show GPU execution, not CPU fallback.
- Increase Docker memory limits if the backend exits while loading a model.
- Use a cloud API if your local GPU does not have enough VRAM.

Match model size to your machine using the hardware requirements guide.

Verify Against Official Docs

GPU scripts and compose overlays can change quickly. For the current helper script names and supported platforms, check the official GitHub repository.

FAQ

Does Odysseus AI require a GPU?

No. Odysseus can run without a GPU when you use cloud APIs or a CPU-only local backend. A GPU matters when you want faster local model inference through Ollama, vLLM, llama.cpp, or another local backend.

Why does Odysseus not see my NVIDIA GPU in Docker?

The host driver may work while Docker still lacks GPU access. Install the NVIDIA Container Toolkit, restart Docker, then verify GPU access from a container before changing Odysseus settings.

Can Docker use an AMD GPU with Odysseus AI?

Yes on supported Linux and ROCm setups. You need the ROCm device mapping and render group permissions. Use the AMD GPU helper script if the Odysseus repo provides one.

Does Apple Silicon GPU passthrough work in Docker?

No. Docker Desktop on macOS does not pass the Apple Silicon Metal GPU into Linux containers for local model inference. Use native Ollama on macOS and connect Odysseus to it.

Why is local inference still slow after the GPU is detected?

The selected model may be too large, the backend may still be using CPU, or Docker memory limits may be too low. Check the model backend logs and use a smaller quantized model first.

Installation support

Need hands-on installation support?

Setup Helper is the self-service planner. Installation Support is for users who want a human to review the route, diagnose logs, or get Odysseus running. We confirm scope and price before any payment.