Got Llama 3 running locally on my consumer-grade GPU and set up Open WebUI as the interface for it. The install was easy — following the docs, a single Docker command was all it took, no real technical hurdles.
On the backend, I hooked it up to One API as an aggregation layer, with Cloudflare and Groq configured as upstream providers. Combined with the locally-hosted Llama 3 8B and 70B models, that gives me one unified API surface spanning both local and remote inference.
