diff --git a/omniroute-setup.md b/omniroute-setup.md new file mode 100644 index 0000000..9f9eee8 --- /dev/null +++ b/omniroute-setup.md @@ -0,0 +1,129 @@ +# OmniRoute — Free Multi-Provider LLM Gateway + +> Setup reference for OpenClaw agents. Last updated: 2026-08-06 + +## Overview + +OmniRoute is a free, MIT-licensed AI gateway that aggregates **290+ providers** and **500+ models** through a single OpenAI-compatible endpoint. It provides: +- Auto-fallback across free tiers (~1.6B free tokens/month) +- Smart model routing (`auto/best-coding`, `auto/best-fast`, etc.) +- No API keys needed for local use + +**Repo:** https://github.com/diegosouzapw/OmniRoute +**Version:** 3.8.49 (Docker) +**Local URL:** http://localhost:20128 + +--- + +## Current Deployment (wm's MacBook Pro) + +### Docker Container +```bash +docker run -d \ + --name omniroute-local \ + --restart unless-stopped \ + -p 20128:20128 \ + diegosouzapw/omniroute:latest +``` + +### OpenClaw Provider Config +Added to `~/.openclaw/openclaw.json`: + +```json +{ + "models": { + "providers": { + "omniroute": { + "api": "openai-completions", + "baseUrl": "http://localhost:20128/v1", + "models": [ + {"id": "auto/best-coding", "name": "OmniRoute Best Coding"}, + {"id": "auto/best-fast", "name": "OmniRoute Best Fast"}, + {"id": "auto/best-reasoning", "name": "OmniRoute Best Reasoning"}, + {"id": "auto/best-chat", "name": "OmniRoute Best Chat"} + ] + } + } + } +} +``` + +### Fallback Chain +```json +{ + "agents": { + "defaults": { + "model": { + "primary": "groq/llama-3.3-70b-versatile", + "fallbacks": [ + "omniroute/auto/best-coding", + "omniroute/auto/best-fast", + "omniroute/auto/best-reasoning" + ] + } + } + } +} +``` + +--- + +## Available Auto-Routing Models + +OmniRoute exposes smart `auto/*` prefixes that route to the best available free model: + +| Prefix | Description | Use Case | +|---|---|---| +| `auto/best-coding` | Best free coding model | Code generation, debugging | +| `auto/best-reasoning` | Best reasoning model | Math, logic, analysis | +| `auto/best-fast` | Fastest free model | Quick responses, chat | +| `auto/best-chat` | Best conversational model | General chat | +| `auto/best-vision` | Best vision model | Image understanding | +| `auto/cheap` | Cheapest option | High-volume, low-priority | + +--- + +## CLI Commands Reference + +### Test OmniRoute directly +```bash +openclaw infer model run --model omniroute/auto/best-fast --prompt "Hello" --local +``` + +### Add more OmniRoute models as fallbacks +```bash +openclaw models fallbacks add omniroute/auto/best-chat +openclaw models fallbacks list +``` + +### Check OmniRoute health +```bash +curl -s http://localhost:20128/v1/models | jq '.data | length' +``` + +### Restart OmniRoute +```bash +docker restart omniroute-local +``` + +--- + +## Verified Test Results + +All models tested and passing (2026-08-06): + +| Model | Latency | Test | +|---|---|---| +| `auto/best-coding` | 2624ms | Python reverse string ✅ | +| `auto/best-fast` | 2270ms | Arithmetic (15×37) ✅ | +| `auto/best-reasoning` | 2049ms | Train speed/distance problem ✅ | + +--- + +## Notes for Agents + +- OmniRoute runs as a Docker container on the host — no API keys required +- When primary model (Groq) hits rate limits or fails, fallback auto-kicks to OmniRoute +- OmniRoute internally rotates across free provider tiers to maximize uptime +- The `auto/*` routing prefix handles provider selection internally — no need to specify individual providers +- OmniRoute models are NOT listed by `openclaw models list` by default; they are custom provider models