Add OmniRoute setup reference for agents

2026-08-06 22:06:12 +08:00
parent 8128121220
commit 5c1e7ce188
1 changed files with 129 additions and 0 deletions
@@ -0,0 +1,129 @@
+# OmniRoute — Free Multi-Provider LLM Gateway
+
+> Setup reference for OpenClaw agents. Last updated: 2026-08-06
+
+## Overview
+
+OmniRoute is a free, MIT-licensed AI gateway that aggregates **290+ providers** and **500+ models** through a single OpenAI-compatible endpoint. It provides:
+- Auto-fallback across free tiers (~1.6B free tokens/month)
+- Smart model routing (`auto/best-coding`, `auto/best-fast`, etc.)
+- No API keys needed for local use
+
+**Repo:** https://github.com/diegosouzapw/OmniRoute  
+**Version:** 3.8.49 (Docker)  
+**Local URL:** http://localhost:20128
+
+---
+
+## Current Deployment (wm's MacBook Pro)
+
+### Docker Container
+```bash
+docker run -d \
+  --name omniroute-local \
+  --restart unless-stopped \
+  -p 20128:20128 \
+  diegosouzapw/omniroute:latest
+```
+
+### OpenClaw Provider Config
+Added to `~/.openclaw/openclaw.json`:
+
+```json
+{
+  "models": {
+    "providers": {
+      "omniroute": {
+        "api": "openai-completions",
+        "baseUrl": "http://localhost:20128/v1",
+        "models": [
+          {"id": "auto/best-coding", "name": "OmniRoute Best Coding"},
+          {"id": "auto/best-fast", "name": "OmniRoute Best Fast"},
+          {"id": "auto/best-reasoning", "name": "OmniRoute Best Reasoning"},
+          {"id": "auto/best-chat", "name": "OmniRoute Best Chat"}
+        ]
+      }
+    }
+  }
+}
+```
+
+### Fallback Chain
+```json
+{
+  "agents": {
+    "defaults": {
+      "model": {
+        "primary": "groq/llama-3.3-70b-versatile",
+        "fallbacks": [
+          "omniroute/auto/best-coding",
+          "omniroute/auto/best-fast",
+          "omniroute/auto/best-reasoning"
+        ]
+      }
+    }
+  }
+}
+```
+
+---
+
+## Available Auto-Routing Models
+
+OmniRoute exposes smart `auto/*` prefixes that route to the best available free model:
+
+| Prefix | Description | Use Case |
+|---|---|---|
+| `auto/best-coding` | Best free coding model | Code generation, debugging |
+| `auto/best-reasoning` | Best reasoning model | Math, logic, analysis |
+| `auto/best-fast` | Fastest free model | Quick responses, chat |
+| `auto/best-chat` | Best conversational model | General chat |
+| `auto/best-vision` | Best vision model | Image understanding |
+| `auto/cheap` | Cheapest option | High-volume, low-priority |
+
+---
+
+## CLI Commands Reference
+
+### Test OmniRoute directly
+```bash
+openclaw infer model run --model omniroute/auto/best-fast --prompt "Hello" --local
+```
+
+### Add more OmniRoute models as fallbacks
+```bash
+openclaw models fallbacks add omniroute/auto/best-chat
+openclaw models fallbacks list
+```
+
+### Check OmniRoute health
+```bash
+curl -s http://localhost:20128/v1/models | jq '.data | length'
+```
+
+### Restart OmniRoute
+```bash
+docker restart omniroute-local
+```
+
+---
+
+## Verified Test Results
+
+All models tested and passing (2026-08-06):
+
+| Model | Latency | Test |
+|---|---|---|
+| `auto/best-coding` | 2624ms | Python reverse string ✅ |
+| `auto/best-fast` | 2270ms | Arithmetic (15×37) ✅ |
+| `auto/best-reasoning` | 2049ms | Train speed/distance problem ✅ |
+
+---
+
+## Notes for Agents
+
+- OmniRoute runs as a Docker container on the host — no API keys required
+- When primary model (Groq) hits rate limits or fails, fallback auto-kicks to OmniRoute
+- OmniRoute internally rotates across free provider tiers to maximize uptime
+- The `auto/*` routing prefix handles provider selection internally — no need to specify individual providers
+- OmniRoute models are NOT listed by `openclaw models list` by default; they are custom provider models