Add OmniRoute setup reference for agents
This commit is contained in:
@@ -0,0 +1,129 @@
|
|||||||
|
# OmniRoute — Free Multi-Provider LLM Gateway
|
||||||
|
|
||||||
|
> Setup reference for OpenClaw agents. Last updated: 2026-08-06
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
OmniRoute is a free, MIT-licensed AI gateway that aggregates **290+ providers** and **500+ models** through a single OpenAI-compatible endpoint. It provides:
|
||||||
|
- Auto-fallback across free tiers (~1.6B free tokens/month)
|
||||||
|
- Smart model routing (`auto/best-coding`, `auto/best-fast`, etc.)
|
||||||
|
- No API keys needed for local use
|
||||||
|
|
||||||
|
**Repo:** https://github.com/diegosouzapw/OmniRoute
|
||||||
|
**Version:** 3.8.49 (Docker)
|
||||||
|
**Local URL:** http://localhost:20128
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current Deployment (wm's MacBook Pro)
|
||||||
|
|
||||||
|
### Docker Container
|
||||||
|
```bash
|
||||||
|
docker run -d \
|
||||||
|
--name omniroute-local \
|
||||||
|
--restart unless-stopped \
|
||||||
|
-p 20128:20128 \
|
||||||
|
diegosouzapw/omniroute:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
### OpenClaw Provider Config
|
||||||
|
Added to `~/.openclaw/openclaw.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"models": {
|
||||||
|
"providers": {
|
||||||
|
"omniroute": {
|
||||||
|
"api": "openai-completions",
|
||||||
|
"baseUrl": "http://localhost:20128/v1",
|
||||||
|
"models": [
|
||||||
|
{"id": "auto/best-coding", "name": "OmniRoute Best Coding"},
|
||||||
|
{"id": "auto/best-fast", "name": "OmniRoute Best Fast"},
|
||||||
|
{"id": "auto/best-reasoning", "name": "OmniRoute Best Reasoning"},
|
||||||
|
{"id": "auto/best-chat", "name": "OmniRoute Best Chat"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Fallback Chain
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"agents": {
|
||||||
|
"defaults": {
|
||||||
|
"model": {
|
||||||
|
"primary": "groq/llama-3.3-70b-versatile",
|
||||||
|
"fallbacks": [
|
||||||
|
"omniroute/auto/best-coding",
|
||||||
|
"omniroute/auto/best-fast",
|
||||||
|
"omniroute/auto/best-reasoning"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Available Auto-Routing Models
|
||||||
|
|
||||||
|
OmniRoute exposes smart `auto/*` prefixes that route to the best available free model:
|
||||||
|
|
||||||
|
| Prefix | Description | Use Case |
|
||||||
|
|---|---|---|
|
||||||
|
| `auto/best-coding` | Best free coding model | Code generation, debugging |
|
||||||
|
| `auto/best-reasoning` | Best reasoning model | Math, logic, analysis |
|
||||||
|
| `auto/best-fast` | Fastest free model | Quick responses, chat |
|
||||||
|
| `auto/best-chat` | Best conversational model | General chat |
|
||||||
|
| `auto/best-vision` | Best vision model | Image understanding |
|
||||||
|
| `auto/cheap` | Cheapest option | High-volume, low-priority |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## CLI Commands Reference
|
||||||
|
|
||||||
|
### Test OmniRoute directly
|
||||||
|
```bash
|
||||||
|
openclaw infer model run --model omniroute/auto/best-fast --prompt "Hello" --local
|
||||||
|
```
|
||||||
|
|
||||||
|
### Add more OmniRoute models as fallbacks
|
||||||
|
```bash
|
||||||
|
openclaw models fallbacks add omniroute/auto/best-chat
|
||||||
|
openclaw models fallbacks list
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check OmniRoute health
|
||||||
|
```bash
|
||||||
|
curl -s http://localhost:20128/v1/models | jq '.data | length'
|
||||||
|
```
|
||||||
|
|
||||||
|
### Restart OmniRoute
|
||||||
|
```bash
|
||||||
|
docker restart omniroute-local
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Verified Test Results
|
||||||
|
|
||||||
|
All models tested and passing (2026-08-06):
|
||||||
|
|
||||||
|
| Model | Latency | Test |
|
||||||
|
|---|---|---|
|
||||||
|
| `auto/best-coding` | 2624ms | Python reverse string ✅ |
|
||||||
|
| `auto/best-fast` | 2270ms | Arithmetic (15×37) ✅ |
|
||||||
|
| `auto/best-reasoning` | 2049ms | Train speed/distance problem ✅ |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes for Agents
|
||||||
|
|
||||||
|
- OmniRoute runs as a Docker container on the host — no API keys required
|
||||||
|
- When primary model (Groq) hits rate limits or fails, fallback auto-kicks to OmniRoute
|
||||||
|
- OmniRoute internally rotates across free provider tiers to maximize uptime
|
||||||
|
- The `auto/*` routing prefix handles provider selection internally — no need to specify individual providers
|
||||||
|
- OmniRoute models are NOT listed by `openclaw models list` by default; they are custom provider models
|
||||||
Reference in New Issue
Block a user