mcp-config: enrich discovered models with limits, cost and capabilities

/v1/models returns ONLY model ids, so every model was deployed as {} and
opencode ran all of them on defaults -- no context window (so it cannot manage
truncation), no cost, and no idea whether the model can call tools. opencode
requires this metadata in the models map: it has no /v1/models discovery of its
own, and models.dev only covers known providers, not a private LiteLLM gateway.

sync.sh now also queries /model/info and maps LiteLLM's metadata onto opencode's
per-model spec:

  max_input_tokens          -> limit.context
  max_output_tokens         -> limit.output
  input/output_cost_per_token -> cost.input/output  (per-token -> $ per 1M)
  supports_function_calling -> tool_call
  supports_reasoning        -> reasoning
  supports_vision           -> attachment

Best-effort and backwards-compatible: /model/info is outside the llm_api_routes
scope most virtual keys have, so an un-updated key 403s and falls back to
names-only -- exactly today's behaviour, no regression.

Note the gateway only reports metadata for models in LiteLLM's registry. The
self-hosted vLLM entries (qwen3-*) carry no model_info in the gateway's
model_list, so they still deploy bare. Fixing that is a gateway-side change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-13 14:13:24 -04:00
co-authored by Claude Opus 4.8
parent cd09df07f6
commit 4cde3f5346
+53 -2
View File
@@ -94,13 +94,25 @@ _mcp_sync_refresh() {
"$MCP_SYNC_GATEWAY/v1/models" 2>/dev/null)" || _models=""
fi
# 2b. Enrich with per-model metadata. /v1/models returns ONLY ids — no context
# window, no cost, no capability flags — and opencode needs those or it
# runs every model on defaults (it cannot manage truncation without a
# context limit, and won't know a model can call tools). /model/info has
# them. Best-effort: a key still scoped to llm_api_routes 403s here, and
# we fall back to names-only, exactly as before.
_info=""
if [ -n "${LITELLM_KEY:-}" ]; then
_info="$(curl -fsS -m 8 -H "Authorization: Bearer $LITELLM_KEY" \
"$MCP_SYNC_GATEWAY/model/info" 2>/dev/null)" || _info=""
fi
# 3. Merge and deploy. python3 validates the base (don't clobber good config
# with an error page), injects the discovered models, and — when discovery
# failed — carries over the previous deployed models map so a gateway
# hiccup never empties the model list. Without python3, deploy the raw
# base as-is (no discovery).
if command -v python3 >/dev/null 2>&1; then
MCP_SYNC_BASE="$_base" MCP_SYNC_MODELS="$_models" \
MCP_SYNC_BASE="$_base" MCP_SYNC_MODELS="$_models" MCP_SYNC_MODELINFO="$_info" \
MCP_SYNC_PY_DEST="$MCP_SYNC_DEST" MCP_SYNC_PY_PROVIDER="$MCP_SYNC_PROVIDER" \
python3 - <<'PYEOF' 2>/dev/null || true
import json, os, tempfile
@@ -118,10 +130,49 @@ try:
except Exception:
raise SystemExit(0) # invalid base: keep last-good deployed config
# LiteLLM /model/info -> opencode's per-model spec. Only fields the gateway
# actually reports are emitted; a model LiteLLM knows nothing about (a
# self-hosted vLLM entry with no model_info in the gateway's model_list) simply
# stays {}, which is what every model got before this existed.
def spec_for(mi):
spec = {}
limit = {}
if mi.get("max_input_tokens"):
limit["context"] = mi["max_input_tokens"]
if mi.get("max_output_tokens"):
limit["output"] = mi["max_output_tokens"]
if limit:
spec["limit"] = limit
# LiteLLM prices PER TOKEN; opencode (like models.dev) uses $ per 1M tokens.
cost = {}
for src, dst in (("input_cost_per_token", "input"),
("output_cost_per_token", "output")):
if mi.get(src) is not None:
cost[dst] = mi[src] * 1_000_000
if cost:
spec["cost"] = cost
for src, dst in (("supports_function_calling", "tool_call"),
("supports_reasoning", "reasoning"),
("supports_vision", "attachment")):
if mi.get(src) is not None:
spec[dst] = bool(mi[src])
return spec
info = {}
try:
for entry in (json.loads(os.environ.get("MCP_SYNC_MODELINFO") or "{}")
.get("data", [])):
name = entry.get("model_name")
if name:
info[name] = spec_for(entry.get("model_info") or {})
except Exception:
pass
models = {}
try:
data = json.loads(os.environ.get("MCP_SYNC_MODELS") or "{}")
models = {m["id"]: {} for m in data.get("data", []) if m.get("id")}
models = {m["id"]: dict(info.get(m["id"], {}))
for m in data.get("data", []) if m.get("id")}
except Exception:
pass
if not models: