Hermes Agent Integration

AVELIN API is the recommended backend for Hermes Agent — the open-source CLI AI assistant by Nous Research. AVELIN is an AI Laboratory that builds Cross-Model MoE (Mixture-of-Experts) models — trained AI systems that fuse knowledge from multiple model architectures into single unified models. We build and fuse models, we don't just route them. Hermes is a long-horizon agentic coding tool that runs terminal commands, edits files, searches the web, spawns subagents, and maintains persistent memory across sessions.

Why AVELIN + Hermes? Hermes sessions routinely run 100–500 turns with multi-file refactors, parallel subagent spawns, and 1M-token context windows. AVELIN is purpose-built for exactly this workload — powered by Cross-Model MoE technology built in our AI Lab, unifying 9 specialist model families into a single API with automatic failover, 65K+ output tokens per request, and specialist models that match frontier quality at a fraction of the cost. We build and fuse models — we don't just route them.


The fastest path. This single command installs Hermes (if it isn't already) and configures all AVELIN models, the default model/API, and Firecrawl-powered web search + scrape — then asks for your AVELIN API key and runs unattended.

curl -fsSL https://avelin.ai/docs/install/hermes.sh | bash

When prompted, paste your AVELIN key (sk-avelin-...) and press Enter. The script will:

  • Install Hermes via the official installer (skips this step if Hermes is already installed)
  • Register every AVELIN model and set avelin-coding-pro as the default
  • Point web search and scrape at Firecrawl (via AVELIN)
  • Wire voice (STT/TTS) on the same key
  • Back up any existing ~/.hermes/config.yaml and ~/.hermes/.env before changing them

Already have Hermes installed? Use the configure-only script — same result, no install step:

curl -fsSL https://avelin.ai/docs/install/hermes-configure.sh | bash

When it finishes, verify the connection:

hermes -z "hello from avelin"

Prefer to configure things by hand, or want a custom provider/model layout? The Manual Setup below covers every option in detail.


Manual Setup

Step 1: Install Hermes Agent

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

For Windows (PowerShell):

iex (irm https://hermes-agent.nousresearch.com/install.ps1)

Reload your shell: source ~/.bashrc or source ~/.zshrc

Step 2: Get your AVELIN API key

Sign up at avelin.ai and generate an API key. It will look like sk-avelin-....

Step 3: Configure Hermes

AVELIN supports two API formats. Choose based on your needs:

Option A: Anthropic-Compatible (Recommended)

Best for: Thinking support, structured tool calls, better streaming

hermes model

When prompted, select Anthropic or Custom Anthropic-compatible and enter:

FieldValue
Base URLhttps://api.avelin.ai/v1
API Keysk-avelin-...
Modelavelin-coding-ultra (or any AVELIN model)

Or manually edit ~/.hermes/config.yaml:

model:
  default: avelin-coding-ultra
  provider: anthropic
  base_url: https://api.avelin.ai/v1
  context_length: 1000000
  max_tokens: 65536
  thinking:
    enabled: true
    budget_tokens: 10000

Add your API key to ~/.hermes/.env:

ANTHROPIC_API_KEY=sk-avelin-...

Option B: OpenAI-Compatible

Best for: Compatibility with OpenAI SDKs, simpler setup

hermes model

When prompted, select Custom / OpenAI-compatible and enter:

FieldValue
Base URLhttps://api.avelin.ai/v1
API Keysk-avelin-...
Modelavelin-coding-ultra (or any AVELIN model)
Context length1000000
Max tokens65536

Or manually edit ~/.hermes/config.yaml:

model:
  default: avelin-coding-ultra
  provider: custom
  base_url: https://api.avelin.ai/v1
  context_length: 1000000
  max_tokens: 65536

Add your API key to ~/.hermes/.env:

CUSTOM_API_KEY=sk-avelin-...

Step 4: Verify the Connection

hermes chat

Ask: "What model are you using?" — it should respond with your AVELIN model name.

Test tool calling: "List the files in the current directory" — it should execute a terminal command.

Step 5: Start Coding

hermes chat --model avelin-coding-ultra

Or use the TUI (recommended for better UX):

hermes --tui

FeatureAnthropic EndpointOpenAI Endpoint
Thinking control✅ Explicit thinking.budget_tokens❌ Not supported
Thinking output✅ Separate thinking blocks⚠️ Mixed in reasoning_content
Tool calling✅ Structured tools with input_schema✅ OpenAI function format
Streaming✅ Server-Sent Events with typed events✅ NDJSON chunks
System prompts✅ Top-level system parameter⚠️ In messages array

Hermes was designed with Anthropic's API in mind. The Anthropic endpoint unlocks full thinking support, which is critical for complex coding tasks.


Model Families & Selection Guide

AVELIN provides 9 specialist models across 3 categories, built with Cross-Model MoE technology in our AI Lab — trained mixture-of-experts systems that fuse multiple model architectures into a single unified model (not just routing).

Enterprise licensing: All AVELIN models are available for sovereign deployment on customer GPUs. Request a POC for enterprise licensing to run AVELIN on your own infrastructure.

Intelligence Family — General Reasoning

For complex reasoning, planning, and general-purpose tasks where raw intelligence matters most.

ModelContextMax OutputBest For
avelin-ultra256K65KHighest capability reasoning, hard problems
avelin-pro256K65KPro-tier general tasks, balanced quality/speed
avelin-fast256K65KFast responses, no thinking, simple tasks

When to use: Architecture decisions, complex debugging, multi-step reasoning, tasks where correctness is more important than speed. The Intelligence family delivers frontier-class reasoning at ~up to 85% lower cost than comparable direct-provider access.

Recommended for Hermes: Use avelin-ultra or avelin-pro as your primary model when working on complex codebases where reasoning quality is critical.

Coding Family — Software Engineering

Purpose-built for code generation, refactoring, and multi-file engineering tasks with 1M token context.

ModelContextMax OutputBest For
avelin-coding-ultra1M65KComplex architecture, system design, large-scale planning
avelin-coding-pro1M65KLarge codebases, multi-file refactors, heavy editing
avelin-coding-fast1M65KStandard coding tasks, single-file edits, routine work

When to use: Any software engineering work — from quick edits to full-system refactors. The 1M context window means entire repositories fit in context without chunking. The Coding family delivers specialist coding performance at ~up to 86% lower cost than comparable direct-provider access.

Recommended for Hermes: Use avelin-coding-ultra or avelin-coding-pro as your daily driver for coding sessions. The 1M context is essential for Hermes' multi-file workflows. This is the default family (avelin-coding-pro is the install script default).

Agentic Family — Autonomous Agents

Optimized for tool use, multi-step execution, and long-horizon agent loops where reliability and instruction-following matter most.

ModelContextMax OutputBest For
avelin-agentic-pro256K65KStandard agentic tasks, tool-heavy workflows
avelin-agentic-ultra256K65KHigh-capability agentic tasks, complex multi-step plans
avelin-agentic-fast256K65KCost-optimized high-volume agent swarms, parallel subagents

When to use: When Hermes is operating as an autonomous agent — spawning subagents, executing multi-step plans, or running parallel tasks. The Agentic family is purpose-built for agent workloads at up to 87% lower cost than comparable direct-provider access.

Recommended for Hermes: Use avelin-agentic-ultra when you need maximum reliability in long autonomous sessions. Use avelin-agentic-fast for subagent spawns and high-volume parallel tasks where cost efficiency matters.

Quick Selection Matrix

Your TaskRecommended ModelWhy
Daily coding sessionsavelin-coding-pro1M context, strong coding AA score, good balance
Complex architectureavelin-coding-ultraHighest coding AA score (52), 1M context
Hard reasoning problemsavelin-ultraHighest Intelligence AA score (55)
Autonomous multi-stepavelin-agentic-ultraTop Agentic AA score (67), 1M context
Parallel subagentsavelin-agentic-fastLowest cost per agent, 256K sufficient for subtasks
Quick lookups & searchesavelin-fastFastest response time, lowest cost
Budget-conscious codingavelin-coding-fastAdequate for standard tasks at lowest coding-tier cost

All Models at a Glance

ModelFamilyContextMax OutputThinking
avelin-agentic-proAgentic256K65K✅
avelin-agentic-ultraAgentic256K65K✅
avelin-ultraIntelligence256K65K✅
avelin-proIntelligence256K65K✅
avelin-coding-ultraCoding1M65K✅
avelin-coding-proCoding1M65K✅
avelin-fastIntelligence256K65K❌
avelin-coding-fastCoding1M65K✅
avelin-agentic-fastAgentic256K65K✅

Thinking is enabled by default on pro and ultra tiers; the fast tiers run with reasoning off by default for latency.


Advanced Configuration

Thinking Support (Anthropic Endpoint Only)

Enable deep reasoning for complex tasks:

model:
  default: avelin-coding-ultra
  provider: anthropic
  base_url: https://api.avelin.ai/v1
  thinking:
    enabled: true
    budget_tokens: 10000  # Tokens allocated for thinking

Toggle thinking during a session:

/thinking on
/thinking off

Context Compression

Hermes automatically compresses long conversations. Tune for your workflow:

compression:
  enabled: true
  threshold: 0.50           # Compress at 50% of context limit
  target_ratio: 0.20        # Preserve 20% of threshold as recent tail
  protect_last_n: 20        # Min recent messages to keep uncompressed
  protect_first_n: 3        # Pin first 3 non-system head messages

Multi-Model Strategy

Use different models for different tasks — take advantage of AVELIN's specialist families:

model:
  default: avelin-coding-ultra
  providers:
    - name: avelin
      provider: anthropic
      base_url: https://api.avelin.ai/v1
      models:
        - avelin-coding-ultra   # Complex architecture
        - avelin-coding-pro        # Daily coding
        - avelin-agentic-fast       # Subagent spawns (AA 38, low cost)
        - avelin-fast               # Quick lookups (AA 47, no thinking)

Switch models mid-session:

/model avelin-fast               # Quick lookups, simple searches
/model avelin-coding-pro        # Back to coding
/model avelin-coding-ultra   # Complex architecture work
/model avelin-agentic-ultra       # Long autonomous agent runs

Tool Gateway

Enable web search, image generation, and TTS:

hermes gateway setup

Or manually add to ~/.hermes/config.yaml:

gateway:
  web_search: true
  image_gen: true
  tts: true

Terminal Backends

Run commands in isolation:

terminal:
  backend: docker           # local, docker, ssh, modal, daytona
  docker_image: ubuntu:22.04
  docker_mount_cwd: true

For SSH:

terminal:
  backend: ssh
  ssh_host: your-server.com
  ssh_user: dev
  ssh_key: ~/.ssh/id_rsa

Best Practices

1. Start with Anthropic Endpoint

The Anthropic endpoint provides better thinking support and is what Hermes was designed for. Only use OpenAI-compatible if you have specific SDK requirements.

2. Choose Your Family by Workload

Match the model family to your session type:

  • Coding sessions (editing files, refactoring, writing code) → Coding family (avelin-coding-pro or avelin-coding-ultra). The 1M context window lets Hermes hold your entire repository without compression.

  • Reasoning-heavy sessions (architecture decisions, debugging tricky issues, planning) → Intelligence family (avelin-ultra or avelin-pro). These are the deepest reasoning tiers.

  • Autonomous agent runs (multi-step plans, parallel subagents, long background tasks) → Agentic family (avelin-agentic-ultra for reliability, avelin-agentic-fast for volume). These tiers are tuned for reliable tool use.

3. Use Appropriate Context Length

  • Coding tasks: Set context_length: 1000000 (1M tokens) for full repository access
  • Large codebases: All Coding family models support 1M context
  • Agentic tasks: all Agentic family models support 256K
  • Simple tasks: Intelligence family models at 256K are sufficient

Hermes requires minimum 64K context for reliable tool calling.

4. Enable Thinking for Complex Tasks

thinking:
  enabled: true
  budget_tokens: 10000

Thinking helps with:

  • Multi-file refactors
  • Architecture decisions
  • Debugging complex issues
  • Security-sensitive code

5. Use Model Switching to Optimize Cost

Switch models strategically within a session to balance quality and cost:

Switch toFor
avelin-fastFile listings, simple searches, quick lookups
avelin-coding-fastStandard single-file edits, routine tasks
avelin-coding-proMulti-file refactors, large codebase work
avelin-coding-ultraSystem design, complex architecture
avelin-agentic-fastSpawning parallel subagents

6. Leverage Subagents

Hermes can spawn parallel subagents — use cost-efficient models for subagent work:

User: Refactor the auth module and update all tests

Hermes spawns:
  → Subagent 1: Refactor auth module (avelin-coding-pro)
  → Subagent 2: Update test suite (avelin-coding-pro)

For high-volume subagent work, consider avelin-agentic-fast to reduce per-agent costs while maintaining the 256K context each subagent needs.

7. Use Persistent Memory

Hermes remembers facts across sessions. Teach it your preferences:

Remember: I prefer functional programming patterns
Remember: Our codebase uses TypeScript strict mode

Verification & Testing

After setup, verify everything works:

Test 1: Basic Chat

hermes chat
> Hello, what model are you using?

Expected: Model name in response

Test 2: Tool Calling

> List the files in the current directory

Expected: Terminal command execution, file listing

Test 3: Thinking (Anthropic endpoint)

> Think step-by-step about how to implement a rate limiter

Expected: Extended reasoning before response

Test 4: Multi-turn Context

> My name is Alice
> What's my name?

Expected: "Alice"

Test 5: File Operations

> Create a file called test.py with a hello world function
> Read the file back

Expected: File created and read successfully


Troubleshooting

"Context too long" errors

Solution: Set context_length: 1000000 in config.yaml. AVELIN Coding models support 1M context; Intelligence and Agentic models support 256K.

Tool calls failing or malformed

Solution: Ensure you're using a model with tool support (all AVELIN models except avelin-fast have full tool calling).

"Invalid API key" error

Solution:

  • Verify your key starts with sk-avelin-
  • Check it's in the right env var: ANTHROPIC_API_KEY for Anthropic endpoint, CUSTOM_API_KEY for OpenAI endpoint
  • Ensure no extra whitespace in the key

Slow first response

Solution: Normal — the model thinks before responding. Streaming will start once reasoning completes.

"Context length too small" warning

Solution: Hermes requires minimum 64K context. Set:

context_length: 1000000

Model not found

Solution: Use exact model names from the table above. Common mistakes:

  • ❌ avelin-coding-ultra-200k (no context suffix)
  • ✅ avelin-coding-ultra

Streaming not working

Solution: Streaming is enabled by default. If not working:

  • Anthropic endpoint: Uses Server-Sent Events
  • OpenAI endpoint: Uses NDJSON chunks
  • Both should stream automatically

Rate limiting

Solution: AVELIN has generous rate limits. If you hit them:

  • Hermes retry logic handles it automatically
  • Consider switching to avelin-fast or avelin-agentic-fast for high-frequency tasks
  • Contact AVELIN support for higher limits

Example Session

$ hermes --tui

╭─ hermes ─────────────────────────────────╮
│ model: avelin-coding-ultra           │
│ provider: anthropic (avelin.ai)          │
│ context: 0% (0/1M)                       │
│ thinking: enabled                        │
╰──────────────────────────────────────────╯

You> I need to add rate limiting to our FastAPI app. Add middleware,
     write tests, and update the README.

Hermes> [thinking...]
        Let me understand the project structure first...
        
        [reads files, searches codebase, plans approach]

        Here is my plan:
        1. Add RateLimitMiddleware in middleware/rate_limit.py
        2. Configure per-route limits in the router
        3. Write pytest tests covering happy path + 429 responses
        4. Update README with the new configuration options

        Shall I proceed? [Y/n]

You> yes

Hermes> [creates middleware/rate_limit.py — 47 lines]
        [modifies main.py — adds middleware registration]
        [creates tests/test_rate_limit.py — 128 lines]
        [updates README.md — adds Rate Limiting section]

        Done. All 4 files updated, 6 tests passing.
        Want me to run the full test suite?

Comparison with Other Providers

FeatureAVELINOpenRouterAnthropic Direct
TechnologyCross-Model MoE (trained)RoutingSingle model
Thinking support✅ Full⚠️ Limited✅ Full
Context length256K–1MVaries200K
Specialist families9 models, 3 categoriesManyClaude only
Automatic failover✅ Yes✅ Yes❌ No
Cost vs direct48–up to 87% cheaperVariesList price
Rate limitsGenerousVariesStrict