LLM Providers

All LLM providers for the ask-rb ecosystem. A single gem containing every provider — each with its own wire format, auth, and capabilities, all implementing the same Ask::Provider contract.

gem "ask-llm-providers"

Quick Start

require "ask-llm-providers"

provider = Ask::Provider.resolve(:opencode_go).new
response = provider.chat(
  [{ role: "user", content: "Say OK in one word." }],
  model: "deepseek-v4-flash"
)
response.content
# => "OK"

Every provider in the gem registers itself with Ask::Provider, so you can look one up by slug instead of hardcoding a class. The OpenAI-compatible providers read their key from the matching *_API_KEY env var; the canonical providers (OpenAI, Anthropic, …) resolve theirs through Ask::Auth.

Provider Transformation Contract

Every provider implements the Ask::LLM::ProviderConfig interface, which separates wire-format concerns into five public methods:

provider.build_request(messages, model:, tools: nil, temperature: nil, stream: nil, schema: nil)
  # => Hash (provider-native request payload)

provider.parse_response(body, model)
  # => Ask::Message

provider.parse_stream(raw, stream, model, &block)
  # => yields Ask::Chunks

provider.format_tools(tools)
  # => Array (provider-native tool format)

provider.format_message(msg)
  # => Hash (provider-native message format)

This makes each wire-format concern testable in isolation and adding a new provider mechanical — implement five methods and the provider works. Inspired by LiteLLM’s BaseConfig pattern.

Canonical Providers

These providers have distinct wire formats. Each is a dedicated class with its own serialization, streaming, and auth.

Provider	Class	Capabilities	Auth
OpenAI	Ask::Providers::OpenAI	Chat, streaming, tools, vision, embeddings, image gen, transcription	OPENAI_API_KEY
Anthropic	Ask::Providers::Anthropic	Chat, streaming, tools, vision, thinking, prompt caching	ANTHROPIC_API_KEY
Google Gemini	Ask::Providers::Google	Chat, streaming, tools, vision, embeddings, file upload	GEMINI_API_KEY
Amazon Bedrock	Ask::Providers::Bedrock	Chat, streaming, tools, vision	AWS credentials chain
Ollama	Ask::Providers::Ollama	Chat, streaming, tools, embeddings (local)	None needed
Mistral AI	Ask::Providers::Mistral	Chat, streaming, tools, structured output, embeddings	MISTRAL_API_KEY
Cloudflare	Ask::Providers::Cloudflare	Chat, streaming, vision	CLOUDFLARE_API_KEY + Account ID

OpenAI-Compatible Providers

These share OpenAI’s wire format. Each is configured via a registry entry — no subclass, no new file. Adding a new provider is one line in lib/ask/llm/openai_compatible.rb.

Provider	Env Var	Capabilities
DeepSeek	DEEPSEEK_API_KEY	Chat, streaming, tools, thinking
OpenRouter	OPENROUTER_API_KEY	Chat, streaming, tools, vision, thinking, structured output
Groq	GROQ_API_KEY	Chat, streaming, tools, vision
Together	TOGETHER_API_KEY	Chat, streaming, tools
Fireworks	FIREWORKS_API_KEY	Chat, streaming, tools
Cerebras	CEREBRAS_API_KEY	Chat, streaming, tools
xAI	XAI_API_KEY	Chat, streaming, tools, vision, thinking
Perplexity	PERPLEXITY_API_KEY	Chat, streaming
Moonshot	MOONSHOT_API_KEY	Chat, streaming
DeepInfra	DEEPINFRA_API_KEY	Chat, streaming, tools
Anyscale	ANYSCALE_API_KEY	Chat, streaming, tools
SambaNova	SAMBANOVA_API_KEY	Chat, streaming, tools
Nebius	NEBIUS_API_KEY	Chat, streaming, tools
Nvidia NIM	NVIDIA_NIM_API_KEY	Chat, streaming, tools
Friendli	FRIENDLI_API_KEY	Chat, streaming, tools
Hyperbolic	HYPERBOLIC_API_KEY	Chat, streaming, tools
Novita	NOVITA_API_KEY	Chat, streaming, tools
Nscale	NSCALE_API_KEY	Chat, streaming, tools
Featherless	FEATHERLESS_API_KEY	Chat, streaming, tools
AI/ML API	AIML_API_KEY	Chat, streaming, tools
AI21	AI21_API_KEY	Chat, streaming, tools
Meta (Llama)	LLAMA_API_KEY	Chat, streaming, tools
GitHub Models	GITHUB_API_KEY	Chat, streaming, tools, vision
OpenCode	OPENCODE_API_KEY	Chat, streaming, tools
OpenCode Go	OPENCODE_GO_API_KEY	Chat, streaming, tools
Mimo	MIMO_API_KEY	Chat, streaming

Using a Model Through a Different Provider

A model is registered under a specific provider in the catalog (e.g., deepseek-v4-flash under the deepseek provider). But you may want to use it through a different provider that serves the same model — for instance, deepseek-v4-flash served by opencode_go.

Pass the provider: parameter to override which provider serves the model:

require "ask-agent"

session = Ask::Agent::Session.new(
  model: "deepseek-v4-flash",
  provider: :opencode_go,  # use opencode_go instead of the default deepseek provider
  tools: []
)

Or when using Ask::Agent::Chat directly:

require "ask-agent"

chat = Ask::Agent::Chat.new(
  model: "deepseek-v4-flash",
  provider: :opencode_go
)
chat.model_id  # => "deepseek-v4-flash"

This works because the agent checks provider: first, then falls back to the catalog’s provider. It also means you can use any OpenAI-compatible provider with any model name the provider supports, without adding model entries to the catalog.

The provider: parameter is available on both Ask::Agent::Session.new and Ask::Agent::Chat.new — it passes through to the underlying provider resolution.

Provider Registration

All providers auto-register with Ask::Provider on gem load:

require "ask-llm-providers"

Ask::Provider.providers.keys
# => [:openai,
#  :anthropic,
#  :gemini,
#  :bedrock,
#  :ollama,
#  :mistral,
#  :cloudflare,
#  :openai_codex,
#  :aiml,
#  :ai21,
#  :anyscale,
#  :cerebras,
#  :deepinfra,
#  :deepseek,
#  :featherless,
#  :fireworks,
#  :friendli,
#  :github,
#  :groq,
#  :hyperbolic,
#  :meta,
#  :mimo,
#  :moonshot,
#  :nebius,
#  :novita,
#  :nscale,
#  :nvidia_nim,
#  :opencode,
#  :opencode_go,
#  :github_copilot,
#  :openrouter,
#  :perplexity,
#  :sambanova,
#  :together,
#  :xai]
Ask::Provider.resolve(:openai)  # => Ask::Providers::OpenAI

Adding a new OpenAI-compatible provider

Add one entry to the registry — no new file, no subclass:

# lib/ask/llm/openai_compatible.rb
OPENAI_COMPATIBLE = {
  groq: { api_base: "https://api.groq.com/openai/v1", api_key_env: "GROQ_API_KEY",
          capabilities: { chat: true, streaming: true, tool_calls: true, vision: true } },
  # ... add yours here
}

Special quirks are handled declaratively:

deepseek: { api_base: "https://api.deepseek.com", api_key_env: "DEEPSEEK_API_KEY",
            reasoning_content: true },  # injects reasoning_content for tool call messages

openrouter: { api_base: "https://openrouter.ai/api/v1", api_key_env: "OPENROUTER_API_KEY",
              extra_headers: { "HTTP-Referer" => "...", "X-Title" => "ask-rb" } },

opencode_go: { api_base: "https://opencode.ai/zen/go/v1", api_key_env: "OPENCODE_GO_API_KEY" },

Streaming

stream = provider.chat(
  [{ role: "user", content: "Tell me a story" }],
  model: "deepseek-v4-flash",
  stream: true
) do |chunk|
  print chunk.content
end

Tool Calls

tools = [{
  name: "get_weather",
  description: "Get weather for a location",
  parameters: {
    type: "object",
    properties: { location: { type: "string" } },
    required: ["location"]
  }
}]

response = provider.chat(
  [{ role: "user", content: "What's the weather in NYC?" }],
  model: "deepseek-v4-flash",
  tools: tools
)
# response.tool_call? => true
# response.tool_calls => [{ name: "get_weather", arguments: "..." }]

Capabilities Introspection

require "ask-llm-providers"

Ask::Providers::OpenAI.capabilities
# => {chat: true,
#  streaming: true,
#  tool_calls: true,
#  vision: true,
#  thinking: true,
#  structured_output: true,
#  embed: true,
#  transcribe: true,
#  paint: true,
#  moderate: true,
#  prompt_caching: true}

Ask::Providers::Ollama.local?   # => true
Ask::Providers::OpenAI.local?   # => false

Error Handling

Provider errors map to structured Ask::Error types:

Ask::RateLimitError       # 429
Ask::Unauthorized         # 401/403
Ask::ServerError          # 500
Ask::ServiceUnavailable   # 503
Ask::ContextLengthExceeded # context_length_exceeded
Ask::ProviderError        # other errors

Model Catalog

The gem automatically populates Ask::ModelCatalog when loaded:

require "ask-llm-providers"

model = Ask::ModelCatalog.find("deepseek-v4-flash")
model.provider           # => "deepseek"
model.context_window     # => 1000000
model.max_output_tokens  # => 384000
model.supports?(:vision) # => false

Models are defined in JSON files under lib/ask/llm/models/ with pricing, context windows, and capabilities. Short names are resolved via aliases:

require "ask-llm-providers"

Ask::LLM::Aliases.resolve("claude-sonnet-4")
# => "claude-sonnet-4-6"

Development

git clone https://github.com/ask-rb/ask-llm-providers
cd ask-llm-providers
bundle exec rake test