Explore the expanded provider coverage, new AI operations, conversation controls, and Rails integration in RubyLLM 2.0.
After reading this guide, you will know:
How much more of each provider’s API you can use in 2.0.
How to use tool approvals, citations, thinking, caching, and fallbacks.
Which new APIs you can use for media, documents, and search.
How batches, usage tracking, and workflows support larger applications.
How these features fit into Rails.
RubyLLM 2.0 expands the framework across conversations, agents, media, documents, and Rails. You can use much more of each provider’s API through Ruby methods, with consistent results, streaming, and usage tracking. You also get more control over how conversations run and persist.
1.16 already supported chat, tools, agents, structured output, thinking, embeddings, images, transcription, and moderation. 2.0 adds video, speech, OCR, reranking, files, batches, and a shared API for provider-hosted tools. It also extends the existing APIs with more controls and richer results.
Red cells show built-in support added in 2.0. Gray cells were already supported in 1.16. Outlined cells with a × mark missing integrations; use “Missing in 2.0” to find them. Select a cell for its sources and implementation notes.
Red cells mark built-in support added in 2.0, including features previously available through raw options or partial integration. Select a cell for evidence.
No rows match these filters.
The Provider API Coverage page shows current support, with source references and remaining gaps. The comparison above uses today’s documented provider features for both versions, so it includes features providers introduced after 1.16.
Providers and Protocols
Cohere, Deepgram, ElevenLabs, and Ollama Cloud join the built-in providers, bringing the total to seventeen.
Providers and protocols are now separate. A provider supplies authentication, endpoints, model catalogs, and service-specific behavior. A protocol handles request formats, response parsing, and streaming. RubyLLM selects the protocol for the model and operation, so your application keeps the same API across providers.
A new provider can reuse an existing protocol. The provider gem generator creates the package, configuration, and tests to get started.
Human Approval for Tools
A tool can now require approval before it runs. In a Rails app with a Post model, you can let an agent prepare a post while leaving publication to a person:
classPublishPost<RubyLLM::Tooldescription"Publishes a draft post"requires_approvaldefexecute(post_id:)Post.find(post_id).update!(published: true)"Published post #{post_id}"endendchat=RubyLLM.chat.with_tools(PublishPost)chat.ask"Publish post 42."chat.awaiting_approval?# => true when the model requests publication
The tool call stays pending and ask returns. Your application can show the proposed action, collect a decision, and continue from its approval handler:
Use deny to reject a call. The model receives the decision and can respond to it. See Tool Approvals.
You also get explicit control over the conversation loop. ask_later stages a question, generate asks the model for one response, run_tools executes pending tools, and step advances the conversation by one generation or tool execution. Use them to set limits, hand work to another agent, or run one turn per job. See Agentic Workflows.
Citations
Citations now have a common result object for document references, web search, and grounding. Enable document citations, ask a question, and read the passages the model used:
chat=RubyLLM.chat(model: "claude-sonnet-5").with_citationsresponse=chat.ask"What are the report's main findings?",with: "report.pdf"response.citations.eachdo|citation|putscitation.cited_textputscitation.start_pageend
Web citations expose url and title; document citations can include page or character locations. RubyLLM also collects citations while streaming and saves them with Rails messages. See Citations for supported providers and citable search results from your own tools.
Thinking Controls
1.16 let you set thinking effort and token budgets. In 2.0, you can also let RubyLLM choose the model’s default thinking settings:
chat=RubyLLM.chat(model: "claude-sonnet-5").with_thinkingresponse=chat.ask"Find the flaw in this argument: every square is a rectangle, so every rectangle is a square."putsresponse.content
Use with_thinking(effort: :high) or with_thinking(budget: 10_000) when you need a specific setting. with_thinking(false) turns thinking off where the model allows it. The defaults follow the model when you change it, including during a fallback.
You can also request thinking summaries on supported models and read them through response.thinking. See Thinking.
Prompt Caching
Prompt caching now has a common API across supported providers. Enable it with with_caching, and mark a reusable prefix with cache_until_here:
chat=RubyLLM.chat(model: "claude-sonnet-5").with_cachingchat.with_instructions(File.read("support-policy.txt")).cache_until_herechat.ask"Can I return an order after 20 days?"response=chat.ask"What if the item arrived damaged?"response.tokens.cache_read
The boundary marks the end of the policy, before the changing questions. Boundaries also persist on Rails messages. Providers still set the minimum prefix length, lifetime, and supported models; a cache hit is not guaranteed.
You can also create reusable cache resources with RubyLLM.cache on Gemini and Vertex AI. See Prompt Caching for automatic caching, boundaries, and cache resources.
Model Fallbacks
Choose another model to try when a request fails with a transient provider or network error:
chat=RubyLLM.chat(model: "gpt-5.6-luna").with_fallbacks("claude-sonnet-5")response=chat.ask"Explain Ruby pattern matching with an example."
Configure credentials for both providers. The conversation keeps its tools, schema, and settings, so choose fallback models that support the features you use. Usage tracking includes the failed attempts as well as the successful one. See Model Fallbacks.
Video and Speech Generation
Generate a video and save it with the same pattern you use for images:
video=RubyLLM.animate"A red panda typing Ruby code, with rain at the window"video.save"red_panda.mp4"
animate waits for the result. animate_later returns a job you can check and collect later. See Video Generation for animating images and choosing a provider.
Turn text into speech, too:
speech=RubyLLM.speak"Welcome to the Ruby study group."speech.save"welcome.mp3"
An empty speaker_names array asks the model to identify speakers without assigning known names. Supported models can also stream the transcript as it arrives. See Audio Transcription for live transcription, speaker labels, and timing formats.
OCR, Multimodal Embeddings, and Reranking
The new OCR API extracts Markdown from PDFs and scanned images:
Use OCR when you need the document’s text for indexing, extraction, or later model calls.
Reranking orders search results by how well they answer a question:
documents=["Invoices arrive by email.","Reset your password in Settings."]ranked=RubyLLM.rerank("How do I reset my password?",documents,model: "rerank-v3.5")putsranked.results.first.document
Moderation also supports configured Bedrock guardrails. Text checks use the existing RubyLLM.moderate API and report the guardrail’s assessment and usage without requiring a generation model.
Provider Tools
Models can use tools hosted by the provider, including web search, code execution, and remote MCP servers. Enable them on a chat with with_provider_tools, or declare them on an agent:
classResearchAgent<RubyLLM::Agentmodel"claude-sonnet-5"instructions"Research the question and cite your sources."provider_tools:web_searchendresponse=ResearchAgent.new.ask"What changed in the latest Ruby release?"response.citations.each{|citation|putscitation.url}
Combine provider tools with tools that run your Ruby code. Both work with streaming and follow-up questions. See Provider Tools.
Hosted Research
Run a provider’s research agent and read its report:
report=RubyLLM.research("Find the official Ruby documentation and explain where its API reference lives.",provider: :vertexai,agent: "deep-research-preview-04-2026")putsreport.content
research_later returns a job ID you can save, retrieve, poll, or cancel. Vertex AI supports this through its Deep Research agent, including remote MCP tools. See Hosted Research for credentials, tools, citations, and recovery.
Batch Processing
Submit chats or embedding requests to a provider’s batch API when the results can arrive later. Stage the questions with the same chat settings you use for interactive work:
chats=["Ruby blocks","Rails migrations"].mapdo|topic|RubyLLM.chat(model: "gpt-5.6-luna").with_instructions("Explain the topic in one paragraph.").ask_later(topic)endbatch=RubyLLM.batch(chats)
Save batch.id. Another process can find the batch and check whether it has finished:
Once complete, batch.messages returns the results in submission order. Batch pricing and turnaround depend on the provider; RubyLLM uses batch rates when calculating the results’ costs. See Batches for polling, failures, embedding batches, and conversations with tools.
Tokenization and Token Counting
Inspect the token IDs a model uses for your text:
result=RubyLLM.tokenize("Ruby makes AI useful.",model: "grok-4.3")result.idsresult.count
For a complete chat input, use chat.count_tokens before asking the model to generate a response:
chat=RubyLLM.chat(model: "claude-sonnet-5").with_instructions("Review the contract for renewal terms.")chat.count_tokens("What should I check in a renewal clause?")
See Tokenization for the standalone counting API, supported inputs, and the difference between input counts and billed usage.
Usage and Cost Tracking
Usage tracking now follows individual provider attempts, including retries, fallbacks, and cancelled requests. Read normalized token counts and costs through the same objects:
chat=RubyLLM.chatresponse=chat.ask"Explain Ruby fibers in one paragraph."response.tokens.inputresponse.tokens.outputresponse.cost.totalchat.cost.total
An answer that required several attempts includes their reported usage. Unknown usage or pricing stays nil, so missing information does not look like a free request.
In Rails, the usage ledger records attempts separately from messages and keeps the costs calculated at completion. Updating model prices later does not rewrite that history. See Tokens and Costs.
Workflow Instrumentation
Group a piece of work with RubyLLM.workflow and name its steps. Calls inside each step carry the workflow and step identifiers in their instrumentation events:
RubyLLM.workflow("Summarize meeting")do|workflow|transcript=workflow.step("Transcribe")doRubyLLM.transcribe("meeting.wav").textendworkflow.step("Summarize")doRubyLLM.chat.ask("List the decisions and action items:\n#{transcript}").contentendend
Use ordinary Ruby for branching, loops, and concurrency. Rails sends the events through ActiveSupport::Notifications; plain Ruby applications can configure an instrumenter. See Instrumentation for connecting your logs and tracing tools.
Rails Persistence and Durable Agents
The new loop controls and approval decisions also work on persisted conversations. Declare an agent with your application’s chat model and the PublishPost tool from above:
classEditorialAgent<RubyLLM::Agentchat_modelChatmodel"gpt-5.6-luna"toolsPublishPostendchat=EditorialAgent.create!chat.ask"Publish post 42."
Once a user approves a pending call, an approval handler or job can reload the agent and continue:
The saved transcript records completed work. If a job stops before saving a result, that operation may run again, so tools need to tolerate retries. Durable Agents shows how to run turns with Active Job and resume after interruptions.
RubyLLM now owns the model-registry, tool-call, usage, and batch tables. These records describe the framework’s work, so RubyLLM can evolve their schema without asking every application to maintain its own supporting models. Your app owns its chats and messages.
The upgrade runs in phases, with cleanup in a later deployment. Optional copy mode keeps a controlled route back to 1.16; conversations written by 2.0 remain hidden during that rollback.
The Rails integration uses the same Ruby API with Active Record persistence, Active Storage attachments, and Hotwire streaming. The generators set up those pieces in conventional Rails directories.
The model registry uses the same RubyLLM.models API in plain Ruby and Rails, backed by a file cache or RubyLLM’s database table. Browse Models to compare providers, capabilities, and prices.
API Consistency
The API uses one name for each concept across chats, agents, and persisted records. For example, max_output_tokens replaces max_tokens, and provider-specific request options use with_provider_options. Responses expose typed token counts, costs, citations, and other results through readers.
The upgrade guide lists the renames and Rails migration steps. Two other additions are context compaction with with_compaction, and file storage with RubyLLM.upload and RubyLLM.download.