fix: refresh and persist RubyLLM model registry on initialization

RubyLLM uses a static bundled models.json as its model registry. New
models released by providers (e.g., gpt-5.1, gpt-5.2) are not available
until the gem is updated, causing ModelNotFoundError for self-hosted
users who configure newer models.

Refresh the model registry asynchronously via a background thread during
Llm::Config.initialize! to fetch the latest model list from configured
providers and models.dev, then persist with save_to_json. The bundled
registry is available immediately; the refresh updates it in the
background without blocking the first LLM request.

Use double-checked locking with a mutex to prevent concurrent threads
from spawning duplicate refresh operations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
aakashb95
2026-02-17 22:36:14 +05:30
co-authored by Claude Opus 4.6
parent f4900f45d5
commit d11e058db2
2 changed files with 19 additions and 15 deletions
+12 -15
View File
@@ -2,8 +2,7 @@ require 'ruby_llm'
module Llm::Config
DEFAULT_MODEL = 'gpt-4.1-mini'.freeze
REGISTRY_TTL = 1.hour
REGISTRY_PATH = Rails.root.join('tmp/ruby_llm_models.json').to_s
INIT_MUTEX = Mutex.new
class << self
def initialized?
@@ -13,9 +12,13 @@ module Llm::Config
def initialize!
return if @initialized
configure_ruby_llm
refresh_model_registry
@initialized = true
INIT_MUTEX.synchronize do
return if @initialized
configure_ruby_llm
refresh_model_registry
@initialized = true
end
end
def reset!
@@ -34,18 +37,12 @@ module Llm::Config
private
def refresh_model_registry
if fresh_registry_file?
RubyLLM.models.load_from_json!(REGISTRY_PATH)
else
Thread.new do
RubyLLM.models.refresh!
RubyLLM.models.save_to_json(REGISTRY_PATH)
RubyLLM.models.save_to_json
rescue StandardError => e
Rails.logger.warn "Failed to refresh RubyLLM model registry: #{e.message}"
end
rescue StandardError => e
Rails.logger.warn "Failed to refresh RubyLLM model registry: #{e.message}"
end
def fresh_registry_file?
File.exist?(REGISTRY_PATH) && File.mtime(REGISTRY_PATH) > REGISTRY_TTL.ago
end
def configure_ruby_llm
+7
View File
@@ -0,0 +1,7 @@
RSpec.configure do |config|
config.before do
models_instance = RubyLLM.models
allow(models_instance).to receive(:refresh!)
allow(models_instance).to receive(:save_to_json)
end
end