aakashb95andClaude Opus 4.6
73cecdbc26
RubyLLM uses a static bundled models.json as its model registry. New models released by providers (e.g., gpt-5.1, gpt-5.2) are not available until the gem is updated, causing ModelNotFoundError for self-hosted users who configure newer models. Refresh the model registry synchronously during Llm::Config.initialize! to fetch the latest model list from configured providers and models.dev, then persist with save_to_json. This runs once per process on the first LLM request, adding ~1-2s to that request. Falls back gracefully to the bundled registry on failure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
58 lines
1.3 KiB
Ruby
58 lines
1.3 KiB
Ruby
require 'ruby_llm'
|
|
|
|
module Llm::Config
|
|
DEFAULT_MODEL = 'gpt-4.1-mini'.freeze
|
|
|
|
class << self
|
|
def initialized?
|
|
@initialized ||= false
|
|
end
|
|
|
|
def initialize!
|
|
return if @initialized
|
|
|
|
configure_ruby_llm
|
|
refresh_model_registry
|
|
@initialized = true
|
|
end
|
|
|
|
def reset!
|
|
@initialized = false
|
|
end
|
|
|
|
def with_api_key(api_key, api_base: nil)
|
|
context = RubyLLM.context do |config|
|
|
config.openai_api_key = api_key
|
|
config.openai_api_base = api_base
|
|
end
|
|
|
|
yield context
|
|
end
|
|
|
|
private
|
|
|
|
def refresh_model_registry
|
|
RubyLLM.models.refresh!
|
|
RubyLLM.models.save_to_json
|
|
rescue StandardError => e
|
|
Rails.logger.warn "Failed to refresh RubyLLM model registry: #{e.message}"
|
|
end
|
|
|
|
def configure_ruby_llm
|
|
RubyLLM.configure do |config|
|
|
config.openai_api_key = system_api_key if system_api_key.present?
|
|
config.openai_api_base = openai_endpoint.chomp('/') if openai_endpoint.present?
|
|
config.logger = Rails.logger
|
|
end
|
|
end
|
|
|
|
def system_api_key
|
|
InstallationConfig.find_by(name: 'CAPTAIN_OPEN_AI_API_KEY')&.value
|
|
end
|
|
|
|
def openai_endpoint
|
|
InstallationConfig.find_by(name: 'CAPTAIN_OPEN_AI_ENDPOINT')&.value
|
|
end
|
|
end
|
|
end
|