RubyGems - llm_ruby - Versions diffs - 0.1.0 - Mend

llm_ruby 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (15) hide show

checksums.yaml +7 -0
data/.rspec +3 -0
data/.standard.yml +3 -0
data/LICENSE.txt +21 -0
data/README.md +153 -0
data/Rakefile +10 -0
data/lib/llm/clients/open_ai/response.rb +42 -0
data/lib/llm/clients/open_ai.rb +109 -0
data/lib/llm/info.rb +94 -0
data/lib/llm/provider.rb +0 -0
data/lib/llm/response.rb +3 -0
data/lib/llm/stop_reason.rb +9 -0
data/lib/llm/version.rb +1 -0
data/lib/llm.rb +50 -0
metadata +185 -0

checksums.yaml ADDED Viewed

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: dfe908817dd406ae16aca4130133b9a421b18333cdecc4ad870635dd997be500
+  data.tar.gz: 105ae0dcc30918686abcf8d01d99605d2f70f41eebdd2737744bc2bf27c6575c
+SHA512:
+  metadata.gz: 5b9643df8771735111f18f52182b6f217231ca92c890e3456f21c3937850bf2c9ac668730cd1aeae8b43330d5a3c84eab6b419c60aa5d54d70f405793e4463ad
+  data.tar.gz: 820687838675aeaadde8e5b7c5b7f7f45bfdf10beb74b90e2b594cd80d5e7accc38ace2deec56a6a2fcdc3de6a369af41405c5523da3e4c2b47f6e584c28f3fd

data/.rspec ADDED Viewed

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/.standard.yml ADDED Viewed

@@ -0,0 +1,3 @@
+# For available configuration options, see:
+#   https://github.com/standardrb/standard
+ruby_version: 3.1.0

data/LICENSE.txt ADDED Viewed

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+Copyright (c) 2024 TODO: Write your name
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md ADDED Viewed

@@ -0,0 +1,153 @@
+# LLMRuby
+LLMRuby is a Ruby gem that provides a consistent interface for interacting with various Large Language Model (LLM) APIs, with a current focus on OpenAI's models.
+## Installation
+Add this line to your application's Gemfile:
+```ruby
+gem 'llm_ruby'
+```
+And then execute:
+```
+$ bundle install
+```
+Or install it yourself as:
+```
+$ gem install llm_ruby
+```
+## Usage
+### Basic Usage
+```ruby
+require 'llm'
+# Initialize an LLM instance
+llm = LLM.from_string!("gpt-4")
+# Create a client
+client = llm.client
+# Send a chat message
+response = client.chat([{role: :user, content: "Hello, world!"}])
+puts response.content
+```
+### Streaming Responses
+LLMRuby supports streaming responses:
+```ruby
+require 'llm'
+# Initialize an LLM instance
+llm = LLM.from_string!("gpt-4")
+# Create a client
+client = llm.client
+# Define the on_message callback
+on_message = proc do |message|
+  puts "Received message chunk: #{message}"
+end
+# Define the on_complete callback
+on_complete = proc do |stop_reason|
+  puts "Streaming complete. Stop reason: #{stop_reason}"
+end
+# Send a chat message with streaming enabled
+response = client.chat(
+  [{role: :user, content: "Hello, world!"}],
+  stream: true,
+  on_message: on_message,
+  on_complete: on_complete
+)
+puts response.content
+```
+### Using the Response Object
+The response object returned by the `client.chat` method contains several useful fields:
+- `content`: The final content of the response.
+- `raw_response`: The raw response payload for non-streaming requests and the array of chunks for streaming requests.
+- `stop_reason`: The reason why the response generation was stopped.
+Here is an example of how to use the response object:
+```ruby
+# Initialize an LLM instance
+llm = LLM.from_string!("gpt-4")
+# Create a client
+client = llm.client
+# Send a chat message
+response = client.chat([{role: :user, content: "Hello, world!"}])
+# Access the response fields
+puts "Response content: #{response.content}"
+puts "Raw response: #{response.raw_response}"
+puts "Stop reason: #{response.stop_reason}"
+```
+## Available Models
+LLMRuby supports various OpenAI models, including GPT-3.5 and GPT-4 variants. You can see the full list of supported models in the `KNOWN_MODELS` constant:
+| Canonical Name            | Display Name           | Provider |
+|---------------------------|------------------------|----------|
+| gpt-3.5-turbo             | GPT-3.5 Turbo          | openai   |
+| gpt-3.5-turbo-0125        | GPT-3.5 Turbo 0125     | openai   |
+| gpt-3.5-turbo-16k         | GPT-3.5 Turbo 16K      | openai   |
+| gpt-3.5-turbo-1106        | GPT-3.5 Turbo 1106     | openai   |
+| gpt-4                     | GPT-4                  | openai   |
+| gpt-4-32k                 | GPT-4 32K              | openai   |
+| gpt-4-1106-preview        | GPT-4 Turbo 1106       | openai   |
+| gpt-4-turbo-2024-04-09    | GPT-4 Turbo 2024-04-09 | openai   |
+| gpt-4-0125-preview        | GPT-4 Turbo 0125       | openai   |
+| gpt-4-turbo-preview       | GPT-4 Turbo            | openai   |
+| gpt-4-0613                | GPT-4 0613             | openai   |
+| gpt-4-32k-0613            | GPT-4 32K 0613         | openai   |
+| gpt-4o                    | GPT-4o                 | openai   |
+| gpt-4o-mini               | GPT-4o Mini            | openai   |
+| gpt-4o-2024-05-13         | GPT-4o 2024-05-13      | openai   |
+| gpt-4o-2024-08-06         | GPT-4o 2024-08-06      | openai   |
+## Configuration
+Set your OpenAI API key as an environment variable:
+```
+export OPENAI_API_KEY=your_api_key_here
+```
+## Development
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+To install this gem onto your local machine, run `bundle exec rake install`.
+## Contributing
+Bug reports and pull requests are welcome on GitHub at https://github.com/contextco/llm_ruby.
+## License
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+## Acknowledgements
+This gem is developed and maintained by [Context](https://context.ai).

data/Rakefile ADDED Viewed

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+RSpec::Core::RakeTask.new(:spec)
+require "standard/rake"
+task default: %i[spec standard]

data/lib/llm/clients/open_ai/response.rb ADDED Viewed

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+class LLM::Clients::OpenAI::Response
+  def initialize(raw_response)
+    @raw_response = raw_response
+  end
+  def to_normalized_response
+    LLM::Response.new(
+      content: content,
+      raw_response: parsed_response,
+      stop_reason: normalize_stop_reason
+    )
+  end
+  def self.normalize_stop_reason(stop_reason)
+    case stop_reason
+    when "stop"
+      LLM::StopReason::STOP
+    when "safety"
+      LLM::StopReason::SAFETY
+    when "max_tokens"
+      LLM::StopReason::MAX_TOKENS_REACHED
+    else
+      LLM::StopReason::OTHER
+    end
+  end
+  private
+  def content
+    parsed_response.dig("choices", 0, "message", "content")
+  end
+  def normalize_stop_reason
+    self.class.normalize_stop_reason(parsed_response.dig("choices", 0, "finish_reason"))
+  end
+  def parsed_response
+    @raw_response.parsed_response
+  end
+end

data/lib/llm/clients/open_ai.rb ADDED Viewed

@@ -0,0 +1,109 @@
+# frozen_string_literal: true
+require "httparty"
+require "event_stream_parser"
+class LLM::Clients::OpenAI
+  include HTTParty
+  base_uri "https://api.openai.com/v1"
+  def initialize(llm:)
+    @llm = llm
+  end
+  def chat(messages, options = {})
+    parameters = {
+      model: @llm.canonical_name,
+      messages: messages,
+      temperature: options[:temperature],
+      response_format: options[:response_format],
+      max_tokens: options[:max_output_tokens],
+      top_p: options[:top_p],
+      stop: options[:stop_sequences],
+      presence_penalty: options[:presence_penalty],
+      frequency_penalty: options[:frequency_penalty],
+      tools: options[:tools],
+      tool_choice: options[:tool_choice]
+    }.compact
+    return chat_streaming(parameters, options[:on_message], options[:on_complete]) if options[:stream]
+    resp = post_url("/chat/completions", body: parameters.to_json)
+    Response.new(resp).to_normalized_response
+  end
+  private
+  def chat_streaming(parameters, on_message, on_complete)
+    buffer = +""
+    chunks = []
+    output_data = {}
+    wrapped_on_complete = lambda { |stop_reason|
+      output_data[:stop_reason] = stop_reason
+      on_complete&.call(stop_reason)
+    }
+    parameters[:stream] = true
+    proc = stream_proc(buffer, chunks, on_message, wrapped_on_complete)
+    parameters.delete(:on_message)
+    parameters.delete(:on_complete)
+    _resp = post_url_streaming("/chat/completions", body: parameters.to_json, &proc)
+    LLM::Response.new(
+      content: buffer,
+      raw_response: chunks,
+      stop_reason: output_data[:stop_reason]
+    )
+  end
+  def stream_proc(buffer, chunks, on_message, complete_proc)
+    each_json_chunk do |_type, event|
+      next if event == "[DONE]"
+      chunks << event
+      new_content = event.dig("choices", 0, "delta", "content")
+      stop_reason = event.dig("choices", 0, "finish_reason")
+      buffer << new_content unless new_content.nil?
+      on_message&.call(new_content) unless new_content.nil?
+      complete_proc&.call(Response.normalize_stop_reason(stop_reason)) unless stop_reason.nil?
+    end
+  end
+  def each_json_chunk
+    parser = EventStreamParser::Parser.new
+    proc do |chunk, _bytes, env|
+      if env && env.status != 200
+        raise_error = Faraday::Response::RaiseError.new
+        raise_error.on_complete(env.merge(body: try_parse_json(chunk)))
+      end
+      parser.feed(chunk) do |type, data|
+        next if data == "[DONE]"
+        yield(type, JSON.parse(data))
+      end
+    end
+  end
+  def post_url(url, **kwargs)
+    self.class.post(url, **kwargs.merge(headers: default_headers))
+  end
+  def post_url_streaming(url, **kwargs, &block)
+    self.class.post(url, **kwargs.merge(headers: default_headers, stream_body: true), &block)
+  end
+  def default_headers
+    {
+      "Authorization" => "Bearer #{ENV["OPENAI_API_KEY"]}",
+      "Content-Type" => "application/json"
+    }
+  end
+end

data/lib/llm/info.rb ADDED Viewed

@@ -0,0 +1,94 @@
+# frozen_string_literal: true
+module LLM::Info
+  KNOWN_MODELS = [
+    # Semantics of fields:
+    # - canonical_name (required): A string that uniquely identifies the model.
+    #   We use this string as the public identifier when users choose this model via the API.
+    # - display_name (required): A string that is displayed to the user when choosing this model via the UI.
+    # GPT-3.5 Turbo Models
+    {
+      canonical_name: "gpt-3.5-turbo",
+      display_name: "GPT-3.5 Turbo",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-3.5-turbo-0125",
+      display_name: "GPT-3.5 Turbo 0125",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-3.5-turbo-16k",
+      display_name: "GPT-3.5 Turbo 16K",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-3.5-turbo-1106",
+      display_name: "GPT-3.5 Turbo 1106",
+      provider: :openai
+    },
+    # GPT-4 Models
+    {
+      canonical_name: "gpt-4",
+      display_name: "GPT-4",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4-32k",
+      display_name: "GPT-4 32K",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4-1106-preview",
+      display_name: "GPT-4 Turbo 1106",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4-turbo-2024-04-09",
+      display_name: "GPT-4 Turbo 2024-04-09",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4-0125-preview",
+      display_name: "GPT-4 Turbo 0125",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4-turbo-preview",
+      display_name: "GPT-4 Turbo",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4-0613",
+      display_name: "GPT-4 0613",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4-32k-0613",
+      display_name: "GPT-4 32K 0613",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4o",
+      display_name: "GPT-4o",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4o-mini",
+      display_name: "GPT-4o Mini",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4o-2024-05-13",
+      display_name: "GPT-4o 2024-05-13",
+      provider: :openai
+    },
+    {
+      canonical_name: "gpt-4o-2024-08-06",
+      display_name: "GPT-4o 2024-08-06",
+      provider: :openai
+    }
+  ].freeze
+end

data/lib/llm/provider.rb ADDED Viewed

File without changes

data/lib/llm/response.rb ADDED Viewed

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+LLM::Response = Struct.new(:content, :raw_response, :stop_reason, keyword_init: true)

data/lib/llm/stop_reason.rb ADDED Viewed

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+module LLM::StopReason
+  STOP = :stop
+  SAFETY = :safety
+  MAX_TOKENS_REACHED = :max_tokens
+  OTHER = :other
+end

data/lib/llm/version.rb ADDED Viewed

	@@ -0,0 +1 @@
1	+ # frozen_string_literal: true

data/lib/llm.rb ADDED Viewed

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+require "zeitwerk"
+loader = Zeitwerk::Loader.for_gem
+loader.inflector.inflect(
+  "llm" => "LLM",
+  "open_ai" => "OpenAI"
+)
+loader.setup
+class LLM
+  def initialize(model)
+    @canonical_name = model[:canonical_name]
+    @display_name = model[:display_name]
+    @provider = model[:provider]
+    @client_class = LLM::Clients::OpenAI # TODO: Allow alternative client classes.
+  end
+  def client
+    client_class.new(llm: self)
+  end
+  attr_reader :canonical_name,
+    :display_name,
+    :provider
+  private
+  attr_reader :client_class
+  class << self
+    def all!
+      known_models
+    end
+    def from_string!(model_string)
+      model = known_models.find { |model| model.canonical_name == model_string }
+      raise ArgumentError, "Unknown model: #{model_string}" unless model
+      model
+    end
+    private
+    def known_models
+      @known_models ||= LLM::Info::KNOWN_MODELS.map { |model| new(model) }
+    end
+  end
+end

metadata ADDED Viewed

@@ -0,0 +1,185 @@
+--- !ruby/object:Gem::Specification
+name: llm_ruby
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Alex Gamble
+autorequire:
+bindir: exe
+cert_chain: []
+date: 2024-09-13 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: event_stream_parser
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+- !ruby/object:Gem::Dependency
+  name: httparty
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.22.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.22.0
+- !ruby/object:Gem::Dependency
+  name: zeitwerk
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 2.6.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 2.6.0
+- !ruby/object:Gem::Dependency
+  name: dotenv
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.1.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.1.0
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: standard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.31.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.31.0
+- !ruby/object:Gem::Dependency
+  name: vcr
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.3.1
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.3.1
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.23.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.23.0
+description:
+email:
+- [email protected]
+- [email protected]
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".rspec"
+- ".standard.yml"
+- LICENSE.txt
+- README.md
+- Rakefile
+- lib/llm.rb
+- lib/llm/clients/open_ai.rb
+- lib/llm/clients/open_ai/response.rb
+- lib/llm/info.rb
+- lib/llm/provider.rb
+- lib/llm/response.rb
+- lib/llm/stop_reason.rb
+- lib/llm/version.rb
+homepage: https://context.ai
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://context.ai
+  source_code_uri: https://github.com/contextco/llm_ruby
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 3.1.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.5.16
+signing_key:
+specification_version: 4
+summary: A client to interact with LLM APIs in a consistent way.
+test_files: []