Integrate AI with Rails: AI bootcamp for Developers – Day 4 -Practical Course – Part 3

Great. Now we can make the first real LLM request.

We’ll keep this step deliberately small. Our goal is not to build the complete AI assistant yet.

The goal is simply:

Rails
  ↓
Ai::Client
  ↓
OpenAI API
  ↓
LLM
  ↓
Response

Once this works, we’ll build the Rails service layer around it.

Step 5.11 – Add the OpenAI Ruby SDK

Rather than manually constructing HTTP requests, we’ll start with the official Ruby SDK.

1. Add the gem

Open our Gemfile and add:

gem "openai"

Then run:

bundle install

Verify:

bundle info ruby-openai

You should see where Bundler installed the gem.

Why use an SDK?

We could use Ruby’s Net::HTTP ourselves:

Ruby
  ↓
Net::HTTP
  ↓
HTTP request
  ↓
OpenAI

But then we’d have to manually handle:

  • authentication headers
  • JSON encoding
  • HTTP errors
  • response parsing
  • request formatting

The SDK gives us:

Ruby
  ↓
OpenAI Ruby SDK
  ↓
HTTP
  ↓
OpenAI

Important point: An SDK doesn’t eliminate the HTTP API. It is an abstraction over it.

Step 5.12 – Verify the gem

Run:

bin/rails console

Then:

require "openai"

It should return:

=> true

or possibly:

=> "openai"

depending on the gem’s load behavior.

Then:

OpenAI

should resolve without a NameError.

Exit:

exit

Step 5.13 – Let’s inspect the SDK before using it

This is something I want you to develop as a senior Ruby developer habit.

Instead of blindly copying code from a blog, let’s see what API the installed gem exposes.

Run:

bundle info ruby-openai

Then:

bin/rails console

Inside console:

require "openai"

Then:

OpenAI::Client.instance_method(:initialize).parameters

This tells us what the client’s constructor expects.

Also try:

OpenAI::Client.instance_methods(false)

We’re learning to inspect a Ruby library rather than treating it as magic.

Step 5.14 – Create the OpenAI client

Now let’s modify:

app/services/ai/client.rb

We’ll start with:

class Ai::Client
  def initialize
    @api_key = Rails.application.credentials.dig(:openai, :api_key)

    raise "OpenAI API key is missing" if @api_key.blank?

    @client = OpenAI::Client.new(api_key: @api_key)
  end
end

Now we have:

Ai::Client
   │
   ├── reads Rails credentials
   │
   └── creates OpenAI SDK client

Step 5.15 – Test initialization

Run:

bin/rails console

Then:

client = Ai::Client.new

It should return something similar to:

#<Ai::Client:0x...>

No request has happened yet.

That’s important.

We’ve only done:

Rails credentials
       ↓
API key
       ↓
OpenAI::Client

Stop here

Don’t call the LLM yet.

I want you to complete these steps first:

1. Gemfile

gem "openai"

2. Install

bundle install

3. Verify

bundle info ruby-openai

4. Update

app/services/ai/client.rb

with the code above.

5. Test

bin/rails c
client = Ai::Client.new

One note

The Ruby OpenAI SDK’s API can change between versions, so don’t blindly copy the exact request syntax from older tutorials. That’s why we’re checking the version we’ve actually installed before writing the API call.

Now we’ve:

“OpenAI client initialized.”

We’ll make our first actual LLM request and inspect the complete response, including:

response
model
output
usage
input tokens
output tokens

That will lead directly into why we added those fields to our Message model.

@client = OpenAI::Client.new(api_key: @api_key)

We’ll use our installed SDK’s API, not older ruby-openai examples. The current official openai Ruby SDK documents OpenAI::Client.new(api_key: ...) and the Responses API as the current interface. (GitHub)

Step 5.16 – Make the First Real LLM Request

For this step, we’ll do one simple request and inspect the response.

We are not integrating it with Conversation or Message yet.

Our goal is:

Rails console
    ↓
Ai::Client
    ↓
OpenAI Responses API
    ↓
LLM
    ↓
Response

1. Add a chat method

Open:

app/services/ai/client.rb

Change it to:

class Ai::Client
  def initialize
    @api_key = Rails.application.credentials.dig(:openai, :api_key)

    raise "OpenAI API key is missing" if @api_key.blank?

    @client = OpenAI::Client.new(api_key: @api_key)
  end

  def chat(message)
    @client.responses.create(
      model: "gpt-5.2",
      input: message
    )
  end
end

The SDK’s current Responses API accepts model and input for creating a response. (GitHub)

Why input: message?

We’re deliberately starting with the simplest possible request:

input: "Explain Ruby blocks in simple terms"

Later we’ll send structured conversation history:

input: [
  { role: :system, content: "..." },
  { role: :user, content: "..." }
]

The Responses API supports both simple input and structured message input. (GitHub)

2. Start Rails console

bin/rails c

Create the client:

client = Ai::Client.new

Now make the request:

response = client.chat(
  message: "Explain Ruby blocks in simple terms."
)

This is the moment our application makes an actual network request.

3. Inspect the response

First:

response.class

Then:

response

Don’t worry if the output is large.

The current official Ruby SDK returns typed response objects and the response contains the generated output plus metadata such as usage. (GitHub)

But if you get the following output, we can change the model which has free API calls:

ai-assistant(dev):013> client = Ai::Client.new
ai-assistant(dev):003> res = ai.chat('I want to be a expert in Ruby language')
app/services/ai/client.rb:11:in 'Ai::Client#chat': {url: "https://api.openai.com/v1/responses", status: 429, body: {error: {message: "You have no credits remaining. Add credits to continue using the API at https://platform.openai.com/settings/organization/billing/.", type: "insufficient_quota", param: nil, code: "credit_balance_exhausted"}}} (OpenAI::Errors::RateLimitError)
        from (ai-assistant):3:in '<compiled>'

4. Get the generated text

Try:

response.output_text

You should get a normal answer such as:

A Ruby block is a chunk of code that can be passed to a method...

This is the first important distinction:

response
    ↓
entire API response
response.output_text
    ↓
just the model's text

Don’t immediately throw away the full response. We need the metadata later.

5. Inspect the model

Try:

response.model

This tells you which model actually generated the response.

That’s relevant to our messages.model column.

6. Inspect usage

Now:

response.usage

You should see token-related information.

Inspect it:

response.usage.input_tokens

and:

response.usage.output_tokens

These are directly related to the fields we added earlier:

messages
-------------------
input_tokens
output_tokens

So our database design is now connected to a real API response.

LLM response
     │
     ├── model
     ├── output text
     └── usage
          ├── input_tokens
          └── output_tokens

The SDK’s response models expose usage information as part of the response. (GitHub)

7. One very important experiment

Ask a second question:

response2 = client.chat(
  message: "What is my name?"
)

You’ll probably notice the model doesn’t know your name from the previous request.

That’s intentional.

We made two independent requests:

Request 1
"Explain Ruby blocks"
Request 2
"What is my name?"

The LLM does not automatically receive our previous request.

This is going to become extremely important when we implement:

Conversation
    ↓
Messages
    ↓
Prompt Builder
    ↓
LLM

Our Rails application will be responsible for providing the appropriate conversation context.

8. One thing to notice

We’ve built:

app/services/ai/client.rb

and now:

Ai::Client.new.chat(...)

works.

That’s already a valuable architectural boundary:

Rails application
       │
       ▼
   Ai::Client
       │
       ▼
OpenAI SDK
       │
       ▼
OpenAI API

Our controllers won’t need to know:

  • how authentication works,
  • how the SDK works,
  • which API endpoint is used,
  • how OpenAI responses are represented.

That’s why we created the abstraction.

Stop Here

Run these commands one by one:

bin/rails c
client = Ai::Client.new
response = client.chat(
  message: "Explain Ruby blocks in simple terms."
)

Then inspect:

response.output_text
response.model
response.usage
response.usage.input_tokens
response.usage.output_tokens

Don’t paste our API key or any sensitive output anywhere.

Now: “First LLM request works.”

Then we’ll do the next important step: inspect the raw response structure and improve Ai::Client so it returns a clean Ruby object to the rest of our Rails application.

Unknown's avatar

Author: Abhilash

Hi, I’m Abhilash! A seasoned web developer with 15 years of experience specializing in Ruby and Ruby on Rails. Since 2010, I’ve built scalable, robust web applications and worked with frameworks like Angular, Sinatra, Laravel, Node.js, Vue and React. Passionate about clean, maintainable code and continuous learning, I share insights, tutorials, and experiences here. Let’s explore the ever-evolving world of web development together!

Leave a comment