For a Senior Rails developer, simply knowing what RAG, LLM, Agents, and embeddings mean is not enough. In an int., you may be asked:

“Okay, let’s build an AI feature in Rails. How would you structure it?”

You should be able to open your laptop and actually build one.

So let’s turn Day 4 into a hands-on mini-project in this blog that we’ll build incrementally. We won’t rush through the whole application in one answer.

Build an AI Application with Ruby on Rails

Project: AI Chat Assistant

We’re going to build a real Rails application that evolves throughout this course.

The final architecture will look approximately like this:

                         ┌──────────────────┐
                         │     Browser      │
                         │   Chat UI        │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Rails Controller │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Chat Service     │
                         └────────┬─────────┘
                                  │
                     ┌────────────┴────────────┐
                     ▼                         ▼
              Conversation              Prompt Builder
                  DB                         │
                                            ▼
                                   ┌──────────────────┐
                                   │   AI Client      │
                                   └────────┬─────────┘
                                            │
                                            ▼
                                   ┌──────────────────┐
                                   │      LLM         │
                                   │ OpenAI / Claude  │
                                   └────────┬─────────┘
                                            │
                                            ▼
                                   Response Formatter
                                            │
                                            ▼
                                      Rails / Browser

And later we’ll evolve it into:

                         AI Rails Application
                                  │
       ┌──────────────────────────┼──────────────────────────┐
       │                          │                          │
       ▼                          ▼                          ▼
     Chat                       RAG                       Agents
       │                          │                          │
       ▼                          ▼                          ▼
   LLM API                   pgvector                    Tools
                                  │                          │
                                  ▼                          ▼
                             Documents                 Business APIs

That will give you practical experience across LLM → RAG → Agents.

What We Are Going to Build

Our application will start simple.

Version 1

User
 ↓
Rails
 ↓
LLM API
 ↓
Response

Then we’ll progressively add:

Version 2

Conversation
 ├── User message
 ├── Assistant response
 ├── User message
 └── Assistant response

Version 3

Streaming:

LLM
 ↓
token
 ↓
token
 ↓
token
 ↓
Browser

Version 4

Production architecture:

Controller
    ↓
Chat Service
    ↓
Prompt Builder
    ↓
AI Client
    ↓
Provider

Version 5

RAG:

Question
   ↓
Embedding
   ↓
pgvector
   ↓
Relevant Documents
   ↓
Prompt
   ↓
LLM

Version 6

Agent:

User
 ↓
Agent
 ├── Search Product
 ├── Find Order
 ├── Search Documentation
 └── Create Support Ticket

This is why I recommend we build one application throughout the AI bootcamp, rather than writing isolated examples.

Practical Course Roadmap

We’ll divide the practical Day 4 into 10 stages.

Then Day 5 can build on this application to introduce agents.

Stage 1 – Create the Rails Application

We’ll use:

• Ruby
• Rails
• PostgreSQL
• OpenAI API initially
• RSpec/Minitest depending on your preference
• dotenv/credentials for secrets
• Turbo/Stimulus where useful

The important thing is:

We won’t use a huge AI framework initially.

I want you to understand what is actually happening underneath.

Later we can compare this approach with Ruby AI libraries/frameworks.

Step 1 – Create Rails App

Assuming Rails is installed:

rails new ai_assistant -d postgresql

Move into the application:

cd ai_assistant

Create database:

bin/rails db:create

Run it:

bin/rails server

Then open:

http://localhost:3000

At this point:

Browser
   ↓
Rails
   ↓
PostgreSQL

works.

No AI yet.

Why Start This Way?

This is important for ints.

We don’t want to hide everything behind an AI gem.

You need to understand:

HTTP Request
     ↓
Rails
     ↓
Ruby
     ↓
HTTP Client
     ↓
AI Provider

Once you understand this, an SDK becomes just an abstraction.

Stage 2 – Configure AI Credentials

Never do this:

api_key = "sk-xxxxx"

Never commit API keys to Git.

We’ll use Rails credentials or environment variables.

Conceptually:

Rails Application
       │
       ▼
Configuration
       │
       ▼
OPENAI_API_KEY

For local development, we’ll configure the key securely.

Stage 3- Make Your First LLM Request

This is our first major milestone.

We’ll create:

app/
└── services/
    └── ai/
        └── client.rb

Initially:

class Ai::Client
  def initialize
    ...
  end
  def chat(messages:)
    ...
  end
end

Then:

client = Ai::Client.new
response = client.chat(
  messages: [
    {
      role: "user",
      content: "Explain Ruby blocks in simple terms"
    }
  ]
)

And eventually:

Ruby
 ↓
Ai::Client
 ↓
OpenAI API
 ↓
LLM
 ↓
JSON Response
 ↓
Ruby

This is the most important practical exercise of Day 4.

You will see exactly what an LLM API actually returns.

Stage 4 – Understand the Raw API Response

We’re not immediately going to hide the response.

We’ll inspect things like:

response
├── id
├── model
├── choices
│   └── message
│       ├── role
│       └── content
└── usage
    ├── input tokens
    └── output tokens

This connects directly with Day 1.

Remember:

Tokens
 ↓
Cost
 ↓
Latency
 ↓
Context

You’ll actually see token usage in a real application.

Stage 5 – Build the Rails Chat Application

Now we’ll create:

User
 ↓
Chat page
 ↓
POST /conversations/:id/messages
 ↓
Rails Controller
 ↓
AI Service
 ↓
LLM
 ↓
Response
 ↓
Browser

We’ll create models such as:

User
Conversation
Message

A conversation:

Conversation
    │
    ├── Message
    │      role: user
    │      content: "What is Ruby?"
    │
    ├── Message
    │      role: assistant
    │      content: "Ruby is..."
    │
    ├── Message
    │      role: user
    │      content: "Who created it?"
    │
    └── Message
           role: assistant
           content: "Yukihiro Matsumoto..."

Stage 6 – Database Design

We’ll design this properly rather than putting everything into one table.

For example:

conversations
-----------------
id
user_id
title
created_at
updated_at

and:

messages
-----------------
id
conversation_id
role
content
model
input_tokens
output_tokens
created_at

Potentially later:

total_tokens
latency_ms
finish_reason

Now you’re thinking like a senior engineer.

Stage 7 – Build Conversation Context

This is where you’ll see something very important.

The LLM doesn’t automatically remember our database conversation.

If we have:

User:
My name is Abhilash.
Assistant:
Nice to meet you.
User:
What's my name?

Rails must send appropriate history back to the LLM:

[
  {
    role: "user",
    content: "My name is Abhi."
  },
  {
    role: "assistant",
    content: "Nice to meet you."
  },
  {
    role: "user",
    content: "What's my name?"
  }
]

Therefore:

Your Rails application manages conversation memory.

This is a very important int. concept.

Stage 8 – Prompt Builder

Eventually we don’t want:

messages = [
  ...
]

scattered everywhere.

We’ll create:

Ai::PromptBuilder

Architecture:

Conversation
     ↓
Prompt Builder
     ↓
System Prompt
     +
Conversation History
     +
Current User Message
     ↓
LLM

For example:

Ai::PromptBuilder.new(
  conversation: conversation,
  user_message: message
).build

This is where your Rails architecture skills become important.

Stage 9 – Streaming

After normal request/response works, we’ll make it feel like ChatGPT.

Instead of:

User
 ↓
 [wait 5 seconds]
 
 ↓ Complete response

we’ll have:

User
 ↓
Rails
 ↓
LLM
 ↓
"Ruby"
 ↓
" is"
 ↓
" a"
 ↓
" programming"
 ↓
" language"

The browser updates progressively.

We’ll investigate Rails approaches such as:

SSE
Turbo Streams
Action Cable

And we’ll discuss when each is appropriate.

Stage 10 – Production Concerns

Then we’ll deliberately break our application.

We’ll simulate:

LLM timeout
LLM rate limit
Invalid response
API unavailable
Malformed JSON

We’ll build:

Ai::Client
     │
     ├── timeout
     ├── retry
     ├── rate limit
     └── provider error

We’ll also add:

Authentication
Authorization
Rate limiting
Logging
Token tracking
Cost tracking

This is where my 15 years of backend experience can help.

Stage 11 – Testing

We’ll write tests around:

Ai::Client

Does it call the provider?
Does it handle errors?
Does it parse the response?

Ai::PromptBuilder

Does it create the correct messages?
Does it include conversation history?

Ai::ChatService

Does it save the user message?
Does it call the AI?
Does it save the response?

We’ll mock the external AI service.

The tests should not depend on a live LLM API.

Final Day 4 Application

At the end of the practical course, you’ll have something approximately like:

                         Browser
                            │
                            ▼
                    ┌───────────────┐
                    │ Chat UI       │
                    └───────┬───────┘
                            │
                            ▼
                    ┌───────────────┐
                    │ Controller    │
                    └───────┬───────┘
                            │
                            ▼
                    ┌───────────────┐
                    │ Chat Service  │
                    └───────┬───────┘
                            │
                  ┌─────────┴─────────┐
                  ▼                   ▼
          ┌──────────────┐    ┌──────────────┐
          │ Conversation │    │Prompt Builder│
          │ PostgreSQL   │    └──────┬───────┘
          └──────────────┘           │
                                     ▼
                              ┌──────────────┐
                              │  AI Client   │
                              └──────┬───────┘
                                     │
                                     ▼
                              ┌──────────────┐
                              │     LLM      │
                              └──────┬───────┘
                                     │
                                     ▼
                              Response
                                     │
                                     ▼
                               Browser

But We Won’t Stop There

This application will become our AI laboratory for the remaining bootcamp.

Day 5

We’ll add:

AI Agent
   │
   ├── Product Search Tool
   ├── Order Lookup Tool
   └── Documentation Search Tool

Day 6

We’ll add:

RAG
Documents
   ↓
Chunks
   ↓
Embeddings
   ↓
pgvector
   ↓
Semantic Search
   ↓
LLM

and discuss production concerns.

Day 7

We’ll turn everything into:

                         AI Rails Application
                                  │
              ┌───────────────────┼───────────────────┐
              │                   │                   │
              ▼                   ▼                   ▼
             LLM                 RAG                Agent
              │                   │                   │
              ▼                   ▼                   ▼
         Prompting            pgvector             Tools
              │                   │                   │
              └───────────────────┼───────────────────┘
                                  ▼
                           Production System

Then we’ll do AI int. questions + AI system design + coding exercises.

How I Suggest We Learn Each Stage

This is important.

I don’t want to dump 500 lines of Rails code on you.

For every stage, we’ll follow:

1. Understand

I’ll explain:

What are we building?

2. Why

Why do we need this architecture?

3. Build

We’ll create the files/commands one at a time.

4. Run

You’ll execute it on your Mac.

5. Inspect

We’ll look at:

Rails logs
HTTP requests
API responses
PostgreSQL records
LLM responses
Token usage

6. Break it

I’ll give you scenarios such as:

What happens if the AI provider times out?

You solve it.

7. Questions

We’ll turn what you just built into question / answers.

This is particularly important for you because your objective isn’t merely to build an AI app – it’s to be able to explain the architecture during a senior-level discussions.

Our Practical Course

So I suggest we proceed in this exact order:

DAY 4 PRACTICAL
Part 1
│
├── 1. Create Rails app
│
├── 2. Configure AI API credentials
│
├── 3. Make first LLM API request
│
├── 4. Inspect raw response
│
├── 5. Build Ai::Client
│
├── 6. Build Chat UI
│
├── 7. Conversation + Message models
│
├── 8. Build Ai::ChatService
│
├── 9. Build PromptBuilder
│
├── 10. Conversation memory
│
├── 11. Streaming
│
├── 12. Error handling
│
├── 13. Token/cost tracking
│
├── 14. Security
│
├── 15. Tests
│
└── 16. Production architecture

Start Here: Part 1

Your first assignment is simply to create the application.

On your Mac:

ruby -v
rails -v
psql --version

Then:

rails new ai_assistant -d postgresql
cd ai_assistant
bin/rails db:create
bin/rails server

Verify:

http://localhost:3000

Once that works, don’t start building anything else yet.

Since we’ve already created:

app/services/ai/client.rb

we’ll now build the database layer.

For now, don’t create all models at once. We’ll create one model, migrate it, inspect the database, understand why we designed it this way, and only then move to the next model.

Part 1 – Create Conversation

Our AI application needs to remember conversations.

Think of ChatGPT:

Conversation
    │
    ├── User message
    ├── AI response
    ├── User message
    └── AI response

So we’ll have two main models:

Conversation
     │
     └── has_many :messages

and later:

Message
     │
     └── belongs_to :conversation

For the moment, we’ll create only Conversation.

Step 1 – Check your current directory

From your Rails application’s root:

pwd

You should be somewhere like:

.../ai_assistant

Then:

ls

You should see something similar to:

Gemfile
Gemfile.lock
app
config
db
lib
public
...

If you’re already in your ai_assistant directory, continue.

Step 2 – Generate the Conversation model

Run:

bin/rails generate model Conversation title:string

You can also use:

bin/rails g model Conversation title:string

Both commands do the same thing.

Rails should generate something similar to:

invoke  active_record
create    db/migrate/XXXXXXXXXXXXXX_create_conversations.rb
create    app/models/conversation.rb

Step 3 – Understand what Rails created

Open:

app/models/conversation.rb

You’ll initially see:

class Conversation < ApplicationRecord
end

At this point, the model doesn’t have any associations.

That’s okay.

Step 4 – Inspect the migration

Open:

db/migrate/XXXXXXXXXXXXXX_create_conversations.rb

You’ll see something like:

class CreateConversations < ActiveRecord::Migration[8.1]
  def change
    create_table :conversations do |t|
      t.string :title
      t.timestamps
    end
  end
end

The exact Rails migration version will depend on your Rails version.

What does this mean?

Rails is asking PostgreSQL to create approximately:

conversations
-------------------------
id
title
created_at
updated_at

Step 5 – Run the migration

Now execute:

bin/rails db:migrate

You should see something similar to:

== ... CreateConversations: migrating =====================
-- create_table(:conversations)
   -> 0.00xxs
== ... CreateConversations: migrated ======================

Now the table exists in PostgreSQL.

Step 6 – Verify using Rails

Open Rails console:

bin/rails console

or:

bin/rails c

Then:

Conversation

You should get:

Conversation (call 'Conversation.connection' to establish a connection)

Now:

Conversation.column_names

You should see something similar to:

[
  "id",
  "title",
  "created_at",
  "updated_at"
]

This is a good habit for you as a senior Rails developer:

Don’t blindly trust generated migrations. Inspect what Rails actually created.

Step 7 – Create a Conversation

Still inside Rails console:

conversation = Conversation.create(title: "My first AI conversation")

You should get something like:

#<Conversation id: 1, title: "My first AI conversation", ...>

Now:

conversation.id

You should get:

1

And:

Conversation.all

should return your conversation.

Step 8 – Check PostgreSQL directly

This is particularly useful for your int. preparation because I want you to understand both Rails and the database underneath it.

Exit Rails console:

exit

Then connect to your database:

bin/rails dbconsole

You’ll enter psql.

Run:

\d conversations

You should see something approximately like:

Column       | Type
-------------+--------------------------
id           | bigint
title        | character varying
created_at   | timestamp
updated_at   | timestamp

Then:

SELECT * FROM conversations;

You should see your test conversation.

Exit:

\q

Why are we starting with Conversation?

Eventually our application will look like:

Conversation
     │
     │ has_many
     ▼
 Messages
     │
     ├── user
     ├── assistant
     ├── user
     └── assistant

For example:

Conversation #1
Title: Ruby Question
Message #1
role: user
content: "What is a Ruby block?"
Message #2
role: assistant
content: "A Ruby block is..."
Message #3
role: user
content: "Can you give me an example?"
Message #4
role: assistant
content: "Sure..."

The Conversation represents the container, while Message represents each individual interaction.

One Important Design Decision

You may notice that our earlier architecture discussed:

Conversation
    user_id
    title

We’re deliberately not adding user_id yet.

Why?

Your newly created Rails app may not have an authentication/User model yet.

We don’t want to introduce Devise/authentication just to learn AI.

We’ll first make the AI application work.

Later we can add:

User
  │
  └── has_many :conversations

That keeps today’s exercise focused.

Your Current State

You should now have:

app/
├── models/
│   └── conversation.rb
│
└── services/
    └── ai/
        └── client.rb
db/
└── migrate/
    └── XXXXX_create_conversations.rb

And PostgreSQL:

conversations
-------------------------
id
title
created_at
updated_at

Stop Here

Don’t create Message yet.

First execute these steps:

bin/rails g model Conversation title:string
bin/rails db:migrate
bin/rails c

Then inside Rails console:

Conversation.column_names

and:

conversation = Conversation.create(title: "My first AI conversation")

Then verify:

Conversation.all

Now Our “Conversation model is done.”

Next step: create the Message model and I’ll explain why role, content, model and token-related columns belong there.

Now I’ll create the Message model. This is the most important model in our AI chat application because it represents the actual conversation between the user and the LLM.

Step 2 – Create the Message model

Our structure will become:

Conversation
    │
    ├── Message
    ├── Message
    ├── Message
    └── Message

For example:

Conversation #1
│
├── User      → "What is Ruby?"
├── Assistant → "Ruby is a programming language..."
├── User      → "Who created it?"
└── Assistant → "Ruby was created by..."

The Message table needs to know:

• which conversation it belongs to
• who/what produced it (user or assistant)
• the actual message
• which AI model generated the response
• token usage, which we’ll use later for cost tracking

Step 1 – Generate the model

From your Rails application’s root directory:

bin/rails generate model Message \
  conversation:references \
  role:string \
  content:text \
  model:string \
  input_tokens:integer \
  output_tokens:integer

You can also write it as one line:

bin/rails g model Message conversation:references role:string content:text model:string input_tokens:integer output_tokens:integer

Rails should generate:

app/models/message.rb
db/migrate/XXXXXXXXXXXXXX_create_messages.rb

Step 2 – Inspect the generated migration

Open:

db/migrate/XXXXXXXXXXXXXX_create_messages.rb

You’ll see something similar to:

class CreateMessages < ActiveRecord::Migration[8.0]
  def change
    create_table :messages do |t|
      t.references :conversation, null: false, foreign_key: true
      t.string :role
      t.text :content
      t.string :model
      t.integer :input_tokens
      t.integer :output_tokens
      t.timestamps
    end
  end
end

Your Rails migration version may differ.

Understand conversation:references

This is important.

When we wrote:

conversation:references

Rails generated:

t.references :conversation, null: false, foreign_key: true

This creates:

conversation_id

in the messages table.

So our database relationship becomes:

conversations
----------------
id
title
messages
----------------
id
conversation_id
role
content
model
input_tokens
output_tokens
created_at
updated_at

The important connection is:

messages.conversation_id
            │
            ▼
conversations.id

That’s a standard relational database foreign key.

Step 3 – Run the migration

Execute:

bin/rails db:migrate

You should see something like:

== CreateMessages: migrating ===============================
-- create_table(:messages)
   -> ...
== CreateMessages: migrated ================================

Now PostgreSQL has the messages table.

Step 4 – Inspect PostgreSQL

Let’s verify what actually happened.

Run:

bin/rails dbconsole

Then:

\d messages

You should see something approximately like:

Column          | Type
----------------+--------------------------
id              | bigint
conversation_id | bigint
role            | character varying
content         | text
model           | character varying
input_tokens    | integer
output_tokens   | integer
created_at      | timestamp
updated_at      | timestamp

And importantly, you’ll see a foreign key from:

conversation_id

to:

conversations.id

You can also run:

SELECT * FROM messages;

Currently there should be no records.

Exit:

\q

Step 5 – Inspect the generated Rails model

Open:

app/models/message.rb

Rails should have generated:

class Message < ApplicationRecord
  belongs_to :conversation
end

Rails automatically added:

belongs_to :conversation

because we used:

conversation:references

Now we need the other side of the relationship.

Step 6 – Add has_many to Conversation

Open:

app/models/conversation.rb

Currently it probably looks like:

class Conversation < ApplicationRecord
end

Change it to:

class Conversation < ApplicationRecord
  has_many :messages, dependent: :destroy
end

Now our Rails relationship is:

Conversation
    │
    │ has_many
    ▼
Messages

and:

Message
    │
    │ belongs_to
    ▼
Conversation

Step 7 – Test the association

Open Rails console:

bin/rails console

First find your conversation:

conversation = Conversation.first

Then:

conversation.messages

It should return:

[]

because we haven’t created any messages yet.

Now create a user message:

message = conversation.messages.create(
  role: "user",
  content: "What is Ruby?"
)

Now:

message

You should get something similar to:

#<Message
 id: 1,
 conversation_id: 1,
 role: "user",
 content: "What is Ruby?",
 ...
>

Step 8 – Check the relationship

Now run:

conversation.messages

You should see your message.

And:

message.conversation

should return the conversation.

This demonstrates the two-way ActiveRecord relationship:

conversation.messages
        ↓
      Message
message.conversation
        ↓
    Conversation

Why do we need role?

This is extremely important for an AI application.

The LLM needs to distinguish between:

user
assistant
system

For example:

{
  role: "user",
  content: "What is Ruby?"
}

and:

{
  role: "assistant",
  content: "Ruby is a programming language..."
}

Later, Rails will retrieve these database records and transform them into the messages we send to the LLM.

So:

PostgreSQL
Message
role = "user"
content = "What is Ruby?"
        ↓
Rails transforms it
        ↓
LLM API
{
  role: "user",
  content: "What is Ruby?"
}

This is the bridge between our database and the LLM API.

Why model?

Suppose today we use one model:

some-current-model

Later we change to another model.

We want to know which model generated each response.

For example:

Message #1
model = model-A
Message #2
model = model-B

This becomes valuable for:

• debugging
• cost analysis
• performance analysis
• comparing models
• auditing

We don’t need to populate it for user messages.

Why input_tokens and output_tokens?

Remember Day 1?

Input tokens
      +
Output tokens
      =
Usage

Suppose an AI response used:

input_tokens  = 500
output_tokens = 200

We can store that information.

Later we can calculate:

How much did this conversation cost?
How much did this user cost?
Which model is expensive?
Which endpoint consumes the most tokens?

This is exactly the kind of thing you should do in a senior-level AI system design.

One thing we’re deliberately NOT doing yet

You may wonder:

Why don’t we add validations for role?

For example:

validates :role, inclusion: {
  in: %w[user assistant system]
}

We’re going to discuss this next.

There is an interesting design question here:

Should role be a Ruby enum?

For example:

enum :role, {
  user: "user",
  assistant: "assistant",
  system: "system"
}

We’ll discuss why a string enum is useful here, what gets stored in PostgreSQL, and what tradeoffs exist before finalising the model.

Our Current Database

After completing this step, you should have:

conversations
-------------------------
id
title
created_at
updated_at
        │
        │ 1 → many
        ▼
messages
-------------------------
id
conversation_id
role
content
model
input_tokens
output_tokens
created_at
updated_at

And Rails:

class Conversation < ApplicationRecord
  has_many :messages, dependent: :destroy
end

class Message < ApplicationRecord
  belongs_to :conversation
end

Stop Here

Please execute only these steps now:

bin/rails g model Message conversation:references role:string content:text model:string input_tokens:integer output_tokens:integer
bin/rails db:migrate
bin/rails console

Then test:

conversation = Conversation.first
message = conversation.messages.create(
  role: "user",
  content: "What is Ruby?"
)
conversation.messages
message.conversation

Now Our “Message model done.”

Next step: properly design Message.role and validations, and after that we’ll make our first real LLM API call through your Ai::Client.

to be continued..