Let’s move directly into RAG Part 2: embeddings.

One important correction before we code: the free model list you fetched earlier contains no free embedding model slug. OpenRouter currently lists liquid/lfm2.5-embedding-350m as a free embedding model, producing 1,024-dimensional vectors. OpenRouter’s embeddings API is OpenAI-compatible, so we can use the same Ruby SDK/base URL. (OpenRouter)

That means our existing vector(1536) column is the wrong dimension for the free embedding model we’ll use. We’ll fix that now.

RAG Part 2 – Ai::EmbeddingService

Our target architecture:

DocumentChunk
      │
      ▼
Ai::EmbeddingService
      │
      ▼
OpenRouter Embedding API
      │
      ▼
1024-dimensional vector
      │
      ▼
document_chunks.embedding

Then later:

User question
      ↓
Embedding
      ↓
pgvector similarity search
      ↓
Relevant chunks
      ↓
PromptBuilder
      ↓
LLM

Step 1 – Change the vector dimension

We originally created:

t.vector :embedding, limit: 1536

But our free model produces 1,024 dimensions.

Generate a migration:

bin/rails g migration ChangeDocumentChunkEmbeddingDimension

Open the migration and use:

class ChangeDocumentChunkEmbeddingDimension < ActiveRecord::Migration[8.1]
  def change
    remove_column :document_chunks, :embedding, type: :vector

    add_column :document_chunks, :embedding, :vector, limit: 1024
  end
end

Since our chunks don’t contain embeddings yet, removing and recreating the column is fine.

Run:

bin/rails db:migrate

Verify:

bin/rails dbconsole

\d document_chunks

You want:

embedding | vector(1024)

Then:

\q

Step 2 – Add the embedding model constant

Open:

app/services/ai/client.rb

Keep your existing chat models and add:

EMBEDDING_MODEL = "liquid/lfm2.5-embedding-350m:free"

So conceptually:

class Ai::Client
  MODELS = [
    "minimax/minimax-m3:free",
    "google/gemma-4-31b-it:free",
    "nvidia/nemotron-3-super-120b-a12b:free"
  ].freeze

  EMBEDDING_MODEL = "liquid/lfm2.5-embedding-350m"

  BASE_URL = "https://openrouter.ai/api/v1"

  # ...
end

Notice that this model is not a :free slug in the model ID you should send. OpenRouter currently lists this embedding model itself as free.

Check: https://openrouter.ai/models?output_modalities=embeddings

Step 3 – Add embeddings to Ai::Client

Add:

def embed(text:)
  response = @client.embeddings.create(
    model: EMBEDDING_MODEL,
    input: text
  )

  {
    embedding: response.data.first.embedding,
    model: response.model,
    input_tokens: response.usage&.prompt_tokens
  }
end

So your client now has two responsibilities:

chat()
embed()

Both communicate with the same OpenRouter endpoint, but use different models/endpoints. OpenRouter provides an OpenAI-compatible /embeddings API for this. (OpenRouter)

Step 4 – Test the raw embedding request

Open Rails console:

bin/rails c

Then:

client = Ai::Client.new

Now:

result = client.embed(
  text: "Ruby on Rails is a web application framework."
)

Inspect:

result.keys

You should get:

[:embedding, :model, :input_tokens]

Now:

result[:embedding].length

You should get:

1024

This is an important RAG checkpoint.

You’ve just proven:

text
 ↓
embedding model
 ↓
1024 numbers

Now inspect the first few values:

result[:embedding].first(5)

You’ll see floating-point numbers.

Don’t worry about the actual values. Their position in vector space is what matters.

Step 5 – Create Ai::EmbeddingService

Now we introduce the application-level service.

Create:

app/services/ai/embedding_service.rb

Use:

class Ai::EmbeddingService
  def initialize(ai_client: Ai::Client.new)
    @ai_client = ai_client
  end

  def call(text:)
    result = @ai_client.embed(text: text)

    result[:embedding]
  end
end

Why create another service when Ai::Client already has embed?

Because these are different responsibilities:

Ai::Client

How do I communicate with OpenRouter?

Ai::EmbeddingService

How does our application generate an embedding?

That distinction becomes useful once we introduce:

• chunking
• batch embeddings
• document indexing
• retries
• persistence

Step 6 – Generate an embedding for a real chunk

We already created our Ruby Guide document.

Open console:

bin/rails c

Then:

chunk = DocumentChunk.first

Check:

chunk.content

Now:

embedding = Ai::EmbeddingService.new.call(
  text: chunk.content
)

Verify:

embedding.length

Expected:

1024

Step 7 – Save the vector

Now:

chunk.update!(embedding: embedding)

Then:

chunk.reload

And:

chunk.embedding.length

You should get:

1024

We now have our first actual vector stored in PostgreSQL.

Error: I cannot update embedding vector column with Ruby Array embedding data

I have tested to storing the embedding. But it seems to be Rails does not know / there is a Type mismatch for embedding ruby array data and db vector data type

➜  ai_assistant git:(main) ✗ rails c
Loading development environment (Rails 8.1.3.1)
ai-assistant(dev):001> chunk = DocumentChunk.first

embedding = Ai::EmbeddingService.new.call(
  text: chunk.content
)

> chunk.update!(embedding: embedding)
(ai-assistant):5:in '<compiled>': can't quote Array (TypeError)

          raise TypeError, "can't quote #{value.class.name}"
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Check the solution here: https://railsdrop.com/update-embedding-vector-column-with-ruby-array-embedding-data-from-llm/

Step 8 – Embed all three chunks

We currently have:

Chunk 1 → Ruby blocks
Chunk 2 → Ruby modules
Chunk 3 → Ruby classes

Run:

service = Ai::EmbeddingService.new

Then:

DocumentChunk.find_each do |chunk|
  chunk.update!(
    embedding: service.call(text: chunk.content)
  )
end

Now:

DocumentChunk.where(embedding: nil).count

should return:

0

And:

DocumentChunk.count

should return:

3

Step 9 – Verify directly in PostgreSQL

Run:

bin/rails dbconsole

Then:

SELECT
  id,
  chunk_index,
  vector_dims(embedding)
FROM document_chunks;

Expected:

 id | chunk_index | vector_dims
----+-------------+------------
 1  | 0           | 1024
 2  | 1           | 1024
 3  | 2           | 1024

This is a very useful RAG sanity check.

Step 10 – Now perform our FIRST semantic search

This is the exciting part.

Take a query:

"What allows Ruby code to be reused?"

Generate its embedding:

query_embedding = service.call(
  text: "What allows Ruby code to be reused?"
)

Now we need PostgreSQL to compare that vector against all the chunk vectors.

pgvector provides operators including cosine distance (<=>) and inner product; cosine distance is a common choice for semantic search. (OpenRouter)

Run this in Rails console:

results = DocumentChunk
  .where.not(embedding: nil)
  .order(
    Arel.sql(
      "embedding <=> '#{query_embedding}'"
    )
  )
  .limit(3)

Why we’re stopping at this exact point

We’ve now completed the embedding generation side:

Document
   ↓
Chunk
   ↓
EmbeddingService
   ↓
OpenRouter
   ↓
1024-d vector
   ↓
PostgreSQL

The next piece is the actual retrieval:

Question
   ↓
Query embedding
   ↓
pgvector
   ↓
ORDER BY cosine distance
   ↓
Top K chunks

That is the point where RAG becomes real.

Then we’ll build Ai::VectorSearchService and make the first semantic search against PostgreSQL – the most important practical RAG step after embeddings.

to be continued ..