Let’s move directly into RAG Part 2: embeddings.
One important correction before we code: the free model list you fetched earlier contains no free embedding model slug. OpenRouter currently lists liquid/lfm2.5-embedding-350m as a free embedding model, producing 1,024-dimensional vectors. OpenRouter’s embeddings API is OpenAI-compatible, so we can use the same Ruby SDK/base URL. (OpenRouter)
That means our existing vector(1536) column is the wrong dimension for the free embedding model we’ll use. We’ll fix that now.
RAG Part 2 – Ai::EmbeddingService
Our target architecture:
DocumentChunk
│
▼
Ai::EmbeddingService
│
▼
OpenRouter Embedding API
│
▼
1024-dimensional vector
│
▼
document_chunks.embedding
Then later:
User question
↓
Embedding
↓
pgvector similarity search
↓
Relevant chunks
↓
PromptBuilder
↓
LLM
Step 1 – Change the vector dimension
We originally created:
t.vector :embedding, limit: 1536
But our free model produces 1,024 dimensions.
Generate a migration:
bin/rails g migration ChangeDocumentChunkEmbeddingDimension
Open the migration and use:
class ChangeDocumentChunkEmbeddingDimension < ActiveRecord::Migration[8.1]
def change
remove_column :document_chunks, :embedding, type: :vector
add_column :document_chunks, :embedding, :vector, limit: 1024
end
end
Since our chunks don’t contain embeddings yet, removing and recreating the column is fine.
Run:
bin/rails db:migrate
Verify:
bin/rails dbconsole
\d document_chunks
You want:
embedding | vector(1024)
Then:
\q
Step 2 – Add the embedding model constant
Open:
app/services/ai/client.rb
Keep your existing chat models and add:
EMBEDDING_MODEL = "liquid/lfm2.5-embedding-350m:free"
So conceptually:
class Ai::Client
MODELS = [
"minimax/minimax-m3:free",
"google/gemma-4-31b-it:free",
"nvidia/nemotron-3-super-120b-a12b:free"
].freeze
EMBEDDING_MODEL = "liquid/lfm2.5-embedding-350m"
BASE_URL = "https://openrouter.ai/api/v1"
# ...
end
Notice that this model is not a :free slug in the model ID you should send. OpenRouter currently lists this embedding model itself as free.
Check: https://openrouter.ai/models?output_modalities=embeddings
Step 3 – Add embeddings to Ai::Client
Add:
def embed(text:)
response = @client.embeddings.create(
model: EMBEDDING_MODEL,
input: text
)
{
embedding: response.data.first.embedding,
model: response.model,
input_tokens: response.usage&.prompt_tokens
}
end
So your client now has two responsibilities:
chat()embed()
Both communicate with the same OpenRouter endpoint, but use different models/endpoints. OpenRouter provides an OpenAI-compatible /embeddings API for this. (OpenRouter)
Step 4 – Test the raw embedding request
Open Rails console:
bin/rails c
Then:
client = Ai::Client.new
Now:
result = client.embed(
text: "Ruby on Rails is a web application framework."
)
Inspect:
result.keys
You should get:
[:embedding, :model, :input_tokens]
Now:
result[:embedding].length
You should get:
1024
This is an important RAG checkpoint.
You’ve just proven:
text ↓embedding model ↓1024 numbers
Now inspect the first few values:
result[:embedding].first(5)
You’ll see floating-point numbers.
Don’t worry about the actual values. Their position in vector space is what matters.
Step 5 – Create Ai::EmbeddingService
Now we introduce the application-level service.
Create:
app/services/ai/embedding_service.rb
Use:
class Ai::EmbeddingService
def initialize(ai_client: Ai::Client.new)
@ai_client = ai_client
end
def call(text:)
result = @ai_client.embed(text: text)
result[:embedding]
end
end
Why create another service when Ai::Client already has embed?
Because these are different responsibilities:
Ai::Client
How do I communicate with OpenRouter?
Ai::EmbeddingService
How does our application generate an embedding?
That distinction becomes useful once we introduce:
- chunking
- batch embeddings
- document indexing
- retries
- persistence
Step 6 – Generate an embedding for a real chunk
We already created our Ruby Guide document.
Open console:
bin/rails c
Then:
chunk = DocumentChunk.first
Check:
chunk.content
Now:
embedding = Ai::EmbeddingService.new.call(
text: chunk.content
)
Verify:
embedding.length
Expected:
1024
Step 7 – Save the vector
Now:
chunk.update!(embedding: embedding)
Then:
chunk.reload
And:
chunk.embedding.length
You should get:
1024
We now have our first actual vector stored in PostgreSQL.
Error: I cannot update embedding vector column with Ruby Array embedding data
I have tested to storing the embedding. But it seems to be Rails does not know / there is a Type mismatch for embedding ruby array data and db vector data type
➜ ai_assistant git:(main) ✗ rails c
Loading development environment (Rails 8.1.3.1)
ai-assistant(dev):001> chunk = DocumentChunk.first
embedding = Ai::EmbeddingService.new.call(
text: chunk.content
)
> chunk.update!(embedding: embedding)
(ai-assistant):5:in '<compiled>': can't quote Array (TypeError)
raise TypeError, "can't quote #{value.class.name}"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Check the solution here: https://railsdrop.com/update-embedding-vector-column-with-ruby-array-embedding-data-from-llm/
Step 8 – Embed all three chunks
We currently have:
Chunk 1 → Ruby blocksChunk 2 → Ruby modulesChunk 3 → Ruby classes
Run:
service = Ai::EmbeddingService.new
Then:
DocumentChunk.find_each do |chunk|
chunk.update!(
embedding: service.call(text: chunk.content)
)
end
Now:
DocumentChunk.where(embedding: nil).count
should return:
0
And:
DocumentChunk.count
should return:
3
Step 9 – Verify directly in PostgreSQL
Run:
bin/rails dbconsole
Then:
SELECT
id,
chunk_index,
vector_dims(embedding)
FROM document_chunks;
Expected:
id | chunk_index | vector_dims
----+-------------+------------
1 | 0 | 1024
2 | 1 | 1024
3 | 2 | 1024
This is a very useful RAG sanity check.
Step 10 – Now perform our FIRST semantic search
This is the exciting part.
Take a query:
"What allows Ruby code to be reused?"
Generate its embedding:
query_embedding = service.call(
text: "What allows Ruby code to be reused?"
)
Now we need PostgreSQL to compare that vector against all the chunk vectors.
pgvector provides operators including cosine distance (<=>) and inner product; cosine distance is a common choice for semantic search. (OpenRouter)
Run this in Rails console:
results = DocumentChunk
.where.not(embedding: nil)
.order(
Arel.sql(
"embedding <=> '#{query_embedding}'"
)
)
.limit(3)
Why we’re stopping at this exact point
We’ve now completed the embedding generation side:
Document
↓
Chunk
↓
EmbeddingService
↓
OpenRouter
↓
1024-d vector
↓
PostgreSQL
The next piece is the actual retrieval:
Question
↓
Query embedding
↓
pgvector
↓
ORDER BY cosine distance
↓
Top K chunks
That is the point where RAG becomes real.
Then we’ll build Ai::VectorSearchService and make the first semantic search against PostgreSQL – the most important practical RAG step after embeddings.
to be continued ..
The Rails Drop 