Step 13.3 – Build Semantic Search
We now have:
Document
↓
DocumentChunk
↓
EmbeddingService
↓
OpenRouter embedding model
↓
vector(1024)
↓
PostgreSQL
Now we need the retrieval side:
Question
↓
EmbeddingService
↓
query vector
↓
pgvector
↓
nearest chunks
pgvector’s cosine-distance operator is <=>; cosine similarity is 1 - cosine_distance. (GitHub)
Because we’re already using Neighbor, we’ll use its ActiveRecord integration rather than constructing SQL manually.
1. Add has_neighbors
Open:
app/models/document_chunk.rb
It should have:
class DocumentChunk < ApplicationRecord
belongs_to :document
has_neighbors :embedding
validates :content, presence: true
validates :chunk_index, presence: true
end
You’ve already added this while fixing vector persistence, so just verify it exists.
2. Create Ai::VectorSearchService
Create:
app/services/ai/vector_search_service.rb
Use:
class Ai::VectorSearchService
DEFAULT_LIMIT = 1
def initialize(embedding_service: Ai::EmbeddingService.new)
@embedding_service = embedding_service
end
def call(query:, limit: DEFAULT_LIMIT)
embedding = @embedding_service.call(text: query)
DocumentChunk.has_embedding
.nearest_neighbors(:embedding, embedding, distance: "cosine")
.limit(limit)
end
end
class DocumentChunk < ApplicationRecord
.....
scope :has_embedding, -> { where.not(embedding: nil) }
end
The important piece is:
.nearest_neighbors(
:embedding,
embedding,
distance: "cosine"
)
Conceptually, that becomes a pgvector nearest-neighbor query using cosine distance. pgvector supports cosine distance through <=>.
3. Test semantic search
You already have three chunks:
Chunk 1Ruby blocks are chunks of code passed to methods.Chunk 2Ruby modules allow code to be organized and reused.Chunk 3Ruby classes define objects and their behavior.
Let’s test with a query that doesn’t use the exact wording from the second chunk.
Run:
bin/rails c
Then:
search = Ai::VectorSearchService.new
Now:
results = search.call( query: "How can I reuse code in Ruby?")
Inspect:
results.map(&:content)
You should ideally see the modules chunk near the top:
"Ruby modules allow code to be organized and reused."
That’s our first semantic retrieval.
4. See the ranking
I want you to see why the result was selected.
Ask for the distance:
results.map do |chunk|
{
id: chunk.id,
content: chunk.content,
distance: chunk.neighbor_distance
}
end
Depending on your Neighbor version, the distance accessor may be exposed differently. If neighbor_distance isn’t available, don’t spend time debugging it yet; the returned ordering is the important part for this checkpoint.
The conceptual result is:
Chunk 2 distance 0.18 ← best
Chunk 1 distance 0.62
Chunk 3 distance 0.71
For cosine distance:
smaller distance = more similar
and:
cosine similarity = 1 - distance
So a distance of 0.18 corresponds to similarity 0.82.
5. Why this is semantic search
Our question:
How can I reuse code in Ruby?
The document says:
Ruby modules allow code to be organized and reused.
There isn’t necessarily a literal phrase match for:
"How can I reuse code"
Yet the embedding vectors are close enough for the chunk to rank highly.
That’s the difference:
Keyword search"reuse code" ↓exact words
versus:
Semantic search"reuse code" ↓meaning ↓embedding ↓vector similarity
This reads nicely.
6. The RAG pipeline now has two halves
We have completed:
Indexing
Document ↓Chunk ↓Embedding ↓Vector ↓PostgreSQL
Retrieval
Question ↓Embedding ↓Vector similarity ↓Top-K chunks
Put them together:
INDEXING
│
▼
Document → Chunks → Embeddings → pgvector
▲
│
similarity
│
Question → Embedding ─────────────────┘
│
▼
Top chunks
That is the core of RAG.
7. Add a simple test
Create:
test/services/ai/vector_search_service_test.rb
A basic test can use a fake embedding service, because we don’t want every test to call the embedding API.
require "test_helper"
class Ai::VectorSearchServiceTest < ActiveSupport::TestCase
test "returns nearest document chunks" do
document = Document.create!(title: "Ruby Guide", source: "test")
document.document_chunks.create!(
content: "Ruby blocks are passed to methods.",
chunk_index: 0,
embedding: Array.new(1024, 0.1)
)
document.document_chunks.create!(
content: "Ruby modules allow code reuse.",
chunk_index: 1,
embedding: Array.new(1024, 0.2)
)
fake_embedding_service = Minitest::Mock.new
fake_embedding_service.expect(
:call,
Array.new(1024, 0.2),
text: "How do I reuse Ruby code?"
)
service = Ai::VectorSearchService.new(
embedding_service: fake_embedding_service
)
results = service.call(
query: "How do I reuse Ruby code?",
limit: 1
)
assert_equal 1, results.size
assert_equal "Ruby modules allow code reuse.", results.first.content
fake_embedding_service.verify
end
end
Because our vectors are artificial, this test is mainly verifying the service’s wiring. For higher-confidence semantic-search tests, we’d later use controlled fixtures or a small integration test.
Next – The Actual RAG Answer
We’re now one step away from having a real RAG feature.
Currently:
Question ↓VectorSearchService ↓Relevant chunks
Next we’ll do:
Question ↓VectorSearchService ↓Top 5 chunks ↓PromptBuilder ↓LLM ↓Answer grounded in document
We’ll modify Ai::PromptBuilder so it can accept retrieved context and implement:
Ai::RagService
That will be the point where our Rails app goes from “I can search vectors” to “I have built a RAG application.”
to be continued ..
The Rails Drop 