Tag: semantic search

Integrate AI with Rails: Day 11 – RAG : Build Semantic Search

Step 13.3 – Build Semantic Search

We now have:

Document
  ↓
DocumentChunk
  ↓
EmbeddingService
  ↓
OpenRouter embedding model
  ↓
vector(1024)
  ↓
PostgreSQL

Now we need the retrieval side:

Question
   ↓
EmbeddingService
   ↓
query vector
   ↓
pgvector
   ↓
nearest chunks

pgvector’s cosine-distance operator is <=>; cosine similarity is 1 - cosine_distance. (GitHub)

Because we’re already using Neighbor, we’ll use its ActiveRecord integration rather than constructing SQL manually.

1. Add has_neighbors

Open:

app/models/document_chunk.rb

It should have:

class DocumentChunk < ApplicationRecord
  belongs_to :document

  has_neighbors :embedding

  validates :content, presence: true
  validates :chunk_index, presence: true
end

You’ve already added this while fixing vector persistence, so just verify it exists.

2. Create Ai::VectorSearchService

Create:

app/services/ai/vector_search_service.rb

Use:

class Ai::VectorSearchService
  DEFAULT_LIMIT = 1

  def initialize(embedding_service: Ai::EmbeddingService.new)
    @embedding_service = embedding_service
  end

  def call(query:, limit: DEFAULT_LIMIT)
    embedding = @embedding_service.call(text: query)

    DocumentChunk.has_embedding
                 .nearest_neighbors(:embedding, embedding, distance: "cosine")
                 .limit(limit)
  end
end


class DocumentChunk < ApplicationRecord
  .....

  scope :has_embedding, -> { where.not(embedding: nil) }
end

The important piece is:

.nearest_neighbors(
  :embedding,
  embedding,
  distance: "cosine"
)

Conceptually, that becomes a pgvector nearest-neighbor query using cosine distance. pgvector supports cosine distance through <=>.

3. Test semantic search

You already have three chunks:

Chunk 1
Ruby blocks are chunks of code passed to methods.
Chunk 2
Ruby modules allow code to be organized and reused.
Chunk 3
Ruby classes define objects and their behavior.

Let’s test with a query that doesn’t use the exact wording from the second chunk.

Run:

bin/rails c

Then:

search = Ai::VectorSearchService.new

Now:

results = search.call(
  query: "How can I reuse code in Ruby?"
)

Inspect:

results.map(&:content)

You should ideally see the modules chunk near the top:

"Ruby modules allow code to be organized and reused."

That’s our first semantic retrieval.

4. See the ranking

I want you to see why the result was selected.

Ask for the distance:

results.map do |chunk|
  {
    id: chunk.id,
    content: chunk.content,
    distance: chunk.neighbor_distance
  }
end

Depending on your Neighbor version, the distance accessor may be exposed differently. If neighbor_distance isn’t available, don’t spend time debugging it yet; the returned ordering is the important part for this checkpoint.

The conceptual result is:

Chunk 2   distance 0.18   ← best
Chunk 1   distance 0.62
Chunk 3   distance 0.71

For cosine distance:

smaller distance = more similar

and:

cosine similarity = 1 - distance

So a distance of 0.18 corresponds to similarity 0.82.

5. Why this is semantic search

Our question:

How can I reuse code in Ruby?

The document says:

Ruby modules allow code to be organized and reused.

There isn’t necessarily a literal phrase match for:

"How can I reuse code"

Yet the embedding vectors are close enough for the chunk to rank highly.

That’s the difference:

Keyword search
"reuse code"
        ↓
exact words

versus:

Semantic search
"reuse code"
        ↓
meaning
        ↓
embedding
        ↓
vector similarity

This reads nicely.

6. The RAG pipeline now has two halves

We have completed:

Indexing

Document
   ↓
Chunk
   ↓
Embedding
   ↓
Vector
   ↓
PostgreSQL

Retrieval

Question
   ↓
Embedding
   ↓
Vector similarity
   ↓
Top-K chunks

Put them together:

             INDEXING
                 │
                 ▼
Document → Chunks → Embeddings → pgvector
                                      ▲
                                      │
                                  similarity
                                      │
Question → Embedding ─────────────────┘
                                      │
                                      ▼
                                  Top chunks

That is the core of RAG.

7. Add a simple test

Create:

test/services/ai/vector_search_service_test.rb

A basic test can use a fake embedding service, because we don’t want every test to call the embedding API.

require "test_helper"

class Ai::VectorSearchServiceTest < ActiveSupport::TestCase
  test "returns nearest document chunks" do
    document = Document.create!(title: "Ruby Guide", source: "test")

    document.document_chunks.create!(
      content: "Ruby blocks are passed to methods.",
      chunk_index: 0,
      embedding: Array.new(1024, 0.1)
    )

    document.document_chunks.create!(
      content: "Ruby modules allow code reuse.",
      chunk_index: 1,
      embedding: Array.new(1024, 0.2)
    )

    fake_embedding_service = Minitest::Mock.new

    fake_embedding_service.expect(
      :call,
      Array.new(1024, 0.2),
      text: "How do I reuse Ruby code?"
    )

    service = Ai::VectorSearchService.new(
      embedding_service: fake_embedding_service
    )

    results = service.call(
      query: "How do I reuse Ruby code?",
      limit: 1
    )

    assert_equal 1, results.size
    assert_equal "Ruby modules allow code reuse.", results.first.content

    fake_embedding_service.verify
  end
end

Because our vectors are artificial, this test is mainly verifying the service’s wiring. For higher-confidence semantic-search tests, we’d later use controlled fixtures or a small integration test.

Next – The Actual RAG Answer

We’re now one step away from having a real RAG feature.

Currently:

Question
 ↓
VectorSearchService
 ↓
Relevant chunks

Next we’ll do:

Question
 ↓
VectorSearchService
 ↓
Top 5 chunks
 ↓
PromptBuilder
 ↓
LLM
 ↓
Answer grounded in document

We’ll modify Ai::PromptBuilder so it can accept retrieved context and implement:

Ai::RagService

That will be the point where our Rails app goes from “I can search vectors” to “I have built a RAG application.”

to be continued ..