Ranjith Vutnoor

Enterprise RAG and semantic search

A production-oriented retrieval system for complex technical documentation, designed around relevance, evidence sufficiency, citations and measurable quality.

Role
AI/ML engineering
Focus
Retrieval + grounding
Stack
Python · Qdrant · LLMs
Status
Production work

Context

Technical documentation is high-volume, deeply nested and sensitive to product version, configuration and terminology. A useful assistant cannot rely on a single nearest-neighbour search followed by a model call. It needs a retrieval system that produces sufficient, traceable evidence under real production constraints.

Public-scope noteThis case study describes my engineering responsibilities at a pattern level. Client-specific names, source documents, internal prompts, infrastructure details and proprietary code are intentionally omitted.

System architecture

The retrieval path was designed as a sequence of independently measurable stages:

1 · UnderstandClassify the question and identify product, feature and intent.
2 · ExpandGenerate multiple search formulations to improve recall.
3 · RetrieveSearch vector and lexical indexes with metadata constraints.
4 · RerankPrioritise evidence using semantic relevance.
5 · AnswerGenerate a grounded response with source traceability.

Document ingestion

I worked on ingestion paths that preserved document structure, stable identifiers and source URLs while processing headings, body text, tables, figures and code blocks. Incremental metadata comparisons allowed changed topics to be updated without rebuilding the complete vector collection.

Retrieval and reranking

The search flow combined dense embeddings, multiple expanded queries, candidate aggregation, deduplication and reranking. Retrieval thresholds and candidate counts were treated as tunable system parameters rather than fixed assumptions.

Reflective retrieval

A sufficiency check evaluated whether the retrieved context could support a complete answer. When evidence was weak, the workflow generated alternative search phrases and performed another retrieval pass instead of producing a premature response.

Evaluation

Retrieval quality was evaluated independently from answer style. This helps distinguish a search failure from a generation failure.

  • MRR for the position of the first relevant result.
  • NDCG for graded relevance across the ranked list.
  • Hit rate for whether relevant evidence appeared within the retrieval window.
  • Human relevance labels to build a reusable evaluation set.
  • Golden-set regression to detect quality changes across model, prompt and retrieval updates.

Reliability engineering

Production quality also required citation validation, structured-output handling, retries, failure fallbacks, database migrations, observability and repeatability checks. These controls made model changes reviewable and reduced the risk of silent quality regressions.

What I learned

  • Retrieval improvements are often more valuable than adding complexity to the final prompt.
  • Evaluation datasets become product infrastructure: they should be versioned, reviewed and expanded from real failures.
  • Source traceability must be designed into ingestion and retrieval—not added at the end.
  • Model migrations need quality, latency, cost and reliability checks, not only syntax changes.
Next case study

EvalStudio — LLM quality workspace