RAG Fails Quietly
Hallucinations often come from bad retrieval, not a bad model. If the wrong chunk is fetched, the LLM will confidently answer the wrong question.
Chunking and Metadata
Align chunk size with how users ask questions. Attach metadata — product, locale, document type, freshness — and filter before similarity search.
Evaluate Continuously
Maintain a golden set of questions with expected citations. Score retrieval hit rate and answer faithfulness on every pipeline change.
Guardrails
Refuse when confidence is low, cite sources in the UI, and log prompts/retrievals for debugging. Production RAG is an observability problem as much as an ML problem.