AuditRAG
- Period
- March 2025
- Status
- Shipped
- Role
- Founder and engineer
Hybrid search (dense + sparse + RRF), auto-generated eval datasets, and per-query latency/cost observability.
Production-grade Retrieval-Augmented Generation platform. Upload any PDF, chat with it using hybrid search (dense + sparse + RRF fusion), and get cited answers. The system auto-generates evaluation datasets on ingestion, runs quality harnesses (exact match, token F1), and tracks per-query observability — latency percentiles, token usage, and cost. Built with FastAPI, Qdrant, PostgreSQL, and a React + TypeScript frontend. Demonstrates the full RAG lifecycle: ingestion, retrieval, generation, evaluation, and monitoring.
A useful retrieval system needs more than a chat interface. Ingestion quality, retrieval behavior, citations, evaluation, latency, and cost all need to remain visible after deployment.
AuditRAG covers the complete retrieval lifecycle, from PDF ingestion through hybrid search and cited generation to automated evaluation and per-query observability.
How it works
Ingest
Parse, segment, and prepare uploaded documents.
Retrieve
Run dense and sparse search, then fuse the rankings.
Answer
Generate a response grounded in cited source material.
Evaluate
Measure answer quality and inspect operational signals.
Key decisions
Combine retrieval signals
Dense and sparse retrieval are fused with reciprocal rank fusion instead of relying on one search strategy.
Evaluate during ingestion
The system generates evaluation material as documents enter the pipeline, making quality checks part of the workflow.
Make operations inspectable
Latency percentiles, token usage, and cost are tracked alongside answer quality and citations.
Technologies
Python / FastAPI / Qdrant / PostgreSQL / React / TypeScript / OpenAI / Anthropic