Enterprise knowledge RAG and MCP tool-service platform
Built private knowledge retrieval for specialist PDFs, connecting document processing, hybrid retrieval, ranking evaluation, and citation-bearing tool services so AI IDEs and agents can access traceable knowledge results.
Work and methods: Enterprise knowledge RAG and MCP tool-service platform
- Hybrid retrieval: Implemented dense and BM25 retrieval, RRF fusion, and cross-encoder reranking. Logged candidates, rankings, and latency at each stage to diagnose retrieval failures and iterate on strategies.
- Document ingestion: Built an end-to-end pipeline for PDF parsing, Markdown conversion, semantic chunking, metadata enrichment, embeddings, and upserts. Created over 8,000 Chroma chunks from NIST AI governance documents and BioASQ biomedical and clinical research material.
- Data quality: Added quality detection, rule-based repairs, and auditing for joined and broken English words. Repaired 468 text instances in the test documents, reducing joined-word strings by 61.1% and chunks requiring manual review by 19.9%.
- Isolated evaluation: Created golden sets of 40 manually reviewed items for NIST AI and 60 officially expert-annotated items for BioASQ. With separate development/test sets and strict relevance annotations, blind-test Hit Rate@5 reached 92.5% and 93.33%, respectively, with reported relative improvements of 37.5% and 21.66% over baselines on the same test sets.
- Service delivery: Exposed citation-bearing tools such as query_knowledge_hub, list_collections, and get_document_summary via MCP stdio/JSON-RPC, with a Streamlit console and Ragas/custom evaluation entry points.
Python, RAG, MCP, BM25, RRF, Cross-Encoder, ChromaDB, SQLite, Streamlit, Ragas












