CASPER Chatbot

CASPER started as a proof of concept and grew into a production RAG system. My scope was the ingestion side: taking arbitrary links, parsing and chunking them cleanly, landing artifacts in S3, and indexing into OpenSearch with metadata that made retrieval quality measurably better on engineering-flavored questions.

The GitHub scraper was where the biggest quality gains came from — code and READMEs turned out to be far denser signal than the original web sources, and the dedup layer kept the index from getting polluted as repos got re-crawled.