Retrieval-Augmented Generation and vector search. Embeddings, chunking strategies, vector DBs, and production RAG pipelines.
Fine-tuned Mistral 7B on 100k SQL query pairs with DPO. It now beats GPT-4 on Spider benchmark by 12%. Runs on a single RTX 4090. Model weights and training code included.
Everyone's building RAG pipelines but getting mediocre results. I switched to fine-tuning domain-specific 3B models and the quality difference is staggering. Here's my cost analysis.
Retrieval-Augmented Generation and vector search. Embeddings, chunking strategies, vector DBs, and production RAG pipelines.