LLM Adaptation Techniques. Evaluating RAG Strategies

Abstract

This thesis explores the application of Retrieval-Augmented Generation (RAG) systems to optimize question answering tasks, addressing limitations of Large Language Models (LLMs) in scalability, efficiency, and domain adaptability. A theoretical foundation is established, highlighting RAG’s role in integrating external knowledge to enhance language models. A RAG pipeline is implemented and evaluated through experiments analyzing embedding models, similarity metrics, retrieval parameters (k), and re-ranking using cross-encoders. Results demonstrate that re-ranking improves retrieval accuracy, even with noisy, large-scale datasets, and highlight trade-offs between retrieval scope and generative performance. This study underscores RAG’s potential as a scalable alternative to finetuning, enabling efficient adaptation to dynamic datasets. Future research could explore advanced RAG variants and hybrid methods for broader applications. The corresponding code notebook can be found on the following GitHub repository, https://github.com/XiscoCasta/LLM-adaptation-techniques.-Evaluating-RAG-models