Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter
  Impact on Performance and Efficiency
Large language models achieve high task performance yet often hallucinate or
rely on outdated knowledge. Retrieval-augmented generation (RAG) addresses
these gaps by coupling generation with external search. We analyse how
hyperparameters influence speed and quality in RAG systems, covering Chroma and
Faiss vector stores, chunking policies, cross-encoder re-ranking, and
temperature, and we evaluate six metrics: faithfulness, answer correctness,
answer relevancy, context precision, context recall, and answer similarity.
Chroma processes queries 13% faster, whereas Faiss yields higher retrieval
precision, revealing a clear speed-accuracy trade-off. Naive fixed-length
chunking with small windows and minimal overlap outperforms semantic
segmentation while remaining the quickest option. Re-ranking provides modest
gains in retrieval quality yet increases runtime by roughly a factor of 5, so
its usefulness depends on latency constraints. These results help practitioners
balance computational cost and accuracy when tuning RAG systems for
transparent, up-to-date responses. Finally, we re-evaluate the top
configurations with a corrective RAG workflow and show that their advantages
persist when the model can iteratively request additional evidence. We obtain a
near-perfect context precision (99%), which demonstrates that RAG systems can
achieve extremely high retrieval accuracy with the right combination of
hyperparameters, with significant implications for applications where retrieval
quality directly impacts downstream task performance, such as clinical decision
support in healthcare.
68248d74e7c1cd4c9b255e16
Adel Ammar
admin_assigned
2025-05-14T12:54:00.928Z
647c60cd36e109abce3b3b15
/avatars/d702ec97cc2cde5ca7b5d16feb5f45c9.svg
Adel Ammar
user
ammaradel
68248d74e7c1cd4c9b255e17
Anis Koubaa
admin_assigned
2025-05-14T12:54:09.177Z
6458b291d1d994ec8105e2b7
/avatars/1031491368645c5abf5b20af74dc627b.svg
Anis Koubaa
user
akoubaa
68248d74e7c1cd4c9b255e18
Omer Nacar
claimed_verified
2025-05-15T10:32:22.640Z
628f7a71dd993507cfcbe587
https://cdn-avatars.huggingface.co/v1/production/uploads/628f7a71dd993507cfcbe587/3frAcMgfBCx-OJsrdAhnb.png
Omartificial Intelligence Space
user
Omartificial-Intelligence-Space
68248d74e7c1cd4c9b255e19
Wadii Boulila
admin_assigned
2025-05-14T12:54:21.036Z
67bde4936c7c7de3a68608d1
/avatars/e08afba50d8aae8f7698ccba4c1e7b4a.svg
Wadii Boulila
user
wboulila
Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter
  Impact on Performance and Efficiency
Large language models achieve high task performance yet often hallucinate or
rely on outdated knowledge. Retrieval-augmented generation (RAG) addresses
these gaps by coupling generation with external search. We analyse how
hyperparameters influence speed and quality in RAG systems, covering Chroma and
Faiss vector stores, chunking policies, cross-encoder re-ranking, and
temperature, and we evaluate six metrics: faithfulness, answer correctness,
answer relevancy, context precision, context recall, and answer similarity.
Chroma processes queries 13% faster, whereas Faiss yields higher retrieval
precision, revealing a clear speed-accuracy trade-off. Naive fixed-length
chunking with small windows and minimal overlap outperforms semantic
segmentation while remaining the quickest option. Re-ranking provides modest
gains in retrieval quality yet increases runtime by roughly a factor of 5, so
its usefulness depends on latency constraints. These results help practitioners
balance computational cost and accuracy when tuning RAG systems for
transparent, up-to-date responses. Finally, we re-evaluate the top
configurations with a corrective RAG workflow and show that their advantages
persist when the model can iteratively request additional evidence. We obtain a
near-perfect context precision (99%), which demonstrates that RAG systems can
achieve extremely high retrieval accuracy with the right combination of
hyperparameters, with significant implications for applications where retrieval
quality directly impacts downstream task performance, such as clinical decision
support in healthcare.
RAG systems improve task performance by integrating external search, with hyperparameters influencing speed and quality, revealing trade-offs and best practices in configuration.
Retrieval-augmented generation (RAG) Chroma Faiss chunking policies cross-encoder re-ranking temperature faithfulness answer correctness answer relevancy context precision context recall answer similarity semantic segmentation corrective RAG workflow