Answers.Body: <p>Indeed RAGs, while powerful for context-specific retrieval and synthesis, struggle with exact quantitative reasoning over a mere semantically similar subset as described in your case. The key to solve this problem is recognizing the current limitations of LLMs and complementing them with tools built for specific tasks such as SQL engine plugin for data aggregation queries instead of vector retrieval.</p>
  <p>One solution could use LangChain's <a href="https://python.langchain.com/api_reference/langchain/agents.html" rel="nofollow noreferrer">agent</a> which can leverage different types of <a href="https://python.langchain.com/api_reference/core/tools.html" rel="nofollow noreferrer">tools</a> based on its LLM's inferred task from the user query, such as your simple user query about exhaust work frequency where the the default agent's LLM most likely can infer exact database query intent correctly and route to a database toolkit. However, for highly specialized or complex decision-making tasks, additional custom heuristics (like custom decision trees, rule-based engines, or ANNs) may be required.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-21T07:09:05.740Z
Answers.LastActivityDate: 2024-11-21T07:09:05.740Z
Answers.UserDisplayName: cinch
Body: <p>RAG is great for empowering LLMs with context-specific data. However, given the way that RAG works, the LLM still cannot reason about the entire data set at once. For example, say an AI assistant for a auto repair shop was empowered with work notes from past repairs, if you were to ask the bot, &quot;How often do repairs include exhaust work?&quot;, it would likely pull a subset of data points (repair orders that include things like &quot;replaced muffler&quot;) and give you a number that is completely false. This is because RAG systems just return a list of <em>n</em> relevant results.</p>
  <p>This seems extra dangerous to me because most users of the system would think the system can provide very context-specific results, and it can sometimes, but for some types of questions, it misleads.</p>
  <p>In this scenario, I imagine it would be hard for the LLM to even know that it's lying because it is basing it's answer on real data - just incomplete data.</p>
  <p>So how do we solve this?</p>
  
Tags: <large-language-models>
Title: Can LLMs with RAG reliably know the types of solutions they can provide?
UserDisplayName: PICyourBrain
ThreadText: # Can LLMs with RAG reliably know the types of solutions they can provide?
  
  (Asked by: username_0 on 2024-11-20)
  
  RAG is great for empowering LLMs with context-specific data. However, given the way that RAG works, the LLM still cannot reason about the entire data set at once. For example, say an AI assistant for a auto repair shop was empowered with work notes from past repairs, if you were to ask the bot, "How often do repairs include exhaust work?", it would likely pull a subset of data points (repair orders that include things like "replaced muffler") and give you a number that is completely false. This is because RAG systems just return a list of *n* relevant results.
  
  This seems extra dangerous to me because most users of the system would think the system can provide very context-specific results, and it can sometimes, but for some types of questions, it misleads.
  
  In this scenario, I imagine it would be hard for the LLM to even know that it's lying because it is basing it's answer on real data - just incomplete data.
  
  So how do we solve this?
  
  **Tags:** large-language-models
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-11-21. Score: 1
  
  Indeed RAGs, while powerful for context-specific retrieval and synthesis, struggle with exact quantitative reasoning over a mere semantically similar subset as described in your case. The key to solve this problem is recognizing the current limitations of LLMs and complementing them with tools built for specific tasks such as SQL engine plugin for data aggregation queries instead of vector retrieval.
  
  One solution could use LangChain's [agent](https://python.langchain.com/api_reference/langchain/agents.html) which can leverage different types of [tools](https://python.langchain.com/api_reference/core/tools.html) based on its LLM's inferred task from the user query, such as your simple user query about exhaust work frequency where the the default agent's LLM most likely can infer exact database query intent correctly and route to a database toolkit. However, for highly specialized or complex decision-making tasks, additional custom heuristics (like custom decision trees, rule-based engines, or ANNs) may be required.
  
  
  ---
  
  