content.raw: ✅New paper to ensure valid LLM output with SOTA LLMs like GPT4 by mixing it with OSS LLMs
content.type: text
content.value: ✅New paper to ensure valid LLM output with SOTA LLMs like GPT4 by mixing it with OSS LLMs
content.type: new_line
content.type: new_line
content.raw: Paper: arxiv.org/abs/2401.09967
content.type: text
content.value: Paper: arxiv.org/abs/2401.09967
content.type: new_line
content.type: new_line
content.raw: Great paper showing how strong proprietary AI like #GPT4 can be paired with #OSS LLM to ensure LLM output validity, e.g. valid JSON. 
content.type: text
content.value: Great paper showing how strong proprietary AI like #GPT4 can be paired with #OSS LLM to ensure LLM output validity, e.g. valid JSON. 
content.type: new_line
content.type: new_line
content.raw: Many devs complain that #LLMs cannot be reliably used in production if the output is not valid, for instance, if one wants to use LLMs to generate SQL queries or JSON, it is crucial that the output is valid.
content.type: text
content.value: Many devs complain that #LLMs cannot be reliably used in production if the output is not valid, for instance, if one wants to use LLMs to generate SQL queries or JSON, it is crucial that the output is valid.
content.type: new_line
content.type: new_line
content.raw: Frameworks have arisen to constrain the outputs of the LLM to follow some constraints, like outlines (
content.type: text
content.value: Frameworks have arisen to constrain the outputs of the LLM to follow some constraints, like outlines (
content.href: https://github.com/outlines-dev/outlines
content.raw: https://github.com/outlines-dev/outlines
content.type: link
content.raw: ), but they assume access to logits.
content.type: text
content.value: ), but they assume access to logits.
content.type: new_line
content.type: new_line
content.raw: This makes them incompatible with proprietary LLMs like GPT4 that don’t share logits, so one can only use open-source LLMs that are much less performant.
content.type: text
content.value: This makes them incompatible with proprietary LLMs like GPT4 that don’t share logits, so one can only use open-source LLMs that are much less performant.
content.type: new_line
content.type: new_line
content.raw: This paper shows how can use powerful proprietary LLMs like GPT4 to create a first unconstrained sketch and refine it using an OSS model like Llama 2 where logits are accessible, to rewrite the sketch following some specific constraints.
content.type: text
content.value: This paper shows how can use powerful proprietary LLMs like GPT4 to create a first unconstrained sketch and refine it using an OSS model like Llama 2 where logits are accessible, to rewrite the sketch following some specific constraints.
content.type: new_line
content.type: new_line
content.raw: They show that GPT4 Precision can be increased by 14% (43% before, 57% after), by boosting it with constrained output on information extraction on Wiki-NRE!
content.type: text
content.value: They show that GPT4 Precision can be increased by 14% (43% before, 57% after), by boosting it with constrained output on information extraction on Wiki-NRE!
rawContent: ✅New paper to ensure valid LLM output with SOTA LLMs like GPT4 by mixing it with OSS LLMs
  
  Paper: arxiv.org/abs/2401.09967
  
  Great paper showing how strong proprietary AI like #GPT4 can be paired with #OSS LLM to ensure LLM output validity, e.g. valid JSON. 
  
  Many devs complain that #LLMs cannot be reliably used in production if the output is not valid, for instance, if one wants to use LLMs to generate SQL queries or JSON, it is crucial that the output is valid.
  
  Frameworks have arisen to constrain the outputs of the LLM to follow some constraints, like outlines (https://github.com/outlines-dev/outlines), but they assume access to logits.
  
  This makes them incompatible with proprietary LLMs like GPT4 that don’t share logits, so one can only use open-source LLMs that are much less performant.
  
  This paper shows how can use powerful proprietary LLMs like GPT4 to create a first unconstrained sketch and refine it using an OSS model like Llama 2 where logits are accessible, to rewrite the sketch following some specific constraints.
  
  They show that GPT4 Precision can be increased by 14% (43% before, 57% after), by boosting it with constrained output on information extraction on Wiki-NRE!
reactions.reaction: ❤️
reactions.users: s3nh clem osanseviero samusenps Dlbk AsadullaH777 merve yahma lunarflu sbarman25 Aseetpatel21