content.raw: "What Evidence Do Language Models Find Convincing?" is a new paper that explores what types of evidence and argumentation techniques language models find convincing when presented with ambiguous, open-domain questions that have conflicting answers online. 
content.type: text
content.value: "What Evidence Do Language Models Find Convincing?" is a new paper that explores what types of evidence and argumentation techniques language models find convincing when presented with ambiguous, open-domain questions that have conflicting answers online. 
content.type: new_line
content.type: new_line
content.raw: Keypoints:
content.type: text
content.value: Keypoints:
content.type: new_line
content.raw: * Dataset: It introduces "ConflictingQA," a dataset of controversial questions and real-world evidence paragraphs supporting both "yes" and "no" answers.
content.type: text
content.value: * Dataset: It introduces "ConflictingQA," a dataset of controversial questions and real-world evidence paragraphs supporting both "yes" and "no" answers.
content.type: new_line
content.raw: * Convincingness Metric: It uses the "paragraph win rate" - when shown two conflicting paragraphs, this measures how often a model predicts the answer that aligns with a given paragraph's stance.
content.type: text
content.value: * Convincingness Metric: It uses the "paragraph win rate" - when shown two conflicting paragraphs, this measures how often a model predicts the answer that aligns with a given paragraph's stance.
content.type: new_line
content.raw: * Current models rely on the relevance of the content to the query, while largely ignoring stylistic features such as whether a text contains scientific references or if it is written with a neutral tone.
content.type: text
content.value: * Current models rely on the relevance of the content to the query, while largely ignoring stylistic features such as whether a text contains scientific references or if it is written with a neutral tone.
content.type: new_line
content.type: new_line
content.raw: Congrats to the authors for their work!
content.type: text
content.value: Congrats to the authors for their work!
content.type: new_line
content.type: new_line
content.raw: Paper: 
content.type: text
content.value: Paper: 
content.label: What Evidence Do Language Models Find Convincing? (2402.11782)
content.raw: https://huggingface.co/papers/2402.11782
content.id: 2402.11782
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2402.11782
content.type: new_line
content.raw: Code: 
content.type: text
content.value: Code: 
content.href: https://github.com/AlexWan0/rag-convincingness
content.raw: https://github.com/AlexWan0/rag-convincingness
content.type: link
rawContent: "What Evidence Do Language Models Find Convincing?" is a new paper that explores what types of evidence and argumentation techniques language models find convincing when presented with ambiguous, open-domain questions that have conflicting answers online. 
  
  Keypoints:
  * Dataset: It introduces "ConflictingQA," a dataset of controversial questions and real-world evidence paragraphs supporting both "yes" and "no" answers.
  * Convincingness Metric: It uses the "paragraph win rate" - when shown two conflicting paragraphs, this measures how often a model predicts the answer that aligns with a given paragraph's stance.
  * Current models rely on the relevance of the content to the query, while largely ignoring stylistic features such as whether a text contains scientific references or if it is written with a neutral tone.
  
  Congrats to the authors for their work!
  
  Paper: https://huggingface.co/papers/2402.11782
  Code: https://github.com/AlexWan0/rag-convincingness
reactions.reaction: ❤️
reactions.users: samusenps dillfrescott osanseviero gsarti victor clem Kukedlc victorshenbr sbrandeis