readme: ---
  annotations_creators:
  - expert-generated
  language:
  - en
  language_creators:
  - found
  license: apache-2.0
  multilinguality:
  - monolingual
  pretty_name: message-classification
  size_categories:
  - n=10K
  source_datasets:
  - https://github.com/zeloru/small-english-smalltalk-corpus
  tags:
  - message-classification
  - question-detection
  widget:
  - text: "How are you"
  - text: "Hello there, how are you"
  - text: "Hello there, nice to meet you"
  - text: "The highest mountain of Switzerland is the Dufourspitze"
  - text: "Which ANN algorithm has Apache Lucene implemented"
  task_categories:
  - text-classification
  task_ids:
  - semantic-similarity-scoring
  ---
  
  ## Table of Contents
  - [Description](#description)
    - [Summary and intended uses](#summary-and-intended-uses)
    - [Languages](#languages)
  - [Dataset Structure](#dataset-structure)
    - [Data Fields](#data-fields)
    - [Data Splits](#data-splits)
  - [Dataset Creation](#dataset-creation)
    - [Curation Rationale](#curation-rationale)
    - [Source Data](#source-data)
    - [Annotations](#annotations)
  - [Considerations for Using the Model](#considerations-for-using-the-model)
    - [Known Limitations](#known-limitations)
  
  ## Description
  ### Summary and intended uses
  
  Model to detect whether a sentence is a question.
  
  For example
  
  - __Question__: How are you
  - __Question__: Hello there, how are you
  - __Other__: Hello there, nice to meet you
  - __Other__: The highest mountain of Switzerland is the Dufourspitze
  - __Question__: Which ANN algorithm has Apache Lucene implemented
  - __Other__: Hi Everyone, we have a new blog post that you all might be interested in: "Why is Vector Search so fast?"
  
  The model can be used by bots (e.g. https://ukatie.com) to detect questions inside chatrooms, like for example Slack, MS Teams, Discord or Matrix.
  
  ### Languages
  
  So far, English is the only supported language.
  
  ## Dataset Structure
  
  ### Data Fields
  
  Text: Short input sentence, e.g. "Which ANN algorithm has Apache Lucene implemented"
  
  Label: __Question__ or __Other__
  
  ### Data Splits
  
  Question: 10K samples
  
  Other: 10K samples
  
  
  Training:  18K samples shuffled
  
  Validation: 2K samples shuffled
  
  ## Dataset Creation
  
  ### Curation Rationale
  
  Simple, short and basic language examples were chosen, because those contain the same kind of words and word placements as long-winded questions with rare words.
  
  Also the goal of this is to detect questions in a chatroom, which is a medium where people often use very short sentences and a lot of greetings or small-talk.
  
  ### Source Data
  
  #### Initial Data Collection
  
  https://github.com/zeloru/small-english-smalltalk-corpus
  
  It is scraped data from ESL language learning material.
  
  Out of the already scraped data, only samples from certain conversations were taken, because of quality issues with some of them.
  
  Because we also want to detect questions that are missing a questionmark, most of the samples had the questionmark removed. The same was done for the "other" label where "." were removed from the end of a sentence. This was done, so these identifiers don't become the only feature the model looks at.
  
  ### Annotations
  
  #### Annotation process
  
  The annotations of "question" or "other" were done automatically by taking questions in the conversations as "question" and the answers as "other"
  
  ## Considerations for Using the Model
  
  ### Known Limitations
  
  There seems to be an inbalance of greeting word combinations in the beginning of sentences, for example "Hi, has anyone deployed X in Y" is falsely not detected as a question because of the "Hi, " part. This issue will be addressed in updates.
  
  Sentences in the form of "Wondering if 'question'..." and "I'm asking for help about 'question'..." are seemingly hard to detect and need more samples in the data.
  
  Code fragments in input sentences are sometimes detected as questions. If code is present, it should probably be filtered out beforehand.
  