readme: ---
  datasets:
  - ElKulako/stocktwits-crypto
  language: 
  - en
  tags:
  - cryptocurrency
  - crypto
  - BERT
  - sentiment classification
  - NLP
  - bitcoin
  - ethereum
  - shib
  - social media
  - sentiment analysis
  - cryptocurrency sentiment analysis
  
  ---
  
  # CryptoBERT
  CryptoBERT is a pre-trained NLP model to analyse the language and sentiments of cryptocurrency-related social media posts and messages. It was built by further training the [vinai's bertweet-base](https://huggingface.co/vinai/bertweet-base) language model on the cryptocurrency domain, using a corpus of over 3.2M unique cryptocurrency-related social media posts. 
  (A research paper with more details will follow soon.)
  ## Classification Training
  The model was trained on the following labels: "Bearish" : 0, "Neutral": 1, "Bullish": 2
  
  CryptoBERT's sentiment classification head was fine-tuned on a balanced dataset of 2M labelled StockTwits posts, sampled from [ElKulako/stocktwits-crypto](https://huggingface.co/datasets/ElKulako/stocktwits-crypto). 
  
  CryptoBERT was trained with a max sequence length of 128. Technically, it can handle sequences of up to 514 tokens, however, going beyond 128 is not recommended.
  
  # Classification Example
  ```python
  from transformers import TextClassificationPipeline, AutoModelForSequenceClassification, AutoTokenizer
  model_name = "ElKulako/cryptobert"
  tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
  model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels = 3)
  pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer, max_length=64, truncation=True, padding = 'max_length')
  # post_1 & post_3 = bullish, post_2 = bearish
  post_1 = " see y'all tomorrow and can't wait to see ada in the morning, i wonder what price it is going to be at. 😎🐂🤠💯😴, bitcoin is looking good go for it and flash by that 45k. "
  post_2 = "  alright racers, it’s a race to the bottom! good luck today and remember there are no losers (minus those who invested in currency nobody really uses) take your marks... are you ready? go!!" 
  post_3 = " i'm never selling. the whole market can bottom out. i'll continue to hold this dumpster fire until the day i die if i need to." 
  df_posts = [post_1, post_2, post_3]
  preds = pipe(df_posts)
  print(preds)
  
  
  ```
  
  ```
  [{'label': 'Bullish', 'score': 0.8734585642814636}, {'label': 'Bearish', 'score': 0.9889495372772217}, {'label': 'Bullish', 'score': 0.6595883965492249}]
  ```
  
  ## Training Corpus
  CryptoBERT was trained on 3.2M social media posts regarding various cryptocurrencies. Only non-duplicate posts of length above 4 words were considered. The following communities were used as sources for our corpora:
  
  
  (1) StockTwits - 1.875M posts about the top 100 cryptos by trading volume. Posts were collected from the 1st of November 2021 to the 16th of June 2022. [ElKulako/stocktwits-crypto](https://huggingface.co/datasets/ElKulako/stocktwits-crypto)
  
  (2) Telegram - 664K posts from top 5 telegram groups: [Binance](https://t.me/binanceexchange), [Bittrex](https://t.me/BittrexGlobalEnglish), [huobi global](https://t.me/huobiglobalofficial), [Kucoin](https://t.me/Kucoin_Exchange), [OKEx](https://t.me/OKExOfficial_English). 
  Data from 16.11.2020 to 30.01.2021. Courtesy of [Anton](https://www.kaggle.com/datasets/aagghh/crypto-telegram-groups).
  
  (3) Reddit - 172K comments from various crypto investing threads, collected from May 2021 to May 2022
  
  (4) Twitter - 496K posts with hashtags XBT, Bitcoin or BTC. Collected for May 2018. Courtesy of [Paul](https://www.kaggle.com/datasets/paul92s/bitcoin-tweets-14m).