readme: ---
  license: mit
  ---
  # UCTopic
  
  This repository contains the code of model UCTopic and an easy-to-use tool UCTopicTool used for <strong>Topic Mining</strong>, <strong>Unsupervised Aspect Extractioin</strong> or <strong>Phrase Retrieval</strong>.
  
  Our ACL 2022 paper [UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining](https://arxiv.org/abs/2202.13469).
  
  # Quick Links
  
    - [Overview](#overview)
    - [Pretrained Model](#pretrained-model)
    - [Getting Started](#getting-started)
      - [UCTopic Model](#uctopic-model)
      - [UCTopicTool](#uctopictool)
    - [Experiments in Paper](#experiments)
      - [Requirements](#requirements)
      - [Datasets](#datasets)
      - [Entity Clustering](#entity-clustering)
      - [Topic Mining](#topic-mining)
    - [Pretraining](#pretraining)
    - [Contact](#contact)
    - [Citation](#citation)
  
  # Overview
  
  We propose UCTopic, a novel unsupervised contrastive learning framework for context-aware phrase representations and topic mining. UCTopic is pretrained in a large scale to distinguish if the contexts of two phrase mentions have the same semantics. The key to pretraining is positive pair construction from our phrase-oriented assumptions. However, we find traditional in-batch negatives cause performance decay when finetuning on a dataset with small topic numbers. Hence, we propose cluster-assisted contrastive learning(CCL) which largely reduces noisy negatives by selecting negatives from clusters and further improves phrase representations for topics accordingly.
  
  # Pretrained Model
  Our released model:
  |              Model              | Note|
  |:-------------------------------|------|
  |[uctopic-base](https://drive.google.com/file/d/1XQzi4E9ctdI373CK5O-pXQyBvOONssp1/view?usp=sharing)| Pretrained UCTopic model based on [LUKE-BASE](https://arxiv.org/abs/2010.01057)
  
  Unzip to get `uctopic-base` folder.
  
  # Getting Started
  We provide an easy-to-use phrase representation tool based on our UCTopic model. To use the tool, first install the uctopic package from PyPI
  ```bash
  pip install uctopic
  ```
  Or directly install it from our code
  ```bash
  python setup.py install
  ```
  
  ## UCTopic Model
  After installing the package, you can load our model by just two lines of code
  ```python
  from uctopic import UCTopic
  model = UCTopic.from_pretrained('JiachengLi/uctopic-base')
  ```
  The model will automatically download pre-trained parameters from [HuggingFace's models](https://huggingface.co/models). If you encounter any problem when directly loading the models by HuggingFace's API, you can also download the models manually from the above table and use `model = UCTopic.from_pretrained({PATH TO THE DOWNLOAD MODEL})`.
  
  To get pre-trained <strong>phrase representations</strong>, our model inputs are same as [LUKE](https://huggingface.co/docs/transformers/model_doc/luke). Note: please input only <strong>ONE</strong> span each time, otherwise, will have performance decay according to our empirical results.
  
  ```python
  from uctopic import UCTopicTokenizer, UCTopic
  
  tokenizer = UCTopicTokenizer.from_pretrained('JiachengLi/uctopic-base')
  model = UCTopic.from_pretrained('JiachengLi/uctopic-base')
  
  text = "Beyoncé lives in Los Angeles."
  entity_spans = [(17, 28)] # character-based entity span corresponding to "Los Angeles"
  
  inputs = tokenizer(text, entity_spans=entity_spans, add_prefix_space=True, return_tensors="pt")
  outputs, phrase_repr = model(**inputs)
  ```
  `phrase_repr` is the phrase embedding (size `[768]`) of the phrase `Los Angeles`. `outputs` has the same format as the outputs from `LUKE`.
  
  ## UCTopicTool
  We provide a tool `UCTopicTool` built on `UCTopic` for efficient phrase encoding, topic mining (or unsupervised aspect extraction) or phrase retrieval.
  
  ### Initialization
  
  `UCTopicTool` is initialized by giving the `model_name_or_path` and `device`.
  ```python
  from uctopic import UCTopicTool
  
  topic_tool = UCTopicTool('JiachengLi/uctopic-base', device='cuda:0')
  ```
  
  ### Phrase Encoding
  
  Phrases are encoded by our method `UCTopicTool.encode` in batches, which is more efficient than `UCTopic`.
  ```python
  phrases = [["This place is so much bigger than others!", (0, 10)],
             ["It was totally packed and loud.", (15, 21)],
             ["Service was on the slower side.", (0, 7)],
             ["I ordered 2 mojitos: 1 lime and 1 mango.", (12, 19)],
             ["The ingredient weren't really fresh.", (4, 14)]]
  
  embeddings = topic_tool.encode(phrases) # len(embeddings) is equal to len(phrases)
  ```
  **Note**: Each instance in `phrases` contains only one sentence and one span (character-level position) in format `[sentence, span]`.
  
  Arguments for `UCTopicTool.encode` are as follows,
  * **phrase** (List) - A list of `[sentence, span]` to be encoded.
  * **return_numpy** (bool, *optional*, defaults to `False`) - Return `numpy.array` or `torch.Tensor`.
  * **normalize_to_unit** (bool, *optional*, defaults to `True`) - Normalize all embeddings to unit vectors.
  * **keepdim** (bool, *optional*, defaults to `T