Hi @cezary,
The suggestions from @lewtun are great and worth investing.
When I approached a problem like this a couple avenues that I explored:

Gensim TFIDF is a good baseline.
Summarization + Sentence Transformers: GitHub - UKPLab/sentence-transformers: Sentence Embeddings with BERT & XLNet 114
2a) Topic Clustering from Sentence Transformers:  Clustering — Sentence-Transformers documentation 293

If you go down the BERT route, I would recommend using a BERT fine tuned to a task similar to yours and extracting the hidden states as suggested by @lewtun , an example of extracting the hidden states is here: Domain-Specific BERT Models · Chris McCormick 169


Best of luck,
Chris