https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/datasets/ted2020-source-files.zip
Usage
You can easily load the dataset using Pandas:
```python
import pandas
import csv
df = pandas.read_csv('ted2020.tsv.gz', sep='\t', keep_default_na=False, encoding='utf8', quoting=csv.QUOTE_NONE)
```
Or with Python CSV reader:
```python
import csv
import gzip
with gzip.open('ted2020.tsv.gz', 'rt', encoding='utf8') as fIn:
   reader = csv.DictReader(fIn, delimiter='\t', quoting=csv.QUOTE_NONE )
   for line in reader:
      print(line)
```
Statistics
| Language | Number of sentences |
| --- | --- |
| Total number of sentences: | 10,544,174 |
| en | 427,436 |
| pt-br | 413,689 |
| es | 412,885 |
| fr | 406,538 |
| ar | 403,716 |
| zh-tw | 400,871 |
| zh-cn | 399,092 |
| ko | 395,661 |
