June 2025 data update

README.md

@@ -8,14 +8,14 @@ library_name: txtai
 tags:
 - sentence-similarity
 datasets:
-- NeuML/wikipedia-20250123
+- NeuML/wikipedia-20250620
 ---
 
 # Wikipedia txtai embeddings index
 
 This is a [txtai](https://github.com/neuml/txtai) embeddings index for the [English edition of Wikipedia](https://en.wikipedia.org/).
 
-This index is built from the [Wikipedia January 2025 dataset](https://huggingface.co/datasets/neuml/wikipedia-20250123). Only the first paragraph of the [lead section](https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Lead_section) from each article is included in the index. This is similar to an abstract of the article.
+This index is built from the [Wikipedia June 2025 dataset](https://huggingface.co/datasets/neuml/wikipedia-20250620). Only the first paragraph of the [lead section](https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Lead_section) from each article is included in the index. This is similar to an abstract of the article.
 
 It also uses [Wikipedia Page Views](https://dumps.wikimedia.org/other/pageviews/readme.html) data to add a `percentile` field. The `percentile` field can be used
 to only match commonly visited pages.
@@ -65,7 +65,7 @@ Performance was evaluated using the [NDCG@10](https://en.wikipedia.org/wiki/Disc
 
 ## Build the index
 
-The following steps show how to build this index. These scripts are using the latest data available as of 2025-01-23, update as appropriate.
+The following steps show how to build this index. These scripts are using the latest data available as of 2025-06-20, update as appropriate.
 
 - Install required build dependencies
 ```bash
@@ -75,7 +75,7 @@ pip install ragdata mwparserfromhell
 - Download and build pageviews database
 ```bash
 mkdir -p pageviews/data
-wget -P pageviews/data https://dumps.wikimedia.org/other/pageview_complete/monthly/2025/2025-01/pageviews-202501-user.bz2
+wget -P pageviews/data https://dumps.wikimedia.org/other/pageview_complete/monthly/2025/2025-06/pageviews-202506-user.bz2
 python -m ragdata.wikipedia.views -p en.wikipedia -v pageviews
 ```
 
@@ -85,7 +85,7 @@ python -m ragdata.wikipedia.views -p en.wikipedia -v pageviews
 from datasets import load_dataset
 
 # Data dump date from https://dumps.wikimedia.org/enwiki/
-date = "20250123"
+date = "20250620"
 
 # Build and save dataset
 ds = load_dataset("neuml/wikipedia", language="en", date=date)
@@ -95,7 +95,7 @@ ds.save_to_disk(f"wikipedia-{date}")
 - Build txtai-wikipedia index
 ```bash
 python -m ragdata.wikipedia.index \
-       -d wikipedia-20250123 \
+       -d wikipedia-20250620 \
        -o txtai-wikipedia \
        -v pageviews/pageviews.sqlite
 ```

config.json

@@ -14,15 +14,15 @@
   "content": true,
   "dimensions": 768,
   "backend": "faiss",
-  "offset": 6332694,
+  "offset": 6393125,
   "build": {
-    "create": "2025-02-10T20:02:33Z",
-    "python": "3.9.21",
+    "create": "2025-07-03T15:11:29Z",
+    "python": "3.10.18",
     "settings": {
-      "components": "IVF2251,SQ8"
+      "components": "IVF2262,SQ8"
     },
     "system": "Linux (x86_64)",
-    "txtai": "8.3.0"
+    "txtai": "8.7.0"
   },
-  "update": "2025-02-10T20:02:33Z"
+  "update": "2025-07-03T15:11:29Z"
 }

documents

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f0346befb651ac69494d4de63c62096099cf6190019c301f94cca5bcbff57301
-size 3405090816
+oid sha256:8455e7fc929c97437db0cb8cbef3adf9d5243cdfc272ff2d97c6c48a9892d415
+size 3440136192

embeddings

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:118afed43b574b98db1019f4ef928254e0a04af82f43ad6497e4bd720a808627
-size 4921109952
+oid sha256:536aea23237a8c9673980a020b567d1913cc29cbc51b8e6b4a21c4797fedab1b
+size 4968038288