The Wikidata Embedding Project (Wikimedia Deutschland, with Jina AI and DataStax) publishes jina-embeddings-v3 vectors for ~23M Wikidata entities — every item with at least one Wikipedia article. The dataset is CC0, refreshed (a May-2026 snapshot build landed 2026-07-17), and served two ways:
- a live API at wd-vectordb.wmcloud.org (Astra-backed, single-QID lookups), and
- a bulk release on Hugging Face (philippesaade/Wikidata_Vectors_0.2, 44M rows / ~123 GB parquet, unsorted by QID).
It would be convenient if they could be more easily accessible to Toolforge & PAWS for research and tools. A higher rate limit would be great, but making the dumps readily available would be a simple step forward.
Concretely, from our research project (Wikipedia readership analysis): extracting vectors for ~57K articles required streaming (initially ~42 GB, now 123 GB) of parquet from Hugging Face to filter out ⟨~120 MB⟩ of vectors.
A lot of the other data sources that one may want to combine with this, are already readily co-located — pagelinks, page_props (page↔QID), and the Wikidata entity dumps are all on the dumps mount next to Toolforge/Cloud compute.