Page MenuHomePhabricator

Integrate Denelezh and merge stats tables
Closed, ResolvedPublic8 Estimated Story Points

Description

Integrate Denelezh statistics, i.e. per-professions stats.

Merge all current tables into a single one:

CREATE TABLE persons (
    date          TEXT NOT NULL,
    qid           TEXT NOT NULL,
    birth_year    INT  NOT NULL,
    gender_id     TEXT NOT NULL REFERENCES gender_labels (qid) ON DELETE RESTRICT,
    PRIMARY KEY (date, wiki_name, wiki_lang, country_id, profession_id, gender_id)
) STRICT;

CREATE TABLE persons_countries (
    date       TEXT NOT NULL,
    person_id  TEXT NOT NULL,
    country_id TEXT NOT NULL REFERENCES countries (qid) ON DELETE RESTRICT,
    FOREIGN KEY (date, person_id) REFERENCES persons (date, qid) ON DELETE RESTRICT,
    PRIMARY KEY (date, person_id, country_id)
) STRICT;

CREATE TABLE persons_professions (
    date          TEXT NOT NULL,
    person_id     TEXT NOT NULL,
    profession_id TEXT NOT NULL REFERENCES professions (qid) ON DELETE RESTRICT,
    FOREIGN KEY (date, person_id) REFERENCES persons (date, qid) ON DELETE RESTRICT,
    PRIMARY KEY (date, person_id, profession_id)
) STRICT;

Pro:

  • Makes it possible to build queries combining multiple criteria.
  • Easier to query from the search page.

Cons:

  • The database’s size may increase substentially because of no longer aggregating human elements.

Event Timeline

Danya triaged this task as High priority.Apr 5 2026, 10:44 PM
Danya renamed this task from Integrate Denelezeh to Integrate Denelezh.Apr 25 2026, 7:00 PM
Danya updated the task description. (Show Details)
Danya set the point value for this task to 4.May 1 2026, 10:43 AM
Danya changed the task status from Open to In Progress.May 10 2026, 7:23 PM
Danya moved this task from [deleted] to Doing on the Tool-wiki-gender-stats board.
Danya renamed this task from Integrate Denelezh to Integrate Denelezh and merge stats tables.May 10 2026, 9:01 PM
Danya updated the task description. (Show Details)
Danya changed the point value for this task from 4 to 5.

I’m getting timeouts on QLever with the new queries, we may need to use wiki replicas instead. Hopefully, this would solve the issue. If not, then we will have to resort to downloading and parsing dumps.

Danya changed the task status from In Progress to Stalled.May 13 2026, 7:11 PM
Danya changed the point value for this task from 5 to 6.

Stalled until server is migrated to Cloud VPS (T425489).

Danya changed the task status from Stalled to In Progress.May 13 2026, 7:19 PM
Danya changed the task status from In Progress to Stalled.
Danya moved this task from [deleted] to Doing on the Tool-wiki-gender-stats board.
Danya changed the status of subtask T423012: Use dumps from Open to In Progress.May 14 2026, 8:38 AM
Danya changed the point value for this task from 6 to 13.May 14 2026, 8:45 AM
Danya changed the point value for this task from 13 to 8.
Danya changed the task status from Stalled to In Progress.May 14 2026, 11:35 PM