Page MenuHomePhabricator

Dumps ingester crashes for invalid wiki languages
Closed, ResolvedPublic3 Estimated Story PointsBUG REPORT

Description

If the dump being ingested contains an invalid wiki language code, the dumps ingester crashes with the following error:

Traceback (most recent call last):
  File "/workspace/ingest_dump.py", line 143, in main
    ingest_dump(file, dump_date, db, mode, log_to_stdout)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/server/services/dumps_ingester.py", line 576, in ingest_dump
    elements_count = _ingest_human_elements(
        dump_file, date, db, print_to_console
    )
  File "/workspace/server/services/dumps_ingester.py", line 376, in _ingest_human_elements
    db.save_person_data(
    ~~~~~~~~~~~~~~~~~~~^
        date,
        ^^^^^
    ...<5 lines>...
        wikis,
        ^^^^^^
    )
    ^
  File "/workspace/server/services/storage_service.py", line 542, in save_person_data
    raise ValueError(f"invalid wiki language: {wiki}")
ValueError: invalid wiki language: commons.wikipedia.org

First impacted version: 0.4.0

Related Objects

Event Timeline

Danya claimed this task.
Danya triaged this task as Unbreak Now! priority.
Danya lowered the priority of this task from Unbreak Now! to High.
Danya set the point value for this task to 3.
Danya changed the task status from Open to In Progress.Sun, Aug 9, 5:35 PM
Danya edited parent tasks, added: T434396: [VERSION] 0.4.1; removed: T432922: [VERSION] 0.5.0.
Danya moved this task from Next Version to Doing on the Tool-wiki-gender-stats board.
Danya moved this task from Doing to Done on the Tool-wiki-gender-stats board.