Page MenuHomePhabricator
Create Task
Maniphest T329353

PHP Notice: Trying to access array offset on value of type null (in TranslationUnitDTO)
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error
normalized_message
[{reqId}] {exception_url}   PHP Notice: Trying to access array offset on value of type null
exception.trace
from /srv/mediawiki/php-1.40.0-wmf.21/extensions/ContentTranslation/includes/DTO/TranslationUnitDTO.php(134)
#0 /srv/mediawiki/php-1.40.0-wmf.21/extensions/ContentTranslation/includes/DTO/TranslationUnitDTO.php(134): MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.40.0-wmf.21/extensions/ContentTranslation/scripts/dump-corpora.php(146): ContentTranslation\DTO\TranslationUnitDTO->toCorporaDumpArray(boolean)
#2 [internal function]: CXCorporaDump::{closure}(ContentTranslation\DTO\TranslationUnitDTO)
#3 /srv/mediawiki/php-1.40.0-wmf.21/extensions/ContentTranslation/scripts/dump-corpora.php(147): array_map(Closure, array)
#4 /srv/mediawiki/php-1.40.0-wmf.21/maintenance/includes/MaintenanceRunner.php(585): CXCorporaDump->execute()
#5 /srv/mediawiki/php-1.40.0-wmf.21/maintenance/doMaintenance.php(99): MediaWiki\Maintenance\MaintenanceRunner->run()
#6 /srv/mediawiki/php-1.40.0-wmf.21/extensions/ContentTranslation/scripts/dump-corpora.php(316): require_once(string)
#7 /srv/mediawiki/multiversion/MWScript.php(118): require_once(string)
#8 {main}
Impact
Notes

These are currently happening on a very high rate (~2200 of these in the last 15 min) in snapshot1008.

Details

MediaWiki Version
1.40.0-wmf.21
SubjectRepoBranchLines +/-
CX corpora dump: Check if type exists before sanitizingmediawiki/extensions/ContentTranslationmaster+1 -1
CX corpora query API: Handle null blob types for translation unitsmediawiki/extensions/ContentTranslationmaster+3 -0
CX corpora dump: Filter out translation units without sourcemediawiki/extensions/ContentTranslationmaster+10 -2
Customize query in gerrit

Related Objects

Event Timeline

Zabe created this task.Feb 10 2023, 10:09 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 10 2023, 10:09 AM
Zabe added a project: Language-Team.Feb 10 2023, 10:10 AM
Krinkle renamed this task from PHP Notice: Trying to access array offset on value of type null to PHP Notice: Trying to access array offset on value of type null (in TranslationUnitDTO).Feb 15 2023, 12:12 AM
jnuche subscribed.Mar 2 2023, 1:14 PM

Still happening as of wmf.25

One such req from today: https://logstash.wikimedia.org/goto/056a3d0a8ca80fa3193257fd0837d74b

Added filter (accounting for TranslationUnitDTO) to MW new errors board.

jnuche added a comment.Mar 2 2023, 1:23 PM

Actually, the null access seems to now be happening in a different function call. New backtrace:

exception.trace
from /srv/mediawiki/php-1.40.0-wmf.24/extensions/ContentTranslation/includes/DTO/TranslationUnitDTO.php(109)
#0 /srv/mediawiki/php-1.40.0-wmf.24/extensions/ContentTranslation/includes/DTO/TranslationUnitDTO.php(109): MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.40.0-wmf.24/extensions/ContentTranslation/includes/ActionApi/ApiQueryContentTranslationCorpora.php(46): ContentTranslation\DTO\TranslationUnitDTO->toCustomArray(array, boolean)
#2 [internal function]: ContentTranslation\ActionApi\ApiQueryContentTranslationCorpora::ContentTranslation\ActionApi\{closure}(ContentTranslation\DTO\TranslationUnitDTO)
#3 /srv/mediawiki/php-1.40.0-wmf.24/extensions/ContentTranslation/includes/ActionApi/ApiQueryContentTranslationCorpora.php(45): array_map(Closure, array)
#4 /srv/mediawiki/php-1.40.0-wmf.24/includes/api/ApiQuery.php(678): ContentTranslation\ActionApi\ApiQueryContentTranslationCorpora->execute()
#5 /srv/mediawiki/php-1.40.0-wmf.24/includes/api/ApiMain.php(1901): ApiQuery->execute()
#6 /srv/mediawiki/php-1.40.0-wmf.24/includes/api/ApiMain.php(878): ApiMain->executeAction()
#7 /srv/mediawiki/php-1.40.0-wmf.24/includes/api/ApiMain.php(849): ApiMain->executeActionWithErrorHandling()
#8 /srv/mediawiki/php-1.40.0-wmf.24/api.php(90): ApiMain->execute()
#9 /srv/mediawiki/php-1.40.0-wmf.24/api.php(45): wfApiMain()
#10 /srv/mediawiki/w/api.php(3): require(string)
#11 {main}
Nikerabbit triaged this task as Medium priority.Mar 13 2023, 11:23 AM
Nikerabbit edited projects, added Language-Team (Language-2023-January-March); removed Language-Team.
ngkountas claimed this task.Mar 21 2023, 12:16 PM
gerritbot added a comment.Mar 21 2023, 12:16 PM

Change 901568 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):

[mediawiki/extensions/ContentTranslation@master] CX corpora dump: Filter out translation units without source

https://gerrit.wikimedia.org/r/901568

gerritbot added a project: Patch-For-Review.Mar 21 2023, 12:16 PM

Change 901569 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):

[mediawiki/extensions/ContentTranslation@master] CX corpora query API: Handle null blob types for translation units

https://gerrit.wikimedia.org/r/901569

ngkountas moved this task from Quarter Backlog to In Review on the Language-Team (Language-2023-January-March) board.Mar 21 2023, 12:16 PM
gerritbot added a comment.Mar 22 2023, 5:09 AM

Change 901568 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] CX corpora dump: Filter out translation units without source

https://gerrit.wikimedia.org/r/901568

gerritbot added a comment.Mar 22 2023, 5:11 AM

Change 901569 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] CX corpora query API: Handle null blob types for translation units

https://gerrit.wikimedia.org/r/901569

Maintenance_bot removed a project: Patch-For-Review.Mar 22 2023, 5:30 AM
ReleaseTaggerBot added a project: MW-1.41-notes (1.41.0-wmf.2; 2023-03-27).Mar 22 2023, 6:00 AM
ngkountas moved this task from In Review to Check after deployment on the Language-Team (Language-2023-January-March) board.Mar 26 2023, 4:39 PM
Nikerabbit moved this task from Needs Triage to Bugs on the ContentTranslation board.Mar 27 2023, 11:40 AM
ngkountas moved this task from Check after deployment to Needs QA on the Language-Team (Language-2023-January-March) board.Mar 30 2023, 10:42 AM
Pginer-WMF edited projects, added Language-Team (Language-2023-April-June); removed Language-Team (Language-2023-January-March).Mar 31 2023, 1:23 PM
Pginer-WMF moved this task from Quarter Backlog to Needs QA on the Language-Team (Language-2023-April-June) board.
hashar subscribed.Apr 7 2023, 1:15 PM

There is another one that generated 2M PHP Notices since 11h20 UTC this morning:

labels.normalized_message
[{reqId}] {exception_url}   PHP Notice: Trying to access array offset on value of type null
error.stack_trace
from /srv/mediawiki/php-1.41.0-wmf.3/extensions/ContentTranslation/includes/DTO/TranslationUnitDTO.php(153)
#0 /srv/mediawiki/php-1.41.0-wmf.3/extensions/ContentTranslation/includes/DTO/TranslationUnitDTO.php(153): MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.41.0-wmf.3/extensions/ContentTranslation/scripts/dump-corpora.php(146): ContentTranslation\DTO\TranslationUnitDTO->toCorporaDumpArray(boolean)
#2 [internal function]: CXCorporaDump::{closure}(ContentTranslation\DTO\TranslationUnitDTO)
#3 /srv/mediawiki/php-1.41.0-wmf.3/extensions/ContentTranslation/scripts/dump-corpora.php(147): array_map(Closure, array)
#4 /srv/mediawiki/php-1.41.0-wmf.3/maintenance/includes/MaintenanceRunner.php(681): CXCorporaDump->execute()
#5 /srv/mediawiki/php-1.41.0-wmf.3/maintenance/run.php(51): MediaWiki\Maintenance\MaintenanceRunner->run()
#6 /srv/mediawiki/multiversion/MWScript.php(140): require_once(string)
#7 {main}

It is a systemd timer on snapshot1008:

NEXT                         LEFT           LAST                         PASSED        UNIT                                            ACTIVATES
n/a                          n/a            Fri 2023-04-07 09:10:00 UTC  4h 2min ago   xlation-dumps.timer                             xlation-dumps.service
hashar merged a task: T334285: Snapshot host cause a ContentTranslation notice: PHP Notice: Trying to access array offset on value of type null.Apr 7 2023, 1:16 PM
ArielGlenn subscribed.Apr 7 2023, 2:31 PM
Hokwelum subscribed.Apr 11 2023, 3:23 PM
ArielGlenn added a comment.Apr 11 2023, 3:27 PM

The job is of course the content translation dumps, which run once a week. They call the maintenance script extensions/ContentTranslation/scripts/dump-corpora.php which I guess the Language Team knows the innards of.

EChukwukere-WMF subscribed.Apr 12 2023, 3:10 AM

@ArielGlenn @Zabe could there be any specific steps to reproduce this ?

gerritbot added a comment.Apr 13 2023, 9:25 AM

Change 908484 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):

[mediawiki/extensions/ContentTranslation@master] CX corpora dump: Check if type exists before sanitizing

https://gerrit.wikimedia.org/r/908484

gerritbot added a project: Patch-For-Review.Apr 13 2023, 9:25 AM
thcipriani moved this task from Untriaged to Feb 2023 on the Wikimedia-production-error board.Apr 13 2023, 3:52 PM
EChukwukere-WMF added a comment.Apr 17 2023, 4:13 AM

Completed content translation of an article and unable to reproduce the above said issue.

Marked as QA PASS

EChukwukere-WMF moved this task from Needs QA to Done on the Language-Team (Language-2023-April-June) board.Apr 17 2023, 4:13 AM
ngkountas added a comment.Apr 19 2023, 8:35 AM

@EChukwukere-WMF This task cannot really be tested using the CX or SX application. We can only track the logstash and see if the errors disappear to verify that it is fixed, moving back to "Needs QA" to keep an eye on it for some time, until we stop getting those errors.

ngkountas moved this task from Done to Needs QA on the Language-Team (Language-2023-April-June) board.Apr 19 2023, 8:35 AM
EChukwukere-WMF added a comment.Apr 20 2023, 2:26 PM

@ngkountas By logstash you mean the network tab in the browser correct ?

ngkountas added a comment.Apr 21 2023, 10:46 AM

@EChukwukere-WMF Logstash is the tool we use to collect, track and manage the logs from production Wikis. You should be able to access it here: https://logstash.wikimedia.org

gerritbot added a comment.May 2 2023, 7:22 AM

Change 908484 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] CX corpora dump: Check if type exists before sanitizing

https://gerrit.wikimedia.org/r/908484

Maintenance_bot removed a project: Patch-For-Review.May 2 2023, 7:30 AM
ReleaseTaggerBot edited projects, added MW-1.41-notes (1.41.0-wmf.10; 2023-05-23); removed MW-1.41-notes (1.41.0-wmf.2; 2023-03-27).May 16 2023, 4:01 AM
Pginer-WMF edited projects, added Language-Team (Language-2023-July-September); removed Language-Team (Language-2023-April-June).Jun 30 2023, 11:43 AM
Pginer-WMF moved this task from Quarter Backlog to Needs QA on the Language-Team (Language-2023-July-September) board.
hashar unsubscribed.Jul 3 2023, 1:17 PM
Nikerabbit closed this task as Resolved.Aug 3 2023, 8:14 AM
Nikerabbit moved this task from Needs QA to Done on the Language-Team (Language-2023-July-September) board.
Nikerabbit subscribed.

Last occurrence of this warning in Logstash was in May 5, 2023.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL